What exactly are big and small ends in communication protocols?

In IoT application development, the communication protocols obtained from embedded projects often indicate big-endian and small-endian modes. So what are big-endian and small-endian?

Big and small endian

In communication protocols, endian is an important concept that involves the storage of multi-byte data (such as integers, floating-point numbers, etc.) in memory. Endian determines the order in which the most significant byte (MSB) and the least significant byte (LSB) of the data are arranged in the memory address. It is crucial for cross-platform communication and data exchange because different hardware platforms may use different byte orders.

1. "Big Endian": high-order bytes are stored at low addresses in memory, and low-order bytes are stored at high addresses in memory. For example, a 16-bit integer 0x1234 is represented in memory as 0x12 0x34 in big-endian mode.
2. "Little Endian": The low-order bytes are stored at the low addresses of the memory, and the high-order bytes are stored at the high addresses of the memory. For the same 16-bit integer 0x1234, in the little-endian mode, the representation in the memory is 0x34 0x12.

picture

In the communication protocol, it is necessary to clearly specify which byte order to use to ensure that the sender and receiver can interpret the data correctly. Otherwise, if the sender uses big endian and the receiver uses little endian (or vice versa), confusion may occur during data transmission.

A common practice is to explicitly specify the byte order in the communication protocol, which ensures the correct interpretation of the data regardless of the software and hardware platforms of the sender and receiver. It may also be necessary to add some specific tags or metadata to the communication protocol to indicate the byte order of the data, so that the receiver can dynamically adjust its byte order interpretation based on these tags.

picture

For example, many network protocols (such as TCP/IP) use big endian, while Intel architectures such as x86 and x86_64 use little endian. When communicating across platforms, byte order mismatches can cause problems, and it is usually necessary to convert byte order when sending and receiving data.

In programming, sometimes you need to write specific code to handle the conversion of byte order to ensure the correct interpretation of data. For example, in C language, you can use functions such as htonl, ntohl, htons, ntohs, etc. to handle the conversion between network byte order and host byte order. The "h" in these function names stands for host, "n" stands for network, "s" stands for short, and "l" stands for long.

Endian conversion

In Java, you can directly use the ByteBuffer class to convert the endianness. ByteBuffer supports big endian and little endian, and can dynamically change the byte order at runtime.

For integer types (such as int, short, etc.), you can use the ByteBuffer's order() method to set the byte order, and then use methods such as putInt() and getShort() to read and write data.

 import java.nio.ByteBuffer; import java.nio.ByteOrder; public class EndianConversion { public static void main(String[] args) { int data1 = 0x12345678; short data2 = 0x1234; // 使用ByteBuffer进行端序转换ByteBuffer buffer1 = ByteBuffer.allocate(6); // 分配足够的空间ByteBuffer buffer2 = ByteBuffer.allocate(6); // 设置为小端序并写入数据buffer1.order(ByteOrder.LITTLE_ENDIAN); buffer1.putInt(data1); buffer2.order(ByteOrder.LITTLE_ENDIAN); buffer2.putShort(data2); // 翻转到大端序并读取数据buffer1.flip(); // 准备从缓冲区读取数据buffer1.order(ByteOrder.BIG_ENDIAN); int bigEndian1 = buffer1.getInt(); buffer2.flip(); buffer2.order(ByteOrder.BIG_ENDIAN); short bigEndian2 = buffer.getShort(); System.out.println("原int值: " + Integer.toHexString(data1)); System.out.println("大端模式int值: " + Integer.toHexString(bigEndian1)); System.out.println("原short值: " + Integer.toHexString(data2 & 0xFFFF)); System.out.println("大端模式short值: " + Integer.toHexString(bigEndian2 & 0xFFFF)); } }

For floating-point types (such as float and double), ByteBuffer can also be used for endian conversion.

 import java.nio.ByteBuffer; import java.nio.ByteOrder; public class FloatEndianConversion { public static void main(String[] args) { float data = 123.45f; // 使用ByteBuffer进行端序转换ByteBuffer buffer = ByteBuffer.allocate(4); // 分配足够的空间// 设置为小端序并写入数据buffer.order(ByteOrder.LITTLE_ENDIAN); buffer.putFloat(data); // 翻转到大端序并读取数据buffer.flip(); // 准备从缓冲区读取数据buffer.order(ByteOrder.BIG_ENDIAN); float bigEndian = buffer.getFloat(); System.out.println("原float值: " + data); System.out.println("大端模式float值: " + bigEndian); } }

The endian conversion of floating point numbers does not actually need to change the order of its internal bits. The IEEE 754 standard defines the format of floating point numbers. No matter which platform, as long as it is interpreted according to this standard, its value is consistent. If you need to store or transmit floating point numbers in the form of bytes and want the receiver to interpret these bytes in a different byte order, you need to use ByteBuffer for conversion.

In C, standard library functions are usually used for endian conversion, which allow developers to convert between network byte order (big endian) and host byte order. The network byte order is big endian, while the host byte order depends on the specific hardware architecture (it may be big endian or little endian).

For 16-bit and 32-bit integers, you can use the htons (host to network short), ntohs (network to host short), htonl (host to network long), and ntohl (network to host long) functions for conversion.

 #include <stdio.h> #include <arpa/inet.h> int main() { uint16_t short_host_order = 0x1234; uint32_t long_host_order = 0x12345678; // 转换到网络字节序（大端序） uint16_t short_net_order = htons(short_host_order); uint32_t long_net_order = htonl(long_host_order); // 转换回主机字节序uint16_t short_back_to_host = ntohs(short_net_order); uint32_t long_back_to_host = ntohl(long_net_order); printf("Host order short: %04x\n", short_host_order); printf("Network order short: %04x\n", short_net_order); printf("Back to host order short: %04x\n", short_back_to_host); printf("Host order long: %08x\n", long_host_order); printf("Network order long: %08x\n", long_net_order); printf("Back to host order long: %08x\n", long_back_to_host); return 0; }

For floating point numbers, there is no direct endian conversion function because the representation of floating point numbers includes an exponent and a mantissa, which are stored in memory in a complex way. Usually, a solution is to convert the floating point number to an integer type (such as uint32_t), then do the endian conversion, and then convert back to a floating point number.

 #include <stdio.h> #include <string.h> #include <arpa/inet.h> int main() { float f = 123.45f; uint32_t *int_ptr; uint32_t int_val; float f_net, f_back; // 将浮点数转换为整数memcpy(&int_val, &f, sizeof(f)); // 转换到网络字节序int_val = htonl(int_val); // 将整数转换回浮点数memcpy(&f_net, &int_val, sizeof(f_net)); // 转换回主机字节序int_val = ntohl(int_val); memcpy(&f_back, &int_val, sizeof(f_back)); printf("Original float: %f\n", f); printf("Network order float: %f\n", f_net); printf("Back to host order float: %f\n", f_back); return 0; }

The default byte order in the example is little endian. In actual applications, you can use the __BYTE_ORDER__ macro in GCC to check the byte order and perform endian conversion when necessary. For endian conversion of floating-point numbers, you need to pay attention to the impact of the IEEE 754 standard on floating-point representation, as well as the differences that may occur on different platforms and compilers.