How to implement a custom serial communication protocol?

[[402368]]

This article is reprinted from the WeChat public account "strongerHuang", the author is strongerHuang. Please contact the WeChat public account "strongerHuang" to reprint this article.

Some beginners always think that communication protocol is a very complicated knowledge, and think it is very profound, which leads to them not knowing how to learn it.

At the same time, occasionally some readers ask questions about the serial port custom communication protocol. Today, let’s write about the serial port communication protocol. It is not as difficult as you think.

1. What is the communication protocol?

The communication protocol is not difficult to understand. It is a protocol that must be followed for communication between two (or more) devices.

Baidu Encyclopedia's explanation:

Communication protocols refer to the rules and agreements that two entities must follow to complete communication or services. In order for multiple data communication systems in different geographical locations to work together to achieve information exchange and resource sharing, they must have a common language. What to communicate, how to communicate, and when to communicate must follow certain mutually acceptable rules. This rule is the communication protocol.

Relatively speaking, many readers have bought some modules based on serial communication. Many modules based on serial communication on the market have custom communication protocols, some of which are relatively simple and some are relatively complex.

Let's take a very simple serial communication protocol as an example: for example, only a temperature value is transmitted, and there are only three bytes of communication protocol:

Doesn't this look simple? It is also a communication protocol.

It’s just that the application scenarios of this communication protocol are relatively simple (one-to-one between two devices), and at the same time, it has many disadvantages.

2. Problems caused by overly simple communication protocols

I believe everyone understands the communication protocol with only three bytes above. Although it can communicate and transmit data, it has a series of problems.

For example: multiple devices are connected to a bus (such as 485), how to determine to whom the transmission is sent? (no device information)

For example: In a noisy environment, can you ensure that the transmitted data is correct? (No verification information)

For example: I want to transmit multiple data of uncertain length, what should I do? (no length information).

I believe that friends who have done custom communications understand the above series of problems.

Therefore, more "protocol information" should be agreed upon in the communication protocol to ensure the integrity of the communication.

3. Common contents of communication protocol

The communication protocol based on the serial port is usually not too complicated. Because of the serial port communication rate, anti-interference ability and other reasons, it is a very lightweight communication protocol compared to the communication protocol such as TCP/IP.

Therefore, in addition to some common communication protocols (such as Modubs and MAVLink) based on serial port communication, engineers often customize communication protocols according to their own project conditions.

The following briefly describes some key points of common custom communication protocols.

(These are some common agreement contents. The agreement contents may be different in different situations.)

1. Frame header

The frame header is the beginning of a frame of communication data.

Some communication protocols have only one frame header, while others have two, for example: 5A and A5 as frame headers.

2. Device address/type

Device address or device type is usually used between multiple devices to facilitate the distinction between different devices.

In this case, it is necessary to describe various device type information in the protocol or appendix to facilitate developers' coding queries.

Of course, for some fixed communications between two devices, this option may not be available.

3. Commands/Instructions

Commands/instructions are quite common, and different operations are usually distinguished by different commands.

For example: Temperature: 0x01; Humidity: 0x02;

4. Command type/function code

This option further supplements the command, such as read and write operations.

For example: Read Flash: 0x01; Write Flash: 0x02;

5. Data length

The data length option may be moved to the front device address by some protocols, and the command information will be counted in the "length".

This is mainly to facilitate the statistics of received data length during protocol (receiving) parsing.

For example, sometimes you need to transmit one valid data, sometimes you need to transmit multiple valid data, or even transmit an array of data. At this time, the transmitted frame of data is indefinite length data, so it must be constrained by [data length].

Some are one byte in length, ranging from 0x01 to 0xFF. Some may require more to be transmitted at one time, so they are represented by two bytes, ranging from 0x0001 to 0xFFFFF.

Of course, some communications have a fixed length (for example, only two data, temperature and humidity, are transmitted), and the protocol may not have this option.

6. Data

There is no need to describe the data, it is the real data you transmit, such as temperature: 25℃.

7. Frame tail

Some protocols may not have a frame tail, which should be an optional option.

8. Verification code

The checksum is a relatively important content. Generally, formal communication protocols have this option. The reason is very simple. Communication can be easily disturbed or other reasons can cause errors in transmitted data.

If there is a check code, data transmission errors can be avoided more effectively.

There are many ways to verify the code. Checksum and CRC are relatively common and are used as verification methods in custom protocols.

Another point is that some protocols may place the checksum at the second to last position and the frame tail at the last position.

4. Communication protocol code implementation

There are many ways to implement custom communication protocols in code. As the saying goes, "all roads lead to Rome". You only need to write the implementation code according to your protocol.

Of course, when implementing it, you need to consider the actual situation of your project. For example, if there is a lot of communication data, you need to use a message queue (FIFO). For example, if the protocol is complex, it is best to encapsulate the structure.

Here are some codes I used before. They may not describe more details, but some ideas can be used as reference.

1. Message data sending

a. Send each byte directly through the serial port

This is understandable to novices. Here is an example of a previous DGUS serial screen:

 #define DGUS_FRAME_HEAD1 0xA5 //DGUS screen frame header 1
 #define DGUS_FRAME_HEAD2 0x5A //DGUS screen frame header 2 
 
 #define DGUS_CMD_W_REG 0x80 //DGUS write register instruction
 #define DGUS_CMD_R_REG 0x81 //DGUS read register instruction
 #define DGUS_CMD_W_DATA 0x82 //DGUS write data command
 #define DGUS_CMD_R_DATA 0x83 //DGUS read data instruction
 #define DGUS_CMD_W_CURVE 0x85 //DGUS write curve command 
 
 /* DGUS register address */
 #define DGUS_REG_VERSION 0x00 //DGUS version
 #define DGUS_REG_LED_NOW 0x01 //LED backlight brightness
 #define DGUS_REG_BZ_TIME 0x02 //Buzzer duration
 #define DGUS_REG_PIC_ID 0x03 //Display page ID
 #define DGUS_REG_TP_FLAG 0x05 //Touch coordinate update flag
 #define DGUS_REG_TP_STATUS 0x06 //Coordinate status
 #define DGUS_REG_TP_POSITION 0x07 //Coordinate position
 #define DGUS_REG_TPC_ENABLE 0x0B //Touch enable
 #define DGUS_REG_RTC_NOW 0x20 //Current RTCS 
 
 //Write one byte of data to the specified register of the DGDS screen
 void DGUS_REG_WriteWord(uint8_t RegAddr, uint16_t Data)
 {
  DGUS_SendByte(DGUS_FRAME_HEAD1);
  DGUS_SendByte(DGUS_FRAME_HEAD2);
 DGUS_SendByte(0x04); 
 
  DGUS_SendByte(DGUS_CMD_W_REG); //Command
 DGUS_SendByte(RegAddr); //Address 
 
 DGUS_SendByte((uint8_t)(Data>>8)); //data
  DGUS_SendByte((uint8_t)(Data&0xFF));
 } 
 
 //Write one byte of data to the specified address of the DGDS screen
 void DGUS_DATA_WriteWord(uint16_t DataAddr, uint16_t Data)
 {
  DGUS_SendByte(DGUS_FRAME_HEAD1);
  DGUS_SendByte(DGUS_FRAME_HEAD2);
 DGUS_SendByte(0x05); 
 
  DGUS_SendByte(DGUS_CMD_W_DATA); //Command 
 
 DGUS_SendByte((uint8_t)(DataAddr>>8)); //Address
  DGUS_SendByte((uint8_t)(DataAddr&0xFF)); 
 
 DGUS_SendByte((uint8_t)(Data>>8)); //data
  DGUS_SendByte((uint8_t)(Data&0xFF));
 }

b. Send through message queue

Based on the above, use a buf to hold the message, then "package" it into the message queue and send it out through the message queue method (FIFO).

 static uint8_t sDGUS_SendBuf[DGUS_PACKAGE_LEN]; 
 
 //Write one byte of data to the specified register of the DGDS screen
 void DGUS_REG_WriteWord(uint8_t RegAddr, uint16_t Data)
 {
  sDGUS_SendBuf[0] = DGUS_FRAME_HEAD1; //Frame header
  sDGUS_SendBuf[1] = DGUS_FRAME_HEAD2;
 sDGUS_SendBuf[2] = 0x06; //length
  sDGUS_SendBuf[3] = DGUS_CMD_W_CTRL; //Command
 sDGUS_SendBuf[4] = RegAddr; //Address
 sDGUS_SendBuf[5] = (uint8_t)(Data>>8); //data
  sDGUS_SendBuf[6] = (uint8_t)(Data&0xFF); 
 
  DGUS_CRC16(&sDGUS_SendBuf[3], sDGUS_SendBuf[2] - 2, &sDGUS_CRC_H, &sDGUS_CRC_L);
 sDGUS_SendBuf[7] = sDGUS_CRC_H; //Check
  sDGUS_SendBuf[8] = sDGUS_CRC_L; 
 
  DGUSSend_Packet_ToQueue(sDGUS_SendBuf, sDGUS_SendBuf[2] + 3);
 } 
 
 //Write one byte of data to the specified address of the DGDS screen
 void DGUS_DATA_WriteWord(uint16_t DataAddr, uint16_t Data)
 {
  sDGUS_SendBuf[0] = DGUS_FRAME_HEAD1; //Frame header
  sDGUS_SendBuf[1] = DGUS_FRAME_HEAD2;
 sDGUS_SendBuf[2] = 0x07; //length
  sDGUS_SendBuf[3] = DGUS_CMD_W_DATA; //Command
 sDGUS_SendBuf[4] = (uint8_t)(DataAddr>>8); //Address
  sDGUS_SendBuf[5] = (uint8_t)(DataAddr&0xFF);
 sDGUS_SendBuf[6] = (uint8_t)(Data>>8); //data
  sDGUS_SendBuf[7] = (uint8_t)(Data&0xFF); 
 
  DGUS_CRC16(&sDGUS_SendBuf[3], sDGUS_SendBuf[2] - 2, &sDGUS_CRC_H, &sDGUS_CRC_L);
 sDGUS_SendBuf[8] = sDGUS_CRC_H; //Check
  sDGUS_SendBuf[9] = sDGUS_CRC_L; 
 
  DGUSSend_Packet_ToQueue(sDGUS_SendBuf, sDGUS_SendBuf[2] + 3);
 }

c. Use "structure" instead of "array SendBuf" method

The structure is more convenient for array reference and management, so the structure method is more advanced and more practical than the array buf. (Of course, if there are many members, using temporary variables will also cause too much stack to be occupied)

for example:

 typedef struct
 {
 uint8_t Head1; //Frame header 1
 uint8_t Head2; //Frame header 2
 uint8_t Len; //length
 uint8_t Cmd; //Command
 uint8_t Data[DGUS_DATA_LEN]; //data
 uint16_t CRC16; //CRC check
 }DGUS_PACKAGE_TypeDef;

d. Others

There are many ways to send data via serial port, such as using DMA instead of message queue.

2. Message data reception

Serial port message reception usually uses serial port interrupt reception. Of course, there are also rare cases where data is received by polling.

a. Regular interrupt reception

Taking the DGUS serial port screen as an example, a simple and common interrupt receiving method is described:

 void DGUS_ISRHandler(uint8_t Data)
 {
 static uint8_t sDgus_RxNum = 0; //Number
 static uint8_t sDgus_RxBuf[DGUS_PACKAGE_LEN];
 static portBASE_TYPE xHigherPriorityTaskWoken = pdFALSE; 
 
  sDgus_RxBuf[gDGUS_RxCnt] = Data;
 gDGUS_RxCnt++; 
 
 /* Determine the frame header */
 if(sDgus_RxBuf[0] != DGUS_FRAME_HEAD1) //Receive frame header 1
 {
 gDGUS_RxCnt = 0;
 return ;
 }
  if((2 == gDGUS_RxCnt) && (sDgus_RxBuf[1] != DGUS_FRAME_HEAD2))
 {
 gDGUS_RxCnt = 0;
 return ;
 } 
 
 /* Determine the length of a frame of data */
 if(gDGUS_RxCnt == 3)
 {
    sDgus_RxNum = sDgus_RxBuf[2] + 3;
 } 
 
 /* Receive a frame of data */
  if((6 <= gDGUS_RxCnt) && (sDgus_RxNum <= gDGUS_RxCnt))
 {
 gDGUS_RxCnt = 0; 
 
 if(xDGUSRcvQueue != NULL ) //Parsing successful, join the queue
 {
      xQueueSendFromISR(xDGUSRcvQueue, &sDgus_RxBuf[0], &xHigherPriorityTaskWoken);
      portEND_SWITCHING_ISR(xHigherPriorityTaskWoken);
 }
 }
 }

b. Add timeout detection

The received data may be half received and the interrupt may be interrupted for some reason. At this time, timeout detection is also necessary.

For example: use the redundant MCU timer to do a timeout counting process, receive a data, start timing, if the next data is not received within 1ms, discard this packet of data (previously received).

 static void DGUS_TimingAndUpdate(uint16_t Nms)
 {
 sDGUSTiming_Nms_Num = Nms;
 TIM_SetCounter(DGUS_TIM, 0); //Set the count value to 0
 TIM_Cmd(DGUS_TIM, ENABLE); //Start the timer
 } 
 
 void DGUS_COM_IRQHandler(void)
 {
  if((DGUS_COM->SR & USART_FLAG_RXNE) == USART_FLAG_RXNE)
 {
 DGUS_TimingAndUpdate(5); //Update timing (to prevent timeout)
    DGUS_ISRHandler((uint8_t)USART_ReceiveData(DGUS_COM));
 }
 }

c.More

There are many ways to implement receiving, just like sending. For example, receiving can also use the structure method. But one thing is that you need to code according to your actual needs.

5. Finally

The above custom protocol content is for reference only. What you use and how many bytes you occupy depend on your actual needs.

There are many differences in custom communication protocols based on serial ports. For example, the MCU processing power, the number of devices, the communication content, etc. are all related to your custom protocol.

Some may only require a very simple communication protocol to meet the requirements, while others may require a more complex protocol.

Finally, I would like to emphasize two points:

1. The above examples are not complete codes (some details are not described), but are mainly for everyone to learn this programming idea or implementation method.

2. A good communication protocol code must have certain fault tolerance, such as: sending completion detection, receiving timeout detection, data error detection, etc. Therefore, the above code is not a complete code.