Message bus for communication between processes

• 1. Inter-process communication (IPC) in Linux system
• 2. Advantages of Socket-based communication
• 3. MQTT Message Bus
• 4. How to use MQTT message bus in embedded systems
• 5. Mosquitto: A simple test code
• VI. Conclusion
• 7. Resource Download

1. Inter-process communication (IPC) in Linux system

As an embedded software developer, it is common to deal with communication between processes. From the perspective of communication purpose, we can divide the communication between processes into 3 types:

1. For process scheduling: this can be achieved through signals;
2. To share resources: this can be achieved through mutex locks, semaphores, read-write locks, file locks, etc.
3. In order to transfer data: it can be achieved through shared memory, named pipes, message queues, and Sockets.

There are a lot of information and articles about the communication primitives provided by the operating system mentioned above, so I won't go into details here. How should we choose among these methods? According to my personal experience, quality is more important than quantity. Carefully selecting three or four things can fully meet the needs of daily work.

The issue we want to discuss today is mainly the third one: transferring data. Among the above methods of transferring data, my favorite and most commonly used method is Socket communication.

Some friends may say: Socket communication is the same as TCP/IP, and you need to manage the connection, package and subpackage the data yourself, which is also quite troublesome.

Yes, Socket communication itself does require manual processing of these underlying things, but we can give Socket a "coat": use the MQTT message bus to exchange data between various processes in the system. Let's talk about it one by one.

2. Advantages of Socket-based communication

I will not elaborate on this myself, but will directly quote the viewpoint in Professor Chen Shuo's book "Linux Multithreaded Server Programming" (page 65, section 3.4):

1. Cross-host and scalable

Anyway, it is multi-process. If the processing power of one machine is not enough, multiple hosts can be used to handle it. Distribute the processes to multiple machines in the same LAN, and the program can continue to work by changing the Host:Port configuration. On the contrary, the communication methods between processes listed at the beginning of the article cannot cross machines, which limits scalability.

2. The operating system will automatically recycle resources

The TCP port is exclusively used by one process. When the program exits unexpectedly, the operating system will automatically reclaim resources and will not leave garbage in the system. The program can be easily restored after restarting.

3. Recordable and reproducible

Two processes communicate via TCP. If one crashes, the operating system will close the connection, and the other process will sense it almost immediately and failover quickly. Of course, the heartbeat at the application layer is essential. (Additional information: The operating system itself has a keep-alive time for TCP connections, which is 2 hours by default and is global.)

4. Cross-language

The server and client do not have to use the same programming language.

1. Professor Chen Shuo described general Socket communication, so the client and server are generally located on different physical machines.

2. In embedded development, the same programming language is generally used, so cross-language issues can be ignored.

3. MQTT Message Bus

1. MQTT is a communication mechanism

Friends who are familiar with the Internet of Things field must be very familiar with the MQTT message bus. Currently, several major IoT cloud platforms (Amazon, Alibaba Cloud, and Huawei Cloud) provide access methods for the MQTT protocol.

Currently, the best documentation for learning MQTT is IBM's online manual: https://developer.ibm.com/zh/technologies/messaging/articles/iot-mqtt-why-good-for-iot/.

Here, I will list some key information directly:

• The MQTT protocol is lightweight, simple, open and easy to implement;
• MQTT is a messaging protocol based on the Publish/Subscribe paradigm;
• MQTT works on the TCP/IP protocol suite;
• There are three message publishing qualities of service;
• Small transmission, low overhead (fixed-length header is 2 bytes), and minimal protocol exchanges to reduce network traffic;

MQTT message transmission requires a middleware called Broker, which is actually a Server. The communication model is as follows:

1. MQTT Broker needs to be started first;
2. ClientA and ClientB need to connect to Broker;
3. ClientA subscribes to topic_1, ClientB subscribes to topic_2;
4. ClientA sends a message to topic_2, which will be received by ClientB;
5. ClientB sends a message to topic_1, which will be received by ClientA;

A great advantage of topic-based communication is decoupling. A client can subscribe to multiple topics, and any other client connected to the bus can send information to these topics (a client can also send messages to itself).

2. Implementation of MQTT

MQTT is just a protocol. From IBM's online documentation, we can see that many languages ​​have implemented the MQTT protocol, including: C/C++, Java, Python, C#, JavaScript, Go, Objective-C, etc. For embedded development, the following implementations are more commonly used:

• Mosquitto;
• Paho MQTT;
• wolfMQTT;
• MQTTRoute.

Below, we will focus on the compilation and use of the open source implementation of Mosquitto, which is also what I use most in my projects.

3. Design your own communication protocol on top of MQTT

From the above description, it can be seen that the MQTT message bus is a communication mechanism that provides a channel for communication entities to transmit data.

On this channel, we can send data in any format and encoding according to the needs of the actual project. In the project, we most often use plain text in json format, which is also the method recommended by various IoT cloud platforms. If binary data needs to be included in the text data, convert it to BASE64 encoding before sending.

4. How to use MQTT message bus in embedded systems

From the above description, we can see that as long as an MQTT Broker service is running on the server side, each client connected to the bus can flexibly send and receive data to each other.

We can apply this mechanism in the design of embedded applications: MQTT Broker runs as an independent service locally in the embedded system. Other processes that need to interact can send data to each other as long as they connect to the local Broker. The operation model is as follows:

Each process only needs to subscribe to a fixed topic (for example, its own client ID). If other processes want to send data to it, they can send it directly to this topic.

1. A communication framework for embedded systems

I have previously developed an environmental monitoring system that collects PM2.5, PM10 and other pollutant parameters in the atmosphere. It was developed on the Contex A8 platform and required the implementation of functions such as data recording (database) and UI monitoring interface.

The pollutant data sampling hardware module is provided by a third-party company. We only need to use the serial port protocol provided by the module to control the sampling equipment and receive the sampling data. The final communication model is as follows:

• The UI process sends control instructions to the sampling control process through the message bus. After receiving the control instructions, the sampling control process sends the control instructions to the sampling module through the serial port.
• After the sampling control process receives the PM2.5 and other data from the sampling module from the serial port, it sends all the data to the specified topic on the message bus;
• The UI process subscribes to the topic and displays the received data on the screen;
• The database process also subscribes to the topic and, after receiving the data, stores it in the SQLite database;

In this product, the core process is the sampling control process, which is responsible for interacting with the sampling module. By designing UI processing and database processing as independent processes, the complexity of the system is reduced. Even if these two processes crash, it will not affect the core sampling control process.

For example, if the UI process crashes due to an error, it will restart immediately. After restarting, it will know through cache information that sampling is currently being performed. Then the UI process will immediately connect to the message bus, enter the sampling data display interface, and continue to receive and display PM2.5 and other data sent by the sampling control process.

This communication model has another advantage: scalability.

In the later stage of project development, Party A said that it was necessary to integrate a third-party gas module to collect parameters such as NO and SO2 in the atmosphere, and the communication method was RS485.

At this point, it is very simple to expand this functional module. Just write an independent gas parameter process and connect it to the message bus. When this process receives gas parameters such as NO and SO2 from the third-party gas module through RS485, it directly throws it to a topic on the message bus. The UI process and database process subscribe to this topic and can immediately receive gas-related data.

In addition, this design model has some other advantages:

• Parallel development: Each process can be developed in parallel by different people, as long as the communication protocol is defined between them;
• Easy debugging: Since the data sent is manually readable, during the development phase, a monitoring program can be written on the PC and connected to the MQTT Broker in the embedded system, so that messages sent by all processes can be received;
• Communication security: After the product is released, in order to prevent others from eavesdropping on the data (such as the debugging process in 2), you can specify a configuration file for the MQTT Broker that only allows local processes (127.0.0.1) to connect to the message bus.

2. A slightly more complex communication model

In the embedded system framework design just described, each process runs locally, and all messages are sent and received within the system. So, if you need to transmit data to the cloud, or need to receive some control instructions from the cloud, how should you design it?

Just add an MQTT Bridge module! That is, add another process, which connects to the cloud MQTT Broker and the local MQTT Broker at the same time. The communication model is as follows:

When MQTT Bridge receives instructions from the cloud, it forwards them to the local message bus;

When MQTT Bridge receives a local message, it forwards it to the cloud message bus.

5. Mosquitto: A simple test code

The above content mainly discusses the design ideas. When it comes to the code level, I generally use the open source implementation of Mosquitto.

It is very convenient to install and test in Linux system, and the following is a brief explanation.

1. Install and test directly through apt

You can refer to this document (https://www.Vultr.com/docs/how-to-install-mosquitto-mqtt-broker-server-on-ubuntu-16-04) to install and test.

(1) Installation

 sudo apt- add -repository ppa:mosquitto-dev/mosquitto-ppa
 sudo apt-get update  
 sudo apt-get install mosquitto
 sudo apt-get install mosquitto-clients

(2) Testing

The mosquitto broker will start automatically after installation. You can check port 1883 with netstat to confirm.

Receiver: After connecting to the broker, subscribe to the "test" topic.

 mosquitto_sub -t "test"  

Sending end: After connecting to the broker, send the string "hello" to the topic "test".

 mosquitto_pub -m "hello" -t "test"  

When the sending end executes mosquitto_pub, the string "hello" can be received in the terminal window of the receiving end.

2. Manually compile and test from source code

Installation through apt is mainly used for simple learning and testing. If you want to use Mosquitto in project development, you must compile it manually, get the header files and library files, and then copy them to the application for use.

(1) Manually compile and install Mosquitto

My development environment is:

Compiler: gcc (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609

• Mosquitto version: mosquitto-1.4.9
• mosquitto-1.4.9 can be downloaded from the official website or from the network disk at the end of the article. You can also try a higher version.

Compile and install instructions:

 make
 make install prefix=$PWD/install

After successful installation, you can see the output files in the install folder of the current directory:

• bin: mqtt client program;
• include: header files that the application needs to include;
• lib: library files that the application needs to link;
• sbin: mqtt broker service program.

During the compilation process, if you encounter some errors such as: ares.h, uuid.h and other dependent files cannot be found, just install the corresponding development package through the apt command.

(2) The simplest mosquitto client code

In the mosquitto source code, a wealth of sample examples are provided. If you are not willing to explore, you can directly download the Demo sample program in the network disk at the end of the article. After connecting to the message bus, this program subscribes to the topic "topic_01". Of course, you can also modify the code to send messages (call: mosquitto_publish function).

After entering the c_mqtt sample code directory, you can see that it already contains bin, include, and lib directories, which are copied from the installation directory install in (1) above.

After executing the make command, the compilation is successful and an executable file is obtained: mqtt_client.

The testing process is as follows:

Step 1: Start MQTT Broker

In the first terminal window, start the broker program sbin/mosquitto. If you have already started a broker in the above test, you need to kill the previous broker first, because they both use port 1883 by default and cannot coexist.

Step 2: Start the receiving program mqtt_client

In the second terminal window, start mqtt_client, which is the executable program compiled from our sample code. The topic it subscribes to is "topic_01".

 ./mqtt_client 127.0.0.1 1883

• Parameter 1: The IP address of the Broker service. Since it is in the local system, it is 127.0.0.1;
• Parameter 2: port number, usually the default is 1883.

Step 3: Start the sender program bin/mosquitto_pub

In the third terminal window, start bin/mosquitto_pub with the following command:

 ./mosquitto_pub -h 127.0.0.1 -p 1883 -m "hello123" -t "topic_01"  

• Parameter -h: IP address of Broker service, because it is in the local system, it is 127.0.0.1; parameter -p: port number 1883;
• Parameter -m: the content of the message sent;
• Parameter -t: the topic to be sent.

At this point, you can print out the received messages in the second terminal window (mqtt_client).

VI. Conclusion

This article mainly introduces a design pattern in embedded systems: using a message bus to achieve communication between processes, and introduces the open source implementation of Mosquitto.

In actual projects, stricter permission control is also required. For example, when accessing the message bus, a user name, password, and device certificate must be provided. The client name must conform to the specified format, the subscribed topic must conform to a certain format, and so on.

In the next article, we will continue to discuss this topic and give a more specific and practical Demo example.

7. Resource Download

1. mosquitto-1.4.9.tgz

Link: https://pan.baidu.com/s/1izQ3dAlGbHiHwDvKnOSfyg

Password:dozt

2. Mosquitto Demo Sample Code

Link: https://pan.baidu.com/s/1M-dU3xapNbKyk2w07MtDyw

Password: aup3

This article is reprinted from the WeChat public account "IOT Internet of Things Town", which can be followed through the following QR code. To reprint this article, please contact the IOT Internet of Things Town public account.