MQTT protocol, someone finally explained it clearly

[[409407]]

This article is reprinted from the WeChat public account "Uncle Wheat", the author is Caidao and Xiaomai. Please contact Uncle Wheat's public account to reprint this article.

Hello everyone, I am Xiaomai. I recently worked on an IoT project and I would like to share with you the MQTT protocol. As we all know, the MQTT protocol is very commonly used in IoT. If you don’t know much about it, I believe this article can help you get started.

• MQTT protocol
• 1. Characteristics of MQTT protocol
  • Publish and Subscribe
  • QoS (Quality of Service levels)
• 2 MQTT Packet Structure
  • 2.1 MQTT fixed header
  • 2.2 MQTT variable header
  • 2.3 Payload message body
• 3 Environment Construction
  • 3.1 MQTT server construction
  • 3.2 MQTT Client
• 4 Conclusion

MQTT protocol

MQTT (Message Queuing Telemetry Transport) is a "lightweight" communication protocol based on the publish/subscribe model. The protocol is built on the TCP/IP protocol and was released by IBM in 1999.

The biggest advantage of MQTT is that it provides real-time and reliable message services for connecting remote devices with very little code and limited bandwidth.

As an instant messaging protocol with low overhead and low bandwidth usage, it is widely used in the Internet of Things, small devices, mobile applications, etc.

1. Characteristics of MQTT protocol

MQTT is a client-server based publish/subscribe messaging protocol.

The MQTT protocol is lightweight, simple, open, and easy to implement, making it applicable to a wide range of situations, including constrained environments such as machine-to-machine (M2M) communications and the Internet of Things (IoT).

It is widely used in sensors that communicate via satellite links, medical devices that occasionally dial up, smart homes, and some miniaturized devices.

The current version of the MQTT protocol is MQTT v3.1.1, which was released in 2014. In addition to the standard version, there is also a simplified version, MQTT-SN, which is mainly aimed at embedded devices that generally work on TCP/IP networks, such as ZigBee.

MQTT Like HTTP, MQTT runs on top of the Transmission Control Protocol/Internet Protocol (TCP/IP) stack.

MQTT OSI

Publish and Subscribe

MQTT uses the publish/subscribe messaging model, which provides a one-to-many message distribution mechanism, thereby achieving decoupling from applications.

This is a messaging mode where messages are not sent directly from sender to receiver (i.e. point-to-point), but are distributed by an MQTT server (or MQTT Broker).

The MQTT server is the core of the publish-subscribe architecture.

It can be implemented very simply on a single-board computer like a Raspberry Pi or NAS, but it can also be implemented on a mainframe or Internet server.

The server distributes messages and therefore must be a publisher, but never a subscriber!

Clients can publish messages (senders), subscribe to messages (receivers), or both.

A client (also called a node) is an intelligent device such as a microcontroller or a computer with a TCP/IP stack and software that implements the MQTT protocol.

Messages are published under topics that allow filtering. Topics are hierarchical UTF-8 strings. Different topic levels are separated by slashes /.

Let's take a look at the setup below.

• The photovoltaic power station is the publisher.
• The main Topic level is "PV", and this factory publishes two sub-levels "sunshine" and "data";
• "PV/sunshine" is a boolean value (true/fault, can also be 1/0) that the charging station needs to know if it should load the EV (only when the sun is shining :) ).
• The charging station (EVSE) is a subscriber and subscribes to "PV/sunshine" to obtain information from the server.
• "PV/data" on the other hand transmits the instantaneous power generated by the plant in kW, and this topic can be subscribed to, for example, via a computer or tablet to generate a graph of the transmitted power over a day.

This is a simple MQTT application scenario, as shown in the following figure;

MQTT Publish and Subscribe

QoS (Quality of Service levels)

Quality of service is an important feature of MQTT. When we use TCP/IP, the connection is already protected to a certain extent. But in wireless networks, interruptions and interference are frequent, and MQTT is here to help avoid information loss with its quality of service levels. These levels are used when publishing. If a client publishes to an MQTT server, the client will be the sender and the MQTT server will be the receiver. When an MQTT server publishes a message to a client, the server is the sender and the client is the receiver.

QoS 0

At this level, message loss or duplication may occur, and message publishing depends on the underlying TCP/IP network. That is: <= 1

QoS 1

QoS 1 promises that a message will be delivered at least once to a subscriber.

QoS 2

With QoS 2, we guarantee that the message is delivered to the destination only once. To do this, the message with a unique message ID is stored twice, first from the sender and then by the receiver. QoS level 2 has the highest overhead in the network because two flows are required between the sender and the receiver.

2. MQTT data packet structure

• Fixed header, present in all MQTT packets, indicates the packet type and packet class identifier;
• Variable header, which exists in some MQTT packets. The packet type determines whether the variable header exists and its specific content;
• The message body (Payload) exists in some MQTT packets and indicates the specific content received by the client;

The overall MQTT message format is shown in the figure below;

2.1 MQTT fixed header

The fixed header is present in all MQTT packets and has the following structure:

The following is a brief analysis of the fixed header message format;

MQTT message type

**Location:** byte 1, bits 7-4.

A 4-bit unsigned value of the following type:

Flag/DUP

**Location:** byte 1, bits 3-0.

In message types that do not use the flag, the flag is reserved. If an invalid flag is received, the receiver must close the network connection:

• DUP: Publish a copy of the message. Used to ensure reliable transmission of messages. If set to

1, then MessageId is added to the variable length below, and a reply confirmation is required to ensure that the message transmission is completed, but it cannot be used to detect repeated message sending.

• QoS is the quality of service for publishing messages (introduced earlier), that is, the number of times the message is guaranteed to be delivered
  • 00: at most once, i.e.: <= 1
  • 01: At least once, that is: >=1
  • 10: once, that is: =1
  • 11: Reserved
• RETAIN: Releases the retention flag, indicating that the server wants to retain the information pushed this time. If a new subscriber appears, the message will be pushed to it. If set, the message will be released after being pushed to the current subscriber.

Remaining Length

Location: byte 1.

The second byte of the fixed header is used to save the total size of the variable-length header and the message body, but it is not saved directly. This byte can be expanded. Its saving mechanism is that the first 7 bits are used to save the length, and the last bit is used as an identifier. When the last bit is 1, it means that the length is insufficient and two bytes are needed to continue saving. For example: the calculated size after is 0

2.2 MQTT variable header

The MQTT data packet contains a variable header, which resides between the fixed header and the payload. The content of the variable header varies depending on the data packet type, and is often used as a packet identifier:

Many types of packets include a 2-byte packet identification field. These types of packets are:

PUBLISH (QoS > 0), PUBACK, PUBREC, PUBREL, PUBCOMP,

SUBSCRIBE, SUBACK, UNSUBSCRIBE, UNSUBACK

2.3 Payload message body

The Payload message body is the third part of the MQTT data packet. The four types of messages, CONNECT, SUBSCRIBE, SUBACK, and UNSUBSCRIBE, have message bodies:

CONNECT, the message body mainly contains: ClientID, subscribed Topic, Message, and username and password

SUBSCRIBE, the message body is a series of topics to be subscribed and QoS.

SUBACK: The message body is the server's confirmation and response to the topic and QoS requested by SUBSCRIBE.

UNSUBSCRIBE, the message body is the topic to be subscribed.

3. Environment Construction

After introducing the basic theory, let's build a simple MQTT application on the Windows platform and perform a simple application. The overall architecture is shown in the figure below;

[External link image transfer failed. The source site may have an anti-hotlink mechanism. It is recommended to save the image and upload it directly (img-ScRucIVO-1625480723109) (architecture diagram.png)]

3.1 MQTT server construction

Currently, the mainstream platforms of MQTT agents are as follows:

• Mosquitto: https://mosquitto.org/
• VerneMQ: https://vernemq.com/
• EMQTT: http://emqtt.io/

This article will use Mosquitoo for testing. Go to the installation page and download the program that is compatible with your computer system.

Download Page

After successful installation, go to the installation path and find mosquitto.exe;

[External link image transfer failed. The source site may have an anti-hotlink mechanism. It is recommended to save the image and upload it directly (img-YXZupgOv-1625480723111)(image-20210705171401654.png)]

Hold down Shift, right-click a blank space, then open Powershell and open a terminal software normally;

• Enter ./mosquitto.exe -h to view the corresponding help;
• Enter ./mosquitto.exe -p 10086 to start the MQTT service. The listening address is 127.0.0.1 and the port is 10086.

The details are shown in the figure below;

3.2 MQTT Client

Now that the server is set up, the next step is to open the client and publish and subscribe so that the corresponding messages can be transmitted.

Here I use a QT mqtt client program compiled by myself, which is compiled based on the official library of Qt. Let's open this software. In the next issue, I will briefly introduce how to complete this client and set the corresponding parameters:

• Address: 127.0.0.1
• Port: 10086

Then subscribe to the topic and send data to each other, as shown in the following figure;

Combined with the previous pictures, the overall architecture is as follows;

4. Summary

This article briefly introduces the working principle of the MQTT protocol and the corresponding protocol format, briefly introduces some details of the protocol, and specifically cites the corresponding application scenarios. The author's level and ability are limited, and there are inevitably errors and omissions in the article. Please feel free to give me your advice.

This episode ends here. I am Xiaomai. See you next time.