Do you know some new features of RocketMQ 5.0? Let me tell you.

In order to evolve towards cloud native and improve resource utilization and elasticity, RocketMQ adjusted and upgraded its architecture in 5.0.

Proxy layer

Architecture before RocketMQ5.0:

Previously, RocketMQ5.0 used a custom Remoting protocol based on Netty for network communication. Computing and storage were integrated and all in Broker.

Producers and consumers pull routing information from NameServer, and then interact directly with Broker to produce and consume messages.

picture

RocketMQ5.0 architecture:

After 5.0, the elastic stateless proxy mode was introduced, and the responsibilities of the Broker were split.

The computing logic such as client protocol adaptation, permission management, and consumption management are extracted and placed in the Proxy layer, and Broker focuses on data storage.

• In order to better adapt to the cloud-native environment and achieve elastic resource scheduling.

And support for the GRPC protocol was added after 5.0.

• It is Google's open source high-performance RPC framework based on Protobuf serialization.

picture

POP consumption model

Before RocketMQ 5.0:

There are two ways to get messages from Broker: Pull mode and Push mode.

Pull mode:

Consumers need to continuously obtain data from the blocking queue. If there is no data, they will wait. The data in this blocking queue is added by the message pulling thread after pulling messages from the Broker.

Therefore, consumption in Pull mode needs to continuously and actively pull messages from Broker.

Push mode:

A message listener needs to be registered. When a message arrives, the message will be consumed through a callback function. On the surface, it looks like the Broker actively pushes it to the consumer, so it is called push mode.

The underlying layer is still that the consumer pulls data from the Broker and then triggers the callback function to consume the message, but there is no need to constantly determine whether there is a message arriving like in the Pull mode.

RocketMQ5.0

Functions such as load balancing and consumer site management are placed on the Broker side to reduce the burden on the client, making it lightweight, and after 5.0, message-based load balancing is supported.

Message granularity load balancing:

In the message granularity load balancing strategy, multiple consumers in the same consumer group will evenly share all messages in the topic according to the message granularity.

• That is, messages in the same queue can be evenly distributed to multiple consumers in the group for common consumption.

POP message consumption:

First, the client (consumer) sends a Pop request to the server (Broker). After receiving the request, the Broker obtains the message in Pop mode and then returns it to the client.

After the client successfully consumes the message, it sends an ACK request to the Broker to confirm that the message consumption is successful.

picture

Controller Mode

Before RocketMQ5.0, there were two cluster deployment modes, namely Master-Slave mode and Dledger mode.

RocketMQ 5.0 and later introduced the Controller mode, which has the following features:

In the master-slave deployment mode, it has the ability to automatically switch the Master. Before 5.0, DLedger was required.

You can use RocketMQ's native storage and replication capabilities and unify RocketMQ's storage and replication capabilities.

RocketMQ 5.0 has separated the Broker leader election related functions and placed them in the Controller.

Automatic switching of Master is achieved in master-slave deployment mode. Controller can be deployed independently or embedded in NameServer.

Controller under independent deployment:

picture

The deployment diagram embedded in NameServer is as follows:

picture

refer to:

https://rocketmq.apache.org/version/

https://developer.aliyun.com/article/801815

https://rocketmq.apache.org/zh/docs/deploymentOperations/03autofailover/