Netty - Sticky Packets and Half Packets (Part 2)

Continue from the previous article "Introduction and Solution of TCP Sticky Packet and Half Packet (Part 1)"

The previous article introduced sticky packets and half packets and their general solutions. Today, let’s focus on how Netty implements the encapsulation framing solution.

Decoding core process

Previously, we introduced three decoders: FixedLengthFrameDecoder, DelimiterBasedFrameDecoder, and LengthFieldBasedFrameDecoder. They all inherit from ByteToMessageDecoder, which inherits from ChannelInboundHandlerAdapter, and its core method is channelRead. Therefore, let's take a look at the channelRead method of ByteToMessageDecoder:

 @Override
 public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
 if (msg instanceof ByteBuf) {
 CodecOutputList out = CodecOutputList .newInstance();
 try {
 //Convert the incoming message into data
 ByteBuf data = (ByteBuf) msg;
 // The ultimate goal is to put all the data into cumulation
 first = cumulation == null;
 // The first data is put directly into
 if (first) {
 cumulation = data ;
 } else {
 // If it is not the first data, append
 cumulation = cumulator .cumulate(ctx.alloc(), cumulation, data);
 }
 // Decode
 callDecode(ctx, cumulation, out);
 }
 // The following code is omitted because it does not belong to the decoding process
 }

Let's take a look at the callDecode method:

 protected void callDecode(ChannelHandlerContext ctx, ByteBuf in, List < Object > out) {
 try {
 while (in.isReadable()) {
 int out outSize = out.size();
 if (outSize > 0) {
 // The following code is omitted because in the initial state, outSize can only be 0 and it is impossible to enter here
 }
 int oldInputLength = in .readableBytes();
 // When decoding, the handler's remove operation is not executed.
 // Only after decode is executed, start cleaning data.
 decodeRemovalReentryProtection(ctx, in, out);
 // Omit the following code because the following content is not the decoding process

Let's take a look at the decodeRemovalReentryProtection method:

 final void decodeRemovalReentryProtection(ChannelHandlerContext ctx, ByteBuf in, List < Object > out)
 throws Exception {
 // Set the current state to decoding
 decodeState = STATE_CALLING_CHILD_DECODE ;
 try {
 // Decode
 decode(ctx, in, out);
 finally
 // Execute the remove operation of hander
 boolean removePending = decodeState == STATE_HANDLER_REMOVED_PENDING;
 decodeState = STATE_INIT ;
 if (removePending) {
 handlerRemoved(ctx);
 }
 }
 }
 // Subclasses override this method, and each implementation has its own special decoding method
 protected abstract void decode(ChannelHandlerContext ctx, ByteBuf in, List < Object > out) throws Exception;

From the above process, we can conclude that before decoding, the data needs to be written into the cumulation, and after decoding, it needs to be removed through the handler.

Specific decoding process

I just mentioned that the decode method is implemented in the subclass. Let's look at the implementation of the three decoding methods we mentioned one by one.

1. FixedLengthFrameDecoder

The source code is:

 @Override
 protected final void decode(ChannelHandlerContext ctx, ByteBuf in, List < Object > out) throws Exception {
 Object decode decoded = decode(ctx, in);
 if (decoded != null) {
 out.add(decoded);
 }
 }
 protected Object decode(
 @SuppressWarnings("UnusedParameters") ChannelHandlerContext ctx, ByteBuf in) throws Exception {
 // Whether the collected data is less than the fixed length, if it is less than, it means that it cannot be parsed
 if (in.readableBytes() <   frameLength ) {
 return null;
 } else {
 return in.readRetainedSlice(frameLength);
 }
 }

As simple as the name of this class, it decodes with a fixed length. Therefore, when setting up the decoder, you need to pass in the frameLength in the construction method.

2. DelimiterBasedFrameDecoder

The source code is:

 @Override
 protected final void decode(ChannelHandlerContext ctx, ByteBuf in, List < Object > out) throws Exception {
 Object decode decoded = decode(ctx, in);
 if (decoded != null) {
 out.add(decoded);
 }
 }
 protected Object decode(ChannelHandlerContext ctx, ByteBuf buffer) throws Exception {
 // Is the current delimiter a line break delimiter (\n or \r\n)?
 if (lineBasedDecoder != null) {
 return lineBasedDecoder.decode(ctx, buffer);
 }
 // Try all delimiters and choose the delimiter which yields the shortest frame.
 int minFrameLength = Integer .MAX_VALUE;
 ByteBuf minDelim = null ;
 // Other delimiters are used for splitting
 for (ByteBuf delim: delimiters) {
 int frameLength = indexOf (buffer, delim);
 if (frameLength > = 0 && frameLength <   minFrameLength ) {
 minFrameLength = frameLength ;
 minDelim = delim ;
 }
 }
 // The following code is omitted

As its name suggests, the delimiter is its core. It divides delimiters into two categories, only line break delimiters (n or rn) and others. Therefore, it is important to note that you can define multiple delimiters, and it supports them all.

3. LengthFieldBasedFrameDecoder

This class is quite complex, and it is easy to get confused if you look at the methods directly, so I plan to combine the explanation of the class and look at its private variables first.

 * BEFORE DECODE (16 bytes) AFTER DECODE (13 bytes)
 * +------+--------+------+----------------+ +------+----------------+
 * | HDR1 | Length | HDR2 | Actual Content |----- > | HDR2 | Actual Content |
 * | 0xCA | 0x0010 | 0xFE | "HELLO, WORLD" | | 0xFE | "HELLO, WORLD" |
 * +------+--------+------+----------------+ +------+----------------+

• lengthFieldOffset: This field indicates the byte from which the Length field starts. In the above example, the Length field starts from the first byte (HDR1 is the 0th byte), so the value is 0.
• lengthFieldLength: This field represents the number of bytes occupied by the Length field. In the above example, the Length field occupies 2 bytes, so the value is 2.
• lengthAdjustment: This field represents the distance from the end of the Length field to the actual start of the content. In the above example, because the Length field means the entire message (including HDR1, Length, HDR2, and Actual Content, and generally Length refers only to Actual Content), the distance from the end of Length to the actual start of the content (the beginning of HDR1) is equivalent to reducing 3 bytes, so it is -3.
• initialBytesToStrip: How many bytes should be skipped from the end of the Length field when displaying. In the above example, because the actual content starts from HDR1 and the final displayed content starts from HDR2, there is a difference of 3 bytes in the middle, so the value is 3.

The decoding method of this class is relatively complicated. Students who are interested can try to analyze it by themselves.

Summarize

This article mainly explains the three ways of framing in Netty in combination with the source code in Netty. I believe you must have a different understanding.