What does the request data packet go through from sending to receiving?

Previously, we talked about how the domain name in "from entering the URL to successfully viewing the interface in the browser" is converted into an IP address, and learned about things related to DNS. This article will talk about the operation that occurs after DNS resolution - establishing a connection. This is what we often call a three-way handshake.

When you see the three-way handshake, you might say, isn’t this a question that has been asked too many times in interviews?

Isn't the three-way handshake just:

1. The server starts in the CLOSE state and then listens to a port. At this time, the server enters the LISTEN state.
2. The client is also in the CLOSE state at first. The client will send a data packet with the SYN flag to the server to actively initiate a connection. At this time, the client will change to the SYN-SENT state.
3. After the server receives the data packet from the client, it determines through the flag that the client wants to establish a connection. Then it returns a SYN and ACK, and the server status changes to SYN-RCVD.
4. After the client receives the ACK from the server, it will return an ACK to the server. After returning this ACK, the server's status changes to ESTABLISH
5. After the server receives the ACK from the client, the server status changes to ESTABLISH

Isn't that the end? What else is there to talk about?

This article will not cover the various state changes mentioned above, nor what the flags in the packets are, but will focus more on the underlying things, that is, how the data packets sent back and forth are sent out.

In fact, it is not just the three-way handshake when establishing a connection. Many HTTP interfaces called in the browser will communicate with the server.

So how are these requests sent to the server?

Is this even a question? Isn't it just a matter of sending an HTTP request?

Of course, this answer may be what many people who don’t understand the Internet might say.

In fact, a more specific and accurate statement is that it is sent through the protocol stack and the network card.

Among them, the protocol stack is responsible for packaging the data. After packaging, the network card converts the data into electrical signals and sends them out through optical fiber.

Needless to say, the network card is the hardware used to communicate with other computers. The MAC (Medium Access Control) address we often talk about is actually the number of the network card, a unique number determined from the moment it is produced. The MAC address is 48 bits long, that is, 6 bytes, and is expressed in hexadecimal.

Once we know the IP address of the device we are communicating with, we can entrust the protocol stack in the operating system to package the data from the application into data packets and send them out. What exactly is a protocol stack? The protocol stack is actually the sum of a series of network protocols, such as:

• TCP
• UDP
• IP

Different applications may choose different protocols when transmitting data. For example, the browser we use uses the TCP protocol, while the DNS resolution mentioned earlier uses the UDP protocol.

So what does the data go through in the protocol stack? How does it become packets one by one?

Take sending an HTTP request to a server as an example. We know that an HTTP request contains:

• Request Line
• Request Header
• Request Body

HTTP is a protocol belonging to the application layer, and there are many other protocols in the application layer. Each protocol involves different data. How can the protocol stack be compatible with data between different protocols?

The answer is no compatibility. To the protocol stack, all data is just a bunch of binary sequences.

After the protocol stack receives this bunch of binary sequences, does it directly send them to the network card?

I asked, so obviously it's not...

In fact, the protocol stack does not send out data immediately after receiving it, but writes it into the buffer in the memory first. So why not send it directly?

It's actually very simple. Suppose you are at the starting station of a bus. Do you think the bus will leave immediately if someone shows up?

Obviously not, it will wait for a while, and then leave after more passengers get on the bus. But it can't wait too long, otherwise the passengers on the subsequent platforms will have to wait for a long time.

The reason why the protocol stack does not send out the message immediately is the same. There are two basic considerations behind this:

1. The length of the data
2. Waiting time

The data sent by the application layer program may have different lengths. Some may be sent byte by byte, while others may transmit all the data at once.

If data is sent out as soon as it is received, many small packets will be transmitted in the network, which will reduce the efficiency of network transmission.

Therefore, the protocol stack will wait for a while after receiving the data, and will execute the sending operation after the data reaches a certain amount.

However, the protocol stack cannot wait too long, right? If it waits too long, how will the users who are operating in front of the computer feel? This sending delay will make the user experience drop rapidly.

However, it is not easy to strike a balance between the two. If the data packet is too short, the network transmission efficiency will be reduced, and if the waiting time is too long, it will cause transmission delay. Therefore, the protocol stack simply gives the control to the application.

Applications can control which measures to take. For example, our commonly used browsers interact with users in real time, and users are very sensitive to the response speed of the entire page, so they generally send data directly, even if the data does not reach a "certain amount".

What exactly is this “certain amount”?

Indeed, the above only says a certain amount, a certain amount, so what is this amount?

To understand this we need to know two parameters, namely:

1. MTU (Maximum Transmission Unit)
2. MSS (Maximum Segment Size) maximum segment size

MTU actually represents the maximum length of the data packet on the above path, which is generally 1500 bytes. And we need to know that the data packet is composed of the following parts:

1. Various header information
2. Real data

Subtract the size of various header data from the MTU, and what remains is the MSS, which is the actual data.

After knowing the composition of data packets and the concepts of MTU and MSS, we can proceed to the next step. If the data sent at a time does not exceed the MSS, it can be sent directly.

What if it exceeds the MSS? For example, the data length of the request I sent when I posted this article may exceed the MSS.

Splitting of overlong packets

At this time, the data needs to be split according to the length of MSS, and the split data is put into different data packets. After the split, it can be sent to the target server.

TCP ensures that the communicating server can receive the data packet. Each byte is numbered during transmission. For example, if the data transmitted this time is 1 - 1000 bytes, then the ACK returned by the server will be 1001, which means that there is no packet loss.

All the packets sent are temporarily stored in the buffer. If an error occurs during transmission, retransmission can be used as compensation. This is why there is no compensation mechanism in the data link layer (such as network cards, routers, hubs, etc.). Once an error is detected, they will directly discard the packet. Then the transport layer can retransmit it.

What if the network is very congested and the server has not returned?

On the server side, when we interact with other third parties, do we set a timeout? If we don't set a timeout, do we have to wait here forever?

The same is true for TCP. When the client is waiting for the server to respond, there will be a time called ACK waiting time, which is actually the timeout time.

When the network is congested, you can actually think of it as a traffic jam. At this time, the ACK will be slow to return. If the return time is so long that the client thinks the server has not received it, it may resend it.

And it is possible that the ACK arrives just after the retransmission. Although the server can detect the duplication of packets by sequence number and will not cause errors, it is nonsense to add meaningless duplicate packets when the network is already under heavy load. This is obviously not acceptable.

How to avoid the above situation? The answer is simple. Just extend the ACK waiting time a little bit. This way, the above problems can be avoided to a certain extent. However, if you think about it, you should know that the longer the time is, the better. If it is longer, users will be overwhelmed.

In addition to network fluctuations affecting the ACK return time, the physical distance of communication is also an influencing factor. To put it bluntly, it is impossible to set a fixed time for this thing. So, in fact, this waiting time is dynamically adjusted. If the return is a little slow this time, I will extend the waiting time a little next time. If the speed of returning ACK is very good, then the waiting time will be reduced accordingly.

The above concept also has a familiar name, called timeout retransmission.

Let's imagine a more extreme situation. Suppose your communication network cable is cut, or even the computer room is on fire. At this time, no matter how many times you resend, it will be useless. Then TCP will continue to send requests in an infinite loop, right?

Of course, this situation was taken into consideration when TCP was designed. After several invalid retransmissions, it will forcibly interrupt communication and throw an error to the application.

The question is, after the client sends a data packet to the server, while waiting for the ACK, does it really just wait for the ACK and do nothing else?

Of course not, this is extremely wasteful of resources and reduces communication efficiency. After sending a data packet, it will directly send the next packet without waiting for the return of ACK. This is the sliding window.

But there is a problem with this. The application sends packets too frequently, causing the server to be unable to receive them.

As mentioned above, when an application sends a message, it will store the sent data in a buffer. The same is true for the receiver. After receiving the message, the receiver will store the data in a buffer, and then reassemble the received data in the buffer to restore it to the data originally sent by the application.

However, if the data is sent too fast and exceeds the speed of reassembly, the buffer will be filled. Once the buffer is filled, subsequent data can no longer be received, and packet loss occurs.

How does TCP solve this problem? The answer is flow control. In order to prevent the transmission party from sending too fast and causing packet loss, which in turn triggers the above timeout retransmission mechanism, the transmission speed of the sender is determined according to the receiving party's acceptance capacity. This mechanism is flow control.

This mechanism works on the receiver. In the TCP packet header, a 16-bit field is used to represent the window size, which is a very important tuning parameter. The larger this number is, the larger the receiver's buffer is, and the more data can be received. When confirming the response, the receiver will write its remaining window size and send it to the sender along with the ACK.

TCP Flow Control

If the size received by the sender is 0, it will stop sending data. This will cause a problem. If the next response (that is, the window size is not 0) is lost in the process, the sender will enter a deadlock and wait for each other. Therefore, the sender will periodically send window detection data segments to the receiver.

Well, that's all about sending data packets. I'll talk about TCP congestion control later if I have the chance.