What happens when you enter a URL in the browser (Part 2): TCP module encapsulation and transmission mechanism

This series of articles will start with an introduction to the basic knowledge of computer networks, focusing on "what happens after you enter a URL into a browser". The previous section briefly introduced http message encapsulation and dns request to obtain the target IP.

Continued from the previous article: What happens when you enter a URL in the browser (Part 1) Uncovering the entire process of DNS request and resolution

This section will introduce how HTTP messages are further processed in the protocol stack and sent to the network. The protocol stack here refers to the TCP/IP protocol stack.

For TCP transmission, the sending and receiving of requests need to be done through a connection; this is not the case for UDP transmission. This section will first introduce the TCP transmission process and then UDP transmission.

Before that, some related concepts need to be introduced.

1. Protocol Stack

A protocol stack is a software program composed of multiple protocol modules with layers. Each protocol module usually communicates with two other protocol modules above and below it. The lowest level protocol always describes the interaction with physical hardware, adding more features for each higher level.

The protocol is the specification and convention that computers need to follow when communicating on the network, and the protocol stack is the implementation of this specification and convention, that is, the specific code and function for the upper layer to call. Each layer of protocol module serves the protocol module of the upper layer. The protocol module of this layer only needs to call the method of the lower layer module to complete a certain operation without paying attention to its specific implementation.

2. Sockets and Connections

For TCP transmission, the sending and receiving of requests need to be done through a connection. We can think of a connection as an end-to-end pipeline. Both ends of this pipeline can be either the sender or the receiver, and data flows in both directions in this pipeline. The entrance and exit of this pipeline are called sockets, which are on the client and server sides respectively. Both parties need to create a socket before they can establish a connection. After the connection is established, requests and responses are sent through the socket and transmitted through this pipeline.

Of course, the pipeline is just a metaphor. There is no such pipeline in reality.

For UDP transmission, although there is no need to establish a connection, a socket must still be created and packets must be sent and received through the socket.

3. What is a socket

The essence of a socket is data in a memory space that stores communication control information such as IP addresses, port numbers, and communication operation status. The protocol stack uses this control information when performing operations, such as when encapsulating a message, it needs to know the IP and port of the communication partner, when the protocol stack needs to know how long it has been since the data was sent and has not been returned in order to resend it, when comparing the sequence number to see if it is correct, and when saving the window size so that it can be brought to the TCP header to notify the other party of the communication. All of this requires querying the socket's control information.

The protocol stack works according to the control information recorded in the socket.

Executing netstat in Windows can display the socket content. As shown in the following figure, each line is a socket:

• The first column: Protocol, usually TCP or UDP, indicates whether the socket is created for TCP or UDP transmission.
• The second column: local IP and port; if multiple network cards are installed, different IPs will be displayed; 0.0.0.0 means no IP address is bound.
• The third column: the remote end's IP and port; 0.0.0.0 means that communication has not started yet and the IP and port are not bound; in addition, the socket in the UDP protocol does not bind the other party's address and port, so *:* is displayed.
• Fourth column: Communication status; LISTENING: Waiting for the other party to connect; ESTABLISHED: The connection is complete and communication is in progress.
• Fifth column: Process ID of the program using the socket.

Back to the topic.

When the client program (browser) gets the target IP address, it will go through the following steps: create a socket, connect, send data, receive data, disconnect, and delete the socket. As shown in the figure below

1. Create a socket

The application (browser) calls the socket() method, and then the work is switched to the protocol stack of the operating system. The protocol stack will first allocate a memory space for storing the socket and write the initial state to it. The protocol stack will return a descriptor (an integer) representing the socket to the browser.

Later, when the browser needs to send or receive data (connect/read/write), it needs to provide this descriptor to the protocol stack. The protocol stack will find the corresponding socket according to this descriptor and perform operations.

2. Establish a connection

After creating the socket, the application (browser) calls the connect method, and the work is switched to the operating system's protocol stack, which is responsible for connecting the local socket to the server's socket.

The essence of connection is that the two communicating parties exchange control information and record it in the sockets of both parties. Each subsequent data transmission and reception must carry the control information in the socket. For example, the server's IP and port are recorded in the local socket, and the server is informed of the IP and port of the requesting party, so that the server records the client's IP and port in the server socket, in addition to other control information.

In addition, during the connection operation, the communicating parties will also allocate a memory space for temporarily storing the sent and received data.

4. TCP control information (TCP header)

The control information mentioned above will be written into the header of the TCP message and sent to the server, as shown below:

The above fields are stored in the socket, and when the header needs to be constructed, the information is taken from the socket. During the continuous communication between the two parties, the socket will be updated (such as window and sequence number).

5. Data packets transmitted by TCP

Whether connecting, sending or receiving data, or disconnecting, both parties in communication need to send data packets.

For connection and disconnection, there is no need to pass application (browser) data, so the data packet only contains control information (that is, header information). When sending and receiving data, the packet will contain control information and data block content. The data of the application may be very large, so the protocol stack will divide the data into data blocks, add header control information to each data block to make a packet and then send it.

As shown below:

Now we formally introduce the connection process:

• The application (browser) calls the socket library's connect(<socket descriptor>, <server IP and port>, ...)
• The work switches to the TCP/IP protocol stack. The protocol stack constructs a TCP header (TCP message) containing a lot of control information (ports, windows, sequence numbers, ACK numbers, etc. of both communicating parties), and sets the SYN bit in the header to 1, indicating that the message is to request the server to establish a connection.
• The TCP module passes the constructed TCP message (which only contains header information but no data content) to the IP module, which further encapsulates it into a packet (adds IP header and MAC header). The IP module then converts the packet into an electrical signal through the network card and transmits it to the server via the network cable. (How the packet finds the server based on the IP address and the transmission process of the packet in the network will be introduced later).
• After the packet reaches the server (reaching the server protocol stack through the network cable and network card), the corresponding socket of the server is found according to the port specified in the message. The server socket changes from the listening state to the connecting state, and the control information in the message is written into the socket for storage.
• After that, the server's protocol stack will also encapsulate the response message. This process is similar to the above A/B/C steps, and the SYN and ACK positions of the TCP header in the message are 1, which means that the server initiates a connection request to the sender (SYN=1) and receives the sender's packet (ACK=1).
• After the response packet reaches the sender, the protocol stack changes the socket connection status to connected and saves the server's IP and port to the socket. It then encapsulates a packet with ACK 1 to the server to indicate that it has received the connection request packet from the server.
• The connection operation is completed only after the server receives the packet with ACK 1 from the client.

At this point, the client and server sockets have saved each other's information, and the control flow has returned from the operating system to the application, after which the client's HTTP messages can actually be sent. The connection will remain until the close is executed.

The above process of exchanging three data packets between the two parties is also the familiar "three-way handshake" process.

http messages are encapsulated into network packets and sent

After the connection is completed, the application (browser) will call the write() method of the socket library to pass the data (encapsulated http message) to the protocol stack, which will send it to the server.

Before talking about the data sending and receiving process, it is necessary to introduce some details and key points of data sending and receiving:

6. Network Packet

Packet is the name of network data at the network layer (such as IP packet and Ethernet packet).

A packet consists of a header that records control information and the contents of the packet. The IP header and MAC header are added at the network layer, so they can both be called packets:

The TCP module is responsible for adding TCP header control information to the http message and passing this data to the next layer IP module, which encapsulates the IP header and MAC header to form a network packet. The TCP header records the port numbers of both communicating parties, and the IP header records the IP addresses of both communicating parties. Both the data in the connection stage and the data sent and received will be encapsulated and sent by the IP module.

IP header:

PS: IP addresses are not assigned to computers but to network cards. Therefore, if a computer contains multiple network cards, it can have multiple IP addresses. If the client and server hosts have multiple IP addresses, which IP address is filled in the IP header depends on which network card the protocol stack wants to send the data packet, and to which network card of the server the data packet is sent.

MAC header:

The next chapter will introduce the IP header and MAC header. Here we only need to know that after adding these two headers, the data becomes a network packet, which is then passed to the network card through the IP module. When it is passed to the network card, the network packet officially leaves the protocol stack.

Network data names of other layers:

• Frame: The unit of data at the data link layer
• Packet: The unit of data at the network layer
• Segment: The unit of data at the transport layer
• Message: The unit of data at the application layer

7. Data sending timing

The protocol stack does not necessarily send out data immediately after receiving it. It may also store the data in an internal send buffer and wait until the data accumulates to a certain level before sending it.

The reason for this is that the amount of data that an application passes to the protocol stack each time is uncertain. Some applications pass the data to the protocol stack word by word or line by line, while others pass all the data of the entire request to the protocol stack at once. If the application passes very little data to the protocol stack each time, and the protocol stack sends the data immediately after receiving it, a large number of small packets will be sent, resulting in reduced network efficiency.

How much data is passed to the protocol stack each time is determined by the application.

Whether the protocol stack sends data immediately after receiving it from the application is also determined by some options specified when the application makes a system call.

For example, for a session-based application such as a browser, it will specify that the protocol stack send data immediately to avoid delays.

If the application decides to let the protocol stack cache the data before sending it, the protocol stack will decide when to send it based on two factors: one is the maximum data length that the network packet can accommodate, and the other is the frequency of data transmission by the application (the time interval between each data transmission).

The maximum data length that a network packet can contain: The protocol stack stipulates that the maximum length that a network packet can contain is MTU (1500 bytes in Ethernet), that is, IP header + TCP header + real data <= 1500. Excluding the header, the maximum length of the real data is MSS = 1500 - 20 - 20 = 1460. When the protocol stack receives data close to or exceeding the length of MSS, it can avoid the problem of sending a large number of small packets.

The time interval for the application to transmit data:

When the application does not send messages frequently, if it has to wait until the length is close to the MSS each time, it will cause a large sending delay. There is a timer in the protocol stack. After a certain period of time, even if the buffer data does not reach the MSS, it will be encapsulated as a packet and sent out.

8. Big Data Splitting

Generally, HTTP messages in GET request mode are not very long and can be contained in one network packet (the packet here refers to the packet sent by the IP module). However, if you want to transmit a form or even upload a file, the data size (MSS) of a packet may exceed that of the data in the send buffer. In this case, the data size in the send buffer will be larger than the MSS. At this time, the TCP module of the protocol stack will split the data in the send buffer (that is, the http message) into multiple data blocks in units of MSS, and each block will be individually added with a TCP header and encapsulated as a network packet for transmission.

When splitting data, the TCP module will calculate the first byte of each block of data is the nth byte of the overall data, and use this as the sequence number in the TCP header of each block of data (the sequence number is the data sequence number of the application rather than the length sequence number of the network packet).

The receiver can get the length of each data block by subtracting the header length from the length of each network packet received. Then the sequence number of the next received packet can be determined by the current latest sequence number + data block length. The ACK number of the response packet returned by the receiver should be the current latest sequence number + data block length + 1, and the sequence number that the sender should send next time should also be the ACK number of the receiver's last response packet.

For example: the current sequence number is A, and the length of the packet is 1460. Then the response packet ACK number should be A+1460+1. The sequence number of the next packet that the receiver should receive should be A+1460+1 = A + 1461. If the sequence number of the next packet received by the receiver is greater than A+1461, it means that a packet is missing in the middle.

The sequence number tells the receiver how many bytes of data the sender has sent, and the ACK tells the sender how many bytes of data the receiver has received (so by the same token, the sender can use the ACK number to determine whether the receiver's reply packet has been lost).

Every time a network packet is sent by the sender, the receiver will receive a reply packet containing an ACK number. This mechanism is called a confirmation response mechanism.

In actual communication, the sequence number does not start from 1, but an initial value calculated using a random number. During the connection phase, when both parties set SYN to 1, the initial sequence number will also be set and sent to each other by both parties (similarly, the initial ACK number is also notified to the other party during the connection phase).

That is to say, there are two sequence numbers during communication, one is generated by the client, and the other is generated by the server, because TCP data transmission and reception are bidirectional (for example, the server will reply when receiving the client's packet, and the server will also actively send packets when returning response data [here does not refer to ACK packets, but the network packets returning http response messages], so the client is not only the sender, and the server is not only the receiver, but both the client and the server can be both the sender and the receiver). Similarly, there are two ACK numbers.

as follows:

The sequence number and ACK number appear to tell the communicating parties how much data was sent and how much data was received each time a packet was sent, so as to determine whether there was packet loss. Packet loss will trigger the TCP retransmission mechanism, which is a unique error compensation mechanism of the TCP module. Once the network card, router, or hub detects an error, it will directly discard the packet instead of retransmitting it.

9. ACK number waiting time of retransmission mechanism

After the sender sends a packet, it will wait for the receiver to return a reply packet with an ACK number (the sender will not do nothing during this waiting process. This involves the sliding window to be introduced later, so we will not mention it here). This waiting time is called the ACK number waiting time. If no reply packet is received within this period of time, the sender will think that the packet is lost in the network and will resend the packet.

If the sender loses the packet, or the receiver loses the reply packet after the sender's packet arrives at the receiver, or the reply packet from the receiver returns slowly due to network congestion, the receiver may wait for more than the ACK timeout period and cause retransmission.

How does the TCP module set a suitable ACK waiting time?

Due to the different distances between different servers, or the different network congestion conditions in different network environments (for example, the ACK number may be returned in a few milliseconds in a local area network, but it may take hundreds of milliseconds to return in the case of congestion on the Internet), the ACK waiting time is not a fixed time, but a dynamically changing time. The TCP module will continue to measure the return time of multiple ACK numbers. If the return of the first few ACK numbers is slow, the waiting time will be extended. If the first few ACK numbers can be returned immediately, the waiting time will be shortened.

Each retransmission due to exceeding the waiting time will extend the timeout period. If no confirmation packet is received after multiple retransmissions, the connection will be disconnected.

10. Fast retransmission of retransmission mechanism

Fast retransmission is a more efficient retransmission method than timeout retransmission. When the receiver receives out-of-order messages (such as 1-3-2, or 1-3-4), it will immediately return duplicate ACK packets to the sender without delay. After the sender receives the same ACK number three times, it will resend the lost packet. As shown below:

In the figure, the first packet from the sender arrives, and the receiver returns a reply packet with ACK number ACK2; but the sender's second message is lost, and then the 3rd to 5th packets arrive at the receiver. The receiver knows from the sequence number of the confirmation packet that the second packet has not been sent, so it returns a duplicate ACK number (ACK2) to the 3rd to 5th packets.

After receiving multiple repeated response packets with ACK number ACK2, the receiver knows that the second packet has not been sent to the receiver, so it will resend the second packet (the packet with sequence number ACK2) and resend the third to fifth packets (because the latest ACK number obtained by the sender is ACK2, it will resend all packets with sequence numbers after ACK2). After receiving the second to fifth packets, the receiver will reply with a packet with ACK number ACK6.

If the SACK option is used, the sender will only resend the second packet and not 3 to 5.

11. Brief description of sliding window

It would be a waste if the TCP module does nothing while sending a packet and waiting for a response. To reduce this waste, the TCP module uses a sliding window to send packets, which means that it does not wait for a packet to be responded before sending the next packet, but sends multiple packets and receives multiple response packets continuously. This is shown below:

When the receiver receives a packet, it will first store it in the receiving buffer. The TCP module will obtain the packet from the buffer and calculate the ACK number of the packet, assemble the data blocks of multiple packets, restore them to the original data, and pass them to the receiving application.

If packets arrive faster than they can be processed and delivered to the application, the buffer will accumulate and eventually overflow, and after the overflow, the receiver will be unable to accept subsequent packets.

As shown in the figure below, after the receiver receives the packet, it will immediately take the packet out of the receive buffer and process it. The space in the buffer will be released, and then the remaining space in the receive buffer (also the window size) will be noted in the TCP header of the return packet. In this way, after the sender receives the return packet, it will know that the amount of data sent next time should not exceed the size of this window.

As the receiver continues to send data, it automatically calculates how much of the window size it has used, and stops sending packets when the window size is calculated to be 0. During this process, if the receiver replies, the sender will update the window size in its socket based on the TCP header window size in the reply packet (the automatic calculation process is 4380->2920->1460, at which point the receiver returns an ACK packet and informs that the window size is 2920, the sender will update the current window size to 2920, and then continue to automatically calculate from 2920, 2920->1460->0, and then pause until the receiver's ACK packet arrives at the sender again, the sender updates the window size again, and starts sending packets again). This is the basic idea of ​​the sliding window.

This diagram is for the sake of explanation, and deliberately shows a situation where the receiver does not have time to process the received packets, resulting in the buffer being filled. In fact, the receiver will start processing immediately after receiving the data. If the receiver has high performance and the processing speed is faster than the packet arrival rate, the buffer will be emptied immediately and the sender will be notified through the window field.

Also, the figure only shows the operation of sending data from right to left. In fact, like the sequence number and ACK number, the sending operation is also carried out in both directions.

Another thing to note is that the receiver will not send a reply packet for every packet from the sender, because the receiver sends a reply packet to notify the sender of the ACK number and window size (that is, to tell the sender how many packets I have received and how many packets I can still receive from you). Therefore, after the receiver receives multiple packets from the sender continuously within a period of time, it may only send one reply packet that records the current latest ACK number (that is, the ACK number corresponding to the last incoming packet) and the latest window size. The ACK number in the middle will be omitted, which reduces the number of packets and improves the efficiency of network communication.

When the server receives all the packets of a HTTP message from the client and restores it to an http message, the server application processes the http request, encapsulates the response data into an http response message, and then encapsulates it into packets through the protocol stack and returns it to the client. At this time, the server becomes the sender and the client becomes the receiver.

1. Receive http response message

After the protocol stack sends the http message in multiple packets, the workflow returns to the application (browser), which calls read() to receive the response message. The workflow is transferred to the protocol stack again.

After the response message arrives at the client host, it will be temporarily stored in the receiving buffer. The protocol stack will try to get data from the buffer. If there is no data in the buffer, the protocol stack will suspend the receiving work and do other things. The data will not be passed to the application until the message arrives, that is, the data will be copied to the memory address specified by the application (if the amount of data is large, it will not be passed to the application at once, but will be passed multiple times).

In this step, many details are skipped, such as the protocol stack will check the received data block and TCP header to determine whether there is data loss; update the window size to the server; splice multiple blocks of data into the original data according to the sequence number, etc. These processes are similar to the process when the client sends the sender.

2. The server disconnects and deletes the socket

The party that has sent the data will actively initiate the disconnection operation. In Web communication, the server is the last party to send data and will actively close the connection.

• The server application will call the close function of the Socket library, and the workflow will switch to the protocol stack. The protocol stack will generate a TCP header with the FIN bit set to 1 and entrust the IP module to send it to the client. At this time, the server will enter the close waiting state and record it in the socket.
• After the client receives the packet from the server to close the connection, it will enter the close waiting state and record it in the socket. It will also send an ACK packet to the server to indicate that it has received the request to close the connection.
• The client will not close the connection immediately, but wait for the application to receive all the data in the receive buffer (wait for the read() process to end).
• When the data in the receiving buffer in the operating system is received, the workflow returns to the application. The application calls the close function of the Socket library, and the workflow goes to the protocol stack. The client's protocol stack also generates a TCP packet with the FIN bit set to 1 and sends it to the server through the IP module, telling the server that it can close the connection.
• After receiving a TCP packet with the FIN bit set to 1, the server will return an ACK packet. After receiving the ACK packet, the client will close the connection (closing the connection actually means deleting the socket), but the server will not close the connection immediately (that is, it will not delete the socket immediately), but will enter the time-wait waiting state (marking the socket state as time-wait). After the time-wait state ends, the server will actually close the connection and delete the socket.

The above process is the well-known "four waves" process.

The server does not close the connection immediately but enters the time-wait state to prevent misoperation. For example, when the client is the one that actively initiates the disconnection (assuming that the client's socket is bound to port 54305), the client will be the last to send the ACK packet. If the ACK packet is lost, the server will wait for a period of time and will not receive the ACK packet and will resend the FIN packet to the client.

If the client does not have a time-wait state but directly closes the connection after sending the ACK packet (that is, deletes the socket and releases port 54305), then the client may generate a new socket for other connections and bind it to port 54305. After the FIN packet resent by the server reaches the client, the client finds the new socket based on the receiver port in the packet header, which will cause the connection on the new socket to be closed.

The time-wait state usually lasts for several minutes.

Appendix: Using UDP protocol to send and receive packets

Compared with TCP, UDP does not require a connection to be established when sending packets (the two communicating parties exchange control messages), so data transmission and reception reduces overhead and transmission delays and is more efficient than TCP. However, UDP does not have the secure transmission mechanisms of TCP, such as confirmation and response mechanisms, retransmissions, and windows.

Control information of UDP header:

UDP is suitable for the following scenarios:

a. Short data

If all data can be sent to the other end with just one packet, there is no need to establish a connection to save the IP and port of both parties and other information in the socket. Moreover, less data means fewer packets are transmitted, which reduces the possibility of packet loss and does not need to monitor the delivery status of packets like TCP. If a packet is lost, the protocol stack will not resend the data packet automatically if it does not receive a reply from the other party. The application can organize the resending of data by itself (the developer of the application needs to write the retry logic).

b. Video and audio data

Audio and video data must be delivered within the specified time. If they are delivered late, they will miss the playback opportunity, resulting in audio and video freezes. If errors are checked and resent by receiving confirmation responses like TCP, the resending process will take a certain amount of time, so the resent data may have missed the playback opportunity.

In addition, the lack of some packets in audio and video data will not cause serious problems, but will cause some distortion or freeze, which is generally acceptable. Using UDP to send data will be more efficient.

The above operations are all about the transmission and reception of data in the transport layer TCP module. In fact, TCP messages also need to be encapsulated into packets by adding IP headers and MAC headers in the IP module of the network layer and then entrusted to the network card for transmission.

Next section preview: What happens when you enter a URL in the browser (Part 3) IP module encapsulation, ARP protocol, IP protocol and network card