HTTP connection management diagram

The HTTP protocol is very important to us programmers. No matter which language you use, HTTP is the key point you need to know.

This is not an article that simply introduces the basic concepts of HTTP. If you are not very familiar with the basic concepts of HTTP, I recommend you to read the article about HTTP basics written by cxuan - "After reading this article on HTTP, you will have no problem arguing with the interviewer."

So we assume that everyone here has some knowledge and understanding of HTTP.

Let’s start this article.

[[414997]]

TCP with HTTP

As we all know, HTTP, an application layer protocol, transmits data based on TCP. When you want to access a resource (a resource is a URL on the Internet), you need to first resolve the IP address and port number of the resource, and then establish a TCP connection with the server where the IP and port number are located. Then the HTTP client initiates a service request (GET) message, and the server responds to the server's request message. When there is no need to exchange messages, the client will close the connection. The following diagram illustrates this process very well.

The above picture well illustrates the whole process of HTTP from establishing connection -> initiating request message -> closing connection, but the above process also ignores a very important point, that is, the process of TCP establishing connection.

TCP needs to go through three handshakes and exchange three messages to establish a connection. I believe everyone is familiar with this process. If you are still unclear about the process of TCP establishing a connection, you can first read cxuan's article TCP connection management.

Since HTTP is located on the upper layer of TCP, the timeliness (performance) of the HTTP request -> response process depends largely on the performance of the underlying TCP. Only after understanding the performance of TCP connections can we better understand the performance of HTTP connections and thus implement high-performance HTTP applications.

We usually call a complete request -> response process an HTTP transaction.

So I will usually write it as HTTP transaction later, just make sure you understand what is going on.

Our next focus will start with the performance of TCP.

HTTP latency loss

Let's review the HTTP transaction process above. Which processes do you think will cause HTTP transaction delay? As shown in the following figure:

As can be seen from the figure, the following factors mainly affect the latency of HTTP transactions:

• The client will determine the server's IP and port number based on the URL. The main factor here is the delay in DNS converting the domain name into an IP address. DNS will initiate a DNS query to query the server's IP address.
• The second delay is the delay when TCP establishes a connection. The client sends a connection request message to the server and waits for the server to send back a response message. Each new TCP connection will have a delay.
• Once the connection is established, the client will request data from the server. This delay is mainly the delay for the server to read the request message from the TCP connection and process the request.
• The delay in the server transmitting the response message to the client.
• The last delay is the TCP connection closing delay.

The optimization of the last point is also a focus of this article.

HTTP Connection Management

Imagine a problem. Suppose a page has five resources (elements). Each resource requires the client to open a TCP connection, obtain the resource, and disconnect. Moreover, each connection is opened serially, as shown in the following figure:

Serial means that these five connections must be in sequence, and there will not be a situation where more than two connections are open at the same time.

The five resources above require five connections to be opened. It is not a big deal if the resources are few, as the CPU can handle it. But what if the page resources reach hundreds or more? Is it necessary to open a separate connection for each resource? This will obviously increase the processing pressure of the CPU dramatically and cause a lot of delays, which is obviously unnecessary.

Another disadvantage of serial is that some browsers cannot know the size of the object before the object is loaded, and the browser needs the object size information to place them in a reasonable position on the screen. Therefore, the screen will not display anything until enough objects are loaded. This will cause the object to be loaded all the time, but we think the browser is stuck.

So, is there a way to optimize HTTP performance? That's a good question, of course there is.

(1) Parallel connection

This is the most common and easiest to think of connection method. HTTP allows the client to open multiple connections and execute multiple HTTP transactions in parallel. After adding parallel connections, the request process of the entire HTTP transaction is as follows.

Using parallel connections will overcome the idle time and bandwidth limitations of a single connection. Because each transaction has a connection, the delays can overlap, which will increase the loading speed of the page.

However, parallel connections are not necessarily fast. If the bandwidth is insufficient, the page response speed may even be worse than that of serial connections. This is because in parallel connections, each connection will compete for the use of effective bandwidth, and each object will be loaded at a slower speed. It is possible that connection 1 has loaded 95%, connection 2 has occupied 80% of the bandwidth, connection 3, connection 4... Although each object is loading, there is no response on the page.

Moreover, opening a large number of connections will consume a lot of memory resources, thus causing performance problems. The five connections discussed above are relatively few. A complex web page may have dozens or even hundreds of embedded objects. In other words, the client can open hundreds of connections, and many clients send requests at the same time, which can easily become a performance bottleneck.

It seems that parallel connections are not necessarily "fast". In fact, parallel connections do not speed up the transmission of pages. Parallel connections only create an illusion, which is a common problem of all parallel connections.

(2) Persistent Connection

Web clients often open connections to the same site, and an application that initiates a request to a server is likely to make more requests to the server in the near future, such as to fetch more images. This property is called site locality.

Therefore, HTTP 1.1 and HTTP1.0 allow HTTP to keep the connection open after executing a transaction. This open state actually refers to the open state of TCP, so that the next HTTP transaction can reuse this connection.

A TCP connection that remains open after an HTTP transaction ends is called a persistent connection.

Non-persistent connections are closed after each transaction ends, whereas persistent connections remain open after each transaction ends. Persistent connections remain open between transactions until the client or server decides to close them.

Long connections also have disadvantages. If a single client does not initiate requests very frequently, but there are many connected clients, the server will crash sooner or later.

There are generally two options for persistent connections: HTTP 1.0 + keep-alive; HTTP 1.1 + persistent.

The default connections of versions before HTTP 1.1 are all non-persistent connections. If you want to use persistent connections on older versions of HTTP, you need to specify the Connection value as Keep-Alive.

HTTP 1.1 versions are all persistent connections. If you want to disconnect, you need to specify the Connection value as close. This is also the version factor of the two selection methods we mentioned above.

The following is a comparison of HTTP transactions using persistent connections and using serial HTTP transaction connections:

This figure compares the time loss of HTTP transactions on serial connections and persistent connections. It can be seen that HTTP persistent connections save the time of connection opening and connection closing, so the time loss is reduced.

Another interesting aspect of persistent connections is the Connection option. Connection is a common option, which is a header that both the client and the server have. The following is a request-response diagram of a client and server with a persistent connection:

As can be seen from this picture, the persistent connection mainly uses the Connection header, which means that Connection is the implementation method of persistent connection. So below we mainly discuss the Connection header.

Connection header

The Connection header has two purposes:

• Used together with Upgrade to perform protocol upgrades
• Managing persistent connections

(1) Used together with Upgrade to upgrade the protocol

HTTP provides a special mechanism that allows an established connection to be upgraded to a new protocol. The general syntax is as follows:

 GET /index.html HTTP/1.1
 Host: www.example.com
 Connection: upgrade
 Upgrade: example/1, foo/2

HTTP/2 explicitly prohibits the use of this mechanism, which belongs only to HTTP/1.1.

That is to say, the client initiating Connection:upgrade indicates that this is a request for connection upgrade. If the server decides to upgrade this connection, it will return a 101 Switching Protocols response status code and an Upgrade header field of the protocol to be switched to. If the server does not (or cannot) upgrade this connection, it will ignore the Upgrade header field sent by the client and return a regular response: for example, return 200.

(2) Managing persistent connections

We said above that there are two ways of persistent connection, one is HTTP 1.0 + Keep-Alive; the other is HTTP 1.1 + persistent.

 Connection: Keep-Alive
 Keep-Alive: timeout = 10 , max = 500  

In HTTP 1.0 + Keep-Alive mode, the client can include the Connection: Keep-Alive header request to keep a connection open.

One thing to note here: the Keep-Alive header only keeps the request active. After sending a Keep-Alive request, the client and server do not necessarily agree to a Keep-Alive session. They can close an idle Keep-Alive connection at any time, and the client and server can limit the number of transactions handled by a Keep-Alive connection.

The Keep-Alive header has the following options:

• timeout: This parameter estimates how long the server hopes to keep the connection active.
• max: This parameter follows the timeout parameter and indicates how many transactions the server can open persistent connections for.

Keep-Alive This header is optional, but can only be used if Connection: Keep-Alive is provided.

There are certain limitations on the use of Keep-Alive. Let's discuss the limitations of the use of Keep-Alive.

Keep-Alive Usage Limitations and Rules

• In HTTP/1.0, Keep-Alive is not used by default, and the client must send a Connection: Keep-Alive request header to activate the Keep-Alive connection.

• By detecting whether the response contains the Connection: Keep-Alive header field, the client can determine whether the server closed the connection after sending the response.

• Proxies and network managers must enforce the Connection header rules. They must delete the header fields and the Connection header itself before forwarding or caching the message, because Connection is a Hop-by-Hop header, which is only valid for a single forwarding and will become invalid if forwarded to a cache/proxy server.

• Strictly speaking, you should not establish a Keep-Alive connection with a proxy server that you are not sure supports the Connection header to prevent the dumb proxy problem, which we will discuss below.

Keep-Alive and the dumb proxy problem

Here I will first explain what a proxy server is, and then talk about the dumb proxy problem.

(1) What is a proxy server?

A proxy server is a medium that obtains network information on behalf of the client. In layman's terms, it is a transit station for network information.

(2) Why do we need a proxy server?

The most common use is that we need to use a proxy server to access some websites that our clients cannot directly access. In addition, the proxy server has many other functions, such as caching, which can reduce costs and save bandwidth; real-time monitoring and filtering of information. The proxy server is also a client relative to the target server (the server that ultimately obtains the information). It can obtain the information provided by the server. The proxy server is a server relative to the client, and it is up to it to decide what information to provide to the client, so as to achieve the monitoring and filtering functions.

The dumb proxy problem occurs on the proxy server. More specifically, it occurs on the proxy server that cannot recognize the Connection header and does not know that it will delete the Connection header after sending the request.

Suppose a web client is talking to a web server through a dumb proxy server, as shown in the following diagram:

Let me explain the picture above:

• First, the Web client sends a message to the proxy, which contains the Connection: Keep-Alive header, hoping to remain active after this HTTP transaction. The client then waits for a response to determine whether the other party allows a persistent connection.

• The dumb proxy (it is inappropriate to define it as a dumb proxy here. We often look at what it does first and then characterize it. Now this server has not yet behaved as a dumb proxy, but it has been characterized as such) received the HTTP request, but it does not understand the Connection header, nor does it know what Keep-Alive means, so it just sends the message to the server along the forwarding link. However, the Connection header is a Hop-by-Hop header, which is only applicable to single-link transmission, so the proxy server should not send it to the server again, but it still sends it, and some difficult things will happen later.

• When the forwarded HTTP request reaches the server, it will mistakenly think that the other party wants to maintain a Keep-Alive persistent connection. After evaluation, the server responds that it agrees to have a Keep-Alive conversation, so it sends back a Connection: Keep-Alive response and it reaches the dumb proxy server.

• The dumb proxy server will directly send the response to the client. After receiving the response, the client knows that the server can use a persistent connection. However, at this time, both the client and the server know that they need to use Keep-Alive persistent connections, but the dumb proxy server knows nothing about Keep-Alive.

• Since the proxy knows nothing about Keep-Alive, all the data it receives will be sent to the client, and then wait for the server to close the connection, but the proxy server thinks it should remain open, so it does not close the connection. In this way, the dumb proxy server keeps hanging there waiting for the connection to be closed.

• After the client sends the next HTTP transaction, the dumb proxy will directly ignore the new HTTP transaction because it does not think that there will be other requests on the same connection, so it will directly ignore the new request.

This is the Keep-Alive dumb proxy.

So how do we solve this problem? Use Proxy-Connection

Proxy-Connection solves dumb proxy

Netscape proposed a method of using the Proxy-Connection header. First, the browser will send the Proxy-Connection extension header to the proxy instead of the officially supported Connection header. If the proxy server is a dumb proxy, it will directly send the Proxy-Connection to the server, and the server will ignore this header when it receives the Proxy-Connection, which will not cause any problems. If it is a smart proxy server, when it receives the Proxy-Connection, it will directly replace the Proxy-Connection with the Connection header and send it to the server.

HTTP/1.1 Persistent Connections

HTTP/1.1 gradually stopped supporting Keep-Alive connections and replaced Keep-Alive with an improved design called persistent connection. This improved design is also a persistent connection, but it has a better working mechanism than HTTP/1.0.

Unlike the Keep-Alive connection of HTTP/1.0, HTTP/1.1 uses a persistent connection by default. Unless otherwise specified, HTTP/1.1 assumes that all connections are persistent. If you want to close the connection after the transaction ends, you need to explicitly add a Connection: close header to the message. This is a very important difference from previous HTTP protocol versions.

There are also some restrictions and rules for using persistent connections:

• First, after sending the Connection: close request, the client cannot send more requests on this connection. This also means that if the client does not want to send other requests, it can use Connection: close to close the connection.
• HTTP/1.1 proxies MUST be able to manage separate persistent connections between the client and server, with each persistent connection valid for a single transfer.
• It is best for the client to maintain only two persistent connections to any server or proxy to prevent overloading the server.
• The connection can only be maintained for a long time if the length of the entity part is consistent with the corresponding Content-Length, or if chunked transfer encoding is used.

Pipeline Connection

HTTP/1.1 allows the use of request pipelines on persistent connections. This is another performance optimization compared to Keep-Alive connections. A pipeline is a carrier that carries HTTP requests. We can put multiple HTTP requests into a pipeline, which can reduce the network round-trip time and improve performance. The following figure is a schematic diagram of using serial connection, parallel connection, and pipelined connection:

There are several limitations to using pipelined connections:

• If the HTTP client cannot verify that the connection is persistent, pipelining should not be used.
• HTTP responses must be sent back in the same order as the requests, because HTTP has no concept of sequence numbers, so if the responses are out of order, there is no way to match them with the requests.
• HTTP clients must be prepared that the connection may be closed at any time, and be prepared to resend any outstanding pipelined requests.

HTTP Close Connection

All HTTP clients, servers, or proxies can close an HTTP transport connection at any time. Usually the connection is closed after a response, but it may also happen in the middle of an HTTP transaction.

However, the server cannot be sure whether the client has any data to send at the moment of closing. If this happens, the client will have a write error during the data transmission process.

Even if there is no error, the connection can be closed at any time. If the connection is closed during the transaction transmission, you need to reopen the connection and try again. If it is a single connection, it is not a big deal, but if it is a pipelined connection, it is worse, because the pipelined connection will throw a large number of connections in the pipeline. If the server is shut down at this time, a large number of connections will not respond and need to be rescheduled.

If an HTTP transaction gets the same result whether it is executed once or n times, then we consider the transaction to be idempotent. Generally, GET, HEAD, PUT, DELETE, TRACE and OPTIONS methods are considered idempotent. The client should not send any non-idempotent request in a pipelined manner, such as POST, otherwise it will cause uncertain consequences.

Since HTTP uses TCP as the transport layer protocol, closing an HTTP connection is actually the same process as closing a TCP connection.

There are three types of HTTP connection closing: full closing, half closing, and normal closing.

The application can close either the TCP input or output channel, or both at the same time. Calling the socket close() method will close both the input and output channels at the same time, which is called a full close. You can also call the socket shutdown method to close the input or output channel separately, which is called a half-close. The HTTP specification recommends that when the client and server suddenly need to close the connection, they should close it normally, but it does not say how to do it.