HTTP 2.0 is a bit explosive!

[[420793]]

Hey guys, hello everyone, this is programmer cxuan, welcome to read my latest article.

In this article, we will talk about HTTP 2.0, what changes HTTP 2.0 has made based on HTTP 1.1, and what features HTTP 2.0 has. So without further ado, let's start this article.

The original HTTP

When HTTP was first introduced, it was only used to obtain content on the web, generally for page access. At that time, the page content was not as rich as it is now, there were not many interactive scenarios, and there was no huge and complicated CSS and JS, so the page loading speed was very fast. However, with the emergence of web 2.0 and more content being displayed, more exquisite layouts, and more user interaction scenarios, the page content has become larger and larger, making the page loading speed slower and slower.

HTTP bottleneck

There are two main factors that affect an HTTP network request: bandwidth and latency.

First, let's talk about bandwidth. If we are still at the dial-up stage, bandwidth bottlenecks are likely to occur because the amount of data transmitted per unit time is very small. But now with the continuous development of communication technologies such as optical fiber, 10Mbps, 100Mbps, and even 1000Mbps have entered every household, and we no longer have to worry about bandwidth becoming a bottleneck for network requests.

Then all that's left is delay.

There are three main types of delay:

• Browser blocking (HOL blocking): The browser will block requests for some reasons. The browser can only have 4 connections to the same domain name at the same time (this may vary depending on the browser kernel). If the maximum number of connections exceeds the browser's limit, subsequent requests will be blocked.
• DNS Lookup: The browser needs to know the IP address of the target server to establish a connection. The system that resolves domain names to IP addresses is DNS. This can usually be achieved by using DNS cache results to reduce this time.
• Initial connection: HTTP is based on the TCP protocol. The browser can only piggyback the HTTP request message during the third handshake at the earliest to establish a real connection. However, these connections cannot be reused, which causes each request to go through three handshakes and slow start. The three-way handshake has a more obvious impact in high-latency scenarios, and the slow start has a greater impact on large file requests.

One of the most complained about HTTP 1.0 is that connections cannot be reused. Every time there is a new request, the three-way handshake and four-wave handshake process will be repeated, and the establishment and release of the connection requires a lot of server resources. It can still cope with pages with fewer requests, but as the number of requests continues to increase, HTTP 1.0 is becoming more and more difficult to handle.

However, HTTP 1.0 protocol headers can set Connection:Keep-Alive. Setting Keep-Alive in the header allows connections to be reused for a certain period of time. The specific length of reuse time can be controlled by the server, usually around 15 seconds. After HTTP 1.1, the default value of Connection is Keep-Alive. If you want to turn off connection reuse, you need to explicitly set Connection:Close.

Another problem that HTTP is complained about the most is its head of blocking. The head of blocking problem will cause the bandwidth to not be fully utilized, resulting in subsequent requests being blocked.

If five requests are sent at the same time, if the first request is not processed, the subsequent requests will not be processed, as shown in the following figure

If the first request is not processed, then the four requests 2, 3, 4, and 5 will be directly blocked on the client side, and can only be sent one by one after request 1 is processed. When the network is unobstructed, the performance is not greatly affected. However, once request 1 does not reach the server for some reason, or the request is not returned in time due to network congestion, all subsequent requests will be affected, causing subsequent requests to be blocked indefinitely, and the problem becomes more serious.

However, in HTTP 1.1, a pipelining design was proposed. Pipelining is used to solve the head-of-line blocking problem, as shown in the following figure.

Although this pipeline design seems to be able to solve the blocking problem at first glance, because the three requests in the right figure are not sent after the response arrives, but are sent directly one by one, but in fact, it is not the case.

Pipelining is not a panacea, it also has many drawbacks:

Because only idempotent requests such as GET and HEAD can use pipelining, non-idempotent requests such as POST cannot use it because there may be a sequence dependency between requests.

In fact, the head-of-line blocking problem is not completely solved, because the responses returned by the server still have to be returned in sequence, that is, the returned requests are FIFO - first sent, first returned.

Most HTTP proxy servers do not support pipelining.

There are problems negotiating with older servers that do not support pipelining.

Precisely because of so many problems, major browser manufacturers either do not support pipelining at all, or turn off the pipelining mechanism by default, and the conditions for enabling it are very harsh.

SPDY

Although HTTP1.0 and HTTP 1.1 have so many problems, the industry has come up with various optimization methods, but these methods are only temporary solutions and not fundamental solutions. It was not until 2020 that Google proposed the SPDY solution that people began to look at and solve the problems of the old version of the HTTP protocol itself from a positive perspective, which directly accelerated the birth of HTTP 2.0.

Let's first talk about what SPDY is and what are its features?

Understanding SPDY

The goal of SPDY is to solve the defects of HTTP, namely latency and security. We have been discussing latency above. As for security, although we did not talk about it in detail above, HTTP plain text transmission is indeed a problem. If the goal is to reduce latency, both HTTP at the application layer and TCP at the transport layer have room for adjustment. However, TCP, as a lower-level protocol, has existed for decades and is now deeply rooted in the global network infrastructure. If it is to be changed, it will inevitably cause damage to the bones and the industry's response will inevitably be low. Therefore, SPDY's scalpel is aimed at HTTP.

Reduce latency. The client's single connection and single request, and the server's FIFO response queue are the main causes of latency.

HTTP was originally designed so that the client initiates the request and the server responds. The server cannot actively send content to the client.

Compress HTTP headers. HTTP 1.x headers are getting bigger and bigger. Cookies and user agents can easily increase the size of headers to 1kb or more. And due to the stateless nature of HTTP, headers must be repeated in each request, which wastes traffic.

In order to increase the feasibility of solving these problems, Google, a smart company, avoided starting from the transport layer and intended to use the power of the open source community to increase the spread of the protocol. For protocol users, they only need to set the user agent in the request header and then provide support on the server side, which greatly reduces the difficulty of deployment. The design of SPDY is as follows

As you can see, SPDY is located below HTTP and above SSL, which makes it easily compatible with older versions of the HTTP protocol. SPDY's functions are divided into basic functions and advanced functions. Basic functions are enabled by default, and advanced functions need to be enabled manually.

SPDY Basics

Multiplexing: Multiplexing reduces the overhead of establishing and releasing TCP connections and improves bandwidth utilization by allowing multiple requests to share one connection.

Request prioritization. One problem with multiplexing is that some critical requests will be blocked based on the shared connection. SPDY allows setting a priority for each request so that important requests will be responded to first.

Header compression. As mentioned above, HTTP 1.x headers are often repeated and redundant. Choosing a suitable compression algorithm can reduce the size and number of packets. SPDY can achieve a compression rate of more than 80% for headers.

SPDY Advanced Features

Server push, HTTP can only be sent by the client, the server can only passively send the response. However, after the server push is enabled, the server will inform the server that new content will be pushed through the X-Associated-Content header.

The difference between server hint and server push is that server hint will not push content, but only inform the client that new content is generated. The client still needs to actively initiate a request to download the content. Server hint is notified through X-Subresources header. The general application scenario is that the client needs to query the server status first and then download the resource, which can save one query request.

After the emergence of automatic SPDY, page loading time was reduced by 64% compared to HTTP, and major browser manufacturers also supported SPDY in more than a year after the birth of SPDY. However, the survival time of SPDY is not as long as people imagine. SPDY was born in 2012 and stopped maintenance in 2016. If HTTP 2.0 had not been born, I believe Google might have received more real feedback and data, but SPDY also completed its mission during this period.

A First Look at HTTP 2.0

HTTP 2.0, also written as HTTP/2, is the 2.0 version of the Hypertext Transfer Protocol. Due to the popularity of SPDY, IETF saw the effect of optimization and that HTTP can be optimized by modifying the protocol layer. Therefore, IETF decided to formally consider the plan of HTTP 2.0. In addition, some designers of SPDY were also invited to participate in the design of HTTP 2.0.

HTTP2.0 was designed with different purposes and starting points from SPDY. SPDY is more like a product of Google itself, equivalent to a toy of its own, you can play with it however you want, while HTTP 2.0 was designed for the purpose of universality. Therefore, any design at the beginning will be related to future maintenance issues. If there are any flaws or deficiencies, the impact may be huge, so the issues should be considered very rigorously and carefully.

HTTP 2.0 had some important premises when it was first designed:

This basic model of a client sending a request to a server does not change.

The original protocol headers will not be changed, services and applications using http:// and https:// will not be modified, and there will be no http2://.

Clients and servers using HTTP 1.x can be smoothly upgraded to HTTP 2.0.

Proxy servers that do not understand HTTP 2.0 can downgrade requests to HTTP 1.x.

Before the client determines with the server whether to use HTTP1.x or HTTP 2.0, it needs to determine whether the other party supports HTTP 2.0. Therefore, negotiation must be performed first, that is, the client asks the server, which adds an RTT delay. Our modification of HTTP 1.x is to reduce latency, and now there is an additional RTT, which is obviously unacceptable. Google also encountered this problem when formulating the SPDY protocol. The approach they adopted was to force the negotiation to be completed at the SSL layer, and they also developed a TLS extension called NPN (Next Protocol Negotiation). Although HTTP 2.0 also adopts the same approach, after discussion, HTTP 2.0 finally did not force the SSL layer. HTTP 2.0 did not use NPN, but developed a TLS extension called ALPN (Application Layer Protocol Negotiation). Now, SPDY also plans to migrate to ALPN.

Major changes in HTTP 2.0

HTTP 2.0 has undergone many changes since its design and birth, but for developers and manufacturers, the following points have a greater impact:

Binary format

HTTP 1.x was born using a plain text protocol, and its format mainly consists of three parts: request line, request header, and message body. To identify these three parts, protocol parsing must be done, and protocol parsing is text-based. Text-based parsing has the defects of diversity, and the binary format can only recognize 0 and 1, which is relatively fixed. Based on this consideration, HTTP 2.0 decided to adopt a binary format, which is easy to implement and robust.

The following picture well explains the different message formats used by HTTP1.x and HTTP 2.0.

In HTTP 2.0 messages, length defines the start and end of the entire frame, type defines the type of frame, there are ten types, flags defines some important parameters, stream id is used for flow control, and the remaining payload is the body of the request.

Although the HTTP 2.0 message format looks completely different from HTTP 1.x, in fact, HTTP 2.0 does not change the semantics of HTTP 1.x. It just encapsulates a layer based on HTTP 1.x, as shown in the following figure

As can be seen from the figure above, the request line and request header in HTTP 1.x are encapsulated into HEADERS Frame by HTTP 2.0, and the message body in HTTP 1.x is encapsulated into Data Frame by HTTP 2.0. During debugging, the browser will automatically restore the HTTP 2.0 frame to the HTTP 1.x format.

Connection Sharing

As we mentioned above, HTTP 1.x does not really solve the problem of connection multiplexing, so one of the major problems that HTTP 2.0 needs to solve is connection sharing (MultiPlexing). Connection sharing means that only one connection is needed between the client and the server, so that even data packets from many streams can be mixed together and transmitted through the same connection, and then reconnect different data streams according to the stream id identifiers in different frame headers to assemble them.

What is a stream?

A stream is a virtual channel in a connection that can carry bidirectional message transmission. Each stream has a unique integer identifier. To prevent stream ID conflicts at both ends, streams initiated by the client have odd IDs and streams initiated by the server have even IDs.

We mentioned above that HTTP 1.x does not really solve the problem of connection sharing. Another major factor is that it is impossible to set priorities for different requests, which will cause key requests to be blocked. However, HTTP 2.0 allows you to set different priorities for different streams, and you can also set dependencies between streams. Dependencies and priorities can be adjusted dynamically, which will solve the problem of key requests being blocked.

Header Compression

We also talked about headers in HTTP1.x. Since cookies and user agents do not have memory, requests must be resent with these headers each time. HTTP 2.0 uses encoders to reduce the size of transmitted headers. Both communicating parties will cache a header field table, which can avoid repeated header transmission and reduce the size of the transmission. HTTP 2.0 uses the HPACK compression algorithm.

The main idea of ​​this compression algorithm can be found in the official document https://httpwg.org/specs/rfc7541.html

Server Push

Server Push We have already discussed above that HTTP 2.0 can push the client's content in advance. Because there is no request or connection establishment, the speed of static resources can be greatly improved through server push. Server push has another greater advantage: caching. Cache can also share cache resources between different pages.

The following points should be noted:

1. Push follows the same-origin policy;

2. This server-side push is determined based on the client's request response.

When the server needs to actively push a resource, it will send a frame with a Frame Type of PUSH_PROMISE, which contains the stream id that needs to be created by PUSH. This means that the client is told: I will use this id to send something to you next, and the client is ready to continue. When the client parses the frame and finds that it is a PUSH_PROMISE type, it will prepare to receive the stream that the server wants to push.

HTTP 2.0 flaws

The most amazing thing about HTTP 2.0 is multiplexing. Although multiplexing has many advantages, you can think about it. Although multiplexing is good, it is based on TCP connections. In the case of frequent connections, will it cause pressure on TCP connections? From this perspective, TCP can easily become a performance bottleneck.

Another point is that using HTTP 2.0 will add a TLS handshake process and increase RTT, which we also mentioned above.

In HTTP 2.0, multiple requests are in the same TCP pipeline. Therefore, when HTTP 2.0 packet loss occurs, the entire TCP connection must wait for retransmission, which will block all requests in the TCP connection.

Summarize

In this article, we mainly talked about the protocol changes from HTTP 1.x to SPDY and then to HTTP 2.0, as well as the pain points and disadvantages of HTTP 1.0 and 1.1, the background and discovery of SPDY, the main features of HTTP 2.0, the changes in HTTP 2.0 compared to HTTP 1.x, and its shortcomings.

This article is reprinted from the WeChat public account "Programmer cxuan", which can be followed through the following QR code. To reprint this article, please contact the programmer cxuan public account.