A few pictures, take down HTTPS

[[374308]]

This article is reprinted from the WeChat public account "Xiaolincoding", the author is Xiaolincoding. Please contact Xiaolincoding public account to reprint this article.

I wrote an article about HTTP and HTTPS a long time ago, but the introduction to HTTPS was not detailed enough and only covered the basics. So this time we will take a deeper look at HTTPS and use actual packet capture to take you to take a peek at HTTPS again.

For those who don’t know about symmetric encryption and asymmetric encryption, you can first review my previous article "Hardcore! 30 pictures to explain common HTTP interview questions". This article assumes that everyone already has this knowledge.

TLS handshake process

HTTP is transmitted in plain text. The so-called plain text means that the information communicated between the client and the server is visible to the naked eye, and the content of the communication can be intercepted by any packet capture tool.

Therefore, there are three security risks:

• Risk of eavesdropping, for example, the content of the communication can be obtained on the communication link, and the user number can be easily lost.
• Risks of tampering, such as forced insertion of spam ads, visual pollution, and blinding of users’ eyes.
• There are risks of impersonation, such as impersonating Taobao website, which can easily cause users to lose their money.

HTTPS adds the TLS protocol between the HTTP and TCP layers to address the above risks.

How does the TLS protocol address the risks of HTTP?

• Information encryption: HTTP interactive information is encrypted, so third parties cannot steal it;
• Verification mechanism: Verify whether the information has been tampered with by a third party during transmission. If it has been tampered with, a warning will be issued;
• Identity certificate: proves that Taobao is the real Taobao;

It can be seen that with the TLS protocol, HTTP communication can be guaranteed to be secure. Therefore, before HTTP communication, a TLS handshake is required. The TLS handshake process is as follows:

The figure above briefly summarizes the TLS handshake process, where each "box" is a record. Records are the basic unit of TLS data transmission and reception, similar to segments in TCP. Multiple records can be combined into a TCP packet for transmission, so the TLS handshake can usually be completed after "four messages", which means a delay of 2 RTT is required. Then, HTTP messages can be sent in a secure communication environment to implement the HTTPS protocol.

Therefore, we can find that HTTPS is an application layer protocol. It is necessary to complete the TCP connection establishment first, and then go through the TLS handshake process before a secure communication connection can be established.

In fact, the TLS handshake process may be slightly different for different key exchange algorithms.

Here we will briefly introduce the key exchange algorithm. Due to performance considerations, both parties use symmetric encryption keys when encrypting application information. Symmetric encryption keys cannot be leaked. In order to ensure the security of symmetric encryption keys, asymmetric encryption is used to protect the negotiation of symmetric encryption keys. This task is the responsibility of the key exchange algorithm.

Next, we will take the simplest RSA key exchange algorithm to look at its TLS handshake process.

RSA handshake process

Traditional TLS handshakes basically use the RSA algorithm to implement key exchange. When the TLS certificate is deployed on the server, the certificate file contains a pair of public and private keys. The public key will be passed to the client during the TLS handshake phase, and the private key will remain on the server. Make sure that the private key cannot be stolen.

In the RSA key negotiation algorithm, the client generates a random key and encrypts it with the server's public key before sending it to the server. According to the asymmetric encryption algorithm, messages encrypted with the public key can only be decrypted with the private key. After the server decrypts, both parties get the same key and use it to encrypt application messages.

I used Wireshark to capture the TLS handshake process with RSA key exchange. You can see below that there are four handshakes in total:

I also drew a picture corresponding to the Wireshark packet capture. You can see the process clearly from the picture below:

Then, we will further introduce each TLS handshake.

TLS first handshake

The client will first send a "Client Hello" message. We can understand the literal meaning, which is to "say hello" to the server.

The message contains the TLS version number used by the client, a list of supported cipher suites, and a generated random number (Client Random). This random number will be retained by the server and is one of the materials for generating symmetric encryption keys.

TLS second handshake

When the server receives the "Client Hello" message from the client, it confirms whether the TLS version number is supported, selects a cipher suite from the cipher suite list, and generates a random number (Server Random).

Next, a "Server Hello" message is returned, which contains the TLS version number confirmed by the server, a random number (Server Random), and then a suitable cipher suite is selected from the client's cipher suite list.

As you can see, the cipher suite selected by the server is "Cipher Suite: TLS_RSA_WITH_AES_128_GCM_SHA256".

This cipher suite looks really dizzying, it's such a long list, but it actually has a fixed format and specification. The basic form is "key exchange algorithm + signature algorithm + symmetric encryption algorithm + digest algorithm". Generally, there are two words before the word WITH. The first word is the algorithm for key exchange, and the second word is the algorithm for certificate verification. For example, the cipher suite just now means:

• Since there is only one RSA in the WITH word, it means that both the key exchange algorithm and the signature algorithm during the handshake use RSA;
• The communication after the handshake uses the AES symmetric algorithm, the key length is 128 bits, and the packet mode is GCM;
• The digest algorithm SHA384 is used for message authentication and random number generation;

In the process of "greeting" each other between the client and the server, the client and the server have confirmed the TLS version and the cipher suite used, and you may find that the client and the server will each generate a random number and pass the random number to each other.

So what is the use of this random number? In fact, these two random numbers are used as conditions for generating "session keys" later. The so-called session key is the symmetric encryption key used during data transmission.

Then, in order to prove its identity, the server will send a "Server Certificate" to the client, which contains a digital certificate.

Afterwards, the server sends a "Server Hello Done" message to tell the client that I have given you everything I should give you and this hello is over.

Client Authentication Certificate

Let me stop here. After the client obtains the digital certificate of the server, how can it verify that the digital certificate is authentic and valid?

Digital Certificates and CAs

Before we talk about the process of verifying whether a digital certificate is credible, let's first look at what a digital certificate is. A digital certificate usually contains:

• Public key;
• Owner information;
• Certificate Authority (CA) information;
• The CA's digital signature on this document and the algorithm used;
• Certificate validity period;
• There is some other additional information;

The role of a digital certificate is to authenticate the identity of the public key holder to prevent third parties from impersonating. To put it simply, a certificate is used to tell the client whether the server is legitimate, because only if the certificate is legitimate can the server's identity be trusted.

We use certificates to authenticate the identity of the public key holder (the identity of the server), so where does the certificate come from? And how do we authenticate the certificate?

In order to make the server's public key trusted by everyone, the server's certificates are all signed by CA (Certificate Authority). CA is the public security bureau and notary center in the cyber world, and has a very high credibility. Therefore, it signs each public key. A certificate issued by a trusted party must be trusted.

The reason for signing is that it can prevent middlemen from tampering with the contents of the certificate when obtaining it.

Digital certificate issuance and verification process

The following figure shows the digital certificate issuance and verification process:

The process of CA issuing a certificate is shown on the left side of the above figure:

• First, the CA will package the holder's public key, purpose, issuer, validity period and other information into a package, and then perform hash calculation on this information to obtain a hash value;
• The CA will then use its own private key to encrypt the Hash value and generate a Certificate Signature, which means that the CA has signed the certificate;
• Finally, add the Certificate Signature to the file certificate to form a digital certificate;

The process of the client verifying the server's digital certificate is shown on the right side of the above figure:

First, the client will use the same hash algorithm to obtain the hash value H1 of the certificate;

Usually, the CA's public key information is integrated into the browser and operating system. After receiving the certificate, the browser can use the CA's public key to decrypt the Certificate Signature content and obtain a hash value H2;

Finally, H1 and H2 are compared. If the values ​​are the same, the certificate is trustworthy; otherwise, the certificate is considered untrustworthy.

Certificate Chain

But in fact, there is still a problem of certificate trust chain in the certificate verification process, because the certificate we apply to the CA is generally not issued by the root certificate, but by the intermediate certificate, such as Baidu's certificate. As you can see from the figure below, there are three levels of certificates:

The verification process for this three-level certificate relationship is as follows:

• After receiving the certificate from baidu.com, the client found that the issuer of the certificate was not a root certificate, so it could not verify whether the baidu.com certificate was credible based on the public key in the local root certificate. Therefore, the client found that the issuing authority of the certificate was "GlobalSign Organization Validation CA - SHA256 - G2" based on the issuer in the baidu.com certificate, and then requested the intermediate certificate from the CA.
• After requesting the certificate, I found that the "GlobalSign Organization Validation CA - SHA256 - G2" certificate was issued by the "GlobalSign Root CA". Since the "GlobalSign Root CA" has no superior issuing agency, it is a root certificate, that is, a self-signed certificate. The application software will check whether this certificate has been pre-loaded on the root certificate list. If so, the public key in the root certificate can be used to verify the "GlobalSign Organization Validation CA - SHA256 - G2" certificate. If the verification is found to be successful, the intermediate certificate is considered to be credible.
• After the “GlobalSign Organization Validation CA - SHA256 - G2” certificate is trusted, you can use the public key in the “GlobalSign Organization Validation CA - SHA256 - G2” certificate to verify the credibility of the baidu.com certificate. If the verification passes, you can trust the baidu.com certificate.

In these four steps, at first the client only trusts the root certificate GlobalSign Root CA certificate, then the "GlobalSign Root CA" certificate trusts the "GlobalSign Organization Validation CA - SHA256 - G2" certificate, and the "GlobalSign Organization Validation CA - SHA256 - G2" certificate trusts the baidu.com certificate, so the client also trusts the baidu.com certificate.

In summary, since users trust GlobalSign, baidu.com guaranteed by GlobalSign can be trusted. In addition, since users trust the software vendors of the operating system or browser, GlobalSign root certificates pre-loaded by the software vendors can be trusted.

The operating system usually has some root certificates built in. For example, my MAC computer has so many root certificates built in:

Such layer-by-layer verification constitutes a trust chain. The entire certificate trust chain verification process is shown in the following figure:

The last question is, why is such a complicated process of certificate chain needed? Why doesn't the Root CA issue the certificate directly, but has to have so many intermediate levels?

This is to ensure the absolute security of the root certificate. The more strictly the root certificate is isolated, the better. Otherwise, if the root certificate is lost, the entire trust chain will be in trouble.

TLS third handshake

After the client verifies the certificate, it will continue to move forward if it believes it is credible. Then, the client will generate a new random number (pre-master), encrypt the random number with the server's RSA public key, and send it to the server through the "Change Cipher Key Exchange" message.

After receiving it, the server decrypts it with the RSA private key and obtains the random number (pre-master) sent by the client.

At this point, both the client and the server share three random numbers, namely Client Random, Server Random, and pre-master.

Then, both parties generate a session key (Master Secret) based on the three random numbers they have obtained. It is a symmetric key used to encrypt and decrypt subsequent HTTP request/response data.

After generating the session key, the client sends a "Change Cipher Spec" to tell the server to start sending messages using encryption.

Then, the client sends an "Encrypted Handshake Message (Finished)" message, which summarizes all the previously sent data and encrypts it with the session key (master secret), allowing the server to verify whether encrypted communication is available and whether the previous handshake information has been tampered with in the middle.

It can be found that the TLS handshake data transmitted before "Change Cipher Spec" is all plain text, and the data after that is all ciphertext encrypted with symmetric key.

TLS fourth handshake

The server performs the same operation, sending "Change Cipher Spec" and "Encrypted Handshake Message". If both parties verify that encryption and decryption are OK, the handshake is officially completed.

Finally, the "session key" is used to encrypt and decrypt HTTP requests and responses.

RSA algorithm flaws

The biggest problem with using the RSA key negotiation algorithm is that it does not support forward secrecy. Because the client uses public key encryption to encrypt the random number (one of the conditions for generating symmetric encryption keys) to the server, the server will use the private key to decrypt the random number after receiving it. Therefore, once the server's private key is leaked, all TLS communication ciphertexts intercepted by third parties in the past will be cracked.

In order to solve this problem, the DH key negotiation algorithm was created. Here is a brief introduction to its workflow.

The client and server will each generate a random number and use it as a private key, and then calculate their respective public keys based on the public DH calculation announcement. The two parties will exchange their respective public keys through the TLS handshake, so that both parties have their own private key and the other party's public key. Then both parties will calculate a random number based on the materials they hold. The value of this random number is the same for both parties, which can be used as the key for subsequent symmetric encryption.

During the DH key exchange process, even if a third party intercepts the public key transmitted during the TLS handshake phase, it is impossible to calculate the key without knowing the private key. Moreover, each symmetric encryption key is generated in real time to achieve forward secrecy.

However, due to the computational efficiency of the DH algorithm, the ECDHE key negotiation algorithm emerged later. Most of our websites now use the ECDHE key negotiation algorithm. The process of the ECDHE handshake will be revealed in the next article, so stay tuned.

Original link: https://mp.weixin.qq.com/s/U9SRLE7jZTB6lUZ6c8gTKg