If you were asked to design the SSL/TLS protocol

Preface

Speaking of network communication protocols, I believe everyone is familiar with TCP and HTTP, which can be said to be the cornerstones of today's Internet communication. However, in terms of network security, they have great security risks:

Eavesdropping risk: A third-party attacker can eavesdrop on communications at will, such as obtaining payment account passwords.
Impersonation risk. A third-party attacker can impersonate someone else to communicate with you, such as impersonating a bank website to steal your bank account password.
Tampering risk: A third-party attacker can modify the communication content at will, such as adding a phishing URL to the response.

For this reason, the SSL/TLS protocol came into being. SSL/TLS is a secure communication protocol built on the transport layer and below the application layer. Its main design intention is to eliminate the above-mentioned security risks and ensure network communication security. The HTTPS we are familiar with is built by HTTP + SSL/TLS. It can be said that SSL/TLS is the cornerstone of today's Internet secure communication.

So, if you were asked to design the SSL/TLS protocol, how would you design it?

This article will introduce how to design a simplified version of SSL/TLS step by step from the designer's perspective. At the end of the article, the working mechanism of TLS 1.2 version will be briefly introduced to help everyone have a deeper understanding of the basic principles of the SSL/TLS protocol.

Data encryption based on symmetric encryption algorithm

The risk of eavesdropping is mainly due to the fact that the communicating parties transmit data in plain text on the network, which allows attackers to obtain the content of the communication through simple network packet capture.

To solve the risk of eavesdropping, the best way is to encrypt the data. That is, the client encrypts the data before sending it out; after receiving the ciphertext, the server decrypts and restores the data. This can avoid the spread of plaintext on the network, thereby preventing eavesdropping by third-party attackers.

When it comes to encryption algorithms, many people first think of symmetric encryption algorithms, which are known for their simplicity and efficiency. Symmetric encryption refers to the use of the same key for both encryption and decryption. Common algorithms include DES, AES, etc.

Now, let's try to use a symmetric encryption algorithm to achieve secure communication:

The premise of using symmetric key encryption is that both parties in communication must use the same key to encrypt data. There are two main solutions for offline and online key exchange to achieve this goal:

Offline key exchange, that is, the two parties agree to exchange keys in person offline (for example, through a USB flash drive). This solution can ensure the security of key exchange, but it is difficult to promote and use because in most scenarios, the client and server cannot meet each other.
Online key exchange, that is, transmitting keys through the network. However, the plaintext transmission of keys over the network can also be intercepted by attackers, and such encryption is meaningless.

Therefore, simple symmetric encryption cannot meet the requirements of communication security, and we need to continue to optimize...

Data encryption based on asymmetric encryption algorithm

Asymmetric encryption algorithms refer to encryption and decryption using different keys. These two different keys form a key pair, namely a public key and a private key. The public key is a public key that is available to everyone; the private key is kept confidential. When we use the public key to encrypt data, only the corresponding private key can complete the decryption. Common asymmetric encryption algorithms include RSA, ECC, etc.

Now, let's try to use an asymmetric encryption algorithm to achieve secure communication:

Through asymmetric encryption algorithms, we can both encrypt data and solve the problem of key exchange, thereby eliminating the risk of eavesdropping. However, the biggest disadvantage of asymmetric encryption algorithms is that the encryption and decryption speed is very slow, which is more than 1,000 times slower than symmetric encryption algorithms. Therefore, asymmetric encryption algorithms are usually only suitable for encrypting small amounts of data.

So far, simply using symmetric encryption algorithms or asymmetric encryption algorithms cannot meet the requirements, and further optimization is needed...

Data encryption based on symmetric encryption + asymmetric encryption algorithm

Since the symmetric encryption algorithm has fast encryption and decryption speed, but there is a problem of key exchange; and the asymmetric encryption algorithm can solve the key exchange problem, but the encryption and decryption speed is slow, then we can combine the two algorithms, that is, encrypt data through the symmetric encryption algorithm, and use the asymmetric encryption algorithm to encrypt the symmetric key when exchanging the symmetric key, to ensure that the key will not be eavesdropped by attackers during network transmission.

Now, let's try to use symmetric encryption + asymmetric encryption algorithm to achieve secure communication:

By using the solution of symmetric encryption + asymmetric encryption algorithm, we have eliminated the risk of eavesdropping and there will be no encryption and decryption performance issues, but we still cannot eliminate the risk of impersonation.

Consider the following scenario:

The attacker intercepts the server's public key and saves it.
The attacker pretends to be the server and sends his public key to the client.
The attacker intercepts the symmetric key encrypted with an illegal public key, decrypts it, and obtains the plaintext of the symmetric key and saves it .
The attacker uses the server's public key to re-encrypt the symmetric key, forging it into the client's name and sending it to the server.

After this operation, the attacker can obtain the symmetric key without the knowledge of the client and the server. In this scenario, the attacker switches from passive eavesdropping to active impersonation, making the client and the server mistakenly believe that they are communicating with each other.

Therefore, we need to find a way for the client to ensure that the public key it receives is sent by the real server, that is, to authenticate the real identity of the "server"...

CA certificate-based authentication

Digital Certificate Overview

Quoting the definition from Baidu Encyclopedia:

A digital certificate is a digital certification that marks the identity information of the communicating parties in Internet communications. People can use it to identify each other's identities online.

A digital certificate is like an ID card in the real world, used to identify the legal identity of a network user (person, company, server, etc.). Just like an ID card must be issued by the public security bureau, a trusted digital certificate must also be issued by an authoritative organization, the Certificate Authority (CA). The digital certificate issued by CA is usually called a CA certificate.

A CA certificate mainly contains the applicant's public key, applicant's information, issuing agency CA's information, validity period, certificate serial number and other plain text information. It also contains a CA digital signature. It is the existence of this signature that proves the validity of the certificate.

Digital signatures are based on asymmetric encryption algorithms. When issuing a certificate, the CA first uses a specified algorithm (such as the SHA256 algorithm) to calculate a digital summary of the certificate's plain text information, and then uses the CA's private key to encrypt the summary to form a signature.

The certificate verification mainly includes the following two parts:

Check whether the plain text information of the certificate is valid, such as whether the certificate has expired, whether the domain name is consistent, etc.
Use the public key published by the CA to decrypt the certificate signature and obtain digital summary 1. Then use the same algorithm to calculate the certificate plaintext information to obtain digital summary 2. Compare digital summary 1 and digital summary 2 to see if they are equal.

There is not just one organization that issues certificates. For example, organization A can use the root certificate issued by the CA to issue a second-level certificate to organization B; organization B can use the second-level certificate to issue a third-level certificate to organization C, and so on. This is the so-called certificate chain.

Use CA certificates to authenticate the identities of both parties in communication

Now, we add the CA certificate to authenticate the identities of both parties:

After the CA certificate is introduced, the server's public key is placed in the certificate it provides. When the client verifies the server's certificate, it means that the public key is indeed a legitimate public key from the server. In this way, the subsequent communication process can proceed normally.

However, if the symmetric key remains unchanged, an attacker is likely to crack the symmetric key by brute force. Therefore, we also need to make the symmetric key different for each connection...

Use random numbers to generate symmetric keys

In order to make the symmetric key different for each connection, we can introduce random numbers to generate the symmetric key to ensure its randomness. However, considering that the random numbers currently generated by computers are all pseudo-random numbers, in order to further improve the randomness, we can achieve this goal by generating multiple random numbers.

We can design it like this:

The client and server first generate a random number (random number 1 and random number 2) respectively, and exchange them in the ClientHello and ServerHello messages.
After verifying the identity of the server, the client generates a random number 3 and encrypts it with the server's public key and sends it to the server. (At this point, both parties have the same 3 random numbers) .
The client and server then generate the final symmetric key based on these three random numbers.

In this way, the key generated by three random numbers can better ensure the randomness of the key and reduce the possibility of being cracked by attackers.

Although random number 1 and random number 2 are transmitted in plain text, random number 3 is transmitted in encrypted form, which ensures that it is difficult for attackers to crack the key.

So far, we have successfully prevented attackers from eavesdropping on the communication between the client and the server through various means. However, if the attacker does not intend to eavesdrop on the communication but simply wants to cause damage. For example, the attacker intercepts the ClientHello message and changes the random number 1 to the random number 4, which will cause the keys generated by the client and the server to be inconsistent. In this scenario, although the connection has been established, the client and the server cannot communicate normally:

To this end, we need a mechanism to verify the correctness of all messages during the connection establishment phase (handshake phase) to prevent the establishment of incorrect connections.

Verify the correctness of the handshake message

We can use digital summaries to verify the correctness of all handshake messages. That is, at the end of the handshake phase, both communicating parties use a hash algorithm (such as SHA256) to calculate a digital summary of all messages they receive and send, and then use the previously negotiated symmetric key to encrypt the digital summary and send it to the other party.

When receiving the digital summary ciphertext sent by the other party, first decrypt it with the symmetric key. If the decryption is successful, it means that the key generation is correct. Then compare whether the digital summaries of both parties are consistent. If they are consistent, it means that the message in the handshake phase has not been tampered with, and a correct connection can be established.

Up to now, we have basically eliminated the risk of eavesdropping (through data encryption), impersonation (through certificate authentication), and tampering (through digital summaries). However, in order to establish a secure communication channel, we need to go through multiple steps such as message interaction, encryption and decryption, and identity authentication, which has a certain performance loss.

Because after a handshake, the key has been negotiated and saved by both parties. The next time a connection is established, the key negotiated in the last handshake can be used, thus avoiding the need to renegotiate the key and improving performance. We need a mechanism to reuse sessions to improve the performance of the protocol...

Reusing sessions to improve performance

In order to use the key negotiated last time, we assign a session ID to each connection.

It is generated by the server when the connection is first established and returned to the client via ServerHello.
When creating a connection next time, the client sends the session ID to the server through ClientHello, indicating that it wishes to reuse the session.
After receiving the session ID, the server can send a Finished message to indicate that it agrees to reuse the session, which means that it can continue to use the key negotiated in the last session for secure communication.

So far, we have completed the design of a simplified version of the SSL/TLS protocol. The real SSL/TLS protocol is of course not that simple, but the core ideas and basic principles are similar. It’s just that SSL/TLS adds some mechanisms for better security, scalability and ease of use, such as supporting multiple encryption algorithms and using MAC instead of ordinary digital digests to complete integrity verification.

Below, we will briefly introduce the working mechanism of the actual SSL/TLS protocol.

Overview of SSL/TLS Protocol Mechanism

The SSL protocol (Secure Sockets Layer) is the predecessor of the TLS (Transport Layer Security) protocol. Their versions have evolved as follows, and the latest version is TLS 1.3. In this section, we will analyze the most widely used TLS 1.2 version and introduce the basic working mechanism of SSL/TLS.

SSL 1.0 version -> SSL 2.0 version -> SSL 3.0 version -> TLS 1.0 version -> TLS 1.1 version -> TLS 1.2 version -> TLS 1.3 version .

SSL/TLS Protocol Overview

The SSL/TLS protocol is located between the transport layer and the application layer in the network protocol stack. It can be divided into two layers, with a total of five sub-protocols:

Record Protocol

The bottom layer Record protocol is responsible for encapsulating the upper layer sub-protocols and providing secure communication capabilities:

Private connection. Use symmetric encryption algorithms (such as AES, RC4, etc.) to encrypt data, and in each connection, the encryption keys negotiated by both parties are different, so as to achieve better security. In addition, the Record protocol can also provide unencrypted encapsulation, such as the Hello message in the handshake phase.
Reliable connection. Use MAC (Message Authentication Code, HMAC currently used by TLS is also a type of MAC) to provide integrity verification for data. Similarly, this function can also be not used during the handshake phase.

Handshake Protocol

The upper-layer Handshake protocol is used in the handshake phase to provide identity authentication, encryption algorithm and key negotiation capabilities for both communicating parties:

Identity authentication. The identity authentication of the other end is completed based on the CA certificate, which uses asymmetric encryption technology (such as RSA, DES, etc.). This function is optional, but the usual practice is to perform at least one-way authentication.
Negotiation of security parameters. Complete the negotiation of security parameters (such as encryption algorithms, hash algorithms, keys, etc.), and ensure that attackers cannot obtain keys during the negotiation process.
Reliable negotiation: Ensure that attackers cannot tamper with messages during the negotiation of security parameters.

The Handshake protocol includes the following message types: ClientHello, SeverHello, Certificate, ServerKeyExchange, CertificateRequest, ServerHelloDone, ClientKeyExchange, CertificateVerify, ChangeCipherSpec, and Finished.

Change Cipher Spec Protocol

The Change Cipher Spec protocol is also used in the handshake phase. When one party in the communication sends a Change Cipher Spec message, it means that the key has been negotiated and the key will be used for encrypted transmission starting from the next message.

Alert Protocol

The Alert protocol is only used when the connection is abnormal. The Alert message types defined by the current protocol are as follows:

 close_notify : Indicates that the sender will not send any more messages, which is used to close the connection normally, similar to the FIN message in TCP
 unexpected_message : An unexpected message was received
 bad_record_mac : The MAC in the received message is incorrect, indicating that the message has been tampered with.
 decryption_failed_RESERVED : Decryption failed, for earlier versions of TLS
 record_overflow : message length overflow, ciphertext length does not exceed 2 ^ 14 + 2048 bytes; compressed plaintext does not exceed 2 ^ 14 + 1024 bytes
 decompression_failure : Decompression failed when using compression function
 handshake_failure : The correct security parameters could not be negotiated during the handshake phase.
 no_certificate_RESERVED : TLS is no longer used to be compatible with SSL 3.0
 bad_certificate : Certificate signature verification failed
 unsupported_certificate : An unsupported certificate type was received
 certificate_revoked : received a revoked certificate
 certificate_expired : Received an expired certificate
 certificate_unknown : In addition to the above 4 situations, other certificate exception scenarios
 illegal_parameter : The parameters of the handshake phase message are illegal, such as range overflow, etc.
 unknown_ca : Certificate issued by an untrusted CA
 access_denied : The certificate verification passed, but the sender refused to continue the handshake
 decode_error : message decoding failed
 decrypt_error : security-related steps failed during the handshake phase, such as signature verification failure, Finished message verification failure, etc.
 export_restriction_RESERVED : Used by earlier TLS versions
 protocol_version : protocol version not supported
 insufficient_security : The security algorithm required by the server is not met by the client
 internal_error : protocol internal error
 user_canceled : The user actively closed the connection abnormally
 no_renegotiation : refuse to re-handshake
 unsupported_extension : unsupported extension

Application Data Protocol

The Application Data protocol is used in the communication phase to encapsulate the application layer data, which is then encapsulated by the Record protocol and forwarded through the TCP protocol.

SSL/TLS protocol handshake process

First Handshake

The client sends a ClientHello message to the server to initiate a connection, which carries the following content:

Version: TLS protocol version supported by the client
Random: A random number generated by the client, which is then used to generate the master secret
SessionID: Session ID. If it is not empty, it means the client wants to reuse the session.
CipherSuites: A list of encryption suites supported by the client, which must be carried when SessionID is empty.
CompressionMethods: List of compression algorithms supported by the client
Extensions:

Second handshake

1. The server sends a ServerHello message to the client, which carries the following content:

Version: The selected TLS version. Select the highest version supported by both parties.
Random: A random number generated by the server, which is then used to generate the master secret
SessionID: Session ID. If it is empty, it means that the server starts a new session and does not want it to be reused. If it is the same as the SessionID brought by the client, it means that the session is reused. Otherwise, a new session is started and may be reused in the future.
CipherSuite: The selected cipher suite
CompressionMethod: The selected compression algorithm
Extensions:

2. The server sends a Certificate message to the client, which carries the server's certificate. The certificate must be in the x.509 standard format and contain the server's public key, server domain name, issuer information, validity period, and other information.

Optional. Sent when the client needs to authenticate the server's identity using a certificate .

3. The server sends a Server Key Exchange message to the client, which carries the security parameters used by the client to generate the premaster secret.

Optional, sent when the information carried in the Certificate message cannot support the client to generate a premaster secret .

4. The server sends a CertificateRequest message to the client, requesting the client certificate, which includes the expected certificate type, signature algorithm, and CA list.

Optional. Sent when mutual authentication is enabled .

5. The server sends a ServerHelloDone message to the client, indicating that the server has sent all key exchange-related content.

The third handshake

1. The client sends a Certificate message to the server, which carries the client certificate.

Optional, sent when receiving a CertificateRequest message from the server .

2. The client sends a ClientKeyExchange message to the server, which contains the premaster secret (random number) encrypted with the server's public key, which is then used to generate the master secret.

3. The client sends a CertificateVerify message to the server, which contains the digital signatures of all handshake messages between the two communicating parties so far, used to prove that the private key it owns corresponds to the public key in the certificate sent previously.

Optional. Sent when sending a Certificate message to the server .

4. The client sends a ChangeCipherSpec message to the server, indicating that encrypted transmission will begin from the next message.

5. The client sends a Finished message to the server, which is encrypted and transmitted, and contains the digital summary of all handshake messages to prevent tampering.

The fourth handshake

1. The server sends a ChangeCipherSpec message to the client, indicating that encrypted transmission will begin from the next message.

2. The server sends a Finished message to the client, which is encrypted and transmitted. It contains the digital summary of all handshake messages to prevent tampering.

at last

The SSL/TLS protocol is not absolutely secure. It also has many vulnerabilities that are constantly discovered by hackers. Of course, the SSL/TLS protocol is also constantly being improved. The TLS 1.3 version released in 2018 has made many enhancements based on the TLS 1.2 version. For example, in terms of performance, the handshake phase is reduced from 2-RTT to 1-RTT, and supports 0-RTT mode; in terms of security, encrypted transmission begins after the ServerHello message, and some insecure encryption suites are no longer supported (such as static RSA, Diffie-Hellman, etc.).

Although TLS 1.3 has changed a lot in terms of mechanism, the basic principles remain the same. Therefore, if we fully understand the principles of the SSL/TLS protocol, we can quickly learn the protocol mechanism no matter how the version evolves.