It’s time to understand HTTPS and the encryption behind it

HTTPS (Hypertext Transfer Protocol Secure) is an HTTP channel with security as its goal. Simply put, it is a secure version of HTTP. This article will introduce its principles in depth.

Why HTTPS is needed

The reason for using HTTPS is actually very simple, that is, HTTP is insecure.

[[264137]]

When we send relatively private data to the server, if we use HTTP for communication, security will not be guaranteed.

First of all, during the data transmission process, the data may be captured by the middleman, and then the data will be stolen by the middleman.

Secondly, after the data is obtained by the middleman, the middleman may modify or replace the data and then send it to the server.

After the server receives the data, it cannot determine whether the data has been modified or replaced. Of course, if the server cannot determine whether the data really comes from the client.

In summary, HTTP has three disadvantages:

• Confidentiality of messages cannot be guaranteed
• The completeness and accuracy of the message cannot be guaranteed
• The reliability of the source cannot be guaranteed

HTTPS was created to solve the above problems.

Basic concepts of HTTPS

In order to solve the problems existing in HTTP, HTTPS uses some encryption and decryption, digital certificates, and digital signature technologies. The following introduces the basic concepts of these technologies.

Symmetric encryption vs. asymmetric encryption

In order to ensure the confidentiality of messages, encryption and decryption are needed. The current mainstream encryption and decryption algorithms are divided into symmetric encryption and asymmetric encryption.

①Symmetric encryption (shared key encryption): The client and server share a key to encrypt and decrypt messages. This method is called symmetric encryption.

The client and the server agree on an encryption key. The client uses the key to encrypt the message before sending it, and after sending it to the server, the server uses the key to decrypt the message.

Advantages of symmetric encryption: Symmetric encryption solves the problem of message confidentiality in HTTP.

Disadvantages of symmetric encryption: Symmetric encryption ensures message confidentiality, but because the client and server share a key, the key is particularly easy to leak. Because the risk of key leakage is high, it is difficult to ensure the reliability of the source of the message, the integrity and accuracy of the message.

②Asymmetric encryption (public key encryption): Since the key is so easy to leak in symmetric encryption, we can use an asymmetric encryption method to solve it.

When using asymmetric encryption, both the client and the server have a public key and a private key. The public key can be exposed to the outside world, while the private key is only visible to oneself.

A message encrypted with a public key can only be decrypted with the corresponding private key. Conversely, a message encrypted with a private key can only be decrypted with the public key.

In this way, before sending a message, the client first encrypts the message with the server's public key, and the server decrypts it with its own private key after receiving it.

Advantages of asymmetric encryption:

• Asymmetric encryption uses public keys and private keys to solve the problem of message confidentiality in HTTP and reduce the risk of private key leakage.
• Because messages encrypted with a public key can only be decrypted by the corresponding private key, the source, accuracy, and integrity of the message are guaranteed to a large extent.

Disadvantages of asymmetric encryption:

• Asymmetric encryption requires the recipient's public key to encrypt the message, but the public key is not confidential and can be obtained by anyone, including the middleman.

Then the middleman can do two things. First, the middleman can replace the client's public key with his own when the client and the server exchange public keys.

In this way, the public key obtained by the server will not belong to the client, but to the server. The server cannot determine the correctness of the source of the public key.

The second thing is that the middleman may not replace the public key, but he can intercept the message sent by the client, then tamper with it, and then encrypt it with the server's public key and send it to the server. The server will receive an incorrect message.

• The performance of asymmetric encryption is several times or even hundreds of times slower than that of symmetric encryption, and it consumes more system resources. That is why HTTPS combines the two encryption methods.

Digital Certificates and Digital Signatures

In order to solve the insecurity of the source of public keys in asymmetric encryption, we can use digital certificates and digital signatures to solve it.

①Application for digital certificate

In reality, there are some specialized authoritative organizations that issue digital certificates. We call these organizations certification centers (CA, Certificate Authority).

We (servers) can apply for digital certificates from these CAs. The application process is roughly as follows: first generate a pair of keys locally, then take your public key and other information (such as company name, etc.) to the CA to apply for a digital certificate.

After receiving this information, CA will select a one-way hash algorithm (such as the common MD5) to encrypt the information. The encrypted information is called a summary.

One-way hashing algorithm has a characteristic that it is irreversible. As long as the original content changes a little, the encrypted data will be very different (of course, there is a small possibility of duplication. Interested friends can learn about the pigeon hole principle), thus preventing the information from being tampered with.

Generating the summary is not the end, the CA will also encrypt the summary with its own private key. The encrypted data of the summary is called a digital signature.

The CA will combine our application information (including the server's public key) and the digital signature to generate a digital certificate. The CA will then pass the digital certificate to us.

②How does a digital certificate work?

After obtaining the digital certificate, the server will send the digital certificate to the client. The client needs to use the CA's public key to decrypt the digital certificate and verify the legitimacy of the digital certificate.

So how can we get the CA's public key? Our computers and browsers have built-in root certificates of some authoritative organizations, and these root certificates contain the CA's public key.

The reason why it is a root certificate is that in real life, certification centers are hierarchical, that is, there are certification centers and sub-certification centers below them. It is a tree structure. The computer has the root certificate of the organization built into it, but don’t worry, the public key of the root certificate is also applicable to the sub-level.

The client uses the CA's public key to decrypt the digital certificate. If the decryption is successful, it means that the certificate comes from a legitimate certification authority. After the decryption is successful, the client gets the summary.

At this point, the client will generate a summary of the application information using the same hash algorithm as the CA, and compare it with the decrypted one. If they are the same, it means that the content is complete and has not been tampered with.

The client can obtain the server's public key from the certificate securely and then communicate with the server in a secure asymmetric manner. The server can also obtain the client's public key in the same way.

The following figure illustrates the general certificate application and its use process in a graphical way:

HTTPS Principles

Through the above study, we understand the characteristics, advantages and disadvantages of symmetric encryption and asymmetric encryption, as well as the role of digital certificates.

HTTPS is not implemented using a single technology, but rather fully integrates these technologies based on their characteristics to maximize performance and security.

This integrated technology is called SSL (Secure Sockets Layer). So HTTPS is not a new protocol, it just puts an encrypted shell on HTTP.

To establish HTTPS, let’s take a look at the flow chart:

Here, HTTPS establishment to disconnection is divided into 6 stages and 12 processes. The following will explain the 12 processes one by one:

• The client starts SSL communication by sending a Client Hello message, which contains the specified version of SSL supported by the client and a list of cipher suites (the encryption algorithm used and the key length, etc.).
• When the server can perform SSL communication, it will respond with a Server Hello message. Like the client, the message contains the SSL version and encryption component. The server's encryption component content is filtered out from the received client encryption component.
• The server sends a certificate message, which contains the public key certificate.
• The server sends a Server Hello Done message to notify the client that the initial SSL handshake negotiation is over.
• After the SSL handshake is complete, the client responds with a Client Key Exchange message. The message contains a random string of characters called Pre-master secret used in communication encryption. The message has been encrypted with the public key in step 3.
• The client then continues to send a Change Cipher Spec message, which will prompt the server that the communications after this message will be encrypted using the Pre-master secret key.
• The client sends a Finished message. This message contains the overall checksum of all messages connected so far. Whether this handshake negotiation is successful depends on whether the server can correctly decrypt the message.
• The server also sends a Change Cipher Spec message.
• The server also sends a Finished message.
• After the server and client have exchanged Finished messages, the SSL connection is established. Of course, the communication is protected by SSL. From here, the application layer protocol communication begins, that is, sending HTTP requests.
• Application layer protocol communication, that is, sending HTTP responses.
• The client disconnects. When disconnecting, a close_notify message is sent. The above diagram has some omissions. After this step, a TCP FIN message is sent to close the communication with TCP.

In addition, in the above flowchart, when the application layer sends data, it will attach a message digest called MAC (Message Authentication Code). MAC can detect whether the message has been tampered with, thereby ensuring the integrity of the message.

The following diagram is more detailed than the diagram of the digital certificate (the picture comes from "HTTP Diagram"):

From the above introduction, we can see that HTTPS first uses digital certificates to ensure that the server's public key can reach the client safely and accurately.

Then use asymmetric encryption to securely transmit the shared key, and use the shared key to securely exchange data.

Use of HTTPS

HTTPS is so secure, do we have to use HTTPS for communication in all scenarios? The answer is no.

①Although HTTPS provides a channel for secure message transmission, the encryption and decryption of each message is very time-consuming and consumes system resources.

Therefore, unless we must use HTTPS for communication in some scenarios with high security requirements, such as banking systems and shopping systems, there is no need to use HTTPS in other scenarios with low security requirements.

② Using HTTPS requires the use of a digital certificate, but digital certificates issued by authoritative organizations are generally charged, and the price is not cheap.

Therefore, for some personal websites, if the security requirements are not high, there is no need to use HTTPS.