Understand how HTTPS works

HTTPS, also known as HTTP over TLS. The predecessor of TLS is SSL. TLS 1.0 is usually marked as SSL 3.1, TLS 1.1 is SSL 3.2, and TLS 1.2 is SSL 3.3. This article focuses on version 1.2 of the TLS protocol.

[[274278]]

The following figure describes the relationship between TLS (each subprotocol) and HTTP in the TCP/IP protocol stack:

Credit: Kaushal Kumar Panday From: SSL Handshake and HTTPS Bindings on IIS

Among them, Handshake protocol, Change Ciper Spec protocol and Alert protocol constitute SSL Handshaking Protocols.

Compared with HTTP protocol, HTTPS provides:

• Data integrity: Content transmission is integrity checked
• Data privacy: Content is symmetrically encrypted, and each connection generates a unique encryption key
• Identity authentication: A third party cannot forge the server (client) identity

Among them, data integrity and privacy are guaranteed by TLS Record Protocol, and identity authentication is achieved by TLS Handshaking Protocols.

Overview

The SSL handshake process using the RSA algorithm is as follows:

Source: Keyless SSL: The Nitty Gritty Technical Details

1. [Plain text] The client sends the random number client_random and a list of supported encryption methods
2. [Plain text] The server returns a random number server_random, the selected encryption method and the server certificate chain
3. [RSA] The client verifies the server certificate and uses the public key in the certificate to encrypt the premaster secret and send it to the server
4. The server uses the private key to decrypt the premaster secret
5. Both ends generate a master secret through client_random, server_random and premaster secret, which is used to symmetrically encrypt subsequent communication content.

Certificate (Digital certificate)

So what is a certificate?

What information is included in the certificate:

• Certificate information: expiration date and serial number
• Owner information: name, etc.
• Owner public key

Why does the server send a certificate to the client?

There are so many services on the Internet that require certificates to verify their identities that the client (operating system or browser, etc.) cannot have all the certificates built in and needs to send the certificates to the client through the server.

Why should the client verify the received certificate?

 Client < ------------Attacker <------------Server  
 Forged certificates Intercept requests

How does the client verify the received certificate?

In order to answer this question, we need to introduce digital signature.

 + ---------------------+  
 | A digital signature |
 |( not   to be confused |
 | with a digital |
 |certificate) | + ---------+ +--------+  
 | is a mathematical | ----Hash--->| Message Digest |---Private Key Encryption--->| Digital Signature |  
 |technique used | + ---------+ +--------+  
 | to validate the |
 |authenticity and |
 |integrity of a |
 |message, software |
 | or digital document. |
 + ---------------------+  

A digital signature is generated by encrypting a piece of text using a hash and private key.

Assume that a message is sent between Bob, Susan, and Pat. Susan sends a message to Bob along with a digital signature. After Bob receives the message, he can verify that the message he received is the one sent by Susan.

 + ---------------------+  
 | A digital signature |
 |( not   to be confused |
 | with a digital |
 |certificate) | + ---------+  
 | is a mathematical | ----hash--->| message digest |  
 |technique used | + ---------+  
 | to validate the | |
 |authenticity and | |
 |integrity of a | |
 |message, software |
 | or digital document.
 + ----------------------+ |  
 |
 |
 + --------+ +---------+  
 | Digital Signature | ---Public Key Decryption--->| Message Digest |  
 + --------+ +---------+  

Of course, this assumes that Bob knows Susan's public key. More importantly, like the message itself, the public key cannot be sent directly to Bob over an insecure network.

This is where the Certificate Authority (CA) comes in. There aren't many CAs, and Bob's client has certificates for all trusted CAs built in. The CA digitally signs Susan's public key (and other information) to generate a certificate.

After Susan sends the certificate to Bob, Bob verifies the certificate signature using the public key of the CA certificate.

Bob trusts CA, and CA trusts Susan, which makes Bob trust Susan. This is how the chain of trust is formed.

In fact, Bob's client has a built-in CA root certificate. In the HTTPS protocol, the server will send a certificate chain to the client.

TLS protocol

The TLS protocol includes the TLS Record Protocol and the TLS Handshake Protocol. The flowchart in the overview only involves the TLS Handshake Protocol.

TLS Record Protocol

In the TLS protocol, there are four sub-protocols running on top of the Record protocol:

• Handshake protocol
• Alert protocol
• Change cipher spec protocol
• Application data protocol

The Record protocol plays this role

• On the sending end: segment the data (Record), compress it, add MAC (Message Authentication Code) and encrypt it
• On the receiving end: decrypt the data (Record), verify the MAC, decompress and reassemble

It is worth mentioning that the Record protocol provides data integrity and privacy guarantees, but the Record type and length are publicly transmitted.

Record Protocol has three connection states, which define compression, encryption and MAC algorithms. All records are processed by the algorithm determined by the current state.

TLS Handshake Protocol and Change Ciper Spec Protocol will cause the Record Protocol state to switch.

 empty state ------------------> pending state ------------------> current state  
 Handshake Protocol Change Cipher Spec

The initial current state (Current State) does not specify encryption, compression and MAC algorithms, so before completing a series of actions of the TLS Handshaking Protocols, the data between the client and the server are transmitted in plain text; when TLS completes the handshake process, the client and the server determine the encryption, compression and MAC algorithms and their parameters, and the data (Record) will be processed by the specified algorithm.

Among them, the Record is first encrypted, and then a MAC (message authentication code) is added to ensure data integrity.

TLS Handshaking Protocols

Handshakeing protocols include Alert Protocol, Change Ciper Spec Protocol and Handshake protocol. This article will not introduce Alert Protocol and Change Ciper Spec Protocol in detail.

The handshake process using the RSA algorithm is as follows (mentioned in the overview)

Source: Keyless SSL: The Nitty Gritty Technical Details

The client and server exchange client_random and server_random in plain text in the handshake hello message, use RSA public key to encrypt and transmit the premaster secret, and finally use the algorithm to calculate the master secret respectively. The reason for not using the premaster secret directly is to ensure that the randomness of the secret is not affected by either party.

In addition to using the RSA algorithm to exchange keys on a public channel, you can also use the Diffie-Hellman algorithm. The principle of the Diffie-Hellman algorithm is as follows:

 By Original schema : AJ Han Vinck, University of Duisburg-Essen SVG version: Flugaal [ Public domain], via Wikimedia Commons

Process of exchanging premaster secret using Diffie–Hellman algorithm

Source: Keyless SSL: The Nitty Gritty Technical Details

summary

TLS Handshaking Protocols negotiates the algorithms and required parameters used by TLS Record Protocol and verifies the identity of the server; after negotiation, TLS Record Protocol ensures the integrity and privacy of application layer data.

The core of TLS Handshaking Protocol is to transmit premaster secret over public channels.

Q&A

Why not just use asymmetric encryption to transfer content?

performance

Can HTTPS guarantee normal connection?

no

There are a number of ways in which a man-in-the-middle attacker can attempt to make two entities drop down to the least secure method they support.

The attacker can even directly discard the data packets from both parties.

How does the server verify the client's identity?

Through Client Certificate

This message conveys the client's certificate chain to the server; the server will use it when verifying the CertificateVerify message (when the client authentication is based on signing) or calculating thepremaster secret (for non-ephemeral Diffie-Hellman). The certificate MUST be appropriate for the negotiated cipher suite's key exchange algorithm, and any negotiated extensions.

What does the Alert protocol do?

Closure Alerts: Prevent Truncation Attack

In a truncation attack, an attacker inserts into a message a TCP code indicating the message has finished, thus preventing the recipient picking up the rest of the message. To prevent this, SSL from version v3 onward has a closing handshake, so the recipient knows the message has not ended until this has been performed.

Error Alerts: Error Handling

How is the master secret calculated?

 master_secret = PRF(pre_master_secret, "master secret" ,
 ClientHello.random + ServerHello.random)
 [0..47];

How are encryption, compression and MAC algorithm parameters calculated?

Handshaking Protocols enables the client and server to exchange three parameters: client_random, server_random and master_secret. The parameters required by the algorithm are generated through the following algorithm

 To generate the key material, compute
 key_block = PRF(SecurityParameters.master_secret,
 "key expansion" ,
 SecurityParameters.`server_random` +
 SecurityParameters.`client_random`);
 until enough output has been generated. Then , the key_block is  
 partitioned as follows:
 client_write_MAC_key[SecurityParameters.mac_key_length]
 server_write_MAC_key[SecurityParameters.mac_key_length]
 client_write_key[SecurityParameters.enc_key_length]
 server_write_key[SecurityParameters.enc_key_length]
 client_write_IV[SecurityParameters.fixed_iv_length]
 server_write_IV[SecurityParameters.fixed_iv_length]

The master secret is expanded into a sequence of secure bytes, which is then split to a client write MAC key, a server write MAC key, a client write encryption key, and a server write encryption key

TLS handshake details using the Diffie-Hellman algorithm