The HTTPS certificate revocation mechanism that you don’t care about may cause you catastrophic security problems!

origin

I occasionally watched "The Longest Day in Chang'an". During my nap, I dreamed that I traveled back to the Tang Dynasty and worked as a deputy secretary in the Jing'an Division in Chang'an City for a day. I was not in the division when Xu Bin was killed and disappeared, so I don't know what happened at that time.

Later, Xu Bin woke up. According to his description, "Messenger Lu San" was a secret agent who nearly killed Xu Bin. I was embarrassed that a highly intelligent person who was good at casework was deceived by a few words of an ordinary messenger. Lu San was a messenger and was familiar with the signal of the watchtower, the command transmission system of the Jing'an Division. The news that he was a secret agent spread throughout the organization. This made Zhang Xiaojing and Yao Runeng think that the watchtower system could no longer complete the function of confidential message transmission. In fact, they did not know the watchtower at all.

[[315602]]

The entire watchtower system is composed of a "transmission system + encryption system". As a military-level organization, the Jing'an Division's information transmission is definitely multi-encrypted. Just understanding the watchtower pattern or only having the code book is not enough to decipher the code. As for the influence of the communication that Lu San is a hidden stake, it only needs to replace the code book.

I designed the Watchtower system after learning RSA asymmetric encryption, and I have already evaluated these risks. Even if the key is lost during HTTPS communication... Hmm? What happens if the HTTPS certificate private key is lost? Is there no way to prevent the private key from being used? Thinking of this question, I suddenly woke up from my dream and went to review the certificate revocation mechanism.

doubt

• Who determines when an HTTPS certificate expires?
• Who checked the legitimacy of the certificate?
• When is it triggered?
• Does it affect performance?
• How to revoke a certificate?
• The HTTPS request is initiated by the client (browser). How does it know that the certificate is revoked?
• What is the process for validating an HTTPS certificate?

HTTPS communication process

Everyone knows that the HTTPS handshake is completed after the TCP handshake. The process is very familiar, but it is still necessary to review it:

• In the first phase, the handshakes such as Client Hello and Server Hello are completed, including the response parameters such as the SSL version, random numbers of the server and client, cipher suite, and data compression.
• In the second stage, the server sends the public key of the domain name certificate to the browser (client), and the browser (client) verifies the legitimacy of the certificate.
• In the third stage, the client sends its certificate to the server (in the case of certificate login), and the server detects the client certificate.
• In the fourth stage, key negotiation and symmetric encryption key exchange are completed.

Abbreviation explanation: RN: Random Number; PMS: Pre Master Secret; MS: Master Secret

Regarding the certificate verification in the second stage, I believe many people don’t know much about it, and don’t even know what will be tested, so let’s learn about it below.

Certificate integrity verification

Use RSA public key decryption to verify whether the private key signature on the certificate is legal. If the signature is invalid, it can be determined that the certificate has been modified and an error will be reported directly.

Certificate validity verification

When issuing a certificate, the CA sets a validity period for each certificate, including a start time and an end time. If the current system time is not within the start and end time of the certificate, the certificate is considered invalid.

Certificate revocation status detection

What if the certificate is lost within the validity period? The certificate needs to be revoked, so there is an additional certificate revocation status check. The user notifies the CA service provider of the certificate that needs to be revoked, and the CA service provider notifies the browser of the revocation status of the certificate. Let's take a look at a browser reminder after the certificate is revoked:

Chrome returns NET::ERR_CERT_REVOKED and refuses to continue access. It also does not provide an interface for forced access, and there is no manual click link for continued access.

Verify the issuer

HTTPS digital certificates are used in two roles:

• The certificate issuer has the private key of the signing key.
• The subject of the certificate applicant, the user's browser that uses the certificate's public key for authentication checks that the issuer field of the certificate is the same as the subject field of the superior certificate in the certification path.

To increase security, most PKI implementations verify whether the key, signature, and other information of the issuer are the same as the key of the current certificate. However, for the trust chain, the root certificate is issued by itself, which means that their issuer and subject are the same.

At the same time, these CA root certificates are directly embedded into the system by the operating system, browser, etc. For example:

Check domain name (IP) specifications

The intermediate CA provides granular control over the management and issuance of domain name certificates. The validity scope of the certificate is limited to a fixed domain name, domain name range (including subdomains) or a fixed IP. For example, the following figure shows the DNS information of the HTTPS certificate for https://www.baidu.com.

As shown in the figure above, the DNS scope includes multiple domain names, and the second-level and above domain names support the scope format. It is represented by wildcard characters. However, the second-level domain name scope of .example.com cannot include the third-level domain name abexample.com. At the same time, the DNS scope also supports IP, but IP does not support the scope format, and all IP lists must be included in the list.

Check policy constraints

Legal strategy related testing (omitted).

Certificate revocation status detection method

The above mentioned the verification scope of the browser (client) when verifying the legitimacy of the certificate. For the time being, we will only focus on the detection of certificate revocation information. Next, let's take a closer look at the implementation principles of the two detection mechanisms.

1. Certificate Revocation Lists (CRL)

CA will regularly update and publish certificate revocation lists, Certificate Revocation Lists (CRL), RFC 5280: Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile.

CRLs are distributed in publicly available repositories, and browsers can fetch and consult the CA's latest CRL when validating certificates.

One drawback of this method is that the time granularity of revocation is limited to the CRL publication period. The browser will be notified of the revocation only after all currently published CRLs are scheduled to be updated. The strategies of various signing CA manufacturers are different, some are a few hours, some are a few days, and even a few weeks. In 2015, student papers from several American universities counted the CA certificate revocations at that time, as shown in the following figure:

From this statistic, we can see that the number of CRLs of CA certificate manufacturers varies, most of which are 30-50, while GoDaddy has more than 300 CRL addresses, and nearly 300,000 certificates are revoked, with an average file size of 1M.

CRL information of the certificate

CRL information is written into the extension area of ​​X.509 v when the CA issues the certificate, such as the certificate information of alipay.com:

As you can see, its CRL information is:

• http://crl3.digicert.com/SecureSiteCAG2.crl
• http://crl4.digicert.com/SecureSiteCAG2.crl

(2) CRL Detection Process

You can imagine that when the browser verifies the legitimacy of the certificate, it has to download a 1M file and then verify it. Only after the verification is passed can it connect to the website server it wants to visit, which is a waste of time and efficiency. At the same time, many browsers are located in networks with network access restrictions, which may be in a local area network, such as our village, or the physical distance is very far, and there is a serious network delay problem.

2. Online Certificate Status Protocol (OCSP)

In the description of RFC2560X.509 Internet Public Key Infrastructure Online Certificate Status Protocol – OCSP, the browser requests the revocation status of the certificate from the online OCSP server (also called OCSP Response Server), and the OCSP Server responds. This method avoids the problem of CRL update delays. Similarly, the OCSP information of the X.509 v3 certificate is also stored in the extended information, such as the green box in the picture of the alipay.com certificate.

(1) OCSP detection process

After obtaining the public key certificate of the Web server, the browser begins to verify the legitimacy of the public key. Here, it will send an OCSP Response to the OCSP Server address provided in the extended information of the public key. After receiving the response, it confirms that the certificate is valid and then continues to communicate with the Web server.

(2) Advantages of OCSP

Compared with the CRL method, after the certificate is revoked, the CA Server can immediately send the revocation information to the browser, which takes effect quickly. The response packet is short, unlike the CRL response packet, which is nearly 1M or more.

(3) Disadvantages of OCSP

The first disadvantage is that each time a browser creates an HTTPS request, it needs to connect to the CA OCSP Server for verification. Some browsers have IP addresses that are not connected to the CA OCSP Server, such as our village. Moreover, OCSP verification requires network IO, which takes a long time and seriously affects the user experience of the browser accessing the server.

The second disadvantage: When the browser sends the server HTTPS certificate serial number to the CA OCSP Server, it will also expose the user's privacy and reveal the URL visited by the user to the CA OCSP Server.

(4) Problems arising from the OCSP mechanism

Imagine a scenario where you are a browser company and your browser fails to get a response from the OCSP server when checking the certificate revocation status. What would you do?

• If you choose to reject the certificate information and reject subsequent HTTPS communications, this method is called Hard-fail.
• If you choose to trust the certificate and believe it has not been revoked, this method is called soft-fail.

If it is hard-fail mode, then the prerequisite for the browser to access any HTTPS server depends on the OCSP server, which will be a fatal single point of failure. For you who have senior architecture design experience, would you choose this?

If it is in soft-fail mode, the OCSP server response will be ignored if it cannot be obtained. Then, what is the point of this mechanism? What is the use of you?

(5) OCSP Stapling

The OCSP Stapling solution solves the shortcomings of CRL and OCSP by handing over the process of obtaining certificate revocation status through the OCSP Server to the Web server. The Web server can not only directly query the OCSP information to avoid network access restrictions and the physical distance between the OCSP server and the user, but also cache the query response for use by other browsers. Since the OCSP response is also signed by the CA RSA private key, there is no need to worry about forgery.

• Solved the problem of slow access
• Solved the problem of user privacy leakage

The communication process is as above. However, when the Web server queries the certificate status from the OCSP Server, it also faces the problem of not being able to obtain the OCSP Response. When responding to the browser, the browser still has to face the choice of hard-fail and soft-fail, which is very troublesome for the browser.

(6) OCSP Must-Staple

Faced with the problem of hard-fail and soft-fail, browser manufacturers have different attitudes. At the same time, no matter how the browser chooses, it cannot meet the needs of the majority of domain name users, so it is better to leave this choice to the domain name users themselves. For this reason, OCSP Must-Staple came into being, and the browser must detect the OCSP response. When the domain name certificate is created, the custom settings enable this option and enter this information into the X.509 v3 extension. After the browser reads it, it forces OCSP detection and goes into hard-fail mode. This specification was drafted in X.509v3 Extension: OCSP Stapling Required draft-hallambaker-muststaple-00, but it has not yet been adopted as an RFC standard.

(7) CA vendor support

Currently, the CA vendors that support this extension include Let's Encrypt. If you are using a version prior to openssl 1.1.0, you can use 11.3.6.1.5.5.7.1.24 = DER:30:03:02:01:05 to specify it. RFC For example, when generating a CSR, add the following to openssl.cnf:

 [v3_req ]
 basicConstraints = CA :FALSE
 keyUsage = nonRepudiation , digitalSignature, keyEncipherment
 subjectAltName = @alt_names
 1.3.6.1.5.5.7.1.24 = DER :30:03:02:01:05 If you are using OpenSSL 1.1.0 or higher, you can set it like this: [ v3_req ]
 basicConstraints = CA :FALSE
 keyUsage = nonRepudiation , digitalSignature, keyEncipherment
 subjectAltName = @alt_names

Browser support for certificate revocation on various platforms

1. Mac Safari

After Mac OS X 10.7 (Lion), Safari/macOS does not detect CRLs and OCSP by default, and uses its own key chain system. (There is little information, and I can't find much information from Apple official website)

2. Windows Internet Explorer

Starting from Windows Vista, Internet Explorer 7 has built-in CryptoAPI to support OCSP detection of certificate revocation. The detection scope includes the issuer's certificate and the subject server's certificate.

Why is IE slower than other browsers when accessing HTTPS websites? You should already know the answer.

3. Mozilla Firefox

In 2010, all versions of Mozilla Firefox supported OCSP detection. The OCSP detection mechanism was set as the default mechanism in Firefox 3. In subsequent version updates, Firefox has different strategies for intermediate certificates and domain name certificates.

(1) Verification of the revocation status of an intermediate certificate

In 2015, starting with Firefox 37, Mozilla also enabled the self-developed certificate revocation status check mechanism OneCRL to replace the OCSP mechanism for the detection of intermediate certificates. The purpose is to solve the shortcomings of the CRL and OCSP mechanisms. However, intermediate certificates cannot use the OCSP stapling method and are not allowed to be cached. So... there is also,

RFC 6961 defines a mechanism for stapling OCSP responses for CA certificates. Since FIrefox does not rely on OCSP to validate intermediate certificates, we have no plans to implement support for this.

Firefox officially has no plans to support Multi-staple in the short term.

(2) Verification of the revocation status of a domain name certificate

On the official Firefox WIKI, it is mentioned that the revocation verification for domain name certificates is divided into the following five schemes:

• Solution 1: Short-Lived Certificates. By default, for certificates with a validity period of less than 10 days, verification is skipped directly and they are considered unsafe. This can be configured in the security.pki.cert_short_lifetime_in_days parameter.
• Solution 2: OCSP Stapling, same as RFC specification. If the value of security.OCSP.enabled is 0, the OCSP response is ignored.
• Solution 3: OCSP Must-staple, as specified in the RFC. This can be disabled by setting the security.ssl.enable_ocsp_must_staple or security.ssl.enable_ocsp_stapling parameters.
• Solution 4: OCSP, same as RFC specification. If the OCSP response is not returned within 2 seconds (10 seconds for EV certificates), it will be ignored.
• Solution 5: CRLite, a detection mechanism similar to Chrome CRLSets, is used for OCSP and OCSP stapling failure. Firefox officially plans to use this mechanism to replace CRL as the main detection mechanism (OCSP OCSP stapling failure).

4. Chrome

In 2012, Chrome disabled CRLs and OCSP detection: Google Chrome Will No Longer Check for Revoked SSL Certificates Online, and used a self-designed certificate verification mechanism CRLSets

So, what is the reason for Chrome's choice? Obviously, from the comparison between the two verification methods above, CRL and OCSP, the former is not timely and the latter affects performance. Therefore, Google Chrome decided to build its own certificate revocation status verification system.

Adam Langley, a security engineer at Chrome, wrote an article on his blog: "Revocation checking and Chrome's CRL". Adam Langley has very specific opinions on Chrome's HTTPS certificate verification.

I strongly oppose the use of CRL and OCSP to verify the certificate revocation status. I have written several articles on certificate revocation status detection. At the same time, I also mentioned [What's the story with certificate revocation?] in the security section of the Chromium developer homepage:

But this is not a perfect solution to the problem, from [An Evaluation of the Effectiveness of Chrome's CRLSets]:

The biggest problem with CRLSet mentioned in this article is that it contains too little certificate revocation data. In some cases, it accounts for less than 2% of the mainstream CRL certificate revocation information.

Moreover, the update of CRLSets is not so timely. It is understandable that Chrome sacrifices a little security for the sake of user experience, performance, and user privacy. But it seems to be unworthy of the title of the most secure browser. [Manual dog head]

Summary

As shown in the figure above, in 2015, researchers from Northeastern University, University of Maryland, Duke University and other universities analyzed five popular browsers on the market (multiple platforms, multiple versions) and plotted a table showing the support of each browser under different levels of certificates. (The paper is given in the references) As you can see from the figure, Chrome, which is the most secure browser in our impression, performs surprisingly well. It performs the worst in terms of HTTPS security.

As mentioned above, Chrome ignored the certificate revocation status detection mechanisms such as CRL and OCSP for the sake of user experience such as access speed and privacy, and sacrificed the security support of TLS.

However, IE browser is beyond our expectation. It supports HTTPS security very well. It can be said to be the most secure among these browsers, but... but it opens web pages slowly... As of today (July 16, 2019), 4 years have passed, and the support of many browsers and WebServers has changed a lot. It cannot reflect the current status of the Internet. These data are also for reference.

Appendix: WebServer support status

• The Apache web server has supported OCSP stapling since v2.3.3 (ref).
• The nginx web server has supported OCSP stapling since v1.3.7 (ref).

Summarize

1. The harm of certificate leakage

So, if the certificate is lost, will it have a big impact on the security of the website? Let's look back at the conditions for using the certificate:

• Possess certificate
• Have a domain name
• Browser accesses the server

If you have all of these, then you are the official of the website.

In the security world, there is an attack method called Man-in-the-middle attack. If a hacker obtains a certificate and builds a website with the same domain name, and uses DNS pollution to allow the user's browser to access the domain name, it can become a reverse proxy, parse the user's request, and pretend to be a client to communicate with the real Web server, thereby obtaining the plaintext information of the communication between the two parties and achieving the purpose of the attack.

How do you prevent this situation? The defense method against man-in-the-middle attacks is to enable client authentication of certificates, but your certificate private key is lost again, what can you do? From what we have learned in the previous chapters of this article, in this case, you can only actively apply to the CA manufacturer for certificate revocation, no matter how many browsers support it. After all, the loss can be reduced as much as possible.

What to do if the certificate is leaked?

What should I do if the certificate is leaked? Judging from the browser support, it seems that even if I apply for certificate revocation in time, it cannot provide much protection against the loss of the certificate, as many browsers do not support it very much.

However, some losses can be avoided to some extent, after all, IE browser still has great support in this area.