Why does the HTTP request return 304?

[[402402]]

I believe most web developers are familiar with the HTTP 304 status code. In this article, I will introduce how the HTTP 304 status code and the fresh function in the fresh module implement resource freshness detection based on the Koa cache example. If you don't know the browser's cache mechanism, it is recommended that you read the article "In-depth understanding of the browser's cache mechanism" first.

1. 304 Status Code

How is ETag generated in HTTP? This article introduces how ETag is generated. In the ETag practical section, Abaoge demonstrated how to use the ETag response header and If-None-Match request header to implement resource cache control in actual projects based on libraries such as koa, koa-conditional-get, koa-etag, and koa-static.

 // server.js
 const Koa = require( "koa" );
 const path = require( "path" );
 const serve = require( "koa-static" );
 const etag = require( "koa-etag" );
 const conditional = require( "koa-conditional-get" ); 
 
 const app = new Koa(); 
 
 app.use(conditional()); // Use conditional request middleware
 app.use(etag()); // Use etag middleware
 app.use( // Use static resource middleware
  serve( path.join (__dirname, "/public" ), {
 maxage: 10 * 1000, // Set the maximum period of cache storage in seconds
 });
 ); 
 
 app.listen(3000, () => {
  console.log( "app starting at port 3000" );
 });

After starting the server, we open the Chrome developer tools and switch to the Network tab, then enter the http://localhost:3000/ address in the browser address bar, and then visit the address multiple times (press Enter multiple times in the address bar).

The above figure shows the result of Abaoge's multiple visits. In the figure, we can see the 200 and 304 status codes. The 304 status code indicates that the resource has not been modified since the version specified by the If-Modified-Since or If-None-Match parameter in the request header. In this case, the client still has a previously downloaded copy, so there is no need to retransmit the resource.

Let's take the index.js resource as an example to take a closer look at the 304 response message:

 HTTP/1.1 304 Not Modified
 Last -Modified: Sat, 29 May 2021 02:24:53 GMT
 Cache-Control: max -age=10
 ETag: W/ "29-179b5f04654"  
 Date : Sat, 29 May 2021 02:25:26 GMT
 Connection : keep-alive

For the above response message, the response header contains Last-Modified, Cache-Control and ETag, which are cache-related fields. If you are not familiar with the functions of these fields, you can read the two articles: In-depth understanding of browser cache mechanism and How is ETag generated in HTTP? Next, Abao will explore with you why the request for index.js resource returns 304 after 10 seconds?

2. Why return 304 status code?

In the previous example, we registered three middlewares by using the app.use method:

 app.use(conditional()); // Use conditional request middleware
 app.use(etag()); // Use etag middleware
 app.use( // Use static resource middleware
  serve( path.join (__dirname, "/public" ), {
 maxage: 10 * 1000, // Set the maximum period of cache storage in seconds
 })
 );

The first thing to register is the koa-conditional-get middleware, which is used to handle HTTP conditional requests. In this type of request, the result of the request, and even the status of the request success, will change with the comparison result of the validator and the affected resource. HTTP conditional requests can be used to verify the validity of the cache and eliminate unnecessary control measures.

In fact, the implementation of koa-conditional-get middleware is very simple, as shown below:

 // https://github.com/koajs/conditional-get/blob/master/ index .js
 module.exports = function conditional () {
 return async function (ctx, next ) {
 await next ()
 if (ctx.fresh) {
 ctx.status = 304
 ctx.body = null  
 }
 }
 }

From the above code, we can see that when the fresh property of the request context object is true, the response status code will be set to 304. Therefore, our next focus is to analyze the setting conditions of the ctx.fresh value.

By reading the source code of the koa/lib/context.js file, we know that when accessing the fresh property of the context object, we are actually accessing the fresh property of the request object.

 // Proxy request object
 delegate(proto, 'request' )
 // Omit other agents
 .getter( 'fresh' )
 .getter( 'ips' )
 .getter( 'ip' );

The fresh property on the request object is defined using the getter method, as shown below:

 // node_modules/koa/lib/request.js
 module.exports = {
 // Omit some code
 get fresh() {
 const method = this.method; // Get request method
 const s = this.ctx.status; // Get status code 
 
    if ( 'GET' !== method && 'HEAD' !== method) return   false ; 
 
    // 2xx or 304 as per rfc2616 14.26
    if ((s >= 200 && s < 300) || 304 === s) {
 return fresh(this.header, this.response.header);
 }
 return   false ;
 },
 }

method && 'HEAD' !== method) return false; // 2xx or 304 as per rfc2616 14.26 if ((s >= 200 && s < 300) || 304 === s) { return fresh(this.header, this.response.header); } return false; },}

In the fresh method, freshness detection is performed only when the request is a GET/HEAD request and the status code is 2xx or 304. The corresponding freshness detection logic is encapsulated in the fresh module, so let's analyze how this module detects freshness?

3. How to detect freshness

The fresh module provides a fresh function that supports two parameters: reqHeaders and resHeaders. Within this function, the logic of freshness detection can be divided into the following four parts:

3.1 Determine whether it is a conditional request

 // https://github.com/jshttp/fresh/blob/master/ index .js
 function fresh (reqHeaders, resHeaders) {
  var modifiedSince = reqHeaders[ 'if-modified-since' ]
  var noneMatch = reqHeaders[ 'if-none-match' ] 
 
 // Non-conditional request
  if (!modifiedSince && !noneMatch) {
 return   false  
 }
 }

If the request header does not contain the if-modified-since and if-none-match fields, false is returned directly.

3.2 Determine the cache-control request header

 // https://github.com/jshttp/fresh/blob/master/ index .js
 var CACHE_CONTROL_NO_CACHE_REGEXP = /(?:^|,)\s*? no -cache\s*?(?:,|$)/ 
 
 function fresh (reqHeaders, resHeaders) {
  var modifiedSince = reqHeaders[ 'if-modified-since' ]
  var noneMatch = reqHeaders[ 'if-none-match' ] 
   
  // Always return stale when Cache-Control: no -cache
  // to support end - to - end reload requests
  // https://tools.ietf.org/html/rfc2616# section -14.9.4
  var cacheControl = reqHeaders[ 'cache-control' ]
  if (cacheControl && CACHE_CONTROL_NO_CACHE_REGEXP.test(cacheControl)) {
 return   false  
 }
 }

When the value of the cache-control request header is no-cache, false is returned to support end-to-end reload requests. It should be noted that no-cache does not mean no caching, but means that the resource is cached, but immediately invalidated, and a request will be made next time to verify whether the resource is expired. If you do not cache any responses, you need to set the cache-control value to no-store.

3.3 Check if ETag matches

 // https://github.com/jshttp/fresh/blob/master/ index .js
 function fresh (reqHeaders, resHeaders) {
  var modifiedSince = reqHeaders[ 'if-modified-since' ]
  var noneMatch = reqHeaders[ 'if-none-match' ] 
   
 // Omit some code
  if (noneMatch && noneMatch !== '*' ) {
 var etag = resHeaders[ 'etag' ] // Get the value of the etag field in the response header 
 
 if (!etag) { // If the response header does not set etag, return false directly  
 return   false  
 } 
 
 var etagStale = true // stale: not fresh
    var matches = parseTokenList(noneMatch) // Parse noneMatch
 for (var i = 0; i < matches.length; i++) { // Execute loop matching operation
 var match = matches[i]
      if (match === etag || match === 'W/' + etag || 'W/' + match === etag) {
 etagStale = false  
 break
 }
 } 
 
 if (etagStale) {
 return   false  
 }
 }
 return   true  
 }

In the above code, the parseTokenList function is used to handle the situation of 'if-none-match': ' "bar" , "foo"'. During the parsing process, extra spaces will be removed, and the etag values ​​separated by commas will be split. The purpose of performing loop matching is also to support the following test cases:

 // https://github.com/jshttp/fresh/blob/master/test/fresh.js
 describe( 'when at least one matches' , function () {
  it( 'should be fresh' , function () {
    var reqHeaders = { 'if-none-match' : ' "bar" , "foo"' }
    var resHeaders = { 'etag' : '"foo"' }
    assert.ok(fresh(reqHeaders, resHeaders))
 })
 })

In addition, the W/ (case-sensitive) in the above code indicates the use of a weak validator. Weak validators are easy to generate, but difficult to compare. If the etag does not contain W/, it indicates a strong validator, which is a more ideal choice, but difficult to generate effectively. Two weak etag values ​​for the same resource may be semantically equivalent, but not byte-for-byte identical.

3.4 Determine whether Last-Modified has expired

 // https://github.com/jshttp/fresh/blob/master/ index .js
 function fresh (reqHeaders, resHeaders) {
 var modifiedSince = reqHeaders[ 'if-modified-since' ] // Get the modification time in the request header
  var noneMatch = reqHeaders[ 'if-none-match' ] 
 
 // if-modified-since
 if (modifiedSince) {
 var lastModified = resHeaders[ 'last-modified' ] // Get the modification time in the response header
    var modifiedStale = !lastModified || !(parseHttpDate(lastModified) <= parseHttpDate(modifiedSince)) 
 
 if (modifiedStale) {
 return   false  
 }
 } 
 
 return   true  
 }

The judgment logic of Last-Modified is very simple. When the response header does not set the last-modified field information or the last-modified value in the response header is greater than the modification time corresponding to the if-modified-since field in the request header, the freshness detection result is false, which means that the resource has been modified and is no longer fresh.

After understanding the specific implementation of the fresh function, let's review the difference between Last-Modified and ETag:

• In terms of accuracy, Etag is better than Last-Modified. The time unit of Last-Modified is seconds. If a file is changed multiple times within 1 second, its Last-Modified will not reflect the change, but Etag will change every time, thus ensuring accuracy; in addition, if it is a load-balanced server, the Last-Modified generated by each server may also be inconsistent.
• In terms of performance, Etag is inferior to Last-Modified. After all, Last-Modified only needs to record time, while ETag requires the server to calculate a hash value through a message digest algorithm.
• In terms of priority, when checking the freshness of a resource, the server will give priority to Etag. That is, if the request header of a conditional request carries both the If-Modified-Since and If-None-Match fields, it will first determine whether the ETag value of the resource has changed.

I believe you have a general understanding of why the index.js resource request in the example returns 304. If you are interested in how the koa-etag middleware generates ETag, you can read the article How is ETag generated in HTTP?

4. Cache Mechanism

Strong cache takes precedence over negotiated cache. If strong cache (Expires and Cache-Control) is in effect, the cache is used directly. If it is not in effect, negotiated cache (Last-Modified/If-Modified-Since and Etag/If-None-Match) is performed. The server decides whether to use the negotiated cache. If the negotiated cache fails, the cache of the request is invalid, and 200 is returned. The resource and cache identifier are returned again and stored in the browser cache. If it is in effect, 304 is returned and the cache continues to be used.

The specific caching mechanism is shown in the following figure:

In order to help you better understand the cache mechanism, let's briefly analyze the Koa cache example introduced earlier:

 // server.js
 const Koa = require( "koa" );
 const path = require( "path" );
 const serve = require( "koa-static" );
 const etag = require( "koa-etag" );
 const conditional = require( "koa-conditional-get" ); 
 
 const app = new Koa(); 
 
 app.use(conditional()); // Use conditional request middleware
 app.use(etag()); // Use etag middleware
 app.use( // Use static resource middleware
  serve( path.join (__dirname, "/public" ), {
 maxage: 10 * 1000, // Set the maximum period of cache storage in seconds
 });
 ); 
 
 app.listen(3000, () => {
  console.log( "app starting at port 3000" );
 });

The above example uses koa-conditional-get, koa-etag and koa-static middleware. Their specific definitions are as follows:

4.1 koa-conditional-get

 // https://github.com/koajs/conditional-get/blob/master/ index .js
 module.exports = function conditional () {
 return async function (ctx, next ) {
 await next ()
 if (ctx.fresh) { // If the resource is not updated, return 304 Not Modified
 ctx.status = 304
 ctx.body = null  
 }
 }
 }

The implementation of koa-conditional-get middleware is very simple. If the resource is fresh, it directly returns a 304 status code and sets the response body to null.

4.2 koa-etag

 // https://github.com/koajs/etag/blob/master/ index .js
 module.exports = function etag (options) {
 return async function etag (ctx, next ) {
 await next ()
 const entity = await getResponseEntity(ctx) // Get the response entity object
    setEtag(ctx, entity, options)
 }
 }

Inside the koa-etag middleware, after getting the response entity object, the setEtag function is called to set the ETag. The definition of the setEtag function is as follows:

 // https://github.com/koajs/etag/blob/master/ index .js
 const calculate = require( 'etag' ) 
 
 function setEtag (ctx, entity, options) {
 if (!entity) return  
  ctx.response.etag = calculate(entity, options)
 }

Obviously, the koa-etag middleware uses the etag library to generate the corresponding etag for the response entity.

4.3 koa-static

 // https://github.com/koajs/ static /blob/master/ index .js
 function serve (root, opts) {
  opts = Object.assign(Object. create ( null ), opts)
 // Omit some code
 return async function serve (ctx, next ) {
 await next ()
    if (ctx.method !== 'HEAD' && ctx.method !== 'GET' ) return  
    // response is already handled
    if (ctx.body != null || ctx.status !== 404) return  
 try {
      await send(ctx, ctx.path, opts)
 } catch (err) {
      if (err.status !== 404) {
 throw err
 }
 }
 }
 }

For the koa-static middleware, when the request method is not a GET or HEAD request (which should not contain a response body), it will be returned directly. The processing capability of static resources is actually implemented by the send library.

Finally, in order to help you better understand the processing logic of the above middleware, Brother Abao will take you to briefly review the onion model:

In the above figure, each layer in the onion represents an independent middleware, which is used to implement different functions, such as exception handling, cache processing, etc. Each request will pass through each layer of middleware from the left layer, and when it enters the innermost layer of middleware, it will return layer by layer from the innermost layer of middleware. Therefore, for each layer of middleware, there are two opportunities to add different processing logic in a request and response cycle.

V. Conclusion

In this article, based on Koa's cache example, Abao Ge introduces how the HTTP 304 status code and the fresh function in the fresh module implement resource freshness detection. I hope that after reading this article, you will have a deeper understanding of HTTP and browser cache mechanisms. In addition, this article only briefly introduces Koa's onion model. If you are interested in the onion model, you can continue to read the article How to Better Understand Middleware and Onion Model.

6. Reference Resources

• MDN - HTTP Conditional Requests
• How is ETag generated in HTTP?