What exactly is the cache technology that supports high concurrency?

Author | Jin Peng, Unit: China Mobile Smart Home Operation Center

Labs Guide

Cache refers to a storage device that can exchange data at high speed. It exchanges data with the CPU before the main memory does, so the speed is very fast. So what is the operating logic of cache and what are the cache solutions? This article will focus on this.

Cache is a technology that stores target data in a high-speed operation layer with high accessibility. It aims to quickly read data that is highly used and infrequently updated, avoiding repeated access to the source data layer for requests before access, resulting in additional I/O resource consumption, and then improving the concurrency of the business. In order to increase the operating rate of data access, cached data must also weigh resource capacity and data refresh frequency to set expiration time or set a reasonable cache elimination strategy. Mainstream cache solutions include CDN cache, browser cache, local cache, external cache, etc.

Part 01 CDN Cache

CDN refers to the content distribution network, which caches resources on CDN nodes. It relies on edge servers deployed in various places, and through the load balancing, content distribution, scheduling and other functional modules of the central platform, users can obtain content from the nearest CDN cache node, reduce network latency and increase access speed. Business scenarios that use this caching technology include: static files such as HTML, CSS, JS in front-end projects, downloading firmware packages in the Internet of Things, resource requests in multi-site active architectures, etc. Deploying or caching these files on CDN nodes can improve resource access speed while ensuring the stability of resource access.

Part 02 Browser Cache

Browser cache refers to the process of implementing cache based on the cache rules in the response message after the browser sends a request to the server for the first time and gets the result when communicating with the backend server through the request-response mode under the HTTP protocol. For example, under the HTTP/1.1 protocol, the mandatory cache rule will be implemented based on Cache-Control; when the mandatory cache fails, the cache rule will be determined based on the cache identifier in the request. Browser session persistence is also a cache implementation. The current mainstream technical solutions include Cookie mechanism, Session mechanism, Token mechanism, etc.

Part 03 Local Cache

Local caching means caching the response data in the memory space of the server's application process to reduce the I/O consumption of the query persistence layer. The advantage is that the data does not need to be transmitted across the network, the reading performance is better, and the supported QPS is higher; the disadvantage is that it will occupy the memory space of the application process, so large amounts of data cannot be stored, and when the application process is restarted, the locally cached data will be lost. Caffeine, a local caching solution implemented in the Andlink cloud platform project, has the following dependencies:

 <dependency>
 <groupId>com.github.ben-manes.caffeine</groupId>
 <artifactId>caffeine</artifactId>
 <version>${caffeine.version}</version>
 </dependency> 

Part 04 External Cache or Distributed Cache

Corresponding to the local cache is the external cache or distributed cache, which is independent of the application process deployment and can be designed in cluster mode to ensure data read and write separation, high performance, and high availability. The distributed cache implemented in the Andlink cloud platform project is a solution based on the Spring Boot Cache framework integrating the Redis cluster. This solution can effectively solve common business pain points such as data loss, business concurrency, fault recovery, and dynamic storage expansion. Combined with business needs, Redis provides a relatively complete response plan, such as stand-alone mode, master-slave mode, cluster mode, or sentinel mode. Among them, the sentinel mode can realize health detection and automatic recovery, and is the preferred mode to ensure the stable operation of high-concurrency businesses. The dependency packages of the solution are as follows:

 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-data-redis</artifactId>
 <version>${redis.version}</version>
 </dependency>
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-cache</artifactId>
 <version>${cache.version}</version>
 </dependency> 

Of course, there are many mainstream caching technology solutions, but the general direction is to reduce the underlying performance overhead and improve the high concurrency capability of the business. Currently, the Andlink cloud platform supports hundreds of millions of users and tens of millions of device requests, and caching technology plays a vital role in this. ​