Let's not talk about the principle, let's talk about the ready-to-use thread pool

This article is reproduced from the WeChat public account "JavaGuide". Author: Guide Brother. Please contact the JavaGuide public account for reprinting this article.

[[329762]]

Hello everyone, I am Guide, a technical person with a more positive outlook than the protagonist. Today, let's continue to talk about thread pools~

This article is about 5,000 words long and is absolutely informative. The title is a bit exaggerated, but it is actually a summary of some of the more important points I personally feel when using thread pools.

Thread pool knowledge review

Before starting this article, I will briefly introduce the thread pool. The previous article "Thread Pool Learning Summary that Newbies Can Understand" introduced it in detail.

Why use thread pool?

"Pooling technology is already common. Thread pools, database connection pools, Http connection pools, etc. are all applications of this idea. The idea of ​​pooling technology is mainly to reduce the consumption of each resource acquisition and improve the utilization of resources.

A thread pool provides a way to limit and manage resources (including executing a task). Each thread pool also maintains some basic statistics, such as the number of completed tasks.

Here I would like to borrow the "Art of Java Concurrency Programming" to talk about the benefits of using thread pools:

• Reduce resource consumption. Reduce the consumption caused by thread creation and destruction by reusing the created threads.
• Improve response speed. When a task arrives, it can be executed immediately without waiting for the thread to be created.
• Improve thread manageability. Threads are scarce resources. If they are created without limit, they will not only consume system resources but also reduce system stability. Using thread pools can be uniformly allocated, tuned, and monitored.

Thread pool usage scenarios in actual projects

Thread pools are generally used to execute multiple unrelated time-consuming tasks. Without multithreading, tasks are executed sequentially. Using thread pools allows multiple unrelated tasks to be executed simultaneously.

Suppose we want to perform three unrelated time-consuming tasks. Guide draws a picture to show you the difference before and after using the thread pool.

Note: The following three tasks may do the same thing or they may be different things.

Comparison before and after using thread pool

How to use thread pool?

Generally, a thread pool is created through the ThreadPoolExecutor constructor, and then tasks are submitted to the thread pool for execution.

The ThreadPoolExecutor constructor is as follows:

 /**
 * Create a new ThreadPoolExecutor with the given initial parameters.
 */
 public ThreadPoolExecutor( int corePoolSize, //Number of core threads in the thread pool
 int maximumPoolSize, //The maximum number of threads in the thread pool
 long keepAliveTime, //When the number of threads is greater than the number of core threads, the maximum time that the redundant idle threads survive
 TimeUnit unit, //Time unit
 BlockingQueue<Runnable> workQueue, //Task queue, used to store queues waiting for tasks to be executed
 ThreadFactory threadFactory, //Thread factory, used to create threads, usually the default
 RejectedExecutionHandler handler //Rejection strategy. When there are too many tasks submitted and they cannot be processed in time, we can customize the strategy to handle the tasks.
                             ) {
 if (corePoolSize < 0 ||
          maximumPoolSize <= 0 ||
          maximumPoolSize < corePoolSize ||
 keepAliveTime < 0)
          throw new IllegalArgumentException();
      if (workQueue == null || threadFactory == null || handler == null )
          throw new NullPointerException();
      this.corePoolSize = corePoolSize;
      this.maximumPoolSize = maximumPoolSize;
      this.workQueue = workQueue;
      this.keepAliveTime = unit.toNanos(keepAliveTime);
      this.threadFactory = threadFactory;
 this.handler = handler;
 }

This is a brief demonstration of how to use the thread pool. For a more detailed introduction, please see: "A summary of thread pool learning that even novices can understand."

 private static final int CORE_POOL_SIZE = 5;
 private static final int MAX_POOL_SIZE = 10;
 private static final int QUEUE_CAPACITY = 100;
 private static final Long KEEP_ALIVE_TIME = 1L; 
 
 public   static void main(String[] args) { 
 
 //Use the method recommended by Alibaba to create a thread pool
 //Create through ThreadPoolExecutor constructor custom parameters
     ThreadPoolExecutor executor = new ThreadPoolExecutor(
 CORE_POOL_SIZE,
 MAX_POOL_SIZE,
 KEEP_ALIVE_TIME,
             TimeUnit.SECONDS,
             new ArrayBlockingQueue<>(QUEUE_CAPACITY),
             new ThreadPoolExecutor.CallerRunsPolicy()); 
 
 for ( int i = 0; i < 10; i++) {
         executor. execute (() -> {
 try {
                 Thread.sleep(2000);
             } catch (InterruptedException e) {
                 e.printStackTrace();
 }
             System. out .println( "CurrentThread name:" + Thread.currentThread().getName() + "date:" + Instant.now());
 });
 }
 //Terminate the thread pool
 executor.shutdown();
 try {
         executor.awaitTermination(5, TimeUnit.SECONDS);
     } catch (InterruptedException e) {
 e.printStackTrace();
 }
     System. out .println( "Finished all threads" );
 }

Console output:

 CurrentThread name :pool-1-thread-5date：2020-06-06T11:45:31.639Z
 CurrentThread name :pool-1-thread-3date：2020-06-06T11:45:31.639Z
 CurrentThread name :pool-1-thread-1date：2020-06-06T11:45:31.636Z
 CurrentThread name :pool-1-thread-4date：2020-06-06T11:45:31.639Z
 CurrentThread name :pool-1-thread-2date：2020-06-06T11:45:31.639Z
 CurrentThread name :pool-1-thread-2date：2020-06-06T11:45:33.656Z
 CurrentThread name :pool-1-thread-4date：2020-06-06T11:45:33.656Z
 CurrentThread name :pool-1-thread-1date:2020-06-06T11:45:33.656Z
 CurrentThread name :pool-1-thread-3date：2020-06-06T11:45:33.656Z
 CurrentThread name :pool-1-thread-5date：2020-06-06T11:45:33.656Z
 Finished all threads

Thread Pool Best Practices

Let me briefly summarize what I know about the things that should be paid attention to when using thread pools. There seems to be no article specifically written on this topic on the Internet.

Because the Guide is still relatively new, if there are additions, improvements, or errors, you can let me know in the comment section or communicate with me on WeChat.

1. Declare the thread pool using the ThreadPoolExecutor constructor

The thread pool must be manually declared through the ThreadPoolExecutor constructor. Avoid using newFixedThreadPool and newCachedThreadPool of the Executors class because there may be a risk of OOM.

The drawbacks of Executors returning thread pool objects are as follows:

• FixedThreadPool and SingleThreadExecutor: The queue length of requests is allowed to be Integer.MAX_VALUE, which may accumulate a large number of requests and cause OOM.
• CachedThreadPool and ScheduledThreadPool: The number of threads allowed to be created is Integer.MAX_VALUE. A large number of threads may be created, resulting in OOM.

To put it simply: use bounded queues to control the number of threads created.

In addition to avoiding OOM, the reasons why the two quick thread pools provided by Executors are not recommended are:

• In actual use, you need to manually configure the thread pool parameters such as the number of core threads, the task queue used, the saturation strategy, etc. according to the performance of your machine and business scenarios.
• We should explicitly name our thread pool, which will help us locate problems.

2. Monitor the thread pool running status

You can use some means to detect the running status of the thread pool, such as the Actuator component in SpringBoot.

In addition, we can also use the relevant API of ThreadPoolExecutor to do a simple monitoring. As can be seen from the figure below, ThreadPoolExecutor provides the ability to obtain the current number of threads and active threads in the thread pool, the number of completed tasks, the number of tasks in queue, and so on.

Below is a simple demo. printThreadPoolStatus() will print out the number of threads in the thread pool, the number of active threads, the number of completed tasks, and the number of tasks in the queue every second.

 /**
 * Print thread pool status
 *
 * @param threadPool thread pool object
 */
 public   static void printThreadPoolStatus(ThreadPoolExecutor threadPool) {
      ScheduledExecutorService scheduledExecutorService = new ScheduledThreadPoolExecutor(1, createThreadFactory( "print-thread-pool-status" , false ));
      scheduledExecutorService.scheduleAtFixedRate(() -> {
          log.info( "=========================" );
          log.info( "ThreadPool Size: [{}]" , threadPool.getPoolSize());
          log.info( "Active Threads: {}" , threadPool.getActiveCount());
          log.info( "Number of Tasks : {}" , threadPool.getCompletedTaskCount());
          log.info( "Number of Tasks in Queue: {}" , threadPool.getQueue(). size ());
          log.info( "=========================" );
      }, 0, 1, TimeUnit.SECONDS);
 }

3. It is recommended to use different thread pools for different types of business

Many people have similar questions in actual projects: In my project, multiple businesses need to use thread pools. Should I define one thread pool for each or a common thread pool?

It is generally recommended that different businesses use different thread pools. When configuring the thread pool, configure the current thread pool according to the current business situation. Because the concurrency and resource usage of different businesses are different, focus on optimizing businesses related to system performance bottlenecks.

Let's look at a real accident case! (This case comes from: "An online accident caused by improper use of thread pool" @https://club.perfma.com/article/646639, a very exciting case)

Example Code Overview

The above code may cause deadlock. Why? Let me draw a picture to explain it to you.

Consider this extreme situation:

Suppose the number of core threads in our thread pool is n, the number of parent tasks (charge-collection tasks) is n, and there are two child tasks (child tasks under charge-collection tasks) under the parent task, one of which has been completed, and the other has been placed in the task queue. Since the parent task has used up the core thread resources of the thread pool, the child task cannot be executed normally because it cannot obtain thread resources and has been blocked in the queue. The parent task waits for the child task to complete execution, and the child task waits for the parent task to release thread pool resources, which causes a "deadlock".

The solution is also very simple, which is to add a new thread pool for executing subtasks to serve it specifically.

4. Don’t forget to name the thread pool

When initializing the thread pool, you need to display the name (set the thread pool name prefix) to help locate the problem.

By default, the thread name created is similar to pool-1-thread-n, which has no business meaning and is not conducive to locating problems.

There are usually two ways to name threads in the thread pool:

1). Using guava's ThreadFactoryBuilder

 ThreadFactory threadFactory = new ThreadFactoryBuilder()
                        .setNameFormat(threadNamePrefix + "-%d" )
                        .setDaemon( true ).build();
 ExecutorService threadPool = new ThreadPoolExecutor(corePoolSize, maximumPoolSize, keepAliveTime, TimeUnit.MINUTES, workQueue, threadFactory)

2). Implement ThreadFactor yourself.

 import java.util.concurrent.Executors;
 import java.util.concurrent.ThreadFactory;
 import java.util.concurrent.atomic.AtomicInteger;
 /**
 * The thread factory sets the thread name, which helps us locate the problem.
 */
 public final class NamingThreadFactory implements ThreadFactory { 
 
    private final AtomicInteger threadNum = new AtomicInteger();
    private final ThreadFactory delegate;
    private final String name ; 
 
 /**
 * Create a named thread pool production factory
 */
 public NamingThreadFactory(ThreadFactory delegate, String name ) {
        this.delegate = delegate;
        this. name = name ; // TODO consider uniquifying this
 } 
 
 @Override
 public Thread newThread(Runnable r) {
        Thread t = delegate.newThread(r);
        t.setName( name + " [#" + threadNum.incrementAndGet() + "]" );
 return t;
 } 
 
 }

5. Correctly configure thread pool parameters

When it comes to how to configure parameters for the thread pool, Meituan’s operation is still unforgettable to me (I will mention it later)!

Let's first take a look at the generally recommended ways to configure thread pool parameters in various books and blogs, which can be used as a reference!

General Operation

Many people may even think that it is better to configure the thread pool too large! I think this is obviously problematic. Take a very common example in our lives: more people does not mean that things can be done well, which increases the cost of communication. You only need 3 people to do a thing, but you force 6 people to do it. Will it improve the efficiency? I don't think so. The impact of too many threads is the same as how many people we assign to do things. For the multi-threaded scenario, it mainly increases the cost of context switching. If you don't know what context switching is, you can see my introduction below.

"Context switch: In multithreaded programming, the number of threads is generally greater than the number of CPU cores, and a CPU core can only be used by one thread at any time. In order to allow all these threads to be executed effectively, the CPU adopts a strategy of allocating time slices to each thread and rotating them. When a thread's time slice is used up, it will be ready again for other threads to use. This process is a context switch. In summary, the current task will save its own state before switching to another task after executing the CPU time slice, so that the state of this task can be loaded again when it switches back to this task next time. The process from saving to reloading a task is a context switch. Context switching is usually computationally intensive. That is, it requires considerable processor time. In the dozens or hundreds of switches per second, each switch takes nanoseconds. Therefore, context switching means a lot of CPU time consumption for the system. In fact, it may be the most time-consuming operation in the operating system. Linux has many advantages over other operating systems (including other Unix-like systems), one of which is that its context switching and mode switching consume very little time.

Analogous to the realization that humans in the world do something through cooperation, one thing we can be sure of is that setting the thread pool size too large or too small will cause problems, and the appropriate size is the best.

If the number of thread pools we set is too small, if there are a large number of tasks/requests to be processed at the same time, it may cause a large number of requests/tasks to queue up in the task queue waiting for execution, or even the situation where the task queue is full and the tasks/requests cannot be processed, or a large number of tasks are piled up in the task queue, causing OOM. This is obviously problematic! The CPU is not fully utilized at all.

However, if we set the number of threads to be too large, a large number of threads may compete for CPU resources at the same time, which will cause a large number of context switches, thereby increasing the execution time of the threads and affecting the overall execution efficiency.

There is a simple and widely applicable formula:

• CPU-intensive tasks (N+1): This type of task mainly consumes CPU resources. The number of threads can be set to N (the number of CPU cores) + 1. The extra thread is to prevent occasional page faults or other reasons that cause task suspension. Once the task is suspended, the CPU will be idle, and in this case, the extra thread can make full use of the CPU's idle time.
• I/O intensive tasks (2N): When this type of task is applied, the system will spend most of its time processing I/O interactions, and the thread will not occupy the CPU to process during the period of time when processing I/O, so the CPU can be handed over to other threads. Therefore, in the application of I/O intensive tasks, we can configure more threads, and the specific calculation method is 2N.

How to determine whether it is a CPU-bound task or an IO-bound task?

CPU intensive tasks are tasks that use CPU computing power, such as sorting a large amount of data in memory. Anything involving network reading or file reading is IO intensive. The characteristic of such tasks is that the CPU computing time is very small compared to the time waiting for the IO operation to complete, and most of the time is spent waiting for the IO operation to complete.

Meituan’s tricks

The Meituan technical team introduced the ideas and methods for implementing customizable configuration of thread pool parameters in the article "Java Thread Pool Implementation Principles and Its Practice in Meituan Business".

The idea of ​​the Meituan technical team is to make the core parameters of the thread pool customizable. These three core parameters are:

• corePoolSize : The number of core threads defines the minimum number of threads that can run simultaneously.
• maximumPoolSize: When the tasks stored in the queue reach the queue capacity, the number of threads that can currently run simultaneously becomes the maximum number of threads.
• workQueue: When a new task comes, it will first determine whether the number of currently running threads reaches the number of core threads. If so, the trust will be stored in the queue.

Why these three parameters?

I have said in this article "Thread Pool Learning Summary That Newbies Can Understand" that these three parameters are the most important parameters of ThreadPoolExecutor, and they basically determine the thread pool's task processing strategy.

How to support dynamic configuration of parameters? Let's look at the following methods provided by ThreadPoolExecutor.

Special attention should be paid to corePoolSize. When the program is running, if we call the setCorePoolSize() method, the thread pool will first determine whether the current number of working threads is greater than corePoolSize. If so, the working threads will be recycled.

In addition, you can see that there is no method to dynamically specify the queue length above. Meituan’s method is to customize a queue called ResizableCapacityLinkedBlockIngQueue (mainly by removing the final keyword modification of the capacity field of LinkedBlockingQueue to make it variable).

The final effect of dynamically modifying thread pool parameters is as follows. 👏👏👏

Final effect of dynamically configuring thread pool parameters