Illustration | You call this a thread pool?

[[375802]]

This article is reprinted from the WeChat public account "Low Concurrency Programming", the author is Shan Kesun. Please contact the WeChat public account "Low Concurrency Programming" to reprint this article.

Xiaoyu: Shanke, I recently saw the thread pool and was confused by the various parameters in it. Can you tell me more about it?

Shan Ke: No problem, I am good at this. Let's start with the simplest case. Suppose there is a piece of code that you want to execute asynchronously. Do you need to write the following code?

 new Thread(r).start();

Xiaoyu: Yeah, that seems to be the simplest way to write it.

Flasher: This way of writing can certainly complete the function, but you write it this way, Lao Wang writes it this way, Lao Zhang also writes it this way, and the program is full of methods of creating threads in this way. Can you write a unified tool class for everyone to call?

Xiaoyu: Yes, it is possible. I feel that having a unified tool class is more elegant.

Flasher: If you were asked to design this tool class, how would you write it? I will first give you an interface and you implement it.

 public interface Executor {
 public void execute (Runnable r);
 }

Xiaoyu: Emmm, I might define a few member variables first, such as the number of core threads, the maximum number of threads... anyway, those messy parameters.

Shanke: STOP! Xiaoyu, you are deeply poisoned by the interview manual. Forget all these concepts first. Suppose I ask you to write a simplest tool class. What would you write first?

First Edition

Xiaoyu: Then I might do this

 // New thread: directly create a new thread to run
 class FlashExecutor implements Executor {
 public void execute (Runnable r) {
        new Thread(r).start();
 }
 }

Flash: Yeah, very good, your idea is very good.

Xiaoyu: Ah, isn’t this too low? I thought you would scold me.

How could it be? Doug Lea gave such an example in the JDK source code comments. This is the most basic function. Based on this, do you try to optimize it?

Second Edition

Xiaoyu: How else can we optimize it? Isn't this already achieved with a tool class?

Shanke: Let me ask you a question. If 10,000 people call this tool class to submit tasks, 10,000 threads will be created to execute them. This is definitely not appropriate! Can the number of threads be controlled?

Xiaoyu: This is not difficult. I can put this task r into a tasks queue, and then start only one thread, let's call it the Worker thread, which continuously takes tasks from the tasks queue and executes them. In this way, no matter how many times the caller calls it, there will always be only one Worker thread running, like this.

Shan Ke: Great! This design has three important meanings:

1. Control the number of threads.

2. The queue not only acts as a buffer, but also decouples task submission and execution.

3. The most important point is that it solves the system overhead caused by repeated creation and destruction of threads.

Xiaoyu: Wow, really? Such a small change has so much meaning.

Shanke: Of course, but isn't it a bit too little to have only one background worker thread? And what if the tasks queue is full?

Third Edition

Xiaoyu: Oh, indeed, having only one thread is very difficult in some scenarios. Should I increase the number of Worker threads?

Flasher: That's right, the number of Worker threads needs to be increased, but the specific number should be decided by the user and passed in when calling, let's call it core thread number corePoolSize.

Xiaoyu: Okay, then I’ll design it like this.

1. When initializing the thread pool, directly start corePoolSize worker threads to run first.

2. These workers are infinite loops that take tasks from the queue and execute them.

3. The execute method still puts the task directly into the queue, but discards it directly when the queue is full

Flash: That's perfect. I'll give you a Ferrero Rocher as a reward.

Xiaoyu: Haha, thank you. I’ll eat for a while then.

Flash: Okay, you eat while I talk. Now we have implemented a thread pool that is at least not so ugly, but there are still a few minor flaws. For example, during initialization, a bunch of Worker threads are created and run in vain. If no asynchronous tasks are submitted for execution at this time, this is a bit of a waste.

Xiaoyu: Oh, it seems so!

Shanke: Also, once the queue is full, you just discard the new task. This is a bit rough. Can you let the caller decide how to deal with it?

Xiaoyu: Oh, I never thought that such a gentle girl like me could write such crude code.

Flash: Um, you should swallow the Ferrero first.

Fourth Edition

Xiaoyu: I have finished eating. My brain is a bit slow now. I need to digest the food first. How about you help me analyze it?

Flash: OK, now we make the following improvements.

1. Create Workers on demand: When the thread pool is initialized, corePoolSize worker threads are no longer created immediately. Instead, worker threads are gradually created as callers continue to submit tasks. When the number reaches corePoolSize, the worker threads are stopped and tasks are directly thrown into the queue. Therefore, an attribute must be used to record the number of worker threads that have been created, let's call it workCount.

2. Add rejection strategy: The implementation is to add an input parameter, the type is an interface RejectedExecutionHandler, and the caller determines the implementation class so that the rejectedExecution method is executed after the task submission fails.

3. Adding a thread factory: The implementation is to add an input parameter, the type is an interface ThreadFactory. When adding a worker thread, no longer directly new thread, but call the newThread method of the ThreadFactory implementation class passed in by the caller.

Like below.

Xiaoyu: Wow, you are really awesome. This version should be perfect, right?

Shan Ke: No, no, no, it is still far from perfect. You can think of the next improvements. I can give you a hint.

Flexible thinking

Fifth Edition

Xiaoyu: Flexible thinking? Haha, Shanke, your terminology is becoming less and less human.

Flash: Ahem

Xiaoyu: Oh, I mean you must mean that my code is not flexible, right? But what does flexibility mean?

Shan Ke: Simply put, in this scenario, elasticity refers to whether there is a problem with your code in the two situations where tasks are submitted frequently or very infrequently.

Xiaoyu: Emmm, let me think about it. When the number of submitted tasks increases suddenly, the worker threads and queues are full. I can only use the rejection strategy, which is actually discarded.

Flash: Yes

Xiaoyu: This is indeed too hard. But I thought that the caller can solve this problem by setting a large number of core threads, corePoolSize.

Shanke: It is indeed possible, but in general scenarios, the QPS peak period is very short. To meet this short peak, setting a large number of core threads is a waste of resources. Don't you feel dizzy when you look at the picture above?

Xiaoyu: Yes, then what should we do? It can’t be too big, and it can’t be too small.

Shanke: We can invent a new property called maximumPoolSize. When the core thread number and queue are full, new submitted tasks can still create new worker threads (called non-core threads) until the number of worker threads reaches maximumPoolSize. This can alleviate the peak period temporarily, and users don't need to set too large a number of core threads.

Xiaoyu: Oh, I seem to have some feeling about it, but how to operate it specifically?

Flasher: Xiaoyu, your imagination is not good enough. Then take a look at the demonstration below.

1. At the beginning, it is the same as the previous version. When workCount < corePoolSize, a new Worker is created to execute the task.

2. When workCount >= corePoolSize, stop creating new threads and throw the tasks directly into the queue.

3. However, when the queue is full and workCount < maximumPoolSize, the rejection strategy is no longer used directly. Instead, a non-core thread is created until workCount = maximumPoolSize, and then the rejection strategy is used.

Xiaoyu: Oh, why didn’t I think of this? In this way, corePoolSize is responsible for the number of working threads required in most cases, while maximumPoolSize is responsible for temporarily expanding the number of working threads during peak periods.

Shanke: That's right. Now that we have solved the problem of elasticity during peak hours, we naturally need to consider the trough hours. When there is no task submission for a long time, both the core and non-core threads will be running idle, wasting resources. We can set a timeout keepAliveTime for non-core threads. When no task is obtained from the queue for such a long time, the threads will no longer wait and will be destroyed.

Xiaoyu: Well, this time our thread pool can be temporarily expanded when the QPS peaks, and threads (non-core threads) can be recycled in time when the QPS is low without wasting resources. It is really very flexible.

Shan Ke: Yes, yes. Oh, that's not right. Why did I say it again? Didn't I say that you should think about this version?

Xiaoyu: I want to, too. But you always talk to yourself when you talk about technology, so what can I do?

Shan Ke: Sorry, sorry. Next, can you summarize our thread pool?

Summarize

Xiaoyu: Okay, first of all, its construction method is like this

 public FlashExecutor(
 int corePoolSize,
 int maximumPoolSize,
 long keepAliveTime,
 TimeUnit unit,
        BlockingQueue<Runnable> workQueue,
        ThreadFactory threadFactory,
        RejectedExecutionHandler handler)
 {
 ... // Omit some parameter checks
    this.corePoolSize = corePoolSize;
    this.maximumPoolSize = maximumPoolSize;
    this.workQueue = workQueue;
    this.keepAliveTime = unit.toNanos(keepAliveTime);
    this.threadFactory = threadFactory;
 this.handler = handler;
 }

These parameters are

int corePoolSize: number of core threads

int maximumPoolSize: maximum number of threads

long keepAliveTime: Idle time of non-core threads

TimeUnit unit: the unit of idle time

BlockingQueue workQueue: task queue (thread-safe blocking queue)

ThreadFactory threadFactory: thread factory

RejectedExecutionHandler handler: rejection strategy

The entire task submission process is

Shan Ke: Not bad, not bad. You summarized it yourself. Do I need to explain to you what a thread pool is?

Xiaoyu: Oh my god, I just realized that this seems to be the parameters and principles of the thread pool that I have been unable to figure out!

Shanke: That’s right, and the constructor of the last version of the code is the longest constructor of ThreadPoolExecutor, which is often tested in Java interviews, and the parameter names have not changed.

Xiaoyu: Wow, that’s awesome! I almost forgot what I wanted to do at the beginning, hehe.

Shan Ke: Haha, it feels good to have learned some skills without realizing it, right? It’s almost time for dinner, do you want to go to a Shanxi noodle restaurant together?

Xiaoyu: Oh, I don’t like the color of the tables in that restaurant. Let’s try it next time.

Flash: Oh okay.

postscript

Thread pool is a knowledge point that is often tested in interviews. Many articles on the Internet start directly from its construction method with 7 parameters, and force the meaning of each parameter to be explained to you, leaving people confused.

I hope that after reading this article, these parameters of the thread pool will no longer be memorized by rote, but will come alive in your mind like these animated pictures in this article, so that you can remember them forever~