Using Threads with Networking
A thread pool is a group of threads that can execute multiple tasks, with each task assigned to any idle thread in the pool rather than to a newly created thread.
Hundreds of Small Jobs and a Costly Habit
Imagine a program that must perform hundreds of small computations, such as processing hundreds of pieces of an image. A tempting design is to start a brand-new thread for every piece of work, let each thread do its one job, and then let it finish. That design is easy to picture, but it runs into a practical cost: creating a thread carries a significant amount of overhead, and paying that cost again and again for a large number of small jobs wastes effort.
This article introduces the thread pool, a structure built around the opposite idea. Instead of making one thread per job, a program keeps a modest group of threads on hand and feeds the jobs to them. To understand how this works, we will follow a single job through the system, look at how the job is packaged, and then examine how to decide how large the group should be.
What do you think happens?
If a program has hundreds of small jobs, what should happen to the threads after one job finishes?
Reveal answer
The thread becomes idle again and can be assigned the next job
A thread that finishes its work in a pool does not disappear. It becomes idle again and is ready for the next task that arrives, which is what lets a few threads serve many jobs.
Packaging Work as a Runnable
Before a job can be handed to a thread, the job has to exist as something a program can pass around. In an object-oriented design, each subtask is usually represented by an object. Because a subtask is a computation, its object carries an instance method that performs that computation. Running the subtask then reduces to one action: calling that method.
In the source's own program, that method is named run(), and the task object implements the standard Runnable interface. The point of the interface is that it provides a common, standard way to describe a computational task. Any code that knows how to run a Runnable can therefore run any task that follows the same pattern, whatever the task actually computes.
This separation is what makes the rest of the design possible. Once a task is an object with a run() method, the program does not need to know in advance which thread will eventually execute it. It only needs a way to get the object to some thread that is free to call run(). Task decomposition, the step of splitting a large job into independent subtasks, is the first half of this story, and packaging each subtask as a Runnable is the second.
Why a Pool Beats a Thread per Task
A Runnable could be given its own thread immediately, and that is a legitimate option for a single task. The trouble appears when there are many tasks. Each thread that is created carries overhead, so a program that creates a thread for every one of many tasks spends a large share of its effort on creating and managing threads rather than on the computations themselves. The sensible alternative is to create only a few threads and let each of them execute a number of tasks, one after another, as the work arrives.
A thread pool is a group of threads that can execute multiple tasks, with each task assigned to any idle thread in the pool rather than to a newly created thread.
Tracing a Task Through the Pool
The key question is what happens when a task needs to be performed. Following the source, the answer is that the task is assigned to any idle thread in the pool. Tasks are not tied to particular threads. Each task joins the pool, and whichever pooled thread happens to be idle takes it. The flowchart below traces that path for one submitted task.
Five Image Tiles, Two Pooled Threads
A pool has two threads, A and B, both idle. Five tile tasks are submitted one after another to process five pieces of an image. Trace which thread can take each task. This example is generated for illustration; the source does not specify a particular pool size or submission order.
- 1Tile 1 is submitted
Both threads are idle, so Tile 1 is assigned to any idle thread. Suppose Thread A takes it and calls
run(). - 2Tile 2 is submitted
Thread B is still idle, so Tile 2 is assigned to Thread B.
- 3Thread A finishes Tile 1
Thread A does not disappear. It becomes idle again and is ready for the next task.
- 4Tile 3 is submitted
Thread A is idle, so Tile 3 is assigned to it. No new thread is created.
- 5Tiles 4 and 5 follow the same pattern
Whichever thread is idle takes the next tile. The same two threads carry out all five tasks.
Five tasks were completed by the same two reused threads, with no new thread created for any tile.
Choosing the Pool Size
A pool needs a size, and the right size is not fixed by any universal rule. The source is direct about this: the optimal number of threads is not entirely clear, and it depends on the particular problem being solved. The guiding goal is to keep all of the computer's processors busy. When every processor always has a thread ready to run, the hardware is being used well.
Mistakes Learners Make with Pools
Common mistake Creating a new thread for every task so each one has its own thread
What it looks likeA program with hundreds of tile tasks starts hundreds of threads, one per tile.
Why it failsCreating a thread carries significant overhead, and the program spends effort on thread creation rather than on the computations.
Do this insteadKeep a modest group of threads and submit the tasks to the pool, so threads are reused across many tasks.
Common mistake Assuming a task is permanently tied to one thread
What it looks likeBelieving Tile 3 must always run on Thread A because it ran Tile 1.
Why it failsEach task is assigned to any idle thread in the pool, so no task is tied to a particular thread.
Do this insteadThink of tasks as flowing to whichever pooled thread is idle.
Common mistake Using one thread per processor for every problem
What it looks likeSetting the pool to one thread per processor for a program whose tasks spend long periods waiting on a network connection.
Why it failsThe source says a blocked thread leaves its processor idle unless another thread is ready, so blocking-heavy work benefits from extra threads.
Do this insteadConsider whether tasks block while waiting. If they do, add threads beyond the processor count.
Common mistake Treating the optimal pool size as a fixed universal number instead of a problem-dependent choice
Sizing a Pool for Your Own Tasks
A program must download data for 200 separate requests over a network, and each request spends most of its time waiting for a reply. Another program must compress 200 image tiles, a task that keeps the processor busy the whole time. Using the source's reasoning, explain which program needs more threads in its pool relative to its processor count, and why.
Hints
- Ask whether the thread spends its time doing computation or blocking while it waits.
- The source ties the benefit of extra threads to threads that block while waiting, such as waiting for data over a connection.
- Think about what happens to a processor when its only thread is blocked.
Key Takeaways
- A thread pool is a group of threads that execute multiple tasks, with each task assigned to any idle pooled thread rather than to a newly created thread, so a few threads are reused across many tasks.
- Each subtask is typically packaged as a Runnable object whose
run()method performs the computation, which lets any available thread execute it. - The right pool size depends on the problem: one thread per processor suits computation-bound work, while tasks that block while waiting, such as waiting on a network connection, benefit from extra threads.
- Pools exist to avoid the significant overhead of creating a new thread for each task.