Concepts / Thread Pools and Reuse

Thread Pools and Reuse

ExecutorService is a Java interface in java.util.concurrent that defines services for executing submitted tasks, and it is used to manage thread pools.

  • Programming

Why Reuse Threads

Suppose a program has many small pieces of work that can run in parallel. One approach is to create a new thread for each piece, start it, and let it finish. That works, but the program keeps making and discarding threads, and it has no built-in way to limit how many run at once. Java provides a higher-level tool for this situation called ExecutorService. Instead of managing individual threads yourself, you hand work to an executor and let it manage a group of reusable threads, called a thread pool.

This article follows a plan that starts from running code. You will watch where a submitted task goes, then name the pieces that made that path happen. As you read, try to predict what each line changes before reading the explanation that follows it.

Following a Submitted Task

A task is submitted with executor.execute(task), where task is a Runnable object. A submitted task moves through three stages. It is handed to execute, it waits in a queue of waiting tasks, and then a pool thread removes it from the queue and runs it. Your code runs on one side of this path and the pool threads run on the other side, with the queue between them.

The diagram shows that the thread which runs your task is not the thread that called execute. The submitting code only places the task into the queue. A pool thread is the one that removes the task and executes it. This is the key to tracing any surprising result: follow the task from the queue to a pool thread, not to the submitting thread.

What do you think happens?

When executor.execute(task) returns, has the task finished running?

  • Yes, execute waits until the task has completed

  • No, the task may not have started yet, because execute returns after placing the task in the queue

Reveal answer
Answer:

No, the task may not have started yet, because execute returns after placing the task in the queue

The source states that execute returns immediately after placing the task into a queue of waiting tasks. The call does not wait for the task to run. A pool thread picks the task up later.

Where ExecutorService Lives

ExecutorService is a Java interface in the java.util.concurrent library. It defines services for executing submitted tasks, and it is used to manage thread pools. Because it is an interface, you do not write new ExecutorService() to get one. An interface describes what operations are available without itself being a working object.

Definition

ExecutorService is a Java interface in java.util.concurrent that defines services for executing submitted tasks, and it is used to manage thread pools.

Creating a Fixed-Size Pool

The method this article focuses on is Executors.newFixedThreadPool(n), where n is an int. It creates a thread pool with n threads. The source's example sets n to the number of processors the machine reports, using Runtime.getRuntime().availableProcessors(). That choice makes the pool size match the processor count of the machine running the program.

Code walkthrough
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class PoolDemo {
    public static void main(String[] args) {
        int n = Runtime.getRuntime().availableProcessors();
        ExecutorService executor = Executors.newFixedThreadPool(n);
        for (int i = 1; i <= 5; i++) {
            final int id = i;
            Runnable task = () -> System.out.println("Task " + id + " running");
            executor.execute(task);
        }
        System.out.println("All tasks submitted");
        executor.shutdown();
    }
}
  1. 1
    Lines 1 and 2 import the interface and the factory class, since the code refers to ExecutorService and calls Executors.
  2. 2
    Line 6 asks the runtime how many processors are available and stores that number in n.
  3. 3
    Line 7 creates the pool with n threads. The variable type is the interface ExecutorService.
  4. 4
    Lines 8 to 11 build five Runnable tasks and submit each one with execute. Each task prints its own number when a pool thread runs it.
  5. 5
    Line 12 prints a message from the submitting thread right after the submissions. Because execute does not wait, this message can appear before some tasks have run.
  6. 6
    Line 13 calls shutdown, a closing step. The source pack does not explain it further, so treat it as the final step of this example for now.
  • The pool size n comes from the processor count reported by the machine.
  • The Runnable lambda captures a copy of i in the final variable id.
  • Each execute call hands one task to the pool and returns without waiting.
Output
The Task lines may print in any order, and "All tasks submitted" may appear before some of the task lines. The exact order depends on thread timing.

Worker Threads Reused Across Tasks

A pool exists ahead of time. Work is given to the pool, and the pool's threads pick up that work. The same threads are reused for many tasks rather than being created for each one. After a pool thread finishes one task, it returns to the queue, takes the next waiting task, and runs that one. The thread is not discarded between tasks.

The loop in the diagram is the reuse. The same worker thread cycles through taking, running, and finishing tasks. This is the reason a pool avoids the cost of making and discarding a new thread for every piece of work, which was the problem the article opened with.

When More Tasks Than Threads

Suppose a fixed pool has n threads and receives more than n tasks at once. The extra tasks wait in the queue. They do not start immediately. As each pool thread finishes its current task, it removes the next task from the queue and runs it. So with a pool of two threads and four submitted tasks, two tasks can run at the same time, and the other two wait their turn.

The diagram traces the case above. Tasks 1 and 2 are taken by the two threads. Tasks 3 and 4 sit in the queue. Task 3 starts only when a pool thread is free to take it from the queue, here pool thread 1 after Task 1 finishes. Waiting tasks are not lost and are not run on the submitting thread.

Tracing a Surprising Result

When a submitted task does not behave as you expected, the useful question is where along the path the expectation broke. Check three points in order. First, did the pool get the number of threads you intended, meaning the n passed to newFixedThreadPool? Second, did the task reach the queue through execute? Third, did a pool thread pick it up and run it? A mismatch at the first point is a pool-size issue, a mismatch at the second is a submission issue, and a mismatch at the third is an execution issue.

Worked example

A Log Line That Appears Too Early

A developer submits a task that prints "Report saved" and then immediately prints "Done" from the same method. The developer expected "Report saved" to appear before "Done" and suspects the task ran on the calling thread.

  1. 1
    Check the pool size

    The code creates the pool with Executors.newFixedThreadPool(4). The n is 4, which matches what the developer intended, so the first check passes.

  2. 2
    Check the submission

    The task was passed to executor.execute(task). The call places the task in the queue and returns immediately. It does not wait for the task to run, so "Done" can print first.

  3. 3
    Check the execution

    The task runs later on a pool thread, not on the thread that printed "Done". The order between the two lines is not guaranteed by execute alone.

  4. 4
    Locate the mismatch

    The expectation broke at the submission point. The developer assumed execute would wait for the task. The fix is to make the calling code wait for the result, not to change the pool size.

Result

The surprising order was not a pool-size bug or an execution bug. It came from assuming execute blocks until the task finishes, which the source says it does not do.

Mistakes Beginners Make With Pools

  • Common mistake

    Writing new ExecutorService() to create a pool

    Why it fails

    ExecutorService is an interface, so it cannot be instantiated directly. The source points to the Executors class, whose static methods create ExecutorService objects.

    Do this instead

    Call Executors.newFixedThreadPool(n) and store the result in an ExecutorService variable.

  • Common mistake

    Assuming execute waits for the task to finish

    What it looks like

    Printing a line right after executor.execute(task) and expecting the task's output to appear first.

    Why it fails

    execute returns immediately after placing the task in the queue. The task may not have started yet.

    Do this instead

    Treat execute as handing off work. If the caller needs the result before continuing, the code must wait for it in some other way.

  • Common mistake

    Expecting the submitting thread to run the task

    Why it fails

    The source says that tracing a task means following it from the queue to a pool thread rather than to the submitting thread.

    Do this instead

    When debugging, look for the pool thread that removes the task from the queue and runs it.

  • Common mistake

    Confusing the pool size with the number of tasks

    What it looks like

    Creating a pool of 2 threads and expecting 10 tasks to run simultaneously.

    Why it fails

    A fixed pool has n threads. Extra tasks wait in the queue until a pool thread becomes free.

    Do this instead

    Expect at most n tasks to run at the same time in a fixed pool, with the rest waiting.

  • Common mistake

    Choosing n without thinking about the machine, when the source's example uses the processor count from availableProcessors

Practice Tracing a Task

MEDIUM

A program creates a fixed pool with newFixedThreadPool(2), then calls execute three times in a row with tasks A, B, and C. Each task prints its name. Describe the path of task C from the moment execute(task C) is called until it prints. Name the component that holds task C while it waits, and the kind of thread that eventually runs it.

Hints
  • Start with where execute places the task before any thread runs it.
  • Ask how many pool threads are free while tasks A and B are running.
  • Remember that the thread which runs task C is a pool thread, not the thread that called execute.

Key Takeaways

  • ExecutorService is a Java interface in java.util.concurrent that defines services for executing submitted tasks and is used to manage thread pools. You get one from the Executors class, such as Executors.newFixedThreadPool(n).
  • A fixed-size pool has n threads, and the source's example sets n to the processor count from Runtime.getRuntime().availableProcessors().
  • executor.execute(task) places a Runnable in a queue of waiting tasks and returns immediately. It does not wait for the task to run.
  • Pool threads remove tasks from the queue and execute them, and the same threads are reused across many tasks. Extra tasks wait in the queue when all pool threads are busy.
  • When a task behaves unexpectedly, check in order: the pool size, whether the task reached the queue through execute, and whether a pool thread picked it up and ran it.

Related Topics