Thread Pools and ExecutorService

A Runnable is handed to an ExecutorService with execute, which returns as soon as the task is placed in a queue.

Programming

Why Hand Work to a Pool

Suppose a program has several pieces of work it wants to run in parallel. It is possible to create and manage a separate thread for every piece, but that quickly becomes tedious. The ExecutorService approach separates two responsibilities. The main program describes the work, and a pool of threads carries it out.

This article follows one question through the design. Once the main program has handed a task to the pool, how does it send the task off, and how does it get back an answer when the task produces one? Keep two kinds of task in mind as you read. Some tasks only need to be carried out, with no value coming back. Other tasks compute something and hand a value back. The article follows both kinds from the moment they are submitted to the moment their results are read.

Hand-Off Through execute

Definition

A Runnable object describes a unit of work. To give such an object to an ExecutorService, the program calls execute on the executor and passes the task in: executor.execute(task).

The call does not run the task on the spot in the calling thread. Instead, the task is placed into a queue of tasks waiting to be run. The method returns immediately once the task has been queued. The caller moves on to its next statement without waiting. The threads in the pool are the ones that take tasks off the queue and execute them.

After the caller calls execute(), which thread does what, and at what point does the caller continue while the task is still pending in the queue or running on a worker thread?

Read the sequence as two lanes running at once. The upper lane is the caller, which enqueues the task and continues. The lower lane is the pool, where a worker takes the task from the queue and runs it. The source describes this split as the heart of the design: the code that submits work and the code that performs work run on different threads, and submission is quick because it only involves enqueueing.

java

Runnable task = () -> System.out.println("Working");
executor.execute(task);
System.out.println("Submitted");

Runnable task = () -> System.out.println("Working");
executor.execute(task);
System.out.println("Submitted");

Notes on the code

The lambda describes the work but does not run it yet
execute places the task in the queue and returns
This line may print before or after the word Working, because the caller does not wait for the task

What do you think happens?

After executor.execute(task) returns, does the caller wait for the task body to finish before running the next line?

Yes, the caller waits until the task body completes
No, the caller moves on once the task has been queued

Reveal answer

Answer:

No, the caller moves on once the task has been queued

execute returns immediately once the task is queued. The worker threads take tasks off the queue and run them, so the caller does not wait for the task to finish.

Callable Tasks That Return Values

A Runnable is fine when a task only performs an action. When a task must return a result, the ExecutorService framework offers a different type: Callable<T>. It is a parameterized functional interface. It declares a single method named call, which takes no parameters and returns a value of type T. The type parameter T names whatever kind of value the task produces, so a Callable<Integer> returns an integer and a Callable<String> returns a string.

Aspect	Runnable	Callable<T>
Role	A unit of action	A unit of computation
Value handed back	Nothing is handed back	A value of type T is produced by call()
Method	Performs the action	Single method named call with no parameters
Typical submission	executor.execute(task)	executor.submit(c)
How the caller gets a result	No result mechanism needed	Through a Future<T> returned by submit

Runnable and Callable<T> compared as tasks given to an ExecutorService

The contrast with Runnable is the return value. A Runnable task is a unit of action with nothing handed back, while a Callable task is a unit of computation that yields something. That difference is what makes a separate mechanism for retrieving results necessary.

Futures as Placeholders

A Callable object c is submitted with executor.submit(c). Like execute, the Callable is not run in the calling thread at the moment of submission. It will be executed at some later time. This creates an obvious question: once the computation completes, how does the program get its answer?

The answer is a second interface, Future<T>. A Future<T> stands in for a value of type T that may not be available yet. The call executor.submit(c) returns such a Future, and the Future represents the result of the computation that is still under way. The program holds onto this placeholder and asks it for the value when it needs the value.

What does the caller actually receive from submit(), and how does that Future object relate to the task and to the result slot that will be filled later?

The diagram shows the two paths that start at the moment of submission. One path leads from the main program to the Future, which the main program keeps in hand. The other leads to a pool thread that runs call() at some later time. The value produced by call() reaches the main program only when the main program asks the Future for it. Read the figure as a relay: the Future is the handle that connects the program's present-time request to the pool's later-time computation.

java

Callable<Integer> c = () -> 6 * 7;
Future<Integer> f = executor.submit(c);
System.out.println("Task handed off");

Callable<Integer> c = () -> 6 * 7;
Future<Integer> f = executor.submit(c);
System.out.println("Task handed off");

Notes on the code

The Callable<Integer> promises an integer result from call()
submit returns a Future<Integer>, not the integer itself
The integer 42 is not available on the line where f is assigned

Asking Without Waiting: isDone

A Future defines several methods, and two matter most for retrieving results. The first is isDone(), a boolean-valued method that reports whether the result is available. The source describes it as a check that runs without waiting. The method is a question the program can ask at any time without changing the Future's state; it simply reads the current status.

When the caller invokes get() or isDone() on a Future, which state is the Future in, and what happens to the calling thread if the task has not finished yet?

The diagram captures the only transition that isDone is concerned with. Before the computation completes, the Future reports that its result is not yet available. After completion, it reports that the result is available. Asking the question never moves the Future between these states, so the program can check repeatedly without affecting the computation.

Waiting for Answers: get

The second method, get(), behaves differently. Calling f.get() retrieves the value the Future holds. If the value is not yet available, the call blocks until it is. Blocking means the calling thread pauses at that line and resumes only when the result arrives. Because of this, get is the natural way to wait for an answer, but calling it early will stall the program.

java

try {
    int answer = f.get();
    System.out.println(answer);
} catch (Exception e) {
    System.out.println("The task failed or was interrupted");
}

try {
    int answer = f.get();
    System.out.println(answer);
} catch (Exception e) {
    System.out.println("The task failed or was interrupted");
}

Notes on the code

get() may pause the calling thread here until the value is ready
get() can raise exceptions, so the call sits inside try..catch
The catch block handles failures reported while waiting for or retrieving the value

Worked example

Tracing isDone and get on one submitted Callable

A Callable<Integer> is submitted, and the main program then checks isDone before deciding whether to call get. Trace what the Future reports and what the calling thread does in each case.

1
Submit the task
submit(c) returns a Future<Integer> immediately. The computation is still under way on a pool thread, so the Future is a placeholder.
2
First isDone check
If the computation has not completed, isDone returns false. The check does not wait, and the Future stays in the same state.
3
Calling get too early
If get is called now, the calling thread blocks at that line until the value is available, then resumes with the integer.
4
Calling get after completion
If isDone returns true first, get returns the value without a pause, because the result is already available.

Result

isDone lets the program look without waiting. get is the step that actually waits when the result is not ready yet.

Mistakes Beginners Make with Futures

Common mistake
Assuming execute returns the task's result
What it looks like
```
int x = executor.execute(c);
```
Why it fails
execute is used for Runnable tasks, which carry no value back. It returns as soon as the task is queued.
Do this instead
Use submit with a Callable when a value is needed, and keep the returned Future.
Common mistake
Expecting the next line to run after the task body
What it looks like
```
executor.execute(task);
System.out.println("done");
```
Why it fails
The caller continues once the task is queued, so the message may print before the task has run.
Do this instead
Only rely on a task's effects after waiting for them, for example through a Future and get.
Common mistake
Treating the Future as the value itself
What it looks like
```
int y = f + 1;
```
Why it fails
A Future<T> stands in for a value of type T that may not be available yet. It is a handle, not the number.
Do this instead
Call get on the Future to obtain the value, inside try..catch.
Common mistake
Calling get too early without checking
Why it fails
get blocks the calling thread until the value is available, which can stall the program.
Do this instead
Use isDone to check whether the result is ready, or structure the code so get runs at the point the answer is needed.
Common mistake
Omitting exception handling around get
Why it fails
The source states that get must be wrapped in try..catch because it can raise exceptions.
Do this instead
Place every get call inside a try block with a matching catch.

Tracing a Submission Yourself

MEDIUM

A program calls executor.execute(taskA), then executor.submit(callableB) which returns Future<String> f, then prints a message, then calls f.get(). Trace which of these steps may pause the calling thread, and explain why execute does not pause it even though taskA has not necessarily finished.

Hints

Recall that execute returns once the task is queued
Recall that submit also returns immediately with a Future placeholder
Ask which call is the one defined as blocking until the value is available

Your attempt (not saved)

Wrap-Up

1
execute(task) hands a Runnable to the pool. It places the task in a queue and returns immediately, so the caller does not wait for the task to finish.
2
A Callable<T> has a call method that returns a value of type T. Its result can be retrieved, unlike the action-only Runnable.
3
submit(c) returns a Future<T>, a placeholder for a value that may not be available yet. The value reaches the caller only when the program asks the Future for it.
4
isDone() reports whether the result is available without waiting. get() returns the value and blocks until it is available, so it must be wrapped in try..catch.

Key Takeaways

execute hands a Runnable to the pool, which queues it and returns at once without waiting for the task to run.
Callable<T> produces a value through call(), and submit returns a Future<T> that stands in for that value.
isDone checks availability without waiting, while get retrieves the value and blocks until it is ready.
Calls to get must be wrapped in try..catch because they can raise exceptions.

Why Hand Work to a Pool

Hand-Off Through execute

Callable Tasks That Return Values

Futures as Placeholders

Asking Without Waiting: isDone

Waiting for Answers: get

Mistakes Beginners Make with Futures

Tracing a Submission Yourself

Wrap-Up

Key Takeaways

Related Topics