Producer/Consumer Pattern and Blocking Queues

When several threads work on the same data, a common arrangement is to have some threads produce work while other threads consume it. Producers create items, consumers handle them, and all of them can work at the same time. This arrangement is called the producer/consumer pattern, and it appears whenever the rate at which work is created does not match the rate at which it is handled.

The pattern depends on a data structure that can pause a thread when there is nothing to do or no room to work. That structure is called a blocking queue. In this article you will follow the behavior of a blocking queue step by step: what a consumer does when the queue is empty, what a producer does when a limited queue is full, and how the queue's contents change as tasks are added and removed.

A queue sits between the two groups. Producers add items at one end, and consumers remove items from the other end. The queue is the meeting point, so neither group needs to know the details of the other. However, when the producers and consumers are threads, the queue must be synchronized so that two threads never corrupt it by touching it at the same moment. Synchronization alone does not solve everything, because the waiting has to be handled as well.

Consider what a consumer should do when the queue has nothing in it. A simple design would have the consumer check the queue, see that it is empty, and then either give up or keep checking in a loop. Neither option is satisfying. Giving up loses the work the consumer was meant to do, and repeated checking wastes processor time. What is really wanted is for the consumer to pause until something arrives.

The same problem appears on the other side. A producer facing a full, limited-size queue should be able to pause until room opens up. A plain queue offers neither pause, so the programmer would have to write extra waiting logic around every access. The blocking queue exists so that this waiting is built into the queue operations themselves.

Before the mechanism is explained, commit to an answer. Imagine one consumer thread and one blocking queue. The queue starts with nothing in it, and the consumer calls take() on it. What should happen to the consumer's thread? Decide on your answer before reading on.

The state diagram below shows the two possible paths through take(). If an item is already present, the thread takes it and keeps going without ever pausing. If the queue is empty, the thread moves into the blocked state. It stays there until an item is placed in the queue, and at that moment it resumes and receives the item. Notice that the thread never has to repeatedly check the queue to find out when work has arrived.

Blocking applies in both directions. On the consumer side, a call to take() on an empty queue blocks the caller until an item appears. On the producer side, a call to put(item) on a limited-capacity queue that is already full blocks the caller until a space opens. The same queue can therefore hold back consumers when it is empty and hold back producers when it is full, without any extra logic written by the programmer around the queue.

The diagram below shows the flow. Each producer calls put(item) to hand work to the queue, and each consumer calls take() to receive work from it. Neither a producer nor a consumer needs to know about the other; each only needs to know about the queue. The queue is therefore the single place where work changes hands.

The two groups do not run at the same pace, and the pattern has to cope with that. Sometimes a consumer finds nothing ready to process, and it must wait until a producer makes something. This waiting is a normal part of the pattern, not an error. The reverse situation matters just as much: if producers keep producing faster than consumers can handle, the backlog grows without bound unless a limit is placed on how many items may wait in the queue.

The timeline below traces a smaller sequence of operations on an empty queue. Read it from top to bottom, and note that the queue returns items in the order they were added: the first task placed in is the first task taken out.

Java provides two classes that implement blocking queues: LinkedBlockingQueue and ArrayBlockingQueue. Both are parameterized types, so you specify the type of item the queue will hold. Both are defined in the package java.util.concurrent, and both implement an interface called BlockingQueue. Because they share that interface, the take() and put(item) operations behave the same way on either class, while the capacity rules differ.

Choosing between the classes is really a decision about where back-pressure should come from. A bounded ArrayBlockingQueue lets the queue itself slow producers down when consumers fall behind. An unbounded LinkedBlockingQueue lets producers keep going, which avoids blocking them, but then the queue can keep growing if consumers are slower than producers.

Take() and put(item) are the blocking pair: one waits for items, and the other waits for room. Add(item) is the non-blocking counterpart of put(item). The source recommends put(item) for limited-capacity queues and add(item) for unlimited ones, because add(item) cannot wait for room that will never be needed. Consumers should use take(), which waits for work without polling.

Consider a program that draws images, where each image takes real computation to produce. The source course uses an example called MultiprocessingDemo2, which creates an entirely new thread pool each time it draws an image. That looks wasteful, and the obvious improvement is to create one set of threads when the program starts and reuse them whenever an image needs computing.

A thread pool only works well if its threads can sit idle until work arrives and then carry that work out. Without a blocking queue there is no direct way to make a thread wait for a task to show up. The blocking queue fills that gap: idle pool threads call take() on the queue, block while nothing is available, and resume as soon as a producer hands them a task.