Filtering and Mapping Streams

A stream pipeline usually ends with a terminal operation, and many terminal operations exist to turn a whole stream of values into one answer. Asking for the total of a list of numbers or the largest of a set of values are common examples. Rather than writing a separate terminal operation for every possible question, the Stream API offers a more general tool called reduce, which can express many of these computations.

The same idea also explains a second tool, collect. It answers a different kind of question: instead of producing one value, it gathers the elements of the stream into a new collection. Both tools end a pipeline. This article explains how each one works, why the rule given to reduce has to satisfy a special property, and how to trace a result back to the stage that produced it.

Think of reduce as a rule for combining two values at a time. You hand the stream a rule that takes two inputs and returns one output. The stream then uses that rule repeatedly, merging values until only a single result is left. When the rule is addition, the final result is a sum. When the rule is multiplication, the final result is a product. In general terms, a reduce operation combines the values from a stream using a BinaryOperator, and a sum is one case where the binary operation is addition.

The easiest way to see what reduce does is to watch the accumulated value change. The code below sums five numbers. Notice that the program never writes a loop; the stream applies the combining rule for you.

Read the diagram from left to right. Each box is the accumulated value after one step. Each step takes the previous accumulated value and one new element from the stream, and combines them with the rule. The final box, 14, is the single result that reduce returns.

The Stream API has no ready-made terminal operation for the product of the values in a stream. You can still compute it, because reduce lets you supply the combining rule yourself. The source's own illustration is the product of all the non-zero elements in an array of double values. The code below follows that pattern: a filter removes the zeros, and reduce multiplies what remains. The starting value 1.0 is the value that leaves any number unchanged when multiplied, so it works as the starting point for a product.

The rule you give reduce must be a BinaryOperator, and the source adds a condition: the binary operator should be associative. An operator is associative when the way its applications are grouped does not change the result. The source states the practical consequence directly: the order in which the operator is applied does not matter. Because of this, you should check the property before relying on reduce.

The two cases below show how to tell when grouping matters. The first case is one where it does not, and the second is one where it does.

The collect operation has a different purpose from reduce. The source describes it as accumulating the input elements into a new Collection. Where reduce returns a single combined value, collect returns a collection that holds the elements that reached it. The usual form names the collector as Collectors.toList(), so the elements are gathered into a List.

Read this example from left to right, since that matches the order in which values move through the pipeline. collect only gathers what the filter let through. If freds ever contains a value you did not expect, the filter condition is the first place to check.

When a pipeline gives a wrong answer, the cause is usually one of its stages. The filter stage decides which values reach the reduction, and the reduce stage decides how they are combined. Tracing each stage's input and output shows which one diverged from what you expected. The diagram below shows a pipeline with a filter, a map, and a reduce in sequence.