Following the flow of a Java Stream

How to visualize multiple lazy operations in a Java Stream.

Photo by JJ Shev on Unsplash

Eager is easy, Lazy is Labyrinthine

A few years ago, I wrote a blog that explained the differences between eager and lazy iteration patterns in terms of Stream operation names (filter, map, reduce). I used the peek method of Java Stream with System.out.println to explain the most challenging part of understanding lazy iteration patterns — the order and flow of execution.

Eager is Easy, Lazy is Labyrinthine

While using peek with System.out.println is fine for a blog, there is a better way if we want to encode some of the “how” something works in a test for future maintainers to learn and understand from, without requiring them to execute the code and look at the output.

Don’t print, assert!

The following is code that is similar to the code I wrote in the blog above, where I use peek to understand the inputs that are used for each step in the Stream.

@Test
public void lazyFilterMapReducePrint()
{
    List<Integer> list =
            List.of(1, 2, 3, 4, 5);

    Optional<String> lazy = list.stream()
            .peek(i -> System.out.println("filter: " + i))
            .filter(each -> each % 2 == 0)
            .peek(i -> System.out.println("map: " + i))
            .map(String::valueOf)
            .peek(i -> System.out.println("reduce: " + i))
            .reduce(String::concat);

    Assertions.assertEquals("24", lazy.orElse(""));
}

This code by itself does not tell us anything about the order of execution of the Stream pipeline. We need to run the code to see the output. The following is the output when I run this code.

filter: 1
filter: 2
map: 2
reduce: 2
filter: 3
filter: 4
map: 4
reduce: 4
filter: 5

We can convert the peek code from printing output to adding elements to a List and then asserting equality to an expected ordered List.

@Test
public void lazyFilterMapReduceAssert()
{
    List<Integer> list =
            List.of(1, 2, 3, 4, 5);

    MutableList<String> order =
            Lists.mutable.empty();

    Optional<String> lazy = list.stream()
            .peek(i -> order.add("filter: " + i))
            .filter(each -> each % 2 == 0)
            .peek(i -> order.add("map: " + i))
            .map(String::valueOf)
            .peek(i -> order.add("reduce: " + i))
            .reduce(String::concat);

    List<String> expectedOrder = List.of(
            "filter: 1",
            "filter: 2",
            "map: 2",
            "reduce: 2",
            "filter: 3",
            "filter: 4",
            "map: 4",
            "reduce: 4",
            "filter: 5");

    Assertions.assertEquals(expectedOrder, order);
    Assertions.assertEquals("24", lazy.orElse(""));
}

In this code we take the List of integer values and see them first get evaluated against the Predicate which tests and filters if they are even. When the numbers 2 and 4 are tested, we see they flow on to the next step immediately to be mapped to a String, and then finally reduced together into a concatenated String, which gets returned as an Optional value.

Here’s an eager equivalent of the code above with the output as assertions. The code uses Eclipse Collections types and eager methods, because there is no equivalent eager behavior in the standard Java Collection Framework. The equivalent to the Stream peek method in Eclipse Collections is called tap.

@Test
public void eagerSelectCollectReduceAssert()
{
    ImmutableList<Integer> list =
            Lists.immutable.of(1, 2, 3, 4, 5);

    MutableList<String> order =
            Lists.mutable.empty();

    Optional<String> eager = list
            .tap(i -> order.add("select: " + i))
            .select(each -> each % 2 == 0)
            .tap(i -> order.add("collect: " + i))
            .collect(String::valueOf)
            .tap(i -> order.add("reduce: " + i))
            .reduce(String::concat);

    List<String> expectedOrder = List.of(
            "select: 1",
            "select: 2",
            "select: 3",
            "select: 4",
            "select: 5",
            "collect: 2",
            "collect: 4",
            "reduce: 2",
            "reduce: 4");

    Assertions.assertEquals(expectedOrder, order);
    Assertions.assertEquals("24", eager.orElse(""));
}

The eager version of similar functioning code selects (aka filter) all evens first, then collects (aka map) as Strings the elements that were selected, and finally reduces those elements to the final output. The order of the output matches the order of the method calls in the eager case. This makes it easy to understand and reason about.

The lazy equivalent in Eclipse Collections will have the same order of execution as the Java Stream code. The following is the equivalent code using the LazyIterable type in Eclipse Collections.

@Test
public void lazySelectCollectReduceAssert()
{
    ImmutableList<Integer> list =
            Lists.immutable.of(1, 2, 3, 4, 5);

    MutableList<String> order =
            Lists.mutable.empty();

    Optional<String> lazy = list.asLazy()
            .tap(i -> order.add("select: " + i))
            .select(each -> each % 2 == 0)
            .tap(i -> order.add("collect: " + i))
            .collect(String::valueOf)
            .tap(i -> order.add("reduce: " + i))
            .reduce(String::concat);

    List<String> expectedOrder = List.of(
            "select: 1",
            "select: 2",
            "collect: 2",
            "reduce: 2",
            "select: 3",
            "select: 4",
            "collect: 4",
            "reduce: 4",
            "select: 5");

    Assertions.assertEquals(expectedOrder, order);
    Assertions.assertEquals("24", lazy.orElse(""));
}

IntelliJ to the rescue

The problem of understanding Java Stream flow is common enough that the wonderful developers at JetBrains developed a special debugging feature for Java Stream to help us.

Analyze Java Stream operations | IntelliJ IDEA

The great thing about this tool is that we don’t need to litter calls to peek at various points in the Stream in order to use it. Our code can stay as simple as follows.

@Test
public void lazyFilterMapReduce()
{
    ImmutableList<Integer> list =
            Lists.immutable.of(1, 2, 3, 4, 5);

    Optional<String> lazy = list.stream()
            .filter(each -> each % 2 == 0)
            .map(String::valueOf)
            .reduce(String::concat);

    Assertions.assertEquals("24", lazy.orElse(""));
}

Using the debugging tool

Put a breakpoint in your code and run the code in debug mode. Look for this button with the bubble help text of “Trace Current Stream Chain” in the debugging tab.

There are two modes of viewing a Stream. In split mode, we can look at individual steps in the Stream.

Let’s look at filter first.

Using Split mode to look at filter results

Let’s look at map next.

Using split mode to look at map results

Finally, let’s look at reduce.

Using split mode to look at reduce results

Using the split mode is kind of tedious with multiple steps, so we will switch to flat mode which will show all of the steps in the Stream at once.

This view shows us what happens to the inputs after each stage of the Stream pipeline, but it doesn’t really show us the order in which the execution happens. We are only able to see this using the peek approach with some form of either printable or assertable output.

Final Thoughts

I hope you found this blog helpful. We can use peek in Java Stream, or tap in Eclipse Collections to help us understand the flow of lazy code. With Java Stream, we also have the benefit of being able to use the Stream specific debugging tool in IntelliJ to analyze the inputs and outputs at each stage in the pipeline. It would be nice if someone developed a similar debugging tool for Eclipse Collections LazyIterable type.

Thanks for reading!

I am the creator of and committer for the Eclipse Collections OSS project, which is managed at the Eclipse Foundation. Eclipse Collections is open for contributions.