Core Java Advanced LiveLessons

Streams

Slide navigation: Forward with space bar, → arrow key, or PgDn. Backwards with ← or PgUp.

Understand the stream concept and its benefits

From Iterating to Stream Operations

• Typical iteration: Count long words.

  List<String> words = ...;
  int count = 0;
  for (String w : words) {
      if (w.length() > 12) count++;
  }
  

• How, not what.
• With streams:

  long count = words.stream()
     .filter(w -> w.length() > 12)
     .count();
  

• What, not how.

jshell

import java.util.stream.*;
import java.nio.file.*;
Stream<String> words = Files.lines(Paths.get("/usr/share/dict/words"));
long count = words.filter(w -> w.length() > 12).count();

What Is a Stream?

• A stream is superficially similar to a collection, but a stream does not store its elements.
  • The stream fetches elements on demand.
• Stream operations don’t mutate their source.
  • A stream operation yields another stream (filter) or a result (count).
• Stream operations are lazy when possible.
  • Example: Request only the first five long words.
  • Filtering stops after fifth match.

Stream Workflow

1. Create a stream.
2. Transform with intermediate operations.
   • Each step yields another stream.
   • Nothing is computed yet.
3. Produce result with terminal operation.
   • Forces execution of lazy operations.
   • Afterwards, the streams can no longer be used.

Be able to create streams

Stream Creation

• Turn any collection into a stream:

  Stream<T> stream = collection.stream();

• Turn any array of objects into a stream:

  Stream<T> stream = Stream.of(array);
  

• Stream.of is a varargs method:

  Stream<String> song = Stream.of("gently", "down", "the", "stream");
  

• Stream.empty makes an empty stream:

Stream<String> silence = Stream.empty();

Infinite Streams

• Provide lambda with no arguments to generate elements:

  Stream<String> echos = Stream.generate(() -> "Echo");
  Stream<Double> randoms = Stream.generate(Math::random);
  

• To produce sequences such as 0 1 2 3 ..., use Stream.iterate.
• Supply a “seed” and a function f that is iteratively applied:

  Stream<BigInteger> integers
  = Stream.iterate(BigInteger.ZERO, n -> n.add(BigInteger.ONE));
  

• The stream has elements seed, f(seed), f(f(seed)), and so on.

API Methods that Yield Streams

• A stream of lines in a file:

  Stream<String> lines = Files.lines(path);
  

• Use inside try-with-resources to make sure file is closed:

  try (Stream<String> lines = Files.lines(path)) {
      Process lines
  }

• Breaking a string into a stream of tokens:

  String contents = ...;
  Pattern pattern = Pattern.compile("\\s+"); // Break at white space 
  Stream<String> words = pattern.splitAsStream(contents);
  

lesson02/creating

Java 9 News Flash - Creating Streams

• New API methods yielding streams:
  • Scanner.tokens, Matcher.results, ServiceLoader.stream, LocalDate.datesUntil, StackWalker.walk, ClassLoader.resources, Process.children/descendants, Catalog.catalogs, DriverManager.drivers
  • Common theme: laziness.
• Scanner.tokens gets a stream of tokens, similar to Pattern.splitAsStream from Java 8:

  Stream<String> tokens = new Scanner(path).useDelimiter("\\s*,\\s*").tokens();

• Finite stream iterator:

  BigInteger limit = new BigInteger("10000000");
  Stream<BigInteger> integers
      = Stream.iterate(BigInteger.ZERO,
          n -> n.compareTo(limit) < 0,
          n -> n.add(BigInteger.ONE));

Transform streams into other streams

Transforming Streams

• Streams are immutable—all operations yield new streams (or results).
• filter yields a new stream:

  List<String> words = ...;
  Stream<String> longWords = words.stream().filter(w -> w.length() > 12);

• The argument of filter is a Predicate<T>
• Use map to transform all stream elements:

  Stream<String> lowercaseWords = words.stream().map(String::toLowerCase);
  Stream<String> firstLetters = words.stream().map(s -> s.substring(0, 1));

flatMap

• map takes a Function<T, U> and yields a Stream<U>
• Suppose the result type is also a Stream:

  public static Stream<String> letters(String s)
      // letters("boat") is a Stream ["b", "o", "a", "t"]

• When you map letters to a Stream<String>, you get a Stream<Stream<String>>:

  [... ["y", "o", "u", "r"], ["b", "o", "a", "t"],
  ...]

• To flatten it into a Stream<String>, use flatMap:

  Stream<String> flatResult = words.stream().flatMap(w -> letters(w))
      // Yields [... "y", "o", "u", "r", "b", "o", "a", "t",
  ...]

• Useful to compose Stream-valued functions.
  • Later, we'll use flatMap to compose Optional-valued functions

lesson02/transforming

Extracting Substreams and Concatenating Streams

• limit returns a new stream that ends after a given number of elements (or earlier if the stream is shorter):

  Stream<Double> randoms = Stream.generate(Math::random).limit(100);

• skip does the opposite:

  Stream<String> words = Stream.of(contents.split("\\PL+")).skip(1);

• concat concatenates two streams:

  Stream<String> combined = Stream.concat(letters("Hello"), letters("World"));
      // Yields the stream ["H", "e", "l", "l", "o", "W", "o", "r", "l", "d"]

Other Stream Transformations

• distinct suppresses duplicates (not necessarily adjacent):

  Stream<String> uniqueWords
      = Stream.of("merrily", "merrily", "merrily", "gently").distinct();
      // Only one "merrily" is retained

• sorted yields a stream of the elements in sorted order:

  Stream<String> longestFirst =
      words.stream().sorted(Comparator.comparing(String::length).reversed());

• peek yields a stream with the same elements at the original, calling a function on each retrieved element:

  Object[] powers = Stream.iterate(1.0, p -> p * 2)
      .peek(e -> System.out.println("Fetching " + e))
      .limit(20).toArray();
  

Know how to get answers from stream data

Simple Reductions

• You know how to create and transform streams.
• To get an answer, apply a terminal operation.
• A reduction yields a value from a stream, terminating the processing pipeline:

  long count = words.stream().filter(w -> w.length() > 12).count();
  

• The max/min methods return the largest or smallest stream element:

  Optional<String> largest = words.max(String::compareToIgnoreCase);

• An Optional because the stream might have been empty—see next slide.
• findFirst yields the first element:

  Optional<String> startsWithQ = words.filter(s -> s.startsWith("Q")).findFirst();
  

• Use findAny instead on a parallel stream.
• anyMatch yields true if a predicate has a match:

  boolean aWordStartsWithQ = words.anyMatch(s -> s.startsWith("Q"));

• Also have allMatch and noneMatch.

jshell

import java.util.stream.*;
import java.nio.file.*;

Stream<String> words() throws IOException {
   return Files.lines(Paths.get("/usr/share/dict/words"));
}

words().filter(w -> w.length() > 12).max(String::compareToIgnoreCase);

words().filter(s -> s.startsWith("Qu")).findFirst();

Work with the Optional type

The Optional Type

• Optional<T> wraps an object of type T or no object.
• Safe alternative for “object or null”.
• Only safe if you use it right:
  • Produce an alternative if the value isn't present:

    String result = optionalString.orElse("");
    String result = optionalString.orElseGet(() -> System.getProperty("user.dir"));
    String result = optionalString.orElseThrow(IllegalStateException::new);

  • Consume value only if it is present:

    optionalValue.ifPresent(results::add);
    Optional<Boolean> added = optionalValue.map(results::add);

How Not to Work with Optional Values

• Avoid unchecked get:

  Optional<T> optionalValue = ...;
  optionalValue.get().someMethod()

• That's no safer than:

  T value = ...;
  value.someMethod(); // NPE if value is null

• Can check first:

  if (optionalValue.isPresent()) optionalValue.get().someMethod();

• But that's no easier than the traditional way:

  if (value != null) value.someMethod();

• Better to produce an alternative or consume only if present.

Creating Optional Values

• When you write a method that yields an Optional, produce a wrapped value if there is a result:

  public static Optional<Double> inverse(Double x) {
      if (x != 0) return Optional.of(1 / x);

• Return Optional.empty() if there is no result:

      else return Optional.empty();
  }
  

• ofNullable turns “object or null” into an Optional:

  obj = find(queryData);
  return Optional.ofNullable(obj);
      // Optional.of(obj) if obj not null, or Optional.empty()

Composing Optional Value Functions with flatMap

• Suppose the method f yields an Optional<T> and T has a method g yielding an Optional<U>.
• Can you “compose” f and g by calling s.f().g()?
• No—f returns an Optional<T>, not a T.
• Use flatMap instead:

  Optional<U> result = s.f().flatMap(T::g);
  

• Three possibilities:
  1. If s.f() is Optional.of(t) and t.g() is Optional(r), that's the result.
  2. If s.f() is Optional.of(t) and t.g() is empty, the result is empty.
  3. If s.f() is empty, the result is empty.
• Similar to flatMap on streams if you think of an Optional as a stream of size zero or one.

lesson02/optional

Java 9/10 News Flash - Optional

• New API methods yielding optional values:
  • HttpClient.authenticator/cookieManager/proxy/sslParameters, ServiceLoader.findFirst, Runtime.Version.build/pre/post, ProcessHandle.of, ProcessHandle.Info.command/commandLine/arguments/ startInstant/totalCpuDuration/user
• Java 9 adds three methods to the Optional class.

• optionalValue.ifPresentOrElse(System.out::println, // Consumer
     () -> Runtime.exec("rm -rf /home")); // Runnable

• Optional<Double> result = squareRoot(x).or(
     () -> inverse(x)) // Another Optional, supplied lazily

  • Is squareRoot(x) nonempty? If so, it is the answer.
  • Otherwise, compute inverse(x).

Optional Java 9 News

• Stream<T> stream()
  

  Yields a stream of length 0 or 1.
• Use with Stream.flatMap to drop empty results:

  class Person
  {
     public Optional<String> nickname() // Not everyone has one
     ...
  }
  
  Stream<String> nicknames = people.map(Person::nickname) 
     .flatMap(Optional::stream);

• Java 10: One new method.

String contents = optionalString.orElseThrow();

• Yup—that's the same as get

Optional Cheat Sheet

• Grab the data:

  data = opt.orElse(defaultValue);
  data = opt.orElseGet(() -> ...);
  data = opt.orElseThrow(SomeException::new);
  data = opt.orElseThrow(); // same as get, throws NoSuchElementException, Java 10

• If/then/else with lambdas:

  opt.ifPresent(value -> ...);
  opt.ifPresentOrElse(value -> ..., () -> ...); // Java 9
  

• Transform/substitute:

  anotherOpt = opt.filter(value -> ...).map(value -> ...).or(() -> ...); 
  

Optional Code Smells

• A variable of type Optional should never be null!
• Don't have fields of Optional type.
  • null is manageable inside a class.
• Avoid parameters of Optional type.
  • Clutters up the call site.
  • Optional intended as return type.
• A List of Optional values or a Map with Optional keys is dubious.
  • Instead of List<Optional<String>>, just have List<String>
• Don't create an Optional to make a null check:

  Optional.ofNullable(result).ifPresentOrElse(...); // Way too complex
  

Place stream results into collections and maps

Collecting Results

• Simple reductions yield a single value.
• Often, want to look at a collection of stream results.
• Can call Iterator<T> iter = stream.iterate()
• Can apply a function to each element:

  stream.forEach(System.out::println);
  

• More commonly, you want to collect elements in a data structure.
• toArray yields an array of stream elements.
• Provide the array constructor to get the correct result type:

  String[] result = stream.toArray(String[]::new);
      // stream.toArray() has type Object[]

Using Collectors

• The collect method passes stream elements to a Collector.
• Collectors has many factory methods for collectors:

  List<String> result = stream.collect(Collectors.toList());
  Set<String> result = stream.collect(Collectors.toSet());

• Can control the type of collection:

  TreeSet<String> result = stream.collect(Collectors.toCollection(TreeSet::new));

• Concatenate stream elements:

  String result = stream.collect(Collectors.joining(", "));

• summarizingInt, summarizingLong, summarizingDouble yield object with methods sum, average, max, min:

  IntSummaryStatistics summary = stream.collect(Collectors.summarizingInt(String::length));
  double averageWordLength = summary.getAverage();
  int maxWordLength = summary.getMax();

lesson02/collecting

Collecting into Maps

• Pass key and value extractors to Collectors.toMap:

  Map<Integer, String> idToName = people.collect(
      Collectors.toMap(Person::getId, Person::getName));
  

• Commonly, the values should be the elements themselves:

  Map<Integer, Person> idToPerson = people.collect(
      Collectors.toMap(Person::getId, Function.identity()));

• Can supply a function to resolve key conflicts:

  Stream<Locale> locales = Stream.of(Locale.getAvailableLocales());
  Map<String, String> languageNames = locales.collect(
      Collectors.toMap(Locale::getCountry,
          Locale::getDisplayLanguage,
          (existingValue, newValue) -> existingValue));

Grouping and Partitioning

• Instead of discarding key duplicates, you can collect all matching values:

  Map<String, List<Locale>> countryToLocales = locales.collect(
      Collectors.groupingBy(Locale::getCountry));

• Groups all elements with the same value of the classifier function:

  List<Locale> swissLocales = countryToLocales.get("CH");
      // Yields list of locales [it_CH, de_CH, fr_CH, ...]

• Use partitioningBy when the classifier function has a boolean result:

  Map<Boolean, List<Locale>> englishAndOtherLocales = locales.collect(
      Collectors.partitioningBy(l -> l.getLanguage().equals("en")));
  List<Locale>> englishLocales = englishAndOtherLocales.get(true);

jshell

Stream<Locale> locales() { return Stream.of(Locale.getAvailableLocales()); }

locales().collect(Collectors.toMap(Locale::toString, Locale::getDisplayLanguage))

locales().collect(Collectors.toMap(Locale::getCountry, Locale::getDisplayLanguage))
    // Exception—duplicate keys

Map<String, List<Locale>> countryToLocales = locales().collect(
    Collectors.groupingBy(Locale::getCountry));
countryToLocales.get("CH")
    // See http://archive.ethnologue.com/15/show_language.asp?code=gsw

Downstream Collectors

• groupingBy produces a list for each key.
• Can process these lists with “downstream collector”:

  Map<String, Set<Locale>> countryToLocaleSet = locales.collect(
      groupingBy(Locale::getCountry, toSet()));

• Use static import of java.util.stream.Collectors.* to make the expressions easier to read.
• Examples of useful downstream collectors:

  • Map<String, Long> countryToLocaleCounts = locales.collect(
        groupingBy(Locale::getCountry, counting()));

  • Map<String, Integer> stateToCityPopulation = cities.collect(
        groupingBy(City::getState, summingInt(City::getPopulation)));

  • Map<String, Set<String>> countryToLanguages = locales.collect(
        groupingBy(Locale::getDisplayCountry,
            mapping(Locale::getDisplayLanguage, toSet())));

lesson02/downstream

Java 9/10 News Flash - Collectors

• New collectors filtering, flatMapping:

  Map<String, Set<City>> largeCitiesByState
      = cities.collect(
          groupingBy(City::getState,
              filtering(c -> c.getPopulation() > 500000,
                  toSet())));

  • Why not just cities.filter(...).collect(...)?
  • Gives empty sets for states without large cities.
• Java 10 has collectors toUnmodifiableList, toUnmodifiableSet, toUnmodifiableMap

Understand the concept of reduction operations

Reduction Operations

• reduce is a general mechanism for computing a value from a stream.
• Simplest form repeats applying a binary function:

  List<Integer> values = ...;
  Optional<Integer> sum = values.stream().reduce((x, y) -> x + y);

• In general, computes v₀ op v₁ op v₂ op . . .
• Can supply identity element:

  List<Integer> values = ...;
  Integer sum = values.stream().reduce(0, (x, y) -> x + y)
      // Computes 0 + v0 + v1 + ...

• Now the return type is no longer Optional.

Complex Reductions

• Additional forms of reduce for more complex operations.
• Example: Sum lengths of all strings.
• Can't use simple form because we need a function (Int, String) -> Int.
• Also, when operation is parallelized, we need to combine partial results:

  int result = words.reduce(0,
      (total, word) -> total + word.length(),
      (total1, total2) -> total1 + total2);

• In practice, it's simpler to use map(String::length) and compute the sum.

Work with streams of primitive type values

Primitive Type Streams

• So far, we used Stream<T>, where T is a class (or interface).
• Stream<Integer>, Stream<Double>, etc. are inefficient.
• Use specialized types IntStream, LongStream, DoubleStream instead.
• Also have specialized types OptionalInt, OptionalLong, OptionalDouble
  • Instead of get, call getAsInt, getAsLong, and getAsDouble

Creating Primitive Type Streams

• From varargs and arrays:

  IntStream stream = IntStream.of(1, 1, 2, 3, 5);
  stream = Arrays.stream(values, from, to);

• IntStream and LongStream ranges:

  IntStream zeroToNinetyNine = IntStream.range(0, 100);
  LongStream zeroToTenBillion = LongStream.rangeClosed(0, 10_000_000_000L);

• Random numbers: Random.ints, Random.longs, Random.doubles
• Unicode characters:

  String sentence = "\uD835\uDD46 is the set of octonions.";
  IntStream codes = sentence.codePoints();
      // A stream with elements 0x1D546 0x20 0x69 0x73 0x20 . . .

• From an object stream with mapToInt, mapToLong, mapToDouble:

  IntStream lengths = words.mapToInt(String::length);
  

Primitive Type Stream Methods

• Most methods are analogous to those of object streams.
• The toArray method returns int[], long[], double[]
• Methods with optional result yield OptionalInt etc.
• There are methods sum, average, max, min
• summaryStatistics yields IntSummaryStatistics etc. which simultaneously provide all four results.
• boxed yields an object stream:

  Stream<Integer> integers = IntStream.range(0, 100).boxed();
  

jshell

IntStream.range(0, 10).map(n -> n * n).forEach(System.out::println)
new Random().ints().limit(1000).toArray()

String hello = "Hello \uD83D\uDE3B";
Arrays.toString(hello.codePoints().toArray())

new Random().ints().map(n -> n % 100).limit(1000).max()
new Random().ints().map(n -> n % 100).limit(1000).summaryStatistics()

Speed up stream operations with parallel streams

Parallel Streams

• Parallel streams compute stream results concurrently.
• Faster with large in-memory streams.
• Turn any collection or stream into a parallel stream:

  Stream<String> parallelWords = words.parallelStream();
  Stream<String> parallelWords = Stream.of(wordArray).parallel();

• Beware of race conditions:

  int[] shortWords = new int[12];
  words.parallelStream().forEach(
      s -> { if (s.length() < 12) shortWords[s.length()]++; });
      // Error—race condition!
  System.out.println(Arrays.toString(shortWords));

• Instead, use a stream pipeline:

  Map<Integer, Long> shortWordCounts =
  words.parallelStream()
      .filter(s -> s.length() < 12)
      .collect(groupingBy(String::length, counting()));

Ordering

• Streams from arrays, lists, ranges, generators, iterators, or Stream.sorted are ordered.
• Results are accumulated in stream order, and computations are deterministic.
• Generally, ordering doesn't preclude concurrent processing.
  • For example, stream.map(fun) can be done in segments that are reassembled in order.
• Some operations (distinct, limit) benefit from dropping ordering:

  Stream<String> sample = words.parallelStream().unordered().distinct().limit(n);

• Merging maps is expensive. Remedy: Use Collectors.groupingByConcurrent

  Map<Integer, List<String>> result = words.parallelStream().collect(
      Collectors.groupingByConcurrent(String::length));
      // Values aren’t collected in stream order

lesson02/parallel

Java 9 News Flash - Files.lines

• Files.lines now uses a memory mapped file.
• Efficient with parallel streams.

  Files.lines(pathOfHugeFile)
     .parallel()
     .filter(l -> l.contains("Exception"))
     ...