Project Valhalla: Flat Value Types in Java

.jpg

☟ Download source code

About the Speaker

.jpg

Cay Horstmann
Author of Core Java (13 editions since 1996), Java/JavaScript/Scala for the Impatient, etc.
Java Champion
https://horstmann.com

OpenJDK Projects

.jpg

Amber: Programmer productivity
- Records
- Pattern matching
- String templates
Loom: Virtual threads
Panama: Foreign functions/memory
Leyden: Improve startup time
Valhalla: Value types

Values and References

.jpg

Ever since Java 1.0: Eight primitive types byte short int long float double boolean char
Everything else is a reference type
Different in so many ways:
- Primitive types are “flat”—no indirection
- And “dense”—no headers
- == compares primitives by value, not identity
- Primitives can't be null ⚠️
- Longer primitives can tear ⚠️
  - long and double on 32 bit VM
- Initialized with zero bits (0 or false) ⚠️

Why Do We Care?

.jpg

Would like class instances that are like primitives
E.g. currency, measures, date/time, short/long floats, points, complex, tuples
“Codes like a class, acts like an int”
Performance benefits
- Memory savings
- Better locality
- Less GC work
Avoid painful wrappers: Function<Integer, Boolean> vs. Predicate<Integer> vs. IntFunction<Boolean> vs. IntPredicate

Value Classes

.jpg

Value class gives up identity
==, hashCode derived from fields, not address
No synchronized methods
JVM may (but doesn't have to) optimize layout/performance

Many value classes will be records:

public value record Point(double x, double y) {
   public Point moveBy(double dx, double dy) { return new Point(x + dx, y + dy); }
}

JEP 401

Values and Objects

.png

Values: primitives, references...
...and now value objects
In registers, stack, heap (inside objects)
Objects: on heap

Aside: Using AI to Clean Hand-drawn Diagrams

Sorry about the hand-drawn diagram
I asked ChatGPT to clean it up

Building Valhalla

.jpg

Currently outdated build on https://jdk.java.net/valhalla/
No need to build for simple experiments—use the sandbox https://javaalmanac.io/jdk/valhalla/

Easy to build on Linux:

git clone https://github.com/openjdk/valhalla/
cd valhalla
bash configure

Install any missing packages
Run
```
make images
```
The JDK build is in valhalla/build/linux-x86_64-server-release/images/jdk

If you use SDKMAN!:

sdk install java valhalla /path/to/valhalla/build/linux-x86_64-server-release/images/jdk

Running Valhalla

.webp

value class syntax is supported in current Valhalla build

Be sure to enable preview:

javac --enable-preview --source 25 ...
java --enable-preview ...
jshell --enable-preview

Easy thing to try: Brownian motion

Point p = new Point(0, 0);
for (int i = 0; i < ITERATIONS; i++)
    p = p.moveBy(Math.random() - 0.5, Math.random() - 0.5);
System.out.println(p);

After 100_000_000 iterations:

time java --enable-preview withvalhalla.Brownian
Point[x=619.9117803091456, y=2227.141928668786]
real    0m7.806s

Timing without Valhalla: 0m8.178s
So, what does Valhalla buy us?

Object Allocations

.png

Object header on 64-bit VM: 12 bytes
Two doubles: 16 bytes
4 bytes padding for total of 32 bytes per Point heap object
100_000_000 points ≈ 2.98GiB
Pre-Valhalla: Garbage collector is busy

To see, run

java --enable-preview -XX:StartFlightRecording=filename=novalhalla.jfr novalhalla.Brownian

Load novalhalla.jfr into Java Mission Control
Open Memory tab
See allocations for 100_000_000 Point objects (close to 3GiB), busy GC
Repeat with Valhalla version
24MiB allocated, no GC activity 😻

JIT Code Disassembly

.png

But why isn't Valhalla faster?
Let's look at the generated code
Download hsdis-amd64.so from https://www.chriswhocodes.com/hsdis/ into lib directory
Set environment variable
```
export LD_LIBRARY_PATH=lib
```

Run with these flags:

-XX:+UnlockDiagnosticVMOptions -XX:+PrintAssembly \
-XX:+LogCompilation -XX:LogFile=logfilename

View the logs with JITWatch

java --enable-preview -jar jitwatch-ui-1.4.9-shaded-linux-x64.jar

Click Open Log, then Start
Opcodes at https://shell-storm.org/x86doc/

Hotspot

.jpg

javac generates bytecodes
java interprets bytecodes...
...and compiles them into machine code
C1 compiler: simple optimizations, speeds up startup time
C2 compiler: more complex optimizations for long-running programs
“Hot” methods are compiled (i.e. frequently called—C1: 1,500, C2 10,000)
“Hot” loops are compiled
Methods may be inlined, loops unrolled
In our case:
- novalhalla doesn't inline call to Point.moveBy
- valhalla inlines call to Point.moveBy
- But Math.random is much more expensive 😞

Benchmarking

.webp

Use JMH (Java Microbenchmark Harness) for accurate timing, JVM warmup

Make project with

mvn archetype:generate \
  -DinteractiveMode=false \
  -DarchetypeGroupId=org.openjdk.jmh \
  -DarchetypeArtifactId=jmh-java-benchmark-archetype \
  -DgroupId=com.horstmann \
  -DartifactId=valhalla-benchmark \
  -Dversion=1.0

In POM, set javac.target to 25 and add this inside configuration of maven-compiler-plugin:

<compilerArgs>
   <arg>-verbose</arg>
   <arg>--enable-preview</arg>
   <arg>-proc:full</arg>
   <arg>--add-exports</arg>
   <arg>java.base/jdk.internal.vm.annotation=ALL-UNNAMED</arg>
</compilerArgs>

Compile and run:

mvn clean install
java --enable-preview -jar target/benchmarks.jar Benchmark1

Too Good to Be True?

.jpg

Instead of random movement, transform tweaks and swaps x and y:

/* value */ record Point(double x, double y) {
    ...
    public Point transform() { return new Point(y + 1, 0.5 - x); }
}

Set p = p.transform() 10_000_000 times

Amazing result 🤩

Benchmark                         Mode  Cnt      Score      Error  Units
c.h.novalhalla.Benchmark1.main    avgt   25  69213.839 ± 1526.112  us/op
c.h.withvalhalla.Benchmark1.main  avgt   25      0.006 ±    0.001  us/op

Valhalla is good, but can it be that good?

Fixing the Benchmark

.jpg

With Valhalla, inlined call to transform unrolls loop
And Hotspot notices that p.transform().transform().transform().transform() equals p!
Exercise: Watch JIT
Remedy: More complex transform

If using random numbers, precompute them

Benchmark                         Mode  Cnt      Score      Error  Units
c.h.novalhalla.Benchmark2.main    avgt   25  47725.264 ± 6597.810  us/op
c.h.withvalhalla.Benchmark2.main  avgt   25  24847.389 ± 3120.296  us/op

Solid result: Value type are performant 👍
And remember the lower memory footprint! 😻

Null-Restricted Types

.png

First ⚠️—value type variables can be null:
```
Point target = null;
```
Can often be optimized locally, but not inside objects, arrays
Flattened layout needs a bit for nullness
Null-restricted type: Value class type that can't be null:
```
Point! origin = new Point(0, 0); // Syntax may change
```
Compiler forbids null assignment:
```
origin = null; // Compile-time error
```

Also run-time check:

Point![] path = ...;
Object[] objs = path;
objs[0] = null; // NullPointerException

Will come to all types, not just value types: https://openjdk.org/jeps/8303099

Strict Initialization

.jpeg

Local variables are always explicitly initialized (definite assignment)
What about non-null fields and array elements?
Primitives initialized with zero bits (second ⚠️)
Not appropriate for all values, ex. LocalDate!
Null-restricted fields must be strictly initialized (https://openjdk.org/jeps/8350458)
Instance field: In constructor before call to super(...), or with initializer expression, or assigned in object initializer
Static field: With initializer expression, or assigned in static initializer

Array element: Provide initial value

String![] labels;
labels = new String![]{ "x", "y", "z" };
labels = new String![100]{ "" }; // strawman syntax
labels = new String![100]{ i -> "x"+i }; // strawman syntax

The default value (zero bits) can never be observed!
With JEP 482, field assignment before super possible for arbitrary classes

Trying Phase 2 Features

.jpg

Current Valhalla build supports phase 1 syntax (value modifier)
Non-null ! will come in phase 2
Need annotations in jdk.internal.vm.annotation package for phase 2 features
@NullRestricted on a field (instead of !)
@Strict on a field that is strictly initialized
@LooselyConsistentValue to allow tearing
jdk.internal.value.ValueClass lets you construct null-restricted arrays

To access internals:

javac --add-exports java.base/jdk.internal.vm.annotation=ALL-UNNAMED \
  --add-exports java.base/jdk.internal.value=ALL-UNNAMED ...
java --add-exports java.base/jdk.internal.value=ALL-UNNAMED ...

Atomic Updates

.jpg

References, primitive types ≤ 32 bits are updated atomically
Longer values may tear (second ⚠️ on the intro slide)
A thread may read/write half-updated value
Tearing more likely for longer flattened values
By default, value type updates are atomic
- Today: by not flattening values

Must opt in to allow tearing: (Syntax may change)

value record Point(double x, double y)
   implements LooselyConsistentValue {}

.png

Then users of the class are on the hook: thread confinement, volatile, locks

Tearing

time java --add-exports java.base/jdk.internal.value=ALL-UNNAMED --enable-preview  withtearing.Main
Point[x=7.0, y=6.0]
Point[x=13.0, y=14.0]
...
real    0m0.367s

time java --add-exports java.base/jdk.internal.value=ALL-UNNAMED --enable-preview notearing.Main
real    0m27.675s

Memory Consumption

.jpg

Sleep after allocating value objects/arrays:

Point[] path = (Point[]) ValueClass.newNullRestrictedNonAtomicArray(Point.class, NPOINTS, new Point(0, 0));
for (int i = 1; i < path.length; i++) {
   path[i] = path[i - 1].transform();
}
System.out.println(path[path.length - 1]);
Thread.sleep(60_000);

Value type must have single field or be loosely consistent

javac --add-exports java.base/jdk.internal.vm.annotation=ALL-UNNAMED \
    --add-exports java.base/jdk.internal.value=ALL-UNNAMED --enable-preview --source 25 \
    withflattening/Points.java
java --add-exports java.base/jdk.internal.value=ALL-UNNAMED --enable-preview withflattening.Points

Run jcmd:

jcmd withflattening.Points GC.class_histogram | head

Flattened array with 200_000_000 double 🎉

num     #instances         #bytes  class name (module)
-------------------------------------------------------
   1:             1     1600000016  [Lwithflattening.Point;

Without flattening:

 num     #instances         #bytes  class name (module)
-------------------------------------------------------
   1:     100000000     3200000000  noflattening.Point
   2:             1      400000016  [Lwithvalhalla.Point;

What's You'll Get

.jpg

value modifier
! syntax
Marker interface for “ok if it tears”
Standard library classes considered as value classes even though not so declared:
- Wrappers Integer, Double, etc.
- Optional, OptionalInt, etc.
- java.time classes LocalDate, Instant, etc.
That's it.
“Virtuous collapse” after so many things were tried
May open the door for more progress: deconstruction, templates, ...
Next can of worms: generics

Generics

.jpg

Should ArrayList<int> use int or Object bytecodes?
Array creation for int[] or Integer![] is different than for Object[]
Is List<int> a subtype of the raw List or wildcard List<?> type?
Constraining to value types? (Optional<T!>)
get method of Map<String, int> can't return null

Overloading:

static <T extends Comparable<? super T>> void sort(T[] a)
static void sort(int[] a)

Universal generics first, then specialized generics

History Notes

.jpg

Valhalla was born July 10, 2014
Surface syntax has changed many times, e.g. inline classes
There used to be different bytecodes for references L and values Q—no more
There used to be special bytecodes withfield and defaultvalue—no more
There used to be “implicit constructors” to agree to zero default—no more
Expect more twists and turns with generics

Conclusion

.jpeg

After more than ten years of prototyping, value types will come to a JVM near you real soon now
Some workloads will see significant performance improvements
Primitive types will be approximated by null-restricted wrappers
Conversion/overloading rules will get (even more) complex, but most programmers won't need to care
Generics? Still a way to go before List<int> is backed by an int[]
When value types come, benchmark to see if they are for you

Project Valhalla: Flat Value Types in Java

About the Speaker

OpenJDK Projects

Values and References

Why Do We Care?

Value Classes

Values and Objects

Aside: Using AI to Clean Hand-drawn Diagrams

Building Valhalla

Running Valhalla

Object Allocations

JIT Code Disassembly

Hotspot

Benchmarking

Too Good to Be True?

Fixing the Benchmark

Null-Restricted Types

Strict Initialization

Trying Phase 2 Features

Atomic Updates

Memory Consumption

What's You'll Get

Generics

History Notes

Further Reading

Conclusion