Virtual Threads, Structured Concurrency, Scoped Values:
When to Use Them, and How

.png

Use QR code for Q&A

☟ Download source code

About the Speaker

.jpg

Cay Horstmann
Author of Core Java (13 editions since 1996), Java/JavaScript/Scala for the Impatient, etc.
Java Champion
https://horstmann.com

Virtual Threads

.png

Now two kinds of java.lang.Thread: platform (OS) and virtual
Virtual thread mounted onto carrier OS thread
When virtual thread blocks, it is unmounted; carrier thread mounts another virtual thread
- “Blocking is cheap”
“virtual”: supposed to evoke virtual memory that is mapped to actual RAM
- Do not think “virtual function”
- Naming is hard...
Didn't we have all that with “green threads” way back in Java 1.0?
- When a green thread blocked, it blocked the carrier thread,
JEP 444, final in Java 21

Scoped Values

.png

Share thread-specific values in call stack and subthreads
Similar to thread-local variables
But guaranteed clean up

Example: Framework executes actions, provides context info as scoped value

public class Framework {
    private static final ScopedValue<Request> REQUEST = ScopedValue.newInstance();
    private LinkedHashMap<String, Callable<Response>> actions = new LinkedHashMap<>();
    ...
    public Response execute(Request request, Callable<Response> action) { // In some thread...
        var action = actions.get(request.getServletPath()); // Ignoring path matching...
        return ScopedValue.where(REQUEST, request).call(action); // Bind value
        // Value unbound here
    }
}

Scoped value can be retrieved in call stack, subthreads

    ...
    actions.put(path, () -> {
        var request = Framework.REQUEST.get();
        ...
    });

JEP 506, final in Java 25

Structured Concurrency

Nathaniel Smith: “Start thread and let it run” is like goto
1960s: Structured programming replaces goto with branches, loops, functions
Structured concurrency: Do the same with concurrent tasks

JEP 505, Fifth preview in Java 25

try (var scope = StructuredTaskScope.open()) {
   Supplier<T1> subtask1 = scope.fork(callable1); // runs on virtual thread
   Supplier<T2> subtask2 = scope.fork(callable2); // runs on another virtual thread
   scope.join(); // blocks until completion
   // Process subtask1.get(), subtask2.get()
}

If either subtask fails, the other is canceled and join fails
Can pass alternative completion policy to open

Why Virtual Threads?

.webp

🚫 Not to run a million concurrent tasks
🚫 Not to run tasks faster
👍 To increase throughput when:
- Many more tasks than platform threads
- Mostly blocking

To enable a “direct” programming style:
🚫

client.sendAsync(request, HttpResponse.BodyHandlers.ofString())
   .thenApply(HttpResponse::body)
   .thenApply(this::getImageURLs)
   .thenCompose(this::getImages)
   .thenAccept(this::saveImages)
   .exceptionally(this::ohNoes);

👍

try {
   var response = client.send(request, HttpResponse.BodyHandlers.ofString());
   for (URL url : getImageURLs(response.body())) {
      saveImage(getImage(url));
   }
}
catch (...) { ... }

“Hello World” of Virtual Threads

.jpg

There is an API for creating, starting virtual threads
We don't care since application programmers rarely create their own threads
Frameworks have their own ways to choose between virtual and platform threads
Spring Boot has global configuration:
```
spring.threads.virtual.enabled=true
```

In Quarkus, annotate each service:

@Path("/sleep")
public class Sleep {
    @Path("/virtual")
    @RunOnVirtualThread
    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public String virtual() {
        try {
            Thread.sleep(1_000);
        } catch (InterruptedException __) {
        }
        return Thread.currentThread() + "\r\n";
    }
}

Observe routes /platform and /virtual

Presentation code

cd ~/projects/2025-jcon
mvn install -Dmaven.test.skip=true
java --enable-preview -jar target/quarkus-app/quarkus-run.jar

curl localhost:8080/sleep/platform
curl localhost:8080/sleep/virtual

Throughput

.png

Classic app server has thread pool
Incoming task is submitted to pool, queued until pool thread is available
When the task blocks, the pool thread blocks and cannot do any useful work
Virtual threads are not pooled
Incoming task starts running on virtual thread right away
When it blocks, the carrier thread serves another virtual thread

Do Your Tasks Block?

.webp

bombardier --requests=1000 --connections=1000  --timeout=60s --print=i,p,r http://localhost:8080/sleep/platform
bombardier --requests=1000 --connections=1000 --timeout=60s --print=i,p,r http://localhost:8080/sleep/virtual

Virtual threads great for blocking tasks
But no advantage for tasks that are processor-intensive

curl localhost:8080/search/platform/record
bombardier --requests=100 --connections=100 --timeout=60s --print=i,p,r http://localhost:8080/search/platform/record
bombardier --requests=100 --connections=100 --timeout=60s --print=i,p,r http://localhost:8080/search/virtual/record

Big disadvantage: Virtual threads cannot be preempted
Don't use them for non-blocking tasks

Watching Threads

.png

Run jcmd to get PID of server process

Capture a Java Flight Recorder recording:

jcmd pid JFR.start
bombardier ...
jcmd pid JFR.dump name=1 filename=dumpfile

Then load dump into Java Mission Control
Observe threads:
- Platform threads: Blocking on 200 executor threads
- Virtual threads: Fork join pool carrier threads move along

To see virtual threads, use

jfr configure jdk.VirtualThreadStart#enabled=true jdk.VirtualThreadEnd#enabled=true --output custom.jfc
jcmd pid JFR.start setting=custom.jfc

In JMC, also look at garbage collections
Virtual thread stack frames are collected

Memory Usage

.png

How much memory?

java  --enable-preview -jar target/quarkus-app/quarkus-run.jar
bombardier --requests=10000 --connections=10000 --timeout=60s --print=i,p,r http://localhost:8080/sleep/virtual
jcmd
jcmd pid VM.native_memory

Look for Thread

Web app with virtual threads:

-                    Thread (reserved=64705KB, committed=5353KB)
                            (threads #63)

With platform threads:

-                    Thread (reserved=191070KB, committed=19070KB)
                            (threads #186)

Platform thread has default stack of 1M (on 64 bit VM), change with -Xss
120 threads in pool: 120MB
Virtual thread stack sized to actual usage, stored on heap when parked
Could have large heap usage with lots of parked threads
Increased throughput isn't free
Observe and measure!

Rate Limiting

.webp

Platform threads have an implicit rate limiter: the thread pool
Virtual threads can make many more concurrent requests to feeble resources
Need mechanism for limiting concurrent access
E.g. database connection pool
YOU may need to do the rate limiting
Fetching random daily Wikipedia image
```
curl localhost:8080/image/1 | display
```

Let's fetch five hundred!

bombardier --timeout=60s --connections=500 --requests=500 --print=i,p,r localhost:8080/image/1

That was fast. But not so successful

  HTTP codes:
    1xx - 0, 2xx - 388, 3xx - 0, 4xx - 0, 5xx - 112

Remedy: Counting semaphore

    private static Semaphore sem = new Semaphore(20);
    ...
    sem.acquire();
    try {
         byte[] image = randomImage();
       return ...;
    } finally {
        sem.release();
    }

Now all calls succeed:

bombardier --timeout=60s --connections=500 --requests=500 --print=i,p,r localhost:8080/image/2

Pinning

.jpeg

Loom parks virtual thread when it blocks on network I/O, sleeping, locking
Except, up to Java 23, for intrinsic lock (synchronized)
Then the virtual thread is pinned to the carrier thread
Not a problem if critical section has short duration (e.g. ConcurrentHashMap)
Bad if the virtual thread blocks while holding the lock
The carrier thread cannot serve any other virtual thread
Can happen in 3rd party libraries (e.g. JDBC drivers)

Stack trace on first event:

java -Djdk.tracePinnedThreads=full --enable-preview -jar target/quarkus-app/quarkus-run.jar
curl localhost:8080/sleep/pin

Don't use -Djdk.tracePinnedThreads in production; look for VirtualThreadPinned events in JFR
Pinning also happens when calling native method or foreign function

Carrier thread pool is not expanded

bombardier --timeout=60s --connections=100 --requests=100 --print=i,p,r localhost:8080/sleep/pin

View threads in jconsole

Problem solved with Java 24

File Operations

.jpg

Socket I/O is fully asynchronous, parking virtual threads
File I/O is synchronous
Carrier thread pool may be temporarily expanded
Default max size is 256, can be changed with -Djdk.virtualThreadScheduler.maxPoolSize

Not easy to see expansion of carrier thread pool

bombardier --timeout=60s --connections=200 --requests=200 --print=i,p,r localhost:8080/search/concurrent/record
jconsole

Are there more ForkJoinPool-1-worker-n threads than cores?

You'll probably be fine doing file I/O on virtual threads
Observe and measure!

Is Reactive Dead?

.jpg

Typical introduction to reactive programming
- Reactive eliminates callback hell
- Reactive is declarative
- Reactive can handle backpressure
Virtual threads take care of callback hell
Declarative style often challenging with business logic
- In the tutorial example, what if we want to always add the suggestions to the favorites?
Bounded queues can handle backpressure

In trivial benchmark, Quarkus reactive is comparable with virtual threads

bombardier --timeout=60s --connections=10000 --requests=10000 --print=i,p,r http://localhost:8080/sleep/reactive

Some reports of better reactive performance
- More GC with virtual threads?
- Netty not optimized for virtual threads?

Structured Task Scopes

.svg

Get data from multiple async requests, combine results
Requests can run concurrently
Do you start your own threads to do that?
Then you'll like structured task scopes
Pluggable policies
Automatically handles cancellation, error propagation
Use them instead of ExecutorService.invokeAll/invokeAny

Thread Locals

.jpeg

Every thread has table of thread locals
Use case #1: Share non-threadsafe objects (SimpleDateFormatter, ObjectMapper)
Use case #2: Avoid passing contexts as method parameters
Ok with pool of a few hundred threads
Still ok with tens/hundreds of thousands of virtual threads?
How do you know if you use them?
Add -Djdk.traceVirtualThreadLocals=true to see when virtual thread sets thread local

Brute force: reflectively access fields

public class Thread {
    ...
    ThreadLocal.ThreadLocalMap threadLocals;
    ThreadLocal.ThreadLocalMap inheritableThreadLocals;
}

java --enable-preview --add-opens=java.base/java.lang=ALL-UNNAMED -jar target/quarkus-app/quarkus-run.jar
curl http://localhost:8080/threadlocal/platform
curl http://localhost:8080/threadlocal/virtual
curl http://localhost:8080/threadlocal/platform/verbose
curl http://localhost:8080/threadlocal/virtual/verbose

Scoped Values

.png

Not a drop-in replacement for thread-local variables
Significant efficiency gain over inheritable thread locals
Inherited in structured task scopes
Automatically unloaded

Scoped values still consume per-task memory Run jconsole and watch memory

curl http://localhost:8080/threadlocal/virtual/props
bombardier --timeout=60s --connections=100 --requests=10000 --print=i,p,r http://localhost:8080/threadlocals/platform/props
bombardier --timeout=60s --connections=100 --requests=10000 --print=i,p,r http://localhost:8080/threadlocals/virtual/props
bombardier --timeout=60s --connections=100 --requests=10000 --print=i,p,r http://localhost:8080/threadlocals/scoped/props

Observe and measure!

The End

.jpeg

If your workloads keep all cores busy, you don't care about virtual threads
Sweet spot for virtual threads: Tasks that block a lot
👍 Increased throughput
👍 Direct programming model
Modern web frameworks have support
Watch out for resource overload
Watch out for of pinning (file I/O, synchronized, native methods, foreign functions)
Use structured task scopes for your own concurrent code
Watch out for thread locals, maybe consider scoped values
Observe and measure!
Thank you!
Q&A

Virtual Threads, Structured Concurrency, Scoped Values: When to Use Them, and How

About the Speaker

Virtual Threads

Scoped Values

Structured Concurrency

Why Virtual Threads?

“Hello World” of Virtual Threads

Throughput

Do Your Tasks Block?

Watching Threads

Memory Usage

Rate Limiting

Pinning

File Operations

Is Reactive Dead?

Structured Task Scopes

Thread Locals

Scoped Values

The End

Virtual Threads, Structured Concurrency, Scoped Values:
When to Use Them, and How