Path 4 · lesson 5 of 5
Virtual threads in Java 21
What a virtual thread is, how carriers mount and unmount it, why it helps blocking I/O only, what pinning is in JDK 21, and why the bottleneck moves to the downstream limit.
What you will learn
- Describe a virtual thread in terms of its carrier, its scheduler and where its stack lives while it is blocked
- Predict from a workload whether virtual threads will raise throughput, and explain why they do nothing for CPU-bound work
- Find pinning on JDK 21 with -Djdk.tracePinnedThreads and the jdk.VirtualThreadPinned JFR event, and state what JDK 24 changed
- Explain why throughput stays the same when a downstream limit is the bottleneck, and find that limit
- Bound concurrency with a Semaphore instead of a thread pool, and say what the bound costs
- List what spring.threads.virtual.enabled switches in Spring Boot 3.2 and later, and what it leaves alone
Before you start
The concept
Until Java 21 every java.lang.Thread was a thin wrapper around one operating system thread. Those are now called platform threads. A virtual thread is also a java.lang.Thread, but it is implemented by the JDK and is not tied to one OS thread. It runs your code on a platform thread only while it is using the CPU. When it blocks, the JDK takes it off that platform thread and stores its stack on the Java heap, and the platform thread goes on to run a different virtual thread.
| Platform thread | Virtual thread | |
|---|---|---|
| Backed by | One OS thread for its whole life | Any carrier thread while running, none while blocked |
| Stack | Reserved outside the heap when the thread starts (1,024 KB by default on Linux x64) | Heap objects that grow and shrink with the call depth |
| Scheduled by | The operating system | The JDK: a ForkJoinPool of carrier threads |
| Sensible count | Hundreds to a few thousand | As many as there are concurrent tasks, up to millions |
| Creation | new Thread, Thread.ofPlatform(), thread pools | Thread.ofVirtual(), Thread.startVirtualThread, Executors.newVirtualThreadPerTaskExecutor() |
| Daemon and priority | Configurable | Always a daemon thread, always NORM_PRIORITY |
In jcmd Thread.print | Listed | Not listed. Use jcmd Thread.dump_to_file |
Virtual threads became a final feature in Java 21 with JEP 444. Nothing changes in how you write the code that runs on them: the same blocking calls, the same exceptions and stack traces, the same ThreadLocal, the same debugger. What changes is the cost of a thread that waits.
Program output100,000 blocked virtual threads in one process (VirtualThreadsJava21.java many)
Real JVM outputavailableProcessors=2 before: 18 OS threads in the process 100,000 virtual threads started and blocked after 1,235 ms: 21 OS threads in the process all 100,000 finished after 3,494 ms
Why it exists
A Spring MVC service handles each request on one thread from start to finish. Most of that time the thread is not computing: it waits for a database, another service or a message broker. With platform threads the number of requests in progress can never exceed the number of threads, and OS threads are too expensive to have tens of thousands of. So the thread pool (200 threads in Tomcat by default) becomes the limit long before the CPU does, and requests queue behind threads that are only waiting.
Program output2,000 requests that each wait 100 ms on a remote call, no downstream limit (labs/jvm/virtual-threads)
Real JVM outputplatform threads (pool of 200), no downstream limit: 2,000 requests in 1,027 ms = 1,947 requests/s virtual threads, no downstream limit: 2,000 requests in 127 ms = 15,748 requests/s
Program output8 CPU-bound tasks on 2 CPUs: a platform pool of 2 against one virtual thread per task (VirtualThreadsJava21.java cpu)
Real JVM outputavailableProcessors=2 one task alone: 163 ms of CPU work, so 8 tasks need about 1,304 ms of CPU time platform pool of 2: 8 CPU-bound tasks in 661 ms virtual thread per task: 8 CPU-bound tasks in 666 ms
How it works inside
Carriers and the scheduler
The platform thread a virtual thread is currently running on is its carrier. Carriers belong to the virtual-thread scheduler, which JEP 444 describes as a work-stealing ForkJoinPool that operates in FIFO mode. It is a separate pool from the common pool of the previous lesson. Its parallelism, the number of carriers, defaults to the number of available processors and can be set with jdk.virtualThreadScheduler.parallelism. In a thread dump the carriers are the threads named ForkJoinPool-1-worker-N.
- Mount. The scheduler assigns a runnable virtual thread to a carrier. The virtual thread's frames are placed on the carrier's stack and it runs.
- Unmount. When the virtual thread blocks in the JDK, for example on a socket read,
Thread.sleep,BlockingQueue.take(), aReentrantLockor aSemaphore, its frames are moved to the heap and the carrier is free for another virtual thread. - Resume. When the operation can complete (bytes arrived, the lock was released), the virtual thread is submitted back to the scheduler. It may continue on a different carrier. Code cannot tell:
Thread.currentThread()always returns the virtual thread. - No time slicing. The scheduler does not preempt a virtual thread that keeps computing. It gives up its carrier only when it blocks or finishes.
A few blocking operations do not unmount. In JDK 21 these are many file system operations (an operating system limitation) and Object.wait(). For these the scheduler compensates by temporarily adding a carrier, so the number of platform threads can exceed the processor count, up to jdk.virtualThreadScheduler.maxPoolSize, which defaults to 256.
Pinning in JDK 21
A virtual thread is pinned when it cannot be unmounted. JEP 444 lists two cases: it is executing code inside a synchronized block or method, or it is executing a native method or a foreign function. If a pinned virtual thread blocks, its carrier blocks with it, and the scheduler does not add a carrier to compensate. With 2 carriers, two pinned and blocked virtual threads stop every other virtual thread in the process.
Program output40 virtual threads sleep 100 ms inside synchronized, run with -Djdk.tracePinnedThreads=short (labs/jvm/virtual-threads)
Real JVM outputVirtualThread[#19]/runnable@ForkJoinPool-1-worker-1 reason:MONITOR
VirtualThreads.lambda$pinned$1(VirtualThreads.java:61) <== monitors:1
pinned: 40 virtual threads sleeping 100 ms inside synchronized took 2,011 ms on 2 carrier threads (unpinned it would take about 100 ms)-Djdk.tracePinnedThreads=short prints: the virtual thread, its carrier (ForkJoinPool-1-worker-1), the reason (MONITOR) and the frame that holds the monitor, with file and line number.Program outputThe same sleep inside synchronized and inside a ReentrantLock (VirtualThreadsJava21.java lock)
Real JVM outputavailableProcessors=2 synchronized: 40 virtual threads each sleeping 100 ms while holding the lock: 2,087 ms ReentrantLock: 40 virtual threads each sleeping 100 ms while holding the lock: 106 ms
synchronized the batch takes 2,087 ms. Under a java.util.concurrent.locks.ReentrantLock it takes 106 ms, because a virtual thread that blocks while holding a ReentrantLock unmounts normally. This is the change JEP 444 recommends for JDK 21, and only where a synchronized block guards a blocking operation that runs often.JFRjfr summary and jfr print --events jdk.VirtualThreadPinned for the pinned run
Real JVM output jdk.VirtualThreadPinned 40 560
jdk.VirtualThreadStart 0 0
jdk.VirtualThreadEnd 0 0
jdk.VirtualThreadSubmitFailed 0 0
jdk.VirtualThreadPinned {
startTime = 21:44:11.023 (2026-11-15)
duration = 101 ms
eventThread = "" (javaThreadId = 23, virtual)
stackTrace = [
java.lang.VirtualThread.parkOnCarrierThread(boolean, long) line: 689
java.lang.VirtualThread.parkNanos(long) line: 648
java.lang.VirtualThread.sleepNanos(long) line: 807
java.lang.Thread.sleep(long) line: 507
VirtualThreads.lambda$pinned$1(Object) line: 61
...
]
}jfr summary: 40 jdk.VirtualThreadPinned events, one per task. That event is enabled by default with a threshold of 20 ms, so a normal production recording already contains it. jdk.VirtualThreadStart and jdk.VirtualThreadEnd show 0 because they are disabled by default. The event below gives the duration of the pinned block (101 ms) and the stack: parkOnCarrierThread is the JDK parking the virtual thread without releasing the carrier, and the first application frame is the line to fix. Unlike the system property, this needs no restart with a new flag if a recording is already running.| JDK 21 to 23 | JDK 24 and later (JEP 491) | |
|---|---|---|
Blocking while inside synchronized | Pins the carrier | Unmounts; the carrier is released |
Blocking to enter a synchronized block held by another thread | Blocks the carrier in the JVM | Unmounts |
Object.wait() | Does not unmount; the scheduler compensates with a spare carrier | Unmounts |
| Blocking inside a native method or foreign function frame | Pins | Still pins |
| Blocking in a class initializer or while loading a class | Pins (native frame on the stack) | Still pins; JEP 491 expects these to be rare |
-Djdk.tracePinnedThreads | Prints a stack trace when a thread blocks while pinned (full or short) | Removed; setting it has no effect |
jdk.VirtualThreadPinned JFR event | Recorded when a thread parks while pinned | Kept for the remaining cases, with the reason and the carrier added |
Thread-locals and pooling
- Never pool virtual threads. A pool exists to share something expensive. A virtual thread is cheap and is meant to run one task and end. JEP 444: there is never a need to pool them.
- Thread-locals work, but not as a cache. Code that keeps an expensive object per thread (a buffer, a parser, a connection) relied on a few long-lived pooled threads. With one new virtual thread per task that object is created for every task and never reused. Share such objects another way.
- Request context is fine. A
ThreadLocalthat carries the current user or trace id for the duration of a request behaves as before. Scoped values, a preview API in Java 21 (JEP 446), are the intended replacement for that use. -Djdk.traceVirtualThreadLocals=trueprints a stack trace whenever a virtual thread sets a thread-local value, which helps to find the caching kind during a migration.
Thread dumps
Thread dumpThread names from jcmd <pid> Thread.print while 1,000 virtual threads are blocked
Real JVM output"main" #1 [653780] prio=5 os_prio=0 cpu=55.43ms elapsed=6.56s tid=0x000075765402a9f0 nid=653780 waiting on condition [0x0000757658cac000] ... "Common-Cleaner" #17 [653796] daemon prio=8 os_prio=0 cpu=0.24ms elapsed=6.40s tid=0x0000757654122230 nid=653796 waiting on condition [0x000075763124e000] "ForkJoinPool-1-worker-1" #19 [653797] daemon prio=5 os_prio=0 cpu=5.67ms elapsed=6.22s tid=0x000075765414e770 nid=653797 waiting on condition [0x000075763114e000] "ForkJoinPool-1-worker-2" #83 [653798] daemon prio=5 os_prio=0 cpu=4.49ms elapsed=6.21s tid=0x000075765414f9b0 nid=653798 waiting on condition [0x000075763104e000] "VirtualThread-unparker" #607 [653799] daemon prio=5 os_prio=0 cpu=0.13ms elapsed=6.20s tid=0x00007575c0069f20 nid=653799 waiting on condition [0x0000757630f4e000] "Attach Listener" #1021 [653893] daemon prio=9 os_prio=0 cpu=0.29ms elapsed=0.10s tid=0x00007575c4000ff0 nid=653893 waiting on condition [0x0000000000000000] ...
ForkJoinPool-1-worker-1 and -2, idle, and VirtualThread-unparker, the JDK thread that wakes timed sleeps. A classic dump of a service on virtual threads therefore looks almost empty during an incident. That is expected and it is not evidence that nothing is running.Thread dumpjcmd <pid> Thread.dump_to_file /tmp/threads.txt: the same moment
Real JVM output653778
2026-11-15T21:43:55.804313750Z
21.0.13+9-LTS
...
#19 "ForkJoinPool-1-worker-1"
java.base/jdk.internal.misc.Unsafe.park(Native Method)
java.base/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
java.base/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1893)
java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)
#83 "ForkJoinPool-1-worker-2"
java.base/jdk.internal.misc.Unsafe.park(Native Method)
java.base/java.util.concurrent.locks.LockSupport.parkUntil(LockSupport.java:449)
java.base/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1891)
java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)
#18 "" virtual
java.base/java.lang.VirtualThread.parkNanos(VirtualThread.java:635)
java.base/java.lang.VirtualThread.sleepNanos(VirtualThread.java:807)
java.base/java.lang.Thread.sleep(Thread.java:507)
VirtualThreadsJava21.lambda$hold$4(VirtualThreadsJava21.java:158)
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
java.base/java.lang.VirtualThread.run(VirtualThread.java:329)
#20 "" virtual
java.base/java.lang.VirtualThread.parkNanos(VirtualThread.java:635)
java.base/java.lang.VirtualThread.sleepNanos(VirtualThread.java:807)
java.base/java.lang.Thread.sleep(Thread.java:507)
VirtualThreadsJava21.lambda$hold$4(VirtualThreadsJava21.java:158)
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
java.base/java.lang.VirtualThread.run(VirtualThread.java:329)
...virtual after the (here empty) name. Each one shows where it is blocked: Thread.sleep reached from VirtualThreadsJava21.lambda$hold$4. The full file has 1,000 such entries. To find a bottleneck, group them by their top application frame: grep -A 6 'virtual#x27; /tmp/threads.txt | grep 'com.example' | sort | uniq -c | sort -rn. The format has no thread states, lock addresses or deadlock detection, so keep Thread.print for those. Add -format=json for a file that tools can read.Architecture
Drawing diagram…
Flowchart. Two thousand requests each have a virtual thread. The scheduler, a ForkJoinPool in FIFO mode, mounts runnable virtual threads on two carrier platform threads that run on two CPU cores. Blocked virtual threads are unmounted and parked on the heap. Most of them wait for one of 20 permits of a semaphore or connection pool in front of a downstream service. A reply makes a thread runnable again. A pinned virtual thread blocked inside synchronized keeps its carrier.
flowchart TD REQ["2,000 requests, one virtual thread each"] --> SCH["Scheduler, a ForkJoinPool in FIFO mode"] SCH --> C1["Carrier 1, platform thread"] SCH --> C2["Carrier 2, platform thread"] C1 --> CPU["2 CPU cores"] C2 --> CPU REQ -->|"blocked, so unmounted"| HEAP["Parked virtual threads, stacks on the heap, no OS thread held"] HEAP -->|"1,980 wait for a permit"| LIM["Semaphore or connection pool, 20 permits"] LIM -->|"20 calls in flight"| DOWN["Downstream service"] DOWN -->|"reply makes the thread runnable"| SCH PIN["Pinned virtual thread, blocked inside synchronized on JDK 21"] -->|"keeps its carrier"| C2
- The carriers are few and are sized for the CPUs. They are only needed for the short moments when a request actually computes.
- The box of parked virtual threads is what replaced the thread pool's queue. Waiting requests used to be tasks in a queue in front of 200 threads; now they are blocked virtual threads, which cost some heap each and nothing else.
- The limit in front of the downstream service is the only number in this picture that sets throughput for an I/O-bound request. With 20 permits and 100 ms per call that is 200 requests per second, however many virtual threads exist.
- The pinned thread is the exception to the model: it is blocked and still occupies carrier 2. With one more like it, nothing else in the process can run until one of them wakes up.
Step by step
Drawing diagram…
Sequence diagram. The scheduler mounts virtual thread A on carrier 1, which runs the request code. A issues a blocking socket read with no bytes available, parks, and its frames are copied to the heap. The scheduler mounts virtual thread B on carrier 1. When bytes arrive, A becomes runnable, the scheduler mounts it on carrier 2, it continues after the read, finishes and ends.
sequenceDiagram participant A as Virtual thread A participant S as Scheduler participant C1 as Carrier 1 participant N as Socket participant C2 as Carrier 2 S->>C1: mount A C1->>A: run the request code A->>N: blocking read, no bytes yet A->>S: park, frames are copied to the heap S->>C1: mount virtual thread B Note over C1: carrier 1 keeps working while A waits N-->>S: bytes arrived, A is runnable S->>C2: mount A C2->>A: continue after the read A-->>S: request finished, thread ends
- From the code's point of view the read was an ordinary blocking call that returned some bytes. The unmount and the move to another carrier are invisible to it.
- Carrier 1 was never idle. This is the whole gain: the wait costs a parked object on the heap instead of an OS thread.
- A resumed on carrier 2. Anything that assumes a stable OS thread, such as a native library keeping per-thread state, can be surprised by this.
- If A had made the read inside a
synchronizedblock on JDK 21, step four would not happen: A would stay on carrier 1 and B would have to wait for carrier 2.
In a real system
A team moves an order service to Java 21, sets spring.threads.virtual.enabled=true and runs the load test that used to top out at about 200 requests per second. It still tops out at about 200 requests per second. CPU is at 20%. The first reaction is that virtual threads are not working.
Program outputThe same 2,000 requests when the remote call goes through 20 downstream connections (labs/jvm/virtual-threads)
Real JVM outputplatform threads (pool of 200), 20 downstream connections: 2,000 requests in 10,019 ms = 199 requests/s virtual threads, 20 downstream connections: 2,000 requests in 10,039 ms = 199 requests/s
This is the central production lesson. Virtual threads move the bottleneck to the next limited resource: a JDBC connection pool (HikariCP defaults to 10 connections), an HTTP client's connection pool, a rate limit on a partner API, or the capacity of the downstream service. Throughput improves only if that limit moves, and it may be there for a good reason. The Investigate case virtual-threads-no-faster walks through this diagnosis.
| Before, platform threads | After, virtual threads |
|---|---|
| At most 200 requests are in progress. The rest wait in Tomcat's queue. | Every accepted request is in progress at once. |
| At most 200 threads can ask for a database connection or call a partner at the same moment. | Thousands can. The thread pool was an accidental concurrency limit for everything behind it, and it is gone. |
Overload shows up as a full pool: tomcat.threads.busy at the maximum, requests queueing. | Overload shows up downstream: connection-pool wait time, pool timeouts, HTTP 429 from partners, a database with too many active sessions. |
| A classic thread dump shows 200 threads and what each waits for. | A classic dump shows a few carriers. Use Thread.dump_to_file and group by frame. |
A resource without a limit of its own needs one from you. On platform threads the usual tool was a small thread pool. On virtual threads it is a Semaphore: the thread that wants to make the call acquires a permit, blocks cheaply if none is free, and releases it afterwards.
Program output2,000 virtual-thread requests with and without a Semaphore of 20 (VirtualThreadsJava21.java semaphore)
Real JVM outputavailableProcessors=2 no limit: 2,000 requests in 200 ms = 10,000 requests/s, peak concurrent downstream calls 2,000 Semaphore(20): 2,000 requests in 10,126 ms = 197 requests/s, peak concurrent downstream calls 20
sleep and does not mind; a real partner API or database would answer with errors, timeouts or a slowdown for everyone. Line 3: with Semaphore(20) the peak is exactly 20 and throughput is the 197 requests per second that 20 permits allow. The semaphore did not make anything faster. It chose where the waiting happens: in your process, cheaply and visibly, instead of inside the dependency.What spring.threads.virtual.enabled switches
| Area | With spring.threads.virtual.enabled=true on Java 21 or later |
|---|---|
| Tomcat and Jetty | Use virtual threads for request processing, so controller code runs on a virtual thread. server.tomcat.threads.max no longer has an effect. |
applicationTaskExecutor | Becomes a SimpleAsyncTaskExecutor that uses virtual threads. It serves @Async methods, Spring MVC asynchronous request handling and blocking execution in WebFlux. The spring.task.execution.pool.* properties are ignored. |
taskScheduler | Becomes a SimpleAsyncTaskScheduler that uses virtual threads and ignores pooling properties. |
| Messaging and data | A virtual thread executor is auto-configured for the RabbitMQ and Kafka listeners, and Spring Data Redis' ClusterCommandExecutor uses virtual threads. |
| Not changed | The size of the HikariCP pool, HTTP client connection pools, any ExecutorService bean you define yourself, the common pool, and the speed of your code. |
Java and Spring Boot
A Spring Boot 3.5 service on Java 21: the configuration that turns virtual threads on and bounds the database, a fan-out that uses one virtual thread per subtask, and a small plain-Java file with the thread API. The plain-Java file was compiled and run with JDK 21; the Spring classes were compiled only against hand-written stand-ins for the Spring types, not against the real libraries.
spring: threads: virtual: enabled: true # Tomcat request threads, @Async, scheduling: virtual threads datasource: hikari: maximum-pool-size: 20 # the real concurrency limit for database work connection-timeout: 2000 # ms to wait for a connection before failingmanagement: endpoints: web: exposure: include: health,metrics,threaddumpWhy it is written this way
spring.threads.virtual.enabledisfalseby default and needs Java 21 or later. With it on,server.tomcat.threads.maxhas no effect: there is no request thread pool to size.maximum-pool-sizeis now the number that decides database throughput. HikariCP's own default is 10. Size it from what the database can serve, not from the request rate.- HikariCP's default
connectionTimeoutis 30 seconds. With thousands of virtual threads able to wait for a connection, 30 seconds means a very long queue before the first error. Two seconds turns overload into a fast, visible failure. - The
metricsendpoint exposeshikaricp.connections.pending,hikaricp.connections.activeand thehikaricp.connections.acquiretimer, which are the first numbers to look at after this change.
package com.example.orders;import java.util.List;import java.util.concurrent.ExecutionException;import java.util.concurrent.ExecutorService;import java.util.concurrent.Executors;import java.util.concurrent.Future;import org.springframework.stereotype.Service;@Servicepublic class OrderSummaryService { private final CustomerClient customers; private final OrderRepository orders; private final LoyaltyClient loyalty; public OrderSummaryService(CustomerClient customers, OrderRepository orders, LoyaltyClient loyalty) { this.customers = customers; this.orders = orders; this.loyalty = loyalty; } /** Three blocking calls, made at the same time, written as plain sequential code. */ public OrderSummary summary(long customerId) throws InterruptedException { try (ExecutorService perTask = Executors.newVirtualThreadPerTaskExecutor()) { Future<Customer> customer = perTask.submit(() -> customers.fetch(customerId)); Future<List<Order>> recent = perTask.submit(() -> orders.findRecent(customerId, 10)); Future<Integer> points = perTask.submit(() -> loyalty.points(customerId)); return new OrderSummary(customer.get(), recent.get(), points.get()); } catch (ExecutionException e) { throw new SummaryUnavailableException(customerId, e.getCause()); } } public record OrderSummary(Customer customer, List<Order> recentOrders, int loyaltyPoints) { }}Why it is written this way
Executors.newVirtualThreadPerTaskExecutor()starts a new virtual thread for every submitted task. It is not a pool: nothing is reused and there is no size to tune.- The executor is created per call and closed by try-with-resources.
close()waits for the submitted tasks, so no subtask outlives the request. Creating it is cheap, unlike creating a platform thread pool. Future.get()blocks the calling virtual thread, which unmounts while it waits. Thejoin()inside a small pool that starved it in the CompletableFuture lesson is harmless here, because no scarce thread is held.- Each subtask still needs its real resource:
orders.findRecenttakes a HikariCP connection. Fan-out multiplies the demand on downstream limits by the number of subtasks per request. CustomerClient,OrderRepository,LoyaltyClientand the record types are assumed from the rest of the project.
import java.util.concurrent.ExecutorService;import java.util.concurrent.Executors;import java.util.concurrent.Future;public class VirtualThreadBasics { public static void main(String[] args) throws Exception { // 1. Start one directly. Thread direct = Thread.ofVirtual().name("report-1").start( () -> System.out.println("direct: " + Thread.currentThread())); direct.join(); // 2. One new virtual thread per submitted task. try (ExecutorService perTask = Executors.newVirtualThreadPerTaskExecutor()) { Future<String> result = perTask.submit(() -> { Thread.sleep(100); // unmounts: the carrier is free meanwhile return "executor: " + Thread.currentThread(); }); System.out.println(result.get()); } // 3. What a virtual thread is, as far as the API can tell. Thread probe = Thread.ofVirtual().unstarted(() -> { }); System.out.println("isVirtual=" + probe.isVirtual() + " isDaemon=" + probe.isDaemon() + " priority=" + probe.getPriority()); }}Why it is written this way
- Run it with
java VirtualThreadBasics.java. Printing a virtual thread shows its carrier after the@, for exampleVirtualThread[#19,report-1]/runnable@ForkJoinPool-1-worker-1. That is how you see the scheduler's pool from inside. - Virtual threads have no name unless you give them one, which is why they appear as
""in dumps. Name the ones you start yourself. - The last line prints
isVirtual=true isDaemon=true priority=5. A virtual thread is always a daemon thread, so it does not keep the JVM alive. Spring Boot hasspring.main.keep-alivefor applications in which every remaining thread is virtual or daemon.
Wrong, then right
A stock lookup against a partner API that allows 20 concurrent calls per client. On platform threads it worked, protected without anyone noticing by Tomcat's 200 threads and a patient partner. Then virtual threads were switched on. The Spring classes follow Spring Boot 3.2 or later and were compiled only against hand-written stand-ins for the Spring types, not against the real libraries.
Incorrect implementation
package com.example.stock;import java.time.Instant;import org.springframework.stereotype.Service;import org.springframework.web.client.RestClient;// application.yml now has: spring.threads.virtual.enabled: true@Servicepublic class PartnerStockClient { private final RestClient partner; private final TokenClient tokens; private String token; private Instant tokenExpiry = Instant.MIN; public PartnerStockClient(RestClient.Builder builder, TokenClient tokens) { this.partner = builder.baseUrl("https://api.partner.example").build(); this.tokens = tokens; } /** Called from the controller, so since the migration: on a virtual thread, once per request. */ public int available(String sku) { return partner.get() .uri("/stock/{sku}", sku) .header("Authorization", "Bearer " + bearer()) .retrieve() .body(Integer.class); // nothing limits how many of these run at once } private synchronized String bearer() { if (Instant.now().isAfter(tokenExpiry)) { AccessToken fresh = tokens.fetch(); // blocking HTTP call while holding the monitor token = fresh.value(); tokenExpiry = fresh.expiresAt(); } return token; }}Why it is written this way
- Before the migration at most 200 request threads existed, so at most 200 partner calls could be in flight. That was never a designed limit, but it was a limit. It has been removed and nothing replaced it.
bearer()issynchronizedand makes an HTTP call inside. On JDK 21 the virtual thread doing the refresh is pinned, and every other virtual thread that callsbearer()meanwhile blocks its own carrier while it waits for the monitor.- The code compiles, passes its tests and behaves at low load exactly as before. Both problems need concurrency to appear.
TokenClientand the recordAccessToken(String value, Instant expiresAt)are assumed from the rest of the project.
Why it fails
- Under a traffic peak of 1,500 concurrent requests, 1,500 virtual threads call the partner at once. The partner allows 20 and answers the rest with HTTP 429 or lets them time out. Error rate rises exactly when traffic does, while the service's own CPU and memory look healthy. The old system would have queued the excess in Tomcat instead.
- Throughput does not go up even when the partner tolerates the burst: with 20 calls served at a time and 100 ms per call, the ceiling is 200 lookups per second, as the real run with 20 downstream connections shows. Virtual threads only changed where the other 1,480 requests wait.
- When the token expires, one virtual thread enters
bearer()and blocks in the HTTP refresh while pinned to its carrier. Every request needs the token, so the next virtual threads to arrive block on the same monitor, and on JDK 21 a virtual thread that blocks entering a monitor blocks its carrier too. With 2 or 4 carriers the scheduler has no free carrier within milliseconds, and no virtual thread in the whole process can run until the refresh returns: health checks, other endpoints, everything. If the token endpoint takes 5 seconds, the service is frozen for 5 seconds. - The evidence is thin unless you know where to look. A classic thread dump shows only carriers, stuck in
bearer().-Djdk.tracePinnedThreads=shortprints the frame with<== monitors:1, and a JFR recording hasjdk.VirtualThreadPinnedevents with that stack.
Corrected implementation
package com.example.stock;import java.time.Instant;import java.util.concurrent.Semaphore;import java.util.concurrent.TimeUnit;import java.util.concurrent.locks.ReentrantLock;import org.springframework.stereotype.Service;import org.springframework.web.client.RestClient;@Servicepublic class PartnerStockClient { /** The partner's documented limit for this client. The bound is explicit and lives next to the call. */ private static final int MAX_CONCURRENT_CALLS = 20; private final Semaphore permits = new Semaphore(MAX_CONCURRENT_CALLS, true); private final ReentrantLock tokenLock = new ReentrantLock(); private final RestClient partner; private final TokenClient tokens; private volatile AccessToken token = new AccessToken("", Instant.MIN); public PartnerStockClient(RestClient.Builder builder, TokenClient tokens) { this.partner = builder.baseUrl("https://api.partner.example").build(); this.tokens = tokens; } public int available(String sku) { acquirePermit(); try { return partner.get() .uri("/stock/{sku}", sku) .header("Authorization", "Bearer " + bearer()) .retrieve() .body(Integer.class); } finally { permits.release(); } } private void acquirePermit() { try { if (!permits.tryAcquire(2, TimeUnit.SECONDS)) { throw new PartnerBusyException("no partner permit within 2 s, " + permits.getQueueLength() + " waiting"); } } catch (InterruptedException e) { Thread.currentThread().interrupt(); throw new PartnerBusyException("interrupted while waiting for a partner permit"); } } private String bearer() { AccessToken current = token; if (Instant.now().isBefore(current.expiresAt())) { return current.value(); // the common path takes no lock at all } tokenLock.lock(); // a virtual thread blocked here unmounts try { if (!Instant.now().isBefore(token.expiresAt())) { token = tokens.fetch(); // blocking call under a ReentrantLock: no pinning } return token.value(); } finally { tokenLock.unlock(); } }}Why it is written this way
Semaphore(20, true)is the concurrency limit. A virtual thread waiting intryAcquireis parked on the heap and costs no carrier, so 1,480 waiting requests are cheap.truemakes it fair, so waiting requests are served in arrival order.tryAcquirewith a timeout bounds the wait. Without it, a slow partner makes the number of waiting requests, and the memory they hold, grow without limit.PartnerBusyExceptionshould map to HTTP 503 so that callers can back off.- The permit is released in
finally. A permit lost on an exception is a slow leak that ends with every request timing out inacquirePermit. ReentrantLockreplacessynchronizedaround the blocking refresh. Threads waiting for the lock unmount, so one slow refresh delays the requests that need the token and nothing else.- The token is read from a
volatilefield first, so the lock is only taken when a refresh is due. The check is repeated inside the lock because another thread may have refreshed while this one waited. PartnerBusyExceptionis an unchecked exception assumed from the rest of the project. Do not replace the semaphore withExecutors.newFixedThreadPool(20, Thread.ofVirtual().factory()): that pools virtual threads, which exist to be created per task.
Trade-offs
- Throughput to the partner is still 20 calls at a time. The fix makes the limit explicit and the overload behaviour controlled; it does not raise the ceiling. Only the partner, or a cache in front of it, can do that.
- Requests that do not get a permit within 2 seconds now fail with 503. That is a product decision: a fast refusal instead of a slow answer. Tune the timeout to what the caller can use.
- A fair semaphore gives predictable waiting order and somewhat lower throughput under heavy contention than an unfair one. With calls of 100 ms the difference does not matter; with very short critical sections it can.
ReentrantLockis more code thansynchronizedand needs thetry/finallydiscipline. On JDK 24 and latersynchronizedno longer pins (JEP 491), so this particular change is only needed while you run on JDK 21 to 23.- The limit is per process. Four replicas with 20 permits each send up to 80 concurrent calls. A limit shared across replicas needs a rate limiter outside the JVM or a smaller per-replica number.
Configuration
The JDK settings are system properties of the JDK reference implementation, read at start-up. The Spring properties are from Spring Boot 3.5. Most services need none of the JDK ones; the ones that matter are the limits behind the threads.
jdk.virtualThreadScheduler.parallelismDefault The number of available processors
- What it does
- The number of platform threads available for scheduling virtual threads, that is, the number of carriers.
- When to change it
- Leave it. Raising it to work around pinning hides the cause and puts more runnable threads on the same cores. In a container, check what the JVM thinks the processor count is.
jdk.virtualThreadScheduler.maxPoolSizeDefault 256
- What it does
- The maximum number of platform threads available to the scheduler, which it may reach when it compensates for operations that block the carrier, such as file I/O and Object.wait() in JDK 21.
- When to change it
- Leave it. If a dump shows dozens of
ForkJoinPool-1-worker-Nthreads, find the operation that blocks carriers (often heavy file I/O on virtual threads) and move that work to a platform thread pool.
-Djdk.tracePinnedThreads=full|shortDefault Not set (no tracing)
- What it does
- Prints a stack trace when a virtual thread blocks while pinned.
fullprints the complete stack and highlights native frames and frames holding monitors;shortprints only the problematic frames. - When to change it
- Use it on JDK 21 to 23 in a test environment when a migration is slower than expected. JEP 491 removed it in JDK 24, where setting it has no effect; use the
jdk.VirtualThreadPinnedJFR event there.
-Djdk.traceVirtualThreadLocals=trueDefault false
- What it does
- Triggers a stack trace when a virtual thread sets the value of any thread-local variable.
- When to change it
- Turn it on briefly during a migration to find code that caches expensive objects in thread-locals, which creates one object per task on virtual threads.
spring.threads.virtual.enabledDefault false
- What it does
- Whether to use virtual threads. On Java 21 or later it makes Tomcat and Jetty process requests on virtual threads and replaces the auto-configured task executor and scheduler with virtual-thread versions.
- When to change it
- Turn it on for services whose requests mostly wait on I/O, after you have set explicit limits and timeouts on every downstream resource. It does nothing useful for CPU-bound services.
server.tomcat.threads.maxDefault 200
- What it does
- Maximum amount of worker threads. The Spring Boot reference adds that it does not have an effect if virtual threads are enabled.
- When to change it
- Before the switch this number was also the hidden concurrency limit for everything behind the controllers. Write down what it protected and replace each protection with an explicit limit.
spring.task.execution.simple.concurrency-limitDefault none (documented: -1 indicates no concurrency limit at all)
- What it does
- Sets the maximum number of parallel accesses allowed on the
SimpleAsyncTaskExecutorthat Spring Boot auto-configures when virtual threads are enabled. - When to change it
- Set it when
@Asyncmethods fan out to a limited resource. It bounds all@Asyncwork together, so a semaphore next to the specific call is more precise.
spring.datasource.hikari.maximum-pool-sizeDefault 10 (HikariCP's default)
- What it does
- Maximum number of connections in the pool, idle and in use. When all are in use,
getConnection()blocks for up toconnectionTimeout(30 seconds by default) and then throws. - When to change it
- With virtual threads this is the throughput limit for database work. Size it from what the database can serve and lower
connection-timeout, because many more threads can now wait for it.
spring.main.keep-aliveDefault false
- What it does
- Whether to keep the application alive even if there are no more non-daemon threads.
- When to change it
- Set it to
truefor applications without a web server whose work runs only on virtual threads, for example scheduled jobs: virtual threads are daemon threads and do not keep the JVM running.
Hands-on exercise
Move the bottleneck, then find it again
Run the same blocking workload on a platform pool and on virtual threads, add a downstream connection limit and see that throughput is then identical, and finally switch on pinning to see carriers held.
Steps · 0 of 7 done
Platform pool against virtual threads, with a downstream limit
Simulation · not a real JVMThe simulation starts where the lab does: 2,000 requests that each wait 100 ms, on a pool of 200 platform threads. Change the thread mode, the downstream connection limit and the pinning option.
What you should see
Without a downstream limit the platform pool needs about ten times one call (the real lab measured 1,027 ms) and virtual threads about one call (127 ms). With 20 downstream connections both take about 10 seconds, 199 requests per second in the lab, because 20 x 10 calls per second is the ceiling. Raising the limit raises throughput in proportion. With pinning on, only as many requests progress at once as there are carriers: the lab's 40 pinned tasks took 2,011 ms on 2 carriers. The widget is a simulation; the lab numbers come from a real JVM.
What can go wrong
Expecting more throughput when a downstream limit is the bottleneck
- Symptom
- After enabling virtual threads the load test shows the same requests per second as before, with low CPU.
- Cause
- The thread pool was not the bottleneck, or it was and the next one took over at once: a connection pool, a rate limit or the capacity of a dependency. Virtual threads let more requests wait; they do not make the dependency faster.
- Fix
- Find the limit (connection-pool pending count, a
Thread.dump_to_filegrouped by frame) and decide whether it can move. If it cannot, the throughput you have is the throughput of that resource.
Losing the accidental limit the thread pool provided
- Symptom
- After the switch, traffic peaks produce connection-pool timeouts, HTTP 429 from partners or an overloaded database, none of which happened before.
- Cause
- 200 request threads used to cap concurrent downstream work at 200. With a virtual thread per request the cap is gone and peaks reach the dependencies directly.
- Fix
- Put an explicit bound next to each limited resource: a
Semaphorewith a timedtryAcquirefor calls without a pool of their own, and short acquisition timeouts on connection pools.
Blocking inside synchronized on JDK 21
- Symptom
- Throughput collapses or the whole service stalls for seconds at a time. A classic dump shows the few
ForkJoinPool-1-worker-Ncarriers blocked inside application or library code. - Cause
- A virtual thread that blocks while inside a
synchronizedblock or method is pinned and keeps its carrier. With as many such threads as carriers, no virtual thread can run. - Fix
- Find the frames with
-Djdk.tracePinnedThreads=shortor thejdk.VirtualThreadPinnedJFR event. ReplacesynchronizedwithReentrantLockwhere it guards a blocking call, upgrade libraries that still do this, or move to JDK 24 or later.
Pooling virtual threads
- Symptom
- Code such as
Executors.newFixedThreadPool(50, Thread.ofVirtual().factory())appears, usually to limit concurrency or out of habit. - Cause
- A pool reuses threads to avoid the cost of creating them. Virtual threads have almost no such cost, and reusing them keeps thread-locals alive across tasks for no benefit.
- Fix
- Use
Executors.newVirtualThreadPerTaskExecutor()and limit concurrency with aSemaphoreat the resource that needs the limit.
Thread-local caches of expensive objects
- Symptom
- Allocation rate and GC activity rise after the migration, and a profile shows constructors of buffers, formatters or clients on every request.
- Cause
- The code cached one expensive object per thread, which worked with 200 long-lived threads. Every virtual thread is new, so the object is built for every task and thrown away.
- Fix
- Share the object explicitly: an immutable instance, a small object pool, or a field of a singleton bean if it is thread-safe.
-Djdk.traceVirtualThreadLocals=truelists where thread-locals are set.
Using virtual threads for CPU-bound work
- Symptom
- A computation-heavy endpoint is no faster, and latency of other endpoints gets worse while it runs.
- Cause
- A virtual thread that computes without blocking keeps its carrier, and the scheduler does not time-slice. A few long computations occupy all carriers and delay every other virtual thread.
- Fix
- Run CPU-bound work on a platform thread pool sized to the processors, as in the ForkJoinPool lesson, and keep virtual threads for code that waits.
How to troubleshoot
No throughput gain after enabling virtual threads
Check
jcmd <pid> Thread.dump_to_file -format=json /tmp/threads.json (or the text form, then: grep -A 8 'virtual#x27; /tmp/threads.txt | grep 'com.example' | sort | uniq -c | sort -rn)
- How to read it
- Hundreds or thousands of virtual threads with the same top application frame are waiting for the same resource. If that frame is a connection pool's
getConnectionor a semaphore acquire, that resource is the bottleneck, not the threads.
Requests time out waiting for the database after the switch
Check
The Micrometer gauges `hikaricp.connections.pending` and `hikaricp.connections.active`, and the timer `hikaricp.connections.acquire` (for example GET /actuator/metrics/hikaricp.connections.pending)
- How to read it
activeat the pool maximum withpendingin the hundreds means requests are queueing for connections. Either queries got slower or the pool is the real limit; raising it helps only if the database has capacity.
Service stalls or runs far below expected concurrency on JDK 21
Check
Restart a test instance with -Djdk.tracePinnedThreads=short and reproduce the load
- How to read it
- Each printed block names a virtual thread, its carrier and
reason:MONITOR, followed by the frame marked<== monitors:1. That frame is asynchronizedblock or method holding a blocking call.
You need pinning evidence from production without a restart
Check
jcmd <pid> JFR.start duration=60s filename=/tmp/vt.jfr then jfr print --events jdk.VirtualThreadPinned /tmp/vt.jfr
- How to read it
- The event is enabled by default with a 20 ms threshold. Each one carries the pinned duration and the stack. Many events with the same application frame point at the block to change.
A classic thread dump of the service looks almost empty
Check
jcmd <pid> Thread.print | grep -A 12 'ForkJoinPool-1-worker'
- How to read it
- Virtual threads are not listed there. Carriers parked in
ForkJoinPool.awaitWorkare idle, which is healthy. Carriers that show application frames are running a virtual thread right now; the same blocking frame on all of them in two dumps a few seconds apart means pinning or CPU-bound work.
Unsure whether request handling really runs on virtual threads
Check
Log `Thread.currentThread()` in a controller, or `Thread.currentThread().isVirtual()`
- How to read it
- A virtual thread prints as
VirtualThread[#id,name]/runnable@ForkJoinPool-1-worker-N. A name likehttp-nio-8080-exec-7means a platform thread: the property is not set, the JVM is older than 21, or a custom executor bean took over.
Key takeaways
- A virtual thread is a
java.lang.Threadscheduled by the JDK onto a few carrier threads. Blocked, it is an object on the heap and holds no OS thread. - The gain is concurrency for code that waits: many blocking requests in progress at once. It is not speed, and CPU-bound work gains nothing.
- The thread pool used to be both a bottleneck and an accidental limit. Removing it moves the bottleneck to the next limited resource and exposes that resource to peaks.
- If a downstream limit of 20 allows 200 requests per second, virtual threads deliver 200 requests per second. Throughput changes only when that limit does.
- Bound concurrency with a
Semaphoreand a timed acquire at the resource, never by pooling virtual threads. - On JDK 21 to 23, blocking inside
synchronizedpins the carrier. Find it with-Djdk.tracePinnedThreadsor thejdk.VirtualThreadPinnedJFR event; JDK 24 removed that case (JEP 491). jcmd Thread.printdoes not list virtual threads. Usejcmd <pid> Thread.dump_to_fileand group the threads by frame.
Practical check
Small scenarios, one decision each. Pick what you would do, then read why.
Scenario 1 of 4
A service handles each request with one JDBC query of about 50 ms. HikariCP has 10 connections. On platform threads with Tomcat's default 200 threads the load test peaks at 200 requests per second with CPU at 15%. The team enables virtual threads and reruns the test.
What result should they expect, and what should they look at?
Scenario 2 of 4
On JDK 21 with 4 carriers, a service on virtual threads freezes for about 3 seconds every hour: no endpoint answers, including health checks. A run with -Djdk.tracePinnedThreads=short prints reason:MONITOR and a frame LicenceCache.refresh(LicenceCache.java:48) <== monitors:1. Line 48 is an HTTP call inside a synchronized method that every request passes through.
Which change addresses the cause on JDK 21?
Scenario 3 of 4
A thumbnail service resizes images in memory, about 400 ms of pure CPU per request, on a pod with 4 CPUs. It runs on a platform pool of 4 threads and serves about 10 requests per second. Someone proposes one virtual thread per request 'to handle more load'.
What is the most likely outcome?
Scenario 4 of 4
After moving to virtual threads, a service calls a partner API that documents a limit of 30 concurrent requests per client. At peak, 2,000 requests arrive within a second and the partner starts returning HTTP 429. A developer proposes Executors.newFixedThreadPool(30, Thread.ofVirtual().factory()) for the partner calls.
What is the better way to enforce the limit, and why?
References
References: JEP 444: Virtual Threads (final in JDK 21): scheduler, pinning, tracing, do not pool ↗ · JEP 491: Synchronize Virtual Threads without Pinning (delivered in JDK 24) ↗ · Thread (Java SE 21 API): virtual threads and the jdk.virtualThreadScheduler.* properties ↗ · Oracle Java 21 Core Libraries guide: Virtual Threads (pinning, semaphores, thread-locals) ↗ · JEP 446: Scoped Values (Preview), JDK 21 ↗ · Spring Boot 3.5 reference: Task Execution and Scheduling (virtual threads) ↗ · Spring Boot 3.5 reference: common application properties (spring.threads.virtual.enabled, server.tomcat.threads.max) ↗ · Spring Boot 3.2 release notes: Support for Virtual Threads ↗ · HikariCP README: maximumPoolSize and connectionTimeout defaults ↗ · HikariCP MicrometerMetricsTracker source: hikaricp.connections.* meter names ↗ · Report a problem