Virtual threads: where they help and where they cannot
2,000 requests each wait 100 ms on a remote call. Virtual threads cut 1,027 ms to 127 ms, change nothing when 20 downstream connections are the limit, and get slower when pinned.
The situation
A Spring Boot 3 pricing service on Java 21 refreshes 2,000 prices by calling a remote service that answers in about 100 ms. On a pool of 200 platform threads the refresh takes about 1 second. The team sets spring.threads.virtual.enabled=true, moves the fan-out to one virtual thread per SKU and expects about 100 ms. In production the refresh takes about 10 seconds, the same as before the change, because the HTTP client in front of the pricing service has 20 connections. One code path is even slower than before: it makes the call inside a synchronized block.
Read first: Virtual threads in Java 21, Executors, thread pools and rejection
Break it in the simulation
Platform pool against virtual threads
Simulation · not a real JVMStart on a platform pool of 200 and read the verdict. Switch Threads to virtual, then set Downstream connections to 20 and compare the two rows. Last, turn on Blocks inside synchronized.
Guided run · 0 of 6 stages done
- Leave Threads on platform, Pool size at 200, Requests at 2,000, Call time at 100 ms and Downstream connections at no limit.
Look for
- Total time is 1,027 ms and Requests per second 1,947, as in the lab run.
- Rounds of calls is 10: 2,000 requests go through 200 at a time.
- Stage 1 · Threads is marked as the narrowest stage: 1,800 requests wait in the executor queue at the start.
- Switch Threads to virtual.
Look for
- Total time drops to about 126 ms (127 ms in the lab) and Calls in flight is 2,000.
- No stage is marked. All requests wait on the remote call at the same time.
- The Memory note: 2,000 virtual threads run on 2 carrier threads.
- Set Downstream connections to 20.
- Switch Threads between virtual and platform.
Look for
- Both rows of Platform against virtual, same settings show about 10,036 ms and 199 requests/s. The lab measured 10,019 ms and 10,039 ms.
- The verdict: both take about 10 s, 20 connections × 10 calls per second each = 200 requests/s whatever runs the requests.
- Stage 2 · Downstream connections is the narrowest. On virtual threads 1,980 requests hold a thread and wait for a connection.
- Keep Threads on virtual. Raise Downstream connections from 20 to 100.
- Put it back to 20 and halve Call time to 50 ms.
Look for
- 100 connections: about 2 s. The total follows the downstream limit, not the thread kind.
- A 50 ms call: about 5 s. Only the narrowest stage and the call time matter.
- Set Call time back to 100 ms and Downstream connections to 0 (no limit).
- With Threads on virtual, turn on Blocks inside synchronized.
Look for
- Calls in flight falls to 2 and Total time rises to about 100 s for 2,000 requests. The lab ran 40 such tasks and measured 2,011 ms, which is 20 rounds of 100 ms.
- Stage 1 · Threads reads 2 carrier threads (pinned) and is the narrowest stage.
- The comparison: the platform pool of 200 is now far faster than virtual threads.
- Turn Blocks inside synchronized off. In code this is the change from synchronized to ReentrantLock around the blocking call.
- Set Downstream connections to 20 again.
Look for
- Without a downstream limit the total is back to about 126 ms.
- With 20 connections it is about 10 s again: unpinning removed a penalty, it did not remove the real limit.
Wrong, then right
Incorrect implementation
import java.math.BigDecimal;import java.util.List;import java.util.Map;import java.util.concurrent.ConcurrentHashMap;import java.util.concurrent.ExecutorService;import java.util.concurrent.Executors;import org.springframework.stereotype.Service;@Servicepublic class PriceService { /** Blocking HTTP client for the pricing service. Its connection pool holds 20 connections. */ public interface PricingClient { BigDecimal fetch(String sku); } private final PricingClient pricing; private final Map<String, BigDecimal> cache = new ConcurrentHashMap<>(); private final Map<String, Object> skuLocks = new ConcurrentHashMap<>(); public PriceService(PricingClient pricing) { this.pricing = pricing; } /** "Virtual threads are cheap, so start one per SKU and the refresh takes one call time." */ public Map<String, BigDecimal> refresh(List<String> skus) { try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) { for (String sku : skus) { executor.submit(() -> price(sku)); } } // close() waits for every task return Map.copyOf(cache); } BigDecimal price(String sku) { synchronized (skuLocks.computeIfAbsent(sku, k -> new Object())) { // one caller per SKU BigDecimal cached = cache.get(sku); if (cached != null) return cached; BigDecimal fresh = pricing.fetch(sku); // blocks about 100 ms while holding a monitor cache.put(sku, fresh); return fresh; } }}Why it is written this way
- The fan-out is fine. The expectation in the comment is not: nothing in this class says that only 20 calls can be in flight.
synchronizedaroundpricing.fetchholds a monitor during a blocking call. On JDK 21 that pins the virtual thread to its carrier for the whole 100 ms.- Each SKU has its own lock object, so there is no lock contention at all. The slowdown comes only from pinning, which is why a thread dump shows no BLOCKED threads.
- Compiled with javac 21 against a stub of
@Service; not run inside Spring.
Why it fails
- A virtual thread that waits gives its carrier thread to another virtual thread, so thousands can wait at once. That removes the limit a thread pool sets. It does not make a call shorter: the Java documentation states that virtual threads are not faster threads and exist to provide scale, not speed.
- The connection pool lets 20 calls through at a time. 20 connections × 10 calls per second each = 200 requests per second, so 2,000 requests take about 10 seconds on a pool of 200 platform threads and on 2,000 virtual threads alike. The other 1,980 virtual threads are parked waiting for a connection.
- On JDK 21 a virtual thread that blocks inside a
synchronizedblock or method cannot unmount. The scheduler has one carrier per available processor by default, so on 2 CPUs only 2 pinned calls run at once: 40 calls took 2,011 ms in the lab instead of about 100 ms. - Pinning is silent. Nothing fails and no thread is BLOCKED; requests just queue. You see it with
-Djdk.tracePinnedThreads=shortor the JFR eventjdk.VirtualThreadPinned.
Corrected implementation
import java.math.BigDecimal;import java.util.List;import java.util.Map;import java.util.concurrent.ConcurrentHashMap;import java.util.concurrent.ExecutorService;import java.util.concurrent.Executors;import java.util.concurrent.Semaphore;import java.util.concurrent.locks.ReentrantLock;import org.springframework.stereotype.Service;@Servicepublic class PriceService { /** Blocking HTTP client for the pricing service. Its connection pool holds 20 connections. */ public interface PricingClient { BigDecimal fetch(String sku); } /** The real limit, written down where the calls are made. Keep it equal to the connection pool. */ private static final int PRICING_CONNECTIONS = 20; private final PricingClient pricing; private final Semaphore pricingPermits = new Semaphore(PRICING_CONNECTIONS); private final Map<String, BigDecimal> cache = new ConcurrentHashMap<>(); private final Map<String, ReentrantLock> skuLocks = new ConcurrentHashMap<>(); public PriceService(PricingClient pricing) { this.pricing = pricing; } /** Takes about skus.size() / 20 x call time. The thread kind does not change that. */ public Map<String, BigDecimal> refresh(List<String> skus) { try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) { for (String sku : skus) { executor.submit(() -> price(sku)); } } return Map.copyOf(cache); } BigDecimal price(String sku) throws InterruptedException { ReentrantLock lock = skuLocks.computeIfAbsent(sku, k -> new ReentrantLock()); lock.lock(); // a virtual thread waiting here, or blocked below, gives its carrier back try { BigDecimal cached = cache.get(sku); if (cached != null) return cached; pricingPermits.acquire(); try { BigDecimal fresh = pricing.fetch(sku); cache.put(sku, fresh); return fresh; } finally { pricingPermits.release(); } } finally { lock.unlock(); } }}Why it is written this way
ReentrantLockreplacessynchronizedaround the blocking call. This is the replacement the Java virtual threads guide recommends where pinning is frequent and long-lived.- The
Semaphorestates the downstream limit in the code that makes the calls. The same guide recommends a semaphore, not a thread pool, to limit concurrency with virtual threads. - The comment on
refreshgives the honest estimate: 2,000 SKUs ÷ 20 × 100 ms = about 10 seconds. - Compiled with javac 21 against a stub of
@Service; not run inside Spring.
Trade-offs
- The corrected version is not faster than a platform pool when 20 connections are the limit. It takes the same 10 seconds. What it gains is that 2,000 waiting requests no longer need 2,000 platform threads, and nothing is pinned.
ReentrantLockmust be released infinally, andjcmd <pid> Thread.printlists java.util.concurrent locks only with the-loption.synchronizedis still fine for short in-memory sections.- The semaphore and the connection pool now both hold the number 20. If they drift apart the smaller one wins without telling you.
- Raising the downstream limit moves the load to the pricing service. Do it only after checking what that service can take; otherwise you trade a 10 second refresh for an outage next door.
- With no thread pool there is no queue and no rejection policy to push back on callers. Under overload, requests pile up as waiting virtual threads, so keep explicit limits and timeouts.
Run it on a real JVM
VirtualThreads.java submits 2,000 tasks that each sleep 100 ms, the stand-in for a remote call. It runs them on a fixed pool of platform threads or on one virtual thread per task, with an optional Semaphore as the downstream connection pool. The pinned mode runs 40 virtual threads that sleep inside synchronized, each on its own lock.
With Docker (CPU and memory limits)
cd labs/jvm/virtual-threadsdocker build -t jvm-lab-virtual-threads .# platform pool of 200, then virtual threadsdocker run --rm --cpus=2 --memory=512m jvm-lab-virtual-threads VirtualThreads platform 200docker run --rm --cpus=2 --memory=512m jvm-lab-virtual-threads VirtualThreads virtual# the same two with 20 downstream connections (about 10 s each)docker run --rm --cpus=2 --memory=512m jvm-lab-virtual-threads VirtualThreads platform 200 20docker run --rm --name vt-lab --cpus=2 --memory=512m jvm-lab-virtual-threads VirtualThreads virtual 20# second terminal, during the 10 s run: a dump that includes virtual threadsdocker exec vt-lab jcmd 1 Thread.dump_to_file -format=text /tmp/threads.txtdocker exec vt-lab cat /tmp/threads.txt# pinning, with a trace of the frame that holds the monitordocker run --rm --cpus=2 --memory=512m jvm-lab-virtual-threads -Djdk.tracePinnedThreads=short VirtualThreads pinnedWithout Docker, on a local JDK 21
java -XX:ActiveProcessorCount=2 VirtualThreads.java platform 200java -XX:ActiveProcessorCount=2 VirtualThreads.java virtualjava -XX:ActiveProcessorCount=2 VirtualThreads.java platform 200 20java -XX:ActiveProcessorCount=2 VirtualThreads.java virtual 20java -XX:ActiveProcessorCount=2 -Djdk.tracePinnedThreads=short VirtualThreads.java pinnedThe program: VirtualThreads.java
import java.util.ArrayList;import java.util.List;import java.util.concurrent.*;/** * Lab 7: platform threads versus virtual threads (Java 21). * * 2,000 requests each wait 100 ms on a remote call. * * java VirtualThreads.java platform 200 -> a pool of 200 platform threads * java VirtualThreads.java virtual -> one virtual thread per request * java VirtualThreads.java virtual 20 -> virtual threads, but the remote call goes through a pool of 20 connections * java VirtualThreads.java platform 200 20 * java -Djdk.tracePinnedThreads=short VirtualThreads.java pinned -> blocking inside synchronized pins the carrier (JDK 21) */public class VirtualThreads { static final int REQUESTS = 2_000; static final int CALL_MS = 100; public static void main(String[] args) throws Exception { String mode = args.length > 0 ? args[0] : "virtual"; if (mode.equals("pinned")) { pinned(); return; } int poolSize = mode.equals("platform") ? Integer.parseInt(args[1]) : 0; int connArg = mode.equals("platform") ? 2 : 1; int connections = args.length > connArg ? Integer.parseInt(args[connArg]) : 0; Semaphore connectionPool = connections > 0 ? new Semaphore(connections) : null; ExecutorService executor = mode.equals("platform") ? Executors.newFixedThreadPool(poolSize) : Executors.newVirtualThreadPerTaskExecutor(); long t0 = System.nanoTime(); List<Future<?>> futures = new ArrayList<>(); for (int i = 0; i < REQUESTS; i++) { futures.add(executor.submit(() -> { if (connectionPool != null) connectionPool.acquire(); try { Thread.sleep(CALL_MS); // the remote call } finally { if (connectionPool != null) connectionPool.release(); } return null; })); } for (Future<?> f : futures) f.get(); long ms = (System.nanoTime() - t0) / 1_000_000; executor.shutdown(); System.out.printf("%-8s threads%s%s: %,d requests in %,d ms = %,d requests/s%n", mode, poolSize > 0 ? " (pool of " + poolSize + ")" : "", connections > 0 ? ", " + connections + " downstream connections" : ", no downstream limit", REQUESTS, ms, REQUESTS * 1000L / Math.max(ms, 1)); } /** JDK 21: a virtual thread that blocks while holding a monitor cannot unmount, so it holds its carrier thread. */ static void pinned() throws Exception { Object monitor = new Object(); int carriers = Runtime.getRuntime().availableProcessors(); try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) { long t0 = System.nanoTime(); List<Future<?>> futures = new ArrayList<>(); for (int i = 0; i < 40; i++) { Object lock = new Object(); // a different lock per task: there is no contention at all futures.add(executor.submit(() -> { synchronized (lock) { Thread.sleep(CALL_MS); // blocks while holding a monitor } return null; })); } for (Future<?> f : futures) f.get(); long ms = (System.nanoTime() - t0) / 1_000_000; System.out.printf("pinned: 40 virtual threads sleeping 100 ms inside synchronized took %,d ms on %d carrier threads (unpinned it would take about 100 ms)%n", ms, carriers); } }}The Dockerfile
# Java 21 lab image. Build: docker build -t jvm-lab-virtual-threads .FROM eclipse-temurin:21-jdkWORKDIR /labCOPY VirtualThreads.java .RUN javac -d out VirtualThreads.java# The JVM reads the container's CPU and memory limits (container support is on by default),# so --cpus and --memory on "docker run" change what Runtime.availableProcessors() and the default heap size report.ENTRYPOINT ["java", "-cp", "out"]CMD ["VirtualThreads"]$ java -XX:ActiveProcessorCount=2 VirtualThreads.java platform 200 (then: virtual, platform 200 20, virtual 20)
Program outputFour runs: platform and virtual, without and with a downstream limit
Real JVM outputplatform threads (pool of 200), no downstream limit: 2,000 requests in 1,027 ms = 1,947 requests/s virtual threads, no downstream limit: 2,000 requests in 127 ms = 15,748 requests/s platform threads (pool of 200), 20 downstream connections: 2,000 requests in 10,019 ms = 199 requests/s virtual threads, 20 downstream connections: 2,000 requests in 10,039 ms = 199 requests/s
$ java -XX:ActiveProcessorCount=2 -Djdk.tracePinnedThreads=short VirtualThreads.java pinned
Program outputPinned: 40 virtual threads sleeping inside synchronized
Real JVM outputVirtualThread[#19]/runnable@ForkJoinPool-1-worker-1 reason:MONITOR
VirtualThreads.lambda$pinned$1(VirtualThreads.java:61) <== monitors:1
pinned: 40 virtual threads sleeping 100 ms inside synchronized took 2,011 ms on 2 carrier threads (unpinned it would take about 100 ms)Real run and simulation: what each one shows
- The captured files come from OpenJDK 21.0.13 run directly on Linux x86-64 with -XX:ActiveProcessorCount=2, not in Docker. The Docker commands above were not executed when this was written.
- The real run shows the pinning trace with the exact source line, and real start-up costs: 127 ms for one round of 100 ms calls, of which about 27 ms is creating and scheduling 2,000 tasks. The widget only has a fitted constant for that.
- The simulation lets you change what the lab fixes in its source: the number of requests and the call time, and it shows platform and virtual side by side for every setting.
- The lab pins 40 tasks; the widget pins whatever Requests is set to (2,000 by default), so its total is about 100 s where the lab shows 2,011 ms. The rule is the same: requests ÷ 2 carriers × call time.
- In the lab the remote call is Thread.sleep. A real remote service gets slower when it receives more concurrent calls, which neither the lab nor the widget shows.
How you meet this in production
- The first symptom is the absence of one: after switching to virtual threads, latency and throughput are what they were. Look at the pool metrics of the downstream client or the database pool (active connections at the maximum, threads waiting for a connection) before looking at threads.
- A dump taken with
jcmd <pid> Thread.dump_to_file -format=text <file>includes virtual threads. Hundreds of them waiting in the same connection-acquire frame name the limit. - For pinning, run with
-Djdk.tracePinnedThreads=shortin a test environment, or record JFR and look forjdk.VirtualThreadPinned, which is enabled by default with a 20 ms threshold. - The Investigate case Virtual threads, no faster starts from the same symptom.
References
References: Java SE 21 Core Libraries: Virtual Threads (pinning, tracePinnedThreads, semaphores, adoption guide) ↗ · Java SE 21 API: Thread (virtual threads, carrier threads, scheduler properties) ↗ · Spring Boot 3.5 reference: Virtual threads (spring.threads.virtual.enabled) ↗ · JDK 21 tool reference: jcmd ↗ · Report a problem