Path 4 · lesson 1 of 5
Executors, thread pools and rejection
How ThreadPoolExecutor decides between a thread, the queue and rejection, why an unbounded queue hides overload, and how to find a starved pool in a thread dump.
What you will learn
- State the exact order in which ThreadPoolExecutor handles a submitted task and predict the pool size and queue length after each submission
- Explain why Executors.newFixedThreadPool never rejects and what that costs when arrivals outrun the pool
- Choose between AbortPolicy, CallerRunsPolicy, DiscardPolicy and DiscardOldestPolicy for a given kind of work
- Size a pool and its queue from the arrival rate, the task time and the wait you are willing to accept
- Configure ThreadPoolTaskExecutor, @Async and the Tomcat request pool in Spring Boot 3.x and know their defaults
- Recognise a starved pool from executor metrics and from the pool's threads in a thread dump
Before you start
The concept
An ExecutorService separates submitting work from running it. You hand it a Runnable or Callable; it decides which thread runs the task and when. The implementation behind almost every pool of platform threads in a Java service is ThreadPoolExecutor: a set of worker threads that take tasks from one shared BlockingQueue.
A ThreadPoolExecutor is fully described by five things. Every production question about a pool comes back to one of them.
| Parameter | What it is | What it decides |
|---|---|---|
corePoolSize | Threads the pool creates before it starts queuing | How much work runs at once in normal operation |
maximumPoolSize | Upper limit on threads, used only when the queue is full | Extra capacity for bursts, if the queue is bounded |
workQueue | Where tasks wait when corePoolSize threads are busy | How much waiting work the pool accepts, and so how long a task can wait |
keepAliveTime | How long a thread above core may sit idle before it ends | How fast the pool shrinks after a burst |
RejectedExecutionHandler | What happens to a task the pool cannot take | Who pays for overload: the caller, the task, or nobody visibly |
The factory methods in Executors are fixed choices of those five parameters. Executors.newFixedThreadPool(n) is n core threads, n maximum threads and a LinkedBlockingQueue with no capacity argument, which means a capacity of Integer.MAX_VALUE. Its Javadoc says so directly: the threads operate "off a shared unbounded queue". Executors.newCachedThreadPool() is the opposite choice: zero core threads, no limit on threads and a SynchronousQueue that holds nothing, so every task that finds no idle thread starts a new one.
A pool of 4 with an unbounded queue
Simulation · not a real JVMRequests arrive five times faster than four threads can finish them. Nothing is rejected. Watch the queue length and the wait of the newest task.
Why it exists
A platform thread is an operating system thread with its own stack. Creating one per request works until a slow dependency makes requests pile up: then the service has thousands of threads, each holding a stack, a database connection or a socket, and the dependency that was slow is now flooded. A pool puts a number on how much work runs at once. That number is the pool's real job: it is a concurrency limit, and the queue and the rejection policy say what happens to the work above the limit.
There are only three things a system can do with work it cannot run now: make it wait, refuse it, or slow the sender down. An unbounded queue chooses waiting without limit. A bounded queue with AbortPolicy chooses refusing. CallerRunsPolicy chooses slowing the sender. No setting makes the overload disappear; the settings choose where it shows.
Unbounded queue
- No task is ever rejected, so no error appears anywhere
- Wait time grows for as long as arrivals exceed capacity
- Callers time out on their side while the task is still queued, and the pool then does work nobody is waiting for
- Each queued task holds its captured objects on the heap until it runs
Bounded queue with a rejection policy
- The longest possible wait is known: queue capacity divided by the pool's throughput
- Overload becomes visible at once as rejections or as a slower caller
- The caller can answer quickly with a 503 or a fallback
- You have to pick the capacity and handle the rejection in code
In a request and response service the first symptom of a starved pool is not an exception. It is latency that climbs while CPU stays low: threads are all waiting on something slow, new work waits behind them, and the machine looks idle. The checkout latency case in Practice is this failure.
How it works inside
The order, step by step
execute(task) makes at most three attempts, in this order. The Javadoc states the same rules under "Queuing", and submit and invokeAll go through execute.
- Fewer than
corePoolSizethreads exist: start a new thread and give it this task as its first task. This happens even if other threads are idle. - Otherwise offer the task to the queue. If the queue accepts it, the task waits there until a worker takes it.
- The queue refused it: start a new thread, unless that would exceed
maximumPoolSize. - That failed too: the pool is saturated. Call the
RejectedExecutionHandler. The same handler is called for any task submitted aftershutdown().
Program output8 one-second tasks into core 2, max 4, queue 2 (java ExecutorsThreadPools.java order)
Real JVM outputcorePoolSize=2 maximumPoolSize=4 queue=ArrayBlockingQueue(2) handler=AbortPolicy task 1 -> new core thread poolSize=1 queued=0 task 2 -> new core thread poolSize=2 queued=0 task 3 -> queued poolSize=2 queued=1 task 4 -> queued poolSize=2 queued=2 task 5 -> new thread above core (queue was full) poolSize=3 queued=2 task 6 -> new thread above core (queue was full) poolSize=4 queued=2 java.util.concurrent.RejectedExecutionException: Task ExecutorsThreadPools$Lambda/0x00007a6f4c000c30@7adf9f5f rejected from java.util.concurrent.ThreadPoolExecutor@1540e19d[Running, pool size = 4, active threads = 4, queued tasks = 2, completed tasks = 0] task 7 -> REJECTED poolSize=4 queued=2 task 8 -> REJECTED poolSize=4 queued=2 java.util.concurrent.ThreadPoolExecutor@1540e19d[Running, pool size = 4, active threads = 4, queued tasks = 2, completed tasks = 0] after shutdown(): isShutdown=true isTerminated=false (queued tasks still run) execute() after shutdown(): rejected, pool state is Shutting down awaitTermination(5 s)=true completed=6 isTerminated=true
pool size = 4, active threads = 4, queued tasks = 2. Note the consequence: tasks 5 and 6 arrived later than 3 and 4 but start running before them. After shutdown() the 6 accepted tasks still finish; only new ones are refused.Two details follow from this order. First, threads are created on demand: a new pool has zero threads until tasks arrive, unless you call prestartAllCoreThreads(). Second, a pool with a large bounded queue and a higher maximum grows late: the extra threads appear only after the queue is already full, which is after latency has already suffered.
Three kinds of queue
| Queue | Behaviour | Consequence |
|---|---|---|
SynchronousQueue (direct handoff) | Holds nothing. An offer succeeds only if a thread is waiting to take it. | Every task either starts at once or creates a thread or is rejected. Needs a high maximum, which allows unbounded thread growth. |
LinkedBlockingQueue with no capacity | Accepts everything (capacity Integer.MAX_VALUE). | Never more than corePoolSize threads. Never a rejection. Unbounded wait and unbounded memory under sustained overload. |
ArrayBlockingQueue(n) or LinkedBlockingQueue(n) | Accepts n waiting tasks, then refuses. | Bounded wait. Rejection policy decides what overload looks like. |
The four rejection policies
| Policy | What it does | Use it when |
|---|---|---|
AbortPolicy (the default) | Throws RejectedExecutionException to the thread that called execute. | The caller can fail fast or fall back: an HTTP request that can return 503. |
CallerRunsPolicy | The submitting thread runs the task itself, unless the pool is shut down, in which case the task is discarded. | The work must not be lost and slowing the submitter is acceptable: a batch loop or a message listener. |
DiscardPolicy | Drops the task silently. | Rarely. Only when nobody ever waits for the result, such as a best-effort refresh. A Future for a discarded task never completes. |
DiscardOldestPolicy | Drops the task at the head of the queue and retries the new one. | Rarely. The Javadoc calls it "rarely acceptable". It fits only when a newer task makes an older one worthless. |
Sizing
Size the pool from what the tasks do, not from a habit.
- CPU-bound work (parsing, compression, calculation): more threads than cores gives no extra throughput, only context switches. Start with the number of available processors, which in a container is the CPU limit, not the host's core count. Check it with
Runtime.getRuntime().availableProcessors(). - Blocking work (HTTP calls, JDBC, file I/O): a thread spends most of its time waiting, so the pool needs about arrival rate × time per task threads to keep up. 100 calls per second that each take 200 ms keep 100 × 0.2 = 20 threads busy on average. Add headroom for variance, then check the limit of what you call: 40 threads are pointless against a database pool of 10 connections.
- The queue is sized from the wait you accept. A full queue of
qtasks in front ofnthreads that each taketseconds delays the last task by aboutq × t / n. A queue of 8 before 4 threads at 200 ms is about 400 ms of extra wait at worst. - Do not mix CPU-bound and blocking tasks in one pool. A few slow blocking tasks occupy every thread and the short tasks starve behind them. Give each kind of dependency its own pool so that one slow dependency cannot take the threads of another.
The lab's numbers make the arithmetic concrete. Four threads at 200 ms per task finish 20 tasks per second. Arrivals are one every 10 ms, 100 per second. The pool is short by 80 tasks per second, and the three configurations in the next section differ only in what they do with those 80.
Shutdown
shutdown()stops accepting new tasks and lets running and queued tasks finish. It returns at once; it does not wait.awaitTermination(timeout, unit)blocks until all tasks are done or the timeout passes, and returnsfalseon timeout.shutdownNow()interrupts the running tasks and returns the queued tasks that never started. A task that ignores interruption keeps running.- Since Java 19
ExecutorServiceisAutoCloseable:close()callsshutdown()and then waits for termination, so try-with-resources works for short-lived executors.
import java.util.List;import java.util.concurrent.ArrayBlockingQueue;import java.util.concurrent.ThreadFactory;import java.util.concurrent.ThreadPoolExecutor;import java.util.concurrent.TimeUnit;import java.util.concurrent.atomic.AtomicInteger;public final class PartnerPool { public static ThreadPoolExecutor create() { AtomicInteger n = new AtomicInteger(); ThreadFactory named = r -> new Thread(r, "partner-exec-" + n.incrementAndGet()); return new ThreadPoolExecutor( 16, 16, // core = max: a fixed pool 0, TimeUnit.SECONDS, // no threads above core, so keep-alive is unused new ArrayBlockingQueue<>(32), // at most 32 tasks waiting named, new ThreadPoolExecutor.AbortPolicy()); } /** Stop taking work, give running tasks 20 s, then interrupt what is left. */ public static void stop(ThreadPoolExecutor pool) { pool.shutdown(); try { if (!pool.awaitTermination(20, TimeUnit.SECONDS)) { List<Runnable> neverStarted = pool.shutdownNow(); System.err.println("pool did not drain, dropped " + neverStarted.size() + " queued tasks"); pool.awaitTermination(5, TimeUnit.SECONDS); } } catch (InterruptedException e) { pool.shutdownNow(); Thread.currentThread().interrupt(); } }}Why it is written this way
- Naming the threads is not cosmetic. The prefix is how you find this pool in a thread dump and tell it from
http-nio-8080-exec-andtask-threads. - Core equal to max with a bounded queue is the configuration that is easiest to reason about: 16 running, 32 waiting, the 49th concurrent task is rejected.
stopis the two-phase shutdown:shutdown(), wait, thenshutdownNow()for what is left. Without the wait, a deploy cuts off work that was accepted.- Compiled with javac 21.
Architecture
Drawing diagram…
Flowchart. A submitted task first checks whether fewer than corePoolSize threads exist, and if so starts a core thread. Otherwise it is offered to the queue. If the queue is full and fewer than maximumPoolSize threads exist, a new thread starts. Otherwise the rejection handler runs, with one branch for each of the four policies.
flowchart TD
A["execute(task)"] --> B{"Fewer than corePoolSize threads"}
B -->|"Yes"| C["Start a core thread with this task"]
B -->|"No"| D{"Queue accepts the task"}
D -->|"Yes"| E["Task waits in the queue"]
D -->|"No, queue is full"| F{"Fewer than maximumPoolSize threads"}
F -->|"Yes"| G["Start a thread above core"]
F -->|"No"| H["RejectedExecutionHandler"]
H --> I["AbortPolicy throws to the caller"]
H --> J["CallerRunsPolicy runs it in the caller"]
H --> K["DiscardPolicy drops it"]
H --> L["DiscardOldestPolicy drops the queue head and retries"]- Read it top to bottom: the queue sits between core and max. That single fact explains why a pool configured as core 10, max 50 with an unbounded queue never has more than 10 threads.
- The right-hand exit of the second decision is the only way to reach the third. With an unbounded queue the diagram ends at "Task waits in the queue" for every task, for ever.
- The handler runs in the submitting thread.
AbortPolicythrows there andCallerRunsPolicyruns the task there. That is why the choice of policy depends on who the caller is. - Workers leave the picture after their first task: each one loops, taking the next task from the queue. An idle worker is a thread parked inside the queue's
take().
Step by step
Drawing diagram…
Sequence diagram with a request thread, the pool, its work queue, a worker thread and the rejection handler. Task 1 starts a core thread. Task 5 is offered to the queue and accepted, and the worker takes it when task 1 ends. Task 13 finds the queue full and the pool at its maximum, so the handler throws RejectedExecutionException back to the request thread.
sequenceDiagram participant R as Request thread participant P as ThreadPoolExecutor participant Q as Work queue participant W as Worker thread participant H as Rejection handler R->>P: execute task 1 P->>W: Start core thread with task 1 R->>P: execute task 5 while all core threads are busy P->>Q: offer task 5 Q-->>P: accepted Note over W: Task 1 ends W->>Q: take next task Q-->>W: task 5 R->>P: execute task 13 with queue full and max reached P->>H: rejectedExecution H-->>R: RejectedExecutionException Note over R: The request answers 503 instead of waiting
- The pool never pushes a queued task to a worker. Workers pull: when a task ends, the worker calls
take()on the queue. A task's wait in the queue is therefore the time until some worker finishes what it is doing. - The rejection is synchronous.
executethrows in the request thread before it returns, so the caller knows immediately and can answer with an error or a fallback. - Nothing in this exchange measures the wait in the queue. If you want that number, record the submit time in the task and subtract it when the task starts, as the lab does.
In a real system
The lab reproduces a service that calls a slow dependency through a pool of 4 threads. Each call takes 200 ms and a new request arrives every 10 ms, so 60 requests arrive in 600 ms while the pool can finish only 12 in that time. The same program was run with three configurations.
Program outputUnbounded queue (java PoolExhaustion.java unbounded)
Real JVM outputt= 95 ms submitted=10 active=4 queued= 6 completed= 0 rejected= 0 t= 204 ms submitted=20 active=4 queued=15 completed= 1 rejected= 0 t= 306 ms submitted=30 active=4 queued=22 completed= 4 rejected= 0 t= 407 ms submitted=40 active=4 queued=31 completed= 5 rejected= 0 t= 509 ms submitted=50 active=4 queued=38 completed= 8 rejected= 0 t= 610 ms submitted=60 active=4 queued=47 completed= 9 rejected= 0 mode=unbounded served=60 rejected=0 time to submit all 60 requests: 622 ms (600 ms if nothing blocked the caller) wait in queue before a thread picked the task up: avg 1109 ms, max 2225 ms all work finished after 3036 ms
rejected stays 0 and queued climbs by about 8 every 100 ms: 6, 15, 22, 31, 38, 47. That slope is the gap between arrivals and capacity. No error was raised, yet the average wait before a thread even started a task was 1,109 ms and the worst was 2,225 ms, for work that takes 200 ms. In a service this is the graph where p99 latency climbs while the error rate stays at zero.Program outputQueue of 8 with AbortPolicy (java PoolExhaustion.java abort)
Real JVM outputt= 93 ms submitted=10 active=4 queued= 6 completed= 0 rejected= 0 t= 204 ms submitted=20 active=4 queued= 8 completed= 1 rejected= 7 t= 306 ms submitted=30 active=4 queued= 8 completed= 4 rejected=14 t= 408 ms submitted=40 active=4 queued= 8 completed= 5 rejected=23 t= 509 ms submitted=50 active=4 queued= 8 completed= 8 rejected=30 t= 612 ms submitted=60 active=4 queued= 7 completed=10 rejected=39 mode=abort served=21 rejected=39 time to submit all 60 requests: 622 ms (600 ms if nothing blocked the caller) wait in queue before a thread picked the task up: avg 259 ms, max 397 ms all work finished after 1203 ms
queued stops at 8 and rejected grows instead: 39 of 60 requests were refused. The 21 that were accepted waited at most 397 ms, close to the 8 × 200 / 4 = 400 ms the sizing rule predicts, and the caller was never slowed down (622 ms to submit against 600 ms). Overload is now an explicit, fast failure the caller can handle.Program outputQueue of 8 with CallerRunsPolicy (java PoolExhaustion.java caller-runs)
Real JVM outputmode=caller-runs served=60 rejected=0 time to submit all 60 requests: 2223 ms (600 ms if nothing blocked the caller) wait in queue before a thread picked the task up: avg 219 ms, max 374 ms all work finished after 2635 ms
Finding a starved pool in a thread dump
Metrics tell you a pool is full. A thread dump tells you what its threads are doing. Take one with jcmd <pid> Thread.print and look at the threads with the pool's name prefix. The second program keeps a pool of 4 busy calling a server that accepts the connection and never answers, next to an idle pool of 2.
Program outputThreadPoolExecutor.toString() for both pools
Real JVM outputcheckout java.util.concurrent.ThreadPoolExecutor@2f92e0f4[Running, pool size = 4, active threads = 4, queued tasks = 26, completed tasks = 0] reports java.util.concurrent.ThreadPoolExecutor@5305068a[Running, pool size = 2, active threads = 0, queued tasks = 0, completed tasks = 0]
toString() is the cheapest diagnostic there is: pool size, active threads, queued tasks and completed tasks. The checkout pool has 4 of 4 threads active, 26 tasks queued and 0 completed after half a second. A completed count that does not move is the difference between a busy pool and a stuck one.Thread dumpjcmd <pid> Thread.print: an idle worker and a stuck worker
Real JVM output636259: 2026-11-15 21:41:35 Full thread dump OpenJDK 64-Bit Server VM (21.0.13+9-LTS mixed mode, sharing) ... "report-exec-1" #19 [636455] prio=5 os_prio=0 cpu=0.38ms elapsed=3.31s tid=0x00007ad10813e1e0 nid=636455 waiting on condition [0x00007ad09a9fe000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.13/Native Method) - parking to wait for <0x00000000ffb9efc8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(java.base@21.0.13/LockSupport.java:371) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.13/AbstractQueuedSynchronizer.java:519) at java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.13/ForkJoinPool.java:3780) at java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.13/ForkJoinPool.java:3725) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.13/AbstractQueuedSynchronizer.java:1746) at java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.13/LinkedBlockingQueue.java:435) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.13/ThreadPoolExecutor.java:1070) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.13/ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.13/ThreadPoolExecutor.java:642) at java.lang.Thread.runWith(java.base@21.0.13/Thread.java:1596) at java.lang.Thread.run(java.base@21.0.13/Thread.java:1583) ... "checkout-exec-1" #21 [636465] prio=5 os_prio=0 cpu=0.26ms elapsed=3.30s tid=0x00007ad1081403d0 nid=636465 runnable [0x00007ad09a7fe000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.SocketDispatcher.read0(java.base@21.0.13/Native Method) at sun.nio.ch.SocketDispatcher.read(java.base@21.0.13/SocketDispatcher.java:48) at sun.nio.ch.NioSocketImpl.tryRead(java.base@21.0.13/NioSocketImpl.java:256) at sun.nio.ch.NioSocketImpl.implRead(java.base@21.0.13/NioSocketImpl.java:307) at sun.nio.ch.NioSocketImpl.read(java.base@21.0.13/NioSocketImpl.java:346) at sun.nio.ch.NioSocketImpl$1.read(java.base@21.0.13/NioSocketImpl.java:796) at java.net.Socket$SocketInputStream.read(java.base@21.0.13/Socket.java:1099) at java.net.Socket$SocketInputStream.read(java.base@21.0.13/Socket.java:1093) at ExecutorsThreadPools.callInventory(ExecutorsThreadPools.java:98) at ExecutorsThreadPools.lambda$dump$3(ExecutorsThreadPools.java:81) at ExecutorsThreadPools$Lambda/0x00007ad09c004c00.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.13/ThreadPoolExecutor.java:1144) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.13/ThreadPoolExecutor.java:642) at java.lang.Thread.runWith(java.base@21.0.13/Thread.java:1596) at java.lang.Thread.run(java.base@21.0.13/Thread.java:1583)
report-exec-1 is an idle worker: WAITING (parking) with LinkedBlockingQueue.take called from ThreadPoolExecutor.getTask. Threads in getTask are spare capacity. checkout-exec-1 is busy and stuck: below runWorker there is application code, ExecutorsThreadPools.callInventory, and above it SocketInputStream.read. Its state is RUNNABLE although it uses no CPU (cpu=0.26ms after elapsed=3.30s): a thread blocked in a native socket read is reported as RUNNABLE. In the full dump all four checkout-exec- threads show this same stack. Every thread of a pool in the same frame of the same downstream call is the signature of thread starvation, and the frame names the dependency.In a Spring Boot service the picture has two layers. The Tomcat request threads wait for the pool, and the pool's threads wait for the dependency. A request thread that waits on a future looks like this:
Thread dumpA Tomcat request thread waiting for a pool task
Illustrative, not from a run"http-nio-8080-exec-17" #58 [70412] daemon prio=5 os_prio=0 cpu=312.44ms elapsed=1841.20s tid=0x00007f3a2c0b1a10 nid=70412 waiting on condition [0x00007f39d4dfc000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.13/Native Method) - parking to wait for <0x00000000e1a40d18> (a java.util.concurrent.CompletableFuture$Signaller) at java.util.concurrent.locks.LockSupport.park(java.base@21.0.13/LockSupport.java:221) at java.util.concurrent.CompletableFuture$Signaller.block(java.base@21.0.13/CompletableFuture.java:1864) at java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.13/ForkJoinPool.java:3780) at java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.13/ForkJoinPool.java:3725) at java.util.concurrent.CompletableFuture.waitingGet(java.base@21.0.13/CompletableFuture.java:1898) at java.util.concurrent.CompletableFuture.join(java.base@21.0.13/CompletableFuture.java:2117) at com.example.shipping.ShippingQuoteController.quotes(ShippingQuoteController.java:47) ...
http-nio-8080-exec- threads are parked in join() or get() under one controller method, and all threads of the pool they wait for are in one downstream call, the request pool is not the cause. Raising server.tomcat.threads.max would only let more requests wait.- Latency up, CPU low, errors flat: suspect waiting, not computing. Check the executor's active count against its size and whether the queue is growing.
- Pool threads all in the same downstream frame: the dependency is slow or has no timeout. Fix the timeout first, then the pool.
- Pool threads idle in
getTaskwhile requests are slow: this pool is not the bottleneck. Look at the request pool, the connection pool or a lock. - Pool threads parked in
join()orget()on tasks of the same pool: the pool is waiting for itself. That case is the subject of the next lesson.
Java and Spring Boot
A Spring Boot 3.5 service with two kinds of background work: carrier calls that may fail fast, and audit records that must not be lost. Each gets its own named, bounded pool. The Spring files were written against Spring Boot 3.5 and Spring Framework 6.2 and were not compiled here.
server: tomcat: threads: max: 200 # request threads (the default) min-spare: 10 # the default accept-count: 100 # connections the OS may queue when every thread is busy (the default) mbeanregistry: enabled: true # needed for the tomcat.threads.* metricsspring: task: execution: # Spring Boot's own executor, bean name applicationTaskExecutor mode: force # keep it although this service defines its own Executor beans thread-name-prefix: app-task- pool: core-size: 8 # the default max-size: 16 queue-capacity: 100 keep-alive: 10s shutdown: await-termination: true await-termination-period: 20smanagement: endpoints: web: exposure: include: health,metrics,threaddumpWhy it is written this way
- Out of the box the auto-configured executor has 8 core threads and an unbounded queue, so
max-sizeis ignored and nothing is ever rejected. Settingqueue-capacityis what turnsmax-sizeon. - With
queue-capacity: 100the pool grows from 8 to 16 threads only after 100 tasks are already waiting. If you want 16 threads working before anything queues, setcore-size: 16. - Spring Boot backs off from creating this executor when the application registers an
Executorbean of its own. This service defines two below, somode: forceis needed to keep the auto-configured one as well; addspring.task.execution.mode: forceif MVC async requests and unnamed@Asyncmethods should still use it. await-terminationmakes the executor wait for accepted tasks during shutdown, up to the period, instead of abandoning them. Both are off by default.- The
threaddumpActuator endpoint returns the same information asjcmd Thread.printover HTTP. Do not expose it publicly.
package com.example.shipping;import java.util.concurrent.ThreadPoolExecutor;import org.springframework.context.annotation.Bean;import org.springframework.context.annotation.Configuration;import org.springframework.scheduling.annotation.EnableAsync;import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;@Configuration@EnableAsyncpublic class ExecutorConfig { /** Carrier calls: a request may be refused, it must never wait without limit. */ @Bean public ThreadPoolTaskExecutor carrierExecutor() { ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor(); executor.setThreadNamePrefix("carrier-exec-"); executor.setCorePoolSize(24); executor.setMaxPoolSize(24); executor.setQueueCapacity(48); executor.setRejectedExecutionHandler(new ThreadPoolExecutor.AbortPolicy()); executor.setWaitForTasksToCompleteOnShutdown(true); executor.setAwaitTerminationSeconds(10); return executor; } /** Audit records: nothing may be dropped, so a full queue slows the caller down. */ @Bean public ThreadPoolTaskExecutor auditExecutor() { ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor(); executor.setThreadNamePrefix("audit-exec-"); executor.setCorePoolSize(4); executor.setMaxPoolSize(4); executor.setQueueCapacity(200); executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy()); executor.setWaitForTasksToCompleteOnShutdown(true); executor.setAwaitTerminationSeconds(30); return executor; }}Why it is written this way
ThreadPoolTaskExecutorwraps aThreadPoolExecutor. Its own defaults are a core size of 1, an unlimited maximum and an unlimited queue, which behaves like a single-thread executor. Always set all three.- A positive
queueCapacitygives aLinkedBlockingQueueof that capacity; any other value gives aSynchronousQueue. - Two pools, two policies. A refused carrier call becomes a 503. A full audit queue makes the calling thread write the record itself, which slows that request but loses nothing.
- With the default
AbortPolicy, Spring reports the rejection asTaskRejectedException, a subclass ofRejectedExecutionException. - Spring Boot's Actuator instruments
ThreadPoolTaskExecutorbeans and tags the meters with the bean name, soexecutor.activeandexecutor.queuedexist forcarrierExecutorandauditExecutorseparately.
package com.example.shipping;import org.slf4j.Logger;import org.slf4j.LoggerFactory;import org.springframework.scheduling.annotation.Async;import org.springframework.stereotype.Service;@Servicepublic class AuditTrail { private static final Logger log = LoggerFactory.getLogger(AuditTrail.class); private final AuditRepository repository; public AuditTrail(AuditRepository repository) { this.repository = repository; } @Async("auditExecutor") public void record(AuditEvent event) { try { repository.save(event); } catch (RuntimeException e) { log.error("audit record lost: {}", event, e); throw e; } }}Why it is written this way
@Async("auditExecutor")names the pool. Without a name,@Asyncuses the application's default executor, and unrelated features end up sharing one pool.@Asyncworks through a Spring proxy. A call from another method of the same class goes straight to the method and runs synchronously on the caller's thread.- The method returns
void, so the caller cannot see a failure. Spring passes the exception to anAsyncUncaughtExceptionHandler, which only logs it by default. Catch and log with context here, or return aCompletableFuture. AuditRepositoryandAuditEventare assumed from the rest of the project.
Wrong, then right
A shipping service asks three carriers for a quote on every request. Normal traffic is 40 requests per second and a carrier answers in about 150 ms, so 120 calls per second keep about 18 threads busy. The first version works for months, then one carrier slows down to 2 seconds per call. Neither version was compiled here; both are written against Spring Boot 3.5.
Incorrect implementation
package com.example.shipping;import java.util.ArrayList;import java.util.List;import java.util.concurrent.ExecutorService;import java.util.concurrent.Executors;import java.util.concurrent.Future;import org.springframework.context.annotation.Bean;import org.springframework.context.annotation.Configuration;import org.springframework.web.bind.annotation.GetMapping;import org.springframework.web.bind.annotation.PathVariable;import org.springframework.web.bind.annotation.RestController;import org.springframework.web.client.RestClient;@Configurationclass CarrierConfig { @Bean(destroyMethod = "shutdown") ExecutorService carrierExecutor() { return Executors.newFixedThreadPool(20); // unbounded queue, threads named pool-1-thread-N } @Bean RestClient carriers(RestClient.Builder builder) { return builder.baseUrl("https://carriers.internal.example").build(); // no timeouts }}@RestControllerclass ShippingQuoteController { private static final List<String> CARRIERS = List.of("north", "rapid", "blue"); private final ExecutorService carrierExecutor; private final RestClient carriers; ShippingQuoteController(ExecutorService carrierExecutor, RestClient carriers) { this.carrierExecutor = carrierExecutor; this.carriers = carriers; } @GetMapping("/quotes/{orderId}") List<Quote> quotes(@PathVariable String orderId) throws Exception { List<Future<Quote>> pending = new ArrayList<>(); for (String carrier : CARRIERS) { pending.add(carrierExecutor.submit(() -> carriers.get() .uri("/{carrier}/quotes/{orderId}", carrier, orderId) .retrieve() .body(Quote.class))); } List<Quote> quotes = new ArrayList<>(); for (Future<Quote> future : pending) { quotes.add(future.get()); // waits for as long as it takes } return quotes; }}Why it is written this way
Executors.newFixedThreadPool(20)is aThreadPoolExecutorwith 20 core threads, 20 maximum threads and an unboundedLinkedBlockingQueue. It cannot reject.- The
RestClienthas no connect or read timeout, so one call holds a pool thread for as long as the carrier keeps the connection open. future.get()has no timeout either, so the Tomcat request thread waits for the pool thread, which waits for the carrier.- The pool's threads are called
pool-1-thread-7. In a dump with several pools nobody can tell which is which.
Why it fails
- With one carrier at 2 seconds, a third of the 120 calls per second take 2 s instead of 0.15 s. Those 40 slow calls per second alone need 40 × 2 = 80 threads. The pool has 20, so within a second every pool thread is inside a call to the slow carrier.
- The queue is unbounded, so
submitkeeps succeeding. Calls to the two healthy carriers now wait in the same queue behind the slow ones. The queue grows by tens of tasks per second and no exception is thrown anywhere. - Each request thread sits in
future.get()with no timeout. Tomcat has 200 request threads by default and 40 requests arrive per second, so in about 5 seconds every request thread is parked. New connections wait in the accept queue and clients start to time out, including clients of endpoints that never touch a carrier. - What you see in production: p99 latency rising from 200 ms to many seconds, CPU falling, zero errors in the service's own logs, heap usage climbing with the queue, and a thread dump with 20
pool-1-thread-Nthreads inSocketInputStream.readand 200http-nio-8080-exec-Nthreads inFutureTask.get. - Recovery is slow too. When the carrier comes back, the pool first works through the backlog of calls whose callers gave up long ago.
Corrected implementation
package com.example.shipping;import java.net.http.HttpClient;import java.time.Duration;import java.util.List;import java.util.Objects;import java.util.concurrent.CompletableFuture;import java.util.concurrent.TimeUnit;import org.springframework.context.annotation.Bean;import org.springframework.context.annotation.Configuration;import org.springframework.core.task.TaskRejectedException;import org.springframework.http.HttpHeaders;import org.springframework.http.HttpStatus;import org.springframework.http.ResponseEntity;import org.springframework.http.client.JdkClientHttpRequestFactory;import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;import org.springframework.web.bind.annotation.GetMapping;import org.springframework.web.bind.annotation.PathVariable;import org.springframework.web.bind.annotation.RestController;import org.springframework.web.client.RestClient;@Configurationclass CarrierConfig { @Bean ThreadPoolTaskExecutor carrierExecutor() { ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor(); executor.setThreadNamePrefix("carrier-exec-"); executor.setCorePoolSize(24); executor.setMaxPoolSize(24); executor.setQueueCapacity(48); // AbortPolicy is the default handler executor.setWaitForTasksToCompleteOnShutdown(true); executor.setAwaitTerminationSeconds(10); return executor; } @Bean RestClient carriers(RestClient.Builder builder) { HttpClient http = HttpClient.newBuilder().connectTimeout(Duration.ofMillis(500)).build(); JdkClientHttpRequestFactory factory = new JdkClientHttpRequestFactory(http); factory.setReadTimeout(Duration.ofMillis(800)); return builder.baseUrl("https://carriers.internal.example").requestFactory(factory).build(); }}@RestControllerclass ShippingQuoteController { private static final List<String> CARRIERS = List.of("north", "rapid", "blue"); private final ThreadPoolTaskExecutor carrierExecutor; private final RestClient carriers; ShippingQuoteController(ThreadPoolTaskExecutor carrierExecutor, RestClient carriers) { this.carrierExecutor = carrierExecutor; this.carriers = carriers; } @GetMapping("/quotes/{orderId}") ResponseEntity<List<Quote>> quotes(@PathVariable String orderId) { List<CompletableFuture<Quote>> pending; try { pending = CARRIERS.stream() .map(carrier -> CompletableFuture .supplyAsync(() -> fetch(carrier, orderId), carrierExecutor) .orTimeout(1, TimeUnit.SECONDS) .exceptionally(failure -> null)) .toList(); } catch (TaskRejectedException overloaded) { return ResponseEntity.status(HttpStatus.SERVICE_UNAVAILABLE) .header(HttpHeaders.RETRY_AFTER, "1") .build(); } List<Quote> quotes = pending.stream() .map(CompletableFuture::join) // at most 1 s per future because of orTimeout .filter(Objects::nonNull) .toList(); return quotes.isEmpty() ? ResponseEntity.status(HttpStatus.BAD_GATEWAY).build() : ResponseEntity.ok(quotes); } private Quote fetch(String carrier, String orderId) { return carriers.get() .uri("/{carrier}/quotes/{orderId}", carrier, orderId) .retrieve() .body(Quote.class); }}Why it is written this way
- The read timeout of 800 ms is the most important line. It caps how long one call can hold a pool thread, which caps how many threads a slow carrier can occupy: 40 calls per second × 0.8 s = 32, more than the pool, so the queue and the rejection still matter.
- 24 threads with a queue of 48: at the normal 150 ms per call the pool drains 160 calls per second, so a full queue adds about 48 / 160 = 0.3 s of wait at worst.
- A full queue makes
supplyAsyncthrowTaskRejectedExceptionin the request thread. The endpoint answers 503 withRetry-Afterat once instead of holding a request thread. orTimeout(1, SECONDS)bounds the wait of the request thread, including time spent in the queue.exceptionallyturns a failed or late carrier into a missing quote, so two healthy carriers still produce an answer.- If the second or third
supplyAsyncis rejected, the earlier ones were already accepted and still run. Their results are discarded. That waste is bounded by the pool and the read timeout. JdkClientHttpRequestFactoryandRestClientneed Spring Framework 6.1 or later (Spring Boot 3.2 or later).
Trade-offs
- Some requests now fail with 503 during overload. That is deliberate: the alternative was all requests failing slowly. Clients need to handle 503 and
Retry-After. - Timeouts turn a slow success into a failure. A carrier that sometimes needs 900 ms now loses those quotes; the timeout has to be set from the carrier's real latency distribution, not guessed.
- One slow carrier can still fill the shared pool and cause rejections for the two healthy ones. Full isolation needs a pool or a concurrency limit per carrier, which is more threads and more configuration.
- A timed-out future does not stop the HTTP call. The pool thread stays busy until the read timeout fires, so the two limits must be chosen together.
- The pool size, queue size and timeouts are now numbers somebody has to revisit when traffic or the carriers change. Alert on rejections and on queue length so that the numbers are checked by data.
Configuration
ThreadPoolExecutor has no flags: its settings are constructor arguments. The Spring Boot properties below are from the Spring Boot 3.5 reference. Where the reference lists no default, the value in effect comes from ThreadPoolTaskExecutor.
spring.task.execution.pool.core-sizeDefault 8
- What it does
- Core threads of the auto-configured
applicationTaskExecutor, which runs@Asyncmethods and Spring MVC async requests when you define no executor of your own. - When to change it
- Size it from the blocking work it runs. With the default unbounded queue this is also the maximum, so it is the pool's whole capacity.
spring.task.execution.pool.queue-capacityDefault none listed (unbounded: ThreadPoolTaskExecutor uses Integer.MAX_VALUE)
- What it does
- Capacity of the queue. The reference notes that an unbounded capacity does not increase the pool and therefore ignores
max-size. - When to change it
- Set it. Choose the capacity from the wait you accept: capacity × task time ÷ threads.
spring.task.execution.pool.max-sizeDefault none listed (ThreadPoolTaskExecutor uses Integer.MAX_VALUE)
- What it does
- Maximum threads. The pool expands to it only when tasks are filling the queue. Ignored if the queue is unbounded.
- When to change it
- Has an effect only together with
queue-capacity. Setting it equal tocore-sizegives a fixed pool that is easier to reason about.
spring.task.execution.pool.keep-aliveDefault 60s
- What it does
- Time a thread may stay idle before it is terminated.
- When to change it
- Lower it if a burst leaves many idle threads behind. It rarely matters for a fixed pool.
spring.task.execution.pool.allow-core-thread-timeoutDefault true
- What it does
- Whether core threads may also time out, so an idle pool shrinks to zero threads.
- When to change it
- Leave it on unless the first requests after a quiet period must not pay for thread creation.
spring.task.execution.thread-name-prefixDefault task-
- What it does
- Prefix of the names of the executor's threads.
- When to change it
- Change it to something that identifies the application's pool in thread dumps and logs.
spring.task.execution.shutdown.await-terminationDefault false
- What it does
- Whether the executor waits for tasks to complete on shutdown.
await-termination-periodsets the longest wait and has no default. - When to change it
- Turn it on with a period shorter than the platform's termination grace period, or accepted tasks are abandoned on every deploy.
server.tomcat.threads.maxDefault 200
- What it does
- Maximum number of Tomcat worker threads, which is the number of requests handled at once by blocking controllers.
server.tomcat.threads.min-spare(default 10) is the minimum kept alive. - When to change it
- Raise it only when request threads are busy with useful waiting and the dependencies behind them can take more. Raising it during starvation makes more requests wait.
server.tomcat.accept-countDefault 100
- What it does
- Maximum queue length for incoming connection requests when all request processing threads are in use.
server.tomcat.max-connections(default 8192) limits connections accepted and processed at one time. - When to change it
- Time spent in this queue is invisible to the application's own request timer. Keep it small enough that clients fail fast rather than wait behind a stuck service.
spring.threads.virtual.enabledDefault false
- What it does
- On Java 21 or later, runs request handling and the auto-configured task executor on virtual threads. The pool properties above then have no effect.
- When to change it
- It removes the thread limit, not the limit of what you call. See the virtual threads lesson before turning it on.
Hands-on exercise
Three answers to the same overload
Drive a pool of 4 threads at five times its capacity and compare what an unbounded queue, AbortPolicy and CallerRunsPolicy each do with the excess, then size the pool so that nothing queues.
Steps · 0 of 7 done
Pool of 4, queue of 8, 100 requests per second
Simulation · not a real JVMChange one control at a time: the queue bound, the rejection policy, the thread count and the task time. Watch active threads, queue length, queue wait and rejections.
What you should see
The pool finishes 20 tasks per second against 100 arriving. With the queue of 8 and abort, most requests are rejected and accepted ones wait no more than about 400 ms; the real lab rejected 39 of 60. Unbounded, nothing is rejected and the queue and the wait grow without limit; the real lab reached 47 queued and a wait of 2,225 ms after 60 requests. Caller-runs rejects nothing and slows submission to what the pool can take; the real lab needed 2,223 ms to submit instead of 600. Rejections stop at about 20 threads, which is 100 × 0.2, and return when the task time doubles.
What can go wrong
Setting a maximum pool size with an unbounded queue
- Symptom
- The pool is configured as core 10, max 100, yet a thread dump under heavy load shows exactly 10 worker threads and latency keeps rising.
- Cause
- Threads above core are created only when the queue refuses a task.
LinkedBlockingQueuewithout a capacity, and Spring's default queue capacity, never refuse. - Fix
- Give the queue a capacity, or set core equal to max and treat the pool as fixed. Verify with
executor.pool.sizeunder load.
Using the default executor for everything
- Symptom
- A slow report export makes unrelated
@Asyncemails and MVC async requests late.task-1totask-8are all in the export code. - Cause
- Spring Boot's auto-configured executor has 8 threads and an unbounded queue, and every unnamed
@Asyncmethod shares it. - Fix
- Define a named, bounded executor per kind of work and reference it with
@Async("name").
Exceptions that vanish inside submit()
- Symptom
- A background task stops having any effect. Nothing is logged.
- Cause
submitwraps the task in aFutureand stores any exception there. If nobody callsget(), nobody sees it.executewould have passed it to the thread's uncaught exception handler.- Fix
- Keep and check the
Future, or catch and log inside the task. For@Async voidmethods, log in the method or register anAsyncUncaughtExceptionHandler.
CallerRunsPolicy on a request thread
- Symptom
- After the pool's queue fills, the HTTP request pool fills as well, and some requests run the slow task with the wrong security or transaction context.
- Cause
- The rejected task runs in the thread that submitted it, with that thread's thread-locals, and that thread is blocked for the task's whole duration.
- Fix
- Use
AbortPolicyand a fast error for request paths. KeepCallerRunsPolicyfor loops and listeners where slowing the producer is the intent.
Sizing from the host's core count inside a container
- Symptom
- A CPU-bound pool of 32 threads in a container limited to 2 CPUs: throughput is poor and the container is throttled.
- Cause
- The size was taken from the machine the code was written on. The JVM sees the container's CPU limit, and extra runnable threads only add context switches.
- Fix
- Derive CPU-bound pool sizes from
Runtime.getRuntime().availableProcessors()at start-up and log the value.
Never shutting the pool down
- Symptom
- The JVM does not exit after
mainreturns, or a redeploy drops work that had been accepted. - Cause
- Pool threads are non-daemon by default and stay alive waiting for tasks. Stopping the process without
shutdown()andawaitTerminationabandons queued tasks. - Fix
- Call
shutdown()thenawaitTermination, or let Spring manage the bean and setsetWaitForTasksToCompleteOnShutdown(true)with a termination period.
How to troubleshoot
Latency climbs while CPU stays low and the error rate is flat
Check
Micrometer gauges `executor.active`, `executor.pool.size` and `executor.queued` (tag `name` = the executor bean), for example GET /actuator/metrics/executor.queued?tag=name:carrierExecutor
- How to read it
- Active equal to pool size with a growing queue means the pool is saturated.
executor.queue.remainingat 0 means a bounded queue is full and the next submission is rejected.
Need to know what a saturated pool's threads are doing
Check
jcmd <pid> Thread.print > dump.txt, then grep -A 12 '"carrier-exec-' dump.txt
- How to read it
- Workers in
ThreadPoolExecutor.getTaskare idle. Workers with application frames aboverunWorkerare busy: if they all show the same downstream call, that dependency is the cause. Take a second dump 10 seconds later; the same threads in the same frames means stuck, not merely busy.
Suspect the Tomcat request pool is exhausted
Check
Micrometer gauges `tomcat.threads.busy` and `tomcat.threads.config.max` (needs server.tomcat.mbeanregistry.enabled=true), and grep -c '"http-nio-' dump.txt
- How to read it
- Busy equal to max means every request thread is occupied and new connections wait in the accept queue. Look at where those threads are parked before raising the limit.
Clients report timeouts but the service's own request timer looks healthy
Check
Compare client-side latency or load balancer latency with the `http.server.requests` timer
- How to read it
- A gap means requests wait before the application sees them, in the accept queue or for a request thread. The server-side timer starts only when a thread picks the request up.
RejectedExecutionException or TaskRejectedException in the logs
Check
Read the exception message: `[Running, pool size = 4, active threads = 4, queued tasks = 2, completed tasks = 0]`
- How to read it
Runningwith full threads and a full queue is saturation.Shutting downorTerminatedmeans a task was submitted aftershutdown(), which is a lifecycle bug and not a capacity problem.
Heap grows together with latency
Check
jcmd <pid> GC.class_histogram | head -20
- How to read it
- Many
LinkedBlockingQueue$Node,FutureTaskor lambda instances point at an unbounded executor queue holding waiting tasks.
Key takeaways
- A task goes to a core thread, then to the queue, then to a thread above core, then to the rejection handler, in that order.
- With an unbounded queue the maximum pool size is never used and nothing is ever rejected. Overload shows up only as waiting.
Executors.newFixedThreadPooland Spring Boot's default task executor both have unbounded queues. Bound them deliberately.- The rejection policy decides who pays for overload:
AbortPolicythe caller with an error,CallerRunsPolicythe caller with time, the discard policies the task. - Blocking work needs about arrival rate × task time threads. CPU-bound work needs about one thread per available processor.
- A pool can only be as healthy as the timeouts of the calls its tasks make. Bound the call before tuning the pool.
- In a dump, idle workers are in
getTask. A starved pool has every worker in the same downstream frame, often reported as RUNNABLE.
Practical check
Small scenarios, one decision each. Pick what you would do, then read why.
Scenario 1 of 4
A pool is created with corePoolSize 10, maximumPoolSize 50 and new LinkedBlockingQueue<>() as its queue. During a traffic peak the queue holds 4,000 tasks and p99 latency is 30 seconds. A thread dump shows exactly 10 worker threads, all busy.
Why did the pool not grow towards 50 threads?
Scenario 2 of 4
A Kafka listener thread hands each record to a pool of 8 threads with a queue of 100 for a database write of about 50 ms. During a database slowdown the queue fills. Every record must be written, and the team is happy for consumption to slow down while the database is slow.
Which configuration fits these requirements?
Scenario 3 of 4
p99 latency of a checkout API went from 300 ms to 9 s. CPU is at 6 percent. A thread dump shows 200 http-nio-8080-exec threads parked in CompletableFuture.join under CheckoutController, and all 16 payment-exec threads RUNNABLE in SocketDispatcher.read0 under PaymentClient.authorize.
What should be changed first?
Scenario 4 of 4
An image service resizes uploads, which is pure CPU work of about 80 ms per image, on a pool of 64 threads. It runs in a container limited to 4 CPUs. At peak, throughput is about 45 images per second and the container reports CPU throttling.
Which change is most likely to help?
References
References: Java SE 21 API: ThreadPoolExecutor (core and maximum pool sizes, queuing, rejected tasks) ↗ · Java SE 21 API: Executors (newFixedThreadPool, newCachedThreadPool) ↗ · Java SE 21 tool reference: jcmd (Thread.print, GC.class_histogram) ↗ · Spring Boot 3.5 reference: Task Execution and Scheduling ↗ · Spring Boot 3.5: Common Application Properties (spring.task.execution.*, server.tomcat.*) ↗ · Spring Boot 3.5 reference: Metrics (task execution and Tomcat metrics) ↗ · Spring Framework 6.2 reference: Task Execution and Scheduling (ThreadPoolTaskExecutor, @Async) ↗ · Spring Framework 6.2 API: ThreadPoolTaskExecutor ↗ · Report a problem