Path 3 · lesson 2 of 6
Race conditions: visibility and atomicity
Lost updates, check-then-act races and flags that are never seen: two separate problems, how each one looks in production, and why both pass tests and fail under load.
What you will learn
- Explain a lost update from the three bytecode steps of
count++and the interleaving that loses one - Recognise check-then-act in service code, including when every single call is on a thread-safe collection
- Separate atomicity from visibility and say which of the two
volatileprovides - Describe how a stop flag can be written and never seen, and what in the JVM makes that legal
- Give three reasons these defects pass unit tests and appear under production load
- Turn a suspected race into evidence with a latch-based stress run and a thread dump
Before you start
The concept
A race condition exists when the result of a piece of code depends on how the operations of two or more threads happen to interleave. Most of the time the interleaving is harmless and the result is right. Sometimes it is not, and nothing in the code changed in between.
A Spring @Service is a singleton by default: the container creates one instance and hands the same object to every caller. Every request thread therefore runs the same methods on the same fields. Any field that changes after startup is shared mutable state, and shared mutable state is where races live.
Two different things can go wrong with it. They have different causes and different fixes, and confusing them is the most common mistake in this area.
Atomicity failure
- An action that the code treats as one step is really several, and another thread gets in between them.
- Read-modify-write:
count++,total += amount,map.put(k, map.get(k) + 1). - Check-then-act:
if (!map.containsKey(k)) map.put(k, load(k)),if (stock > 0) stock--,if (instance == null) instance = create(). - Symptom: lost updates, duplicates, limits exceeded. Numbers that are a little wrong.
- Needs mutual exclusion or a single atomic operation: a lock, an atomic class, an atomic method of a concurrent collection.
Visibility failure
- A write by one thread is not seen by another, although the code runs in the order you expect.
- A stop flag, a configuration value, a reference to a refreshed cache.
- Symptom: a thread keeps acting on an old value, possibly for ever. Behaviour that is completely wrong for one thread and fine for the rest.
- Needs a happens-before edge between the write and the read:
volatile, a lock, or a hand-over throughjava.util.concurrent. - No amount of visibility makes a compound action atomic.
The three constructs you will reach for map onto the two problems like this. The following lessons take each in detail.
| Tool | Visibility | Atomicity of a compound action | Lesson |
|---|---|---|---|
volatile field | Yes | No | synchronized, volatile and happens-before |
synchronized or a Lock | Yes | Yes, for everything done while holding the lock | Locks |
AtomicInteger, LongAdder | Yes | Yes, for one variable and one method call | Atomics and CAS |
ConcurrentHashMap atomic methods | Yes | Yes, for one key and one method call (computeIfAbsent, merge, putIfAbsent) | Concurrent collections |
Why it exists
These defects do not crash anything. They produce numbers that are slightly off and behaviour that is occasionally wrong, at a rate that rises with load. By the time someone notices, the evidence is a discrepancy between two systems.
- Counters drift. The request counter in the service says 9.6 million, the load balancer says 10 million. Nobody trusts either.
- Limits leak. A quota of 1,000 requests lets 1,004 through, or 1,500. A stock of 1 is sold twice.
- Work is duplicated. An expensive cache entry is loaded by five threads at once on a cold start, and the downstream service sees five times the calls exactly when it is slowest.
- A switch does not switch. A feature flag or a shutdown flag is set and one worker carries on as before.
- Collections break.
HashMapis documented as not synchronised: if several threads use it and at least one modifies it structurally, it must be synchronised externally. Without that its behaviour is not defined by its contract.
The cost is rarely the bug itself. It is the days spent on a defect that cannot be reproduced on a laptop, and the reflex fix of making every method synchronized, which trades a correctness problem for the throughput problem shown in lock contention.
How it works inside
One line of Java, four instructions
hits++ on a field looks atomic and is not. This is what javac produces for void record() { hits++; }:
Program outputjavap -c -p 'QuotaRace$HitCounter'
Real JVM output
void record();
Code:
0: aload_0
1: dup
2: getfield #7 // Field hits:I
5: iconst_1
6: iadd
7: putfield #7 // Field hits:Igetfield reads the current value onto the thread's own operand stack, iconst_1 and iadd compute the new value there, putfield writes it back. Between getfield and putfield the thread works on a private copy. Any write another thread makes in that window is overwritten.| Step | Thread A | Thread B | Field hits |
|---|---|---|---|
| 1 | getfield reads 41 | 41 | |
| 2 | getfield reads 41 | 41 | |
| 3 | iadd computes 42 | iadd computes 42 | 41 |
| 4 | putfield writes 42 | 42 | |
| 5 | putfield writes 42 | 42 |
The JIT compiler turns these bytecodes into a handful of machine instructions, and on a multi-core machine two threads can execute them truly at the same time, so the window does not need a context switch to be hit. The lost updates lab measures how often it is hit when 8 threads do nothing else:
Program outputlabs/jvm/lost-updates: java LostUpdate.java
Real JVM output8 threads x 200,000 increments, expected 1,600,000 int, no synchronisation result 734,295 lost 865,705 (54.1%) 19 ms volatile int result 951,024 lost 648,976 (40.6%) 15 ms synchronized block result 1,600,000 lost 0 ( 0.0%) 34 ms AtomicInteger.incrementAndGet result 1,600,000 lost 0 ( 0.0%) 10 ms LongAdder.increment result 1,600,000 lost 0 ( 0.0%) 18 ms
int lost 865,705 of 1,600,000 increments, more than half. The volatile int lost 648,976: volatile made every write visible and did nothing for the gap between read and write. The three variants that make read-add-write one indivisible action lost nothing.Check-then-act: the decision goes stale
The second family has the same shape with a condition in the middle: read some state, decide, then act on the decision. By the time the thread acts, the state it checked may have changed. Using a thread-safe collection does not help, because each call is atomic and the pair is not. extras/RaceVisibility.java lets 8 threads populate one cache key, first with containsKey followed by put, then with computeIfAbsent:
Program outputjava -XX:ActiveProcessorCount=2 -Xmx64m RaceVisibility.java checkthenact
Real JVM output8 threads race to create one cache entry, 20,000 rounds containsKey then put loads 21,078 (expected 20,000) rounds with more than one load 665 worst round 8 loads computeIfAbsent loads 20,000 (expected 20,000) rounds with more than one load 0 worst round 1 loads
containsKey then put on a ConcurrentHashMap, 665 of 20,000 rounds loaded the value more than once, and in the worst round all 8 threads did. That is 3.3% of rounds, with every thread released by a latch at the same instant. computeIfAbsent performs the check and the insertion as one atomic invocation: exactly one load per round.Read the other number too: in 96.7% of rounds the broken version behaved correctly, under the most hostile timing a test can arrange. A test that runs the scenario once will pass about 29 times out of 30.
Visibility: the write that never arrives
Now the other problem. A worker spins on while (!stop). One second later the main thread sets stop = true and waits up to five seconds for the worker to end. There is exactly one write and it is as atomic as a write can be.
Program outputRaceVisibility.java stopflag: plain field, volatile field, and plain field with the JIT compiler off
Real JVM output$ java -XX:ActiveProcessorCount=2 -Xmx64m RaceVisibility.java stopflag plain stop flag is plain boolean main: wrote stop = true after 1000 ms worker: still running 5003 ms after the write, state RUNNABLE (it never saw the write) $ java -XX:ActiveProcessorCount=2 -Xmx64m RaceVisibility.java stopflag volatile stop flag is volatile boolean main: wrote stop = true after 1000 ms worker: stopped 6 ms after the write, after 88,123,974 loop iterations $ java -XX:ActiveProcessorCount=2 -Xmx64m -Xint RaceVisibility.java stopflag plain stop flag is plain boolean main: wrote stop = true after 1000 ms worker: stopped 3 ms after the write, after 21,987,788 loop iterations
stop = true. Second run: the only change is the keyword volatile, and the worker stops within milliseconds. Third run: the plain field again, with -Xint so that all code is interpreted, and the worker stops. In these runs the hang needed compiled code.The memory model explains why this is legal. The loop body never writes stop, and nothing in the loop creates a happens-before edge with the writer. So the compiler may assume that, as far as this thread can tell, the field does not change, read it once, and compile the loop as if it were while (true). The language specification describes this permission with a loop of the same shape, and adds that Thread.sleep and Thread.yield have no synchronisation semantics: the compiler does not have to reload a cached value after them. The third run is consistent with that explanation; it does not prove what machine code the compiler generated.
- The hang is permitted, not promised. The same code may stop correctly on another JDK build, another CPU, with a logging call in the loop, or before the loop has been compiled. That is what makes it dangerous: it can appear after an unrelated change.
- A thread dump of the hung program shows the worker
RUNNABLEin the loop with itscpu=time climbing. It looks like a busy thread, not like a bug. - Visibility failures also affect references. A thread can keep using an old cache object after another thread has swapped in a new one.
Why these defects pass tests
- The window is tiny. An increment takes nanoseconds: the lab did 1,600,000 of them in 19 ms. Two requests have to be inside those nanoseconds together. At 10 requests a second that almost never happens; at 5,000 a second on 8 cores it happens all day.
- Unit tests are single-threaded. A test that calls
tryConsumea thousand times from one thread exercises none of this. - Short tests run different code. Methods start in the interpreter and are compiled only once they are hot. The stop-flag run above stops correctly when interpreted. A test that lasts 50 ms may never run the compiled loop that production runs for weeks. See JIT compilation.
- Fewer cores, fewer overlaps. A CI container with one CPU can only interleave threads at context switches. Production with 16 cores runs them truly in parallel.
- Failure is silent. A lost update throws nothing. The test would have to compare a count against an independent source of truth, and most do not.
The quota service from the next section, tested the way a careful engineer might test it, with 8 threads and a latch, 200 times over:
Program outputjava -XX:ActiveProcessorCount=2 -Xmx64m QuotaRace.java
Real JVM outputlimit 1,000 per tenant; 8 threads x 500 requests for one tenant; 200 rounds, fresh quota each round HashMap, read-check-write exactly 1,000 granted in 192 of 200 rounds over the limit in 8 rounds most granted 1,500 AtomicInteger, compareAndSet exactly 1,000 granted in 200 of 200 rounds over the limit in 0 rounds most granted 1,000
One more permission of the memory model belongs here: a non-volatile long or double may be written as two separate 32-bit halves, so in a data race a reader could see half of each of two values. Declare shared long and double fields volatile or guard them; the specification states that volatile reads and writes of them are always atomic.
Architecture
Drawing diagram…
A decision flowchart for a field of a singleton bean. If it never changes after startup it should be final and no race is possible. If it changes by plain overwrite the problem is visibility only and a volatile field is enough. If the new value depends on the old value or on a check, atomicity is also needed: an atomic class for one number, an atomic map method for one map entry, or one lock around several fields or steps.
flowchart TD
S["Field of a singleton bean"] --> Q1{"Does it change after startup?"}
Q1 -->|"no"| SAFE["Make it final. No race is possible"]
Q1 -->|"yes"| Q2{"Does the new value depend on the old one, or on a check?"}
Q2 -->|"no, plain overwrite"| VIS["Visibility only"]
Q2 -->|"yes"| ATOM["Atomicity and visibility"]
VIS --> V["volatile field"]
ATOM --> Q3{"How much state is involved?"}
Q3 -->|"one number"| A["AtomicInteger or LongAdder"]
Q3 -->|"one map entry"| C["computeIfAbsent, merge, compute"]
Q3 -->|"several fields or steps"| L["One lock around all of them"]- The first question removes most fields. Injected collaborators and configuration read at startup should be
final; a final field set in the constructor cannot be part of a race after the object is published. - The second question separates the two problems. A flag that is only ever set, or a reference that is only ever replaced, needs visibility. Anything of the form read, compute, write needs atomicity as well.
- The third question picks the smallest tool that covers the whole compound action. The tool has to cover everything between the read and the write, which is why a lock is the answer as soon as two fields must stay consistent with each other.
- Every branch on the right also gives visibility. You never need
volatilein addition to a lock or an atomic for the same variable.
Step by step
Drawing diagram…
A sequence diagram with two request threads and one shared counter for a tenant. Both threads read the value 999, both check that it is below the limit of 1000, and both write 1000. Two requests were granted but the counter increased by only one, so the tenant received one request more than the limit.
sequenceDiagram
participant A as Request thread A
participant F as usedByTenant for acme
participant B as Request thread B
A->>F: read used
F-->>A: 999
B->>F: read used
F-->>B: 999
A->>A: check 999 is under 1000, allowed
B->>B: check 999 is under 1000, allowed
A->>F: write 1000
B->>F: write 1000
Note over A,B: two requests granted, the counter moved by one
A->>F: next request reads 1000 and is refused- Both reads happen before either write. Each thread then holds a private copy of 999 and nothing tells it that the copy is out of date.
- The check and the write are separate steps. The decision made at the check is acted on later, when it may no longer be true.
- The damage is one extra request and one lost count per collision. It accumulates quietly: after enough collisions the counter is well below the true usage.
- Making the field
volatilechanges nothing in this diagram. Both threads already read the latest value; the problem is what happens between the read and the write.
In a real system
You will almost never catch a race in the act. You infer it, and then you prove it. The inference comes from a pattern of symptoms:
| What you observe | Likely defect | How to confirm |
|---|---|---|
| A count in the service is lower than the same count from an independent source, and the gap grows with traffic | Read-modify-write on a plain or volatile field, or on a map value | Find the field. Is every write a single atomic call or under one lock? |
| A limit is exceeded by a small amount at peak, never off-peak | Check-then-act | Look for if on shared state followed by a write to it, with no lock spanning both |
| An expensive load runs several times for the same key right after a deploy or a cache expiry | Check-then-act on a cache (get, test for null, put) | Count load log lines per key during a cold start |
| One thread behaves as if a flag or setting had never changed; a restart cures it | Visibility: plain field, written by another thread | Two thread dumps: the thread is RUNNABLE in the same loop with cpu= rising |
ConcurrentModificationException, or a collection in an impossible state | A non-thread-safe collection shared between threads | Find who else holds a reference to the collection |
To prove it, write the smallest possible stress run: many threads, a CountDownLatch to release them together, many rounds, and an invariant checked at the end. That is all QuotaRace.java is. Run it on a machine with at least two cores and let it run long enough for the code to be compiled. A run that fails once in 200 rounds is proof. A run that passes 200 times is not proof of correctness, only the absence of a failure. For library-grade concurrent code, OpenJDK publishes a dedicated harness for this kind of test, jcstress (opens in a new tab).
For visibility defects the thread dump is the only runtime evidence. This is how a thread stuck on a stale flag looks: nothing about it says stuck.
Thread dumpWhat a worker spinning on a stale flag looks like (format of jcmd <pid> Thread.print)
Illustrative, not from a run"worker" #19 [7412] daemon prio=5 os_prio=0 cpu=5980.42ms elapsed=6.01s tid=0x00007f3a2c4d1a60 nid=7412 runnable [0x00007f3a11bfd000] java.lang.Thread.State: RUNNABLE at RaceVisibility.lambda$stopFlag$0(RaceVisibility.java:34) at RaceVisibility$Lambda/0x00007f39a0001a18.run(Unknown Source) at java.lang.Thread.runWith(java.base@21.0.13/Thread.java:1596) at java.lang.Thread.run(java.base@21.0.13/Thread.java:1583)
cpu= value that is almost equal to elapsed=. The captured program output above reports the same state for the real worker: RUNNABLE, five seconds after the write.In a service, the place to look first is every singleton with a field that is not final. In a typical Spring Boot codebase that is a short list: counters, caches, rate limiters, circuit-breaker state, lazily initialised clients, and flags. Lock contention and CPU cost from the opposite mistake, too much locking, are the subject of CPU pinned with flat traffic.
Java and Spring Boot
A usage meter in a Spring Boot 3.x gateway, with three kinds of shared state and the smallest sufficient guard for each. It was compiled with javac 21 against a stand-in stub for @Service, not against Spring itself.
package com.example.gateway.usage;import java.util.Map;import java.util.concurrent.ConcurrentHashMap;import java.util.concurrent.atomic.AtomicInteger;import java.util.concurrent.atomic.LongAdder;import java.util.stream.Collectors;import org.springframework.stereotype.Service;/** Singleton bean called from every request thread. Each piece of state has one stated guard. */@Servicepublic class ApiUsageMeter { // Write-heavy statistics: one LongAdder per route, created atomically. private final ConcurrentHashMap<String, LongAdder> callsByRoute = new ConcurrentHashMap<>(); // A value that code branches on: one atomic variable, updated with single atomic calls. private final AtomicInteger inFlight = new AtomicInteger(); // Written rarely by one admin thread, read by all: volatile is enough because the write // does not depend on the current value. private volatile int maxInFlight = 200; /** Returns false when the request should be shed. */ public boolean tryEnter(String route) { callsByRoute.computeIfAbsent(route, r -> new LongAdder()).increment(); int limit = maxInFlight; // one read, then use the local copy while (true) { int current = inFlight.get(); if (current >= limit) return false; // check ... if (inFlight.compareAndSet(current, current + 1)) return true; // ... and act, atomically } } public void exit() { inFlight.decrementAndGet(); } public void setMaxInFlight(int max) { maxInFlight = max; } /** A report, not a snapshot: each sum() may miss increments that are in progress. */ public Map<String, Long> callsByRoute() { return callsByRoute.entrySet().stream() .collect(Collectors.toMap(Map.Entry::getKey, e -> e.getValue().sum())); }}Why it is written this way
computeIfAbsent(route, r -> new LongAdder())is one atomic invocation, so two threads meeting a new route cannot each install their own adder and lose the other's counts.LongAdder.increment()is atomic. Its documentation recommends it overAtomicLongwhen many threads update a sum used for statistics, at the expense of higher space consumption.tryEnteris a check-then-act made atomic withcompareAndSet: the write only succeeds if the value is still the one that was checked. If another thread got in between, the loop re-reads and decides again.maxInFlightis copied into a local once. Reading a volatile field twice in one method can give two different values.maxInFlightis volatile and not atomic because its writer overwrites it without reading it. That is the one case where volatile alone is correct with any number of writers.callsByRoute()is honest about what it returns:LongAdder.sum()is documented as not an atomic snapshot when updates are in progress. For monitoring that is fine. Do not branch on it.
Wrong, then right
A per-tenant daily quota in an API gateway, called once per request by whichever request thread is handling it. Both versions were compiled with javac 21 against a stand-in stub for @Service. Their logic is exactly what labs/jvm/extras/QuotaRace.java ran to produce the output shown earlier.
Incorrect implementation
package com.example.gateway.quota;import java.util.HashMap;import java.util.Map;import org.springframework.stereotype.Service;@Servicepublic class QuotaService { private final Map<String, Integer> usedByTenant = new HashMap<>(); // shared by every request thread private final int dailyLimit = 1_000; /** Called once per request, from whichever Tomcat thread handles it. */ public boolean tryConsume(String tenantId) { int used = usedByTenant.getOrDefault(tenantId, 0); // read if (used >= dailyLimit) { // check return false; } usedByTenant.put(tenantId, used + 1); // act: writes back a value based on a stale read return true; } public int used(String tenantId) { return usedByTenant.getOrDefault(tenantId, 0); }}Why it is written this way
- It reads like correct sequential code, and with one request at a time it is.
- Three defects in six lines: a check-then-act on
used, a read-modify-write throughgetOrDefaultandput, and aHashMapshared between threads with no synchronisation.
Why it fails
getOrDefaultandputare separate steps. Two request threads for the same tenant can both read 999, both pass the check and both write 1000: two requests are granted and the count rises by one. Every such collision lets one request through that should have been counted.- The check uses a value that can be stale by the time
putruns, so the limit itself is not enforced. In the stress run the service granted more than its limit of 1,000 in 8 of 200 rounds, up to 1,500. HashMapis documented as requiring external synchronisation when several threads use it and one of them adds a mapping. Here every first request of a new tenant is a structural modification that races with reads and writes for other tenants, so even a tenant whose own requests never overlap is exposed.- There is also no happens-before edge between a write by one request thread and a read by another, so in principle a thread may work with older state than the map holds.
- In production the symptom is not an exception. It is a tenant billed for 1,000 calls whose access log shows 1,030, and only on busy days.
Corrected implementation
package com.example.gateway.quota;import java.util.concurrent.ConcurrentHashMap;import java.util.concurrent.atomic.AtomicInteger;import org.springframework.stereotype.Service;@Servicepublic class QuotaService { private final ConcurrentHashMap<String, AtomicInteger> usedByTenant = new ConcurrentHashMap<>(); private final int dailyLimit = 1_000; public boolean tryConsume(String tenantId) { AtomicInteger used = usedByTenant.computeIfAbsent(tenantId, id -> new AtomicInteger()); while (true) { int current = used.get(); if (current >= dailyLimit) { return false; } if (used.compareAndSet(current, current + 1)) { // fails if another thread got in between return true; } } } public int used(String tenantId) { AtomicInteger used = usedByTenant.get(tenantId); return used == null ? 0 : used.get(); }}Why it is written this way
ConcurrentHashMap.computeIfAbsentcreates each tenant's counter exactly once, atomically.- The check and the increment are joined by
compareAndSet(current, current + 1): it writes only if the value is stillcurrent. A thread that lost the race re-reads and checks the limit again, so the limit holds. - Not
incrementAndGet()followed by a comparison: that would count refused requests too, and the counter would run past the limit. dailyLimitis final and the map reference is final, so neither needs any guard.
Trade-offs
- The quota is correct per JVM, not per service. With three replicas each instance grants 1,000. A shared limit needs shared state, for example an atomic increment in Redis or a conditional update in the database, with a network call on every request.
- Under heavy contention on one tenant,
compareAndSetfails and the loop retries. Each retry is cheap and nothing blocks, but the work per request is no longer constant. - The map grows by one entry per tenant and never shrinks. A daily quota also needs a reset, and resetting while requests are in flight is another compound action that has to be designed, for example by replacing the whole map reference held in a volatile field.
- The code is harder to read than the broken version. A
synchronizedmethod would be shorter and equally correct, at the price of making every tenant's requests queue behind one lock.
Hands-on exercise
Lose an update by hand, then try to lose one with each fix
Use the simulation to produce a lost update with an interleaving you choose yourself, see why volatile does not prevent it, and check that synchronized and atomic modes make the same interleaving impossible.
Steps · 0 of 6 done
Two threads, one counter
Simulation · not a real JVMStep each thread through read, add and write in the order you choose. The counter is only as correct as the interleaving allows. This is a simulation, not a running JVM.
What you should see
Run one after the other, the threads reach 6. As soon as both threads read the same value before either writes, one increment is lost and the total falls short; with every increment interleaved that way, far short. In volatile mode the outcome is the same: each thread reads the current value, and two threads can still read it together. In synchronized mode the second thread cannot start its read until the first has written, so the losing interleaving cannot be built and the result is always 6. Atomic mode also always ends at 6. The real program in the lab shows the same ranking at scale: plain lost 54.1%, volatile 40.6%, synchronized, AtomicInteger and LongAdder lost nothing.
What can go wrong
Reaching for volatile to fix a counter
- Symptom
- After adding
volatilethe counts are still low, perhaps a little less so. - Cause
volatilegives visibility.count++needs atomicity: it is a read, an add and a write, and volatile does not join them.- Fix
AtomicIntegerorAtomicLongwhen the value is used in decisions,LongAdderfor statistics, or a lock when other state must change with it.
Trusting a thread-safe collection to make your logic thread-safe
- Symptom
- Duplicate loads, duplicate inserts or exceeded limits in code that only ever touches a
ConcurrentHashMap. - Cause
- Each method is atomic.
containsKeythenput, orgetthenput, is two methods with a gap between them. - Fix
- Use the single atomic method that expresses the whole action:
computeIfAbsent,putIfAbsent,mergeorcompute. Keep the function you pass short: other updates to the map may be blocked while it runs.
Declaring the defect fixed because the test is green
- Symptom
- A stress test passes locally and in CI, and the discrepancy returns in production.
- Cause
- The window is nanoseconds wide, the test machine has few cores, and a short test may not run compiled code. A pass says very little.
- Fix
- Fix by construction: identify the guard for each shared field and check that every access uses it. Use the stress run to demonstrate the defect before the fix, not to certify the code after it.
Adding a sleep or a log line to make the flag visible
- Symptom
- The hang disappears after a
Thread.sleep(1)or alog.debugis added to the loop, and returns months later. - Cause
- The change happened to alter what the JIT compiler did.
Thread.sleepandThread.yieldhave no synchronisation semantics, so nothing was actually guaranteed. - Fix
- Create a real happens-before edge: make the flag
volatile, or use interruption as in the previous lesson.
Synchronising the write and not the read
- Symptom
- Updates are no longer lost, yet a reporting thread sometimes shows stale or inconsistent values.
- Cause
- A lock orders only the threads that take it. A reader that does not lock has no happens-before edge with the writers.
- Fix
- Guard reads with the same lock as writes, or publish the value through a volatile or atomic field.
How to troubleshoot
A counter or total is lower than an independent count
Check
javap -c -p -cp target/classes com.example.gateway.quota.QuotaService
- How to read it
- Shows the bytecode of the class that owns the counter.
getfield,iadd,putfieldon a shared field with nomonitorenteraround it, in a method that is notsynchronized, is an unguarded read-modify-write.
You suspect a race and need to demonstrate it
Check
java -XX:ActiveProcessorCount=2 -Xmx64m labs/jvm/extras/QuotaRace.java
- How to read it
- The pattern, not this exact file: many threads released by one latch, hundreds of rounds, an invariant checked after each. Any round that breaks the invariant proves the defect. Give the JVM at least 2 CPUs; with one, threads interleave only at context switches.
A worker ignores a flag that another thread has set
Check
jcmd <pid> Thread.print | grep -A 6 '"<thread name>"' (twice, a few seconds apart)
- How to read it
- RUNNABLE on the same loop line both times with
cpu=growing by about the interval means the thread is spinning and has not seen the write. Check the declaration of the field in the loop condition: if it is not volatile and the loop takes no lock, that is the cause.
The same key is loaded several times after a restart
Check
grep -c 'loading rates for EUR/USD' application.log
- How to read it
- Use whatever line your loader logs. More than one load per key per expiry means the cache is populated with check-then-act. Replace it with
computeIfAbsentor a cache library that loads once per key.
A stop-flag hang cannot be reproduced locally
Check
java -Xint -cp app.jar com.example.Main versus the same command without -Xint
- How to read it
-Xintruns everything in the interpreter. If the hang appears only without it, compiled code is involved, which fits a missing happens-before edge. This is a diagnostic experiment only: never run production with-Xint.
Key takeaways
- A race condition needs shared state that changes. In a Spring service that means the non-final fields of singleton beans.
- Atomicity and visibility are two problems.
count++and check-then-act fail on atomicity; a flag that is never seen fails on visibility. volatilegives visibility only. On the real JVM a volatile counter still lost 40.6% of its increments.- A thread-safe collection makes each call atomic, not your sequence of calls. Use
computeIfAbsent,mergeorcomputefor compound actions on a map. - A visibility failure is permitted by the memory model, not guaranteed: in these runs the plain stop flag hung with compiled code and stopped when interpreted.
- A green test proves little: the broken quota passed 192 of 200 hostile rounds. Find the guard for every shared field instead.
Practical check
Small scenarios, one decision each. Pick what you would do, then read why.
Scenario 1 of 4
A singleton @Service keeps private volatile long processed; and each of 16 request threads calls processed++ once per order. Over a day the order table gains 2,400,000 rows and the metric reports 2,391,600. There are no errors in the log.
What explains the gap, and what closes it?
Scenario 2 of 4
After a deploy, a pricing service makes 40 identical calls to a slow tariff API within one second and then settles to none. The code is Tariff t = cache.get(id); if (t == null) { t = client.fetch(id); cache.put(id, t); } return t; and cache is a ConcurrentHashMap.
Which change removes the burst?
Scenario 3 of 4
A worker thread runs while (!cancelled) { step(); } where cancelled is a plain boolean set by a request thread. In testing it always stopped within a millisecond. In production, after a JDK patch upgrade, one worker in a hundred never stops and holds a CPU until the pod is restarted.
What is the correct reading of this history?
Scenario 4 of 4
A team adds a test for a reserve() method on a stock service: 8 threads, a latch, 1 unit of stock, assert that exactly one reservation succeeds. It passes 50 times in a row on a 2 CPU build agent. The method is if (stock > 0) { stock--; return true; } return false; on a plain field.
What should they conclude?
References
References: Java Language Specification, Java SE 21, chapter 17: Threads and Locks (data races, 17.3 sleep and yield, 17.4.5 happens-before, 17.7 long and double) ↗ · Java SE 21 API: java.util.concurrent, Memory Consistency Properties ↗ · Java SE 21 API: ConcurrentHashMap (computeIfAbsent, putIfAbsent, merge are atomic) ↗ · Java SE 21 API: AtomicInteger (incrementAndGet, compareAndSet) ↗ · Java SE 21 API: LongAdder (when to prefer it, sum is not an atomic snapshot) ↗ · Java SE 21 API: HashMap (not synchronized; must be synchronized externally) ↗ · JDK 21 tool reference: java (-Xint, -XX:ActiveProcessorCount) ↗ · Spring Framework reference: Bean scopes (singleton is the default) ↗ · OpenJDK jcstress: the Java Concurrency Stress harness ↗ · Report a problem