The recurring scenario of this series is OrderService: a Spring Boot 3 / Java 21 service whose GET /orders/{id} calls a customer lookup, the InventoryService and the PaymentGateway over HTTP. In this chapter we isolate exactly that slice. Each request performs three sequential blocking calls of 50 ms and almost no CPU work, and we offer it a steady 1,000 requests per second. The service runs on a classic thread-per-request model: one thread owns a request from the first byte to the last.

A request-handling thread has two kinds of time: C, the time it burns on a CPU, and W, the time it spends blocked (socket read, DB wait, lock wait). Here C is about a millisecond and W is 150 ms. Two relationships govern everything on this page.

Same code, same load, same machine. One variable changes: the number of worker threads. Each row is a separate 10-second run of PoolLab at 1,000 offered req/s (10,000 requests).

Verification of Little's Law: at 200 threads the sampler saw on average 148.8 requests inside the service; lambda x W = 1,000 x 0.153 s = 153. The law holds to within sampling error, which is why you can use it to predict pool size from two numbers you already have (RPS and latency).

Even with the right size, a pool needs a policy for overload. Java's ThreadPoolExecutor admits work in a fixed order, and most production incidents come from not knowing it: core threads first, then the queue, then extra threads up to max, then rejection.

Not every wait is I/O. When threads queue for the same lock the pool looks busy (threads in BLOCKED state), CPU stays low, and adding threads makes it worse. Two shapes to recognise: a long critical section (a legacy synchronized block that does I/O or a 2 ms recomputation) and a hot counter (many threads hammering one cache line).

Result of that run: 8 threads completed 1,621 refreshes in 4 seconds (about 405/s) - barely below the 500/s single-thread ceiling. Seven of the eight threads were BLOCKED on the monitor.

The second shape is hot-spot contention on tiny critical sections. ContentionLab increments one shared counter from 1 to 12 threads, using four primitives. The critical section is nanoseconds long, so the cost is the lock hand-off and cache-line ping-pong, not waiting for work.

Reading it: with one thread the lock-based counters are fast (no contention, thin locks); as threads are added they do not scale and often get slower, because every increment needs exclusive ownership of the same cache line and the lock transfers between cores. LongAdder keeps per-thread cells and only sums on read, so it keeps scaling. The price: sum() is not an atomic snapshot - fine for metrics, wrong for a sequence number.

CompletableFuture.supplyAsync(supplier) without an executor runs on ForkJoinPool.commonPool() - a pool sized for CPU-bound work (availableProcessors() - 1). It is also the pool used by parallel streams. Putting blocking calls on it is the classic way to turn a 12-core server into an 11-call-at-a-time server.

A virtual thread (JEP 444, final in JDK 21) is a java.lang.Thread scheduled by the JVM, not the OS. When it blocks in a supported operation (socket I/O, Thread.sleep, LockSupport.park, java.util.concurrent locks, queues), the JVM saves its stack to the heap and frees the carrier thread to run another virtual thread. The code stays plain blocking code; you simply stop being limited to a few hundred OS threads.

Virtual threads multiply the number of waiting tasks you can hold. They do nothing about the number of tasks the CPUs can execute. A CPU-bound request (hashing an order payload, compressing a response, rendering a PDF) occupies a carrier for its whole duration.

On JDK 21 a virtual thread that blocks while holding a monitor (inside a synchronized block or method) cannot unmount: it stays mounted and blocks its carrier. If every carrier is pinned, no other virtual thread can run - the service looks hung even though thousands of virtual threads exist. This is the most common reason a "we turned on virtual threads" rollout disappoints.

Virtual threads remove the thread limit. They do not remove the limit of whatever the threads are waiting for: the Hikari pool (default maximumPoolSize = 10), the PaymentGateway's rate limit, a licence server. With 200 platform threads the thread pool was an accidental throttle. With one virtual thread per request there is no throttle left, so overload goes straight to the bounded resource, and every waiter holds memory while it waits.

PREVIEW in JDK 21 Structured concurrency (JEP 453, StructuredTaskScope) treats a group of concurrent subtasks as one unit of work: subtasks are forked inside a scope, the scope is joined, and when the block exits no subtask can still be running. It needs --enable-preview on JDK 21 and its API may change in later releases, so treat the code below as a technique to know, not as something to ship without that in mind.

Everything above ran from plain Java files, no Spring needed. Compile and run them on any JDK 21 machine; your absolute numbers will differ (core count, laptop vs server, noisy neighbours) but the shapes - the cliff at N = lambda x W, the BLOCKED pile-up, the pinned carriers - will reproduce.

• Q: How many threads should a Tomcat/Spring Boot service use?
  Start from Little's Law: threads ≥ throughput x time-in-service (1,000/s x 0.15 s = 150). Add headroom (~30%). CPU-bound services need about cores + 1; I/O-bound need many more. Then check the downstream pools (Hikari, HTTP client) support the same concurrency - the smallest pool in the chain sets throughput.
• Q: CPU is at 10% but latency is high and throughput is flat. What do you check?
  Pool saturation and queueing: busy-thread and queue-depth metrics, then thread dumps. Most threads in Socket.read = waiting on a downstream; most BLOCKED on one monitor address = lock contention; all workers busy with a growing queue = pool too small for lambda x W.
• Q: Why is Executors.newFixedThreadPool dangerous?
  Its queue is an unbounded LinkedBlockingQueue. Overload never produces rejection; it produces unbounded latency and eventually OutOfMemoryError. Build ThreadPoolExecutor with a bounded queue and an explicit rejection policy (we saw 16 of 30 tasks rejected under AbortPolicy).
• Q: What does the order of ThreadPoolExecutor admission mean in practice?
  Core threads first, then the queue, then threads up to max, then rejection. So with a large queue "max" is never reached - a common surprise when people raise maxPoolSize to absorb load and nothing changes.
• Q: What problem do virtual threads solve, and what do they not solve?
  They make blocking cheap: one thread per request without an OS-thread limit (10,000 started in 316 ms vs 9116 ms here, throughput 985/s with no tuning). They do not speed up CPU-bound work and do not remove limits of downstream resources like a DB pool.
• Q: What is pinning and how do you find it?
  On JDK 21 a virtual thread that blocks inside synchronized (or a native frame) keeps its carrier. With parallelism 4 we measured 400 x 50 ms taking 5073 ms instead of 87 ms. Find it with -Djdk.tracePinnedThreads=full, JFR jdk.VirtualThreadPinned, or jcmd Thread.dump_to_file; fix by using ReentrantLock or JDK 24+ (JEP 491).
• Q: Why shouldn't you pool virtual threads?
  The pool bounds concurrency to its size, recreating the problem. Create one per task and bound resource use with a Semaphore / bulkhead in front of the scarce resource.
• Q: Is CompletableFuture.supplyAsync(blockingCall) fine?
  No: it runs on the common ForkJoinPool (cores - 1 threads, 11 here), so 1,200 blocking 50 ms calls took 5537 ms instead of ~50-100 ms with a proper executor. Always pass an executor for blocking work and add timeouts.
• Q: When is a lock-free counter better than synchronized?
  When many threads update the same hot variable. LongAdder scaled with threads in our run while synchronized and even AtomicLong did not. For metrics use it; for multi-field invariants use a short critical section.
• Q: Is structured concurrency production-ready?
  On JDK 21 it is a preview API (JEP 453) requiring --enable-preview; the API can still change. The ideas (cancel siblings on failure, no leaked tasks) can be approximated with executors and timeouts today.