Java Performance Engineering: From JVM to AWS
A practical, measurement-driven guide to diagnosing, optimising and scaling Java and Spring Boot services. Every page is a lab: observe a symptom, form a hypothesis, measure, diagnose, change one thing, and prove the fix with numbers. This page is the control room — start from the symptom you are seeing in production.
The Loop: Measure, Find, Fix, Prove
Most performance advice fails for one reason: it hands you a setting without telling you how to know whether the setting helps. Raising a heap, switching a garbage collector or doubling a pool are all hypotheses. Engineering is the discipline of testing them. Every page in this guide repeats one loop.
Understand
How Java, Spring, databases and AWS behave under load — the mechanisms, not just the knobs.
Diagnose
Find bottlenecks with profilers, metrics, traces and load tests instead of guessing.
Optimise & verify
Make a measurable improvement and validate it under a realistic workload.
Where the Time Actually Goes
A slow API is rarely slow in one place. A single request crosses the client, a load balancer, a container, the JVM, Spring, a connection pool, the network and a database — and a problem at any layer looks like “the service is slow” from outside. The first skill is attributing the latency to a layer before touching anything.
Symptom → Likely Cause → Tool → Fix
This is the table to bookmark. Find what you are seeing in production, check the likely causes in order, run the listed tool to confirm one, and follow the link to the lab page that teaches the fix with measurements. Several causes can overlap — the tool column is how you tell them apart.
| Symptom | Likely causes | First tool | Go to |
|---|---|---|---|
| High CPU | Hot loop or regex, excessive serialisation, GC threads, lock spinning, JIT compiling after a deploy | JFR / async-profiler CPU profile, top -H | 05 Profiling, 04 JIT |
| High p99, fine p50 | GC pauses, lock contention, slow downstream tail, retries, CPU throttling, connection-pool waits | Latency histogram + distributed trace of slow requests | 01 Measure, 10 Distributed |
| Latency spikes at a fixed interval (e.g. every 30 s) | Periodic GC, scheduled job, cache TTL expiry stampede, connection churn, container CPU throttling | GC log timeline, JFR, correlate with scheduler/TTL | 03 GC, 09 Caching |
| Memory keeps growing | Leak (static map, listener, ThreadLocal), unbounded cache or queue, native/direct memory | Heap histogram, heap dump + dominator tree | 03 Memory, 05 Profiling |
| Frequent or long GC | High allocation rate, undersized heap, wrong collector for the goal, humongous objects | GC logs (-Xlog:gc*), JFR allocation profile | 03 GC |
| Pod OOMKilled, heap looks half full | Container limit is heap plus metaspace, threads, direct buffers, code cache | Container events, NMT (-XX:NativeMemoryTracking) | 12 AWS Compute |
| Threads blocked / requests queue up | Lock contention, exhausted pool, blocking call on an event loop, pinned virtual threads | Thread dumps (jcmd Thread.print) | 06 Concurrency |
| Connection is not available, request timed out | Pool too small or leaked connections, slow queries holding connections, long transactions | Hikari metrics (pending/acquire), thread dump | 08 Database |
| Database latency spikes | Missing index, bad plan, N+1, lock waits, connection storm | Query plan (EXPLAIN ANALYZE), slow-query log, DB insights | 08 Database |
| Throughput stops rising when adding load or instances | Saturated shared resource (DB, lock, pool), serial fraction (Amdahl/USL), client is the limit | Load test with step ramp, USE method per resource | 11 Capacity, 02 Load tests |
| Slow right after deploy or scale-out | Cold JIT, empty caches, class loading, readiness probe too early | Warm-up curve, startup logs, CDS/AOT comparison | 04 JIT & Startup, 13 Scaling |
| 502 / 504 from the load balancer | Keep-alive or idle-timeout mismatch, upstream timeouts, unhealthy targets, saturation | ALB access logs, target response time vs request time | 07 Network |
| One slow dependency drags everything | No timeout, retry storm, no bulkhead, fan-out tail amplification | Trace waterfall, per-dependency latency and error rate | 10 Distributed |
| Queue lag growing | Consumers slower than producers, poison message retries, small batches, too few partitions | Consumer lag / queue-depth metric, consumer profile | 10 Messaging |
| Cold starts hurt (Lambda / scale-to-zero) | JVM startup, class loading, framework init, small memory (and thus CPU) setting | Init duration in logs, startup timeline | 04 Native, 12 Lambda |
| AWS bill keeps rising | Over-provisioned instances, idle headroom, scaling on the wrong signal, DB tier too big, no caching | Cost Explorer, cost per request, utilisation vs requests | 14 Cost |
OrderService: One Slow System, Fifteen Labs
Rather than fifteen disconnected toy examples, every page works on a slice of one recurring system: OrderService, a Spring Boot 3 / Java 21 REST service on AWS. It has been written to be realistically bad — the kind of service that works fine in a demo and falls over in production.
| Built-in flaw | What it causes | Fixed in |
|---|---|---|
| N+1 queries when listing a customer’s orders | Dozens of round trips per request | 08 Database |
No index on orders.customer_id | Sequential scans that grow with the table | 08 Database |
| Connection pool far too small (or far too big) | Pool waits; or a database drowning in connections | 08, 11 |
| No caching of product and customer lookups | Repeated identical queries | 09 Caching |
| Payment and Inventory called one after the other | Latencies add; one slow call stalls everything | 06, 10 |
| New HTTP connection per downstream call, no timeouts | Handshake cost; hangs under failure | 07 Network |
| Default JVM flags in a 2-vCPU, 4 GB container | Odd heap size, GC choice, throttling | 03, 12 |
| Oversized entity JSON, no compression | CPU on serialisation, large payloads | 07 Spring |
The Six-Step Lab Format
Every deep chapter in this guide uses the same six steps, so you can apply the same structure to your own production incidents.
| Metric | Why we always record it |
|---|---|
| Average, p95, p99 latency | The average hides the tail; the tail is what users and SLOs feel |
| Throughput (requests/s) | A latency win that halves throughput is not a win |
| CPU utilisation | Shows whether you freed capacity or just shifted the bottleneck |
| Resource-specific counters | GC pause, pool pending, queries per request — the metric your change targets |
The 15 Labs, in Four Groups
Read in order the first time — the groups build on each other. Afterwards, use the symptom map above as your entry point.
A · Foundations
01 · Measure First
USE/RED, percentiles, coordinated omission, Little’s Law, Amdahl and USL.
02 · Benchmarking & Load Tests
JMH pitfalls, open vs closed workload models, building a trustworthy load test.
B · The JVM
03 · Memory & GC
Heap layout, G1 vs ZGC vs Shenandoah, GC logs, leaks.
04 · JIT, Startup & Native
Tiered compilation, warm-up, CDS, CRaC and GraalVM — honestly compared.
05 · Profiling Toolbox
CPU, allocation, lock and wall-clock profiles; JFR; dumps; runbooks.
06 · Concurrency & Virtual Threads
Pool sizing, contention, pinning, when virtual threads help and when they do not.
C · Spring & the Data Tier
07 · Spring, HTTP & Network
Tomcat, MVC vs WebFlux, serialisation, TCP/TLS/HTTP2, keep-alive, timeouts.
08 · Database Performance
N+1, indexes, query plans, pool sizing, locks, pagination.
09 · Caching
Cache-aside, hit-ratio math, stampede, invalidation, Caffeine vs Redis.
10 · Distributed Systems & Messaging
Tail amplification, retries, backpressure, SQS and Kafka throughput.
D · AWS & Production
11 · Capacity & Failure Scenarios
Capacity maths, the latency hockey stick, game-day scenarios.
12 · AWS Compute & Containers
JVM in containers, OOMKilled, CPU throttling, Graviton, Lambda.
13 · Scaling & Observability
Autoscaling signals, Micrometer, OpenTelemetry, SLOs, trace reading.
14 · Cost & Capstone
Cost per request and the end-to-end OrderService optimisation.
15 · Scenario Q&A
“p99 spikes every 30 s — what do you do?” and 50+ more.
How to Read the Numbers
Measured here
Experiments marked real output were run on a 12-core Linux machine with Java 21 and the results pasted in. They show the shape and the method. Your absolute numbers will differ with hardware, JVM build and load — that is exactly why you should rerun them.
Illustrative
Anything that needs real AWS services (Aurora, Lambda cold starts, CloudWatch, Fargate throttling, prices) is labelled illustrative. Cost figures use example list prices — always check current AWS pricing for your region.
Versions matter
The guide targets Java 21 LTS and Spring Boot 3.x. Where behaviour differs by JDK (virtual-thread pinning, generational ZGC, preview APIs) the page states the version.
No magic flags
You will not find a “fast JVM flags” list. A flag is only presented next to the evidence that motivates it and the measurement that shows its effect.
- Performance work is a loop: measure, find the bottleneck, change one thing, prove it.
- A slow request is attributed to a layer first — the biggest slice caps what any fix can achieve.
- The symptom map is the entry point; the labs teach the fixes with measurements.
- Start at 01 · Measure First.