Performance · 04

JIT, Startup & Native Image

A freshly started OrderService instance is slow: the first HTTP request took 255 ms and it needed thousands of requests before latency settled at a fifth of its early value. Nothing is wrong with the code. The JVM starts by interpreting bytecode and only compiles what proves hot. This lab measures that warm-up from the first request, reads -XX:+PrintCompilation and PrintInlining, shows escape analysis and deoptimisation in action, then measures real Spring Boot startup levers one at a time (C1-only, lazy init, AppCDS, Spring AOT) and compares them honestly with CRaC and GraalVM native image, separating what was measured here from general guidance.

12
Chapters
255 ms
First request
18x
First vs steady batch
7
Startup runs per variant
Hub · 01 Measure First · 02 Benchmarking & Load Testing · 03 JVM Memory & GC · 04 JIT, Startup & Native Image · 05 Profiling Toolbox · 06 Concurrency & Virtual Threads · 07 Spring, HTTP & Network · 08 Database Performance · 09 Caching · 10 Distributed Systems & Messaging · 11 Capacity & Failure · 12 AWS Compute & the JVM · 13 Scaling & Observability · 14 Cost & Capstone · 15 Scenario Q&A
01
Chapter One - The Problem

Cold JVM, Slow First Requests

Scenario: OrderService is deployed to a new ECS/Fargate task or Kubernetes pod (a rolling deploy, or the autoscaler adding capacity). The load balancer sends it traffic as soon as the readiness check passes. Users on that instance see the slowest requests of the whole day. Two things are slow: startup (getting to "ready") and warm-up (reaching peak speed after ready).

The first request is the slowest: default JVM vs interpreter-only (WarmLab, real data)
100 1000 5000 0 25 50 75 100 125 150 batch number (each batch = 200 orders x 20 lines) microseconds / batch default tiered -Xint Real data from WarmLab on this 12-core machine; the very first batch is the first request the JVM ever sees.
1 Problem
▶
A fresh JVM runs the same pricing code 18x slower on its very first batch (2524 μs vs 141 μs steady), and the Spring Boot app answered request #1 in 255 ms against ~0.8 ms at steady state.
2 Hypothesis
▶
Two causes: (1) the JVM executes bytecode in an interpreter until methods are hot, then compiles them in the background; (2) class loading, Spring bean creation, Jackson and Tomcat initialisation all run once, on the first requests.
3 Measurement
▶
WarmLab (plain Java) prints the time of every batch from the very first one, 5 launches per mode; Probe sends 3,000 sequential HTTP requests to a Spring Boot 3.3.4 app right after startup.
4 Diagnosis
▶
Batch time falls from 2524 μs to 141 μs in about 14 batches; the Spring app's median latency per 100 requests falls from 3.83 ms to 0.75 ms over 3,000 requests.
5 Optimization (one variable)
▶
Chapters 3-8 change one thing at a time: the JIT mode, then the startup levers.
6 Verification
▶
Each chapter ends with real before/after numbers.
Real output - Probe: latency of the first requests after startup, then median ms per block of 100 requests (3,000 requests)
first requests (ms): #1=254.7 #2=5.8 #3=4.3 #5=5.6 #10=4.7 #50=3.7 #100=3.9 median ms per block of 100 requests: 3.83 3.14 2.91 2.61 1.94 1.72 1.70 1.56 1.47 1.46 1.33 1.53 1.71 1.79 1.56 1.41 1.56 1.47 1.10 0.89 0.94 0.81 0.80 0.76 0.78 0.78 0.88 0.86 0.84 0.75
Real Spring Boot app: median request latency per block of 100 sequential requests after startup (ms)
1 2 3 4 0 500 1000 1500 2000 2500 3000 requests since startup (first request alone took 254.7 ms, off the chart) ms median ms / 100 requests One JVM, one client thread, localhost. Latency keeps dropping for ~2,000 requests as C2 compiles the Tomcat/Spring/Jackson path.
Key insight: Think of the JIT as a kitchen that starts by cooking every dish by hand from the recipe book (interpreter). After a dish has been ordered a few hundred times it writes a quick card (C1); after thousands of orders it invests in a custom station (C2), betting that tomorrow's orders look like today's. A new kitchen, or a bet that fails, means a slow stretch.
⚠ Common mistakes
  • Load-testing a freshly started instance and calling the numbers "steady state". Discard a warm-up window (see Benchmarking) or you measure the JIT, not your code.
  • Reading "Started in 2.2 seconds" as time-to-ready. That log line excludes JVM boot and the first-request cost: in our best runs wall time to the first response was ~3.0 s when the Boot log said 2.2 s.
02
Chapter Two - Mental Model

Tiers, Counters, Speculation

HotSpot compiles lazily and in tiers. Counters per method (invocations and loop back-edges) decide when a method moves up. C2 results are speculative: they assume what the profile saw (this call site only ever sees one class; this branch is never taken) and install a guard; if the guard fails the code is thrown away (deoptimisation) and execution falls back.

Interpreter -> C1 -> C2, and back (deoptimisation)
Tiered compilation in HotSpot (JDK 21): method states and the counters that move them Tier 0 interpreter counts calls and loops Tier 3 C1 + full profiling (types, branches) ~30% slower than 1 Tier 4 C2: speculative, inlines, scalar- replaces objects Tier 1 / 2 C1 without or with limited profiling (trivial / C2 busy) invocations > 200 or loops > 60,000 (back-edge) invocations > 5,000 or invocations+loops > 15,000 DEOPT: a speculation failed (new type, uncommon branch) -> "made not entrant", back to the interpreter / tier 3 Thresholds are the real JDK 21.0.2 defaults printed by -XX:+PrintFlagsFinal on this machine: Tier3InvocationThreshold=200, Tier3BackEdgeThreshold=60000, Tier4InvocationThreshold=5000, Tier4CompileThreshold=15000, Tier4BackEdgeThreshold=40000, CICompilerCount=4
Tier 0 (interpreter)
▶
Executes bytecode one instruction at a time. Slowest (>20x slower than compiled code in our -Xint run: 3087 μs per batch vs 141 μs), but starts instantly and gathers the first counters.
Tier 3 (C1 with profiling)
▶
Compiled quickly (milliseconds); instruments the code to record receiver types and branch frequencies. About 30% slower than non-profiled C1 code, so methods spend as little time as possible here.
Tier 4 (C2)
▶
Slow, optimising compiler running on background threads (CICompilerCount = 4 here). Uses the profile for inlining, escape analysis, loop unrolling and vectorisation. Peak speed, biggest cost in CPU and code cache.
On-stack replacement (OSR)
▶
A loop that is hot inside a method that is called once (main) is compiled mid-execution; PrintCompilation marks these with %.
Deoptimisation
▶
"made not entrant" in PrintCompilation: compiled code is invalidated (superseded by a higher tier, or a speculation failed). A burst of those in production often follows a deploy or a change in traffic mix.
Compiler threads compete with your app
▶
On a 1-2 vCPU container the compiler threads take CPU from request threads exactly when warm-up needs it; this is why small tasks warm up slowly and throttle (see the AWS page).
Flag (JDK 21)Default hereMeaning
-XX:+TieredCompilationtrueUse C1 and C2 together
-XX:TieredStopAtLevel=14Stop at C1: fast warm-up, lower peak, smaller memory
-XX:Tier3InvocationThreshold / Tier4InvocationThreshold200 / 5000Invocations before C1 / C2 compile
-XX:Tier3BackEdgeThreshold / Tier4BackEdgeThreshold60000 / 40000Loop iterations before OSR compile
-XX:CICompilerCount4 (ergonomic)Number of compiler threads
-XX:MaxInlineSize / FreqInlineSize35 / 325 bytecodesLargest cold / hot method to inline
-XX:MaxInlineLevel15Maximum inlining depth
-XintoffInterpreter only (diagnosis, never production)
03
Chapter Three - Lab: Warm-up Curve

Measuring the Warm-up Curve

Same program (WarmLab: price an order of 20 lines through a polymorphic discount and an immutable Money record), same machine. One variable changes: the compilation mode. Each mode was launched 5 times; the chart shows the median time of every batch.

1 Problem
▶
How slow is the cold start really, and which JIT mode is best for it?
2 Hypothesis
▶
Default tiered starts with C1 and reaches C2 within tens of milliseconds; C1-only reaches steady state sooner but stays slower; C2-only has to interpret much longer; interpreter-only never improves.
3 Measurement
▶
java [flag] WarmLab 150 x 5 launches per mode; median per batch; steady value = median of batches 110-150.
4 Diagnosis
▶
See the chart and table. Default: 14 batches (about 7 ms of cumulative work) to get within 2x of steady. Interpreter-only stays at ~3087 μs per batch.
5 Optimization (one variable)
▶
Change only the JIT flag.
6 Verification
▶
Table below.
Lab result: time per batch of 200 pricing calls (microseconds, log scale, median of 5 JVM launches)
100 1000 5000 0 25 50 75 100 125 150 batch number (each batch = 200 orders x 20 lines) microseconds / batch default tiered -XX:TieredStopAtLevel=1 -XX:-TieredCompilation -Xint Real data from WarmLab on this 12-core machine; the very first batch is the first request the JVM ever sees.
ModeBatch 1 (μs)Batch 2Steady (μs, median of batches 110-150)Batches to reach 2x steadyTotal ms for 150 batches
default tiered (C1 then C2)25245331411423.1
-XX:TieredStopAtLevel=1 (C1 only)360035316612655.6
-XX:-TieredCompilation (C2 only)46573051167944.9
-Xint (interpreter only)3859335229021453.4
Lab result: total time to price the first 30,000 orders (150 batches), ms, median of 5 launches
default tiered 23 ms -XX:TieredStopAtLevel=1 56 ms -XX:-TieredCompilation 45 ms -Xint 453 ms Log-scaled bars. Interpreter-only never gets fast; C1-only starts fast but plateaus higher; C2-only starts slowest.
Real output - WarmLab, default mode, first 10 batches of one launch (microseconds per batch of 200 orders)
1 2524 2 372 3 388 4 411 5 448 6 329 7 811 8 324 9 312 10 313

Reading the result. Steady-state speed of C1-only is 166 μs vs 141 μs for tiered (the C2 advantage on this code is modest, 18%), while C1 reaches its plateau earlier. For a short-lived or scale-to-zero process that never runs long enough to amortise C2 this trade can be right; for a long-running service C2 pays back. This is exactly the question the startup chapter answers with a real application.

⚠ Common mistakes
  • Using -Xint or -Xcomp for "benchmarks". Neither resembles production; -Xint was 21x slower here.
  • One run. The five launches differ (batch 2 ranged widely because compilation timing races with the program); medians and several launches are the minimum for warm-up claims.
04
Chapter Four - Lab: PrintCompilation

Reading -XX:+PrintCompilation

-XX:+PrintCompilation writes one line each time a method is compiled or invalidated. It is the cheapest way to see which methods the JVM considers hot, which tier they reached, and which were thrown away.

1 Problem
▶
Which methods get compiled, in what order, and what does "made not entrant" mean?
2 Hypothesis
▶
Small accessors become tier 1; the hot method climbs 0 -> 3 -> 4; each tier-3 version is replaced by tier 4 and invalidated.
3 Measurement
▶
java -XX:+PrintCompilation WarmLab 150, filter for WarmLab methods.
4 Diagnosis
▶
See the excerpt and the column legend.
5 Optimization (one variable)
▶
None (diagnostic).
6 Verification
▶
Used in later chapters to prove inlining and deoptimisation.
Real output - java -XX:+PrintCompilation WarmLab 150 | grep WarmLab (first lines; times in ms since JVM start)
51 10 1 WarmLab$Line::qty (5 bytes) 51 11 1 WarmLab$Line::sku (5 bytes) 51 14 3 WarmLab$Money::<init> (10 bytes) 51 12 1 WarmLab$Line::price (5 bytes) 52 17 3 WarmLab$Money::plus (17 bytes) 52 18 3 WarmLab$Loyalty::apply (6 bytes) 52 19 3 WarmLab$Bulk::apply (16 bytes) 52 20 3 WarmLab::priceOrder (115 bytes) 53 22 4 WarmLab$Loyalty::apply (6 bytes) 53 21 1 WarmLab$Money::cents (5 bytes) 53 18 3 WarmLab$Loyalty::apply (6 bytes) made not entrant 53 23 4 WarmLab$Money::<init> (10 bytes) 54 14 3 WarmLab$Money::<init> (10 bytes) made not entrant 115 128 4 WarmLab$Money::plus (17 bytes) 5 117 17 3 WarmLab$Money::plus (17 bytes) made not entrant 117 140 4 WarmLab::priceOrder240 (115 bytes)
ColumnExampleMeaning
timestamp52Milliseconds since JVM start when the compile was queued/finished
compile id20Sequence number; the same id with "made not entrant" refers to that earlier compile
flags% (OSR), s, !, b, n% on-stack replacement; n native wrapper; ! has exception handlers
tier1, 3, 41 = C1 trivial, 3 = C1 with profiling, 4 = C2
method (size)WarmLab::priceOrder (115 bytes)Bytecode size; methods under 35 bytes inline by default, hot ones up to 325
made not entranttier 3 priceOrder at 125 msNo new calls enter this code: replaced (normal) or speculation failed (deopt)
Compilation timeline of one hot method
Life of WarmLab::priceOrder in the real PrintCompilation log (times in ms since start) 0 ms 25 ms 50 ms 75 ms 100 ms 125 ms interpreter -> tier 3 -> tier 4 (C2): batch time falls from 2524 to 309 microseconds in this window 52 ms: queued tier 3 (C1 + profile) 117 ms: tier 4 (C2) compile ready 125 ms: tier 3 version "made not entrant"
How to use it in production without drowning in output
  • -XX:+PrintCompilation on a staging pod after a deploy: how long until your hot methods reach tier 4? Look for compile storms (hundreds of lines per second) and repeated "made not entrant" for the same method (deopt loops).
  • For a continuous view use JFR events jdk.Compilation and jdk.Deoptimization (see Profiling Toolbox) instead of text logs.
  • jcmd PID Compiler.codecache shows code cache usage (240 MB reserved here).
05
Chapter Five - Lab: Inlining and Escape Analysis

Inlining and Escape Analysis

The two optimisations that make Java object-heavy code (records, small DTOs, Optional, iterators) fast are inlining (replace a call by the callee's body, exposing more optimisation) and escape analysis (an object that never leaves the method need not be allocated: its fields live in registers).

1 Problem
▶
Is new Money(...) in a hot loop an allocation? Is discount.apply() a virtual call?
2 Hypothesis
▶
C2 inlines small, hot, monomorphic/bimorphic calls and removes allocations of non-escaping objects.
3 Measurement
▶
-XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining for the inlining decisions; EscapeLab with ThreadMXBean.getThreadAllocatedBytes and with -XX:-DoEscapeAnalysis.
4 Diagnosis
▶
PrintInlining shows inline (hot) for both discount implementations (with a 50/50 type profile) and for Money::plus. EscapeLab: with escape analysis the loop allocates 24 bytes per iteration, without it 48.
5 Optimization (one variable)
▶
One flag: -XX:-DoEscapeAnalysis.
6 Verification
▶
Loop time 5.2 ns vs 10.2 ns per iteration (median of 3).
Real output - PrintInlining for the tier-4 compile of WarmLab::priceOrder (trimmed)
@ 5 WarmLab$Money::<init> (10 bytes) inline (hot) @ 1 java.lang.Record::<init> (5 bytes) inline (hot) @ 1 java.lang.Object::<init> (1 bytes) inline (hot) @ 33 WarmLab$Line::price (5 bytes) accessor @ 43 WarmLab$Line::sku (5 bytes) accessor @ 59 WarmLab$Line::qty (5 bytes) accessor @ 62 WarmLab$Bulk::apply (16 bytes) inline (hot) @ 62 WarmLab$Loyalty::apply (6 bytes) inline (hot) \-> TypeProfile (11178/22356 counts) = WarmLab$Loyalty \-> TypeProfile (11178/22356 counts) = WarmLab$Bulk @ 78 WarmLab$Line::qty (5 bytes) accessor @ 87 java.lang.Math::round (70 bytes) (intrinsic) @ 92 WarmLab$Line::sku (5 bytes) accessor @ 100 WarmLab$Money::<init> (10 bytes) inline (hot) @ 1 java.lang.Record::<init> (5 bytes) inline (hot) @ 1 java.lang.Object::<init> (1 bytes) inline (hot)
Inlining decisions
What C2 did to priceOrder (from PrintInlining): one flat compiled body priceOrder (115 bytes) loop over 20 lines calls on every line Line::price / sku / qty accessor Money::<init> inline (hot) Money::plus inline (hot) Discount::apply (interface call) TypeProfile 11178/22356 = Loyalty 11178/22356 = Bulk -> BOTH inlined + guard Math.round -> (intrinsic) stub Money allocations -> scalar-replaced where non-escaping
Real output - EscapeLab, escape analysis ON (3 runs)
n=50000000 time=187 ms 3.75 ns/iter allocated=1144 MB (24.0 bytes/iter) sink=245000000 n=50000000 time=260 ms 5.21 ns/iter allocated=1144 MB (24.0 bytes/iter) sink=245000000 n=50000000 time=307 ms 6.15 ns/iter allocated=1144 MB (24.0 bytes/iter) sink=245000000
Real output - EscapeLab, -XX:-DoEscapeAnalysis (3 runs)
n=50000000 time=509 ms 10.18 ns/iter allocated=2288 MB (48.0 bytes/iter) sink=245000000 n=50000000 time=512 ms 10.25 ns/iter allocated=2288 MB (48.0 bytes/iter) sink=245000000 n=50000000 time=323 ms 6.47 ns/iter allocated=2288 MB (48.0 bytes/iter) sink=245000000
Lab result: ns per loop iteration (median of 3 runs) and bytes allocated per iteration
escape analysis ON (default) 5.2 ns 24 bytes/iter escape analysis OFF 10.2 ns 48 bytes/iter EscapeLab: total = total.plus(new Money(i & 7)); with C2 escape analysis the temporary Money is scalar-replaced; the accumulating one is not.
Java - EscapeLab: the loop under test
record Money(long cents) { Money plus(Money o) { return new Money(cents + o.cents); } } static long run(int n) { Money total = new Money(0); for (int i = 0; i < n; i++) total = total.plus(new Money(i & 7)); // new Money(i & 7) never escapes the loop body return total.cents(); }

One of the two allocations per iteration disappeared (48 -> 24 bytes). The other, the running total, is carried across iterations (a loop-carried value that C2 here did not eliminate), which is a reminder that escape analysis is an optimisation that sometimes applies, not a guarantee. Do not design code around it; measure allocation in a profile (Profiling Toolbox).

PrintInlining messageMeaningWhat to do
inline (hot)Inlined because the call site is hot (up to FreqInlineSize = 325 bytecodes)Nothing - desired
accessorTrivial getter inlinedNothing
(intrinsic)Replaced by a hand-tuned stub (Math.round here)Nothing
callee is too largeAbove the size limit; stays a real callSplit the method if it is hot and the call is a bottleneck
no static bindingInterface/virtual call whose target is only known from the profileFine if the type profile is mono/bimorphic (chapter 6)
TypeProfile (n/m counts) = ClassC2 inlines the observed receivers with a type guardKeep call sites narrow on hot paths
⚠ Production gotchas
  • Relying on escape analysis to justify allocation-heavy code. It is defeated by large methods (inlining budget), objects stored in fields/collections, or megamorphic calls. If allocation matters, check bytes/op as above.
  • Gigantic hot methods. Methods above ~8,000 bytecodes are not compiled at all by default (-XX:-DontCompileHugeMethods is the escape hatch); methods above 325 bytecodes are not inlined even if hot.
06
Chapter Six - Lab: Speculation and Deoptimisation

When the Bet Fails: Deoptimisation

C2 optimises for what it has seen. A call site that has only ever seen one implementation is compiled as a direct, inlined call guarded by a cheap type check. If a second implementation appears the code becomes bimorphic; with three or more it becomes a genuine interface call. Each change is a deoptimisation followed by recompilation, and the steady-state cost of the call site goes up.

1 Problem
▶
After a deploy that enables a new payment method, the pricing hot path gets 30-70% slower even though no code on that path changed.
2 Hypothesis
▶
A call site that was monomorphic became bimorphic/megamorphic (a new class started flowing through it), invalidating the inlined code.
3 Measurement
▶
DeoptLab: one call site fed with 1, then 2, then 4 implementations; -XX:+PrintCompilation to see "made not entrant".
4 Diagnosis
▶
ns/call rises with each added receiver type; PrintCompilation shows DeoptLab::loop being made not entrant and recompiled when the shape changes.
5 Optimization (one variable)
▶
Changing only the number of receiver types at the call site.
6 Verification
▶
See table.
Real output - DeoptLab (3 launches)
1 receiver type (monomorphic) 1.19 ns/call 2 receiver types (bimorphic) 2.09 ns/call 4 receiver types (megamorphic) 3.32 ns/call 1 receiver type (monomorphic) 1.19 ns/call 2 receiver types (bimorphic) 2.14 ns/call 4 receiver types (megamorphic) 3.58 ns/call 1 receiver type (monomorphic) 1.13 ns/call 2 receiver types (bimorphic) 2.02 ns/call 4 receiver types (megamorphic) 4.24 ns/call
Lab result: cost of ONE interface call site (ns/call, median of 3 runs) as the receiver types grow
1 receiver type 1.19 ns 2 receiver types 2.09 ns 4 receiver types 3.58 ns DeoptLab. Same call site, same loop: the JIT first inlines the single implementation, then must guard, then falls back to a vtable/itable call.
Real output - java -XX:+PrintCompilation DeoptLab (the loop is invalidated and recompiled when D2 appears, then again for D3/D4)
23 8 % 3 DeoptLab::loop @ 5 (39 bytes) 23 9 3 DeoptLab::loop (39 bytes) 24 10 % 4 DeoptLab::loop @ 5 (39 bytes) 29 8 % 3 DeoptLab::loop @ 5 (39 bytes) made not entrant 29 11 4 DeoptLab::loop (39 bytes) 29 10 % 4 DeoptLab::loop @ 5 (39 bytes) made not entrant 34 9 3 DeoptLab::loop (39 bytes) made not entrant 34 12 % 4 DeoptLab::loop @ 5 (39 bytes) 178 12 % 4 DeoptLab::loop @ 5 (39 bytes) made not entrant 179 116 % 4 DeoptLab::loop @ 5 (39 bytes) 183 11 4 DeoptLab::loop (39 bytes) made not entrant 183 118 3 DeoptLab::loop (39 bytes) 187 119 4 DeoptLab::loop (39 bytes) 191 118 3 DeoptLab::loop (39 bytes) made not entrant 409 116 % 4 DeoptLab::loop @ 5 (39 bytes) made not entrant 409 129 % 3 DeoptLab::loop @ 5 (39 bytes) 411 119 4 DeoptLab::loop (39 bytes) made not entrant 413 132 % 4 DeoptLab::loop @ 5 (39 bytes) 415 129 % 3 DeoptLab::loop @ 5 (39 bytes) made not entrant 415 133 4 DeoptLab::loop (39 bytes)
Guard, uncommon trap, recompile
Speculative inlining and its guard compiled loop ds[i].apply(x) inlined D1 body class == D1 ? yes fast path 1.2 ns no: D2 appears uncommon trap deoptimise: "made not entrant", re-profile, recompile cost of the failed bet: interpreter / tier 3 for a few ms, then a new compile with 2 types
Receiver types at one call siteJIT resultMeasured costTypical Spring example
1 (monomorphic)Inlined behind a class check1.19 nsA single implementation of a service interface
2 (bimorphic)Both inlined, guarded2.09 nsTwo payment strategies
3+ (megamorphic)Real itable/vtable call, no inlining through it3.58 nsHandler/filter chains, visitor over many node types
⚠ Production gotchas
  • Panicking about a 3 ns difference. It matters in a tight loop executed millions of times (JSON serialisers, expression evaluators), not in a request that waits 150 ms on I/O. Profile first.
  • Warm-up scripts that only exercise one path. A warm-up that trains the JIT on one payment type gets deoptimised the first time real traffic uses another; warm up with a representative mix.
  • Deopt storms. Rarely, a method flips between compiled and deoptimised repeatedly (look for the same method id many times in PrintCompilation). Report to the library owner or avoid the pattern.
07
Chapter Seven - Lab: Spring Boot Startup

Startup Levers: C1, Lazy Init, AppCDS

Startup is a separate problem from warm-up: it decides how long a new instance takes to become ready (deploys, autoscaling, restarts after OOM) and how much you pay on scale-to-zero platforms. The app measured here is a real Spring Boot 3.3.4 service (Tomcat, Jackson, JDBC starter with an embedded in-memory database on the classpath, 12 controllers and 12 services; no database work is done). It is small - real services with JPA, Security and Actuator take longer, so absolute numbers will be higher; the relative effect of each lever is what to take away.

1 Problem
▶
A new instance needs ~3.0 s from JVM launch to its first successful response (the Boot log said "Started ... in 2.2 seconds"), and the scaling policy waits for it.
2 Hypothesis
▶
Class loading/verification, bean creation and C2 compile CPU dominate startup; reduce them with a class-data archive (CDS), less eager initialisation, or less aggressive compilation.
3 Measurement
▶
startup.sh launches the jar, Probe sends the first request as soon as the port opens; 7 interleaved rounds so slow host moments hit every variant; report best and median.
4 Diagnosis
▶
See table (best of 7 interleaved runs is the primary number because other processes on this shared host randomly slowed individual runs: medians are shown in brackets and are visibly noisier). Combined, the best wall time to first response fell by 42%.
5 Optimization (one variable)
▶
One lever at a time, then the combination.
6 Verification
▶
Table and chart below.
Lab result: wall time from launching the JVM to the first successful HTTP response (s), best of 7 interleaved runs
baseline 3.0 s median 3.4 C1 only (TieredStopAtLevel=1) 2.2 s median 6.2 lazy initialisation 2.8 s median 4.7 AppCDS archive 2.6 s median 4.1 AppCDS + C1 + lazy 1.7 s median 2.2 Spring Boot 3.3.4 web + JDBC starter app (12 controllers, 12 services), Java 21.0.2, shared host: bars are the best run; the annotation is the (noisier) median.
Variant"Started in" s: best of 7 (median)JVM launch -> first response s: best of 7 (median)RSS MB (median)Wall-time saved vs baseline
Baseline (default JVM flags)2.2 (2.6)3.0 (3.4)243-
-XX:TieredStopAtLevel=11.6 (5.0)2.2 (6.2)16329%
--spring.main.lazy-initialization=true2.2 (3.7)2.8 (4.7)2437%
AppCDS archive (-XX:SharedArchiveFile)2.0 (3.0)2.6 (4.1)31415%
AppCDS + C1 only + lazy init1.2 (1.6)1.7 (2.2)16942%
Shell - the commands behind each row
# baseline java -jar orderservice-1.0.jar # C1 only: skip C2 (smaller peak, but no C2 compile threads/CPU during startup) java -XX:TieredStopAtLevel=1 -jar orderservice-1.0.jar # lazy initialisation: beans are created on first use (moves cost to the first request!) java -jar orderservice-1.0.jar --spring.main.lazy-initialization=true # AppCDS (JDK 13+ dynamic archive): training run writes the archive at exit, later runs map it java -XX:ArchiveClassesAtExit=app.jsa -jar orderservice-1.0.jar # run once, exercise it, stop it java -XX:SharedArchiveFile=app.jsa -jar orderservice-1.0.jar # JDK 19+ shortcut: create the archive automatically on first run and reuse it afterwards java -XX:+AutoCreateSharedArchive -XX:SharedArchiveFile=app.jsa -jar orderservice-1.0.jar
Real output - startup-results (two interleaved rounds of the harness; "Started in" comes from the Spring Boot log)
baseline run1 started_in=2.609s first_response_ms=3428 rss_mb=244 c1only run1 started_in=5.046s first_response_ms=6229 rss_mb=160 lazy run1 started_in=5.849s first_response_ms=7918 rss_mb=233 appcds run1 started_in=3.57s first_response_ms=4715 rss_mb=314 appcds-c1-lazy run1 started_in=2.38s first_response_ms=3164 rss_mb=168 baseline run2 started_in=2.545s first_response_ms=3246 rss_mb=242 c1only run2 started_in=1.567s first_response_ms=2159 rss_mb=164 lazy run2 started_in=2.155s first_response_ms=2804 rss_mb=224 appcds run2 started_in=1.964s first_response_ms=2567 rss_mb=297 appcds-c1-lazy run2 started_in=1.444s first_response_ms=1957 rss_mb=169
LeverHow it helpsCost / catch (measured or known)
-XX:TieredStopAtLevel=1Compiler threads stop at C1: less startup CPU, RSS 163 MB vs 243 MBLower peak throughput (chapter 3: ~18% slower steady state on WarmLab); for long-running CPU-heavy services keep C2
Lazy initialisationFewer beans created before "ready"The cost moves to the first request that needs each bean; failures (bad config) show up late; in our runs the gain was small (7% best-of-7, inside the host noise)
AppCDSApplication classes are read from a pre-parsed memory-mapped archive: no parse/verifyArchive tied to exact JDK + classpath; mapped archive inflates RSS (314 MB vs 243 MB here, file-backed pages); needs a training run in the build
CombinedEffects mostly add upMore moving parts in the image build
⚠ Production gotchas
  • Trusting a single startup number. Baseline "Started in" ranged from 2.2 s to 4.0 s across 7 interleaved runs here, and some variants lost whole runs to host noise. Always use several runs and the best/median pair.
  • Optimising "Started in" instead of time-to-ready. With lazy init the log time drops, but readiness probes that call real endpoints may fail or the first user pays.
  • Stale CDS archive. If the jar changes, the archive is rejected (the JVM falls back silently unless -Xshare:on). Rebuild it in CI with the artifact.
  • C1-only in production without a throughput test. We measured an ~18% slower steady state on a tiny kernel; a JSON-heavy service can differ more.
08
Chapter Eight - AOT, CRaC, Native Image

Spring AOT, CRaC and GraalVM Native Image: What Was Measured and What Was Not

Beyond flags there are three ways to change what happens at startup: do work at build time (Spring AOT), snapshot a running process (CRaC / Lambda SnapStart), or remove the JVM from the equation (GraalVM native image). Only Spring AOT was measured here; the others are described from documented behaviour and clearly marked.

1 Problem
▶
Spring Boot still spends startup time scanning, reflecting and creating proxies at runtime.
2 Hypothesis
▶
Spring AOT (build-time processing) generates bean definitions as plain Java and reflection hints, removing classpath scanning and part of the reflection at startup; the JVM still JITs.
3 Measurement
▶
Add spring-boot-maven-plugin goal process-aot; run the jar with -Dspring.aot.enabled=true; the log says "Starting AOT-processed OrderServiceApplication"; 7 interleaved rounds against the normal jar.
4 Diagnosis
▶
Median "Started in" 2.24 s vs 2.58 s; JVM launch to first response 3.0 s vs 3.5 s.
5 Optimization (one variable)
▶
One change: AOT-processed application classes.
6 Verification
▶
About 13% faster startup on the JVM; native image is the larger consumer of this work.
Real output - AOT harness (interleaved: base2 = normal jar, aot = AOT-processed jar with -Dspring.aot.enabled=true)
base2 run1 started_in=2.197s first_response_ms=2819 rss_mb=242 aot run1 started_in=1.702s first_response_ms=2472 rss_mb=227 base2 run2 started_in=2.299s first_response_ms=3062 rss_mb=238 aot run2 started_in=2.022s first_response_ms=2733 rss_mb=240 base2 run3 started_in=3.641s first_response_ms=4688 rss_mb=253 aot run3 started_in=2.909s first_response_ms=4193 rss_mb=226 base2 run4 started_in=3.805s first_response_ms=4974 rss_mb=223 aot run4 started_in=2.531s first_response_ms=3459 rss_mb=230
pom.xml / shell - Spring AOT on the JVM (Spring Boot 3.x)
<plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> <executions><execution><id>process-aot</id><goals><goal>process-aot</goal></goals></execution></executions> </plugin> $ mvn package $ java -Dspring.aot.enabled=true -jar target/orderservice-1.0.jar ... Starting AOT-processed OrderServiceApplication v1.0 using Java 21.0.2 ...
Honest comparison matrix
Startup vs peak throughput vs memory vs build effort: GENERAL GUIDANCE (not all measured here) Startup Peak throughput Memory (RSS) Build / ops effort Measured here? JIT default slow best (C2) highest low yes AppCDS + flags faster same similar low yes Spring AOT (JVM) faster same similar medium see chapter 8 CRaC / SnapStart fast restore depends on snapshot similar high (special JDK) no GraalVM native image milliseconds lower (no C2) lowest high (slow builds) no Only the first two rows (and part of the third) were measured in this lab; the rest is qualitative and varies by app and tooling version.
JIT (default)AppCDS + flagsSpring AOT (JVM)CRaC / Lambda SnapStartGraalVM native image
Startupbaseline (3.0 s best to first response here)42% less wall time (best of 7), combined with C1/lazy13% less "Started in" hereRestore of a warm snapshot: typically tens to hundreds of ms (documented behaviour, not measured here)Typically tens of ms (documented, not measured here)
Peak throughputBest: C2 with live profileSameSameSame as the JVM that was checkpointed (state captured after warm-up can already be hot)Generally lower than C2 for long-running load unless profile-guided optimisation is used (general guidance; verify)
MemoryHighest (JIT + metaspace)RSS up when archive is mapped (314 vs 243 MB here)SimilarSimilar to the JVMLowest RSS typically
Build / opsNoneTraining run + archive in CIBuild plugin stepSpecial JDK (CRaC) or Lambda SnapStart; uniqueness/entropy/connection hooks requiredSlow builds (minutes), reflection/proxy hints, fewer libraries supported, different debugging and profiling
Debug / observeFull (JFR, jcmd, agents)FullFullFullLimited: no JIT, different JFR/agent support
Measured here?yesyesyesno (no CRaC JDK / Lambda in this environment)no (no native-image toolchain here)
When is each worth it? (decision guidance, not measurements)
  • Long-running service on ECS/EKS (deploys a few times a day, steady load): JIT default + AppCDS or AOT if startup gates your autoscaling. Peak throughput matters more than a few seconds of startup.
  • Lambda / scale-to-zero / short batch jobs: startup dominates cost and latency; evaluate SnapStart (managed CRaC), native image, or C1-only. Test with your real handler; this lab did not run Lambda.
  • CPU-throttled small tasks: JIT compilation competes with request threads; a snapshot or AOT shortens the period of poor latency.
  • Whatever you choose, decide with a measurement of your startup, first-request latency and throughput at p99, not a vendor chart.
⚠ Production gotchas
  • Assuming native image is an automatic win. It trades peak throughput, dynamic features (reflection, proxies, classpath scanning) and observability tooling for startup and memory; closed-world assumptions fail at runtime if hints are missing.
  • Checkpointing (CRaC / SnapStart) with open connections or secrets in memory. Restored clones share random seeds, DB connections and tokens unless hooks reset them.
09
Chapter Nine - Deployments

Warm-up and Startup in Deployments

Warm-up and startup are deployment problems as much as JVM problems. A new instance that is "healthy" but cold gets a full share of traffic and a worse p99. This chapter is guidance; the shapes are illustrative, not measurements, because this environment has no AWS access.

Deployment timeline: why a fresh instance hurts p99
ILLUSTRATIVE shape (not measured): a new instance joins the load balancer cold latency of the new task start + readiness JIT warm-up (C1 -> C2) steady state Mitigations: readiness gates, ALB slow start (target-group attribute), warm-up requests, CDS/AOT/CRaC to shorten the first segment. Autoscaling that reacts to CPU adds capacity that is itself slow for the first minutes (see Scaling & Observability).
Readiness must mean "warm enough"
▶
A TCP or shallow HTTP check passes as soon as Tomcat listens. Add a readiness check that has exercised the main paths (see Actuator health groups), or a startup warm-up routine before reporting READY.
Ramp traffic
▶
Application Load Balancer target groups support a slow-start duration (a configurable ramp-up period for new targets); use it instead of sending a full share instantly.
Warm-up requests
▶
Replay a representative mix (not one payload: chapter 6) against the instance before it joins; 2,000-3,000 requests were enough to reach steady latency in our Spring app.
Scale ahead of load
▶
Cold instances add capacity late and badly; keep headroom and scale on leading signals (queue depth, request rate), see Scaling & Observability.
Shorten the cold part
▶
AppCDS/AOT/CRaC reduce startup; they do not replace JIT warm-up unless the snapshot was taken after warm-up.
⚠ Production gotchas
  • Deploying at peak with a rolling update that replaces 50% of tasks at once. Half the fleet is cold exactly when load is highest; lower the batch size or deploy off-peak.
  • Health check timeouts shorter than startup. The orchestrator kills the task before it ever becomes ready (a restart loop); use a startup probe / grace period sized from a measurement like chapter 7.
10
Chapter Ten - Try It Yourself

The Programs

All code was compiled and run with JDK 21.0.2. The Spring Boot app is an ordinary starter-web + starter-jdbc project; the harness is a few lines of bash plus Probe.java.

Shell - the commands
javac WarmLab.java EscapeLab.java DeoptLab.java for m in "" "-Xint" "-XX:TieredStopAtLevel=1" "-XX:-TieredCompilation"; do java $m WarmLab 150 > warm.txt; done java -XX:+PrintCompilation WarmLab 150 | grep WarmLab java -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining -XX:+PrintCompilation WarmLab 150 java EscapeLab ; java -XX:-DoEscapeAnalysis EscapeLab java DeoptLab ; java -XX:+PrintCompilation DeoptLab | grep "DeoptLab::loop" java -XX:+PrintFlagsFinal -version | grep -E 'Tier[34]|CICompilerCount|MaxInlineSize|FreqInlineSize'
Java - WarmLab.java (complete, as run)
import java.util.*; /** * WarmLab - the pricing hot path of OrderService, measured from the very first request. * Prints "batch microseconds" where one batch = 200 orders x 20 line items; time printed in microseconds. Watch the first batches. */ public class WarmLab { interface Discount { double apply(double price, int qty); } record Bulk() implements Discount { public double apply(double p, int q) { return q >= 10 ? p * 0.9 : p; } } record Loyalty() implements Discount { public double apply(double p, int q) { return p * 0.97; } } record Money(long cents) { Money plus(Money o) { return new Money(cents + o.cents); } } record Line(int sku, int qty, double price) {} static final Discount[] DISCOUNTS = { new Bulk(), new Loyalty() }; static Money priceOrder(Line[] lines, int customer) { Money total = new Money(0); for (Line l : lines) { double p = l.price(); Discount d = DISCOUNTS[(l.sku() + customer) & 1]; p = d.apply(p, l.qty()); total = total.plus(new Money(Math.round(p * l.qty() * 100) + (l.sku() % 7))); } return total; } static long sink; public static void main(String[] args) { int batches = args.length > 0 ? Integer.parseInt(args[0]) : 60; Line[] lines = new Line[20]; for (int i = 0; i < lines.length; i++) lines[i] = new Line(i * 13, 1 + i % 12, 9.99 + i); for (int b = 1; b <= batches; b++) { long t0 = System.nanoTime(); for (int o = 0; o < 200; o++) sink += priceOrder(lines, o).cents(); long us = (System.nanoTime() - t0) / 1000; System.out.printf("%3d %7d%n", b, us); } System.out.println("sink=" + sink); } }
Java - EscapeLab.java (complete, as run)
import java.lang.management.ManagementFactory; /** Escape analysis: does "new Money(...)" in a hot loop really hit the heap? */ public class EscapeLab { record Money(long cents) { Money plus(Money o) { return new Money(cents + o.cents); } } static long run(int n) { Money total = new Money(0); for (int i = 0; i < n; i++) total = total.plus(new Money(i & 7)); // both objects never escape the loop return total.cents(); } public static void main(String[] a) { com.sun.management.ThreadMXBean mx = (com.sun.management.ThreadMXBean) ManagementFactory.getThreadMXBean(); long tid = Thread.currentThread().getId(); long sink = 0; for (int i = 0; i < 200; i++) sink += run(100_000); // warm up int n = 50_000_000; long b0 = mx.getThreadAllocatedBytes(tid); long t0 = System.nanoTime(); sink += run(n); long ns = System.nanoTime() - t0; long bytes = mx.getThreadAllocatedBytes(tid) - b0; System.out.printf("n=%d time=%d ms %.2f ns/iter allocated=%d MB (%.1f bytes/iter) sink=%d%n", n, ns / 1_000_000, (double) ns / n, bytes >> 20, (double) bytes / n, sink); } }
Java - DeoptLab.java (complete, as run)
/** Speculation and deoptimisation: a call site that was monomorphic becomes megamorphic. */ public class DeoptLab { interface Discount { double apply(double p); } static class D1 implements Discount { public double apply(double p) { return p * 0.90; } } static class D2 implements Discount { public double apply(double p) { return p * 0.95; } } static class D3 implements Discount { public double apply(double p) { return p * 0.97; } } static class D4 implements Discount { public double apply(double p) { return p * 0.99; } } static double loop(Discount[] ds, int n) { double s = 0; for (int i = 0; i < n; i++) s += ds[i & (ds.length - 1)].apply(i); // THE call site return s; } static double sink; static void phase(String name, Discount[] ds) { for (int i = 0; i < 20; i++) sink += loop(ds, 200_000); // let the JIT settle on this shape long t0 = System.nanoTime(); int n = 100_000_000; sink += loop(ds, n); System.out.printf("%-34s %.2f ns/call%n", name, (double) (System.nanoTime() - t0) / n); } public static void main(String[] a) { phase("1 receiver type (monomorphic)", new Discount[] { new D1() }); phase("2 receiver types (bimorphic)", new Discount[] { new D1(), new D2() }); phase("4 receiver types (megamorphic)", new Discount[] { new D1(), new D2(), new D3(), new D4() }); System.out.println("sink=" + sink); } }
Java - Probe.java (first-request and warm-up probe, as run)
import java.net.URI; import java.net.http.*; import java.util.*; /** Probe PORT N : waits for the port, then sends N sequential GETs and prints the latency (ms) of request 1, 2, 5, 10, 50, 100, ... plus per-100 medians. */ public class Probe { public static void main(String[] a) throws Exception { int port = Integer.parseInt(a[0]); int n = Integer.parseInt(a[1]); HttpClient c = HttpClient.newHttpClient(); HttpRequest r = HttpRequest.newBuilder(URI.create("http://localhost:" + port + "/orders-api/orders/7")).build(); while (true) { try (var s = new java.net.Socket("localhost", port)) { break; } catch (Exception e) { Thread.sleep(20); } } double[] ms = new double[n]; for (int i = 0; i < n; i++) { long t0 = System.nanoTime(); c.send(r, HttpResponse.BodyHandlers.ofString()); ms[i] = (System.nanoTime() - t0) / 1e6; if (i == 0) System.out.println("FIRST_DONE_EPOCH_MS=" + System.currentTimeMillis()); } StringBuilder sb = new StringBuilder("first requests (ms):"); for (int i : new int[] {0, 1, 2, 4, 9, 49, 99}) if (i < n) sb.append(String.format(" #%d=%.1f", i + 1, ms[i])); System.out.println(sb); sb = new StringBuilder("median ms per block of 100 requests:"); for (int b = 0; b + 100 <= n; b += 100) { double[] blk = Arrays.copyOfRange(ms, b, b + 100); Arrays.sort(blk); sb.append(String.format(" %.2f", blk[50])); } System.out.println(sb); } }
Things to try next
  • Run WarmLab with -XX:CICompilerCount=2 (fewer compiler threads) and watch the curve stretch: this is what a 2-vCPU container looks like.
  • Add -XX:Tier4InvocationThreshold=500 to see C2 arrive earlier (and compile more code that never gets hot).
  • Run the Spring app with -XX:+PrintCompilation and count compile events per second during the first 10 s.
11
Chapter Eleven - Interview Angles

JIT and Startup Questions with Evidence

  • Q: Why is the first request slow in a Java service?
    Class loading and initialisation (Spring beans, Jackson, Tomcat) run once on first use and the code runs in the interpreter until the JIT compiles it. Our app took 255 ms for request #1 vs ~0.8 ms at steady state.
  • Q: Explain tiered compilation.
    Interpreter (tier 0) profiles; C1 (tier 3) compiles quickly with profiling; C2 (tier 4) recompiles hot code with aggressive, profile-based optimisation. Thresholds on this JDK: 200 invocations for C1, 5,000 (or 15,000 invocations+loops) for C2.
  • Q: What does "made not entrant" mean?
    A compiled version is no longer used for new calls: it was replaced by a higher tier (normal) or a speculation failed and the method deoptimised. Many repeats for the same method indicate a deopt loop.
  • Q: What is escape analysis and does it guarantee no allocation?
    C2 can scalar-replace objects that do not escape. It is best-effort: in EscapeLab one of two allocations per iteration disappeared (48 -> 24 bytes) but the loop-carried one stayed.
  • Q: Why can a deploy change performance of unchanged code?
    Type profiles change: a call site that was mono/bimorphic becomes megamorphic (we measured 1.19 -> 3.58 ns per call) and the inlined code is deoptimised.
  • Q: How would you reduce Spring Boot startup time?
    Measure first; then AppCDS (class archive), Spring AOT, trimming auto-configuration/classpath, lazy init (with care), and on small CPUs consider C1-only. In our app the combination cut the best wall time to first response by 42%.
  • Q: Is GraalVM native image faster?
    It starts much faster and uses less memory, but usually has lower peak throughput than C2 on long-running workloads, slow builds and reflection constraints. Choose it for scale-to-zero or CLI-like workloads after measuring.
  • Q: What is CRaC / SnapStart?
    Checkpoint/restore of a running JVM: a warm process is snapshotted and restored quickly. Needs support in the JDK/platform and care with connections, secrets and randomness after restore.
  • Q: Does lazy initialisation make the app faster?
    It makes the log line shorter by deferring bean creation to first use; time-to-ready may improve but first requests pay and configuration errors surface late.
12
Chapter Twelve - Summary

Cheat Sheet and Recap

Cheat sheet
SymptomCauseEvidenceAction
Slow first requests after deployCold JVM + first-use initialisationLatency per 100 requests falling for thousands of requestsWarm-up requests, slow start, readiness; AppCDS/AOT
Autoscaled instances hurt p99 for minutesJIT warm-up and compile CPU on small tasksPrintCompilation / jdk.Compilation after start; CPU throttlingPre-scale; larger CPU; shorter startup
Perf regression with no code changeDeoptimisation / megamorphic call site"made not entrant" bursts; type profileKeep hot call sites narrow; profile
Startup > probe timeoutHealth probe too tightWall time to first responseStartup probe; AppCDS; lazy init with care
Scale-to-zero cold start too longJVM startup + initFirst-response timeSnapStart / native image / C1-only (measure!)
∑ JIT and startup - what to remember
  • Warm-up is real: first batch 18x slower than steady in our lab; the Spring app needed ~2,000 requests to settle (3.83 ms -> 0.75 ms).
  • Tiers: interpreter -> C1 (profiling) -> C2 (speculative). Thresholds 200 / 5,000 invocations on JDK 21.
  • PrintCompilation shows tiers and "made not entrant"; PrintInlining shows what C2 inlined and why not.
  • Escape analysis and inlining are bets based on the profile; changing traffic can deoptimise them (1.19 -> 3.58 ns/call).
  • Startup levers measured: C1-only, lazy init, AppCDS, Spring AOT (about 13% faster "Started in") and their combination (42% less wall time to first response).
  • CRaC/SnapStart and native image were not measured here; they trade peak throughput, tooling and complexity for startup.
  • Always measure time to ready and p99 after ready, with several interleaved runs.
Related pages: