A cache is a faster, smaller, possibly stale copy of data that is expensive to produce or fetch. It earns its place in two ways: it lowers latency (the answer is nearby) and it lowers load on the source of truth (the database sees only misses). It costs you three things: staleness, memory, and a new failure mode (the cache itself, plus the code that keeps it correct). Phil Karlton's joke about cache invalidation being one of the two hard problems in computer science is the most quoted line in the field because it is true.

Five names cover almost every design. They differ in who loads on a miss and when writes reach the database. The Spring annotation side (@Cacheable, @CachePut, @CacheEvict) is in Spring: Caching and Resilience; this page is about the behaviour behind the annotations.

Invalidation is the act of making sure a cache stops serving a value that is no longer true. There are four tools. Real systems use a TTL as a safety net plus one precise method on top.

Event-driven invalidation is how you keep many caches (several services, local L1 caches on every instance) correct without the writer knowing about them. The reliable form writes the change and an outbox row in the same database transaction, a relay publishes the event, and each cache owner consumes it and invalidates. Because delivery is at least once, invalidating twice must be harmless (deleting a key twice is). See Messaging fundamentals for the delivery guarantees and Kafka Architecture for a log that can carry the events.

Versioned keys are the cheapest way to invalidate a whole class at once. If the value format changes in a deployment, or a bulk import rewrites a table, bump shop:v3 to shop:v4: every old key becomes unreachable instead of being found and deleted one by one, and the TTL or the eviction policy reclaims the memory. The same idea gives you safe rolling deployments: instances running the old and the new code use different prefixes and never read each other's incompatible values.

A cache and a database are two stores updated by separate calls, so there is no moment where both change atomically. The worst symptom is stale data that stays stale until the TTL. The diagram shows the most common sequence in cache-aside with delete-on-write.

Why it happens: step 2 reads the old value, the whole write (steps 3 and 4) completes, and only then does step 5 write the old value into the cache. It needs a slow reader and a fast writer, which is rare per request but certain at enough traffic. What you can do, in order of cost:

A second race: two writers, update-on-write. Writer 1 sets the DB to 1, writer 2 sets it to 2, writer 2 writes the cache (2), then writer 1 writes the cache (1): cache says 1, DB says 2. This is why delete-on-write is the default advice. If you do update the cache, include a version and make the cache write conditional.

A cache stampede (thundering herd, dog-piling) happens when a popular entry expires or is evicted and many requests miss at the same instant. Each one runs the expensive loader, so the database receives a burst of identical queries at exactly the moment the cache stopped protecting it. The burst makes the database slower, which makes the loaders slower, which keeps more requests in flight: a feedback loop that can take the database down.

Fifty threads asked for the same cold key and the database was called once. The second line shows negative caching in action (chapter 6): the missing id was loaded once and the empty answer was cached for the other two lookups. The statistics line is Caffeine's own recordStats() data.

A cache stores what exists. A request for an id that is not in the database gets a miss every single time, so it always reaches the database. A scanner, a bug or an attacker iterating random ids can therefore bypass the cache completely. The defences: negative caching and a Bloom filter, plus ordinary input validation and rate limiting.

An avalanche is the same problem at scale: a large share of the cache disappears at once and the database gets the full load. Three common causes: (1) lots of keys written together with the same TTL, (2) a cache restart or failover with no persistence, (3) a deployment that changes the key prefix for everyone. All three are about the cold cache, and the database must be able to survive it, or the traffic must be limited until the cache is warm again.

A hot key is one that receives a disproportionate share of traffic: the home page, a trending product, a feature flag read on every request. In a sharded cache all requests for one key land on the node that owns it, so adding nodes does not help. In Redis Cluster, for example, a key lives in exactly one slot (see Redis, chapter 5).

HTTP has caching built in, and it is the cheapest cache you can use because the request never reaches your servers. Two mechanisms work together: freshness (the response says how long it may be reused without asking) and validation (once stale, the cache asks the origin "has it changed?" with a conditional request). The directives below were checked against MDN's Cache-Control reference (2026-10-31).

The hit ratio is hits / (hits + misses). Two formulas make it concrete. Database load is requests x (1 - hit ratio). Average latency is h x t_hit + (1 - h) x (t_hit + t_load), because a miss still pays the cache lookup. The example below uses made-up round numbers: a 1 ms cache lookup and a 20 ms database load.

Two consequences. First, the database must survive the miss traffic, and a cold cache means h near 0: at 10,000 req/s you need either capacity for 10,000 req/s of loads or a plan to shed load (chapter 10). Second, average latency hides the tail: with h = 0.99 the 99th percentile request is a miss, so p99 is the database latency, not the cache latency. Watch percentiles, not the mean.

Caches work because access is skewed: a small part of the data takes most of the traffic. The cache needs to hold that working set, not the whole dataset. Measure it: log key accesses for a day, sort by frequency, and plot the cumulative share of requests against the number of keys. If, say, 5 % of keys served 80 % of requests (a made-up skew for illustration), a cache holding about 5 % of the data reaches roughly an 80 % hit ratio, and each extra percent of capacity buys less.

The output of INFO stats on a lightly used test server looked like keyspace_hits:34, keyspace_misses:7, evicted_keys:0, expired_keys:5 (real values from the session used for the Redis page). The ratio of hits to misses, not the raw counts, is what you alert on.

A cache is an optimisation, so the system must still work, within limits, without it. Most outages involving caches share one pattern: the cache stops absorbing 90-99 % of reads, the database receives all of them, and the database falls over.

Libraries hide the mechanics, so here is the whole thing in about a hundred lines of plain Java 17+: a bounded (LRU) cache, expiring entries (TTL with optional jitter), and single-flight loading so that concurrent misses share one loader call. It is for understanding; in production use Caffeine, which adds much better eviction (W-TinyLFU), asynchronous refresh and statistics.

How it works. lookup takes the lock, returns a fresh entry or removes an expired one (lazy expiry), and a LinkedHashMap in access order makes the eldest entry the least recently used one (LRU: store evicts it when the size exceeds the capacity). getNaive is the classic cache-aside code, so concurrent misses all call the loader. get is single-flight: the first miss puts a CompletableFuture into the inFlight map and becomes the leader; later misses for the same key find the future and wait on it. The leader re-checks the cache once it has won, because a previous leader may have just finished (without that re-check a thread can slip in between a leader's store and its inFlight.remove and load again). A loader failure completes the future exceptionally, is not cached, and the next call retries.

• Q: Name the main caching patterns.
  Cache-aside (app loads and fills), read-through (cache loads), write-through (synchronous write to cache and DB), write-behind (asynchronous DB write), refresh-ahead (reload before expiry). They differ in who loads on a miss and when the database is written.
• Q: Why delete the cache entry on write instead of updating it?
  Delete lets the next reader load the truth and avoids two writers leaving the cache in a different order than the database. Update saves a miss but needs versioning or a conditional write.
• Q: Describe the read/write race in cache-aside.
  A reader misses and reads the old row, a writer updates the DB and deletes the cache entry, then the reader writes its old value into the cache. It stays stale until TTL. Mitigate with TTL, versioned values with conditional set, delayed double delete, or event-driven invalidation from the database log.
• Q: What is a cache stampede and how do you prevent it?
  Many concurrent misses on an expired hot key all hit the database. Use single-flight, a distributed lock or lease, jittered TTLs, probabilistic early refresh (XFetch), stale-while-revalidate, and protect the source with limits.
• Q: How does probabilistic early expiration work?
  Readers recompute a bit before expiry with rising probability: recompute if now - delta * beta * ln(rand()) >= expiry, where delta is the last recompute time. One reader usually refreshes early, so there is no herd at the deadline.
• Q: Cache penetration versus avalanche versus stampede?
  Penetration: requests for nonexistent keys always miss (negative caching, Bloom filter). Avalanche: many keys expire or the cache dies together (jitter, persistence, replicas, shedding). Stampede: concurrent misses on one hot key (single-flight).
• Q: How does a Bloom filter help and what are its limits?
  It answers "definitely not present" or "possibly present" in small memory, so nonexistent keys are rejected before the cache or database. False positives are tunable (about 9.6 bits per element for 1 %); standard filters cannot delete and must be updated when data is added.
• Q: What is a hot key and how do you handle it?
  One key getting disproportionate traffic, which overloads the shard that owns it. Add a local L1 cache, replicate the value under suffixed keys, or use client-side caching with server-assisted invalidation.
• Q: Design a two-level cache. What about invalidation?
  L1 in-process (small, seconds-minutes TTL), L2 shared (Redis), database behind. On write: update DB, delete L2, publish an event so every instance drops its L1 entry. A short L1 TTL bounds staleness if an event is lost.
• Q: Explain no-cache versus no-store.
  no-cache allows storing but requires revalidation before each reuse; no-store forbids storing. Neither means "not cached" by itself in the colloquial sense.
• Q: What is an ETag?
  A validator for a response version. The client sends If-None-Match; if it matches, the server returns 304 Not Modified with no body.
• Q: How do you cache static assets safely?
  Fingerprint the file name and serve Cache-Control: public, max-age=31536000, immutable; a new build produces a new URL.
• Q: How do you size a cache?
  Measure the working set (key frequency distribution), multiply entries by value size plus overhead, add headroom, set maxmemory or maximumSize, and watch hit ratio and evictions. Do not size by table row count.
• Q: Why is the hit ratio alone not enough?
  It hides tail latency (p99 is a miss) and cost per miss. Track per-endpoint hit ratio, load time percentiles, evictions and memory.
• Q: What if the cache goes down?
  Short timeouts, circuit breaker on the cache client, fail open to the database under a concurrency limit, serve stale or degrade, then warm up before restoring full traffic. Ask: could the database survive an empty cache?
• Q: Write-through or write-behind?
  Write-through is consistent and slower; write-behind is fast but acknowledged writes can be lost if the cache dies before the flush. Use write-behind only for loss-tolerant data, with a durable queue.
• Q: What goes wrong if the cache key omits the tenant or user?
  Responses are shared across users: a data leak. The key must include every input that changes the answer.
• Q: Scenario: the home page API is slow every day at 09:00 exactly.
  Likely synchronised TTLs on keys warmed at the same time, or a daily deploy emptying L1. Check expiry metrics and load times at that hour; add jitter, stale-while-revalidate and pre-warming.