Your database is doing the same work millions of times. The same product page read by thousands of users a minute. The same user profile fetched on every API request. The same configuration value queried on every webhook. Caching is the discipline of doing that work once and reusing the result — trading a small risk of staleness for an enormous reduction in latency and load.

A cache is a faster, smaller storage layer that sits between a requester and the original data source. Everything from the L1 cache on your CPU to a CDN edge in São Paulo is the same idea applied at a different scale. The hierarchy below is why caching works at all: each level is roughly 10–1000× slower than the one above it. A cache hit one level up pays for many cache misses below.

A cache hit is the entire point: data found in cache, returned in microseconds, source untouched. A miss is the slow path: query the source, store the result, return it. The interesting metric is the hit ratio — the percentage of requests served from cache. A healthy production cache sits at 80–99% depending on the workload. Below 50%, you are paying the cost of cache complexity without getting the benefit; either the working set doesn't fit, the eviction policy is wrong, or you are caching the wrong layer.

Where you write to determines what staleness, durability, and latency look like. Three policies cover almost every real system. Pick consciously.

Caches have finite memory. When the cache is full and a new entry needs space, the eviction policy decides who dies. The policy you pick should match your access pattern; a default LRU on a workload that wants LFU is just slow with extra steps.

Caching is the engineer's favourite tool because it is conceptually simple and the wins look enormous in benchmarks. The real-world wins come with strings attached: invalidation logic, security boundaries, and the temptation to use a cache to paper over a problem that should be fixed at the source.

In 2010 this was a real debate. In 2026 it is mostly settled. Redis is the default for nearly every modern caching workload because it does everything Memcached does plus a long list of features you will eventually want. Memcached still wins one specific niche: extreme raw throughput for tiny string values where you don't need any of Redis's extras.

An in-process cache is a hash map living inside your application. Zero network hop, zero serialization, fastest possible. Until your app runs on more than one machine — at which point each instance has its own copy and they disagree. A distributed cache (Redis, Memcached) costs you a 1ms network hop, but every backend sees the same view.

A popular cache key expires. In the next millisecond, ten thousand requests simultaneously miss the cache. All ten thousand hit the database at once. The database, which was sized for the cached load, falls over. Symptoms: traffic looks normal, then a single moment of cache eviction takes the entire system down. This is the classic thundering-herd problem applied to caching, and the solutions are well-known and rarely implemented.

Three solutions, in increasing sophistication: (1) mutex/lock on miss — only one request fetches; the rest wait. (2) probabilistic early expiration — recompute the value before TTL with a small random probability per request, so popular keys refresh asynchronously and never all expire at once. (3) background refresh — a worker proactively refreshes hot keys before they expire. Pick the simplest one that fits your traffic.

Despite a textbook full of caching patterns, two cover almost every production system: cache-aside and read-through. The third pattern most teams need but don't implement is cache warming — the difference between a graceful deployment and a database that catches fire every time you ship.