Spring · 13

Caching & Resilience

Two techniques that keep a Spring service fast and alive: caching removes repeated work, and Resilience4j (retry, circuit breaker, rate limiter, bulkhead, time limiter) contains failures. This page shows what the proxy does on every call, how Caffeine and Redis differ, why caches stampede and go stale, how the breaker state machine decides, and how to order the annotations - then builds a cache and a breaker from scratch in plain Java.

Chapters

Resilience patterns

Providers

Plain-Java builds

Hub · 01 IoC & DI · 02 Bean Scopes & Lifecycle · 03 AOP & Proxies · 04 Boot Auto-Configuration · 05 REST & MVC · 06 Validation & Errors · 07 Spring Data JPA · 08 Transactions · 09 Security Architecture · 10 JWT, OAuth2 & OIDC · 11 Configuration & Profiles · 12 Actuator & Observability · 13 Caching & Resilience · 14 Testing · 15 Interview Q&A

Chapter One - Why It Exists

Repeated Work and Failing Dependencies

Every production Spring service eventually meets two physical facts. First, some work is expensive and repeated: the same product, the same exchange rate, the same permission set is loaded thousands of times a minute, and the answer changes once an hour. Second, every remote dependency you call will, sooner or later, be slow, wrong, or gone. Caching attacks the first fact by not doing the work again. Resilience patterns attack the second by containing the damage when a dependency misbehaves. They look like different topics but share one trait in Spring: both are applied as annotations on a proxy, so they share the same traps (self-invocation, proxy visibility, ordering).

Same traffic, two designs

The second half of the picture matters more than most teams expect. A slow dependency is far more dangerous than a dead one: a dead one fails fast, but a slow one lets requests pile up, each holding a servlet thread (Tomcat defaults to 200) until the whole service stops answering - even for endpoints that never touch the slow dependency. This is a cascading failure, and it is what retry limits, timeouts, bulkheads and circuit breakers exist to stop.

Before: hand-rolled cache and an unprotected call

Java - the code that every team writes first

@Service
public class PriceService {
  private final Map<Long, Price> cache = new HashMap<>();   // unbounded, not thread-safe, never expires
  private final PricingClient client;

  public PriceService(PricingClient client) { this.client = client; }

  public Price priceFor(long productId) {
      Price p = cache.get(productId);
      if (p == null) {
          p = client.fetch(productId);          // no timeout, no retry policy, no fallback
          cache.put(productId, p);              // stale forever, memory grows forever
      }
      return p;
  }
}

⚠ What is wrong with this

HashMap shared across request threads: lost updates and, historically, infinite loops during resize. Use a real cache, never a bare map.
No eviction policy: the map grows until java.lang.OutOfMemoryError: Java heap space. Symptom: heap graph climbs for days, GC time rises, pod restarts.
No TTL: price changes never appear. The bug report reads "we updated the price but customers still see the old one until restart".
The caching logic is tangled into business code, so it cannot be turned off for a test or tuned per environment.

After: declarative cache + guarded call

Java - same behaviour, 2 annotations and a policy in YAML

@Service
public class PriceService {
  private final PricingClient client;

  public PriceService(PricingClient client) { this.client = client; }

  @Cacheable(cacheNames = "prices", key = "#productId")
  @CircuitBreaker(name = "pricing", fallbackMethod = "cachedOrDefault")
  @Retry(name = "pricing")
  public Price priceFor(long productId) {
      return client.fetch(productId);
  }

  private Price cachedOrDefault(long productId, Throwable cause) {
      return Price.unavailable(productId);        // degrade, do not 500
  }
}

The method body is now only the business call. Whether the result is cached, for how long, in which store, and what happens when the call fails are all policy - changeable in configuration without touching the code. The rest of this page shows exactly how each piece works at runtime so that you can predict its behaviour instead of trusting it.

Key insight: A cache is a memory and a circuit breaker is a fuse. The memory saves you from repeating expensive work; the fuse saves the house from burning down when one appliance shorts. You want both, and you want to know which one tripped when the lights go out.

Chapter Two - Mental Model

Interceptors on a Proxy

Both features are implemented the same way: Spring wraps your bean in a proxy (see AOP & Proxies) whose interceptor runs logic before and after your method. For caching that interceptor is CacheInterceptor, a subclass of CacheAspectSupport, registered by @EnableCaching (Boot turns this on automatically when spring-boot-starter-cache is present and a CacheManager can be built). For resilience it is a set of Resilience4j aspects (CircuitBreakerAspect, RetryAspect, ...) registered by the Resilience4j Spring Boot starter on top of Spring AOP.

You annotate a method with @Cacheable

▶

Spring scans bean methods at startup via AnnotationCacheOperationSource and builds a CacheOperation (cache names, key expression, condition) per method

A caller invokes the bean

▶

It really calls the proxy; CacheInterceptor.invoke runs first, evaluates the key (SpEL or KeyGenerator) and asks the CacheManager for the named Cache

The Cache.get(key) returns a value

▶

The proxy returns it immediately; your method body never executes. Side effects inside it also do not happen

The lookup misses

▶

The interceptor invokes your method, then Cache.put(key, result), then returns the result. @CachePut always does the invoke+put; @CacheEvict does evict(key)

Someone calls this.priceFor(1) from the same class

▶

No proxy is involved - no lookup, no store. This single mistake explains most "my cache does nothing" reports

Resilience annotations are stacked on the same method

▶

Each adds its own interceptor; the order decides whether a retry re-checks the breaker, whether a cache hit even reaches the breaker, and what the fallback sees

@Cacheable on a miss: the proxy call sequence

Things this sequence tells you for free:

Only public methods called through the proxy are intercepted; private methods can never be intercepted, and relying on non-public visibility is fragile.
The key is computed before the method runs, so a SpEL key can use parameters (#id, #p0, #root.methodName) but #result is only available in unless (and in @CacheEvict/@CachePut expressions evaluated after invocation).
Caching and resilience compose: a @Cacheable hit returns without ever entering a @CircuitBreaker that sits below it in the chain.

Chapter Three - Spring Cache Abstraction

@Cacheable, @CachePut, @CacheEvict and the Key

The abstraction lives in org.springframework.cache. You program against two small interfaces - CacheManager (a registry: getCache(name)) and Cache (get, put, evict, clear, and get(key, Callable) for atomic load) - and the annotations decide which operations to apply.

The four annotations

Annotation	Runs method?	Cache effect	Typical use
`@Cacheable`	Only on miss	get; on miss invoke then put	Read-through lookups
`@CachePut`	Always	Invoke then put (overwrites)	Update methods that return the new value
`@CacheEvict`	Always	evict(key) or clear() with `allEntries=true`; `beforeInvocation` flag	Delete / update where the new value is not returned
`@Caching`	Per its parts	Groups several of the above on one method	Evict from two caches, put into a third

@CacheConfig(cacheNames = "products") on the class supplies defaults (cache name, key generator, manager) so that methods do not repeat them.

Java - a complete, realistic service (Spring Boot 3, Java 17)

@Service
@CacheConfig(cacheNames = "products")
public class ProductService {
  private static final Logger log = LoggerFactory.getLogger(ProductService.class);
  private final ProductRepository repo;

  public ProductService(ProductRepository repo) { this.repo = repo; }

  // Key = the id. unless: never cache "not found" so a later insert becomes visible immediately.
  @Cacheable(key = "#id", unless = "#result == null")
  public ProductView find(long id) {
      log.info("DB hit for product {}", id);
      return repo.findById(id).map(ProductView::from).orElse(null);
  }

  // condition is checked BEFORE invocation: only cache lookups for "small" pages.
  @Cacheable(key = "'cat:' + #category + ':' + #page", condition = "#page < 3")
  public List<ProductView> byCategory(String category, int page) {
      return repo.findByCategory(category, PageRequest.of(page, 20)).map(ProductView::from).toList();
  }

  // The method returns the fresh object, so refresh the cached copy in the same call.
  @CachePut(key = "#result.id()")
  @Transactional
  public ProductView update(long id, UpdateProduct cmd) {
      Product p = repo.findById(id).orElseThrow(() -> new ProductNotFound(id));
      p.apply(cmd);
      return ProductView.from(p);
  }

  // Delete: evict the entry and every cached category page (they may contain it).
  @Caching(evict = {
      @CacheEvict(key = "#id"),
      @CacheEvict(cacheNames = "productLists", allEntries = true)
  })
  @Transactional
  public void delete(long id) { repo.deleteById(id); }
}

Java - enabling it (Boot does this automatically with spring-boot-starter-cache; explicit form shown)

@Configuration
@EnableCaching
class CacheConfig { }

How the key is built

Situation	Key used	Consequence
No `key` attribute, no params	`SimpleKey.EMPTY`	One entry per cache - fine for "load all config", wrong if you expected per-user data
No `key`, exactly one param	The param itself	Uses its `equals/hashCode`; a mutable key object that changes after caching can never be found again
No `key`, several params	`new SimpleKey(params...)`	All params participate - including a `Pageable` or `Locale` you did not mean to key on
`key = "#id"` (SpEL)	Evaluated expression	Needs parameter names: compile with `-parameters` (Boot parent/Gradle plugin do) or use `#a0`/`#p0`
`keyGenerator = "myGen"`	Your `KeyGenerator` bean	Use for conventions such as prefixing tenant id; `key` and `keyGenerator` are mutually exclusive

Attribute	Evaluated	Can see	Use it for
`condition`	Before invocation (and before the lookup)	Parameters only	Skip the cache entirely for some inputs - neither read nor write
`unless`	After invocation	Parameters and `#result`	Look up normally but refuse to store some results (null, empty, error markers)
`sync = true`	On miss	-	Make concurrent misses for one key run the method once (single cache only)

⚠ Production gotchas

Self-invocation: this.find(id) from another method in the same class bypasses the proxy. Symptom: the log line "DB hit" appears on every call. Fix: move the cached method to another bean or inject the bean into itself via a collaborator.
Cached Optional: Spring unwraps Optional return values and stores the content, so Optional.empty() is stored as null (when null caching is allowed). Combine with unless = "#result == null" - note that SpEL #result refers to the unwrapped value.
Mutable cached objects: with an in-memory cache the same instance is handed to every caller. One caller adds an element to a cached List and every other caller sees it. Return immutable records or defensive copies.
Missing -parameters: key = "#id" evaluates to null and Spring throws java.lang.IllegalArgumentException: Null key returned for cache operation. Fix the compiler flag or use #p0.
@Cacheable on a method that is also @Transactional: the cache write happens after the method returns but before the transaction commits if the cache interceptor is inner; with a later rollback the cache holds data that never existed. See chapter 05.
Caching inside a controller or repository interface default method works only if that bean is proxied; always confirm with a second call and a log line.

Chapter Four - Providers

Caffeine vs Redis, TTL and Serialization

Spring Cache is only the front door; the provider decides speed, memory, TTL, eviction and visibility across instances. Boot auto-configures one when it finds it on the classpath (spring.cache.type overrides auto-detection; none disables caching). Without any provider you get a ConcurrentMapCacheManager: unbounded, no TTL, fine for tests only.

Where the cache lives decides what "consistent" means

Aspect	Caffeine (local)	Redis (shared)
Where data lives	JVM heap of each pod	Separate server / cluster
Read latency	Tens to hundreds of nanoseconds	Network round trip, typically around a millisecond
Eviction	Size-based (W-TinyLFU), time-based, weak/soft refs	TTL, plus Redis `maxmemory-policy`
Per-cache TTL in Spring	Needs per-cache config (custom manager); the Boot spec is global	`RedisCacheConfiguration.entryTtl` per cache name
Consistency across pods	None - every pod has its own copy	Single copy, shared
Values	Any object, no serialization	Must serialize (JDK by default, JSON recommended)
Survives restart	No	Yes (subject to persistence config)
Failure mode	OOM if unbounded; cold start	Redis down: every call errors unless you handle it

YAML - Caffeine with Boot auto-configuration

spring:
cache:
  type: caffeine
  cache-names: products,prices          # pre-create; otherwise created on first use
  caffeine:
    spec: maximumSize=10000,expireAfterWrite=10m,recordStats   # ONE spec shared by all caches

Java - per-cache TTLs with Caffeine (build the caches yourself)

@Configuration
@EnableCaching
class CaffeineConfig {

  @Bean
  CacheManager cacheManager() {
      SimpleCacheManager manager = new SimpleCacheManager();
      manager.setCaches(List.of(
          build("products", 10_000, Duration.ofMinutes(30)),
          build("prices",    5_000,  Duration.ofSeconds(30))));
      return manager;
  }

  private static CaffeineCache build(String name, long max, Duration ttl) {
      return new CaffeineCache(name, Caffeine.newBuilder()
          .maximumSize(max)
          .expireAfterWrite(ttl)
          .recordStats()                  // required for hit/miss metrics
          .build());
  }
}

Java - Redis cache with JSON values and per-cache TTL

@Configuration
@EnableCaching
class RedisCacheConfig {

  @Bean
  RedisCacheManager cacheManager(RedisConnectionFactory factory) {
      RedisCacheConfiguration defaults = RedisCacheConfiguration.defaultCacheConfig()
          .entryTtl(Duration.ofMinutes(10))                 // default is NO expiry
          .disableCachingNullValues()
          .prefixCacheNameWith("shop:v3:")                  // bump v3 when the value shape changes
          .serializeValuesWith(RedisSerializationContext.SerializationPair
              .fromSerializer(new GenericJackson2JsonRedisSerializer()));

      return RedisCacheManager.builder(factory)
          .cacheDefaults(defaults)
          .withCacheConfiguration("prices", defaults.entryTtl(Duration.ofSeconds(30)))
          .withCacheConfiguration("products", defaults.entryTtl(Duration.ofHours(1)))
          .transactionAware()                               // defer put/evict until commit
          .build();
  }
}

What ends up in Redis

redis-cli

127.0.0.1:6379> KEYS shop:v3:prices::*
1) "shop:v3:prices::42"
127.0.0.1:6379> TTL "shop:v3:prices::42"
(integer) 27
127.0.0.1:6379> GET "shop:v3:prices::42"
"{"@class":"com.shop.PriceView","productId":42,"amount":19.99}"

Key format is <prefix><cacheName>::<key>. The @class property is what lets the generic JSON serializer rebuild the right type - and also why renaming or moving a class makes old entries undeserializable.

⚠ Production gotchas

Default JDK serialization: caching a class that is not Serializable fails with org.springframework.data.redis.serializer.SerializationException: Cannot serialize (cause java.io.NotSerializableException). Records and DTOs need implements Serializable or switch to JSON.
Deploy changes the class shape: a rolling deploy mixes old and new pods reading each other's entries. Symptom: SerializationException: Could not read JSON / InvalidClassException spikes during deploys. Fix: version the cache prefix (v3) or tolerate unknown properties.
No TTL on Redis: defaultCacheConfig() has no expiry, so the dataset only grows until Redis evicts or you flush. Always set entryTtl.
Redis outage = application outage: a connection failure surfaces as RedisConnectionFailureException from the cache call, failing the request. Provide a CacheErrorHandler (via CachingConfigurer.errorHandler()) that logs and falls through to the real method.
One global Caffeine spec: spring.cache.caffeine.spec gives every cache the same size/TTL. If you need different TTLs, build caches explicitly as above.
No recordStats: Caffeine hit/miss metrics are all zero; you will think the cache is unused.

Java - fail open when the cache store is down

@Configuration
@EnableCaching
class ResilientCacheConfig implements CachingConfigurer {
  private static final Logger log = LoggerFactory.getLogger(ResilientCacheConfig.class);

  @Override
  public CacheErrorHandler errorHandler() {
      return new CacheErrorHandler() {
          public void handleCacheGetError(RuntimeException e, Cache c, Object key)   { log.warn("cache get failed {}:{}", c.getName(), key, e); }
          public void handleCachePutError(RuntimeException e, Cache c, Object key, Object v) { log.warn("cache put failed {}:{}", c.getName(), key, e); }
          public void handleCacheEvictError(RuntimeException e, Cache c, Object key)  { log.warn("cache evict failed {}:{}", c.getName(), key, e); }
          public void handleCacheClearError(RuntimeException e, Cache c)             { log.warn("cache clear failed {}", c.getName(), e); }
      };
  }
}

Note the trade-off in the handler above: failing open on evict errors can leave stale data in the cache. For correctness-critical evictions rethrow instead, or rely on a short TTL as the safety net.

Chapter Five - Cache Problems

Stampede, Penetration and Consistency

A cache is a second copy of the truth, and every second copy can disagree with the first. Three failure families account for almost every cache incident: stampede (a hot key expires and everyone recomputes at once), penetration (requests for keys that do not exist bypass the cache every time) and staleness (the cache keeps serving what the database no longer says).

Cache stampede (thundering herd) timeline

Problem	What happens	Fix	Spring tool
Stampede / thundering herd	One hot key expires; N threads miss and query the DB simultaneously	Single-flight (one loader per key), refresh-ahead, serve stale while revalidating	`@Cacheable(sync = true)`; Caffeine `refreshAfterWrite` with a `LoadingCache`
Avalanche	Many keys share the same TTL and all expire together after a bulk load or restart	Randomize TTL (jitter), stagger warm-up	Custom `RedisCacheWriter` / TTL function (`RedisCacheConfiguration.entryTtl(TtlFunction)` in newer Spring Data Redis)
Penetration	Lookups for non-existent ids (typos, scans, attackers) miss every time and hit the DB	Cache the negative result briefly; Bloom filter; validate id format first	Allow `null` caching with a short-TTL cache; do not use `unless=#result==null` there
Stale data	Writer updated DB, cache still holds old value	Evict/put on write, short TTL as safety net, versioned keys	`@CachePut`, `@CacheEvict`, `transactionAware()`
Inconsistent evict timing	Evict runs, then a concurrent reader re-populates from the not-yet-committed old row	Evict after commit, optionally delayed double-delete	`TransactionAwareCacheManagerProxy`, `@TransactionalEventListener(AFTER_COMMIT)`

Stampede: what sync = true actually does

Java - single-flight inside one JVM

@Cacheable(cacheNames = "rates", key = "#currency", sync = true)
public ExchangeRate rateFor(String currency) {
  return rateClient.fetch(currency);       // 200 ms; with sync=true, 8 concurrent callers -> 1 fetch
}

With sync = true the interceptor calls Cache.get(key, Callable) instead of get + put, so the provider can take a lock around the load. Constraints: one cache only, no other cache operation on the method, and the guarantee is only as strong as the provider. Caffeine gives you exact single-flight inside one JVM. With a shared Redis cache the default lock is per JVM, so ten pods can still send ten loads - add a distributed lock or a short randomized refresh if the load is truly expensive.

Staleness: ordering writes and cache operations

Strategy	Write path	Read path	Risk
Cache-aside (what @Cacheable does)	Update DB, then evict the key	Miss: load from DB, then put	Race: a reader that loaded the old row before the update can put it back after your evict
Write-through	Update DB and cache together (`@CachePut`)	Reads always hit cache	Two writes, not atomic; a failed cache write leaves old data
Write-behind	Update cache, flush to DB asynchronously	Reads hit cache	Data loss if the node dies before flush - rarely right for business data
TTL only	Nothing	Entry expires on its own	Staleness window = TTL; simplest, often good enough for reference data

Java - evict only after the transaction commits

@Service
class ProductAdmin {
  private final ProductRepository repo;
  private final ApplicationEventPublisher events;

  ProductAdmin(ProductRepository repo, ApplicationEventPublisher events) {
      this.repo = repo; this.events = events;
  }

  @Transactional
  public void rename(long id, String name) {
      repo.findById(id).orElseThrow().rename(name);
      events.publishEvent(new ProductChanged(id));       // delivered at commit time
  }
}

@Component
class ProductCacheInvalidator {
  private final CacheManager caches;
  ProductCacheInvalidator(CacheManager caches) { this.caches = caches; }

  @TransactionalEventListener(phase = TransactionPhase.AFTER_COMMIT)
  void on(ProductChanged e) {
      caches.getCache("products").evict(e.id());          // committed data is now visible to readers
  }
}

record ProductChanged(long id) {}

⚠ Production gotchas

Evict-before-commit race: thread A evicts inside its transaction; thread B reads, misses, loads the old committed row and caches it; A commits. The cache is stale until TTL. The after-commit evict above (or transactionAware()) narrows the window; a short TTL closes it.
unless = "#result == null" on a public lookup: every request for a missing id reaches the DB. During a scan or bot attack the DB sees the full request rate. Cache the miss for a few seconds instead.
allEntries = true on a large Redis cache: Spring clears by pattern scan/delete of the cache's key prefix, which can be slow on millions of keys and blocks other work. Prefer targeted evicts or version the key prefix.
Caching per-user data under a global key: a key that omits the user id serves one user's data to another - a security incident, not a performance bug. Review every key expression for the full set of inputs that affect the result.
Diagnosis: temporarily log hit/miss with the cache metrics (chapter 12), compare the cached value's timestamp with the row's updated_at, and look for the exact write path that skipped the evict (bulk SQL updates, other services, migrations).

Chapter Six - Build It Yourself

A Tiny Cache with TTL, LRU and Single-Flight

To see why single-flight matters and how TTL and LRU really work, here is a ~60-line cache in plain Java - no Spring. It has lazy TTL expiry (checked on read, using an injectable clock so the demo is deterministic), LRU eviction through an access-ordered LinkedHashMap, and a ConcurrentHashMap of in-flight loads so concurrent misses share one loader call - the same idea as sync = true and Caffeine's LoadingCache.

TinyCache.java

import java.util.LinkedHashMap;
import java.util.Map;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.Function;
import java.util.function.LongSupplier;

public class TinyCache<K, V> {
  private record Entry<V>(V value, long expiresAt) {}

  private final int maxSize;
  private final long ttlMillis;
  private final LongSupplier clock;
  private final Map<K, Entry<V>> store;                       // access-ordered => LRU
  private final ConcurrentHashMap<K, CompletableFuture<V>> inFlight = new ConcurrentHashMap<>();
  int hits, misses, evictions;

  TinyCache(int maxSize, long ttlMillis, LongSupplier clock) {
      this.maxSize = maxSize;
      this.ttlMillis = ttlMillis;
      this.clock = clock;
      this.store = new LinkedHashMap<>(16, 0.75f, true) {
          @Override protected boolean removeEldestEntry(Map.Entry<K, Entry<V>> eldest) {
              boolean evict = size() > TinyCache.this.maxSize;
              if (evict) evictions++;
              return evict;
          }
      };
  }

  private synchronized V lookup(K key) {
      Entry<V> e = store.get(key);
      if (e == null) return null;
      if (clock.getAsLong() >= e.expiresAt()) { store.remove(key); return null; }   // lazy expiry
      return e.value();
  }

  private synchronized void put(K key, V value) {
      store.put(key, new Entry<>(value, clock.getAsLong() + ttlMillis));
  }

  /** cache-aside with single-flight: concurrent misses for one key share ONE loader call. */
  V get(K key, Function<K, V> loader) {
      V cached = lookup(key);
      if (cached != null) { synchronized (this) { hits++; } return cached; }
      synchronized (this) { misses++; }
      CompletableFuture<V> mine = new CompletableFuture<>();
      CompletableFuture<V> existing = inFlight.putIfAbsent(key, mine);
      if (existing != null) return existing.join();           // follower: wait for the leader
      try {
          V loaded = loader.apply(key);                       // leader: the only one that loads
          put(key, loaded);
          mine.complete(loaded);
          return loaded;
      } catch (RuntimeException ex) {
          mine.completeExceptionally(ex);
          throw ex;
      } finally {
          inFlight.remove(key);
      }
  }

  public static void main(String[] args) throws Exception {
      long[] now = {0};
      TinyCache<String, String> cache = new TinyCache<>(2, 1_000, () -> now[0]);
      AtomicInteger loads = new AtomicInteger();
      Function<String, String> db = k -> { loads.incrementAndGet(); return k.toUpperCase(); };

      System.out.println("-- TTL --");
      cache.get("a", db); cache.get("a", db);
      System.out.println("t=0     get a twice      loads=" + loads + " hits=" + cache.hits + " misses=" + cache.misses);
      now[0] = 999;  cache.get("a", db);
      System.out.println("t=999   get a            loads=" + loads + " (still fresh)");
      now[0] = 1_000; cache.get("a", db);
      System.out.println("t=1000  get a            loads=" + loads + " (expired -> reload)");

      System.out.println("-- LRU (max 2) --");
      cache.get("b", db); cache.get("a", db); cache.get("c", db);   // b is least recently used
      System.out.println("after a,b,a,c   evictions=" + cache.evictions);
      int before = loads.get();
      cache.get("b", db);
      System.out.println("get b again     reloaded=" + (loads.get() - before == 1) + " (b was evicted)");

      System.out.println("-- stampede: 8 threads, same cold key --");
      TinyCache<String, String> shared = new TinyCache<>(10, 60_000, System::currentTimeMillis);
      AtomicInteger slowLoads = new AtomicInteger();
      Function<String, String> slow = k -> {
          slowLoads.incrementAndGet();
          try { Thread.sleep(200); } catch (InterruptedException ignored) {}
          return "v-" + k;
      };
      ExecutorService pool = Executors.newFixedThreadPool(8);
      CountDownLatch start = new CountDownLatch(1);
      for (int i = 0; i < 8; i++) pool.submit(() -> { start.await(); return shared.get("hot", slow); });
      start.countDown();
      pool.shutdown(); pool.awaitTermination(5, TimeUnit.SECONDS);
      System.out.println("single-flight   loader calls=" + slowLoads.get() + " (without it: up to 8)");
  }
}

$ javac TinyCache.java && java TinyCache

-- TTL --
t=0     get a twice      loads=1 hits=1 misses=1
t=999   get a            loads=1 (still fresh)
t=1000  get a            loads=2 (expired -> reload)
-- LRU (max 2) --
after a,b,a,c   evictions=1
get b again     reloaded=true (b was evicted)
-- stampede: 8 threads, same cold key --
single-flight   loader calls=1 (without it: up to 8)

What the run proves:

Expiry is an inclusive boundary in this design: at t=999 the entry lives, at t=1000 it is gone. Real caches define this precisely too; Caffeine expires lazily and also cleans up on write.
LRU needs access order: touching a before inserting c made b the eviction victim. Caffeine uses the smarter W-TinyLFU policy, which also resists scans that would flush a pure LRU.
The 8-thread run is real concurrency: one thread becomes the leader, seven join() its future. Remove the inFlight map and the printed number rises to between 2 and 8, depending on timing.
This version has a flaw worth finding: a loader that returns null caches nothing and followers receive null - exactly the negative-caching decision from chapter 05.

Chapter Seven - Resilience4j

Retry, Backoff and Timeouts

Everything from here on is Resilience4j, the library Spring Boot teams use for fault tolerance (Hystrix is end-of-life; Spring Cloud CircuitBreaker is a thin abstraction that can sit on top of it). For Boot 3 you add io.github.resilience4j:resilience4j-spring-boot3, spring-boot-starter-aop (the annotations are AOP aspects) and spring-boot-starter-actuator (for health, metrics and event endpoints). Each pattern is a separate decorator with its own name-keyed configuration under resilience4j.<pattern>.instances.<name>.

Retry

Retry re-invokes a failed call after a wait. It is the cheapest tool and the easiest to misuse: retries multiply load on a dependency that is already struggling and are only safe for idempotent operations (GET, PUT, a POST with an idempotency key). Retrying a non-idempotent payment POST can charge twice.

Retry with exponential backoff (blocking)

YAML - retry configuration

resilience4j:
retry:
  instances:
    pricing:
      maxAttempts: 4                         # total calls INCLUDING the first (default 3)
      waitDuration: 500ms                    # default 500ms
      enableExponentialBackoff: true
      exponentialBackoffMultiplier: 2        # 500ms, 1s, 2s
      enableRandomizedWait: true             # jitter so clients do not retry in lock-step
      retryExceptions:                       # retry ONLY these ...
        - java.io.IOException
        - org.springframework.web.client.ResourceAccessException
        - org.springframework.web.client.HttpServerErrorException
      ignoreExceptions:                      # ... and never these (client errors are not transient)
        - org.springframework.web.client.HttpClientErrorException

Java - applying it

@Service
public class PricingGateway {
  private final RestClient rest;

  public PricingGateway(RestClient.Builder builder) {
      this.rest = builder.baseUrl("https://pricing.internal").build();
  }

  @Retry(name = "pricing", fallbackMethod = "defaultPrice")
  public Price fetch(long productId) {
      return rest.get().uri("/prices/{id}", productId).retrieve().body(Price.class);
  }

  // Same params + a trailing exception parameter; called only after retries are exhausted.
  private Price defaultPrice(long productId, Exception ex) {
      return Price.unavailable(productId);
  }
}

⚠ Production gotchas

Retrying everything: without retryExceptions, a 400 Bad Request (a bug in your request) is retried three times. Limit retries to transient errors: timeouts, connection resets, 502/503/504.
maxAttempts off by one: maxAttempts: 3 means one call plus two retries.
Blocking waits: annotation-based retry sleeps on the request thread. With 3 attempts and a 2 s backoff, one bad dependency holds threads for seconds; combine with a short timeout and a bulkhead.
Retry amplification: if service A retries 3x against B, and B retries 3x against C, a failing C receives 9 calls per original request. Retry at one layer only.
Retry + @Transactional in the same method: each attempt must run in a fresh transaction. Put @Retry on the outer caller or ensure the retry aspect wraps the transactional proxy, never the other way round.

Timeouts are the real first line of defence

A retry, a breaker or a bulkhead cannot help if a call never returns. Set connect and read timeouts on the HTTP client itself - on Spring Boot 3.4+ via spring.http.client.connect-timeout and spring.http.client.read-timeout, or on the request factory (JdkClientHttpRequestFactory, HttpComponentsClientHttpRequestFactory) for earlier versions. Choose them from the dependency's p99 latency, not from hope. Resilience4j's TimeLimiter (chapter 09) is for async calls; it cannot interrupt a blocking socket read.

Chapter Eight - Circuit Breaker

The State Machine, Window and Thresholds

A circuit breaker watches recent calls and, once the dependency looks unhealthy, stops calling it for a while. Rejected calls fail in microseconds with io.github.resilience4j.circuitbreaker.CallNotPermittedException instead of tying up a thread for a timeout. That does two things: the caller's threads stay free, and the sick dependency gets quiet time to recover instead of being hammered by retries.

Circuit breaker state machine (Resilience4j)

Config key	Default	Meaning
`slidingWindowType`	`COUNT_BASED`	Window is the last N calls, or (`TIME_BASED`) the last N seconds
`slidingWindowSize`	100	N for the window above
`minimumNumberOfCalls`	100	No tripping until this many calls are recorded - the most common reason "the breaker never opens" in tests
`failureRateThreshold`	50 (%)	Failure % at/above which the breaker opens
`slowCallDurationThreshold` / `slowCallRateThreshold`	60 s / 100 %	Calls slower than the duration count as slow; open when the slow % reaches the rate. Default effectively disables slow-call tripping
`waitDurationInOpenState`	60 s	How long to stay OPEN before allowing probes
`permittedNumberOfCallsInHalfOpenState`	10	Probe calls in HALF_OPEN; their failure rate decides CLOSED vs OPEN
`automaticTransitionFromOpenToHalfOpenEnabled`	false	false: the OPEN→HALF_OPEN move happens when the first call arrives after the wait. true: a background timer does it
`recordExceptions` / `ignoreExceptions`	any exception recorded	Which exceptions count as failures; ignored ones are neither success nor failure

YAML - a production-shaped breaker

resilience4j:
circuitbreaker:
  configs:
    default:
      slidingWindowType: COUNT_BASED
      slidingWindowSize: 20
      minimumNumberOfCalls: 10
      failureRateThreshold: 50
      slowCallDurationThreshold: 2s
      slowCallRateThreshold: 80
      waitDurationInOpenState: 30s
      permittedNumberOfCallsInHalfOpenState: 5
      automaticTransitionFromOpenToHalfOpenEnabled: true
      registerHealthIndicator: true
      ignoreExceptions:
        - com.shop.pricing.PriceNotFoundException      # business "404" is not a dependency failure
  instances:
    pricing:
      baseConfig: default
management:
health:
  circuitbreakers:
    enabled: true

Java - breaker with typed fallbacks

@Service
public class PricingGateway {
  private static final Logger log = LoggerFactory.getLogger(PricingGateway.class);
  private final RestClient rest;
  private final PriceSnapshotStore lastKnown;

  public PricingGateway(RestClient rest, PriceSnapshotStore lastKnown) {
      this.rest = rest; this.lastKnown = lastKnown;
  }

  @CircuitBreaker(name = "pricing", fallbackMethod = "fallbackPrice")
  public Price fetch(long productId) {
      Price p = rest.get().uri("/prices/{id}", productId).retrieve().body(Price.class);
      lastKnown.save(p);
      return p;
  }

  // Chosen when the breaker is OPEN: the most specific matching exception type wins.
  private Price fallbackPrice(long productId, CallNotPermittedException ex) {
      log.warn("pricing breaker open, serving last known price for {}", productId);
      return lastKnown.find(productId).orElseGet(() -> Price.unavailable(productId));
  }

  // Chosen for any other failure that propagated out of the call.
  private Price fallbackPrice(long productId, Throwable ex) {
      log.error("pricing call failed for {}", productId, ex);
      return Price.unavailable(productId);
  }
}

⚠ Production gotchas

Breaker never opens in a test: minimumNumberOfCalls defaults to 100. Ten failing calls will not trip a default breaker. Lower it for tests or send enough calls.
Fallback swallows the failure: if the method itself catches the exception and returns a default, the breaker sees a success and never opens. Let the exception propagate to the aspect and handle it in the fallbackMethod.
Fallback signature mismatch: the fallback must live in the same class, return the same type and take the same parameters plus a trailing exception. Otherwise startup/first call fails with NoSuchMethodException: class ... does not have fallback method (wording varies by version).
Business exceptions counted as failures: a 404 or validation error opens the breaker and takes down a perfectly healthy dependency. Put them in ignoreExceptions.
One breaker for many endpoints: a slow /reports endpoint trips the breaker and blocks fast /prices calls. Use one named instance per dependency capability.
Diagnosis: GET /actuator/circuitbreakers shows each instance's state and failure rate; GET /actuator/circuitbreakerevents lists the recent STATE_TRANSITION and NOT_PERMITTED events; the metric resilience4j_circuitbreaker_state graphs transitions.

Chapter Nine - Load Limiters

Rate Limiter, Bulkhead and Time Limiter

Retry and circuit breaker react to failures. The next three limit load: how many calls per unit of time (rate limiter), how many at the same time (bulkhead) and how long any one may take (time limiter).

Pattern	Question it answers	Key settings (defaults)	Rejection exception
`@RateLimiter`	How many calls per period may I make (or accept)?	`limitForPeriod` 50, `limitRefreshPeriod` 500 ns, `timeoutDuration` 5 s (how long to wait for a permit)	`RequestNotPermitted`
`@Bulkhead` (semaphore)	How many calls may be in flight at once?	`maxConcurrentCalls` 25, `maxWaitDuration` 0	`BulkheadFullException`
`@Bulkhead(type = THREADPOOL)`	Run calls on an isolated, bounded thread pool	`coreThreadPoolSize`, `maxThreadPoolSize`, `queueCapacity` (100); method must return `CompletableFuture`	`BulkheadFullException`
`@TimeLimiter`	How long may an async call take?	`timeoutDuration` 1 s, `cancelRunningFuture` true; method must return `CompletableFuture`	`java.util.concurrent.TimeoutException`

Bulkhead: isolate failures like compartments in a ship

Without a bulkhead, one slow dependency can absorb all 200 Tomcat threads. A semaphore bulkhead caps concurrent calls to that dependency at, say, 20; the 21st caller fails immediately (or waits up to maxWaitDuration) while the other 180 threads keep serving unrelated endpoints.

YAML

resilience4j:
bulkhead:
  instances:
    reports:
      maxConcurrentCalls: 10        # at most 10 report calls in flight
      maxWaitDuration: 0ms          # do not queue - fail fast
ratelimiter:
  instances:
    partnerApi:
      limitForPeriod: 100           # 100 calls ...
      limitRefreshPeriod: 1s        # ... per second
      timeoutDuration: 0            # reject immediately instead of waiting for a permit
timelimiter:
  instances:
    slowSearch:
      timeoutDuration: 800ms
      cancelRunningFuture: true

Java - the three annotations in use

@Service
public class ReportClient {
  private final RestClient rest;
  public ReportClient(RestClient rest) { this.rest = rest; }

  @Bulkhead(name = "reports", fallbackMethod = "busy")
  public Report build(long id) {
      return rest.get().uri("/reports/{id}", id).retrieve().body(Report.class);
  }

  private Report busy(long id, BulkheadFullException ex) {
      throw new ResponseStatusException(HttpStatus.SERVICE_UNAVAILABLE, "reports busy, retry shortly");
  }
}

@Service
public class SearchClient {
  private final RestClient rest;
  public SearchClient(RestClient rest) { this.rest = rest; }

  // TimeLimiter needs a future; the work runs on whatever executor completes it.
  @TimeLimiter(name = "slowSearch", fallbackMethod = "empty")
  public CompletableFuture<List<Hit>> search(String q) {
      return CompletableFuture.supplyAsync(() ->
          rest.get().uri("/search?q={q}", q).retrieve().body(new ParameterizedTypeReference<List<Hit>>() {}));
  }

  private CompletableFuture<List<Hit>> empty(String q, TimeoutException ex) {
      return CompletableFuture.completedFuture(List.of());
  }
}

@RestController
class PartnerController {
  private final PartnerService service;
  PartnerController(PartnerService service) { this.service = service; }

  // Server-side rate limit: protect YOUR service from a noisy caller.
  @RateLimiter(name = "partnerApi")
  @GetMapping("/partner/data")
  Data data() { return service.load(); }
}

⚠ Production gotchas

Unhandled RequestNotPermitted returns 500: map it in a @RestControllerAdvice to 429 Too Many Requests (add a Retry-After header) - see Validation & Error Handling.
Per-instance, not global: a Resilience4j rate limiter counts inside one JVM. Ten pods with limitForPeriod: 100 allow 1000/s overall. For a true global limit use a gateway or a Redis-based limiter.
TimeLimiter cannot kill a blocked thread: cancelRunningFuture only calls Future.cancel(true); a task stuck in a socket read does not stop. The HTTP client timeout is what actually frees the thread.
Unbounded executor under a CompletableFuture: supplyAsync without an executor uses the common ForkJoinPool, which you share with the whole JVM. Pass a dedicated, bounded executor.

Chapter Ten - Combining Patterns

Ordering, Layering and Fallbacks

You can put several annotations on one method. They are not applied in the order you type them; each is a separate Spring AOP aspect with a fixed order. The Resilience4j default, outermost to innermost, is Retry → CircuitBreaker → RateLimiter → TimeLimiter → Bulkhead → your method. Each aspect's order can be overridden with resilience4j.<pattern>.<pattern>AspectOrder properties (for example resilience4j.circuitbreaker.circuitBreakerAspectOrder); Spring's Ordered rule applies, lower value = further out.

Resilience4j annotation nesting (default aspect order)

What you want	Consequence of the default nesting	Watch out for
A retry that respects the breaker	Retry is outside, so every attempt goes through the breaker and is recorded	When the breaker opens, the retry sees `CallNotPermittedException` and may retry it three more times pointlessly - add it to the retry `ignoreExceptions`
The breaker to count each attempt	Yes - three attempts = three records	A single logical request can contribute several failures to the window
Bulkhead limits real concurrency	Innermost: only calls that passed the rate limiter and are not yet timed out occupy a permit	Retries re-acquire a permit each attempt
Cache hits to skip all of this	Put `@Cacheable` on an outer bean method that delegates to the resilient gateway	On the same method, order between the cache interceptor and resilience aspects is not something to rely on
Per-call timeout inside retry	TimeLimiter is inside Retry, so each attempt gets its own timeout	Total worst case = attempts x timeout + backoff waits

Java - a layered design that is easy to reason about

@Service
public class PriceService {                       // layer 1: cache (outer bean)
  private final PricingGateway gateway;
  public PriceService(PricingGateway gateway) { this.gateway = gateway; }

  @Cacheable(cacheNames = "prices", key = "#productId", sync = true)
  public Price priceFor(long productId) {
      return gateway.fetch(productId);          // call goes through the gateway PROXY
  }
}

@Service
public class PricingGateway {                     // layer 2: resilience (inner bean)
  @Retry(name = "pricing")                      // outermost resilience aspect
  @CircuitBreaker(name = "pricing", fallbackMethod = "stale")
  @Bulkhead(name = "pricing")                   // innermost
  public Price fetch(long productId) { /* HTTP call with a client read-timeout */ return null; }

  private Price stale(long productId, Throwable ex) { return Price.unavailable(productId); }
}

⚠ Production gotchas

Fallback result gets cached: the cache sits outside the gateway, so a degraded Price.unavailable() returned by the fallback is stored for the full TTL. Mark degraded values and exclude them with unless = "#result.degraded()".
Retry beyond the user's patience: total latency budget = attempts x (timeout + wait). If the browser gives up after 10 s, a 4 x 5 s policy keeps working for a client who is gone.
Fallbacks that call the same dependency: a fallback that hits a second remote service has its own failure modes. Prefer last-known value, static default or a clear 503.
Annotations on a method called from the same class: same self-invocation trap - none of the aspects run. Verify with the retry metric or a failing stub in a test.
Virtual threads / reactive code: annotation-based blocking retry and bulkhead semantics differ with Mono/Flux return types (use the Reactor operators) - do not mix blocking decorators into reactive chains.

A forward-looking note: newer Spring Framework generations add core resilience annotations (@Retryable, @ConcurrencyLimit) so simple retry/limit cases need no extra library; this page targets the Spring Boot 3 / Resilience4j combination that most production codebases still use.

Chapter Eleven - Build It Yourself

A Tiny Circuit Breaker

The state machine from chapter 08 is small enough to write in an afternoon. This version is count-based (last windowSize calls), uses an injectable clock, and models CLOSED / OPEN / HALF_OPEN with probe calls. It is a teaching model: Resilience4j evaluates the half-open outcome after all permitted probes complete, whereas this one trips as soon as the probe failure share reaches the threshold - so a still-sick backend is detected one probe sooner.

TinyBreaker.java

import java.util.ArrayDeque;
import java.util.Deque;
import java.util.function.LongSupplier;
import java.util.function.Supplier;

public class TinyBreaker {
  enum State { CLOSED, OPEN, HALF_OPEN }
  static class CallNotPermittedException extends RuntimeException {
      CallNotPermittedException(String m) { super(m); }
  }

  private final int windowSize, minCalls, halfOpenPermits;
  private final double failureRateThreshold;
  private final long openWaitMillis;
  private final LongSupplier clock;
  private final Deque<Boolean> window = new ArrayDeque<>();   // true = failure (count-based window)
  private State state = State.CLOSED;
  private long openedAt;
  private int probesStarted, probeFailures, probeSuccesses;

  TinyBreaker(int windowSize, int minCalls, double failureRate, long openWaitMillis, int halfOpenPermits, LongSupplier clock) {
      this.windowSize = windowSize; this.minCalls = minCalls; this.failureRateThreshold = failureRate;
      this.openWaitMillis = openWaitMillis; this.halfOpenPermits = halfOpenPermits; this.clock = clock;
  }

  synchronized State state() { return state; }

  <T> T call(Supplier<T> action) {
      synchronized (this) {
          if (state == State.OPEN) {
              if (clock.getAsLong() - openedAt >= openWaitMillis) {
                  transition(State.HALF_OPEN);
                  probesStarted = probeFailures = probeSuccesses = 0;
              } else {
                  throw new CallNotPermittedException("CircuitBreaker is OPEN and does not permit further calls");
              }
          }
          if (state == State.HALF_OPEN) {
              if (probesStarted >= halfOpenPermits)
                  throw new CallNotPermittedException("CircuitBreaker is HALF_OPEN and permits no more probe calls");
              probesStarted++;
          }
      }
      try {
          T result = action.get();
          record(false);
          return result;
      } catch (RuntimeException ex) {
          record(true);
          throw ex;
      }
  }

  private synchronized void record(boolean failure) {
      if (state == State.HALF_OPEN) {
          if (failure) probeFailures++; else probeSuccesses++;
          if (probeFailures > 0 && (double) probeFailures / halfOpenPermits * 100 >= failureRateThreshold) trip();
          else if (probeFailures + probeSuccesses == halfOpenPermits) { window.clear(); transition(State.CLOSED); }
          return;
      }
      window.addLast(failure);
      if (window.size() > windowSize) window.removeFirst();
      long failures = window.stream().filter(f -> f).count();
      double rate = 100.0 * failures / window.size();
      if (window.size() >= minCalls && rate >= failureRateThreshold) trip();
  }

  private void trip() { openedAt = clock.getAsLong(); transition(State.OPEN); }

  private void transition(State to) {
      System.out.println("    [state] " + state + " -> " + to);
      state = to;
  }

  public static void main(String[] args) {
      long[] now = {0};
      TinyBreaker cb = new TinyBreaker(10, 5, 50.0, 30_000, 3, () -> now[0]);
      boolean[] backendUp = {true};
      Supplier<String> backend = () -> {
          if (!backendUp[0]) throw new IllegalStateException("503 from payment-service");
          return "ok";
      };

      System.out.println("1) healthy traffic");
      for (int i = 0; i < 5; i++) cb.call(backend);
      System.out.println("   5 calls ok, state=" + cb.state());

      System.out.println("2) backend dies");
      backendUp[0] = false;
      for (int i = 1; i <= 8; i++) {
          try { cb.call(backend); }
          catch (CallNotPermittedException e) { System.out.println("   call " + i + " -> short-circuited: " + e.getMessage()); }
          catch (RuntimeException e) { System.out.println("   call " + i + " -> backend failure: " + e.getMessage()); }
      }

      System.out.println("3) 30s later, backend still down");
      now[0] += 30_000;
      for (int i = 1; i <= 3; i++) {
          try { cb.call(backend); } catch (RuntimeException e) { System.out.println("   probe " + i + " -> " + e.getMessage()); }
      }
      System.out.println("   state=" + cb.state());

      System.out.println("4) another 30s, backend recovered");
      now[0] += 30_000;
      backendUp[0] = true;
      for (int i = 1; i <= 3; i++) System.out.println("   probe " + i + " -> " + cb.call(backend));
      System.out.println("   state=" + cb.state());
  }
}

$ javac TinyBreaker.java && java TinyBreaker

1) healthy traffic
 5 calls ok, state=CLOSED
2) backend dies
 call 1 -> backend failure: 503 from payment-service
 call 2 -> backend failure: 503 from payment-service
 call 3 -> backend failure: 503 from payment-service
 call 4 -> backend failure: 503 from payment-service
  [state] CLOSED -> OPEN
 call 5 -> backend failure: 503 from payment-service
 call 6 -> short-circuited: CircuitBreaker is OPEN and does not permit further calls
 call 7 -> short-circuited: CircuitBreaker is OPEN and does not permit further calls
 call 8 -> short-circuited: CircuitBreaker is OPEN and does not permit further calls
3) 30s later, backend still down
  [state] OPEN -> HALF_OPEN
 probe 1 -> 503 from payment-service
  [state] HALF_OPEN -> OPEN
 probe 2 -> 503 from payment-service
 probe 3 -> CircuitBreaker is OPEN and does not permit further calls
 state=OPEN
4) another 30s, backend recovered
  [state] OPEN -> HALF_OPEN
 probe 1 -> ok
 probe 2 -> ok
  [state] HALF_OPEN -> CLOSED
 probe 3 -> ok
 state=CLOSED

Reading the transcript:

The [state] CLOSED -> OPEN line prints while call 5 is being recorded, before the caller's message: 5 successes + 5 failures = 10 recorded calls, 50% failure, minCalls of 5 met, threshold 50% reached. Call 4 was not enough (4 of 9 = 44%).
Calls 6-8 never reach the backend: that is the entire value of the pattern - three rejected calls cost nanoseconds, not three 30-second timeouts.
After the wait the first call flips OPEN to HALF_OPEN (no background timer - the default Resilience4j behaviour), and that call is a real probe against the backend.
Probe 2's failure reopens the breaker for another full wait, so probe 3 is rejected without a call. Production breakers use the same "any bad probe reopens" idea.
What this model leaves out: time-based windows, slow-call detection, ignored exceptions, metrics and thread-safety under heavy contention - which is what the library provides.

Chapter Twelve - Operate It

Metrics, Health and Tests

Caches and breakers fail silently by design: a cache that never hits looks like a slow service, a breaker stuck open looks like a dead dependency. Wire them into your observability from day one (see Actuator & Observability).

Signal	Where	What to alert on
Cache hit ratio	Micrometer `cache.gets` with tag `result=hit\|miss` (Prometheus: `cache_gets_total`); Caffeine needs `recordStats`	Ratio drops suddenly after a deploy (key change, TTL too short)
Cache size / evictions	`cache.size`, `cache.evictions`	Evictions rising while size sits at the max: cache too small
Breaker state	`resilience4j_circuitbreaker_state`, `/actuator/circuitbreakers`, health indicator (`registerHealthIndicator: true`)	State = open for longer than expected
Rejected calls	`resilience4j_circuitbreaker_not_permitted_calls_total`, `resilience4j_ratelimiter_*`, bulkhead available permits	Sustained rejections: capacity or downstream problem
Retry outcome	`resilience4j_retry_calls_total` with `kind=successful_with_retry\|failed_with_retry\|...`	High `successful_with_retry` ratio: dependency flaky even if users see success

Testing the policy, not just the happy path

Java - integration test that trips the breaker (JUnit 5 + AssertJ + MockRestServiceServer-style stub)

@SpringBootTest(properties = {
  "resilience4j.circuitbreaker.instances.pricing.minimumNumberOfCalls=4",
  "resilience4j.circuitbreaker.instances.pricing.slidingWindowSize=4",
  "resilience4j.circuitbreaker.instances.pricing.failureRateThreshold=50"
})
class PricingGatewayBreakerTest {

  @Autowired PricingGateway gateway;
  @Autowired CircuitBreakerRegistry registry;
  @MockitoBean PricingClient client;               // @MockBean before Boot 3.4

  @Test
  void opensAfterRepeatedFailuresAndServesFallback() {
      when(client.fetch(anyLong())).thenThrow(new IllegalStateException("503"));

      for (int i = 0; i < 4; i++) {
          assertThat(gateway.fetch(1L).available()).isFalse();     // fallback each time
      }

      assertThat(registry.circuitBreaker("pricing").getState())
          .isEqualTo(CircuitBreaker.State.OPEN);

      reset(client);
      gateway.fetch(1L);                                            // rejected: backend untouched
      verifyNoInteractions(client);
  }
}

The pattern: lower the thresholds through properties, drive failures with a mock, assert on CircuitBreakerRegistry state and on the absence of calls once open. Reset breaker state between tests (registry.circuitBreaker("pricing").reset()) or the order of your tests will matter. More on test setup in Testing Spring Applications.

Chapter Thirteen - Interview Angles

Questions You Should Be Able to Answer

Crisp answers for the questions that come up in interviews on this topic.

Q: Why does my @Cacheable method run every time?
Almost always self-invocation (calling this.method()), a private/final method or class that cannot be proxied, no @EnableCaching/cache starter, or a key that changes every call (a Pageable or timestamp parameter). Verify by checking that the bean is a proxy and logging the key.
Q: @Cacheable vs @CachePut?
@Cacheable skips the method on a hit. @CachePut always runs the method and overwrites the entry. Never put both on the same method - their semantics conflict.
Q: How is the default cache key generated?
SimpleKeyGenerator: no args gives SimpleKey.EMPTY, one arg uses the arg, several args wrap them in a SimpleKey. Override with SpEL key or a KeyGenerator.
Q: What does sync = true guarantee?
Concurrent misses for the same key invoke the method once and share the result - within what the provider can lock. Caffeine: per JVM exactly. Shared Redis: not cluster-wide by default.
Q: Local vs distributed cache - how do you choose?
Local (Caffeine) for read-heavy, small, tolerant-of-staleness data and lowest latency; distributed (Redis) when all instances must agree or the data set is large. Often both: Caffeine L1 with short TTL, Redis L2.
Q: What is a cache stampede and how do you prevent it?
Many requests recompute an expired hot key simultaneously. Single-flight loading (sync=true), refresh-ahead, jittered TTLs, serving stale during refresh, and request coalescing.
Q: How do you keep the cache consistent with the database?
Cache-aside with evict/put on write, evict after commit, short TTL as a backstop, versioned keys for schema changes. Accept that a cache is eventually consistent unless you give up its speed.
Q: Circuit breaker states and transitions?
CLOSED (calls flow, outcomes recorded) → OPEN when failure/slow rate ≥ threshold over the window and minimum calls met; OPEN rejects with CallNotPermittedException for waitDurationInOpenState; then HALF_OPEN allows N probes, which decide CLOSED or OPEN again.
Q: Retry vs circuit breaker - why both?
Retry hides transient blips; the breaker stops you from retrying a dependency that is clearly down. Retry without a breaker amplifies load on a sick dependency; a breaker without retry exposes users to single hiccups.
Q: In what order do Resilience4j annotations apply?
Retry → CircuitBreaker → RateLimiter → TimeLimiter → Bulkhead → method (outer to inner), regardless of the order you write them. Overridable via the *AspectOrder properties.
Q: Bulkhead vs rate limiter?
Rate limiter bounds calls per time (throughput); bulkhead bounds calls at once (concurrency). A slow dependency exhausts concurrency long before it exceeds a rate limit, so bulkheads protect threads and rate limiters protect the callee.
Q: Why is a retry on a POST dangerous?
POST is not idempotent: if the first request succeeded but the response was lost, the retry creates a duplicate. Use idempotency keys or restrict retries to idempotent methods.

Chapter Fourteen - Summary

Cheat Sheet and Recap

Cheat sheet

Need	Use	Remember
Skip repeated reads	`@Cacheable(key=..., unless=...)`	Proxy only; key must include every input; set TTL on the provider
Refresh on write	`@CachePut` / `@CacheEvict` (after commit)	Evict-then-race: keep a short TTL as a backstop
Protect from stampede	`sync=true`, jitter, refresh-ahead	Provider decides the strength of the lock
Survive a blip	`@Retry` + timeouts	Idempotent calls only; maxAttempts includes the first call
Stop hammering a dead dependency	`@CircuitBreaker` + fallback	minimumNumberOfCalls, ignoreExceptions, never swallow the exception
Cap concurrency	`@Bulkhead`	One per dependency; fail fast
Cap throughput	`@RateLimiter`	Per JVM, not global
Bound async latency	`@TimeLimiter`	Client timeouts are the real fix

∑ Caching & Resilience - what to remember

Both are proxies. Self-invocation, private methods and unproxied beans silently disable every annotation on this page.
Cache = second copy of the truth. Design the key, the TTL and the invalidation path together; a cache without a TTL is a memory leak with a delay.
Caffeine for speed, Redis for sharing - and serialize JSON, version your prefix, fail open on cache errors.
Stampede, penetration, avalanche are different problems with different fixes: single-flight, negative caching, TTL jitter.
Timeouts first. Retry, breaker and bulkhead all assume calls eventually return.
Breaker = rate over a window, with a minimum call count; it recovers through HALF_OPEN probes.
Order is outer to inner: Retry, CircuitBreaker, RateLimiter, TimeLimiter, Bulkhead.
Measure it: hit ratio, breaker state, rejected calls, retry outcomes.

Related pages:

AOP & Proxies - why every annotation here is a proxy and what self-invocation does.
Transactions - commit ordering, after-commit events, and retry vs transaction boundaries.
Actuator & Observability - exposing breaker endpoints and cache metrics.
Configuration & Profiles - per-environment TTLs and thresholds.
Testing Spring Applications - how to test the policies above.
Interview Q&A - more questions across all topics.

← 12 Actuator & Observability
Spring hub
14 Testing →

LearningTree

Caching & Resilience | Spring Guide

Caching & Resilience

Repeated Work and Failing Dependencies

Interceptors on a Proxy

@Cacheable, @CachePut, @CacheEvict and the Key

Caffeine vs Redis, TTL and Serialization

Stampede, Penetration and Consistency

A Tiny Cache with TTL, LRU and Single-Flight

Retry, Backoff and Timeouts

The State Machine, Window and Thresholds

Rate Limiter, Bulkhead and Time Limiter

Ordering, Layering and Fallbacks

A Tiny Circuit Breaker

Metrics, Health and Tests

Questions You Should Be Able to Answer

Cheat Sheet and Recap