Redis In Depth
Redis is a data structure server that lives in memory. This page builds the model from the ground up: the single command thread and why it matters, each data type with real commands and complexity, expiry and eviction, what RDB and AOF can and cannot save, replication, Sentinel and Cluster, transactions and scripts, streams versus Pub/Sub, the lock and rate-limiter patterns, memory and performance, the licence and Valkey situation, Spring Data Redis with Lettuce, and a hands-on session in Docker.
An In-Memory Data Structure Server
Redis (REmote DIctionary Server) is a server that keeps data in memory and exposes it as data structures - strings, hashes, lists, sets, sorted sets, streams and more - through a simple command protocol (RESP). That is the key difference from a plain key-value cache: the server understands the structure of the value, so ZINCRBY leaderboard 10 asha or HINCRBY user:42 logins 1 is a single atomic server-side operation instead of a read-modify-write across the network.
The execution model explains most of Redis's behaviour. A single thread executes commands one after another, so every command (and every MULTI/EXEC block or Lua script) is atomic with respect to the others. Since Redis 4.0 some work runs in background threads (for example freeing large objects with UNLINK, closing files and fsync), and since Redis 6 you can enable I/O threads (io-threads) that read and write sockets in parallel while command execution stays on the main thread. In the Redis 7.4.12 and 8.10.3 Docker images used for this page CONFIG GET io-threads returned 1, which means no extra I/O threads unless you configure them. Valkey 9.1 advertises a redesigned I/O threading model on valkey.io; check the docs of your version before tuning this.
| Question | Redis answer | Consequence |
|---|---|---|
| Where is the data? | In RAM; disk only for persistence | Dataset must fit in memory (or you evict). Latency is microseconds on the server side; the network dominates |
| What is a value? | A typed data structure with its own commands | Push work to the server: increments, set operations, range queries, rankings |
| How are commands executed? | One at a time on one thread | Atomic by construction. One slow command blocks everyone |
| How do I scale reads / data size? | Replicas for reads and failover; Cluster to shard data | Chapter 5 |
| Is it durable? | Optionally: RDB snapshots, AOF log, or both | Default is snapshots only; you can lose recent writes (chapter 4) |
| What is it good at? | Caches, sessions, counters, rate limits, leaderboards, queues and streams, locks | Anything small, hot and structurally simple |
| What is it bad at? | Large cold datasets, ad hoc queries, strong multi-key consistency | Use a database for the system of record; Redis in front of it |
"Redis" now names more than one thing, and the licence matters if you ship or host it. Everything below was read from redis.io (the 2024-03-20 licensing blog post, the 2025-05-01 AGPLv3 post and the legal/licenses page) and valkey.io on 2026-10-31.
| When | What happened | Source |
|---|---|---|
| Redis up to 7.2.x | BSD 3-clause open source licence | redis.io licenses page: "7.2.x and earlier remain subject to the BSD3 license" |
| 2024-03-20 | Redis announced that, starting with Redis 7.4, it is dual-licensed under RSALv2 and SSPLv1 (source-available, not OSI open source) and no longer BSD | redis.io blog "Redis Adopts Dual Source-Available Licensing" |
| March 2024 | Valkey, a Linux Foundation project, was started as a BSD fork "right before the transition" (the GitHub repository was created on 2024-03-22) | Valkey README; GitHub repository metadata |
| 2025-05-01 | Redis announced AGPLv3 as an additional option starting with Redis 8 (OSI-approved), and integrated JSON, Time Series, probabilistic types and the Query Engine into core Redis 8 | redis.io blog "Redis is now available under the AGPLv3 open source license" |
| Now (2026-10-31) | Redis 8 is offered under a tri-licence (RSALv2, SSPLv1, AGPLv3). Valkey stays BSD; its latest release was 9.1.3 (2026-10-26) | redis.io licenses page; valkey.io |
What this means in practice: the wire protocol and the commands in this page work the same on Redis and on Valkey (the examples were run against redis:7 = 7.4.12, redis:8 = 8.10.3 and valkey/valkey:8/valkey/valkey:9). Managed services (ElastiCache, Memorystore, Azure) have their own offerings and their own version lists. New commands diverge: DELEX and SET ... IFEQ exist in Redis 8.4 and later (chapter 8), and each project adds features the other does not. Pin your client library, test on the engine you really deploy, and ask your legal team which licence terms apply if you redistribute or offer Redis as a service.
A tutorial written for Redis 8 (with JSON, search and vector sets built in) will not necessarily work on Redis 7.2, on Valkey, or on a cloud service pinned to an older engine. Check INFO server (redis_version, redis_mode) and MODULE LIST before assuming a feature exists.
Structures, Commands and What They Cost
A Redis key maps to a value of one type, and the type decides which commands work (WRONGTYPE Operation against a key holding the wrong kind of value is what you get otherwise). Because commands are executed one at a time on a single thread, the cost of a command is the cost everyone pays: know the complexity of what you call, especially the O(N) ones.
| Type | Core commands | Typical complexity | Gotcha |
|---|---|---|---|
| String (binary safe, up to 512 MB) | SET GET INCR INCRBY APPEND SETEX MGET | GET SET INCR O(1) | A 100 MB string is allowed and is a latency spike each time it moves. Counters are strings holding integers |
| Hash | HSET HGET HMGET HINCRBY HGETALL HDEL | HGET HSET O(1); HGETALL O(N) | EXPIRE covers the whole hash. Per-field expiry (HEXPIRE) exists from Redis 7.4 (worked on the 7.4.12 image); older engines lack it |
| List | LPUSH RPUSH LPOP RPOP LRANGE BLPOP LTRIM | Push/pop at the ends O(1); LRANGE O(S+N); LINDEX O(N) | Middle access is linear. LTRIM after LPUSH gives a capped "latest N" list |
| Set | SADD SREM SISMEMBER SMEMBERS SINTER SUNION SPOP | SADD SISMEMBER O(1); SMEMBERS O(N); SINTER O(N*M) worst case | SMEMBERS on a million members blocks the server; use SSCAN |
| Sorted set | ZADD ZINCRBY ZRANGE ZRANK ZREVRANGE ZREMRANGEBYSCORE ZCARD | ZADD ZRANK O(log N); ZRANGE O(log N + M) | Ordered by score, then lexicographically by member. Scores are doubles: integers above 2^53 lose precision |
| Stream | XADD XREAD XGROUP XREADGROUP XACK XAUTOCLAIM XTRIM | XADD O(1) without trimming; reads O(N) in entries returned | Grows until you XTRIM or use MAXLEN on XADD (chapter 7) |
| Bitmap (string) | SETBIT GETBIT BITCOUNT BITOP BITFIELD | SETBIT O(1); BITCOUNT O(N) | Offset 4 billion allocates 512 MB for one key. Keep offsets dense |
| HyperLogLog | PFADD PFCOUNT PFMERGE | O(1) per add; PFCOUNT O(1) on one key | Uses up to 12 KB and has a 0.81 % standard error. Cannot list members or delete one |
| Geo (sorted set) | GEOADD GEODIST GEOSEARCH | GEOADD O(log N) per item; GEOSEARCH O(N+log M) | Longitude comes before latitude in every command |
Redis picks a compact internal encoding for small values and switches to a general one as they grow. You can see it with OBJECT ENCODING. This is why a hash with five fields costs a few hundred bytes while a million of them still fits: small hashes, sets and sorted sets use a listpack, a flat array scanned linearly, and convert to a hash table or skiplist past configurable limits.
JSON, search and more. Redis 8 integrates JSON, Time Series, probabilistic data types (Bloom filter and friends) and the Query Engine into core Redis; MODULE LIST on the 8.10.3 image showed search, bf and timeseries loaded. On Redis 7.x or Valkey these come from separate modules or are absent. Because that depends on the engine, this page sticks to the core types.
- A hash lets you read or increment one field (
HINCRBY user:42 logins 1) without touching the rest, and is compact for flat objects. A JSON string (SET user:42 {...}) is the simplest for nested objects you always read and write whole. Choose by access pattern, not fashion.
Naming Keys and Keeping Memory Bounded
Keys are binary-safe strings, but treat them like an API. A good key is readable, namespaced, bounded and versioned.
| Guideline | Example | Why |
|---|---|---|
| Colon-separated namespaces | order:1042:items | Convention, easy to grep and to scan with MATCH order:* |
| Put the entity id in the key, not the value | session:7f3a... not a hash of all sessions | Many small keys can expire and be sharded independently; one giant key cannot |
| Prefix with the service or environment | checkout:prod:cart:42 | Several apps often share one Redis; collisions are silent |
| Add a schema version | shop:v2:price:42 | Bump the version instead of deleting millions of keys when the value format changes |
| Keep keys short but not cryptic | u:42:p is false economy | Key bytes are memory, but 20-40 byte keys are fine; readability saves incidents |
| Put a hash tag only where you need co-location | {user:42}:cart, {user:42}:profile | In Cluster, keys sharing the tag land in one slot (chapter 5) |
| Never build keys from unbounded user input | search: + raw query | Attackers fill memory; hash or length-limit the input |
EXPIRE key seconds (or SET key v EX seconds, PEXPIRE for milliseconds, EXPIREAT for a timestamp) attaches a time to live. TTL returns the remaining seconds, -1 if the key has no expiry and -2 if the key does not exist. PERSIST removes the TTL. **Overwriting a key with SET clears its TTL** unless you pass KEEPTTL.
Expiry is lazy plus active. Lazy: when a command touches an expired key Redis deletes it and behaves as if it never existed. Active: Redis periodically tests a few keys at random among those with a TTL and deletes the expired ones, so memory is reclaimed even for keys nobody reads. A consequence worth knowing: INFO keyspace shows keys=17,expires=2,avg_ttl=..., and a key whose time has passed can still occupy memory for a short while until a cycle or an access removes it. Replicas do not expire keys themselves: they wait for the DEL from the primary.
maxmemory-policy)With maxmemory 0 (the default) Redis uses all memory the OS will give it. In production set maxmemory below the machine or container limit (leave room for replication buffers, fork copy-on-write and fragmentation) and pick a policy. All eight names below were accepted by CONFIG SET maxmemory-policy on the Redis 7.4.12 image (an invalid name returns the list), and the default is noeviction.
| Policy | Evicts | Use when |
|---|---|---|
noeviction (default) | Nothing; writes that need memory fail with an OOM error, reads still work | Redis is a database or queue: losing data is worse than rejecting writes |
allkeys-lru | Least recently used key among all keys | Pure cache with recency-biased access. The usual first choice |
allkeys-lfu | Least frequently used key among all keys | Pure cache where a few keys are very hot for a long time |
allkeys-random | A random key | All keys equally likely to be reused |
volatile-lru | LRU among keys that have a TTL | Mixed use: cache keys carry a TTL, durable keys do not |
volatile-lfu | LFU among keys that have a TTL | Same, frequency based |
volatile-random | Random among keys with a TTL | Rare |
volatile-ttl | Keys with the shortest remaining TTL | You encode importance in the TTL |
The volatile-* policies behave like noeviction when no key has a TTL, a classic way to get OOM errors in a "cache" where someone forgot to set expiries. **Newer Redis 8.x lines add two more names, allkeys-lrm and volatile-lrm (least recently modified)**: they were accepted by Redis 8.10.3 and rejected by 7.4.12 in the test for this page, and appear on the current eviction docs page. Do not rely on them if you run 7.x or Valkey.
LRU and LFU are approximated: Redis samples maxmemory-samples keys (default 5) and evicts the best candidate among them, which is cheap and close to exact LRU. Raising the sample size is more precise and costs CPU.
- Symptom:
used_memoryclimbs for days, then either the OOM killer ends the process (nomaxmemory) or every write fails withOOM command not allowed(noeviction). - Fix: always set
maxmemoryand either a TTL on every cache key or anallkeys-*policy. Watchevicted_keysandexpired_keysinINFO stats.
RDB Snapshots, the AOF and What You Can Lose
Redis is memory first, but it can write to disk so that a restart does not start from empty. There are two mechanisms, usable alone or together, and none of them makes Redis a fully ACID database.
| RDB snapshot | AOF (append-only file) | |
|---|---|---|
| What is stored | Compact point-in-time copy of the dataset (dump.rdb) | The write commands, replayed on restart. Since Redis 7.0 a multi-part AOF: a base file plus incremental files in a directory, tracked by a manifest |
| How it is written | fork(): a child writes the file while the parent keeps serving (copy-on-write) | Parent appends each write to the log; a background rewrite compacts it |
| Loss window after a crash | Everything since the last snapshot (docs: "be prepared to lose the latest minutes of data") | With appendfsync everysec: at most about one second of writes |
| Restart speed | Faster for big datasets | Slower (replay), mitigated by the RDB preamble |
| File size | Smaller | Larger, until rewritten |
| Cost | A fork per snapshot can pause the server for milliseconds, up to about a second on a very large instance | fsync policy trades speed for safety; rewrite also forks |
| Best for | Backups, fast restarts, caches you can rebuild | When you want to lose at most about a second |
| Setting | Default observed | Meaning |
|---|---|---|
save | 3600 1 300 100 60 10000 | Snapshot after 1 hour if >=1 change, after 5 min if >=100 changes, after 1 min if >=10000 changes |
appendonly | no | AOF is off by default. Out of the box Redis is snapshot-only |
appendfsync | everysec | Used when AOF is turned on; the docs call this the suggested and default policy |
aof-use-rdb-preamble | yes | AOF rewrite starts the base file as an RDB, so restarts load faster |
auto-aof-rewrite-percentage / -min-size | 100 / 67108864 (64 MB) | Rewrite when the AOF doubles and is at least 64 MB |
The three appendfsync values: **always fsyncs before replying (very safe, much slower; concurrent writes in a pipeline or from several clients are grouped into one fsync), everysec fsyncs from a background thread once a second, and no leaves it to the OS (Linux typically flushes every 30 seconds, kernel dependent). The Redis docs recommend using both RDB and AOF** if you want data safety "comparable to what PostgreSQL can provide", RDB alone if you can accept minutes of loss, and no persistence at all for a pure cache.
- Even with
appendfsync always, a Redis primary that fails over to an asynchronous replica can lose writes the client saw acknowledged. Persistence protects against restart; it does not give you synchronous multi-node durability. - A fork on a big instance needs copy-on-write memory. If the machine has no spare memory, the snapshot or rewrite can fail or push the box into swap. Leave headroom (chapter 9).
- Turning AOF on by editing the config and restarting can lose data (the server then loads an empty AOF). Follow the documented procedure: enable it live with
CONFIG SET appendonly yes, wait for the rewrite to finish, then persist the setting.
- Pure cache: no persistence (or RDB only, to restart warm). Session store or rate limits you can afford to lose a second of: AOF
everysec. Primary store (rare): AOFeverysecoralwaysplus RDB backups, replicas, and an honest statement of the loss you accept.
Replicas, Sentinel and Cluster
A replica connects to a primary, receives a snapshot, and then a continuous stream of write commands. Replication is asynchronous by default: the primary acknowledges your write to you without waiting for any replica. That keeps latency low and means a primary crash can lose the last writes that never reached a replica.
REPLICAOF host portmakes a node a replica (inredis.confasreplicaof). Replicas are read-only by default; reading from them can return stale data (replication lag).WAIT numreplicas timeoutblocks the calling connection until the replicas have acknowledged the preceding writes of that connection, or the timeout (milliseconds) passes. It returns how many replicas acknowledged. It narrows the loss window; it does not turn Redis into a strongly consistent store, because a failover can still elect a replica that missed something.- A replica that reconnects after a short outage can often do a partial resynchronisation from the primary's replication backlog instead of a full copy.
| Redis Sentinel | Redis Cluster | |
|---|---|---|
| Goal | High availability for a single dataset: monitoring, failure detection, automatic failover, configuration provider | Sharding across primaries plus failover |
| Data size | Limited to one primary's memory | Sum of the primaries |
| Topology | 1 primary + replicas, watched by 3 or more Sentinel processes | 3 or more primaries, each ideally with a replica |
| How clients find the primary | Ask Sentinel (SENTINEL get-master-addr-by-name), the client library does it | Cluster-aware client keeps a slot map; follows MOVED |
| Failover decision | A quorum of Sentinels agrees the primary is unreachable; a majority authorises the failover | Cluster nodes vote; a replica of the failed primary is promoted |
| Multi-key operations | Unrestricted | Only within one slot (CROSSSLOT otherwise); hash tags co-locate keys |
| Databases | Numbered databases work | Only database 0 |
| Pick it when | The dataset fits one machine and you want automatic failover | The dataset or throughput outgrows one machine |
Hash slots. The key space is divided into 16384 slots; a key's slot is CRC16(key) mod 16384. Each primary owns a range of slots. When a client sends a command to a node that does not own the key's slot, the node replies **MOVED slot host:port (a permanent redirect: update your slot map). While a slot is being migrated, the node can reply ASK**, a one-time redirect to the target, which the client must precede with ASKING. Good clients (Lettuce, Jedis, redis-py cluster mode) do all of this for you.
Hash tags. If a key contains {...} with at least one character inside, only that part is hashed. {user:1}:cart and {user:1}:profile land in the same slot, so MULTI/EXEC, Lua scripts and multi-key commands over them work. The trade-off: if you tag everything with the same value you have built one hot shard.
- Multi-key commands across slots fail (
CROSSSLOT).MGET,SUNION,RENAME, Lua scripts and transactions all need their keys in one slot. - **No
SELECT.** Code that uses several logical databases must be changed. - Hot keys and big keys do not shard. One key lives on one primary no matter how large the cluster is.
- By default the cluster stops accepting writes if any slot has no owner (
cluster-require-full-coverage yeswas the observed default). Decide deliberately whether you prefer partial availability.
MULTI/EXEC, WATCH, Lua and Functions
Single commands are atomic. When you need several to run as one unit, Redis offers three tools, each with a different guarantee.
| Tool | What you get | What you do not get |
|---|---|---|
MULTI ... EXEC | Commands are queued, then executed sequentially without any other client's command in between (isolated). If the connection drops before EXEC, none run | No rollback. If one command fails at run time the others still execute |
WATCH key + MULTI/EXEC | Optimistic locking (check-and-set): EXEC aborts and returns nil if a watched key changed since WATCH | You must retry in a loop; heavy contention means many retries |
Lua script (EVAL / EVALSHA) | Reads, decisions and writes in one atomic server-side step, one round trip | A slow script blocks the whole server. In Cluster all keys must be passed in KEYS and share a slot |
Functions (FUNCTION LOAD, FCALL) | Server-side code that is part of the dataset (persisted and replicated) rather than sent by each client | Same blocking caveat as scripts |
LPUSH a x failed at execution time because a is a string (the first INCR created it), yet the commands before and after it took effect. The docs state plainly that Redis does not support rollbacks because it would hurt simplicity and performance. A command with a syntax error is rejected at queue time and the whole transaction is discarded; a run-time error is not rolled back.
For anything with logic (read, decide, write) a Lua script is usually simpler than WATCH retries. The sliding-window rate limiter in chapter 8 is exactly that. Keep scripts short: while one runs, nothing else does.
- Redis transactions give isolation (no interleaving) and all-or-nothing queueing, not rollback and not durability beyond your persistence settings. If you need "debit A, credit B, or neither", model it with idempotent steps or use a database.
Pub/Sub Versus Streams
Redis offers three message-like tools. Choosing the wrong one is a common source of lost messages.
| Pub/Sub | List (LPUSH + BRPOP) | Stream (consumer groups) | |
|---|---|---|---|
| Persistence of messages | None. Fire and forget | Until popped | Until trimmed; survives restarts if persistence is on |
| Delivery semantics | At most once (the docs: "if at all") | At most once once popped (a crashed worker loses the item) | At least once with XACK and redelivery |
| Many consumers each get every message? | Yes (fan-out) | No, competing consumers | Yes, one group per consumer service; inside a group they compete |
| Replay history | No | No | Yes: read from any id |
| Track who has not finished | No | No | Yes: the pending entries list (XPENDING) |
| Typical use | Cache invalidation hints, presence, live dashboards where loss is fine | Very simple job queue | Event pipelines, task queues that must not lose work |
The workflow is XGROUP CREATE once, then each worker loops XREADGROUP GROUP g worker COUNT n BLOCK ms STREAMS key >, processes, and calls XACK. Entries read but never acknowledged stay pending; XAUTOCLAIM key group consumer min-idle-time start moves entries idle longer than the threshold to a live consumer. Cap growth with XADD key MAXLEN ~ 100000 * ... or XTRIM.
- A subscriber that restarts, lags beyond its output buffer limit, or is not yet subscribed misses the message. Symptom: "the email was sometimes never sent". Use a stream with a consumer group and idempotent handlers.
Cache, Sessions, Rate Limits, Locks, Leaderboards, Idempotency
The default caching pattern: the application reads the cache, on a miss reads the database, then writes the cache with a TTL (SET product:42 <json> EX 600). All the difficult parts - stampedes, penetration, invalidation races, sizing - are the subject of page 06, Caching Patterns; the Spring annotation side is in Spring: Caching and Resilience.
A session store is the same idea with a TTL that slides: SET session:<id> <data> EX 1800, and EXPIRE session:<id> 1800 on each request (or GETEX session:<id> EX 1800 to read and extend in one command). Because Redis is shared, the app servers stay stateless. Decide what a lost session costs: a forced login is usually acceptable, which is why a snapshot-only Redis is a common choice.
The simplest limiter counts requests per user per window. The first INCR creates the key, so that is when you set the expiry. The weakness is the boundary: a client can send the limit at 11:59:59 and again at 12:00:00, doubling the rate for a moment.
Keep one sorted set per user, members are request ids and scores are timestamps. For each request: remove entries older than the window, count what is left, add this request if under the limit. This is exact, at the cost of memory proportional to the limit. The script below was run for real (limit 3, window 60 s, timestamps 1100 to 1400 ms, then 70000 ms).
The script takes now as an argument instead of calling TIME inside, which keeps the example deterministic; in production pass the client clock or call redis.call('TIME'). In Cluster the single key rl:user:42 is fine. A cheaper approximation is the sliding window counter (two fixed-window counters weighted by overlap), and a smoother alternative is a token bucket kept in a hash with a script. Pick by how exact the limit must be.
A lock in Redis is a key that exists only while someone holds it: SET lock:order:42 <random token> NX PX 30000. NX means set only if absent, PX gives it a lifetime so a crashed owner cannot hold it forever. The token identifies the owner, so that release is conditional: delete only if the value still equals my token. A plain DEL could delete somebody else's lock after yours expired.
On Redis 8.4 and later the compare-and-delete script is built in: DELEX key IFEQ value deletes the key only if its value equals the argument. It returned 0 for a wrong token and 1 for the right one on the 8.10.3 image, and SET key value IFEQ expected (conditional overwrite, same version) also worked. On 7.x and Valkey use the Lua script.
- Efficiency locks ("avoid doing the same job twice"): fine. If the lock occasionally fails, the cost is duplicate work.
- Correctness locks ("two writers must never both modify this"): a TTL lock cannot guarantee it alone. The owner can be paused (garbage collection, a slow network call, a suspended VM) past the TTL; the lock expires; a second client acquires it; the first wakes up and still believes it holds the lock, exactly as in the diagram. Clocks and failovers add further ways to break mutual exclusion: Redis replication is asynchronous, so a primary can crash after granting a lock and before the replica sees it, and the promoted replica grants the same lock again (the Redis docs make this very point).
- Fencing tokens fix the dangerous case: the lock service hands out an increasing number with each grant, the client sends it with every write, and the resource rejects writes with a token lower than one it has already seen. This needs support in the protected resource (a database column, a conditional update such as
WHERE version = ?). Redis can generate tokens withINCR, but the check must happen where the data lives.
Redlock is the algorithm in the Redis documentation for locking across several independent Redis primaries (typically five): acquire on a majority within a time limit shorter than the lock validity, and treat the lock as held for the remaining validity. On 8 February 2016 Martin Kleppmann published an analysis ("How to do distributed locking") arguing that Redlock is unsafe for correctness locks because it depends on timing assumptions (bounded pauses, bounded clock drift) and has no fencing token; Salvatore Sanfilippo (antirez) answered the next day ("Is Redlock safe?"), defending the design and arguing the assumptions are reasonable. The Redis lock documentation itself links both articles, tells you that if you care about correctness you should implement fencing tokens, and presents Redlock as the algorithm to "model" its guarantees. The honest summary: Redlock is reasonable for efficiency locks; for correctness, rely on fencing or on a system built for consensus (for example a lock or lease in ZooKeeper, etcd or your database). Read both articles and decide for your risk.
- Using a fixed value (
"locked") instead of a unique token, then releasing someone else's lock. - Doing
SETNXthenEXPIREas two commands: a crash between them leaves a lock that never expires. Use oneSET ... NX PX. - A TTL shorter than the work, with no renewal or fencing. Symptom: rare duplicate processing under load or after a GC pause.
- Treating "I got the lock" as proof of exclusivity after a long pause.
A leaderboard is a sorted set: ZINCRBY lb:weekly 10 asha on each score event, ZREVRANGE lb:weekly 0 9 WITHSCORES for the top ten, ZREVRANK lb:weekly asha for one player's position (all O(log N) or O(log N + M)). The session in chapter 2 shows real output. Reset a period by writing to a new key (lb:2026-W44) and letting the old one expire.
An idempotency key makes a retried request safe: SET idem:<client key> <result or "in-progress"> NX EX 86400. If SET returns OK you are the first and you process the request; if it returns nil, the request is a duplicate and you return the stored result or a "being processed" answer. Store the final response under the key once finished. Redis alone is not enough if the business effect is in another database: make the effect itself idempotent as well (see Messaging fundamentals).
What Makes Redis Slow, and What To Watch
| Topic | What to know | How to look |
|---|---|---|
| Per-key overhead | Every key has bookkeeping; millions of tiny keys cost more than their payload. Small hashes use compact encodings | MEMORY USAGE key (real: 96 bytes for a 3-field hash, 56 for an integer string) |
| Big keys | A hash, list or set with millions of entries (or a string of hundreds of MB) makes DEL, replication and failover slow, and in Cluster cannot be split | redis-cli --bigkeys (it samples with SCAN), MEMORY USAGE key SAMPLES 0 |
| Deleting big keys | DEL frees memory in the command thread; UNLINK hands the freeing to a background thread | Prefer UNLINK for large keys |
| Fragmentation | The allocator (jemalloc) cannot always return freed pages; RSS exceeds used_memory | INFO memory: mem_fragmentation_ratio. Well above 1.5 deserves a look; below 1 means swapping |
| Fork headroom | Snapshots and rewrites rely on copy-on-write; a write-heavy instance can need extra memory while the child runs | Keep spare memory; check INFO persistence and latency around BGSAVE |
| Buffers | Client output buffers, replication backlog and AOF buffers count against the machine, and some count against maxmemory | INFO clients, client-output-buffer-limit |
--bigkeys output on the eviction test server (abridged)| Rule | Why | Instead |
|---|---|---|
Do not use KEYS pattern in production | O(N) over the whole keyspace in the command thread: every client waits | SCAN cursor MATCH pattern COUNT n, O(1) per call, incremental. It may return a key more than once and does not see a consistent snapshot, so de-duplicate |
| Avoid O(N) commands on large collections | HGETALL, SMEMBERS, LRANGE 0 -1, ZRANGE 0 -1 on big keys block everyone | HSCAN, SSCAN, ZSCAN, pagination with LRANGE/ZRANGE windows |
| Pipeline independent commands | A round trip costs far more than executing a command; N commands in one batch pay the latency once | Client pipelining (Lettuce does it automatically for concurrent async calls; executePipelined in Spring) |
Use MGET/HMGET/MSET | Fewer round trips | But in Cluster the keys must share a slot |
| Keep scripts short | A script blocks the server for its whole run | Move loops and heavy logic out |
| Watch for hot keys | One key = one thread = one shard | Local (in-process) L1 cache in front, key replication with suffixes (page 06) |
| Set client timeouts and pool sizes | A stalled Redis plus no timeout exhausts application threads | Timeouts on connect and command, circuit breaker around Redis calls |
Tools to look: SLOWLOG GET 10 (commands slower than slowlog-log-slower-than, default 10000 microseconds), LATENCY DOCTOR, INFO commandstats, redis-cli --latency, and MONITOR (only briefly: it slows the server).
KEYS *in a health check or admin script. One run on a large instance caused every client call to time out.- Caching without TTL or
maxmemory: slow memory leak until OOM. - Serialising with the JDK default and storing class names: unreadable keys and values, breaks across deployments (chapter 10).
- One giant hash or set "for convenience": big-key latency, impossible to shard.
- Assuming a replica read is current; assuming
WAITmakes writes durable. - Putting Redis in the request path with no timeout, no fallback and no circuit breaker.
- Storing the system of record in a cache-configured Redis (
allkeys-lru, no AOF).
Lettuce, Templates, Serializers and RedisCacheManager
Spring Boot's spring-boot-starter-data-redis brings Spring Data Redis and the Lettuce client by default (Netty based, thread-safe, one shared connection serves many threads, Jedis is the alternative). The code below was compiled and run against a real redis:7 container with Spring Boot 3.5.16 (Spring Data Redis 3.5.x, Java 17). Property names are the Boot 3 ones: spring.data.redis.host, spring.data.redis.port, spring.data.redis.password, spring.data.redis.timeout.
| Class | Use it for | Serialization |
|---|---|---|
StringRedisTemplate | Keys and values that are strings: counters, locks, tokens, flags, Lua scripts returning numbers | UTF-8 strings. Readable in redis-cli |
RedisTemplate<K, V> | Objects. You configure serializers for key, value, hash key and hash value | Default is JDK serialization (binary, class-bound). Replace it |
RedisCacheManager | Backs @Cacheable / @CacheEvict with Redis (see Spring: Caching and Resilience) | RedisCacheConfiguration per cache: TTL, prefix, serializers |
opsForValue() opsForHash() opsForList() opsForSet() opsForZSet() opsForStream() | Typed views of each data type | Use the template's serializers |
GenericJackson2JsonRedisSerializer writes a @class property so it can rebuild the type. The real value stored for product(42) was {"@class":"demo.RedisPatterns$Product","id":42,"name":"Desk lamp","priceCents":1999} with a TTL of about 60 seconds (the prices cache) under the key shop:v1:prices::42. Spring Data Redis 4.x (Spring Boot 4) adds GenericJacksonJsonRedisSerializer next to the older GenericJackson2JsonRedisSerializer (both classes exist in 4.1.2); check the reference docs of the version you use.
A first version cached a Long (@Cacheable method returning Long). The first call worked; the second call, served from Redis, threw java.lang.ClassCastException: class java.lang.Integer cannot be cast to class java.lang.Long, because JSON has one number type and the value 1999 came back as Integer. Cache a typed object (a record or class) instead of a bare number, or use a serializer configured with the exact type. Always test the second call, the one that hits the cache.
- Set
spring.data.redis.timeout. Give every@Cacheablecache a TTL (entryTtl). Replace JDK serialization. Prefix keys with the application and a version. Usescan, neverkeys, and close theCursor. Remember@Cacheableis proxy based: calling it from inside the same class bypasses the cache.
Run Redis and Valkey in Docker and Poke at Everything
You need Docker and nothing else. The tags below were pulled on 2026-10-31: redis:7 resolved to 7.4.12, redis:8 to 8.10.3, valkey/valkey:8 to 8.1.11 and valkey/valkey:9 to 9.1.3 (read from each image with redis-server --version / valkey-server --version). Tags move; re-check.
valkey-cli and redis-cli)- Types. Build a user hash, a tags set and a leaderboard sorted set. Run
TYPEandOBJECT ENCODINGon each, then add 600 members to the set and check the encoding again. - Expiry.
SET k v PX 200, wait,GET k. ThenSET k v EX 100,SET k v2,TTL k- what happened to the TTL? Retry withKEEPTTL. - Eviction. Start a server with
--maxmemory 3mb --maxmemory-policy allkeys-lru, load 4000 keys of 1 KB, readINFO statsforevicted_keys. Switch tonoevictionwithCONFIG SETand try one more write. - Persistence. Start with
--appendonly yes, write a key,docker restart, read it back. Look inside theappendonlydirvolume. Repeat with persistence off and see the key vanish. - Cluster slots. Start with
--cluster-enabled yes,CLUSTER ADDSLOTSRANGE 0 16383and runCLUSTER KEYSLOTon keys with and without hash tags. - Lock. Reproduce the lock session of chapter 8, then let the TTL expire between two acquires.
- Stream. Start two consumers in a group, kill one without
XACK, and recover its entry withXAUTOCLAIM.
docker rm -f redis7 valkeyremoves the containers. Nothing here writes outside them.
Redis Questions with Crisp Answers
- Q: Why is Redis fast if it is single-threaded?
Data is in memory, structures are simple, there is no locking, an event loop multiplexes many sockets, and most latency is the network. Command execution is one thread; I/O threads and background threads exist around it. Single-threaded execution also makes each command atomic. - Q: What is the difference between Redis and Memcached?
Redis stores typed structures, supports persistence, replication, Lua, streams and Cluster. Memcached is a simpler, multi-threaded string cache with no persistence. Pick Memcached only for a plain cache where you want that simplicity. - Q: Explain expiry versus eviction.
Expiry removes keys whose TTL elapsed, lazily on access and actively by sampling. Eviction removes keys whenmaxmemoryis reached, chosen bymaxmemory-policy. They are independent mechanisms. - Q: Name the eviction policies.
noeviction(default),allkeys-lru,allkeys-lfu,allkeys-random,volatile-lru,volatile-lfu,volatile-random,volatile-ttl; newer Redis 8.x lines addallkeys-lrmandvolatile-lrm.volatile-*only considers keys with a TTL. - Q: RDB versus AOF?
RDB: periodic compact snapshots, fast restart, may lose minutes. AOF: a log of writes, witheveryseclosing about a second, bigger and slower to replay. Using both is the documented recommendation for safety. The default config ran snapshots only (appendonly no). - Q: Can Redis lose acknowledged writes?
Yes. With async replication a failover can promote a replica that missed writes, and witheveryseca crash loses about a second.WAITnarrows but does not eliminate the window. - Q: Sentinel or Cluster?
Sentinel gives failover for one dataset that fits on one primary. Cluster shards data over 16384 slots and also fails over, at the cost of multi-key and database restrictions. - Q: What happens when a Cluster client hits the wrong node?
The node repliesMOVED slot host:port; the client updates its slot map and retries. During migration the node may replyASKfor a one-off redirect withASKING. - Q: What is a hash tag and why use it?
The part of the key inside{}is the only part hashed, so related keys land in one slot and multi-key operations work. Overuse creates hot shards. - Q: Does MULTI/EXEC roll back?
No. Commands are queued and run sequentially without interleaving; a run-time failure leaves the others applied.WATCHadds optimistic check-and-set; Lua gives atomic logic. - Q: Pub/Sub versus Streams?
Pub/Sub is at-most-once with no storage. Streams persist entries, support consumer groups, acknowledgements and redelivery (at-least-once). Use streams for anything that must not be lost. - Q: How do you implement a safe distributed lock?
SET key token NX PX ttl, release only if the token matches (Lua, orDELEX IFEQon Redis 8.4+). For correctness, add fencing tokens checked by the protected resource; a TTL lock alone can overlap after a pause or failover. - Q: Is Redlock safe?
Debated. Kleppmann argued it relies on timing assumptions and lacks fencing; antirez replied it is adequate under stated assumptions. Treat it as suitable for efficiency locks and use fencing or a consensus system when correctness is critical. - Q: Why avoid KEYS in production?
It is O(N) over the whole keyspace on the command thread and blocks every client. UseSCAN, which is incremental but may return duplicates. - Q: How would you find what is eating memory?
INFO memory,redis-cli --bigkeys,MEMORY USAGE key,SCANwith a type filter, andINFO keyspacefor key counts and expiries. - Q: How do you build a rate limiter?
Fixed window:INCRplusEXPIREin a script (boundary burst). Sliding window: sorted set of timestamps, trim, count, add, in a Lua script. Token bucket for smooth limits. - Q: What changed with Redis licensing?
Redis 7.4 (2024-03-20 announcement) moved from BSD to RSALv2/SSPLv1; Redis 8 added AGPLv3 as an option (2025-05-01). Valkey is the BSD-licensed Linux Foundation fork started in March 2024. - Q: Why can
@Cacheablereturn a ClassCastException from Redis?
The JSON serializer loses numeric type information (a Long returns as Integer). Cache typed objects and test the cache-hit path. - Q: Scenario: your Redis memory is at 100 % and writes fail.
Checkmaxmemory-policy(noeviction?), whether keys have TTLs, big keys via--bigkeys,evicted_keys. Fix: add TTLs, choose anallkeys-*policy if it is a cache, shard or scale up if it is real data.
Cheat Sheet and Recap
| Topic | Remember | Value / default (checked) |
|---|---|---|
| Execution | Commands run one at a time; each is atomic | io-threads = 1 on 7.4.12 and 8.10.3 |
| Eviction | maxmemory + maxmemory-policy | Default noeviction, maxmemory 0; sample size 5 |
| Expiry | Lazy plus active; SET clears TTL, KEEPTTL keeps it | TTL -1 no expiry, -2 no key |
| RDB | Snapshots by fork, lose minutes | save 3600 1 300 100 60 10000 |
| AOF | Write log; everysec loses about 1 s | appendonly no, appendfsync everysec |
| Replication | Asynchronous; WAIT narrows the loss window | Replicas read-only |
| Cluster | CRC16 mod 16384; MOVED, ASK, CROSSSLOT; hash tags {} | Database 0 only |
| Transactions | Isolated, no rollback; WATCH for CAS | Lua for atomic logic |
| Pub/Sub vs Streams | At-most-once vs at-least-once with groups | XACK, XAUTOCLAIM |
| Lock | SET k token NX PX ttl + compare-and-delete + fencing | DELEX IFEQ on Redis 8.4+ |
| Never in prod | KEYS, unbounded SMEMBERS/HGETALL | Use SCAN, pagination |
| Spring | StringRedisTemplate, RedisCacheManager, JSON serializer, TTL per cache | Boot 3.5.16, Spring Data Redis 3.5.x |
- An in-memory data structure server: pick the structure whose commands do your work on the server.
- One command at a time gives atomicity and a blocking risk: avoid O(N) commands on big keys.
- **Set
maxmemoryand a policy**, and a TTL on every cache key. Expiry and eviction are different. - Persistence reduces loss, it does not remove it: RDB loses minutes, AOF everysec about a second, failover can lose more.
- Sentinel for failover, Cluster for sharding; Cluster costs multi-key operations and
SELECT. - Streams for reliable messaging, Pub/Sub for fire and forget.
- Locks: a token, a TTL, a conditional release, and fencing where correctness matters.
- Know which Redis you run: licence (BSD to 7.2, RSALv2/SSPLv1 from 7.4, AGPLv3 option from 8) and the Valkey fork.
Checked on 2026-10-31: command behaviour, configuration defaults, eviction policy names and DELEX/IFEQ by running redis:7 (7.4.12), redis:8 (8.10.3) and valkey images; licence facts against redis.io (licensing blog 2024-03-20, AGPLv3 blog 2025-05-01, licenses page) and valkey.io; persistence, replication, cluster, Pub/Sub, transaction and lock statements against redis.io/docs/latest; Spring signatures with javap on spring-data-redis 3.5.13 and 4.1.2 and a compile-and-run on Spring Boot 3.5.16. Not verified: exact release dates of Redis 8.x minors, Valkey-specific persistence defaults, managed-service behaviour (ElastiCache and others).
- 06 Caching Patterns - stampedes, consistency races, invalidation and a build-it cache in plain Java.
- 01 NoSQL Fundamentals - the models and trade-offs behind choosing a store.
- Spring: Caching and Resilience - the annotation side of caching.
- Messaging map - when a stream is not enough.