A rate limiter controls how many requests a client can make to a service within a given time window. Without it, a single misbehaving client — malicious or buggy — can consume all resources and bring down the system for everyone. Rate limiting is not optional for any public-facing API. It protects against DDoS attacks, prevents abuse, ensures fair usage, and keeps your infrastructure costs predictable.

Memory estimation: 10M keys × multiple counters per key. For token bucket: 2 values per key (tokens_remaining + last_refill_timestamp) = approximately 16 bytes per rule per key. With 3 rules per client (per-second, per-minute, per-day) = 48 bytes per client. 10M clients × 48 bytes = approximately 480MB. Redis comfortably holds this in memory with room for overhead. This calculation justifies why Redis (memory-first) is the right store rather than a disk-based database that would never achieve sub-millisecond access at this scale.

The 1ms latency budget is tight. A Redis round-trip in the same datacenter is 0.3–0.5ms. A Lua script execution adds ~0.1ms. Network variance adds another 0.1ms. Total: approximately 0.5–0.7ms for the happy path. This leaves no room for multiple Redis calls per request — the entire check-and-decrement must be one atomic operation. It also means the rate limiter cannot call a database or any service with higher latency. Redis is not a preference here — it is the only option that fits the latency budget.

Rate limiting sounds simple until you realize there are dozens of valid configurations. The questions below determine whether you need a simple in-memory counter or a distributed coordination system — and the answer often depends on business context more than technology.

The simplest rate limiter: a HashMap in application memory. Key = client ID, value = request count + window start time. On each request, check if count exceeds limit. Simple, fast (~0.1ms), zero external dependencies. Works perfectly on a single server — and fails immediately when you have multiple servers behind a load balancer, because each server has its own counter.

Sticky sessions route all requests from a specific client to the same server, effectively making the rate limiter work correctly for that client. This partially solves the split counter problem but introduces new issues: sticky sessions reduce load balancer effectiveness (one server may get all requests from a high-volume client while others sit idle), they break on server restart (the sticky client gets a fresh counter), and they require the load balancer to track client identity (adding complexity and becoming a single point of failure for that routing decision). Sticky sessions are a workaround that trades one problem for three others.

The in-memory counter approach uses a fixed window: reset the counter every minute. This creates a boundary burst problem: a client can make their full limit of requests at 11:59:59, then their full limit again at 12:00:00. They have made 2× their limit in two seconds. At the naive design stage this is less important than the multi-server problem, but it is a reason why the naive approach cannot simply be "fixed" by adding shared state — the algorithm itself needs to change.

The solution to split counters is shared state. Redis provides the perfect combination: in-memory speed (~0.5ms per operation), atomic operations (INCR + EXPIRE), and cluster mode for availability. The rate limiter becomes a thin layer that queries Redis on every request — adding minimal latency while maintaining a single global counter per client.

The atomic rate limit check in one Redis round-trip: the script receives the key, current time, bucket capacity, and refill rate. It reads the current token count and last refill timestamp, calculates how many tokens have been refilled since the last request (elapsed_seconds × refill_rate), caps the total at bucket capacity, checks if at least one token remains, decrements if allowed or returns denied if not, writes the new state, and returns the result. This entire sequence runs on the Redis server without any intermediate network calls. Without Lua, a client would need to do GET (read state), local calculation, and SET (write state) — with a race window between GET and SET where another request could read the same stale state.

For extremely high-traffic clients, a local in-memory cache of the rate limit decision can reduce Redis calls dramatically. After a Redis check confirms a client is within limits, the application server caches the result locally for 100ms. Subsequent requests from the same client within that window skip the Redis call entirely. The trade-off: a client could make up to 100ms × local_rate requests more than their limit before the next Redis check. For most APIs this overage is acceptable. For strict financial APIs (Stripe), it is not. The local cache approach reduces Redis calls by 80–90% for high-frequency clients, directly reducing latency and Redis cluster load.

Five algorithms dominate rate limiter implementations. Each trades off between memory usage, accuracy, burst handling, and complexity. The right choice depends on whether you care more about strict enforcement or graceful burst handling.

Use token bucket when: clients are well-behaved and occasionally burst (page loads, mobile app cold starts). The burst tolerance improves user experience without enabling sustained abuse. This is the right default for developer APIs.

Use leaky bucket when: the downstream service you are protecting cannot handle bursts at all. Constant-rate output protects fragile services. Less common in API rate limiting, more common in network traffic shaping.

Use sliding window counter when: you need better accuracy than fixed window but cannot afford the memory of sliding window log. The 0.1% error is acceptable for almost all API rate limiting use cases. Cloudflare uses this at global scale.

Use fixed window counter when: you need absolute minimum memory and can tolerate the 2× boundary burst. Internal service-to-service rate limiting where clients are trusted and burst tolerance is acceptable.

Use sliding window log only when: you need perfect accuracy and have a very small number of clients (the memory cost is too high at scale). Useful for high-value enterprise customers where contractual accuracy matters more than memory efficiency.

Instead of counting requests, assign a cost to each request based on its computational complexity and limit the total cost per time window. A simple GET request costs 1. A complex aggregation query costs 50. The client's budget depletes by the cost of each request, not by a flat count. This prevents a single expensive query from being treated the same as a trivial one. Implementation: calculate cost at request parse time (before execution), then apply token bucket or sliding window on the cost value rather than request count. Shopify's GraphQL API uses this approach — query complexity is calculated from the AST before execution begins.

Every major API platform has a rate limiter. Their approaches differ based on constraints — Stripe needs absolute accuracy (money involved), GitHub needs to be lenient (developer experience), Cloudflare needs extreme speed (inline on every request globally).

Rate limiting patterns transfer to any system where you need to control resource consumption: API calls, database queries, message queue publishing, email sending, or feature flag rollouts. The principles below apply far beyond their rate limiting origin.

Rate limiters sit on the critical path of every request. When they fail, they either block all legitimate traffic (fail-closed) or allow unlimited abuse (fail-open). The failures below have taken down production systems — usually because the rate limiter itself became the bottleneck it was designed to prevent.