Caching in System Design: Strategies, Eviction, Invalidation, and Stampede Prevention
A practical reference for designing cache-backed read and write paths. You will compare cache-aside, write-through, and write-behind; separate eviction from invalidation; choose freshness and memory policies; and protect popular keys from stampedes when they expire under load.
Why Put a Cache Between the Application and the Database?
Every request that reaches the database pays the cost of retrieving data from the source. A database read might take 1–10 ms, while a cache hit in Redis costs approximately 0.1 ms—an order-of-magnitude difference before you account for database contention or network distance. Repeating those reads also consumes database connections and capacity that other requests need. Azure’s caching guidance describes the same scaling effect: serving repeated reads from a cache reduces latency and contention in the original data store.
The usual placement is straightforward: client → application server → cache → database. The application asks the cache for the requested object. If the object is present, the request is a cache hit and the application can return the cached copy. If it is absent, the request is a cache miss and the application must fetch the object from the database before it can respond and make that result available to later readers. Caching stores that copy closer to the reader so repeated fetches are faster and cheaper.
This works because access is not evenly distributed over time. Temporal locality means that after a user profile is read once, it is likely to be read again in the next few seconds. The same pattern makes a product page an obvious candidate: its description may change only once a week, yet customers may read it thousands of times per minute. Sending every one of those reads to the database is wasteful when most callers need the same value.
Design a platform-scale distributed rate limiter55 min
Critique: find the failure modes in a given design45 min
A cache short-circuits repeated reads, while misses continue to the database.
Follow the solid read path to the cache: a hit returns there, while a miss continues to the database before the result comes back through the cache.
Caching’s job is to serve repeated reads from a nearby copy while accepting—and deliberately controlling—the correctness risk that the copy can become stale.
Cache-Aside: The Application Owns the Read Path
Cache-aside makes the application responsible for the read path. The application checks Redis before it queries Postgres; Redis does not know how to query the database or how to transform a database row. A present, unexpired value is a cache hit and can be returned immediately. A missing or expired value is a cache miss that the application must resolve.
The read sequence
For a user-profile service, use user:{id} as the cache key and store the profile as JSON with a TTL of 5 minutes. On a miss, the service reads the profile from Postgres, writes that result into Redis, and returns the same result to the caller. The cache therefore fills lazily: a cold service starts empty, and only profiles that are actually requested enter the cache. You do not need to pre-load every user profile.
text
function get_user_profile(id):
key = "user:" + id
cached_profile = check the cache for key
if cached_profile exists:
return the cached profile
profile = query Postgres for the profile identified by id
if the database read fails:
return the service's database-error result
write the serialized profile to Redis with a 5-minute TTL
if the cache write fails:
return the database result
return the profile
The application handles both branches: the cache serves hits, while Postgres supplies and repopulates misses.
The ordering matters. The service does not return until it has either obtained a value from Redis or obtained one from Postgres. If Postgres fails, there is no authoritative result to cache, so the request follows the service's database-error path and the cache remains unchanged. If Postgres succeeds but the Redis write fails, the database result is still valid for this request; return it and allow a later request to try the cache again. A cache write is an optimization, not a prerequisite for accepting a successful database read.
An eviction or TTL expiration is operationally the same as a miss at this read boundary: the next request pays for the database query and then attempts to repopulate Redis. That makes cache-aside graceful during cold start, but not free. The first request after a miss or eviction has database latency, and a traffic spike can cause many requests to discover the same missing key at once. Without coordination, each request can issue its own SELECT and cache update, creating the conditions for a cache stampede.
CHECK YOUR UNDERSTANDING
A user-profile service receives a read for a key that is not in Redis. What happens, in order?
SHOW ANSWERHIDE ANSWER
The application checks Redis for user:{id}. On a miss, it queries Postgres for the profile. If the database fails, it returns the database error and does not populate the cache. If the database succeeds, it serializes the profile, writes it to Redis with a 5-minute TTL, and returns the profile. A Redis write failure does not change that successful database result; it only means the next request may miss again.
Worked Example: Caching a Frequently Read Product Page
Consider an e-commerce product page whose description changes once a week but is read thousands of times per minute. The description is a strong cache candidate: the source changes infrequently, while many requests ask for the same value. Without caching, every page view repeats the database read. With caching, the service retrieves the description once, stores a copy in Redis, and reuses that copy for subsequent reads. This is the core payoff of caching for repeatedly read data: fewer database operations and a faster read path.
Make the service-time difference explicit
Take a one-minute slice containing 1,000 product-page reads. If every read goes to the database, the aggregate service time is 1,000 reads × 1 ms, or 1,000 ms, at the low end of the database-read range. At 10 ms per read, it is 1,000 × 10 ms, or 10,000 ms. If those reads are cache hits, Redis takes approximately 0.1 ms per hit: 1,000 × 0.1 ms = 100 ms.
python
1000 * 1 # 1000 ms
1000 * 10 # 10000 ms
1000 * 0.1 # 100 ms
The three products compare 1,000 reads against the stated database and Redis service times.
These are aggregate serialized service-time figures, not elapsed wall-clock time: requests can overlap across workers. The comparison still matters because sending all 1,000 reads to the database consumes 1,000–10,000 ms of database service time, whereas serving the same reads from Redis consumes approximately 100 ms of cache service time. The database therefore handles far fewer repeated reads, leaving its capacity for work that cannot be served from the cache. Azure’s caching guidance describes this pattern as most effective when data is read frequently, changes relatively infrequently, and the original store is slower than the cache; this product description has all three properties.
The arithmetic does not make the cached value authoritative. Redis holds a copy of the description captured at some point in the past. Because the product changes weekly, the design can tolerate some staleness only if the product requirement permits it; otherwise, a description update must trigger an explicit refresh or invalidation. The important design decision is not merely whether Redis is faster. It is how long readers may safely receive the previous copy before the service must obtain the changed description.
Choosing Cache-Aside, Write-Through, or Write-Behind
The choice is about which operation you make eager and which cost you accept. Cache-aside is lazy on reads: the application checks the cache, and on a miss it reads the database, populates the cache, and returns the result. Write-through is eager on writes: each write updates the cache and backing store synchronously before the response is sent. Write-behind is also eager at the cache, but defers the database update: it acknowledges after the cache write and flushes the change asynchronously.
Compare the strategies by ordering, exposure, cost, and the workload they fit.
A cached copy can remain stale; cache and DB coordination is the application's responsibility.
Hits are fast; misses pay the DB read and create tail latency. Only read data is cached.
Read-heavy or selectively accessed data; a user profile or product page. Write behavior is handled separately.
Write-through
A write updates cache and DB synchronously; respond after both complete. Reads use cache.
Keeps cache and DB aligned for completed writes, but does not remove every concurrent-read consistency window.
Every write pays both cache and DB cost; added write latency, but cache is ready for subsequent reads.
Read-heavy systems where fresh cached values matter; not attractive for write-heavy services.
Write-behind
Write cache immediately and acknowledge; flush the change to DB asynchronously.
Lower durability: a cache-node failure before the flush can lose acknowledged writes.
Lower write latency and immediate DB relief; asynchronous flushing adds operational coordination.
Tolerant of recent-write loss, such as a leaderboard score update where losing the last half-second is acceptable.
Financial transfer
Write-through to both stores, or do not cache the write path.
Durability matters more than speed; do not accept an acknowledged write that can disappear before persistence.
Accept the synchronous cost when using write-through; skipping the cache avoids making it part of the transfer path.
Financial transfers and other operations where losing or briefly misrepresenting a write is unacceptable.
The comparison people most often get wrong is treating write-through as a universal consistency fix. It is tempting because the write path waits for both destinations, so the cache and database do not drift simply because the application forgot to update one of them. But a concurrent read can still observe the old value momentarily during a write, and a failure between the two updates still needs a recovery policy. The decision flips on the value of durability and freshness versus write volume: use write-through when read-heavy traffic benefits from an immediately updated cache and the synchronous write cost is acceptable; avoid it or choose another path when updates are frequent and every write paying both costs is painful.
CHECK YOUR UNDERSTANDING
A write-heavy service updates a user's location every second, and a teammate suggests write-through caching. What do you say?
SHOW ANSWERHIDE ANSWER
I would not choose write-through automatically. Every update would pay both the cache and database cost synchronously, which is painful for a write-heavy workload. Use it only if the benefit of keeping the cached location current justifies that write latency; otherwise, keep the high-frequency write path out of write-through caching and choose a strategy whose durability and freshness requirements match the location data.
Eviction Is About Space; Invalidation Is About Correctness
Eviction and invalidation remove entries for different reasons. Cache eviction is space management: the cache has a fixed memory budget, and when it is full, a policy decides which entry to drop so another can fit. Cache invalidation is correctness: when the backing record changes, you decide that the cached copy is no longer valid and must be refreshed or evicted. Confusing the two makes incidents difficult to diagnose: memory pressure can remove a perfectly current value, while a cache with free memory can continue serving a value that the database has already changed.
Choose eviction from the access pattern
When the cache reaches its limit, LRU (Least Recently Used) evicts the entry that has not been touched for the longest time. It is a good default for many workloads because recent access is often a useful signal that an entry will be needed again. LFU (Least Frequently Used) evicts the entry accessed the fewest times, preserving items with durable popularity. That makes LFU a better fit when popularity is highly skewed—for example, viral content receives repeated requests while stale content is rarely touched. Redis documents approximate forms of these policies rather than requiring exact tracking; its LFU uses a probabilistic counter, while its LRU is approximate. Memcached uses a slab-based LRU.
Use the removal trigger and the access pattern together when selecting a cache policy.
Policy
What triggers removal
What it preserves
Workload fit
LRU
Entry has been untouched longest
Recently accessed entries
Good default; beware sequential scans that touch every item once and evict hot data
LFU
Entry has the fewest accesses
Persistently popular entries
Highly skewed popularity, such as viral content versus stale content
TTL
Fixed time limit expires
Entries still within their time limit
Simple expiry; pairs well with cache-aside
TTL plus LRU
Expiry or memory pressure
Entries within their TTL and recently used entries
Common practical combination for bounded, time-limited caches
A sequential scan exposes LRU’s blind spot: the scan touches each item once, making the scan look recent even if those entries will not be reused. As the scan fills the cache, it can push out genuinely hot data. LFU can resist that pattern when the hot entries have accumulated higher access counts, although historical popularity can be less useful when demand changes rapidly. In practice, combining a TTL with LRU gives each entry a freshness boundary while still providing a space policy when the memory budget is reached.
Choose invalidation from the data’s staleness tolerance
A TTL is the simplest invalidation strategy: attach a fixed lifetime to the cached copy, and remove it when that lifetime expires regardless of how often the entry was accessed. The trade-off is explicit: with a TTL of N seconds, users may see a value that is stale for up to N seconds, while the cache avoids needing to coordinate every database update. This works well with cache-aside, where the next miss fetches the current value and repopulates the cache.
When that bound is too wide, invalidate from the write path. After a database update, an event can tell the cache to refresh or evict the affected key. This is more precise than waiting for a TTL, but it creates a coordination problem: the database update and cache update are now two operations that must remain aligned. A delayed or failed cache update can still leave a stale copy, so event-driven invalidation exchanges TTL’s predictable staleness for more implementation complexity.
Make the choice per data type rather than applying one freshness rule everywhere. A follower count being two seconds stale may be acceptable; a bank balance being stale is not. Write-through also does not make every read continuously fresh: even when the cache and database are updated as part of the write, a concurrent read can momentarily observe the old value during that write. If the stale window is unacceptable, bypassing the cache or using a stronger coordination design may be preferable.
CHECK YOUR UNDERSTANDING
A critical record changes in the database while its cache entry has a TTL of 10 minutes. What are two ways to prevent users from seeing stale data for the full 10 minutes?
SHOW ANSWERHIDE ANSWER
Use event-driven invalidation on the database update to evict or refresh the affected cache key immediately, accepting the complexity of coordinating the database update with the cache operation. Alternatively, shorten the TTL so the maximum stale window is smaller, accepting more cache misses and database reads. For data such as a bank balance, bypassing the cache for reads may be safer than accepting either stale window.
When Expiration Turns Into a Cache Stampede
A cache-aside design is not automatically safe under concurrency. The failure appears when a popular key expires while many callers are arriving: each request checks the cache, sees a miss, and independently starts the database read needed to regenerate the same value. The cache no longer absorbs traffic; it amplifies it. The operator sees a sudden rise in identical database queries, connection or query saturation, increased latency, and then errors as the database falls over.
Consider a news homepage whose top-10 article list is cached for 60 seconds. At second 60, the key expires. Thousands of users can miss together, and the database may see a sudden 10× spike from requests that all ask for the same top-10 list. A longer TTL changes when the failure happens, not the synchronized shape of the failure. The protection must spread or collapse regeneration work.
A popular-key miss either duplicates database work or funnels it through one regeneration path.
Compare the unprotected branch, where every miss reaches the database, with the mutex branch, where one caller regenerates and the others reread the cache.
Three ways to prevent duplicate regeneration
Probabilistic early expiry starts refresh slightly before the TTL ends. During a refresh window, each request has a chance to become the refresher; the others continue using the current cached value. Randomizing the decision prevents all callers from choosing the same instant. For the news key, beginning the refresh window at second 58 instead of waiting for second 60 spreads regeneration before synchronized expiration. The cost is that some values are refreshed earlier than necessary and the cache may serve the old copy briefly while refresh runs.
A mutex on miss makes regeneration single-file for each key. The first caller that acquires the key's lock reads the database, repopulates the cache, and releases the lock. Concurrent callers wait; after the lock is released, they reread the cache and return the newly populated value instead of issuing duplicate database reads. The lock adds coordination and waiting latency, and a failed lock holder or failed database read needs an explicit error path.
text
function get(key):
value = cache.get(key)
if value exists:
return value
if try_lock(key):
try:
value = database.read(key)
cache.put(key, value)
return value
except database_error:
return regeneration_error
finally:
unlock(key)
// Another caller owns the lock.
wait for the lock to be released, then check the cache again
value = cache.get(key)
if value exists:
return value
return regeneration_error
The lock holder performs one database read; waiters recheck the cache after the lock is released.
The third option is background refresh. A job or request identifies keys that are about to expire and repopulates them before they become misses. Requests continue to use the current cached value while the refresh runs, so expiration does not create a regeneration cliff. This spends work on keys that may not be requested again, and it requires a scheduler or refresh worker plus a way to select the keys that need warming.
Cache invalidation earns its reputation as one of the two genuinely hard problems in computer science because correctness, timing, and coordination interact. A stampede is the timing-and-coordination version of the problem: the cached value becomes unavailable to many callers at once, and the backing store receives work that the cache was meant to prevent. Treat expiration as a concurrent event, not as a single request crossing a timer boundary.
CHECK YOUR UNDERSTANDING
A homepage feed is cached for 60 seconds and expires at peak traffic. What does the stampede look like, and what concrete technique can prevent it?
SHOW ANSWERHIDE ANSWER
At expiration, thousands of concurrent requests miss the same key and all query the database, producing a sudden 10× spike and potentially overwhelming it. Use a mutex so one request regenerates the value while the others wait and reread the cache, or begin probabilistic refresh in a window such as second 58 so regeneration is spread before expiration.
Defending a Caching Design in a System Design Interview
A strong answer starts with the request path, not the product name: client → application → cache → database. The application checks the cache first. A hit returns the cached copy; a miss reaches the database, then the application or cache populates the cache before responding. State the workload before choosing a policy: is it read-heavy, write-heavy, popularity-skewed, or scan-heavy? That classification determines which trade-offs matter.
Make the trade-offs explicit
For lazy read population, choose cache-aside: the application loads only requested data. Choose write-through when the cache and database should be updated synchronously before acknowledging a write, and when write consistency matters more than write latency. Choose write-behind only when deferred persistence is acceptable and losing bounded, recent writes if the cache fails is an acceptable business outcome. It may fit a leaderboard; it is a poor choice for a financial transfer, where you should use write-through or skip the cache.
Then defend the memory policy against the access pattern. Prefer LFU over LRU when a small set of items is persistently and disproportionately popular. Be cautious with LRU under sequential scans: touching many items once can evict data that is otherwise hot. Ask how stale a value may be, and whether the system will use TTL—with an explicitly accepted staleness window—or event-driven invalidation when a database update must remove or refresh the copy promptly.
The operator’s question
The question that separates an operated design from a textbook answer is: what happens when a popular key expires under peak traffic? A good answer describes concurrent misses and names a guard: a mutex so one request regenerates the value, probabilistic early expiry so refresh begins before expiration, or background refresh that pre-warms expiring keys. It also asks what the database can tolerate while that refresh runs.
CHECK YOUR UNDERSTANDING
A service is write-heavy, popularity is highly skewed, and a popular key expires during peak traffic. What should your answer address before naming a cache strategy?
SHOW ANSWERHIDE ANSWER
State the workload first, then choose a write policy that matches write durability and latency requirements, consider LFU for the skewed popularity, define TTL or event-driven invalidation and its accepted staleness window, and protect regeneration with a mutex, probabilistic early expiry, or background refresh.
KEY TAKEAWAYS
Cache-aside lets the application load only requested data, but concurrent misses can still overload the database.
Write-through keeps cache and database updates synchronous, while write-behind trades durability for lower write latency.
Eviction manages limited cache space; invalidation determines whether a cached value is still trustworthy.
TTL, write-triggered invalidation, and bypassing the cache provide different freshness guarantees.
Mutexes, probabilistic early expiry, and background refresh prevent synchronized regeneration from becoming a database spike.