Cache Stampede
Problem
Cache Stampede
- An ASP.NET Core API caches a price in Redis. The cache entry expires, and 200 requests hit the API in the same second.
- All 200 miss the cache at the same moment and go straight to the database.
- The database runs the same expensive query 200 times, for an answer that is identical every single time.
The database pays for one cache miss, 200 times over. That pile-up is a cache stampede.
Solution
- Let one request through to run the query and refill the cache.
- Hold the other 199 until that refill finishes, then hand them the same result.
- The database sees one query; some requests just wait a few milliseconds longer.
A cache miss is one request's job. Don't let two hundred requests volunteer for it at once.
Analogy
A ticket counter reopens after a short break. The whole waiting crowd rushes the window at once, and the clerk is overwhelmed.
With a token queue, one person is served first and the rest wait their turn. The clerk stays calm, and the line still moves.
A cache needs that token queue when a hot key expires, or a cache stampede is exactly what it gets.
