Cache Stampede
1. The Problem
You've got an ASP.NET Core ProductService that uses a distributed cache such as Azure Managed Redis.
Product information is cached because thousands of customers request the same products.
Normally, the flow is simple:
Customer | v ASP.NET Core API | v Cache | v Product
The problem starts when a popular cache entry expires.
Suppose Product 123 is requested by 5,000 customers.
The cached value expires:
Product 123
|
v
Cache entry expires
|
v
Cache MISS
Now thousands of requests arrive at almost the same time.
And they all discover that the data isn't in the cache.
So they all go to the database.
5,000 requests
|
+----------+----------+
| | |
v v v
Request Request Request
| | |
+----------+----------+
|
v
Database
The cache was supposed to protect the database.
Instead, one cache expiration has suddenly caused a huge burst of database traffic.
This is a Cache Stampede.
2. What's Actually Going Wrong?
The normal Cache-Aside flow is:
Cache HIT | v Return data
When the cache expires:
Cache MISS | v Database | v Populate cache
The problem is what happens when many requests experience the miss at the same time.
Instead of:
Request A --> Cache MISS --> Database
|
v
Populate cache
you get:
Request A --> Cache MISS --> Database Request B --> Cache MISS --> Database Request C --> Cache MISS --> Database Request D --> Cache MISS --> Database Request E --> Cache MISS --> Database ...
Every request thinks:
“The cache doesn't have it. I'll get it from the database.”
The result is a sudden spike in database load.
The cache didn't fail.
The problem is that too many requests are trying to refill the same cache entry at the same time.
3. The Concept
Cache Stampede is a situation where many requests simultaneously miss the cache for the same data and all try to retrieve that data from the underlying data source.
It can happen when:
- A popular cache entry expires.
- A cache is suddenly cleared.
- A large number of entries expire together.
- The cache becomes temporarily unavailable.
The dangerous part is that the requests are often for the same data.
Cache Entry
Product 123
|
v
EXPIRES
|
+---------+---------+
| | |
v v v
MISS MISS MISS
| | |
+---------+---------+
|
v
DATABASE
The database now receives many requests for something that could have been fetched once and then cached again.
4. How Do You Fix It?
The basic idea is:
When many requests miss the same cache entry, don't let all of them rebuild it independently.
One common approach is to allow one request to refresh the cache, while the others wait for or reuse that result.
Conceptually:
Cache MISS
|
v
+----------------+
| Who refreshes |
| the cache? |
+-------+--------+
|
+------+------+
| |
First Others
request wait
| |
v |
Database |
| |
v |
Update cache |
| |
+------+------+
|
v
Return data
The exact mechanism can vary.
For example, an application can use a distributed lock so that only one application instance is responsible for rebuilding a particular cache entry.
This is especially important when your ASP.NET Core application is running on multiple instances.
Without coordination:
Instance A --> Database Instance B --> Database Instance C --> Database Instance D --> Database
With coordination:
Instance A --> Lock --> Database --> Cache Instance B --> Wait Instance C --> Wait Instance D --> Wait
Once the cache is populated, subsequent requests can use the cached value.
5. Another Useful Technique: Stagger Expiration
Imagine you have thousands of cache entries and they all expire at exactly the same time.
You can create another problem:
10:00:00
|
+---- Cache A expires
+---- Cache B expires
+---- Cache C expires
+---- Cache D expires
+---- Cache E expires
|
v
Huge database load
A common technique is to add some randomness to expiration times.
Instead of every item expiring after exactly 10 minutes:
Product A -> 10 minutes Product B -> 10 minutes Product C -> 10 minutes Product D -> 10 minutes
you might use slightly different expiration periods:
Product A -> 9m 42s Product B -> 10m 11s Product C -> 9m 57s Product D -> 10m 23s
This spreads cache misses over time instead of creating one large spike.
6. Visualize It in Everyday Terms
Imagine a popular bakery that sells one particular type of bread.
The bakery keeps today's availability on a board outside.
Customers normally check the board before going inside.
That's the cache.
Now imagine the board suddenly gets wiped clean.
Hundreds of customers arrive at exactly the same time.
Everyone walks inside asking:
“Do you have this bread?”
The bakery staff now has hundreds of people asking the same question.
That's a Cache Stampede.
A better approach is for one staff member to check the inventory and update the board.
Everyone else can then use the updated information.
In this analogy:
- Bakery = Application
- Availability board = Cache
- Inventory system = Database
- Customers = Requests
- Board being wiped = Cache expiration
- Hundreds of customers asking at once = Cache Stampede
- One person refreshing the board = Coordinated cache refresh
The lesson:
When a popular cached value disappears, don't let every request rush to the source at the same time.
7. What Should You Remember?
Think about Cache Stampede as:
CACHE EXPIRES
|
v
CACHE MISS
|
+----------+----------+
| | |
v v v
Request Request Request
| | |
+----------+----------+
|
v
DATABASE OVERLOAD
The key points:
- A Cache Stampede happens when many requests miss the same cache entry at the same time.
- Those requests can all hit the database simultaneously.
- Popular data is especially dangerous because many users may request it together.
- Coordinate cache refreshes so one request can rebuild the entry instead of everyone doing it.
- Staggering expiration times can reduce synchronized cache misses.
The mental model is simple:
A cache miss is normal. Thousands of requests reacting to the same cache miss at once is the problem.
8. .NET + Azure Context
From a .NET engineer's perspective, don't just ask:
“Are we using a cache?”
Also ask:
“What happens when a popular cache entry expires and hundreds of requests miss it at the same time?”
That's the question that leads you to Cache Stampede.
