Exponential Backoff with Jitter
Problem
Thundering Herd
- Your Notification service goes down briefly, then comes back online within seconds.
- Every caller is waiting with the same fixed retry delay, so they all retry at the exact same moment.
- Notification gets hit with the full flood at once, the instant it's back, and goes down again.
Everyone retrying at the same instant recreates the exact spike that just went down. This is the thundering herd problem.
Solution
Exponential Backoff with Jitter
- Each retry waits longer than the last: 1 second, then 2, then 4, and so on.
- A small random amount is added to each wait, so different callers don't land on the exact same instant.
- Retries arrive spread out over time instead of all at once.
Space retries out, and randomize them, so recovery doesn't get flooded by the very callers waiting for it. That combination is exponential backoff with jitter.
Analogy
After a neighborhood-wide blackout, the power company doesn't switch every house back on at the same instant.
Every home's air conditioner and appliances kicking in together would spike demand right back to outage levels. Homes are reconnected a few at a time instead.
Exponential backoff with jitter is that staggered reconnection, for retries instead of homes.
