Resilience¶
Calls to networks, disks and other services fail, stall or get slow. These decorators decide what happens next: try again, stop calling a broken dependency, stop waiting, race a backup request, or return a sensible default.
-
Call again when it raises or returns a bad value, with backoff, jitter,
Retry-After, a time budget and a shared retry budget. -
Reject calls at once while a dependency is down, then probe for recovery.
-
Limit one call or a block of code, or give a whole request one shared budget.
-
Return a default value, or
None, instead of raising. -
Start a backup attempt when a call is slow and take the first success.
-
Bundle several of these into one named, reusable decorator, in the right order.
Which one?¶
| Your problem | Use |
|---|---|
| Errors that usually go away if you try again (dropped connection, 503) | retry |
| A dependency that is down and keeps failing | circuit_breaker |
| A call that can hang or run too long | timeout / deadline |
| Rare slow calls (tail latency) on a replicated service | hedge |
| A failure that should become a default value | fallback / suppress |
| All of the above on one call, in the right order | policy |
These combine well, and order matters: the usual stack is fallback outside, then circuit breaker, then retry, then timeout inside. Policies apply that order for you, and Stacking explains why.