Concurrency and limits

Control how often a function runs, how many calls run at once, and how you run many awaitables or blocking calls without hand-writing the plumbing. Everything here is built for both threads and asyncio.

• Rate limits

  ---

  rate_limit and RateLimiter: at most N calls per time window, reject or wait.

• Throttle and debounce

  ---

  Collapse bursts of calls: run at most once per interval, or once after the calls stop.

• Bulkheads

  ---

  Bulkhead and limit_concurrency: cap concurrent calls and bound the queue.

• Async helpers

  ---

  gather_limit, map_async, amap_iter, race and run_sync.

• run_in_thread

  ---

  Start a blocking call in a thread and get a Future back.

• Periodic tasks

  ---

  periodic: run a function every N seconds in the background, with clean shutdown.

Which one?

You want to...	Use
Allow at most N calls per time window (an API quota)	Rate limits
Share one quota between several functions	RateLimiter
Skip calls that arrive too soon after the last one	throttle
Run once, after a burst of calls has stopped	debounce
Cap how many calls run at the same time	Bulkheads
Run many awaitables or items with a bound	Async helpers
Take the first successful result of several awaitables	race
Call blocking code without waiting for it	run_in_thread
Run something every N seconds in the background	periodic
Make identical concurrent calls share one execution	singleflight

A rate limit counts calls over time; a bulkhead counts calls in flight. Use both when a dependency has a quota and a connection limit.

See also

• Timeout and Hedge for deadlines and parallel retries.
• Testing for fast_mode(), fake_clock() and disable_rate_limits().