What is Token Bucket?

Understanding Token Bucket

A token bucket is a rate-limiting algorithm used to control the rate of traffic or requests while permitting controlled bursts. A bucket holds tokens that are added at a steady configured rate up to a maximum capacity; each unit of traffic must consume a token to proceed. If tokens are available, traffic passes; if the bucket is empty, traffic is delayed or dropped.

The mechanism is defined by two parameters: the token refill rate, which sets the sustained average throughput, and the bucket depth (burst size), which determines how much traffic can pass instantaneously when tokens have accumulated during idle periods. Because unused tokens build up to the bucket's capacity, the algorithm tolerates short bursts above the average rate while still enforcing the long-term limit. This distinguishes it from the leaky bucket algorithm, which smooths output to a strictly constant rate and does not allow bursts.

This matters for both performance and security. In Quality of Service, token-bucket policing on routers enforces traffic contracts and protects links from congestion. As a security control, the same algorithm underpins rate limiting on APIs, login endpoints, and services to throttle abuse, blunt brute-force and credential-stuffing attacks, and absorb application-layer denial-of-service attempts by capping how many requests a client can make over time while still accommodating legitimate bursts.

For example, an API gateway limits each client to 100 requests per second with a burst allowance of 200. Tokens refill at 100 per second into a bucket holding up to 200. A client that has been idle can briefly send 200 requests by spending accumulated tokens, then settles to the sustained 100-per-second rate as the bucket refills. An attacker attempting to flood the endpoint exhausts the tokens almost immediately, and excess requests are rejected with a rate-limit response, protecting the backend.