Token Buckets Preserve Burst Capacity Without Removing Rate Bounds
Token Buckets Preserve Burst Capacity Without Removing Rate Bounds A fixed requests-per-second ceiling treats a brief spike and a sustained flood as the same event. That can be too rigid for services whose callers naturally arrive in clusters. A token bucket separates two constraints: the long-run admission rate and the amount of burst traffic the service is willing to absorb. The model has two parameters. The bucket capacity B is the maximum number of tokens that can accumulate. The refill rate r adds tokens per unit of time, up to B. A request consumes tokens according to its configured cost. If enough tokens are present, the request proceeds; otherwise it is rejected, delayed, or handled by another explicit policy.