Skip to content

Archive

System Design

3 articles
Software Engineering 12 Sep 2026 11 min read

Load Shedding: Reject Work Before Overload Spreads

Load Shedding: Reject Work Before Overload Spreads A service has finite capacity. When offered work exceeds that capacity for long enough, accepting every request can make the service less useful rather than more useful. Queues grow, deadlines expire, memory pressure rises, dependencies receive more traffic, and successful throughput can fall. Load shedding is the deliberate rejection of work that the system cannot serve within an acceptable budget. The goal is not to maximize the number of requests admitted. The goal is to preserve useful service during overload.

Software Engineering Reliability
Software Engineering 11 Sep 2026 13 min read

Concurrency Limits: Bound In-Flight Work

Concurrency Limits: Bound In-Flight Work A service can receive traffic at an acceptable average rate and still collapse because too many operations overlap. The issue is not only how many requests arrive per second. It is also how many requests are active at the same time. A concurrency limit places a cap on active work. When all slots are occupied, additional work must wait, fail fast, or take another explicit path. This simple control can protect database connections, CPU-heavy routines, remote dependencies, worker capacity, and memory that grows with each active operation.

Software Engineering Reliability
Software Engineering 10 Sep 2026 10 min read

Token Bucket Rate Limiting for Controlled Bursts

Token Bucket Rate Limiting for Controlled Bursts A service may handle 100 requests per second comfortably on average while still needing to accept a short burst of 300 requests after a client reconnects. A rigid per-second limit treats those situations as the same problem: once the current window is full, otherwise acceptable work is rejected. Token bucket rate limiting gives you a more useful control. It separates two decisions: how quickly permission to do work is replenished and how much permission may accumulate for a burst. Once you understand those two numbers, you can reason about the limiter without depending on a particular library or platform.

Software Engineering Reliability