Latency

Software Engineering / 21 Sep 2026 / 7 min read

Hedged Requests Cut Tail Latency with Controlled Duplication

Hedged Requests Cut Tail Latency with Controlled Duplication A service can have a healthy median latency while a small fraction of requests take far longer. Queueing, garbage collection, storage stalls, packet loss, noisy neighbors, or uneven replica load can all stretch the slow end of the distribution. For a request that fans out to several dependencies, one slow branch can dominate the entire response. Hedged requests reduce that exposure by starting a second copy after a short delay. The copies target independent execution paths when possible, and the first valid response wins.

Software Engineering Distributed Systems

Software Engineering / 21 Sep 2026 / 6 min read

Hedged Requests Cut Tail Latency at a Capacity Cost

Hedged Requests Cut Tail Latency at a Capacity Cost A service can have acceptable median latency while a small fraction of requests take much longer. Queueing, a cold cache, runtime pauses, transient packet loss, or a slow storage operation can leave one attempt far behind the normal path. At sufficient fan-out, those rare delays become common at the aggregate request boundary. A hedged request starts a second equivalent attempt after the first has remained incomplete for a selected delay. The caller accepts the first useful result and cancels or discards the other attempt.

Software Engineering Distributed Systems

Software Engineering / 20 Sep 2026 / 5 min read

Hedged Requests Trade Extra Work for Lower Tail Latency

A service can have a healthy median latency and still produce occasional requests that take far longer than the rest. Queueing, a slow replica, connection setup, garbage collection, storage stalls, or transient network delay can leave one attempt behind while equivalent capacity elsewhere remains available. A hedged request limits exposure to that single slow path. The client starts one attempt normally. If it is still pending after a configured delay, the client may start a second equivalent attempt. The first acceptable result is used, and the remaining attempt is cancelled when cancellation is supported.

Software Engineering Distributed Systems

Software Engineering / 20 Sep 2026 / 6 min read

Hedged Requests Cut Tail Latency at a Controlled Cost

Hedged Requests Cut Tail Latency at a Controlled Cost A service can have acceptable median latency and still produce a small set of very slow responses. Queueing, runtime pauses, storage contention, packet loss, or a temporarily busy replica can push individual requests far beyond the common case. Hedged requests address that tail by sending a second copy after the first request has been outstanding for a chosen delay. The caller accepts the first valid response and cancels the remaining attempt. The technique trades a bounded amount of extra work for a chance to escape an unusually slow execution path.

Software Engineering Distributed Systems

Tech / 17 Sep 2026 / 7 min read

PCIe ASPM Trades Link Wake Latency for Idle Power

A PCI Express link does not need to remain at full active power while no packets are moving. Active State Power Management, commonly called ASPM, lets compatible link partners enter lower-power link states during idle periods and return to active operation when traffic resumes. The tradeoff is direct: deeper idle states can save more energy, but leaving them takes time. A system therefore balances link power against the latency added to the next transfer.

PCIe ASPM

Tech / 17 Sep 2026 / 6 min read

NIC Interrupt Coalescing Trades CPU Overhead for Packet Latency

A network interface can receive packets faster than a CPU should service one hardware interrupt per packet. Interrupt coalescing addresses that mismatch by allowing the adapter to group completion notifications and interrupt the CPU less often. The tradeoff is explicit. Fewer interrupts reduce interrupt handling and scheduling pressure, but a packet may wait longer before software is told that receive work is ready. The best setting depends on packet rate, latency targets, CPU capacity, and the adapter’s coalescing controls.

Networking NIC

Tech / 16 Sep 2026 / 6 min read

TCP Window Scaling Expands the Receive Window for Fast Long Paths

TCP flow control limits how much data a sender may have outstanding according to the receiving endpoint’s available buffer space. The receiver advertises that limit in the TCP Window field so the sender does not deliver data faster than the receiving stack can accept it. The Window field in the TCP header is 16 bits wide. Without an extension, its largest value is 65,535 bytes. That ceiling can be too small on a path that carries data quickly but has a substantial round-trip time.

Networking TCP

Tech / 16 Sep 2026 / 5 min read

PCIe Active State Power Management Trades Idle Power for Exit Latency

A PCI Express link does not need to stay at its fully active electrical state while no traffic is moving. Active State Power Management, commonly shortened to ASPM, lets compatible link partners place the link into lower-power states during idle periods. The practical tradeoff is simple: deeper idle states can save more power, but returning to active operation takes time. That exit delay becomes part of the latency seen when new traffic arrives.

PCIe Power Management

Tech / 16 Sep 2026 / 5 min read

Network Interrupt Coalescing Batches Packets Before CPU Notification

A network interface can receive packets much faster than a CPU should be interrupted for each individual arrival. At high packet rates, one hardware interrupt per packet would consume substantial processor time in interrupt entry, scheduling, driver work, and return paths. Interrupt coalescing changes that pattern. The adapter waits for a small interval, a packet count, or another implementation-specific threshold before notifying the CPU. Several packet arrivals can then be handled from one notification.

Networking NIC

Tech / 16 Sep 2026 / 5 min read

Network Interrupt Coalescing Batches Packet Notifications

A network interface can receive packets far faster than a processor should handle individual hardware interrupts. If every packet immediately triggered an interrupt, high packet rates could consume substantial CPU time in interrupt handling and context transitions. Interrupt coalescing changes that pattern. The network adapter waits for several packets, a short timer, or another configured threshold before notifying the CPU. One interrupt can then cover multiple received packets. The tradeoff is direct: fewer interrupts reduce per-packet CPU overhead, while waiting to form a batch can add latency.

Networking Ethernet

Tech / 16 Sep 2026 / 5 min read

Nagle Algorithm Batches Small TCP Writes

TCP applications can issue writes much smaller than the network’s maximum segment size. Sending every tiny write as a separate segment can consume disproportionate header and processing overhead. The Nagle algorithm limits that pattern by allowing one small segment to remain in flight while later small writes wait for an acknowledgment or enough queued data to form a larger segment. This behavior reduces streams of tiny TCP segments. It can also add latency when an application expects each small write to leave immediately.

TCP Networking

Tech / 15 Sep 2026 / 5 min read

TCP Nagle Algorithm Batches Small Writes

Applications can hand TCP data in pieces much smaller than the network’s practical segment size. A terminal session, control protocol, or interactive service might produce only a few bytes at a time. Sending every tiny write immediately can create a stream of packets whose headers are much larger than their payloads. The Nagle algorithm reduces that pattern by limiting how aggressively a TCP sender emits new small segments while earlier data is still awaiting acknowledgment.

TCP Networking

Tech / 15 Sep 2026 / 6 min read

Bluetooth Audio Latency Extends Beyond Codec Delay

Wireless headphones can sound clean yet place a game effect or instrument monitor noticeably behind the action on screen. The codec contributes to that delay, but it is only one stage in a longer path from application audio to the listener. A Bluetooth audio connection has buffers on both sides of the radio link. Audio may also pass through resampling, encoding, packet scheduling, decoding, and the receiver’s playback queue. Each stage can add time, and some stages deliberately hold data to keep playback stable when packet delivery is not perfectly regular.

Bluetooth Audio

Tech / 14 Sep 2026 / 6 min read

TCP Fast Open Sends Data During Connection Setup

A conventional TCP connection separates setup from application traffic. The client sends a SYN, the server replies with SYN-ACK, and the client completes the three-way handshake with an ACK. Application data normally follows after that exchange has established the connection. For short transactions, that setup time can be a meaningful part of the total delay. A request may contain only a few hundred bytes, yet it still waits for a network round trip before the server can receive it through the established connection.

TCP Networking

Tech / 09 Sep 2026 / 8 min read

Why Bluetooth Audio Can Lag Behind Video

Bluetooth headphones can sound perfectly normal with music yet make a video feel wrong when a speaker’s lips move before you hear the words. The same delay can be more obvious in a game, where a sound may arrive after the action that caused it. This happens because wireless audio is not sent directly from an app to your ears in one instant. The sound passes through several processing and transmission stages, and each can add a small amount of time. Some devices can hide much of that delay during ordinary video playback, but they cannot compensate equally well in every situation.

Bluetooth Wireless Audio

Tech / 04 Sep 2026 / 8 min read

What Internet Latency Means and Why Speed Is Not the Whole Story

A fast internet plan can download a large file quickly and still make a video call feel awkward. A game can react late even when a speed test reports plenty of bandwidth. Sometimes a website pauses before it starts loading, then finishes quickly once it gets going. These experiences make more sense when you separate speed from latency. Internet speed usually describes how much data a connection can move over time. Latency describes delay: how long data takes to travel from one point to another and, depending on the measurement, back again.

Internet Latency