10.2 CUBIC, BBR, pacing, and active queue management
During the rainy season, queues ahead of the bottleneck link grow longer. A network operator attempts to resolve this by increasing the sender's transmission rate, only to find that latency continues to rise.
The congestion-control algorithm determines how a sender updates its sending model, window, or rate based on feedback. The queue discipline dictates how packets are queued, marked, or dropped at the bottleneck. Pacing controls how the sender distributes packet transmission over time. These three layers interact dynamically, neither can be treated as a standalone fix with a simple "switch to a different TCP algorithm" solution.
1. CUBIC is still a loss-based controller, but window growth no longer advances linearly based solely on ACK count
RFC 9438 defines the standard form of CUBIC. Before a loss event, the window is recorded as W_max. After loss, the window is reduced multiplicatively, and thereafter, the window target primarily follows a cubic curve based on elapsed time since the start of the congestion epoch, centered around W_max:
W_cubic(t) = C * (t - K)^3 + W_maxK introduces a gentle flattening region near the previous W_max, resulting in:
- Faster recovery when significantly below
W_max; - Conservative probing when approaching the previous operating point;
- Continued exploration of new capacity once exceeding the old
W_max.
CUBIC also includes Reno-friendly regions, fast convergence, application-limited behavior, and handling of overflow/precision constraints. A Python class that only implements the cubic formula using max(current, target) is not RFC-compliant CUBIC.
The RFC specifies a multiplicative decrease factor that must be clearly defined: beta_cubic represents a retention ratio (standardly recommended at 0.7). It is incorrect to treat beta=0.3 as a "reduction amount" in one context and cwnd *= beta as a separate value in another.
CUBIC reduces RTT bias, but does not guarantee absolute fairness among flows with different RTTs. Factors such as ACK timing, loss probability, pacing, queue behavior, Reno coexistence, and implementation details still influence the outcome. CUBIC is widely used in many Linux systems, but the claim that "Linux will always default to CUBIC in any distribution by today" is not a reliable guarantee, current host configuration must be verified.
2. BBR Modeling: Delivery Rate and Minimum RTT
The BBR family estimates:
- Bottleneck bandwidth (BtlBw): modeled from recent delivery-rate samples, representing the bottleneck capacity;
- Round-trip propagation time (RTprop): modeled as the minimum RTT within a time window, aiming to reduce the influence of queuing components.
These models are used to set the pacing rate and target in-flight/cwnd, and they detect bandwidth changes through gain cycling and probing. BtlBw * RTprop provides an estimate of BDP, but BBR does not simply stay fixed at this value. It requires handling of ACK aggregation, policer logic, loss/ECN events, startup/drain phases, probe scheduling, and application-limited sample processing.
The statement "BBR doesn't see loss" is inaccurate. BBR does not treat the first loss event as a primary operating point signal like Reno or CUBIC. However, implementations and versions may use loss, ECN, and in-flight bounds to manage excessive congestion. The behavior of BBR v1, v2, and v3 differs significantly, and thus cannot be represented by a simplistic rule (such as "after every 8 RTTs, increase by 1.25x then decrease by 0.75x") as a general model for all versions.
Model-based does not imply automatic fairness or absence of queues. When BBR shares the same drop-tail bottleneck with loss-based flows, or when RTT differences or policer constraints are present, the fairness and queueing behavior must be empirically validated. These aspects remain key focus areas for future improvements.
3. Pacing Reduce bursts, but don't create capacity
A window controller can allow all the data in an entire ACK burst to be sent immediately. Pacing spreads these packets out over time at a controlled rate, reducing microbursts, queue spikes, and loss synchronization.
Setting the pacing rate too high still results in queuing, while setting it too low leads to underutilization. The actual time intervals between packets on the wire are influenced by timer granularity, NIC offload, qdisc, and application burst behavior. TSO (TCP Segmentation Offload) causes the kernel to deliver large segments to the NIC; in this case, the NIC's segmentation and pacing capabilities determine whether the traffic reverts to a burst on the wire.
4. Bufferbloat Means Excessive Queuing Delay, Even with Moderately Sized Buffers
Buffers can absorb short bursts of traffic, but persistently full queues raise the round-trip time (RTT) from its propagation baseline to the time it takes to drain the queue. Loss-based senders must wait for a drop-tail queue to fill before receiving a signal, causing latency-sensitive traffic to experience hundreds of milliseconds of additional delay, even when throughput remains high.
Bufferbloat is evaluated using under-load latency metrics, rather than the configured buffer size alone. When link rates drop, the drain time of the same 1 MB queue increases significantly. This makes variable-rate links (like Wi-Fi or cellular) especially problematic.
queueing delay ≈ queued bytes / bottleneck service rateThis is a rough approximation; actual sojourn time can vary due to scheduling policies, packet size, wireless retransmissions, and rate changes.
5. AQM before queue overflow: mark or drop packets
Active Queue Management (AQM) does not wait until the buffer is full before reacting. Instead, it proactively ECN-marks or drops packets based on queue state, sending congestion signals to the sender before buffer overflow occurs.
CoDel identifies persistent queues by packet sojourn time (the duration a packet spends in the queue) rather than just queue byte count. It relies on parameters or assumptions such as target, interval, MTU, and packet-mode. Although its goal is to reduce the need for manual tuning of link bandwidth or RTT, the claim that "CoDel has no configuration parameters" is incorrect.
FQ-CoDel first partitions flows into queues and schedules them, then applies CoDel-style AQM to each individual queue. This prevents a bulk flow from completely saturating the queue and starving latency-sensitive, sparse flows. Flow hashing can result in collisions, and issues like non-responsive flows and tunnel aggregation must also be considered. CAKE further integrates shaping, differentiating services (diffserv), host isolation, and overhead compensation into a unified framework.
AQM must be deployed in a true bottleneck queue (or one that has been artificially created through shaping) to be effective. If a hidden queue behind the ISP modem is slower than your qdisc, simply switching the qdisc on the LAN interface may not resolve the issue.
6. Observe Before Switching the Algorithm
# Check host Available/Default congestion-control algorithm
sysctl net.ipv4.tcp_available_congestion_control
sysctl net.ipv4.tcp_congestion_control
# Check connection RTT, cwnd, retransmission, pacing/delivery-rate Wait
ss -tin
# Check queue discipline and drop/mark/backlog counter
tc -s qdisc showThe field varies by kernel/version/algorithm. ss When seeing cwnd:100, one must also understand MSS, bytes-in-flight, application-limited state, pacing rate, receive window (rwnd), and RTT, neither can throughput be inferred from a single cwnd value.
To fairly compare CUBIC/BBR/AQM, experiments must control:
- bottleneck rate, base RTT, queue size, and qdisc;
- loss, reordering, and ECN policies;
- number of flows, flow start time, and RTT asymmetry;
- sender and receiver kernel versions and offload configurations;
- measurement duration and warm-up period;
- throughput, p50/p95/p99 latency, loss and retransmission rates, fairness, and CPU utilization.
A single file download completion time measurement is insufficient to draw a general conclusion that "algorithm A is faster than B." Do not directly perform global toggling of sysctl on a shared production host for testing; instead, use network namespaces/testbeds or per-socket configurations, and always have a rollback plan ready.
7. A BDP/queue-delay Calculator
def bdp_bytes(rate_mbps: float, base_rtt_ms: float) -> float:
if rate_mbps <= 0 or base_rtt_ms <= 0:
raise ValueError("rate and RTT must be positive")
return rate_mbps * 1_000_000 / 8 * base_rtt_ms / 1_000
def queue_delay_ms(queued_bytes: int, rate_mbps: float) -> float:
if queued_bytes < 0 or rate_mbps <= 0:
raise ValueError("invalid queue/rate")
return queued_bytes * 8 / (rate_mbps * 1_000_000) * 1_000
assert bdp_bytes(100, 20) == 250_000
assert queue_delay_ms(1_000_000, 20) == 400This is only a fluid-model estimate and not a precise simulator of packet schedulers or wireless links. Its purpose is to check order-of-magnitude: simply queuing 1 MB of data at a 20 Mbps bottleneck takes about 400 ms to drain.
8. Acceptance Questions
- What aspects of the network does CUBIC's
W_max,K, and cubic epoch specifically control? - Why can't we say BBR "completely ignores loss"?
- How does pacing reduce bursts, yet fail to create bottleneck capacity?
- What are the key differences between CoDel and FQ-CoDel?
- Why must AQM be deployed at a genuine, controllable bottleneck to effectively manage queue delay?