10.2 CUBIC, BBR, Pacing and Active Queue Management
Downloads can remain fast while interactive requests become slow because a bottleneck queue is growing. Congestion control decides how much to send, pacing decides when to send, and queue management decides how to schedule, mark, or drop packets. This lesson separates those responsibilities, compares CUBIC, BBR, and AQM, and checks queue-delay estimates numerically.
1. CUBIC grows a cubic target over time
RFC 9438 defines Standards Track CUBIC. Usually classified as loss-based, it also responds to congestion signals such as ECN. Before a loss event, the window is recorded as W_max. After loss, the window is reduced multiplicatively, and thereafter, the window target primarily follows a cubic curve based on elapsed time since the start of the congestion epoch, centered around W_max:
W_cubic(t) = C * (t - K)^3 + W_maxC determines the curve’s scale. K is the time it would take to return to W_max without another congestion event. The curve produces:
- Faster recovery when significantly below
W_max; - Conservative probing when approaching the previous operating point;
- Continued exploration of new capacity once exceeding the old
W_max.
CUBIC also includes Reno-friendly regions, fast convergence, application-limited behavior, and handling of overflow/precision constraints. A Python class that only implements the cubic formula using max(current, target) is not RFC-compliant CUBIC.
The RFC recommends a multiplicative decrease factor beta_cubic of 0.7. It is the fraction of the window retained after reduction. A 100-SMSS window multiplied by 0.7 retains 70 SMSS, not 30.
CUBIC reduces RTT bias, but does not guarantee absolute fairness among flows with different RTTs. ACK timing, loss, pacing, queues, coexistence, and implementation still affect the outcome. RFC 9438 CUBIC is available on many Linux systems, but check the host default and each connection’s actual algorithm instead of inferring them from the distribution name.
2. BBR Modeling: Delivery Rate and Minimum RTT
The BBR family estimates:
- Bottleneck bandwidth (BtlBw): modeled from recent delivery-rate samples, representing the bottleneck capacity;
- Round-trip propagation time (RTprop): modeled as the minimum RTT within a time window, aiming to reduce the influence of queuing components.
These models are used to set the pacing rate and target in-flight/cwnd, and they detect bandwidth changes through gain cycling and probing. BtlBw * RTprop provides an estimate of BDP, but BBR does not simply stay fixed at this value. Its model also needs startup, drain, bandwidth and RTT probing, ACK aggregation, application-limited samples, and offload-batch adjustments.
The statement "BBR doesn't see loss" is inaccurate. BBR does not treat the first loss event as a primary operating point signal like Reno or CUBIC. Different versions use loss rates and in-flight bounds to constrain excessive probing. The behavior of BBR v1, v2, and v3 differs significantly, and thus cannot be represented by a simplistic rule (such as "after every 8 RTTs, increase by 1.25x then decrease by 0.75x") as a general model for all versions.
The version referenced here is working-group draft draft-ietf-ccwg-bbr-06, describing BBRv3 as experimental work. It specifies no dedicated ECN response algorithm. A connection claiming ECN support through ECT must nevertheless treat CE marks as congestion. BBR draft §3.7
Model-based does not imply automatic fairness or absence of queues. When BBR shares the same drop-tail bottleneck with loss-based flows, or when RTT differences or policer constraints are present, the fairness and queueing behavior must be empirically validated. Record the kernel and BBR implementation version; “using BBR” alone cannot reproduce an experiment.
3. Pacing spreads bursts without adding bandwidth
A window controller can allow all the data in an entire ACK burst to be sent immediately. Pacing spreads these packets out over time at a controlled rate, reducing microbursts, queue spikes, and loss synchronization.
Setting the pacing rate too high still results in queuing, while setting it too low leads to underutilization. The actual time intervals between packets on the wire are influenced by timer granularity, NIC offload, qdisc, and application burst behavior. TSO (TCP Segmentation Offload) causes the kernel to deliver large segments to the NIC; in this case, the NIC's segmentation and pacing capabilities determine whether the traffic reverts to a burst on the wire.
4. Bufferbloat Means Excessive Queuing Delay, Even with Moderately Sized Buffers
Buffers can absorb short bursts, but persistent queues add delay to the base RTT. If senders receive feedback only when a drop-tail queue fills, interactive traffic may wait hundreds of extra milliseconds while throughput remains high.
Bufferbloat is evaluated using under-load latency metrics, rather than the configured buffer size alone. When link rates drop, the drain time of the same 1 MB queue increases significantly. This makes variable-rate links (like Wi-Fi or cellular) especially problematic.
queueing delay ≈ queued bytes / bottleneck service rateThis is a rough approximation; actual sojourn time can vary due to scheduling policies, packet size, wireless retransmissions, and rate changes.
5. AQM before queue overflow: mark or drop packets
Active Queue Management (AQM) does not wait until the buffer is full before reacting. Instead, it proactively ECN-marks or drops packets based on queue state, sending congestion signals to the sender before buffer overflow occurs.
CoDel identifies persistent queues by packet sojourn time (the duration a packet spends in the queue) rather than just queue byte count. It has parameters such as target and interval, plus implementation constraints. Although its goal is to reduce the need for manual tuning of link bandwidth or RTT, it still has configuration parameters. RFC 8289
FQ-CoDel first partitions flows into queues and schedules them, then applies CoDel-style AQM to each individual queue. This prevents a bulk flow from completely saturating the queue and starving latency-sensitive, sparse flows. Flow hashing can result in collisions, and non-responsive flows and tunnel aggregation still need consideration. RFC 8290 CAKE further integrates shaping, DiffServ, host isolation, and link-overhead compensation. iproute2 CAKE manual
AQM must be deployed in a true bottleneck queue (or one that has been artificially created through shaping) to be effective. If a hidden queue behind the ISP modem is slower than your qdisc, simply switching the qdisc on the LAN interface may not resolve the issue.
6. Observe before comparing algorithms
These read-only Linux checks need procps and iproute2. They change neither algorithms nor queues:
# Show available and default congestion-control algorithms
sysctl net.ipv4.tcp_available_congestion_control
sysctl net.ipv4.tcp_congestion_control
# Show connection RTT, cwnd, retransmissions, and sending rates
ss -tin
# Show queue disciplines and drop/mark/backlog counters
tc -s qdisc showss generally reports cwnd:100 in segments. Interpret it alongside MSS, in-flight data, application-limited state, pacing, rwnd, and RTT. Fields vary by kernel and algorithm. The available-algorithm list can describe registered algorithms without listing every loadable module.
To fairly compare CUBIC/BBR/AQM, experiments must control:
- bottleneck rate, base RTT, queue size, and qdisc;
- loss, reordering, and ECN policies;
- number of flows, flow start time, and RTT asymmetry;
- sender and receiver kernel versions and offload configurations;
- measurement duration and warm-up period;
- throughput, p50/p95/p99 latency, loss and retransmission rates, fairness, and CPU utilization.
A single file download completion time measurement is insufficient to draw a general conclusion that "algorithm A is faster than B." Do not directly perform global toggling of sysctl on a shared production host for testing; instead, use network namespaces/testbeds or per-socket configurations, and always have a rollback plan ready.
7. Calculate BDP and queue drain time
This Python 3 example uses decimal Mbps and MB with finite numeric inputs. Save it as queue_math.py and run python3 queue_math.py. It performs arithmetic without configuring the network:
def bdp_bytes(rate_mbps: float, base_rtt_ms: float) -> float:
if rate_mbps <= 0 or base_rtt_ms <= 0:
raise ValueError("rate and RTT must be positive")
return rate_mbps * 1_000_000 / 8 * base_rtt_ms / 1_000
def queue_delay_ms(queued_bytes: int, rate_mbps: float) -> float:
if queued_bytes < 0 or rate_mbps <= 0:
raise ValueError("invalid queue/rate")
return queued_bytes * 8 / (rate_mbps * 1_000_000) * 1_000
assert bdp_bytes(100, 20) == 250_000
assert queue_delay_ms(1_000_000, 20) == 400
print(bdp_bytes(100, 20))
print(queue_delay_ms(1_000_000, 20))Expected output:
250000.0
400.0A 100-Mbps path with a 20-ms base RTT has a BDP of about 250000 bytes. Draining 1 MB at a 20-Mbps bottleneck takes about 400 ms. This fluid estimate does not simulate packet scheduling or wireless behavior. A busy link can still leave its users waiting.
8. Acceptance Questions
- How do
W_max, C, and K affect the CUBIC curve? - Why does BBR not completely ignore loss, and why must its implementation version be recorded?
- How does pacing reduce bursts, yet fail to create bottleneck capacity?
- What are the key differences between CoDel and FQ-CoDel?
- Why must AQM be deployed at a genuine, controllable bottleneck to effectively manage queue delay?