10.1 TCP Congestion Window, Recovery, and ECN
Free receiver buffer does not imply spare network capacity. Flow control protects the receiver, while congestion control limits the sender's in-flight load on the network path. Both mechanisms restrict transmission, but they derive feedback from different sources: the receiver advertises its buffer capacity via the advertised receive window (rwnd), while the sender uses signals such as ACKs, packet loss, ECN marks, RTT, and delivery rate to maintain its congestion-control state. At the core is the congestion window (cwnd). This lesson separates window limits, loss recovery, and congestion feedback, then checks phase transitions in a simple model.
1. cwnd limits unacknowledged in-flight data
A simplified send limit:
usable window ≈ max(0, min(cwnd, rwnd) - bytes_in_flight)For example, cwnd=20 KiB, rwnd=12 KiB, and 8 KiB in flight leave about 4 KiB available. If rwnd becomes zero, increasing cwnd cannot bypass it. In-flight estimation is also more complex during recovery.
Transmission also depends on application data supply, socket buffers, pacing, TSO/GSO, Nagle/corking rules, ACK behavior, and scheduling. The window limit alone is not a throughput formula.
cwnd represents the sender's state and is typically measured in bytes; textbooks often simplify it using SMSS (Sender Maximum Segment Size) units. It does not equal the router queue length, nor does it represent the number of bytes sent in a single send().
Bandwidth-delay product (BDP) provides an estimate of how much in-flight data is needed to fully utilize a path without queuing:
BDP = bottleneck bandwidth × round-trip propagation timeMeasured RTT includes propagation, transmission, processing, and queuing delays, so it is not purely a propagation delay. BDP can also vary due to changes in route or link capacity.
2. Initial window and slow start
Experimental RFC 6928 proposes the following upper bound for the initial congestion window (IW):
min(10 * SMSS, max(2 * SMSS, 14600 bytes))This does not require every system to use exactly 10 segments. RFC 6928 is Experimental, not a mandatory Standards Track value. RFC 6928 Actual behavior can be influenced by implementation details, cached route metrics, transport variants, and configuration.
Slow start rapidly increases cwnd when new data receives ACKs, commonly simplified in teaching as "doubling approximately every RTT." Strictly speaking, growth depends on ACK ratio, delayed ACKs, ACK compression or loss, appropriate byte counting, application-limited state, and implementation specifics, not on a clock-based trigger that executes cwnd *= 2 every RTT.
Slow start typically exits when it reaches ssthresh or receives a congestion signal. Slow-start overshoot can inject a large number of packets into the bottleneck queue at once. Modern implementations may use pacing or mechanisms like HyStart++ to detect potential capacity boundaries earlier.
3. Congestion Avoidance and AIMD Intuition
Classic Reno-style congestion avoidance simplifies window growth to approximately one SMSS per RTT, and reduces the window multiplicatively after a loss event. This results in a sawtooth pattern and, under ideal assumptions, demonstrates convergence of efficiency and fairness.
However, the claim that "AIMD guarantees that every flow will inevitably share bandwidth equally" is an overgeneralization. Fairness is influenced by numerous factors including RTT, MSS, number of parallel connections, ACK behavior, loss distribution, AQM (Active Queue Management), algorithm coexistence, and application-limited traffic. A user opening 20 connections can actually achieve more bandwidth than a user with a single connection.
Congestion does not always manifest as packet drops. Queues can introduce long delays before packets are dropped, a typical symptom of bufferbloat. On ECN-capable paths, congestion can be signaled through packet marking without any actual packet loss occurring.
4. Loss Detection Goes Beyond "Three Duplicate ACKs"
Classic fast retransmit uses a duplicate ACK threshold to infer segment loss, offering faster recovery than waiting for the full RTO. However, duplicate ACKs can also result from packet reordering, duplication, or non-ideal ACK behavior, so counting three duplicates does not mathematically prove a packet has been lost.
Modern TCP typically combines several mechanisms:
- Cumulative ACK: Acknowledges a contiguous range of bytes;
- SACK (Selective Acknowledgment): The receiver reports non-contiguous blocks already received, allowing the sender to more precisely identify gaps;
- RACK (Recent Acknowledgment): Uses recent segment transmission times, acknowledgment information, and a reordering window to infer loss;
- TLP (Tail Loss Probe): Sends a tail probe to trigger early ACKs or recovery, avoiding the delay of waiting for full RTO due to tail packet loss;
- RTO (Retransmission Timeout): Acts as a fallback safety net when ACK-based fast recovery is insufficient.
SACK, RACK, and TLP improve loss detection and recovery; they are not congestion-control algorithms. After the sender determines a packet loss, it must still update congestion state using CUBIC, Reno, or another controller. RFC 8985
5. Fast Recovery and PRR
Historical comparisons of Tahoe, Reno, and NewReno have pedagogical value, but should not be presented as a complete state machine for modern Linux TCP:
- The Tahoe teaching model reduces
cwndto 1 SMSS after loss, then proceeds with slow start; - Reno's fast recovery uses the sending rhythm provided by duplicate ACKs to avoid immediately reverting to slow start during a single packet loss;
- NewReno improves recovery for multiple losses within a single window using partial ACKs;
- SACK enables the sender to precisely identify which byte ranges have been received;
- Proportional Rate Reduction (PRR) controls actual transmission during recovery, allowing the flight size to converge more smoothly toward
ssthresh.
PRR’s Standards Track specification is RFC 9937, which obsoletes Experimental RFC 6937. RFC 9937
A code snippet that only shows cwnd = ssthresh + 3 * SMSS represents only a snapshot of Reno's behavior and omits key components such as SACK scoreboard, pipe estimation, PRR, reordering, and retransmission ambiguity.
6. RTO Depends on Smoothed RTT and Variation
RTO should not be fixed at 200 ms. RFC 6298 uses SRTT + max(G, 4 * RTTVAR), where G is clock granularity, and recommends rounding values below 1 second up to 1 second. Systems may use different lower bounds. RFC 6298 §2 TCP maintains a smoothed RTT (SRTT) and RTT variation (RTTVAR) based on RTT samples, then computes RTO using minimum/maximum and backoff policies. Karn's algorithm avoids using ambiguous ACKs from retransmitted segments to produce erroneous RTT samples, and the timestamp option improves measurement accuracy.
A timeout typically signals a more severe disruption in the feedback path (specifically, the loss of continuous ACK timing) causing the congestion controller to become more conservative. However, the exact behavior of timeout responses depends on the RFC, algorithm, and implementation, and cannot be universally summarized by a single statement such as "all timeouts are cwnd=1" across modern TCP stacks.
7. ECN Using Marking to Express Congestion
Classic TCP ECN first negotiates support, after which senders set ECT in the IP header. A queue can mark ECN-capable packets CE, and the receiver reports this using TCP ECE. The sender reduces its sending load according to congestion-control rules and signals its response with CWR. RFC 3168
Extensions such as AccECN provide more detailed feedback; behavior depends on support at both endpoints.
ECN does not guarantee lossless transmission: queues may still drop packets under overload conditions, non-ECN traffic still requires loss signals, and policers can directly discard packets. Its value lies in decoupling congestion signaling from packet loss.
8. A Clearly Labeled RTT-Level Teaching Model
Save this Python 3 example as teaching_window.py and run python3 teaching_window.py. It illustrates slow-start and additive-increase shapes without simulating individual ACKs, SACK, retransmissions, or real TCP timers:
from dataclasses import dataclass
@dataclass
class TeachingWindow:
cwnd_mss: float = 10.0
ssthresh_mss: float = 64.0
def acked_round(self) -> None:
if self.cwnd_mss < self.ssthresh_mss:
self.cwnd_mss = min(self.cwnd_mss * 2, self.ssthresh_mss)
else:
self.cwnd_mss += 1
def congestion_event(self, beta: float = 0.5) -> None:
if not 0 < beta < 1:
raise ValueError("beta must be between 0 and 1")
self.ssthresh_mss = max(2.0, self.cwnd_mss * beta)
self.cwnd_mss = self.ssthresh_mss
state = TeachingWindow()
for _ in range(4):
state.acked_round()
print(state.cwnd_mss)
assert state.cwnd_mss == 65.0
state.congestion_event()
print(state.cwnd_mss)
assert state.cwnd_mss == 32.5Expected output:
20.0
40.0
64.0
65.0
32.5The third round stops at the model's threshold of 64; the fourth enters additive increase and reaches 65. A congestion event retaining 0.5 of the window then gives 32.5. Real threshold transitions and rounding depend on implementation. This model fixes its rules for calculation and does not establish a real connection's per-round window values.
9. Acceptance Questions
- What do
cwnd,rwnd, and bytes-in-flight represent? - Why does slow start only "double approximately every RTT"?
- How do SACK, RACK, TLP, and PRR each address different segments of loss recovery?
- Why does per-connection fairness differ from per-user fairness?
- Why isn't ECN equivalent to a no-drop network?
The next lesson is 10.2 CUBIC, BBR, Pacing and Active Queue Management.