2.2 Message Boundaries, Deadlines, and Backpressure

The first echo connection is established. You slot two spell letters consecutively into the portal, but the opposite beacon only receives a single continuous byte stream: the envelopes are gone, the seals are gone. TCP faithfully delivers bytes, yet never promises to preserve the boundaries of two send for the application.

The post route manager must define themselves how to open letters. They can draw a delimiter at the end, or specify the length first; regardless of the choice, they must limit the size of a letter, how long a half-letter waits, and who stops when the reception hall is full. Mapping back to formal terms, these four items are framing, size limit, deadline, and backpressure.

This lesson implements a four-byte length-prefixed protocol. The example deliberately writes one byte at a time, forcing the parser to handle arbitrary segmentation; then it integrates timeout, deadline, pipelining, and backpressure into the same connection state machine.

1. Four Common Types of Framing

Fixed Length

Each message is exactly N bytes. Easy to parse, but short messages waste space, and variable-length fields require additional rules. Suitable for hardware logging or known-size blocks.

Separator

End with a newline or special sequence:

PING\r\n
SET key value\r\n

The implementation must handle separators spanning two recv, payload escaping, maximum line length, and incomplete lines. It cannot indefinitely wait and grow the buffer.

Length Prefix

[4-byte length][payload]

Binary and empty payloads can be transmitted. After reading the header, the upper limit must be verified before allocating and reading the body; otherwise, a 0xffffffff could trigger memory exhaustion.

Self-descriptive format

JSON objects, HTTP messages, or Protobufs have their own syntax/length rules. They still require incremental parsers and must preserve state on incomplete input. It cannot be assumed that a single recv yields a complete JSON.

2. Correctly Reading a Fixed Number of Bytes

recv_exact must be distinguished:

• Collect N bytes;
• Encountering EOF before any byte indicates a normal end of the message stream;
• After reading a portion, encountered EOF, indicating a truncated frame.

import socket
import struct
import threading

HEADER = struct.Struct("!I")
MAX_FRAME = 1024 * 1024


def recv_exact(connection, length, allow_clean_eof=False):
    data = bytearray()
    while len(data) < length:
        chunk = connection.recv(length - len(data))
        if chunk == b"":
            if allow_clean_eof and len(data) == 0:
                return None
            raise EOFError(
                f"stream ended after {len(data)} of {length} bytes"
            )
        data.extend(chunk)
    return bytes(data)


def encode_frame(payload):
    if len(payload) > MAX_FRAME:
        raise ValueError("frame too large")
    return HEADER.pack(len(payload)) + payload


def recv_frame(connection):
    header = recv_exact(
        connection,
        HEADER.size,
        allow_clean_eof=True,
    )
    if header is None:
        return None

    (length,) = HEADER.unpack(header)
    if length > MAX_FRAME:
        raise ValueError(f"declared frame too large: {length}")
    return recv_exact(connection, length)


def fragmented_writer(connection, frames):
    with connection:
        encoded = b"".join(encode_frame(frame) for frame in frames)
        for byte in encoded:
            connection.sendall(bytes([byte]))
        connection.shutdown(socket.SHUT_WR)


left, right = socket.socketpair()
expected = [b"alpha", b"", b"omega"]
writer = threading.Thread(
    target=fragmented_writer,
    args=(left, expected),
)
writer.start()

with right:
    actual = []
    while True:
        frame = recv_frame(right)
        if frame is None:
            break
        actual.append(frame)

writer.join()
assert actual == expected
print(actual)

socketpair Is sent byte-by-byte without going through the real network, reliably proving that the parser does not depend on packet or sendall boundaries. !I represents an unsigned 32-bit length in network byte order.

3. The Header Might Also Be Split

Many incorrect implementations write:

length = struct.unpack("!I", connection.recv(4))[0]

recv(4) can return only 1–3 bytes. Even if the current header has arrived, signal, scheduling, and buffer states can cause different read outcomes. Both header and body must go through an exact read loop.

Another mistake is treating EOF as "no data temporarily":

if connection.recv(4096) == b"":
    continue

This will busy-wait on a closed connection. A blocking socket blocks or times out when no data is available temporarily; returning empty bytes indicates a graceful EOF.

4. The length field belongs to untrusted input

Check after reading length, before allocating:

• Has the frame size exceeded the protocol's maximum frame?
• Is zero length allowed;
• header Could the algorithm potentially overflow with integer values;
• Is there a size limit after decompression;
• How many in-transit frames can one connection handle;
• Is the total buffer subject to global budget control?

“Up to 1 MiB” is just an example. The actual limit should be determined by business semantics, memory budget, and proxy chain constraints.

Compression protocols must also guard against high-compression-ratio payloads. Short wire length does not mean small decompression memory; compression input, decompression output, and CPU workload must all be limited.

5. Per-operation timeout vs. Total deadline

Set a 5-second timeout for each recv does not mean the entire request is limited to 5 seconds. The remote end could send one byte every 4 seconds, making a 1 MiB message last for weeks.

Calculate remaining budget using a monotonic clock for end-to-end deadlines:

import time


def recv_exact_before(connection, length, deadline):
    data = bytearray()
    while len(data) < length:
        remaining = deadline - time.monotonic()
        if remaining <= 0:
            raise TimeoutError("message deadline exceeded")
        connection.settimeout(remaining)

        chunk = connection.recv(length - len(data))
        if chunk == b"":
            raise EOFError("truncated message")
        data.extend(chunk)
    return bytes(data)

This function expresses the total budget, but isn't a complete production strategy:

• How do you handle a half-frame in a connection after a timeout?
• Should we close the connection, or does the protocol support secure resynchronization?
• Does the request deadline include waiting time and downstream calls?
• Can you retry when a response is partially sent?
• How do you signal cancellation to workers and I/O?

deadline is part of the state machine, rather than adding a number to the socket alone.

6. Backpressure: When the reception hall is full

The length prefix solves how to split messages without expanding the receiving buffer. If the remote end is slow at splitting messages, the messages first accumulate in the socket buffer and then spill over into the application's send queue. Continuing to accept messages without limit merely moves the road congestion into process memory.

Back on the main path, when the application calls send:

1. Byte advances into the local socket send buffer;
2. TCP sends data based on the receiving window and congestion window;
3. Place the peer kernel into the receive buffer;
4. The opposing application ultimately recv.

If the remote end reads slowly, the buffer will fill up level by level. blocking sendall will eventually block, nonblocking send will return would-block, and the pending send queue of the async writer will grow.

The wrong approach is for each producer to unconditionally append responses to the user-space list, turning network backpressure into a process OOM.

Need to make it clear:

• Maximum queued bytes per connection;
• Global maximum queued bytes;
• Ultra-timeout pause reading from upstream, reject new requests, or close slow connections;
• Which messages can be discarded or merged;
• write deadline;
• Fair scheduling to prevent a large response from starving smaller ones.

In async frameworks like asyncio, write typically only passes data to the transport buffer; it's drain/await or the high-water mark that participates in backpressure. Specific API behavior should be referenced in the runtime documentation.

7. Request/Response Order and Request ID

The simplest protocol allows only one request per connection:

send request A
receive response A
send request B
receive response B

Pipelining allows A and B to be in transit simultaneously. If responses must be returned in request order, a slow A will block already completed B; if out-of-order responses are allowed, each frame must carry a request ID.

[length][request-id][type][payload]

Also need to define:

• Is an ID reusable? If so, when?
• How are response, error, and cancel related;
• Which ID space does server push use;
• How to handle the same ID appearing repeatedly;
• After reconnection, does the old ID still hold any significance?

HTTP/2, gRPC, and QUIC streams have already addressed most multiplexing concerns, ordinary applications should not reinvent the wheel.

8. Blocking, Nonblocking, and Readiness

A nonblocking socket returns would-block when it temporarily cannot complete the operation. epoll/kqueue, and similar readiness APIs, inform the event loop that a particular fd "might be ready to read/write" now, without guaranteeing that the next operation will complete all requested work.

The event handler still has to:

• Loop read until a would-block condition occurs or the fair share budget is reached;
• Save parser state for half a header/body;
• Only care about writable when there is data pending to be sent;
• Handle hangup/error and remaining readable data;
• Avoid a loop that monopolizes a constantly active connection;
• Cancel the timer and business tasks before closing.

Edge-triggered mode typically requires draining to would-block, otherwise it might never wait for the next edge. Level-triggered will continue to notify as long as the condition holds, offering more intuitive behavior but possibly causing repeated wakeups.

Async/await hides the state machine inside the compiler/runtime, but it doesn't eliminate these protocol rules.

9. TLS will add another layer of framing

TLS packages application bytes into records. A single application write may correspond to multiple TLS records and multiple TCP segments; a single TCP receive may contain only half of a TLS record.

Applications should use the TLS library's read/write APIs, letting the library manage record, authentication, and reassembly states, and should not manually slice TCP bytes at the underlying encrypted socket level. TLS clean shutdown is distinct from TCP EOF, and security-sensitive protocols must verify receipt of a proper close notification.

10. Different boundaries of UDP framing

UDP preserves datagram boundaries, so a length prefix is typically not needed to reassemble the same datagram across multiple recvfrom. However, applications may still include multiple records within a single datagram or split large messages into multiple datagrams.

Custom UDP fragmentation requires message ID, fragment number, total count, timeout, deduplication, memory limit, and congestion control. Losing one fragment can render the entire message unusable. If your needs are reliable, secure, and multi-stream transmission, evaluate QUIC first rather than building it from scratch.

11. Protocol Test Matrix

Don't just test "normally sending a single message." At least cover:

Input	Expected
header reaches only 1 byte each time	correct reassembly
Split the body into arbitrary chunks	Correctly reassemble
Merge two frames into one recv	Parse two
Zero-length frame	Accept or reject according to protocol
Mid-header EOF	Truncated error
EOF in the middle of body	Truncated error
length exceeds limit	reject before allocation
Slow, byte-by-byte input	Total deadline takes effect
Remote end doesn't read response	queued bytes are bounded
cancel and close occur simultaneously	release only once, state remains consistent

A property-based test can randomly split a byte string with the same encoding and verify that the parser produces the same message sequence for all chunk boundaries.

12. Summary

The TCP protocol design must proactively supplement the things the byte stream does not provide:

• framing defines the boundaries of a message;
• recv_exact process arbitrary splits of header/body;
• The length and decompression result must be bounded before allocation;
• An operation timeout cannot replace an end-to-end deadline;
• Backpressure lets the slow downstream limit the upstream, rather than infinitely accumulating memory;
• Pipelining requires request IDs, cancellation, and response ordering rules;
• readiness/async still needs to maintain half-open, error, and closed states;
• Randomize chunk boundaries in testing and cover truncation and slow clients.

By this point, the beacon tower administrator has already demarcated message boundaries in consecutive bytes and knows that when the hall is full, upstream traffic must be halted. In the next chapter, we open the network observation window to first examine how TCP connections are established and terminated, then track sequence numbers, ACKs, flow control, and retransmissions.