14.3 HTTP, Caching, CDN, and Capacity Optimization
Even with a healthy network path, users may still experience slow performance: unstable cache keys trigger repeated origin requests, responses consume too much bandwidth, load balancers send traffic into overloaded pools, or retries turn one failure into congestion. Application and edge optimizations must be co-designed around user metrics and system capacity.
Locate the Time Before the First Byte
Define the timing origin first. This diagram covers the period after connection setup; a measurement from transfer start, such as curl time_starttransfer, also includes earlier DNS, connection, and TLS work:
request upload
+ network to edge/server
+ edge queue and processing
+ origin/upstream queue and processing
+ network back to client
+ first response bytesTherefore, a high TTFB cannot be automatically attributed to slow server-side code. Use the same trace ID to align these timing components:
| Location | What to Record |
|---|---|
| client/RUM | DNS lookup, connect, TLS handshake, TTFB, protocol, network type |
| CDN/LB | cache status, edge processing time, origin request time, selected backend |
| application | queue time, handler processing time, status, request size |
| database/upstream | pool wait time, RPC duration, attempt count, timeout |
If the application handler takes only 20 ms but the load balancer queue consumes 800 ms, optimizing the SQL query will not improve this particular sample.
DNS: Reducing Dependencies, Not Blindly Increasing TTL
DNS performance is influenced by resolver proximity and caching, CNAME chains, authoritative server availability, DNSSEC, and response size. A webpage referencing many resources does not necessarily result in an equivalent number of DNS queries; multiple resources under the same hostname can share caches and connections.
Optimization principles:
- Minimize unnecessary independent hostnames and CNAME layers that lack architectural value;
- Use reliable, low-latency authoritative DNS services with redundancy;
- Balance TTL between caching efficiency and the need for rapid updates;
- Before making changes, reduce TTL to allow old responses to expire gradually; after the old TTL has fully expired, proceed with the change;
- Monitor
NOERRORempty answers,NXDOMAIN,SERVFAIL, truncation, and DNSSEC; - When publishing both A and AAAA records simultaneously, validate each path separately.
dns-prefetch performs only name pre-resolution; preconnect may complete DNS and connection/TLS setup earlier. These operations consume sockets, CPU, and server resources and should be reserved for a small number of critical origins that users are highly likely to access.
Pre-connecting only prepares the connection for a specific origin, and you should not simply multiply DNS+TCP time by ten for ten resources, because browsers cache DNS lookups and reuse or multiplex connections.
Define the Object’s HTTP Cache Semantics
Determine four key points:
- Which cache can store: shared cache or private browser cache;
- What goes into the cache key: URL, HTTP method, selected request headers, tenant or authentication context;
- How long the response is considered fresh;
- How stale responses are revalidated, and whether stale service is allowed when the origin fails.
Content-Addressed Static Resources
These request/response excerpts show key headers, omitting bodies and other details:
GET /assets/app.7f3a9c1.js HTTP/1.1
Host: example.com
HTTP/1.1 200 OK
Cache-Control: public, max-age=31536000, immutable
Content-Type: text/javascriptLong freshness periods are only safe when the filename changes with the content. If the same URL's content is overwritten, immutable makes it difficult for updates to reach clients in a timely manner.
Documents That Require Revalidation
HTTP/1.1 200 OK
Cache-Control: no-cache
ETag: "revision-42"Unqualified no-cache, as used here, permits storage but requires successful validation before each reuse. no-store forbids storing this request/response; it does not retroactively clear all earlier cached copies and is not access control. A later GET can carry:
If-None-Match: "revision-42"If the resource is unchanged, the server responds with 304, omitting the full representation. However, 304 still includes headers and incurs a network round-trip, so it cannot be called "zero bytes, zero cost." This is conditional GET/HEAD behavior. A failed If-None-Match condition on another method normally returns 412, not 304.
Personalized Content and Vary
If content varies based on Accept-Encoding, language, or other request headers, the response must correctly set Vary. However, including high-cardinality or user-specific headers in the cache key leads to catastrophic cache miss rates.
Responses with authentication or personalization must never be marked public simply to improve cache hit rates. Data leakage across users in a shared cache is far more dangerous than performance degradation. Explicitly use private to restrict shared caching or no-store to prohibit storage. HTTP does not automatically forbid all caching merely because Cookie/Set-Cookie is present. Vary selects variants; it does not authorize users.
Compression: Balancing Content and CPU Cost
Text-based formats like HTML, CSS, JavaScript, and JSON respond well to gzip or Brotli compression. Already-compressed assets such as JPEG, PNG, video files, and ZIP archives yield minimal gains from further compression. Dynamic, high-level compression can increase CPU usage and lead to longer Time to First Byte (TTFB).
curl --compressed --silent --show-error --dump-header - \
--output /dev/null https://example.com/app.js--compressed negotiates encodings supported by curl and decodes the response body, but the server may still send an uncompressed response. Retained response headers continue to describe the encoded representation. Check Content-Encoding, Vary: Accept-Encoding, original size, transmitted size, and server CPU load. Pre-compressing static assets can offset request-time overhead during the build phase, but ensure content types and cache variants are correctly configured.
HTTP Connections and Request Shapes
- HTTP/1.1: Bound concurrent connections per origin, reuse connections, and avoid excessive short-lived connections;
- HTTP/2: Multiplex requests while managing priorities, the failure scope of a single connection, and stream concurrency;
- HTTP/3: Validate UDP/QUIC reachability and fallback, and monitor QPACK and connection migration;
- All versions: Enforce request/response deadlines, set body size limits, and implement backpressure.
"Combining all small files into one large file" is not a universal solution across protocols. While HTTP/2 and HTTP/3 reduce the overhead of multiple requests, each request still incurs header costs, scheduling overhead, and server processing expenses. Over-aggressive bundling can instead expand cache invalidation scope. The decision must be guided by dependency graphs, cache granularity, and actual request waterfall behavior.
CDN: Before Optimizing Hits, Define Cache Keys
A single CDN request can result in one of these outcomes:
- hit: served directly from the edge node;
- miss: fetched from the origin server;
- revalidated: the edge node performs a conditional request to validate the object and then serves it;
- stale: serves an older object during background revalidation or origin failure, within policy;
- bypass: caching is bypassed due to rules, authentication, or request characteristics; the request still proceeds.
Key metrics to monitor:
request hit ratio
byte hit ratio
origin requests and egress
edge TTFB by region
miss/revalidation latency
cache-key cardinality
purge propagation and errorsA high request hit ratio need not save much bandwidth. If 99 objects of 1 KiB hit and one object of 1 MiB misses, the request hit ratio is 99%, but the byte hit ratio is only 99 / (99 + 1024) ≈ 8.8%.
Common issues include: unintended inclusion of cookies or query parameters in the cache key, incorrect handling of encoding or language variants due to Vary, cached error responses, and all edge nodes simultaneously fetching from origin, leading to a thundering herd scenario. These can be mitigated using request coalescing, origin shielding, stale-if-error, and refreshes with jitter. Set consistency and failure limits: stale-while-revalidate and stale-if-error need a maximum stale period, not unlimited old permissions or prices. A CDN purge normally affects that CDN’s caches, not still-fresh copies in browsers. Design versioned content, TTL, and rollback together.
Do not rely on guessing X-Cache or other private headers to determine CDN behavior. Each provider defines headers, logs, and hit status semantics differently. Always refer to the actual vendor documentation and configuration to ensure accurate interpretation.
Load Balancing and Capacity
Load balancing cannot create backend capacity. If every instance is already saturated, distributing requests evenly will only cause all instances to queue up simultaneously.
Key metrics to monitor collectively include:
- active connections or requests
- queue depth and queue time
- success, error, and timeout rates
- per-backend latency and saturation levels
- health check status changes
- retry attempts and retry budget
- connection pool size, ephemeral port availability, and NAT/LB state capacity
Algorithms and Business Costs
Round-robin works well when backend capabilities and request costs are similar. Least connections helps with long-lived connections, but connection count does not always reflect actual workload. Consistent hashing supports affinity and cache locality, yet it triggers rebalancing when members change.
Health checks should establish whether a backend can serve traffic without removing the entire pool for a brief failure of a noncritical dependency. Liveness, readiness, and deep business-level probing serve distinct roles.
Retries Amplify Load
If three call layers each allow at most three attempts, including the first, the deepest dependency may receive 3 × 3 × 3 = 27 requests. This is not three additional retries per layer. Retries should only be used for operations that are safe to retry and must include:
- bounded attempt limits
- exponential backoff with jitter
- end-to-end deadlines
- retry budget enforcement
- idempotency keys or explicit idempotent semantics
- circuit breaking, rate limiting, and overload protection
When a service is already overloaded, unbounded retries can turn a localized failure into a global congestion event.
Large Object Transfer
The performance of large file transfers is constrained by multiple factors including CDN capabilities, range requests, transmission window size, bandwidth, and client read speed. Check the following:
- Whether
Range/206 is supported and properly handled; - Whether the content is suitable for edge caching;
- Whether chunked uploads, resumable transfers, and checksums are needed;
- Whether the application loads the entire object into memory;
- Whether rate limiting and fairness prevent a single user from monopolizing the link;
- Whether download tests include disk write speed in their network measurements.
For resuming a download, verify the 206 Content-Range and the same representation’s strong ETag; If-Range can prevent mixing versions. A server may ignore Range and return a full 200 response. Restart the download in that case instead of appending the entire file to the old fragment. Do not mix offsets across content encodings either.
Using /dev/null helps reduce the impact of local disk I/O, though it still includes client CPU, TLS, and kernel processing overhead.
From Baseline to Revalidation
- Define user SLIs: success rate, p50/p95/p99, throughput, and business outcomes;
- Segment by region, network, protocol, cache status, backend, and other dimensions;
- Use trace data, packet capture, or connection state analysis to identify the bottleneck phase;
- Formulate a falsifiable hypothesis;
- In a representative environment, change only one key variable at a time;
- Monitor both the target metric and guardrails simultaneously;
- Launch in a gradual rollout, with rollback capability retained;
- Document results, including experiments that yield no measurable benefit.
Acceptance Checklist
- [ ] Don't substitute business SLOs with fixed millisecond thresholds;
- [ ] Can explain why TTFB does not equal application processing time;
- [ ] Can distinguish
no-cache,no-storefrom immutable; - [ ] Can write cache key and purge strategies for CDN;
- [ ] Can explain why load balancing cannot resolve overall capacity shortages;
- [ ] Performance changes include baseline, comparison, guardrails, and rollback mechanisms;
- [ ] Validate the critical paths for IPv4, IPv6, HTTP/2, and HTTP/3.
Fourth Volume: Professional Content Summary
This volume begins with how data is layered and encapsulated, progressing through Socket, TCP, HTTP, TLS, WebSocket/gRPC, DNS/CDN/LB, Ethernet, IP, routing, congestion control, QUIC, network diagnostics, and IPv6, culminating in performance measurement and capacity validation.
The method remains consistent: define protocol layers and state machines, separate specification semantics from implementation strategy, constrain conclusions with observed evidence, and validate them through repeatable experiments. Small examples and restrained metaphors can build intuition, but they cannot replace the mechanism.