4.1 HTTP Messages, Methods, and Caching Semantics

TCP delivers bytes; the application still needs to identify the requested resource and report what happened. HTTP defines shared meanings for methods, targets, status codes, and representations, so a client can distinguish success, a missing resource, and a request it might retry later.

Do not equate request/response with text messages. HTTP/1.1 uses a textual start-line and fields, while content may contain arbitrary bytes; HTTP/2 and HTTP/3 use binary frames. All three versions share method, status, field, and representation semantics, but their wire formats differ.

This lesson begins by giving a request a clear and unambiguous meaning. The next lesson will build a server to handle CRLF, body length, persistent connections, and malicious framing.

1. Resource, Representation, and Message

Access:

https://atlas.example/chapters/4

A URI identifies the target resource. The HTML, JSON, or image returned by the server is a representation of that resource at a particular point in an interaction; it does not equate to the resource itself. Resources may change over time, and their representations can vary based on factors such as language, compression capabilities, or access permissions.

A single HTTP exchange consists of a request and one or more responses. The request expresses the client's intent, while the response conveys the outcome using a status code, headers, and optionally, content.

This distinction explains several common behaviors:

• The same URI can return JSON or HTML;
• PUT modifies the state of the target resource, rather than putting a packet into it;
• caches store responses/representations and must know the conditions under which they were generated;
• 204 can indicate a successful operation without including response content.

2. A Round Trip over HTTP/1.1

A client requests the resource for chapter 4:

GET /chapters/4 HTTP/1.1
Host: atlas.example
Accept: text/plain

A blank line marks the end of the header section. The server responds with:

HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Content-Length: 6
Cache-Control: max-age=60

hello

The shape of an HTTP/1.1 message is:

start-line CRLF
zero or more field lines, each ending with CRLF
CRLF
optional message body

The request start-line contains the method, request-target, and version; the response start-line includes the version, status code, and an optional reason phrase. The reason phrase is for human consumption only; client logic must rely solely on the status code.

This illustration does not show packet boundaries. A single recv might receive only half a line, or it might deliver data from two messages sent in the same direction. How to extract a complete HTTP/1.1 message from a stream is the subject of Section 4.2.

The readable http blocks hide some wire details: the request needs a terminating blank line, and response content is hello\n, six bytes. HTTP line endings are CRLF, while the newline inside this content is one LF byte. Editor line counts do not determine Content-Length.

3. Field Name Is Case-Insensitive, but Values Cannot Be Treated Uniformly

Content-Type and content-type refer to the same field name. Parsers typically normalize the name to lowercase, but the syntax for field values is defined individually by each field:

• Content-Length represents a decimal length;
• Content-Type includes a media type and its parameters;
• Cache-Control is a list of directives;
• Set-Cookie has its own syntax and cannot be arbitrarily concatenated with commas;
• Vary lists request fields involved in representation selection.

Therefore, "concatenating all repeated headers with commas" is not a universal rule. Both field sections and individual fields must have clearly defined size limits; the HTTP specification does not provide a globally fixed maximum value for any field.

Headers should not be used to carry arbitrary Chinese text directly. Field values are generally constrained by the US-ASCII range and their specific syntax rules. Fields requiring broader character sets must define their own encoding mechanisms. Business-related text should instead be placed within content that has a well-defined media type and character set.

4. Express Intent with Methods

The method states what action the client intends to take on the target resource. This "verb" is part of a protocol contract, not a naming convention for URLs.

Method	Safe	Idempotent	Common Intent
GET	Yes	Yes	Retrieve the current representation
HEAD	Yes	Yes	Obtain metadata equivalent to GET, without returning response content
OPTIONS	Yes	Yes	Query communication options
POST	No	Usually no	Let the target process the content according to its own semantics
PUT	No	Yes	Create or replace the target state with a given representation
DELETE	No	Yes	Request removal of the current association with the target resource
PATCH	No	Not guaranteed by specification	Modify the resource according to a patch document

Safe means the client's request is read-only in semantic terms; the server may still log, bill, or update statistics. Idempotent means that if the same request is sent multiple times, the server's expected outcome should be identical to that of a single request; response status codes need not be consistent across retries.

For example, the first invocation of DELETE /notes/7 might return 204, while the second returns 404; the effect of deletion remains "the association no longer exists." However, retrying POST /orders could result in two orders being created, unless the application implements an idempotency key or a deduplication mechanism.

5. Method and Body Are Two Separate Rules

"The GET method absolutely cannot have a body" is too absolute. While HTTP/1.1 defines a way to indicate request content, RFC 9110 does not establish a universal semantic meaning for GET content; clients should not send a body unless there is a clear agreement with the origin. Middlewares and proxies may also reject or ignore such content.

The browser Fetch API is stricter: constructing a GET or HEAD request with a body is rejected. Controlling the origin does not mean every client library and proxy will forward such a request.

A method accepting content does not imply that every media type is supported. The server must validate the following conditions:

• Whether the method accepts a body;
• Whether the media type in Content-Type is supported;
• Whether the content framing is complete;
• Whether the size is within acceptable limits;
• Whether the business schema is valid.

These failure types should be handled separately. Malformed framing typically results in a 400 error; unsupported media types are commonly responded to with a 415 error; valid syntax but business logic that cannot process the content may return a 422 error; specific error handling is defined by the API contract.

6. Status Code is a Machine-Readable Category of Response Outcomes

The first digit of a status code defines the broad category:

Range	Category	Intuitive Handling
1xx	Informational	Intermediate information sent before the final response
2xx	Successful	The request has been processed as per the semantics of the status
3xx	Redirection	A redirect or reuse of existing representation is required
4xx	Client Error	The request or the client's state is invalid
5xx	Server Error	The server failed while processing a valid request

In practice, the specific semantics matter more:

Code	Meaning and Boundary
200	Success; the content depends on the HTTP method used
201	A resource has been created; commonly includes Location pointing to the new resource
202	Accepted for processing; does not indicate that the asynchronous task is complete
204	Success with no content body
301 / 308	Permanent redirects; 301 permits historical POST-to-GET rewriting, 308 preserves the method
302 / 303 / 307	Temporary or indirect redirection; 307 preserves the original method, 303 typically switches to GET
304	Conditional GET/HEAD allows cache reuse of existing representation; no message body is included
400	The server cannot or will not process an obvious client error, such as malformed syntax
401	Missing valid authentication credentials; must include WWW-Authenticate with at least one challenge
403	The request is understood but action is denied; re-authentication may not resolve the issue
404	The target resource is not found, or the server refuses to disclose its existence
405	The requested method is not allowed for the target resource; the response must include Allow
409	A conflict exists with the current state of the resource
413	The request content exceeds the server's acceptable size limits
429	Request rate is too high; may include Retry-After
500	An unexpected server failure
502	A gateway or proxy received an invalid response from an upstream server
503	The server is temporarily unavailable, such as due to overload or maintenance; may include Retry-After

Clients should not treat all 4xx errors as permanent failures or all 5xx errors as automatic retries. Retries must also consider whether the method is safely re-issuable, whether the request body can be safely resent, overall deadlines, backoff strategies, and application-level idempotency contracts.

7. Content negotiation must respect Vary

Clients can express preferences using fields like Accept, Accept-Language, and Accept-Encoding. The server might select different representations for the same URI:

GET /guide HTTP/1.1
Host: atlas.example
Accept-Language: zh-CN
Accept-Encoding: gzip

If the response varies based on these request fields, the cache must be aware of the dimension of variation:

Vary: Accept-Language, Accept-Encoding

Omitting the correct Vary header can cause a shared cache to serve a response in a different language or encoding to a request that doesn't match. Conversely, arbitrarily setting Vary: * or adding numerous high-cardinality fields significantly reduces response reuse.

8. Freshness and Validation Are Two Steps

A cache first decides whether it may store a response, then whether the stored response is fresh. Once stale, it can validate that representation upstream without necessarily downloading all content again. The following directives use their common forms without field-name arguments:

Directive	Meaning
max-age=N	Freshness lifetime is N seconds; current age must be less than that lifetime
s-maxage=N	Freshness lifetime for shared caches
private	Shared caches are not allowed to store; private caches may still do so
public	Explicitly allows shared cache storage, even when standard rules would prohibit it
no-cache	Storage is permitted, but reuse requires successful validation before use
no-store	Caching is disallowed; this is not a sufficient measure for preventing eavesdropping or protecting privacy
must-revalidate	Once stale, the response cannot be reused without successful validation

Age estimates the time since origin generation or successful validation, including earlier cache residence; entry into a new cache does not reset it. Freshness is also influenced by Date, Expires, heuristic rules, and shared versus private cache policies; conclusions cannot be drawn solely from a single max-age string.

If corrected initial age is 40 seconds and local residence adds 25 seconds, max-age=60 is already exceeded. This model only adds residence time; the complete algorithm also handles Date, response delay, and clock differences:

freshness_lifetime = 60
corrected_initial_age = 40  # simplified: includes the received Age
resident_time = 25
current_age = corrected_initial_age + resident_time
print(current_age, current_age < freshness_lifetime)  # 65 False

9. ETag Makes Conditional Requests Skip Content

First request:

HTTP/1.1 200 OK
ETag: "chapter-4-v3"
Cache-Control: no-cache
Content-Length: 6

hello

Subsequent request, with client/cache validation:

GET /chapters/4 HTTP/1.1
Host: atlas.example
If-None-Match: "chapter-4-v3"

If the representation hasn't changed, the server responds with a 304 status, sending no content. If it has changed, the server returns a new 200 response with updated content. The request still occurs, but the savings come from avoiding transmission and generation of response content, not from eliminating the request entirely.

This Python 3.9+ model handles conditional GET for an existing resource. Its input is a tuple of entity tags already parsed and validated by an HTTP library, the wildcard "*", or None. It does not parse a raw comma-separated field. If-None-Match uses weak comparison, so W/"chapter-4-v3" also matches:

CURRENT_ETAG = '"chapter-4-v3"'
CONTENT = "hello\n".encode("utf-8")

def conditional_get(if_none_match):
    headers = {
        "ETag": CURRENT_ETAG,
        "Cache-Control": "no-cache",
    }
    if if_none_match is not None and if_none_match != "*":
        if not isinstance(if_none_match, tuple):
            raise ValueError("pass parsed tags as a tuple, or '*'")
    matched = if_none_match == "*" or any(
        tag.removeprefix("W/") == CURRENT_ETAG
        for tag in (if_none_match or ())
    )
    if matched:
        return 304, headers, b""

    headers["Content-Type"] = "text/plain; charset=utf-8"
    headers["Content-Length"] = str(len(CONTENT))
    return 200, headers, CONTENT

first = conditional_get(None)
validated = conditional_get((first[1]["ETag"],))
assert conditional_get(('W/"chapter-4-v3"',))[0] == 304
assert conditional_get("*")[0] == 304

assert first[0] == 200 and first[2] == b"hello\n"
assert validated[0] == 304 and validated[2] == b""
print(first[0], first[1]["Content-Length"])
print(validated[0], len(validated[2]))

Run:

python3 conditional_get.py

Expected output:

200 6
304 0

ETag can be either strong or weak. Last-Modified / If-Modified-Since can also perform validation with lower temporal precision. Using If-Match in modification requests helps prevent lost updates, which is related but distinct from the question of whether to use caching.

If-Match uses strong comparison. After client A updates v3 to v4, client B’s update conditioned on v3 should get 412. The comparison and write must be atomic, or two check-then-write operations can still overwrite each other. Computing an ETag may itself require rendering or reading data, so 304 does not guarantee that all generation work is avoided.

10. Cookie Places State in the User Agent

The core HTTP interaction does not require the server to maintain a client session. Cookies enable the server to request the user agent to store a small piece of state in a response, which is then sent back in subsequent matching requests:

Set-Cookie: session=opaque-token; Path=/; Secure; HttpOnly; SameSite=Lax

Then:

Cookie: session=opaque-token

It's generally best to store an unpredictable session identifier in the cookie, while keeping actual permissions and business state on the server. Several attributes address different boundaries:

• Secure ensures the cookie is sent only over secure channels;
• HttpOnly prevents non-HTTP APIs (such as page scripts) from reading the cookie, though it does not eliminate all XSS risks;
• SameSite controls whether certain cross-site requests carry the cookie;
• Domain and Path determine the scope of when and where the cookie is sent, not a strong authorization boundary;
• expiry controls how long the user agent retains the cookie, but does not guarantee that the server session remains valid during that time.

RFC 6265 defines the base cookie model; successor cookie specifications define extensions such as SameSite. When implementing login flows or cross-site scenarios, developers should base their decisions on the target browser and the latest specifications, and combine these with CSRF tokens, origin checks, and server-side authorization.

11. CORS Is Not Server Authorization

In browsers, the same-origin policy restricts pages from reading responses from cross-origin sources. CORS allows a server to declare which origins are permitted to access its responses; it does not block non-browser clients, nor does it substitute for server-side identity verification or access control.

Some cross-origin requests first send a preflight request:

OPTIONS /notes HTTP/1.1
Host: api.example
Origin: https://app.example
Access-Control-Request-Method: PUT
Access-Control-Request-Headers: content-type

After the server agrees, it returns the appropriate Access-Control-Allow-* header. Only then does the actual request proceed.

Two common pitfalls to watch for:

• The fact that "the browser won't let JavaScript read the response" does not mean the server didn't receive the request; endpoints with side effects must still implement CSRF protection and authorization;
• When the credential mode is set to include, Access-Control-Allow-Origin: * cannot be used in combination with credentials. The server must return a clear origin and exercise caution when setting Access-Control-Allow-Credentials: true.

Preflight requests themselves do not carry credentials. CORS caching and standard HTTP caching are not governed by the same rules or state.

The no-credentials rule is the Fetch requirement for cross-origin preflight; deployed browser and enterprise client-certificate behavior can differ. If Access-Control-Allow-Origin varies with the request Origin, cacheable responses also need Vary: Origin. Select allowed origins from a policy rather than reflecting every request.

12. Common Misconceptions

"HTTP is just a text-based protocol." This is only approximately true for the start-line and headers in HTTP/1.1; the content can be arbitrary bytes, and HTTP/2 and HTTP/3 use binary framing.

"GET requests must have no body." The semantics of GET content are not universally defined, and clients should generally avoid sending bodies. This has nothing to do with whether the wire framing can express it.

"Idempotency means the response is exactly the same every time." Idempotency constrains the intended effect, not the response itself; status codes, timestamps, or representations may vary.

"no-cache means caching is forbidden." It requires revalidation against the origin server; only no-store actually prevents cache storage.

"304 means the request is skipped." A conditional request has already reached the server; a 304 response omits the body entirely.

"CORS blocks attackers." It primarily governs whether user agents supporting Fetch/CORS will expose the response to the page; it does not provide authentication or firewall protection.

13. Exercises and Acceptance Criteria

1. Select the appropriate HTTP method for the following operations: "create task", "mark task 7 as complete", "read task 7", and "delete task 7". Explain which requests can be automatically retried.

2. Describe the differences between HTTP status codes 401, 403, 404, and 405. What additional field is missing from a 405 response?

3. Modify the conditional request script to update both the ETag to v4 and CONTENT to its new value. Verify that the old If-None-Match results in a 200 response and returns updated content.

4. Design responses selected from Chinese and English representations. Specify the necessary Vary required to support this.

5. Explain why Cache-Control: no-store cannot replace TLS.

Acceptance Criteria: The student must be able to clearly articulate the target, method, field, and content from a single request. They must be able to infer the next step from a response status code, without blindly retrying based solely on the status code.

14. Message Semantics Still Need Reliable Framing

The next lesson moves into 4.2 HTTP/1.1 Framing and the Minimal Server. The server must handle partial request lines, repeated Content-Length, slow clients, and connection closure instead of assuming that one recv returned a complete message.

References

• RFC 9110: HTTP Semantics
• RFC 9111: HTTP Caching
• RFC 9112: HTTP/1.1
• RFC 6585: Additional HTTP Status Codes
• RFC 6265: HTTP State Management Mechanism
• RFC 6265bis (Current Draft)
• WHATWG Fetch Standard: CORS Protocol