19.1 Caching Protocols, Consistency Windows, and Invalidation

Caching is not simply about storing a copy of data elsewhere. It introduces a replica between the authoritative data source and read requests, one that may expire, be lost, or be evicted. The performance benefits come from higher hit rates and reduced access costs, but correctness depends entirely on the caching protocol.

First, Write the Cache Contract

Before writing any code, answer these questions:

• Who is the authoritative data source?
• How old can cached data be? Which fields must never be stale?
• After a cache miss, can the data be reconstructed?
• How should the key include tenant, permission, language, and schema version information?
• How are delete, update, and permission revocation events propagated?
• What is the fallback strategy when the upstream source is overwhelmed?

If cached state decides inventory, balances, or permissions, it requires transactional integrity, durable persistence, and auditability on par with the authoritative storage system.

Hit Rate Must Be Evaluated Alongside Cost

The average read latency can be roughly expressed as:

E[latency] = hit_rate * cache_latency
           + miss_rate * (cache_latency + origin_latency)

This still doesn't account for tail latency, serialization, network overhead, invalidation broadcasts, or origin queueing. A high hit rate may not be valuable: if the hits are all low-cost queries and only a few misses involve heavy-weight requests, the p99 latency can still be poor.

Assume a 95% hit rate, a 1 ms cache round-trip, and an additional 20 ms origin read. The simplified mean is 0.95×1 + 0.05×21 = 2 ms. This does not make p99 equal to 2 ms; slow requests still reach the origin and may queue.

Do not treat "80% of requests access 20% of data" as a law. Instead, estimate the working set based on real key frequencies, object sizes, TTLs, reuse intervals, and the cost of origin requests.

Cache-Aside: Application Reads and Invalidation

Basic read flow:

read cache
  hit  -> return
  miss -> read source -> populate cache -> return

The write path typically follows an "update database, then delete cache" pattern:

begin database transaction
update authoritative row
commit
delete cache key

Writing the cache first and then the database risks leaving behind an inconsistent "new fact" if the database operation fails. Deleting the cache before committing the database creates a window during which concurrent reads might re-populate the cache with outdated values. Therefore, it's standard practice to invalidate the cache after the database commit.

However, database commit and cache deletion remain a cross-system dual write: if the database transaction commits successfully but the process crashes before deleting the cache key, the old value persists until its TTL expires. Stronger alternatives include:

• Emitting invalidation events reliably from the database's WAL, outbox, or CDC stream;
• Attaching a source version to cache values to reject older events that would overwrite newer ones;
• For reads that cannot tolerate staleness, directly redirect to the authoritative source or use a shared transactional system;
• Using TTL only as a safety net for eventual consistency, not as a guarantee of correctness.

Successful invalidation still permits an old loader to refill the key:

reader: cache miss -> read source v7 -> pause
writer: commit source v8 -> invalidate cache
reader: resume -> populate cache with v7

Comparing against the cached version alone is insufficient because the deleted key can be empty. Atomically check a separately retained version watermark/generation, or use a proven coordination protocol for fills and invalidations. Retain that watermark for the lifetime of older requests. Otherwise, declare the allowed staleness and depend on eventual refresh.

The concept of "delayed double deletion" relies on an arbitrary, heuristic wait time and cannot account for arbitrarily slow queries, job scheduling pauses, or network latency, making it neither universally reliable nor a formal proof of consistency.

Read-Through and Write-Through Are Only Shifts in Responsibility Location

Read-through involves a cache library or proxy loading source data when a cache miss occurs, presenting the application with a unified read interface. It reduces code duplication, but issues such as loader timeouts, batching, permissions, and negative caching still require careful design.

Write-through synchronously updates the cache and backend before returning a write response. If the cache and backend are not part of the same atomic protocol, it remains necessary to handle scenarios where one side succeeds and the other fails; the operation’s name alone does not guarantee transactional consistency.

Write-behind first accepts writes into the cache and then asynchronously propagates them to the backend. This approach can batch writes and reduce front-end latency, but it makes the cache a temporary authoritative source. As a result, it requires persistent queues, write ordering, idempotency, retry mechanisms, backlog monitoring, and fault recovery. Critical data such as balances or inventory should not be entrusted solely to a single asynchronous thread.

Key Design is Part of Correctness

product:v3:tenant=42:locale=zh-CN:id=9182

A key must at least consider:

• Tenant and permission boundaries to prevent cross-user data leakage;
• Parameters that affect outcomes, such as language, currency, or experiment group;
• Schema and serialization versioning to support safe rollouts;
• Normalized parameter ordering to prevent equivalent requests from generating multiple keys;
• Length and sensitivity of data, avoiding direct exposure of tokens or personal information in the key.

Cache null values must be distinguished from misses; use an explicit envelope to achieve this.

{ "found": false, "source_version": 18 }

Negative cache TTLs should generally be short, since the absence of an object may be quickly remedied by its creation.

TTL is part of the business-defined staleness boundary

TTL selection should consider:

• The business tolerance for data staleness;
• How frequently data is updated;
• The cost of rebuilding and the capacity of the source system;
• Cache size and pressure from eviction;
• How long the system can safely rely on stale values during failures.

Redis checks logical expiration when ordinary keys are read. Delayed background memory reclamation does not mean GET should keep returning an expired value, but Redis cannot expire copies in local caches, proxies, or CDNs. See Redis expiration.

Loading from a lagging replica, filling from a delayed loader, or renewing a full TTL at each layer can extend staleness beyond any one TTL. Define source age, maximum load duration, and renewal rules, then evaluate the worst read path. Unbounded pauses or indefinite renewal prevent a finite bound from being derived by simple addition.

Different fields (such as security policies, ban statuses, and pricing) should not share a single arbitrary TTL for convenience.

Multi-Level Caching and Versioned Invalidation

process local cache -> distributed cache -> source database

Local caching offers the fastest access, but it cannot automatically clear entries when the central Redis instance removes them. Common strategies include:

• Broadcasting invalidation events, though this requires handling event loss and reconnection compensation;
• Using short TTLs at the local level and longer TTLs at the distributed layer;
• Including a generation or version number in the key; after a new version is published, old keys are automatically ignored;
• Periodically sampling data and comparing it against the authoritative source to detect propagation drift.

Versioned keys work well in scenarios involving full deployments or when object versions are known in advance. However, old keys still rely on TTLs or cleanup mechanisms, and version counters must not become unreliable due to inconsistent writes across systems.

Section Checkpoint

• Can identify authoritative sources, staleness windows, and recovery paths after a cache miss or eviction.
• Can analyze invalidation failures and stale-loader races in Cache-Aside.
• Does not equate TTL, double-delete, or write-through mechanisms with proof of consistency.
• Can ensure that every key covers all dimensions affecting both the result and access permissions.

The next section addresses the most dangerous moments in caching: a surge of concurrent cache misses, invalid keys bypassing the cache, and the cascading recovery storm that follows when the entire cache layer fails.