15.2 CAP, Read-Write Consistency, and Degradation Boundaries

The CAP theorem is often depicted as a triangle, leading people to believe that databases can freely choose two out of three properties under normal conditions. In reality, the correct interpretation is that when a network partition occurs and messages across the partition cannot reach their intended destinations, a single operation cannot simultaneously guarantee linear consistency and ensure that all non-faulty nodes return a successful response.

What Do the Three Letters Mean?

• Consistency: Here, this typically refers to linear consistency, every operation appears to happen atomically between the call and the return, and all clients observe the same real-time order. This is not the "C" in ACID, nor does it mean "replicas eventually converge."
• Availability: Every request sent to a non-faulty node eventually receives a non-error response, but the response does not guarantee it contains the most recent write.
• Partition tolerance: The system continues to operate according to specification even when messages between nodes are arbitrarily lost or delayed.

As long as a system is deployed across a network, it's impossible to rely on "choosing between C and A" to eliminate partitions. In practice, the design must answer: when a partition occurs, which operations must maintain consistency and reject service, and which can return stale values or accept writes that will be merged later.

CP and AP Are Operational Strategies, Not Permanent Labels

Suppose there are two copies of inventory, separated by the network.

To maintain linearizable inventory deduction, one side must verify it still holds a legal majority or lease; if it cannot confirm this, the replica refuses to write. This sacrifices availability during partitioning to prevent both sides from selling the last item.

If "like" operations allow both sides to continue accepting requests during a partition, and the system can later merge counts or aggregate events upon recovery, availability is preserved to a greater extent. However, clients during the partition may not see the latest results from the other side.

The same system can apply a CP strategy to login permissions and an AP strategy to view counters. A product name cannot substitute for clear, semantic documentation of each API's behavior.

Consistency Goes Beyond Strong or Eventually Consistent

Common session or ordering guarantees in applications include:

• Read-your-writes: A user can read the value they just wrote;
• Monotonic reads: Within a single session, a user never sees a newer version and then reverts to an older one;
• Monotonic writes: Writes within a session occur in a strictly ordered sequence;
• Causal consistency: The observed order of causally related operations is consistent across all participants;
• Bounded staleness: Reads are guaranteed not to be more than a specified time or version lag behind the current state.

These guarantees can be less expensive than global linear consistency while still offering a better user experience than unconstrained eventual consistency. Interfaces should explicitly define their scope: single key, single partition, within the same region, or global.

Quorum Intersection Is Just a Basic Condition

With N replicas, if a write waits for W responses and a read queries R replicas, R + W > N ensures that the write and read sets have at least one intersection. However, having an intersection does not automatically guarantee linear consistency, additional mechanisms such as version selection, conflict resolution during concurrent writes, failure recovery, read repair, or consensus protocols are still required.

Cassandra's Consistency Level determines how many replicas the coordinator waits for before acknowledging a write; the write is still sent to the target replicas. QUORUM, LOCAL_QUORUM, and ONE may have different latencies and failure domains, especially across data centers, making it insufficient to rely solely on numerical values.

Beyond Latency and Partitioning: PACELC

Even without complete network partitioning, systems continually face trade-offs between latency and consistency: synchronously waiting for additional replicas enhances confirmation strength but increases tail latency and sensitivity to slow replicas. PACELC reminds us that "in the presence of a partition, prioritize availability and consistency; otherwise, prioritize latency and consistency", highlighting that CAP only describes failure boundaries, not a complete decision framework.

Conflict Resolution Must Align with Business Algebra

"Last Write Wins" is simple, but treating physical or client timestamps as final arbiters can let clock skew and late writes override correct data. A more reliable approach depends on whether data can be merged:

• Collections can use deduplication events or collection CRDTs (Conflict-Free Replicated Data Types);
• Counts can be merged by summing per replica;
• Individual fields in user profiles can be merged at the field level;
• Invariants like balances or unique usernames that cannot be arbitrarily merged typically require a single serialized point or a consensus transaction.

Avoid using a generic JSON merge for all conflict scenarios. The merge function must at least satisfy commutativity, associativity, and idempotency to withstand out-of-order delivery and duplicate updates.

Put Semantics into API Contracts

Every critical read-write interface should answer these questions:

1. What does a successful response mean, how many replicas were written, and has it been durably persisted?
2. From where should subsequent reads be made to satisfy read-your-writes?
3. In the event of a network partition or consensus lacking a majority, does the system fail, time out, or accept pending writes for later reconciliation?
4. Can retries result in duplicates? How long are idempotent keys retained?
5. Who detects conflicts, how are they merged, and is the user notified?

These answers are more instructive than simply stating "our database is AP" when designing client implementations or conducting failure scenarios.

A Business-Categorized Example

Data	Partitioning Strategy	Reason
Payment ledger	Reject on uncertain write authority	Prevents double-spending or arbitrary merging
Product descriptions	Accept local stale reads; writes go through authoritative regions	Stale read cost is manageable
Like events	Accept locally, deduplicate, asynchronously merge	Operations are commutative and idempotent
Access control policies	Default to rejection or use time-limited verified snapshots	Risk of unauthorized access is high

"Degradation" must be designed and tested before a partition occurs, never decided reactively after a failure.

Section Checkpoint

• Can explain that CAP applies only under network partition conditions, constraining consistency and availability in such scenarios.
• Can distinguish between linear consistency, ACID consistency, and eventual convergence.
• Will not assert system linear consistency solely based on R + W > N.
• Can define the success, failure, and merge semantics for specific business operations under failure conditions.

References

• Apache Cassandra: Tunable Consistency
• DynamoDB: Read Consistency

The next chapter moves into the implementation layer: how data is sharded, how replica groups elect leaders, and how cross-shard transactions are committed.