16.2 C Concurrency Memory Model

Making a completion flag atomic is only part of the protocol: specify when the reader may access the payload beside it. This lesson uses the C11/C17 thread memory model.

The fact that two threads can access the same memory address does not mean they will observe each other's operations in source order. Compilers are allowed to reorder and eliminate operations as long as the observable behavior of a single thread remains unchanged. CPUs also introduce store buffers, cache coherence mechanisms, and out-of-order execution. The C11 memory model establishes a shared agreement across these layers.

This chapter focuses on thread visibility and atomic operations. Details about object alignment, what restrict guarantees, and where volatile is appropriate are covered in Object Layout, Aliasing, and volatile.

1. Start from the formal conditions of a data race

If two evaluations in different threads:

• access the same memory location;
• at least one of them modifies the location;
• at least one of them is not an atomic operation;
• and there is no happens-before relationship between them;

then they constitute a data race, and the behavior of the C program is undefined.

int counter = 0;

/* Thread A */ counter++;
/* Thread B */ counter++;

counter++ represents a read, computation, and write sequence. Even if the target machine provides non-tearing aligned int loads and stores, these expressions still exhibit unsynchronized conflict. The issue is not merely that "the final addition is missing"; the compiler can perform more aggressive optimizations based on the assumption that "the correct program has no data race."

Three common approaches to fixing this are:

• Use a mutex to protect all conflicting accesses;
• Declare the object as an atomic type and select an appropriate memory order;
• Transfer exclusive ownership with synchronization; apparent non-overlap in wall-clock time does not establish happens-before.

2. happens-before is the logical backbone of visibility reasoning

happens-before is not simply the chronological order of events on a wall clock. It is formed through the transitive composition of several types of relationships:

sequenced-before describes the evaluation order specified by the language within the same thread.

synchronizes-with arises from specific synchronization events, such as:

• An unlock of a mutex followed by a successful subsequent lock on the same mutex;
• An acquire load reading the value published by a release store;
• Synchronization defined by thread creation and termination/join operations;
• Other synchronization primitives explicitly defined by the language or library.

If A sequenced-before a release, and that release synchronizes-with an acquire, and the acquire itself is sequenced-before B, then A happens-before B.

producer: write payload
          release store ready=true
                    │
                    │ synchronizes-with
                    ▼
consumer: acquire load sees true
          read payload

The key constraint is that "the acquire must read the value published by the corresponding release or by a value released in its release sequence." Simply placing a release and an acquire on the same object does not automatically establish a happens-before relationship across all executions.

3. A Correct Publication Protocol

Save as publication.c. The worker first writes a regular object answer, then uses a release store to publish a completion flag. Only after the main thread's acquire load observes ready == true does it proceed to read answer:

#include <pthread.h>
#include <sched.h>
#include <stdatomic.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>

static int answer = 0;
static atomic_bool ready = false;

static void *compute(void *unused) {
    (void)unused;
    answer = 6 * 7;
    atomic_store_explicit(&ready, true, memory_order_release);
    return NULL;
}

int main(void) {
    pthread_t worker;
    if (pthread_create(&worker, NULL, compute, NULL) != 0) {
        return EXIT_FAILURE;
    }

    while (!atomic_load_explicit(&ready, memory_order_acquire)) {
        sched_yield();
    }
    printf("%d\n", answer);

    if (pthread_join(worker, NULL) != 0) {
        return EXIT_FAILURE;
    }
    return answer == 42 ? EXIT_SUCCESS : EXIT_FAILURE;
}

cc -std=c17 -O2 -Wall -Wextra -Werror -pthread publication.c
./a.out
# 42

answer itself is not an atomic object, but its write happens-before the main thread's read, so there is no data race. If the main thread reads answer prematurely (before the acquire succeeds) the protocol is violated. This is a one-time publication: the worker never changes answer after publishing. Reusing ready or overwriting answer requires another ownership handoff. sched_yield is a scheduling hint, not a fairness or termination guarantee.

In this small example, pthread_join could also be used to synchronize the worker's completion; the presence of the flag is retained solely to demonstrate the release protocol. In production code, if a join operation is already required, there's typically no need to busy-wait for a completion flag.

4. The Six Memory Orders and What They Govern

C11 atomic operations can specify:

Order	Applicable to	Core Semantics
relaxed	load/store/RMW/fence	Atomic accesses follow per-object modification order; a relaxed fence has no effect
consume	load/RMW/fence	Dependency-based read ordering, usually avoided in application protocols; a consume fence acts as an acquire fence
acquire	load/RMW/fence	Establishes the acquire side for following accesses when the required publication is observed
release	store/RMW/fence	Establishes the release side for preceding accesses, paired with the required acquire operation
acq_rel	RMW/fence	Combines acquire and release semantics
seq_cst	load/store/RMW/fence	Also imposes the specified single total order on seq_cst operations

Fences synchronize through the required atomic reads and writes; a pair of fences alone does not publish data. A failed CAS takes the load path and obeys failure-order restrictions.

"Stronger" does not mean "automatically locking all ordinary variables." Memory order semantics only propagate visibility under specific conditions: satisfying the required read, modification order, and synchronizes-with relationships.

Relaxed is Suitable for Independent Counting

If a counter does not publish other objects, and only requires that each increment is not lost, relaxed ordering is acceptable:

#include <stdatomic.h>

static atomic_ulong requests = 0;

void record_request(void) {
    atomic_fetch_add_explicit(
        &requests, 1, memory_order_relaxed
    );
}

Each fetch-add still participates atomically in the modification order of requests. However, it does not guarantee that threads reading the counter will simultaneously see other fields of a request object. The unsigned long counter wraps modulo its range; a lifetime total still needs an overflow policy.

Acquire/Release is Suitable for Publishing

The ready from the previous section is one-way publishing: the producer's preceding writes become visible after the consumer's acquire reads the release-published true value.

Start with Seq_cst When It Makes the Proof Clearer

C atomic operations without explicit ordering default to seq_cst. It's typically the easiest to reason about and serves as a good starting point for correctness. Only when performance measurements show that synchronization overhead is significant (and when a complete happens-before proof can be written) should the ordering be weakened.

Do not document fixed statements like "seq_cst is X nanoseconds slower than acquire." Performance costs depend on architecture, operation type, cache line state, contention levels, and may be optimized by the compiler to combine adjacent operations into larger contexts.

5. Atomicity Does Not Mean Lock-Free

_Atomic(T) ensures that operations on the object conform to the language's semantics, but does not guarantee that the implementation uses a single hardware instruction. In certain types or alignment scenarios, compilers and runtimes may employ internal locks.

You can use:

atomic_is_lock_free(&object)

to query the implementation attributes of a specific object. Even if this returns true, the overall algorithm is not necessarily lock-free; it might indefinitely wait under certain conditions. Lock-free, wait-free, and obstruction-free describe guarantees about system progress, not labels for individual variable types.

Atomic operations also cannot enforce composite invariants. For example, if "the balance and transaction state must change together," making both fields atomic individually may still allow observers to see inconsistent combinations. In such cases, mutexes or formally proven multi-word protocols are often clearer and safer.

6. Read-modify-write and Compare-Exchange

atomic_fetch_add, atomic_exchange, and successful compare-exchange are atomic read-modify-write (RMW) operations. Failed compare-exchange performs an atomic load and updates expected. The concept of compare-and-swap (CAS) is:

if (*object == expected) {
    *object = desired;
    success
} else {
    expected = *object
    failure
}

In the C API, when a CAS fails, the actual value is written back to expected. The weak version also allows for spurious failures when the value is equal, so it is typically used within a loop:

int expected = atomic_load_explicit(
    &state, memory_order_relaxed
);
int desired = transform(expected);

while (!atomic_compare_exchange_weak_explicit(
        &state,
        &expected,
        desired,
        memory_order_acq_rel,
        memory_order_relaxed)) {
    desired = transform(expected);
}

This function-body fragment assumes atomic_int state and an int-to-int transform. Compute desired from expected before the first attempt, then recompute from the refreshed expected after failure. transform must tolerate retries without repeating business side effects. The failure ordering cannot be release or acq_rel, and must not be stronger than the success ordering.

CAS loops can still encounter problems such as ABA, starvation, cache line contention, and memory reclamation. Being able to atomically update a machine word does not mean that a correctly implemented lock-free container has been achieved.

7. Mutex is Often More Appropriate Than Elaborate Atomic Protocols

The correctness of a mutex goes beyond simply preventing two threads from entering simultaneously; it establishes synchronization between a successful lock acquisition and the prior holder's unlock. As a result, ordinary access within the critical section can be safely and reliably passed from one thread to another.

Use a mutex when:

• Multiple fields need to be maintained with invariants;
• The critical section may evolve in the future, with additional state being added;
• Low contention paths are already sufficiently fast;
• Condition variables, timeouts, or priority-based scheduling are required;
• The team lacks the long-term capability to maintain lock-free correctness proofs;
• Object reclamation is more complex than atomic updates themselves.

The value of atomic operations lies in expressing a specific shared protocol, not in avoiding the appearance of "locks being less sophisticated." First, define the state machine and progress conditions. Only after that should you choose atomic primitives.

8. Compiler Reordering and CPU Reordering Must Be Separated

The statement "the CPU swapped two lines" often conflates three distinct layers:

1. The C abstract machine permits the compiler to generate instruction sequences different from the source code order;
2. The ISA defines memory ordering constraints across cores;
3. Microarchitecture can execute instructions out of order, as long as the architecture guarantees the required memory consistency.

C atomic memory order simultaneously constrains both compiler reordering and maps to the specific instructions or memory barriers required by the target ISA. The same C code may generate different machine instructions on different architectures, but it must maintain identical language-level semantics.

On x86, code without atomic operations may appear to work, yet it still constitutes a C data race. Moving to ARM only makes the issue more apparent; the error was always present. You cannot base C correctness on the accidental strong ordering behavior of a particular CPU.

9. Why volatile Cannot Fix Thread Communication

Changing ready to volatile int still does not provide atomic synchronization:

• Ordinary read-and-write operations may result in data races;
• volatile does not establish a synchronizes-with relationship;
• it does not publish the adjacent answer;
• it does not support composite atomic operations;
• it does not guarantee cross-thread progress ordering.

The meaning of C volatile is not "read from DRAM every time," nor is it a portable compiler fence directive. Specific usage boundaries are detailed in the accompanying Object Layout Page.

In Java, volatile is a synchronization action defined in the Java Memory Model, with semantics significantly stronger than C volatile. It cannot be broadly equated with all C seq_cst operations. Similarly, the Global Interpreter Lock in Python implementations is not a user-level data protocol. Extension modules, I/O that releases the GIL, and optional free-threaded CPython builds available since 3.13 all require explicit synchronization protocols.

10. Using Tools to Detect Race Conditions

On supported platforms, first try ThreadSanitizer with publication.c above to check that the runtime works. race.c below means a separately saved, complete concurrent test containing a race, not the earlier few-line fragment:

cc -std=c17 -O1 -g -fsanitize=thread   -fno-omit-frame-pointer -pthread race.c
./a.out

It can detect many actual data races in execution, but it is not a proof system:

• Paths that are not executed will not be reported;
• Custom synchronization primitives may require additional annotations;
• Timing and overhead differ from production builds;
• Some platforms or libraries are not supported;
• Logical deadlocks and atomicity violations are not always data races.

Static analysis, code reviews, stress testing, and language-level proofs are still necessary. When fixing reported issues, restore the happens-before relationship, don’t simply add volatile to objects or suppress warnings.

11. Reasoning Exercise

Consider the following two code snippets:

/* A: Only atomic count */
atomic_fetch_add_explicit(&completed, 1,
                          memory_order_relaxed);

If completed is merely a counter, relaxed ordering may hold. However, if the consumer treats an incremented counter as a signal that "the task result is ready," then ordering becomes insufficient.

/* B: Publish an immutable object */
initialize(object);
atomic_store_explicit(&published, object,
                      memory_order_release);

/* another thread: object is a thread-local variable */
object = atomic_load_explicit(&published,
                              memory_order_acquire);
if (object != NULL) {
    use(object);
}

If acquire reads a pointer published by release, then the initialization write becomes visible to the consumer. But further questions remain:

• Does the object remain immutable after publication?
• How do multiple publishers compete with each other?
• When is the object reclaimed, and do readers still hold pointers to it?
• Is the atomic pointer type declared correctly in published?

Lock-free publication is often straightforward, but safe reclamation is significantly more challenging.

12. Summary

The C concurrency memory model transforms "it appears that a write happens before a read" into a verifiable relationship:

• A data race requires conflicting access, at least one non-atomic operation, and the absence of a happens-before relationship;
• Release/acquire semantics only synchronize when a read observes the corresponding release value;
• Relaxed memory ordering guarantees atomicity and modification order for individual objects, without publishing or synchronizing adjacent data;
• Seq_cst introduces a total order on seq_cst operations; mixed relaxed and ordinary accesses do not automatically join that order;
• Atomic types do not guarantee lock-free behavior and do not automatically preserve invariants across multiple fields;
• Mutexes, thread lifetimes, and condition variables all establish synchronization relationships;
• volatile is not a thread communication mechanism.

The next chapter, Assembly Fundamentals and Calling Conventions, explores how compilers translate object accesses, atomic operations, and function calls into actual instructions and ABI specifications of the target ISA.

Check the Language Rules

• C11 committee draft N1570: objects, restrict, atomics
• C23 draft N3096: allocation and initialization changes