9.2 Condition Variables, Semaphores, and Barrier
The threads already use a mutex to protect shared state, but that does not answer "whose turn is it to run?" A data-collection thread must wait for a non-empty queue, four analysis threads must wait for the current phase to finish, and only two workers may use a scarce device at once. Mutual exclusion alone often leaves threads holding a lock while repeatedly polling a condition, which wastes CPU cycles.
9.1 Threads, Data Races, and Mutual Exclusion protects shared invariants. This lesson covers three coordination patterns: waiting for a predicate, limiting concurrent resource use, and synchronizing threads at a phase boundary. A notification does not store business state; the state remains in a protected predicate.
A Condition Variable Does Not Store Notifications
The consumer must wait until the queue is non-empty. Repeatedly checking under the lock, unlocking, and sleeping adds polling overhead and delay. If it checks first and then enters sleep separately, the producer might issue a notification exactly between those two steps, causing the consumer to miss it entirely.
A condition variable implements a single atomic protocol for "release mutex and enter waiting state":
lock mutex
while predicate is false:
cond_wait(condition, mutex)
consume / update state
unlock mutexpthread_cond_wait returns successfully only after reacquiring the mutex. The caller must hold the lock before calling, otherwise the predicate check, wait, and producer update would not occur within the same synchronization protocol.
Notifications themselves are not queued as business events. signal wakes at least one currently blocked waiter, without letting the caller select which one, and broadcast means "all waiters should recheck." If no waiters are present at the time of notification, the notification has no subsequent effect. The actual state is preserved solely within the predicate protected by the mutex.
Why You Must Use while
After a wait returns, the predicate may still be false:
- POSIX allows spurious wakeups;
- After multiple consumers are woken, another thread might acquire the mutex first and remove the data;
- Broadcasts inherently wake up waiters who may not be able to continue;
- The program may later introduce additional paths that modify the predicate.
Thus, while is not merely a defense against a rare OS bug, it embodies the semantics of Mesa-style condition variables. When a thread is woken, it gains only the right to recontest the mutex; it does not receive a "guarantee" that the condition remains true.
A Complete Protocol for a Single-Slot Channel
The producer writes values 1–5, and the consumer sums them. Both occupied and closed are protected by the same mutex; the condition variable is used only for sleep and wake operations.
#define _POSIX_C_SOURCE 200809L
#include <stdbool.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
struct Channel {
pthread_mutex_t mutex;
pthread_cond_t changed;
bool occupied;
bool closed;
int value;
};
struct ConsumerArgument {
struct Channel *channel;
int sum;
};
static void check_pthread(const char *operation, int error) {
if (error != 0) {
fprintf(stderr, "%s: %s\n", operation, strerror(error));
abort();
}
}
static void *produce(void *raw_channel) {
struct Channel *channel = raw_channel;
for (int value = 1; value <= 5; value++) {
check_pthread("lock", pthread_mutex_lock(&channel->mutex));
while (channel->occupied) {
check_pthread("wait",
pthread_cond_wait(&channel->changed,
&channel->mutex));
}
channel->value = value;
channel->occupied = true;
check_pthread("broadcast", pthread_cond_broadcast(&channel->changed));
check_pthread("unlock", pthread_mutex_unlock(&channel->mutex));
}
check_pthread("lock", pthread_mutex_lock(&channel->mutex));
while (channel->occupied) {
check_pthread("wait",
pthread_cond_wait(&channel->changed, &channel->mutex));
}
channel->closed = true;
check_pthread("broadcast", pthread_cond_broadcast(&channel->changed));
check_pthread("unlock", pthread_mutex_unlock(&channel->mutex));
return NULL;
}
static void *consume(void *raw_argument) {
struct ConsumerArgument *argument = raw_argument;
struct Channel *channel = argument->channel;
check_pthread("lock", pthread_mutex_lock(&channel->mutex));
for (;;) {
while (!channel->occupied && !channel->closed) {
check_pthread("wait",
pthread_cond_wait(&channel->changed,
&channel->mutex));
}
if (!channel->occupied && channel->closed) {
break;
}
argument->sum += channel->value;
channel->occupied = false;
check_pthread("broadcast", pthread_cond_broadcast(&channel->changed));
}
check_pthread("unlock", pthread_mutex_unlock(&channel->mutex));
return NULL;
}
int main(void) {
struct Channel channel = {
.mutex = PTHREAD_MUTEX_INITIALIZER,
.changed = PTHREAD_COND_INITIALIZER,
.occupied = false,
.closed = false,
.value = 0,
};
struct ConsumerArgument consumer_argument = {
.channel = &channel,
.sum = 0,
};
pthread_t producer;
pthread_t consumer;
check_pthread("create consumer",
pthread_create(&consumer, NULL, consume,
&consumer_argument));
check_pthread("create producer",
pthread_create(&producer, NULL, produce, &channel));
check_pthread("join producer", pthread_join(producer, NULL));
check_pthread("join consumer", pthread_join(consumer, NULL));
printf("sum=%d\n", consumer_argument.sum);
check_pthread("destroy condition", pthread_cond_destroy(&channel.changed));
check_pthread("destroy mutex", pthread_mutex_destroy(&channel.mutex));
return consumer_argument.sum == 15 ? 0 : 1;
}This example has one producer and one consumer; the producer closes after the slot is drained. External closure and cancellation are outside its protocol. A pthread error aborts the entire demo process. Save the two complete programs as channel.c and barrier.c; compile with cc -std=c17 -O2 -Wall -Wextra -Werror -pthread channel.c -o channel, and use the corresponding names for the barrier.
Here broadcast simplifies the single-slot protocol; signal is sufficient when there is exactly one relevant waiter. When extending to a bounded queue, not_empty and not_full are typically used separately to reduce irrelevant wakeups. Before optimization, it's essential to ensure that every state transition wakes up all potential runnable participants.
Bind Timeouts to the Appropriate Clock
pthread_cond_timedwait uses the clock selected by the condition-variable attributes. If using wall clock, clock synchronization by administrators might cause the deadline to jump. When supported, configure the condition variable to CLOCK_MONOTONIC.
After timeout returns, the predicate must still be checked within the mutex. The arrival of the deadline and a state change from the producer might occur simultaneously. The API returning timeout does not necessarily mean the state remains false.
Spurious wakeups must not reset the timeout budget. Recomputing the full wait duration on each iteration can lead to unbounded total wait time. Instead, compute the absolute deadline once, then loop waiting.
A Semaphore Stores a Permit Count
A counting semaphore maintains a non-negative permit count:
sem_waitacquires a permit; if unavailable, blocks until one becomes available.sem_postreleases a permit, which may wake up a waiting thread.
It's suitable for limiting the number of threads that can simultaneously access a finite resource, such as exactly 8 database connections. Unlike a mutex, a semaphore typically lacks the semantic requirement that "the same owner must release it"; incorrectly incrementing post will artificially inflate the available capacity.
#define _POSIX_C_SOURCE 200809L
#include <errno.h>
#include <semaphore.h>
int perform_operation(void);
static int acquire_permit(sem_t *semaphore) {
while (sem_wait(semaphore) != 0) {
if (errno != EINTR) {
return -1;
}
}
return 0;
}
static int use_limited_resource(sem_t *semaphore) {
if (acquire_permit(semaphore) != 0) {
return -1;
}
int result = perform_operation();
int saved_errno = errno;
if (sem_post(semaphore) != 0 && result == 0) {
result = -1;
saved_errno = errno;
}
errno = saved_errno;
return result;
}This is an integration fragment. Supply perform_operation: it returns 0 on success, or -1 with errno set on failure. The semaphore must be initialized correctly, and the operation must return normally without cancellation. Every normal return path attempts post. If both the operation and post fail, this example preserves the operation error; resource management must also record the post failure instead of claiming that the permit was returned.
Some Unix platforms do not implement unnamed semaphores at all. Where supported, check process-shared capability separately; named semaphores have a different lifecycle. Cross-platform libraries should verify the target implementation and should not assume that Linux behavior represents the full POSIX specification.
A binary-semaphore protocol keeps the count at 0 or 1; initializing a POSIX sem_t to 1 does not enforce that upper bound. It also does not provide mutex semantics such as ownership, priority inheritance, or robust recovery. For protecting critical sections, prefer using a mutex. Use a semaphore only when expressing a limited resource quantity.
A Barrier Joins Workers at Phase Boundaries
Four workers can each process a portion of the data, but the next round of aggregation must wait until all participants have completed their current stage. A barrier tracks the number of participants from the current generation that have arrived; it does not represent how many permits are available for a scarce resource.
The predicate of a barrier is "the current generation has reached N participants." The last participant to arrive advances the generation and wakes up the other threads. If the barrier is reused, only the count (without the generation) can lead to ABA-style confusion: a waiter resuming from the previous generation might misread the next generation’s count.
#define _POSIX_C_SOURCE 200809L
#include <assert.h>
#include <pthread.h>
#include <stdatomic.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
enum { THREADS = 4, ROUNDS = 20 };
struct Barrier {
pthread_mutex_t mutex;
pthread_cond_t changed;
unsigned int participants;
unsigned int arrived;
unsigned int generation;
};
struct WorkerArgument {
struct Barrier *barrier;
};
static _Atomic int arrivals[ROUNDS];
static void check(int error, const char *operation) {
if (error != 0) {
fprintf(stderr, "%s: %s\n", operation, strerror(error));
abort();
}
}
static void barrier_wait(struct Barrier *barrier) {
check(pthread_mutex_lock(&barrier->mutex), "lock");
unsigned int generation = barrier->generation;
barrier->arrived++;
if (barrier->arrived == barrier->participants) {
barrier->arrived = 0;
barrier->generation++;
check(pthread_cond_broadcast(&barrier->changed), "broadcast");
} else {
while (generation == barrier->generation) {
check(pthread_cond_wait(&barrier->changed, &barrier->mutex),
"wait");
}
}
check(pthread_mutex_unlock(&barrier->mutex), "unlock");
}
static void *run_rounds(void *raw_argument) {
struct WorkerArgument *argument = raw_argument;
for (int round = 0; round < ROUNDS; round++) {
atomic_fetch_add_explicit(&arrivals[round], 1,
memory_order_relaxed);
barrier_wait(argument->barrier);
assert(atomic_load_explicit(&arrivals[round],
memory_order_relaxed) == THREADS);
}
return NULL;
}
int main(void) {
struct Barrier barrier = {
.mutex = PTHREAD_MUTEX_INITIALIZER,
.changed = PTHREAD_COND_INITIALIZER,
.participants = THREADS,
.arrived = 0,
.generation = 0,
};
struct WorkerArgument argument = {.barrier = &barrier};
pthread_t threads[THREADS];
for (int index = 0; index < THREADS; index++) {
check(pthread_create(&threads[index], NULL, run_rounds, &argument),
"pthread_create");
}
for (int index = 0; index < THREADS; index++) {
check(pthread_join(threads[index], NULL), "pthread_join");
}
check(pthread_cond_destroy(&barrier.changed), "cond_destroy");
check(pthread_mutex_destroy(&barrier.mutex), "mutex_destroy");
puts("all barrier rounds completed");
return 0;
}The participant count is fixed and positive. Each worker arrives exactly once per round and must not exit early; join all workers before destruction. For ordinary fixed-participant synchronization, use pthread_barrier_t where supported. It does not automatically solve missing participants, cancellation, or dynamic membership; choose a library with an explicit protocol for those requirements.
Read-Heavy Workloads Still Need Measurement
A read-write lock allows multiple readers or a single writer. It's suitable for workloads where the critical section is long enough to allow true parallelism and where writes are rare. However, in the following scenarios, a standard mutex might be a better choice:
- The critical section is very short, and the overhead of read-write bookkeeping exceeds the benefits of parallelism;
- Cache line contention remains high due to reader-count updates;
- Write frequency is not low;
- The policy leads to starvation of either readers or writers;
- The read path still modifies lazy cache or statistics, meaning it's not truly a read-only operation.
Fairness and writer preference are typically implementation-specific policies and should not be assumed. Upgrading a read lock to a write lock can lead to deadlocks unless the API explicitly supports it and the protocol properly handles contention.
Chapter 11 discusses spin/futex and lock implementation details; Chapter 16 explains atomic memory ordering. Here, the read-write lock is treated merely as a measurable strategy, not as a guarantee of a one-order-of-magnitude performance improvement.
Choose Primitives by First Writing Predicates
| Problem | Preferred Expression |
|---|---|
| A group of fields must be modified as a unit | mutex |
| Wait for a queue to be non-empty or not full | mutex + condition variable |
| At most N concurrent users | counting semaphore |
| N participants must meet at a stage | barrier |
| Long read, few writes | consider RW lock after benchmarking |
| Single counter or flag | C atomic, with memory order proven |
Multiple primitives can be combined, but each additional waiting relationship increases the complexity of deadlock avoidance, cancellation handling, and shutdown paths. A bounded queue implemented with a mutex and condition variable is generally easier to maintain and reason about than one simulated with three semaphores, especially in preserving invariants and ensuring close semantics.
Verify the Notification Protocol
- Change the single-slot channel's
whiletoif, add two consumers, and construct a timing sequence where the predicate is consumed first. - Selectively replace
broadcastwithsignalto illustrate why each position is sufficient or would miss a role. - Add cancellation and timeout support to the channel to ensure the producer does not permanently wait after the channel is closed.
- Remove the generation from the reusable barrier and look for errors that span across rounds.
- Use a semaphore to limit concurrent operations to three, and verify that operation success and failure both attempt to return a permit, distinguishing failure of sem_post itself.
- Compare mutex and RW lock performance on the same read-heavy map, reporting critical-section duration and starvation policy.
Runnable Threads Still Wait for a CPU
Synchronization primitives determine when a thread becomes blocked or runnable, but they don't dictate when that runnable thread will eventually get access to the CPU. The next chapter covers CPU Scheduling, comparing metrics like response time, turnaround time, fairness, and deadlines, rather than simply memorizing algorithm names.