1.3 Byte Order and Bit Masks

The central control console sends the same four-byte character to two old machines: one displays 0x12345678, while the other reads it backward. The data isn't corrupted, both machines simply didn't agree on whether to place the most significant byte first or last. Once these bytes leave the machines and travel along the magic relay paths, this disagreement becomes a protocol error.

The previous two lessons covered basic representations of bit patterns and text encoding. This lesson addresses two key issues at the boundary between machines: how multi-byte values are arranged in memory, and how bit masks can be used to extract and update specific bits. We're still in the heart of the computer system, but now we're approaching an interface that network protocols will repeatedly rely upon.

The Same Integer, Two Common Representations

Place the 32-bit integer 0x12345678 into four consecutive bytes. When viewed by address from low to high:

Address offset   +0  +1  +2  +3
Big-endian      12  34  56  78
Little-endian   78  56  34  12

In big-endian (big-endian) format, the most significant byte is stored at the lowest address; in little-endian (little-endian), the least significant byte is placed at the lowest address. This discussion concerns the representation of multi-byte values; a single byte has no internal byte order.

Byte order does not alter the abstract value of the integer. When a CPU writes or reads the value according to its native endianness, it still retrieves 0x12345678. The issue arises when two systems exchange multi-byte object representations without agreeing on a common format.

Internet protocols typically define multi-byte integer fields in network byte order, big-endian. Socket API functions such as htons, htonl, and their corresponding ntoh* functions handle conversions between host and network byte order. However, these functions only cover specific widths and cannot replace a complete serialization design.

Explicitly Encoding at Console Ends

A four-byte sequence of characters won't reveal native byte order if it only moves within a single machine. Only when it crosses between machines or protocol boundaries does the sender need to encode each byte individually, and the receiver must decode them according to the same agreed-upon convention. First, observe the native byte order of the local system, then produce a version that's independent of that native ordering.

The following C17 program performs two tasks: it reads an object's byte representation, and it encodes and decodes a 32-bit big-endian integer in a way that doesn't depend on the host's native byte order.

#include <inttypes.h>
#include <stdint.h>
#include <stdio.h>

static void encode_u32_be(uint32_t value, uint8_t output[4]) {
    output[0] = (uint8_t)(value >> 24);
    output[1] = (uint8_t)(value >> 16);
    output[2] = (uint8_t)(value >> 8);
    output[3] = (uint8_t)value;
}

static uint32_t decode_u32_be(const uint8_t input[4]) {
    return ((uint32_t)input[0] << 24)
         | ((uint32_t)input[1] << 16)
         | ((uint32_t)input[2] << 8)
         | (uint32_t)input[3];
}

int main(void) {
    uint32_t value = UINT32_C(0x12345678);
    const uint8_t *native = (const uint8_t *)&value;
    uint8_t encoded[4];

    printf("native:");
    for (size_t index = 0; index < sizeof value; index++) {
        printf(" %02" PRIX8, native[index]);
    }
    putchar('\n');

    encode_u32_be(value, encoded);
    printf("big-endian: %02" PRIX8 " %02" PRIX8 " %02" PRIX8 " %02" PRIX8 "\n",
           encoded[0], encoded[1], encoded[2], encoded[3]);
    printf("decoded: 0x%08" PRIX32 "\n", decode_u32_be(encoded));
    return 0;
}

Regardless of the native byte order of the system, the last two lines should display:

big-endian: 12 34 56 78
decoded: 0x12345678

The behavior observed via uint8_t is intuitive across common implementations; the C standard's most general rule is that any object can be inspected using unsigned char *. Here, using uint8_t also implicitly assumes that the implementation provides a precise 8-bit unsigned type.

Never write entire C structs directly to files or over the network. Structs may contain padding, field alignment, integer width, and native byte order differences. A stable format should specify width, order, valid ranges, and missing value semantics for each field individually, and then explicitly encode those fields.

Masks Enable Local State Modifications

Bit masks pack multiple boolean states into a single unsigned integer. Suppose bits 0, 1, and 2 represent read, write, and execute permissions respectively:

#include <stdbool.h>
#include <stdint.h>

enum Permission {
    PERM_READ  = UINT32_C(1) << 0,
    PERM_WRITE = UINT32_C(1) << 1,
    PERM_EXEC  = UINT32_C(1) << 2
};

uint32_t permissions = PERM_READ | PERM_WRITE;
permissions |= PERM_EXEC;                       /* Set */
permissions &= ~((uint32_t)PERM_WRITE);         /* Clear */
permissions ^= PERM_EXEC;                       /* Reverse */
bool can_read = (permissions & PERM_READ) != 0; /* Test */

Four fundamental operations are worth memorizing:

Purpose	Expression	Meaning
Set certain bits	`value	= mask`
Clear certain bits	value &= ~mask	Positions where the mask is 1 are set to 0
Flip certain bits	value ^= mask	Positions where the mask is 1 are inverted
Test any one bit	(value & mask) != 0	At least one of the target bits is set

To ensure that all bits in the mask are present, compare (value & mask) == mask. The presence of any single bit versus the presence of all bits is a common semantic confusion in interface design.

Unix permissions 0755 can be read in three groups of three bits: the owner has 7 = 111, group members and others have 5 = 101, resulting in the display rwxr-xr-x. The leading 0 is a traditional C octal literal; in command-line chmod 755 file, tools interpret the parameter according to permission syntax.

C's Shift Operations Are Not Boundary-Free Multiplication or Division

Bitwise operations prefer unsigned types and require checking the shift amount first. For a w-bit integer, the shift amount must satisfy 0 <= count < w; negative values or shift amounts equal to or greater than the width trigger undefined behavior.

It's also important to distinguish between these cases:

• Unsigned right shifts fill the high bits with zeros, resulting in well-defined behavior;
• For negative signed integers, right shifts in C17 are implementation-defined, so it's not safe to assume all platforms perform arithmetic right shifts;
• Signed positive integers can only be safely left-shifted if the result fits within the representable range and all other conditions are met; overflow leads to undefined behavior;
• Unsigned left shifts are computed modulo 2^w, but bits shifted out are lost;
• When writing to the nth bit, use an unsigned constant matching the target type, such as UINT32_C(1) << n, and ensure that n < 32 first.

Therefore, x << 1 cannot be unconditionally substituted for x * 2, and x >> 1 cannot be universally replaced with signed division by 2. Compilers will perform appropriate strength reductions, but source code should first express correct semantics.

The classic XOR swap is not worth using in general code. If two variables point to the same object, the naive version will zero out the values; even in the absence of aliasing, it is typically neither clearer nor faster than using a temporary variable.

A Safe Bit Field Interface

Concentrating checks in functions is more reliable than manually writing shifts at every call site:

#include <stdbool.h>
#include <stdint.h>

static bool set_bit_u32(uint32_t *value, unsigned int bit) {
    if (value == NULL || bit >= 32U) {
        return false;
    }
    *value |= UINT32_C(1) << bit;
    return true;
}

static bool test_bit_u32(uint32_t value, unsigned int bit, bool *result) {
    if (result == NULL || bit >= 32U) {
        return false;
    }
    *result = (value & (UINT32_C(1) << bit)) != 0;
    return true;
}

In real projects, you can also use UINT32_WIDTH (C23) or derive width from types or configuration. Here, the width is fixed at 32 bits because the interface explicitly selects a type that is exactly 32 bits when present, namely uint32_t.

Serialization Boundary Checklist

• Define the format first, then write the code: explicitly specify field width, signedness, byte order, text encoding, and version.
• Validate buffer length before decoding, and check value ranges after reconstructing fields.
• Avoid using unaligned pointer casts to read byte buffers; instead, assemble bytes step by step or copy them into a suitable object before applying protocol-specific conversions.
• Keep bitmask constants and the values they operate on in the same unsigned width to prevent unintended overflow into high bits due to integer promotion.
• Test 0, all-ones values, most significant bit states, edge-case shifts, and round-trip behavior decode(encode(x)) == x.

Hands-on Verify Machine Boundaries

1. Run the example to determine whether the native byte representation of uint32_t on this machine is big-endian or little-endian.
2. Implement encode_u32_le and decode_u32_le, and perform round-trip tests using 0, 1, and 0xFFFFFFFF.
3. Write an interface that sets a bit field of [offset, offset + width), and handle overflow and width == 32 conditions.
4. Explain why directly converting a network buffer into uint32_t * may simultaneously introduce alignment, aliasing, and endianness issues.
5. Compare two test expressions: "any permission exists" versus "all permissions exist".

How Integers Interpret the Most Significant Bit

You now have a safe way to arrange bytes, extract, and update individual bits. The next chapter discusses integers, overflow, and floating-point representation: how the same width interprets positive and negative numbers, why unsigned arithmetic wraps around, and why floating-point numbers cannot precisely represent many decimal fractions.