Proven C Book←↑→

53 Working with bits — idioms and traps

What to know first

chapter 6, Representing integers · two’s complement and what a shift means
chapter 29, Integer operations · the bitwise operators and the “do it on unsigned” rule
chapter 52, Expressions and operators · precedence, and each operator’s contract

Looking back

chapter 29 gave the rule “do bit work on unsigned types”. But why is the sign the problem — is a bit not just a bit?

A. A bit is just a bit, but C always sees a bunch of bits as a value of some type. If that type is signed, the topmost bit acquires the special meaning “sign”, and from there shifting and division behave differently (chapter 6) and some combinations fall outside the contract entirely (chapter 52). Choosing unsigned is removing that special meaning so that bits are really bits. This chapter is about what can be built on top of that.

The need for this chapter, and its context

The book has passed bits three times — their meaning in chapter 6, the operators in chapter 29, the contracts in chapter 52. What has never been gathered in one place is “so how does one actually use them”. Setting flags, building masks, rounding up an address, working a bitmap: this work is everywhere in embedded code, graphics, compression and data structures. Part IX is the part of “what you already learned, again and to the end”, so here is the place.

By the end of this chapter

We begin with the five basic moves (set, clear, flip, test, replace a field), then how masks are built and where they fall off a cliff — a shift count equal to the width, outside the contract. Ten idioms follow, each with why it works, and then the names C23 gave them (<stdbit.h>). After that seven rules, the accidents that happen around bit work, and a look at what the compiler already knows — in the machine code it emits.

The questions this chapter answers

  1. Would bit fields (chapter 49) not spare us all this shifting and masking?

53.1 The five basic moves#

Working with bits comes down to five things. Everything else is a combination.

The five moves are gathered in one table further down; before it, the letters that table uses. All five moves work on v, and all but the test change it.

letterwhat it is
vthe value being changed. The starting value, and where the result lands
nthe position number of the bit in question. The rightmost is number 0
mthe mask — a value with 1s marking “which places do we touch”
sthe position where the field starts (where the 1s of m begin)
xthe new value for that field, still sitting at position 0, not yet shifted

Table 53.1 — The letters used in the five-moves table

First, what 1u << n actually is. 1u has only the rightmost place turned on, and << n pushes it n places to the left. So 1u << n is the value with exactly one place on: position n, counting the right end as 0 — numerically 2𝑛. What matters is that the counting starts at 0, not 1: 1u << 0 is the rightmost bit (value 1) and 1u << 3 is the fourth from the right (value 8).

position    7  6  5  4  3  2  1  0     <- positions count from the right, from 0
1u          0  0  0  0  0  0  0  1     value 1
1u << 3     0  0  0  0  1  0  0  0     value 8   (the 1 moved three places left)
1u << 7     1  0  0  0  0  0  0  0     value 128

So “bit n” means the place n steps to the left of the right end. 1u << n turns on exactly that place, and it is the raw material of all five moves below.

whathow it is writtenwhy it worksremember
setv |= 1u << nOR gives 1 if either side is 1 — other places OR with 0 and staydoing it twice changes nothing
clearv &= ~(1u << n)AND gives 0 if either side is 0 — the mask is 0 only thereforgetting the ~ is a common slip
flipv ^= 1u << nXOR inverts exactly where the mask has 1stwice returns to the start
testv & (1u << n)only that place survives★ the result is that place’s value (2𝑛), not 1
replace a fieldv = (v & ~m) | ((x << s) & m)clear the places m marks, then lay the shifted x on toporder matters — clear, then insert

Table 53.2 — The five moves on a single bit

examples-en/bitwise/bitops.c

// The five basic moves for working with bits.
// All of them on unsigned types, with the width written into the type name.
#include <stdio.h>
#include <stdint.h>
#include <inttypes.h>

static void show(const char *label, uint32_t v)
{
    printf("  %-28s 0x%08" PRIX32 "\n", label, v);
}

int main(void)
{
    uint32_t v = 0x00F0'0000;          // C23's digit separator
    show("start", v);

    // (1) set --- put a 1 in that place
    v |= UINT32_C(1) << 3;
    show("set bit 3        (|=)", v);

    // (2) clear --- AND with a mask that is 0 only there
    v &= ~(UINT32_C(1) << 20);
    show("clear bit 20     (&= ~)", v);

    // (3) test --- the result is 0 or *the value of that bit*, not 1
    printf("  is bit 3 on?  %s\n", (v & (UINT32_C(1) << 3)) ? "yes" : "no");
    printf("  the value of v & (1<<3) is %" PRIu32 ", not 1\n",
           v & (UINT32_C(1) << 3));

    // (4) flip --- XOR inverts exactly where the mask has 1s
    v ^= UINT32_C(0xF0);
    show("flip bits 4..7   (^=)", v);

    // (5) replace a field --- clear it, then shift the new value in
    // Treat bits 8..15 as one eight-bit field.
    uint32_t field = 0xAB;
    uint32_t mask  = UINT32_C(0xFF) << 8;
    v = (v & ~mask) | ((field << 8) & mask);
    show("put 0xAB into bits 8..15", v);

    // Reading it back is the reverse --- shift down, then mask
    printf("  reading it back gives 0x%02" PRIX32 "\n", (v >> 8) & 0xFF);
    return 0;
}

Output

  start                        0x00F00000
  set bit 3        (|=)        0x00F00008
  clear bit 20     (&= ~)      0x00E00008
  is bit 3 on?  yes
  the value of v & (1<<3) is 8, not 1
  flip bits 4..7   (^=)        0x00E000F8
  put 0xAB into bits 8..15     0x00E0ABF8
  reading it back gives 0xAB

Only the last row of the five-moves table, replacing a field, has four letters, and the end of the demo is exactly that case. There, bits 8..15 of v are treated as one field and 0xAB is put into it — m is the mask with just those eight places set (0xFF << 8), s is 8, the position the field starts at, and x is the value to insert, 0xAB. Read in two pieces:

OR the two together and you have “that field replaced, nothing else touched”. Reading it back reverses the steps — (v >> s) & (m >> s), which in the demo is (v >> 8) & 0xFF.

The fourth row of the five-moves table, the test, is where beginners are caught most often. The value of v & (1u << 3) is 8, not 1. Put straight into an if it works (only truth matters there), but stored or compared it goes wrong.

Counter-example. Comparing a bit test against 1

if ((flags & FLAG_READY) == 1) { … }   /* almost always false */

If FLAG_READY is 1u << 5, the test yields 32, not 1. There are two ways out — if only truth is needed, do not compare at all: if (flags & FLAG_READY); and if a 0/1 really is needed, write !!(flags & FLAG_READY) or (flags & FLAG_READY) != 0.

53.1.1 Building masks, and the shift-width cliff#

A mask records “which places are we looking at”. Three ways of building one are enough.

whathowwatch out
a single bit1u << nn must satisfy 0 <= n < width
the low n bits(1u << n) - 1u★ outside the contract when n equals the width
w bits starting at s((1u << w) - 1u) << ss + w must not exceed the width

Table 53.3 — Three masks

Beware the cliff in the second row. Turning on all 32 bits of a uint32_t by writing (1u << 32) - 1 shifts by the full width, which is outside the contract (chapter 52). When the whole width is wanted, choose an expression with no shift at all — UINT32_MAX, or ~UINT32_C(0). If it must be general, split the case: w == 32 ? ~UINT32_C(0) : ((UINT32_C(1) << w) - 1).

53.2 A set of flags — the commonest use#

Holding several yes/no answers in one integer is the first use of bit work.

enum {
    OPT_READ    = 1u << 0,
    OPT_WRITE   = 1u << 1,
    OPT_APPEND  = 1u << 2,
    OPT_BINARY  = 1u << 3,
};

unsigned opts = OPT_READ | OPT_BINARY;      /* start with two on */

opts |= OPT_WRITE;                          /* turn one more on */
opts &= ~OPT_BINARY;                        /* turn one off */

if (opts & OPT_WRITE)          { /* writing is on */ }
if ((opts & (OPT_READ | OPT_WRITE)) == (OPT_READ | OPT_WRITE))
                               { /* *both* are on */ }
if (opts & (OPT_READ | OPT_WRITE))
                               { /* *either* is on */ }

Look at the difference between the last two. Either is a plain AND, but both must ask “is it equal to the mask”. Mixing these two up is a very common bug.

★ An enum is a good place to name these values — the debugger shows the names and the values sit together. But an enumeration constant has type int (still the default in C23), so a place like 1u << 31, beyond the signed width, belongs in a macro or an unsigned constant.

53.3 The idioms — and why they work#

The expressions below have been polished over half a century. Rather than memorising them, follow why each works once and you can rebuild it when you need it.

what it giveshow it is writtenwhy
is it even(x & 1u) == 0the lowest place is the ones place
keep only the lowest set bitx & (0u - x)0u - x is ~x + 1 — below that bit nothing changes, above it everything flips
clear the lowest set bitx & (x - 1u)x - 1 turns that bit into 0 and everything below it into 1s
is it a power of twox != 0 && (x & (x - 1u)) == 0it means exactly one bit is set
round up to a multiple of a (a power of two)(x + a - 1u) & ~(a - 1u)overshoot first, then cut the low bits off
round down to a multiple of ax & ~(a - 1u)just discard the low bits
remainder (unsigned, a a power of two)x & (a - 1u)keep only the low bits
count the 1 bitswhile (x) { x &= x - 1; n++; }“clear the lowest set bit”, repeated — it loops once per set bit
rotate left(x << (n & 31)) | (x >> ((32 - n) & 31))the bits pushed off one end come back in the other. The & 31 guards the n == 0 cliff
choose without branching(x & -(uint32_t)c) | (y & ~-(uint32_t)c)for c of 0 or 1 the mask becomes all-zeros or all-ones

Table 53.4 — The idioms one meets most often

examples-en/bitwise/idioms.c

// The idioms one meets most often --- and why they work.
#include <stdio.h>
#include <stdint.h>
#include <inttypes.h>

// Keep only the lowest set bit of x.
// On unsigned values 0u - x equals (~x + 1). Below the lowest set bit nothing
// changes and above it everything flips, so the AND leaves just that one bit.
static uint32_t lowest_one(uint32_t x) { return x & (0u - x); }

// Clear the lowest set bit. x-1 turns that bit into 0 and all below it into 1s.
static uint32_t clear_lowest_one(uint32_t x) { return x & (x - 1u); }

// Is it a power of two --- the same question as "is exactly one bit set".
static int is_power_of_two(uint32_t x) { return x != 0 && (x & (x - 1u)) == 0; }

// Round up to a multiple of a, where a is a power of two.
static uint32_t align_up(uint32_t x, uint32_t a) { return (x + a - 1u) & ~(a - 1u); }

// Rotate left by n. Safe when n == 0 as well --- see below.
static uint32_t rotate_left(uint32_t x, unsigned n)
{
    return (x << (n & 31)) | (x >> ((32u - n) & 31));
}

// How many 1 bits --- loops once per set bit (Kernighan's way).
static unsigned count_ones(uint32_t x)
{
    unsigned n = 0;
    while (x) { x &= x - 1u; n++; }
    return n;
}

int main(void)
{
    uint32_t x = 0x0000'0B40;          // …1011 0100 0000
    printf("x = 0x%08" PRIX32 "\n", x);
    printf("  lowest set bit  (x & -x)     = 0x%08" PRIX32 "\n", lowest_one(x));
    printf("  clear lowest    (x & (x-1))  = 0x%08" PRIX32 "\n", clear_lowest_one(x));
    printf("  number of 1 bits             = %u\n", count_ones(x));

    puts("powers of two:");
    for (uint32_t v = 0; v <= 5; v++)
        printf("  %" PRIu32 " -> %s\n", v, is_power_of_two(v) ? "yes" : "no");
    printf("  1024 -> %s, 1000 -> %s\n",
           is_power_of_two(1024) ? "yes" : "no", is_power_of_two(1000) ? "yes" : "no");

    puts("rounding up to a multiple of 16:");
    for (uint32_t v = 0; v <= 33; v += 16)
        printf("  align_up(%2" PRIu32 ", 16) = %" PRIu32 "\n", v, align_up(v, 16));
    printf("  align_up(17, 16) = %" PRIu32 "\n", align_up(17, 16));

    puts("rotation keeps every bit --- nothing falls off the end:");
    printf("  rotate_left(0x80000001, 1) = 0x%08" PRIX32 "\n", rotate_left(0x8000'0001u, 1));
    printf("  rotate_left(0x80000001, 0) = 0x%08" PRIX32 "  (n = 0 is safe here)\n",
           rotate_left(0x8000'0001u, 0));
    return 0;
}

Output

x = 0x00000B40
  lowest set bit  (x & -x)     = 0x00000040
  clear lowest    (x & (x-1))  = 0x00000B00
  number of 1 bits             = 4
powers of two:
  0 -> no
  1 -> yes
  2 -> yes
  3 -> no
  4 -> yes
  5 -> no
  1024 -> yes, 1000 -> no
rounding up to a multiple of 16:
  align_up( 0, 16) = 0
  align_up(16, 16) = 16
  align_up(32, 16) = 32
  align_up(17, 16) = 32
rotation keeps every bit --- nothing falls off the end:
  rotate_left(0x80000001, 1) = 0x00000003
  rotate_left(0x80000001, 0) = 0x80000001  (n = 0 is safe here)

Look closely at the rotation. The form usually seen is (x << n) | (x >> (32 - n)), and when n is 0 the right-hand side becomes x >> 32, outside the contract. The pair of & 31 in the demo blocks that cliff, and costs nothing — as we will see, the compiler recognises this shape and folds it into a single rotate instruction.

A common misconception. Swapping with XOR and no temporary is faster

*a ^= *b;  *b ^= *a;  *a ^= *b;      /* do not */

Two things are wrong. First, it is slower. Counting the machine code this machine’s gcc emits at -O2: the version with a temporary is four instructions (two loads, two stores) while the XOR version is six, and three of those wait on the one before — a dependency chain that cannot proceed in parallel. A trick from an age of scarce registers is a loss today.

Second, it gives wrong answers. If both arguments point at the same place (swap(&v, &v)), the first statement, *a ^= *b, zeroes it and the value is gone. The version with a temporary has no such case.

53.4 C23 — the idioms get names#

Many of the idioms above are in fact one machine instruction. The problem was that for a long time there was no standard way to ask for it, so each compiler kept its own name — __builtin_popcount, _BitScanForward — and portable code wrote the idiom out by hand. C23 tidied this up: <stdbit.h>.

functionwhat it giveswritten by hand
stdc_count_oneshow many bits are 1while (x) { x &= x-1; n++; }
stdc_count_zeroshow many bits are 0width minus the count of 1 bits
stdc_leading_zeroshow many zeros run down from the top—
stdc_trailing_zeroszeros running up from the bottom—
stdc_first_leading_oneposition of the first 1 from the top (1-based, 0 if none)—
stdc_first_trailing_oneposition of the first 1 from the bottom—
stdc_bit_widthbits needed to hold the valuewhile (x) { x >>= 1; n++; }
stdc_bit_floor · stdc_bit_ceilnearest power of two at or below / above—
stdc_has_single_bitis it a power of twox && !(x & (x-1))

Table 53.5 — What <stdbit.h> named (there are uc/us/ui/ul/ull variants too)

examples-en/bitwise/stdbit.c

// C23 gave the idioms names --- <stdbit.h>.
// Put the hand-written version and the standard function side by side and check.
#include <stdio.h>
#include <stdint.h>
#include <inttypes.h>
#include <stdbit.h>

static unsigned count_by_hand(uint32_t x)
{
    unsigned n = 0;
    while (x) { x &= x - 1u; n++; }
    return n;
}

static unsigned width_by_hand(uint32_t x)          // bits needed to hold x
{
    unsigned n = 0;
    while (x) { x >>= 1; n++; }
    return n;
}

int main(void)
{
    printf("__STDC_VERSION_STDBIT_H__ = %ld\n", (long)__STDC_VERSION_STDBIT_H__);
    printf("byte order is %s\n",
           __STDC_ENDIAN_NATIVE__ == __STDC_ENDIAN_LITTLE__ ? "little endian"
           : __STDC_ENDIAN_NATIVE__ == __STDC_ENDIAN_BIG__  ? "big endian"
                                                            : "neither");

    uint32_t samples[] = { 0u, 1u, 0x0000'0B40u, 300u, 0xFFFF'FFFFu };
    puts("value       ones(hand/std)  width(hand/std)  bit_ceil   leading zeros");
    for (size_t i = 0; i < sizeof samples / sizeof samples[0]; i++) {
        uint32_t x = samples[i];
        printf("0x%08" PRIX32 "   %2u / %-2u        %2u / %-2u         %-10" PRIu32 " %u\n",
               x,
               count_by_hand(x), (unsigned)stdc_count_ones(x),
               width_by_hand(x), (unsigned)stdc_bit_width(x),
               (uint32_t)stdc_bit_ceil(x),
               (unsigned)stdc_leading_zeros(x));
    }

    // The position-finding functions count from 1, and 0 means "there is none".
    printf("first_leading_one(0x00F0) = %u  (counted from the top, 1-based)\n",
           (unsigned)stdc_first_leading_one(UINT32_C(0x00F0)));
    printf("first_trailing_one(0x00F0) = %u  (counted from the bottom)\n",
           (unsigned)stdc_first_trailing_one(UINT32_C(0x00F0)));
    printf("first_trailing_one(0) = %u  (zero means 'there is none')\n",
           (unsigned)stdc_first_trailing_one(UINT32_C(0)));

    // The power-of-two test got a name too.
    printf("has_single_bit(1024) = %d, has_single_bit(1000) = %d\n",
           (int)stdc_has_single_bit(UINT32_C(1024)),
           (int)stdc_has_single_bit(UINT32_C(1000)));
    return 0;
}

Output

__STDC_VERSION_STDBIT_H__ = 202311
byte order is little endian
value       ones(hand/std)  width(hand/std)  bit_ceil   leading zeros
0x00000000    0 / 0          0 / 0          1          32
0x00000001    1 / 1          1 / 1          1          31
0x00000B40    4 / 4         12 / 12         4096       20
0x0000012C    4 / 4          9 / 9          512        23
0xFFFFFFFF   32 / 32        32 / 32         0          0
first_leading_one(0x00F0) = 25  (counted from the top, 1-based)
first_trailing_one(0x00F0) = 5  (counted from the bottom)
first_trailing_one(0) = 0  (zero means 'there is none')
has_single_bit(1024) = 1, has_single_bit(1000) = 0

Three things to know when using <stdbit.h>: how positions are counted, overflow, and byte order.

First, the position functions count from 1. When stdc_first_trailing_one returns 5 it means “the fifth bit from the bottom”, the 1u << 4 place. Zero means “there is no such bit”, so these give a safe answer even for a value of 0 — unlike most hand-written idioms, which fall apart there.

Second, stdc_bit_ceil can overflow. If the result does not fit the type, the behaviour is outside the contract. That the demo printed 0 for 0xFFFFFFFF is what this machine did, not a promise of the standard.

Third, this header also tells you the byte order. Compare __STDC_ENDIAN_NATIVE__ against __STDC_ENDIAN_LITTLE__ and __STDC_ENDIAN_BIG__ (chapter 3). It is the first standard C way of asking about endianness.

Platform note. There are still places where it cannot be used

<stdbit.h> is a C23 header, so the compiler has to provide it. On this book’s verifying machine gcc 14 does, and __STDC_VERSION_STDBIT_H__ reads 202311. Where older tools must also be supported, the practical compromise is to branch on #if __has_include(<stdbit.h>) and keep the idioms of the previous section on the other side.

And a standard function does not always become one machine instruction. On this machine stdc_count_ones compiles to a library call with default settings, and only folds into a single popcnt once the compiler is told the machine has that instruction (-mpopcnt). The standard unified the name, not the speed.

53.5 Seven rules for working with bits#

rulewhy
do it on unsignedit removes the special meaning of the sign bit. Left-shifting a negative is outside the contract; right-shifting one is implementation-defined (chapters 29 and 52)
write the width into the typethe width of unsigned varies by machine; uint32_t says 32
put a suffix on constants1 << 31 is an int and outside the contract; 1u << 31 is fine. For 64-bit places, UINT64_C(1)
do not forget promotioneven uint8_t operands are widened to int first — flipping and comparing surprise people
check the shift count0 <= n < width, where the width is sizeof(x) * CHAR_BIT
be generous with parentheses&, | and ^ bind more weakly than the comparisons (chapter 52)
give it a nameFIELD_GET(v, KIND) reads better in six months than (v >> 8) & 0xFF

Table 53.6 — Rules for working with bits

The fourth rule is the one that hides. In the flesh:

examples-en/bitwise/promote.c

// The place bit work burns people most often --- integer promotion.
// Anything narrower than int is widened to int before the operation, so what
// you thought was "eight bits flipped" comes out thirty-two bits wide.
#include <stdio.h>
#include <stdint.h>
#include <inttypes.h>

int main(void)
{
    uint8_t c = 0x0F;

    // The type of ~c is int, not uint8_t.
    printf("c             = 0x%02X\n", c);
    printf("~c            = 0x%08X   <- not eight bits\n", (unsigned)~c);
    printf("(uint8_t)~c   = 0x%02X         <- narrow it back yourself\n",
           (unsigned)(uint8_t)~c);
    // Writing (~c == 0xF0) here is stopped by gcc with -Wsign-compare.
    printf("((uint8_t)~c == 0xF0) is %s\n", ((uint8_t)~c == 0xF0) ? "true" : "false");

    // Where char is signed, a byte of 0x80 or more becomes negative when widened,
    // which is why bytes are held in unsigned char.
    char signed_byte = (char)0x80;
    unsigned char plain_byte = 0x80;
    printf("a char holding 0x80, widened  : %d\n", (int)signed_byte);
    printf("an unsigned char holding 0x80 : %d\n", (int)plain_byte);
    printf("masking with 0xFF fixes it    : %d\n", (int)(signed_byte & 0xFF));

    // A mask has a width too --- the width of its type.
    uint64_t wide = 0xFFFF'FFFF'FFFF'FFFFu;
    printf("wide & ~0u          = 0x%016" PRIX64 "   <- ~0u is 32 bits wide\n",
           wide & ~0u);
    printf("wide & ~UINT64_C(0) = 0x%016" PRIX64 "\n", wide & ~UINT64_C(0));
    return 0;
}

Output

c             = 0x0F
~c            = 0xFFFFFFF0   <- not eight bits
(uint8_t)~c   = 0xF0         <- narrow it back yourself
((uint8_t)~c == 0xF0) is true
a char holding 0x80, widened  : -128
an unsigned char holding 0x80 : 128
masking with 0xFF fixes it    : 128
wide & ~0u          = 0x00000000FFFFFFFF   <- ~0u is 32 bits wide
wide & ~UINT64_C(0) = 0xFFFFFFFFFFFFFFFF

Apply ~ to a uint8_t and the result is 0xFFFFFFF0, because c is widened to int before ~ is computed (chapter 30). Getting back to eight bits means narrowing again. For the same reason ~c == 0xF0 is false — and happily gcc points at that spot: comparison of promoted bitwise complement of an unsigned value with constant.

The last two lines of the demo’s output have the same root. ~0u is not “all ones” but “all ones as wide as unsigned”, so ANDing it with a 64-bit value wipes the upper half. A mask has a width too.

53.6 The accidents that happen around bit work#

accidentsymptomwho catches itthe cure
flags & MASK == 0always the same answer — == binds first-Wparentheses(flags & MASK) == 0
1 << 31 on an intoutside the contract — the answer shifts with optimisation-fsanitize=undefined1u << 31
x >> 32 (at or past the width)outside the contract — 0 on one machine, x on anotherthe compiler warns for constantscheck the count, or split the shift
rotation with n == 0becomes x >> 32, outside the contractnobodyadd the & 31
~ on a narrow typethought to be 8 bits, comes out 32sometimes -Wsign-comparenarrow it back: (uint8_t)~c
handling bytes in a char0x80 and above turn negativenobodyuse unsigned char, or & 0xFF
>> 1 instead of division on a signed valuea different answer for negativesnobodywrite division as /
~0u used as a 64-bit maskthe upper half is wipednobody~UINT64_C(0)

Table 53.7 — Accidents around bit work

The seventh row is the trap left behind by the old advice that “shifting is fast division”. It holds for unsigned values, but a shift gives a different answer on signed ones.

In practice. Is −7 halved −3 or −4?

C’s division truncates toward zero (chapter 29) while an arithmetic right shift truncates downward. So -7 / 2 is -3 and -7 >> 1 is -4. The remainder goes the same way — -7 % 8 is -7 while -7 & 7 is 1.

The difference shows up only as a different value — no error, no warning — which is how it slips by. Computing an array index, snapping a coordinate to a grid, putting a hash into a bucket — nothing happens at all for as long as no negative arrives.

And the trick no longer pays. The compiler already turns x / 2 into a shift, with the correction a negative needs so the answer stays right. If division was meant, write division.

53.7 Before optimising by hand#

Many bit tricks were born from the premise “the compiler cannot do this, so a person must”. It is worth checking whether that premise still holds. Here is what this machine’s gcc emits at -O2.

written in the sourcemachine code emittedmeaning
(x << n) | (x >> ((32 - n) & 31))a single rolit recognises the rotation idiom
x & (0u - x)neg + and; with BMI1 enabled, one blsiwhere the instruction exists, it is used
while (x) { x &= x-1; n++; }with -mpopcnt, a single popcnt★ it recognises the whole loop and folds it
stdc_count_ones(x)a library call by default; one popcnt with -mpopcnta standard name does not make an instruction
swapping with XORsix instructions and a dependency chain (four with a temporary)an old trick that became a loss

Table 53.8 — What the compiler already knows — checked on this machine

The third row is the heart of the table. Kernighan’s counting loop is code a person wrote “cleverly”, and the compiler sees through to the intent and replaces it with one instruction. If the trick and the plain version compile to the same machine code, the only difference left is readability.

★ So the order is this. (1) Write it so the meaning shows. (2) Measure (chapter 13). (3) Only if that is not enough, reach for the trick — and leave a comment saying why it is written that way.

53.8 In the flesh — where bits are used#

placewhat it doeswhere in this book
hardware registersread and write several fields inside one wordchapters 98 and 114
flag setsseveral options in one integerthis chapter
character encodingsassembling and taking apart UTF-8 byteschapter 8
colours and pixelsR, G, B and A packed into one word—
bitmaps (bit sets)a million yes/no answers in 15,625 64-bit wordsthis chapter
hashes and random numbersmixing bits with XOR and shiftschapter 107
compression and codingreading and writing bit by bit—

Table 53.9 — Where bit work actually appears

UTF-8 is a good specimen. Splitting one code point into several bytes is, in the end, shifting and masking (chapter 8).

/* U+0800 to U+FFFF become three bytes --- 1110xxxx 10xxxxxx 10xxxxxx */
out[0] = (unsigned char)(0xE0u | (cp >> 12));          /* top four bits */
out[1] = (unsigned char)(0x80u | ((cp >> 6) & 0x3Fu)); /* middle six */
out[2] = (unsigned char)(0x80u | (cp & 0x3Fu));        /* bottom six */

0x3F is the mask meaning “only the low six bits”, while 0x80 and 0xE0 are markers saying what kind of byte this is. The five moves of the first section, used exactly as they were given.

And another shape met often in practice is the bitmap.

examples-en/bitwise/bitset.c

// A bitmap --- the shape bit work most often takes in real programs.
// There is no reason to spend a million bytes on a million true/false answers.
#include <stdio.h>
#include <stdint.h>
#include <string.h>
#include <stdbit.h>

#define BITS_PER_WORD 64
#define WORDS(n)      (((n) + BITS_PER_WORD - 1) / BITS_PER_WORD)

#define N 1000
static uint64_t set[WORDS(N)];

// Which word, and which bit inside it, does index i live in?
// The word size is a power of two, so the division and remainder become a shift and a mask.
static size_t   word_of(size_t i) { return i >> 6; }        // i / 64
static unsigned bit_of (size_t i) { return i & 63u; }       // i % 64

static void bit_set   (size_t i) { set[word_of(i)] |=  UINT64_C(1) << bit_of(i); }
static void bit_clear (size_t i) { set[word_of(i)] &= ~(UINT64_C(1) << bit_of(i)); }
static int  bit_test  (size_t i) { return (set[word_of(i)] >> bit_of(i)) & 1u; }

static size_t bit_count(void)
{
    size_t total = 0;
    for (size_t w = 0; w < WORDS(N); w++)
        total += stdc_count_ones(set[w]);          // one whole word at a time
    return total;
}

int main(void)
{
    printf("%d flags need %zu bytes as a bitmap, %d bytes as one char each\n",
           N, sizeof set, N);

    // The sieve of Eratosthenes --- the textbook use of a bitmap
    memset(set, 0, sizeof set);
    for (size_t i = 2; i < N; i++)
        bit_set(i);                                // everything is "maybe prime" to start
    for (size_t p = 2; p * p < N; p++)
        if (bit_test(p))
            for (size_t m = p * p; m < N; m += p)
                bit_clear(m);

    // Clear the spare bits at the end of the last word so they are not counted ---
    for (size_t i = N; i < WORDS(N) * BITS_PER_WORD; i++)
        bit_clear(i);

    printf("primes below %d: %zu\n", N, bit_count());
    printf("  is 997 prime? %s\n", bit_test(997) ? "yes" : "no");
    printf("  is 999 prime? %s\n", bit_test(999) ? "yes" : "no");

    printf("the first ten:");
    size_t shown = 0;
    for (size_t i = 0; i < N && shown < 10; i++)
        if (bit_test(i)) { printf(" %zu", i); shown++; }
    puts("");
    return 0;
}

Output

1000 flags need 128 bytes as a bitmap, 1000 bytes as one char each
primes below 1000: 168
  is 997 prime? yes
  is 999 prime? no
the first ten: 2 3 5 7 11 13 17 19 23 29

Three things carry this demo. First, splitting an index into a word and a bit is a shift and a mask — the word size is a power of two, so i / 64 becomes i >> 6 and i % 64 becomes i & 63. Second, do not forget the spare bits at the end of the last word — counting them means counting elements that do not exist. Third, work a word at a time — calling stdc_count_ones once per word does sixty-four times less looping than walking bit by bit.

Q. Would bit fields (chapter 49) not spare us all this shifting and masking?

A. The syntax gets easier but the promises get thinner. Which end a bit field is filled from, whether it may straddle a word boundary, which types may be used — all implementation-defined (chapter 49), which makes bit fields hard to use for things whose layout is already fixed, such as hardware registers or file formats. The same structure can be laid out differently by another compiler.

So practice splits along that line. Where the layout is nobody’s business but yours — small flags inside an internal data structure — bit fields are convenient; where the layout is a promise made to the outside, hand-written shifts and masks are the safe choice. It is why the Linux kernel and many device drivers choose the latter.

53.9 Recap#

Recap

  • There are only five moves — set, clear, flip, test, replace a field. Everything else is a combination.
  • A test yields that place’s value (1u << n), not 1. Do not compare it against 1.
  • A mask has a width. ~0u is not a 64-bit mask, and 1u << 32 is outside the contract.
  • Learn why an idiom works rather than the idiom — know why x & (x-1) clears the lowest set bit and the rest follows.
  • C23′s <stdbit.h> gave those idioms standard names. The position functions count from 1, and 0 means “there is none”.
  • On signed values >> 1 is not / 2 and & 7 is not % 8. If division was meant, write division.
  • The compiler recognises the rotation idiom and Kernighan’s counting loop. Measure before reaching for a trick by hand.
  • Where a layout is promised to the outside, prefer hand-written shifts and masks over bit fields.

Bits were the lowest layer of a value. The next chapter is the other side of the same coin — reading the same bunch of bits as a real number. Approximation, error, and what “equal” is supposed to mean.