Proven C Book←↑→

103 How the work turns — vocabulary and procedure

What to know first

chapter 102, Build and test · gates, golden tests, kinds of test
chapter 53, Errors and contracts · the idea of a contract

Looking back

chapter 102 settled what goes where and what to test. Is the rest not just a matter of doing it — what is there left to learn here?

A. What is left is the way of working, and that is a different matter from layout. Working alone, the way may live in your head. But with only three people, the question “have you tested this?” calls up three different things. One thinks of what was run by hand, one of what runs automatically, one of what has not been written yet.

★ So this chapter begins with words. The moment the same thing is called by the same name, the team’s conversation gets shorter, and by that much fewer things go astray.

The need for this chapter, and its context

If chapter 102 is the chapter you open when setting a project up, this is the one you open when keeping it turning. What appears here is machinery large projects really use, yet every piece of it can be used today in a program written alone — better learnt alone, in fact, so that no explanation is needed once there are several of you.

By the end of this chapter

First, four words around testing — the fixture (the ground a test stands on), the witness (a check that cries when a defect returns), the negative check (making sure that check really bites), and the ratchet (a number that must not grow). Then tiers, which decide how deep to run and when; and last, the procedure a single piece of work passes through from arriving to being committed. Each word comes with a one-line definition and one real thing.

The questions this chapter answers

  1. Does writing the scope in advance mean leaving alone what you find along the way?

103.1 Without names, the same thing gets called different things#

First, the words of this chapter in one place. Each returns later with something real beside it, so skimming now is enough.

WordIn one line
fixturethe fixed provisions a test stands on — input files, answer files, a prepared state
witnessone check stood up to say “if this defect returns, I cry”
regressiona fixed defect coming back. What a witness guards against
negative checkbreaking something on purpose to see whether the check really bites
ratcheta baseline number that must not grow — like a cogwheel, it turns one way only
tierthe depth of checking. What was changed decides how far to run
gatethe bundle of automatic checks you cannot pass without (chapter 102)
spillsomething found mid-task but outside this scope — written down and passed on

Table 103.1 — The words of this chapter

★ None of these is the name of a tool. Tools differ from project to project, but the places they are called from are the same. Learn the words and even an unfamiliar repository becomes readable.

103.2 The fixture — the ground a test stands on#

Running one test needs provisions: an input to read, an answer to compare against, a temporary folder made in advance, data holding known values. That bundle of provisions is the fixture.

chapter 102′s golden demonstration already had them. They simply went unnamed.

PlaceWhat
in/two-lines.txtfixture — the fixed input the test is fed
expected/two-lines.txtfixture — the frozen answer
wordcount.cthe subject of the test. Not a fixture
run.shthe frame that runs the test and compares

Table 103.2 — What was a fixture in the golden demonstration

★ A fixture has one condition — it must be the same every time. If yesterday and today differ, then when a test fails you cannot tell “was the code wrong, or did the provisions change”. At that moment the test stops being a thing that gives answers and becomes a thing that adds one more question.

So there are things that must not serve as fixtures.

WhatWhy
the current time, random numbersthey differ every run. Where needed, pin the value in
anything across a networksomeone else’s circumstances turn your test red
files left by an earlier testthe order of tests changes the result — the hardest kind of accident to find
a path that exists only on my machineit does not run for anyone else (chapter 102′s “it works on my machine”)

Table 103.3 — What must not be made a fixture

The word’s root is that same property. A lamp or a pipe fixed into a building is called a fixture, and the sense of not moving about, always being there is what carried over into testing.

103.3 The witness — a check is a defect’s gravestone#

Say you have fixed a bug. What do you leave behind at that spot? Leave nothing and the bug comes back — people change, code is edited, and nobody knows the story any more. A defect that has come back is called a regression.

There is one way to stop it. At every place you fix, stand up one check that says “if this defect returns, I cry”. That is a witness.

★ And the order matters — stand it up before the fix. Put it up afterwards and you cannot know whether it bites. A check that was green from the start and a check that was red because it caught the defect and turned green when it was fixed look identical, and are worth entirely different things.

examples-en/ch103/witness/run.sh

#!/bin/sh
# Stand the witnesses up *before* the fix.
#
#   (1) Run against the version with the defect - a witness must cry (red).
#       If none cries, that witness guards nothing.
#   (2) Run against the fixed version - all must be quiet (green).
#
# The order is the point. A check seen only green is a check you do not know bites.
set -eu
cd "$(dirname "$0")"
CC=${CC:-cc}
CFLAGS="-std=c23 -Wall -Wextra"

printf '== (1) the version with the defect (a witness must cry)\n'
$CC $CFLAGS -o witness-buggy find-buggy.c witness.c
if ./witness-buggy; then
    printf '   * no witness cried - this witness is useless\n'
    rc=1
else
    printf '   -> it cried, as intended\n'
    rc=0
fi

printf '\n== (2) the fixed version (all must be quiet)\n'
$CC $CFLAGS -o witness-fixed find.c witness.c
./witness-fixed || rc=1

rm -f witness-buggy witness-fixed
exit $rc

Output

== (1) the version with the defect (a witness must cry)
  ok   finds a value in the middle
  ok   finds the first
  FAIL * finds the last - one cell of edge  (got -1, wanted 4)
  ok   an absent value gives -1
  ok   an empty array gives -1 too
a witness cried
   -> it cried, as intended

== (2) the fixed version (all must be quiet)
  ok   finds a value in the middle
  ok   finds the first
  ok   * finds the last - one cell of edge
  ok   an absent value gives -1
  ok   an empty array gives -1 too
all five witnesses are quiet

The demonstration shows that order as it is. Taking the common slip of writing lo < hi for lo <= hi in a binary search, five witnesses were stood up first.

  1. Run against the version with the defect and only “finds the last” cries. The other four stay quiet — meaning the witnesses point out where it went wrong as well.
  2. On the fixed version all five are quiet.

And each witness carries a sentence on why it matters.

check("* finds the last - one cell of edge", find(sorted, n, 50),  4);

The sentence, not the count, is the asset. Months later, when this check cries, what a person reads is not “check 3 failed” but “it cannot see one cell of the edge”.

Counter-example. Writing the check after the fix

The commonest order, and the least valuable. A check written against already-fixed code is necessarily green, so there is no way to tell a biting check from one that does nothing at all.

Checks that do nothing really do get written — ones that compare constants without calling the function, ones with the condition the wrong way round, ones that swallow the failure. Such a check reassures people merely by existing, which makes it worse than none.

If there is no time to fix, at least keep the order — write the smallest code that reproduces the defect, watch it cry with your own eyes, and only then fix.

103.4 The negative check — does that check really bite#

Make what the witness order taught into a rule and it reads: when you build a new check, break something once, on purpose. This is called a negative check — it confirms not “does it say absent things are absent” but “does it say present things are present”.

The method is simple. Put in on purpose the defect the check is meant to catch, see whether the check cries, and put it back.

SymptomWhat it means
broken, yet quietthe check does not look at that place — fix it now
broken, and a different check criesthe checks overlap — decide which one owns it
nothing broken, yet it criesthe check cries wolf. Left alone, people learn to ignore checks
only that one cries, as intendeda check worth having

Table 103.4 — What a negative check reveals

In practice. This book is built that way

This book’s checkers (the gates of chapter 102) were all built in this order. Make a check, then deliberately undo the place you fixed and run it. If it catches, restore; if it does not, fix the check.

★ Twice in that process a checker turned out to have been shutting its eyes. Once a regular expression was skipping wholesale over what lay inside Korean quotation marks; another time, the manuscript folds its lines and the check stopped at the newline, missing sentences that spanned several lines entirely. Both checks were green — they simply were not looking at what they were supposed to see.

So one more discipline was added. Being green and looking are not the same thing.

103.5 The ratchet — a number that must not grow#

Every project has numbers that “would be nice at zero but cannot be zero today”: the count of warnings, of defects not yet fixed, of places no test reaches. There are two ways to treat such a number — look away, or stop it growing.

The latter is a ratchet. Write today’s number into a file, and if it grows beyond that the gate cries. The name comes from the cogwheel that turns one way only.

SituationWithout a ratchetWith one
300 warningsnobody notices when it becomes 301it is stopped the moment it becomes 301
you fixed oneit rolls on at 299a person tightens the baseline to 299
in a hurry, you want one moreit grows quietlyraising the baseline means editing that line — a trace is left

Table 103.5 — With a ratchet and without

★ The last row is what a ratchet is really worth. It does not forbid growth; it makes growth visible. If it is truly needed, raise the baseline. Only, at that moment the change lands in the commit, and review gets to ask “why did this grow?”.

And the tightening is done by a person. If the machine lowered the baseline automatically whenever the number fell, then a number that dipped by chance and rose again could not be stopped — the ratchet would be gone.

103.6 Tiers — you cannot always run everything#

The more checks you stand up, the longer running them all takes. And then people start not running them (the same story as make clean in chapter 102). So checking is given depth as well. Deciding how far to run according to what changed is the tier.

DepthWhat runsWhen
shallowbuild, formatting, the fast unit testson every save, seconds
ordinaryall unit tests, the golden testsbefore a commit, tens of seconds
deepintegration, slow tests, builds under several configurationsbefore shipping, minutes

Table 103.6 — An example of dividing the depth

Two disciplines come with it.

  1. What changed decides the depth. Edit only documents and there is no reason to run deep; touch a core data structure and you may not slip by shallow.
  2. ★ It can only be raised. “We are in a hurry, shallow this once” is the first step in the collapse of the discipline. Depth is not a thing you lower to get a pass.

103.7 One piece of work — separating decision from execution#

Now the procedure. From a request arriving to a commit closing it, the steps a large project actually walks are broadly these.

StepWhat you doIf skipped
(1) separatewhat is a decision and what is executiona decision made alone gets overturned later
(2) measureactually measure instead of guessingyou fix the wrong place
(3) write the scopenote in advance what this task will touchthe work spreads without end
(4) fixone meaning at a timethe commit becomes impossible to undo
(5) verifypass the checks at that depthverification happens on someone else’s time
(6) write it downleave the why in the commit messagemonths later nobody knows the reason

Table 103.7 — The steps one piece of work passes

★ The first step is the one most often skipped and the most expensive. A decision and an execution are different kinds of work.

Mix the two and both get worse. Ask about the execution as well and nothing moves; settle the decision alone as well and the whole thing gets undone later.

The second step earns as much. Work that begins with “it seems slow” or “the problem is probably here” often grabs the wrong leg. Plans overturned after measuring are common, and that is normal. When one is overturned, put that in the record — so the next person does not grab the same leg again.

Q. Does writing the scope in advance mean leaving alone what you find along the way?

A. No. It means write it down and pass it on. That is called a spill.

Work always turns up problems outside the scope. There are three options then — fix it now, pretend not to have seen it, or write it down and pass it on. The first mixes two meanings into one commit and makes undoing hard; the second simply loses it. The third is the answer.

The place you write it need be nothing grand. One issue, one line in a backlog, one line in a list at the bottom of a file. The point is keeping the boundary of this task without losing the finding.

103.8 No silent failures — and the commit is the record of a decision#

Last, one discipline runs through this whole chapter. Do not fail silently.

If you truncated, say you truncated. If you could not measure, say you could not. If you skipped a check, say you skipped it. Because silence is always read as “it was checked and it was fine”.

Do thisAnd the reader reads
truncate a value that does not fit“the value went in”
drop a failing check from the list“that check passed”
omit the words “usually works”“it always works”
write 0 for what you could not measure“it was measured and it was 0”

Table 103.8 — The faces of silent failure

★ The last row is the dangerous one. “It is zero” and “the check answered zero” are different sentences. A check’s scope is its promise, so what it does not look at must be written down as well.

Then the commit message. Code says what was done but cannot say why (the same story as the design record in chapter 102). So a commit carries these.

  1. What was fixed and why — and what was undone, if anything was
  2. Which numbers moved, and how (if there is a ratchet)
  3. One line on what was learned — so the next person does not grab the same leg

Recap

What to rememberIn brief
vocabularycalling the same thing by the same name shortens the conversation
fixturethe provisions a test stands on. They must be the same every time
witnessone at every place you fix. “If this defect returns, I cry”
order★ stand the witness up before the fix — the only moment you can tell it bites
negative checkbreak a new check once, on purpose. Being green and looking are not the same
ratchetgrowth is stopped; the tightening is done by a person
tierwhat changed decides the depth. It can only be raised
decision and executionask about the decision, do the execution. Mixing spoils both
spilla finding outside the scope is written down and passed on
silent failuresilence is read as “it was checked and it was fine”

Table 103.9 — How the work turns — what to remember

That is the frame of work built by several people over a long time. The next chapter holds all of it against one field — embedded work, where C is most deeply rooted, and what compilers and tools one works with there.