Proven C Book한국어 GitHub

61 The terrain of the standard library

What to know first

chapter 43, Safe input · the traps of the standard functions
chapter 16, The general shape of compilation · a library is linked in

Looking back

Chapter 16 said the linker finds printf’s body in the standard library and joins it up, and chapter 43 said gets was removed from the standard. Who settles the standard library, and how does it change?

A. The language standard settles it along with the language — the C standard document pins down not only the grammar but also the list of libraries that must be provided and the contract of each function (hence “the C standard library”). The speed of change is as slow as the language’s, and removing something is far slower still — the reason gets’s funeral took decades, and the key to this library’s character.

The need for this chapter, and its context

Part 10 closes with the library’s landscape and part 11 opens with reading it closely; this chapter is the bridge. Without first drawing “what is there”, part 11′s twenty-two chapters read as a list of headers. And explaining here why this library is thin and old is what makes the traps ahead read as marks of an era rather than as defects.

By the end of this chapter

We spread out the landscape of the standard library we have merely been using until now — what toolboxes there are, and what to trust and what to beware of. And why this library has its “thin and old” appearance, down to the historical reason.

The questions this chapter answers

  1. Then may input parsing always be done with sscanf?
  2. Then is the standard library so old that it is better not used?

61.1 The landscape — the toolboxes

Group the headers you meet often into families and the terrain comes into view at a glance.

familyrepresentative headers and content
input and output<stdio.h> — streams, the printf/scanf families, files
strings and memory<string.h> — copying, comparing, searching; <ctype.h> — character classification
numbers<stdlib.h> — conversion and random numbers, <math.h> — mathematical functions, <stdint.h>, <limits.h>, <float.h> — the limits of types
memory management<stdlib.h> — the malloc/free family
time and environment<time.h>, the environment access in <stdlib.h>
contracts and diagnosis<assert.h>, <errno.h>
the modern additions<stdbool.h> (C99, made unnecessary by C23), <stdatomic.h> and <threads.h> (C11), <stdckdint.h> (C23 checked arithmetic)

Table 62.1

Two things stand out in this list. First, it is thin — there are no data structures (lists, hash tables), no regular expressions, no networking, no graphics. Second, it is old — most of it was in C89, and what has newly entered can be counted on the fingers.

61.2 Why it is thin — design and history

The reason lies where C grew up. C was born as a language for making operating systems (chapter 4), and it had to write in the same language not only programs running on top of an operating system but the operating system itself and embedded firmware too. In such places there may be no file system, no dynamic allocation, not even an operating system — so the standard library was narrowed to “the minimum that can exist anywhere” (the reason the standard separately defines a freestanding environment). Abundance was given up and portability gained.

And the thinness came at a price — the world made what it needed in the end, and the result is the forest of platform-specific APIs and third-party libraries. That is the background of the saying “in C, choosing libraries is half the work” — chapter 43′s five disciplines are about what to demand of that choice.

61.3 An anatomy of the output format string

This is the place to take apart head on the function used most. In chapter 22 we learned only the minimal set (%d, %s, %%), but printf’s format is in fact a small language of five pieces.

Reading from the back is quicker to understand. The conversion settles “as what shall it be printed” — this alone is required — and the rest is decoration laid on top.

Write * in the width or precision place and that value can be passed as an argument. When printing a string that travels as “pointer and length”, like chapter 10′s view, %.*s comes in handy.

examples-en/ch61/fmtspec.c

#include <stdio.h>

int main(void) {
    int n = 42;
    unsigned u = 255;
    double x = 3.14159265;
    const char *s = "proven";
    size_t sz = 1234;
    long long big = 9000000000LL;

    /* flags and width: right alignment by default, - for left, 0 to fill with zeros */
    printf("[%d] [%6d] [%-6d] [%06d] [%+d]\n", n, n, n, n, n);

    /* precision: decimal places for reals, a maximum length for strings */
    printf("[%f] [%.2f] [%10.3f] [%e] [%g]\n", x, x, x, x, x);
    printf("[%s] [%10s] [%-10s] [%.3s]\n", s, s, s, s);

    /* bases and characters */
    printf("[%c] [%x] [%X] [%#x] [%o] [%u]\n", 'A', u, u, u, u, u);

    /* length modifiers: they tell the format the width of the argument */
    printf("[%zu] [%lld]\n", sz, big);

    /* passing the width and precision as arguments */
    printf("[%.*s] [%*d]\n", 4, s, 6, n);

    /* the percent character itself */
    printf("100%%\n");
    return 0;
}

Output

[42] [    42] [42    ] [000042] [+42]
[3.141593] [3.14] [     3.142] [3.141593e+00] [3.14159]
[proven] [    proven] [proven    ] [pro]
[A] [ff] [FF] [0xff] [377] [255]
[1234] [9000000000]
[prov] [    42]
100%

There are several things to read here. %06d becoming 000042 is because the 0 flag fills with zeros instead of spaces, and %.3s stopping at pro is because for strings precision is maximum length. The 0x of %#x was attached by the #. %g shortening to 3.14159 is because that conversion automatically chooses the shorter of %e and %f.

The length modifier is not decoration but a contract. As chapter 58 showed, type information does not ride along into variadic arguments, so printf reads the stack at exactly the width the format stated. That is why code printing a size_t with %d quietly goes out of step on 64-bit — 8 bytes were put in and only 4 are taken out.

typeoutput formatnote
int%d %ithe basic form
unsigned int%u %x %ohexadecimal when looking at bits
short%dit is promoted, so %hd is optional
long%ld
long long%lld
size_t%zu★ printing it with %d goes out of step
ptrdiff_t%td
double%f %e %gfloat is promoted and the same
long double%Lf
char (as a character)%cthe argument is promoted to int
char *%sit must be NUL-terminated
void *%pthe form is implementation-defined
bool%dprinted as 0 or 1

Table 62.2

A common misconception. %f when printing a float, %lf for a double

Half right. In output, a float goes over as a double by the default argument promotion (chapter 29), so both are %f — the standard permits %lf too, but it means the same. The real root of the confusion is in input. scanf takes addresses, so no promotion happens and float * and double * must be distinguished — %f is float * and %lf is double *. It is the most famous asymmetry of these two functions: the same letter meaning different things depending on the direction.

61.4 The input format string — what differs

The formats of the scanf family overlap in their letters with output and so look like the same language, but what they do is the opposite and their rules differ. Five points of difference.

First, the argument is not a value but the address of a place to hold it. Leave out the & and it mistakes an integer for an address and tries to write at that address — the reason the & met in chapter 25 is compulsory here.

Second, whitespace in the format means “any number of whitespace characters (none is fine too)”. In output a space is simply one space, but in input it is an instruction to skip. Most conversions skip leading whitespace by themselves — the exceptions are %c and %[, which read whitespace as characters too.

Third, ordinary characters in the format must match the input exactly. "x=%d" requires the input to begin with x=. If it does not match it stops with a matching failure.

Fourth, the return value is not the number of characters printed but the number of items successfully assigned. So when three were to be read and 2 comes back, one was not filled, and that argument is left untouched. If there was no input at all, EOF (a negative value) comes back — it must be distinguished from 0.

Fifth, always give %s a maximum width. A %s without a width writes without knowing the destination’s size, the same danger as chapter 43′s gets.

examples-en/ch61/scanspec.c

#include <stdio.h>

int main(void) {
    int a = 0, b = 0, k;
    double d = 0.0;
    char word[8] = {0};
    char rest[16] = {0};

    /* one space in the format means "any number of whitespace characters" */
    k = sscanf("  12   34", "%d %d", &a, &b);
    printf("ints    : k=%d a=%d b=%d\n", k, a, b);

    /* %s stops at whitespace. A width protects the buffer */
    k = sscanf("hello world", "%7s", word);
    printf("bounded : k=%d word=[%s]\n", k, word);

    /* an ordinary character in the format must match the input exactly */
    k = sscanf("x=5", "x=%d", &a);
    printf("literal : k=%d a=%d\n", k, a);

    /* if it does not match it is a matching failure — 0 is returned and the argument is untouched */
    k = sscanf("y=5", "x=%d", &a);
    printf("mismatch: k=%d (a stays %d)\n", k, a);

    /* a conversion failure is 0 as well — not a number, so nothing is read */
    k = sscanf("abc", "%d", &b);
    printf("nonnum  : k=%d (b stays %d)\n", k, b);

    /* an empty input gives EOF (a negative value) — it must be told apart from 0 */
    k = sscanf("", "%d", &b);
    printf("empty   : k=%d\n", k);

    /* partial success: the front is read and it stops further on */
    k = sscanf("7 oops", "%d %lf", &a, &d);
    printf("partial : k=%d a=%d\n", k, a);

    /* the set specifier: it gathers characters that are not a comma */
    k = sscanf("name,42", "%15[^,],%d", rest, &b);
    printf("set     : k=%d rest=[%s] b=%d\n", k, rest, b);

    /* reals and the length modifier: a double is %lf */
    k = sscanf("3.5", "%lf", &d);
    printf("double  : k=%d d=%.2f\n", k, d);
    return 0;
}

Output

ints    : k=2 a=12 b=34
bounded : k=1 word=[hello]
literal : k=1 a=5
mismatch: k=0 (a stays 5)
nonnum  : k=0 (b stays 34)
empty   : k=-1
partial : k=1 a=7
set     : k=2 rest=[name] b=42
double  : k=1 d=3.50

Going through the output line by line makes the rules visible. mismatch and nonnum both have k=0, and what matters is that the argument remains as it was — use it without checking the value and you mistake the previous value for new input. empty’s -1 means “the input has ended”, not “the format was wrong”. partial is the case where only the leading integer was read and it stopped after. set’s %15[^,] means “at most 15 characters that are not a comma”, used for cutting a line with separators.

typeinput formatargument
int%dint *
unsigned%u %xunsigned *
long%ldlong *
long long%lldlong long *
size_t%zusize_t *
float%ffloat *
double%lfdouble *
long double%Lflong double *
one character%cchar * (it reads whitespace too)
a word%99schar[100] — width compulsory
a set of characters%15[^,]char[16]
skipping%*dread but do not store

Table 62.3

In practice. What one format brought down — the format string vulnerability

We complete here the accident chapter 22 brushed past by name only. Code that puts a user-given string straight into the format position (printf(user)) lets an attacker input %s or %x and read the stack, and in the days when the old %n (which writes the number of characters printed to where the argument points) was still alive it led even to arbitrary memory writes. Around 1999 this class was discovered on a large scale and several servers including wu-ftpd were1 remotely compromised. The lesson is one line — the format string must always be a constant written by the program, and user input goes in only as an argument (printf("%s", user)).

Q. Then may input parsing always be done with sscanf?

A. For simple formats it is enough. But sscanf does not tell you where and why it failed — all it gives back is the number of successful items, so “the third field was not a number” and “the line ended early” cannot be distinguished. Moreover it does not detect integer overflow (give 99999999999 to a %d and it is outside the contract), and you cannot know how many characters were consumed up to the failing place either. When the format grows complex and the input came from somebody else, a tool is needed in which failure appears as a value and the remaining input can be held in the hand — Part XII’s scanner is that answer.

61.5 The places to beware

Gathering the traps this book has met along the way, from the library’s point of view, gives this.

Q. Then is the standard library so old that it is better not used?

A. Not at all — that it is everywhere is an overwhelming virtue. Components existing with the same contract on every platform and every compiler are only these, and most of this book’s examples ran with the standard library alone. The right attitude is not “do not use it” but knowing which function has which contract and choosing accordingly — choosing the editions that take a size, checking return values, and keeping the conventions when using functions that lean on global state. And laying components with checking built in over the repeated danger zones (string assembly, input parsing) — the next chapter is that practice.

The landscape is spread out. The next part (Part XI) walks the regions of this landscape one by one and faces head on the places where accidents really happen in each header — streams and files, strings, conversion and allocation, characters and locales, numbers and time, and diagnosis and control.

Notes

  1. Tim Newsham. 2000. Format String Attacks. White paper, Guardent, Inc. faculty.digipen.edu/~dvolper/copies/tn-usfs.pdf. The class itself is catalogued as CWE-134 — cwe.mitre.org/data/definitions/134.html