54 Several files — splitting and linking

54.1 Translation units and headers

One source file after preprocessing — the result of all the #includes being spread out — is called a translation unit. The compiler always sees one translation unit at a time. A multi-file program means making several translation units into object files separately and having the linker join them into one.

We see the division of labour in a three-file example.

/* A multi-file example — this file compiles seeing only greet's declaration. */
#include "greet.h"

int main(void)
{
    greet("world");
    printf_count();
    return 0;
}

Hello, world!
greet was called 1 times

greet.h (the header) contains only the declaration — the contract signature. greet.c has the definition, and main.c includes only the header, knows the contract and calls. Compilation happens separately for each, and the linker joins the call to greet with its definition.

The header’s #ifndef GREET_H ... #define ... #endif is an include guard. It is the idiom that makes the contents unfold only once even if the header is attached twice through different paths, preventing things that error when declared twice (struct definitions and the like). In practice it is often replaced by the single line #pragma once (not standard, but supported by all the major compilers).

54.2 Linkage — does a name cross file boundaries?

A name has one more property besides scope (chapter 24) — linkage, that is, whether that name is visible from another translation unit.

• External linkage: visible outside the file. The default for functions and global variables.
• Internal linkage: visible only within this translation unit. Attach static to a file-level declaration — the example’s static int calls is the case.

That the word static is used with different meanings in chapter 44 (static lifetime) and here (internal linkage) is C’s famous word recycling — inside a function static means duration, at file level it means linkage.

Internal linkage is a basic weapon in practice. C gives the programmer no way to dig a new name space (chapter 55), so every external name meets in one yard; attach static to helper functions and variables used only inside a file and those names never go out into the yard at all — no collisions, and the compiler can optimise more aggressively (knowing that name cannot be called from another file — an honest signal to chapter 13′s editor). This weapon and the rest of the defences against name collisions are the subject of chapter 56.

54.3 Linking is not one step — three names

So far we have said loosely that “the linker joins things up”. It is in fact three stages, each with a name.¹

A statically linked program ends after the first two. A dynamically linked one goes through all three. That is why the one word “linking” appears at both build time and run time, and confuses people.

★ One practical fact attaches here. The runtime linker brings shared objects into the address space before main is called, but it does not join function symbols until they areactually called. This is lazy binding. So linking against a library you never call costs almost nothing at start-up.

On Linux, LD_BIND_NOW=1 turns the deferral off and joins everything at start. Use it when the delay just after launch matters, or when you want to learn about a link failure now rather than later.

54.4 The real purpose of dynamic linking — not size but the ABI

The benefit of dynamic linking is commonly given as “the executable gets smaller”. Measured, that is true enough.

★ But the size is a by-product. The real purpose lies elsewhere. Dynamic linking detaches the program from a particular version of a library. The system promises “these services in this shape”, and the program relies only on that promise. That is why a library can be fixed without rebuilding the program.

This promise is called the ABI (Application Binary Interface). If an API is a promise at the source level, an ABI is one at the binary level — not only what the function is called, but which register the arguments ride in, how a struct is laid out, where the return value goes.

soname is the part met most often in practice. The number in a name like libfoo.so.2 declares “binary compatibility is kept up to this version”; a change that breaks compatibility raises the number. Old programs still look for .so.2 while new ones look for .so.3, so both can live on one system.

54.5 Position-independent code (PIC) — why libraries must be built this way

A shared library has a problem to solve. The same library lands at a different address in each process. Yet addresses inside machine code have to be somewhere.

The naive answer is “fix up the addresses in the code when loading”. That answer has a fatal cost. ★ The moment you patch a code page, that page can no longer be shared. If eight processes use one library, the code should exist once in physical memory; patch it differently in each and there are eight copies. The very reason shared libraries exist collapses.

Hence position-independent code (PIC). The idea is simple — never patch the code; gather every address that needs fixing into a “table”. A table is data, so having one per process costs little.

It is plain when seen for real. Here is int counter; int bump(void){ return ++counter; } built with -fPIC -shared and taken apart with objdump (x86-64).

bump:
  mov 0x2ec9(%rip),%rdx     <- read counter's address out of the GOT
  mov (%rdx),%eax           <- read the value at that address
  add $0x1,%eax
  mov %eax,(%rdx)

★ Two reads. The very structure that split arrays from pointers in chapter 40 — “read the address first, then read the value” — returns here. The cost of PIC is, in the end, that one step.

The function side goes through the PLT.

bump@plt:
  jmp *0x2fc2(%rip)         <- jump to the GOT entry
  push $0x1                 <- if not yet joined, fall through to here
  jmp  <dynamic linker>     <- the linker fills the address in and jumps again

Those two lines are the machinery of lazy binding. On the first call the dynamic linker is summoned to fill the GOT entry; after that a single first jmp goes straight there.

54.6 When names collide they get replaced — interpositioning

The previous section showed that external linkage is the default. Meet that with dynamic linking and something surprising happens. A function I wrote can quietly stand in for a library function.

This is interpositioning. The dynamic linker searches for a symbol in a fixed order and uses the first one found. So if my program has a global function named malloc, even the malloc calls made inside the library come to mine.

★ Two things must coincide for this to become an accident. One is that functions are global by default; the other is that the compiler does not treat the redefinition as an error. The latter is C’s old attitude — it assumes the programmer meant it.

/* in my file, with no static */
char *mktemp(char *tpl) { ... }      /* a name the standard library also uses */
int   index(int i)      { ... }      /* an old Unix function name */

We have learned how to cross file boundaries. But #include and #ifndef appeared again in this chapter — the layer chapter 16 passed over with “the preprocessor is a text tool that does not know C”. The next chapter opens that layer head on, and also sees the formal stages by which source code becomes a program.