10 Programs and processes — what it is to be run

What to know first

chapter 5, The regions of memory · the sketch that divides memory into code, static, stack and heap

chapter 9, The origin of streams · the band along which characters flow, a line at a time

Looking back

Chapter 5 said a program’s memory divides into code, static, stack and heap. But when, and by whom, are those regions prepared?

A. At the moment the program is run, mostly by the operating system. The executable file lying on the disk has no memory yet — it is merely a blueprint saying “the code is like this, and the static region needs this much”. Reading that blueprint, spreading out the real memory, setting up the workbench, and leaping to the first instruction — that is running, and a program brought to life that way is called a process.

The need for this chapter, and its context

This chapter sits right after streams for two reasons. One — chapter 9 showed a band along which characters flow, so who hands us that band is the very next question. The answer is the operating system: standard input, output and error are channels already open when a process is born. Two — chapter 5 said memory divides like this, and what remained was who makes those regions, and when; that answer is here too.

Having this chapter also keeps the word “runs” from being empty when the first program arrives in chapter 16. And since chapter 12 shows how the hardware implements the isolation between processes described here, this chapter is the seam between process isolation and its hardware implementation — the two stories.

By the end of this chapter

We divide the program that exists as a file from the process that has been run and is alive. We look briefly at what the operating system gives a process (an address space of its own, open files, an exit status), and at how a process is born — the Unix family’s fork and Windows’ way. What appears here is not the C standard. It is also the chapter that first draws the boundary between what the standard promises and what the operating system gives.

The questions this chapter answers

“Memory of its own” sounds odd — isn’t there only one memory attached to the machine?
Then why should we know about processes now?
Why is there an “ended” state at all? Once it is over, why not just disappear?
How is a system call different from a function call? In the code they look the same.

10.1 A program is a noun, a process is a verb#

The distinction is simple.

Program — a file lying on the disk. Dead. It can be copied and moved.
Process — a program that has been run and is alive. It has memory, has a position it has reached, has open files, ends one day and leaves an exit status.

Run the same program three times and three processes arise. The three share the same code but each has its own memory — change a value on one side and the other does not know.

what the OS gives a process	what it is	in this book
address space	a memory layout of its own (chapter 5′s four regions)	chapter 98
execution position	which instruction it is in the middle of	—
the list of open files	the passages including standard input and output	chapters 9 and 78
command-line arguments and environment	the values handed over at running	chapter 59
exit status	the one number it leaves as it ends	chapters 16 and 59

Table 10.1 — What the operating system gives a process

Two things in this list connect straight to later chapters. First, the standard input, output and error we saw in chapter 9 are passages the operating system opened for the process in advance — it is not the program that opens them. Second, the 0 that chapter 16′s hello world returns as it finishes is exactly the exit status, and the side receiving that number is whoever ran this process.

Q. “Memory of its own” sounds odd — isn’t there only one memory attached to the machine?

A. Physically there is one, but the operating system shows each process its own address space. The addresses a process sees are not physical addresses but numbers of that process alone, and hardware moves them to the real places (virtual memory). So even if two processes use identical addresses they touch different places, and if one collapses the other is unharmed.

There is a world without this isolation too — on a small chip running with no operating system there is no concept of a process at all, and one piece of code uses the whole machine (chapter 98). So this chapter’s story is not the story of “everywhere C runs” but the story of when it runs on an operating system.

Seeing it once in action makes “a program is a noun, a process is a verb” concrete.

examples-en/processes/process.c

// A process is born anew on every run --- nothing of the last one remains.
#include <stdio.h>
#include <stdlib.h>

// A value the whole program owns. It looks as if the last run's value should
// still be there, but when the program ends the operating system takes back the memory it sat in.
static int run_count = 0;

static void at_the_end(void)
{
    puts("  the process is ending --- everything it owned goes back");
}

int main(int argc, char **argv)
{
    atexit(at_the_end);          // register a function to call when the process ends

    run_count += 1;
    // argv[0] is the name this program was invoked by. It differs from place to
    // place, so here we only look at whether it is there.
    printf("the program received %d argument(s), and argv[0] is %s\n",
           argc, (argc > 0 && argv[0] != NULL) ? "present" : "absent");
    printf("  run_count in this process: %d\n", run_count);
    puts("  run it again and it prints 1 again --- a new process starts fresh");
    return 0;
}

Output

the program received 1 argument(s), and argv[0] is present
  run_count in this process: 1
  run it again and it prints 1 again --- a new process starts fresh
  the process is ending --- everything it owned goes back

The run_count printed is always 1. Run it twice, run it ten times: still 1 — the program file stays put, but a process is born anew each time. The memory the last run used has already been taken back by the operating system. That the function registered with atexit is called at the end is the other side of the same story: a process has a beginning and an end, and at that end it gives back everything it held.

10.2 How a process is born#

From here it differs by operating system. We look briefly at the two branches: the Unix family’s fork and exec, and Windows’ CreateProcess.

Platform note. The Unix family — fork and exec

The Unix family (Linux, macOS, BSD) divides the making of a process into two steps. This design is the root of the shell and of pipelines (chapter 9).

fork() — it duplicates the present process. The duplicated side (the child) starts with the same code, the same memory contents and the same open files as the parent. The strange thing is that this function returns twice — to the parent it returns the child’s number (its PID), to the child 0. From that return value each knows who it is.
The exec family — it replaces the present process’s contents with another program. The memory is turned wholly into the new program’s and it does not return (if it succeeds).
The wait family — the parent waits for the child to end and receives the exit status.

pid_t pid = fork();
if (pid == 0) {               /* the child */
    execl("/bin/ls", "ls", (char *)0);
    _exit(127);               /* it comes here only if exec failed */
} else if (pid > 0) {         /* the parent */
    int status;
    waitpid(pid, &status, 0);
}

Type one command in a terminal and the shell does exactly this — duplicates itself, replaces the duplicated side with that command, and waits for it to end. The redirection and pipes seen in chapter 9 happen in between as well: after duplicating and before replacing, the child’s input and output passages are changed. Dividing the design into two steps is what made room for that.

Duplication does not mean copying the memory whole. Today’s implementations put it off until a write (copy-on-write) — parent and child share the same place, and only the part one of them writes to is copied at that moment.

Platform note. Windows — CreateProcess

Windows has no fork. Instead it is one step — CreateProcess instructs in one go, “start this program as a new process”. There being no duplication step, the child does not inherit the parent’s memory, and what the parent wishes to hand down (open passages, the environment, the working directory) is stated in the arguments.

The difference in character between the two ways shows itself in porting. The Unix pattern of “duplicate, tweak a little, then replace” does not carry over to Windows as it is and must be rewritten in the form of writing what is needed into the arguments. On the POSIX side — the standard shared by the Unix family, a standard of the operating system rather than of C — there is posix_spawn, which makes this pattern one step.

A common misconception. “Surely the functions for handling processes are in the C standard too”

It is not. Neither fork nor exec nor CreateProcess is part of standard C. What the C standard says about processes is surprisingly little — about as much as that the program starts, main runs, and it ends leaving an exit status. How to make a new process, run another program, or wait for a child is not in the standard (there is system alone, specified only as “it executes a command string”, and what that means is left to the implementation).

Why so stingy — because C must run on machines with no operating system too (chapter 77′s freestanding implementation). C must exist on chips that have no processes, so processes remained outside the standard, in the operating system’s territory.

This sense of the boundary matters through the whole book. What is in the standard works everywhere; what is outside it works only on that operating system. It is also why this book puts the stories tied to an operating system into separate “platform note” boxes.

Q. Then why should we know about processes now?

A. Because three things will soon be needed.

First, where chapter 9′s streams come from — standard input and output are passages already open when the process is born. Second, who receives the exit status 0 that chapter 16′s hello world leaves. Third, that chapter 5′s four regions are separate per process — saying a global variable is “one throughout the program” means one within one process.

The deeper stories — programs running along several strands (threads), communication between processes, signals — are beyond this book’s scope, but the pieces of them the C standard treats are met in their places (signals in chapters 90–91, threads and atomic operations in chapters 94–95).

10.3 The life of a process#

A process does not stay in one state from birth to end. Running, able to run and unable to run are different things, and that difference is the groundwork of the next chapter.

State	What it is	Waiting for what	How it leaves
Running	it is using the core right now	—	it stops by itself, its time runs out, or it ends
Ready	it could run right now but it is not its turn	a turn on the core	the scheduler picks it
Waiting	it cannot run — it waits for something	a disk, input, a time, another process	the thing it waited for happens
Ended	the work is done; only the exit status is left	its parent to collect that value	the parent collects it with `wait`

Table 10.2 — The states a process can be in

★ The third row is the state most often met in practice. When a program is “slow” it is usually not short of cores but waiting — for the disk, for a key press, for the network. And since there is no reason to hold the core while waiting, this is where the operating system hands it to another process.

Q. Why is there an “ended” state at all? Once it is over, why not just disappear?

A. Because one thing is left behind: the number called the exit status. If the side that should receive it (the parent) has not yet done so, the body of the process may go but that one line of record must remain. When the parent collects it, it disappears entirely. If the parent never collects it, only that record drifts on — in the Unix family it is called a zombie, which sounds alarming but does nothing at all.

10.4 User mode and kernel mode — what may be done and what may not#

One question about process isolation remains. If processes cannot touch one another’s memory, who makes that rule hold? If each program were to keep it voluntarily, nothing could stop a program that did not.

So the machine has two modes.

	User mode	Kernel mode
Who runs there	ordinary programs	the operating system
Memory	only its own address space is visible	all of it is visible
Devices	cannot be touched directly	can be touched
Doing what may not be done	it stops right there — the operating system steps in	—
The door across	a system call — ask at an appointed place	returns to the previous mode

Table 10.3 — The machine’s two modes — user mode and kernel mode

This makes what printf really does in chapter 16 a little more precise. Putting characters on a screen means touching a device, which user mode may not do. printf formats the characters, holds them in a tub (a buffer), and in the end asks the operating system. That asking is the system call.

Q. How is a system call different from a function call? In the code they look the same.

A. They do look the same — you write write(…). What happens inside differs. An ordinary call jumps to another address within the same mode; a system call crosses the boundary between modes. So it costs more: the request is handed over in an appointed way, the mode changes, and the operating system checks “may this be granted?” before anything happens.

This adds another layer to why the tub of chapter 9 exists. Crossing the boundary for every single character would mean paying that cost every time, so the characters are gathered and handed over in one go.

Recap

to remember	the point
program / process	a file / a program run and alive
what the OS gives	an address space, open passages, an exit status
each process has its own memory	run the same program several times and the processes do not see one another’s memory
process creation on Unix	two steps, `fork` (duplicate) + `exec` (replace)
process creation on Windows	one step, `CreateProcess`
processes are outside the C standard	creating and managing processes is not the C standard’s territory — the OS provides it

Table 10.4 — Program and process — what to remember

The stage is set. A program is a file; run it and it becomes a process; the operating system hands it memory, channels and an exit status — that is what actually happens when chapter 16′s first program is “run”.

Now it is time to see what happens on that stage. When there are several processes, how is one core shared among them; who decides each switch and what announces it — that is the next chapter. The “waiting” state and the boundary between modes, both just seen, are used there at once. How bold a lie the simple picture of the machine has been all along — memory splitting apart (chapter 12), execution overlapping (chapter 13), the compiler stepping in (chapter 14), and therefore why C cannot help being “an abstract language” (chapter 15) — is what remains of this part.

← PrevContentsNext →