Appendix M — How machines talk: wires, buses and interruption
In the appendix on C without an operating system, devices were handled by address. Write *(volatile uint8_t *)0x10000000 = 'A' and a character went out. But what happens after that one write? Which wires carry which voltages in what order, and how does the other end turn them back into a character?
This appendix is that story. The aim is not to become a hardware engineer but to be able to read the first page of a datasheet.
Platform note. what this appendix rests on, and its limits
Words to know first#
| Word | Meaning | Easily confused |
|---|---|---|
| line, signal | one strand carrying a voltage | “data lines” and “control lines” do different work |
| bus | a bundle of lines shared by several devices | point-to-point links (PCIe, USB) are often buses in name only |
| protocol | the agreement about what goes on the wire and when | same wires, different agreement, and nothing is understood |
| frame | one chunk sent at a time | not “one byte” — markers are attached front and back |
| baud | signal changes per second | not necessarily bits per second (it depends on the modulation) |
| throughput | bytes actually moved per second | always below the theoretical figure, because of the frame’s overhead |
| latency | the time from asking to being answered | throughput can be large while latency is large too |
| controller / target | the side that speaks / the side that answers | modern specifications prefer these words to the older ones |
| full duplex / half duplex | both sides at once / by turns | the number of wires usually decides this |
Table 105.1 — The words used in this appendix
★ The third row from the bottom is the most frequently misunderstood in practice. Throughput and latency are different axes. Fill a truck with hard drives and drive it and the throughput is enormous while the latency is hours. Conversely I2C is slow (100 kbit/s) and the latency of exchanging one byte is under a millisecond.
Four axes that separate connections#
There are dozens of ways to connect things, and only four questions to ask. Settle these four and the rest is detail.
| Axis | One side | The other | What it settles |
|---|---|---|---|
| number of wires | serial — queued on one strand | parallel — at once on several | pin count and top speed (Figure 105.1) |
| timing | asynchronous — no clock line | synchronous — a separate clock line | whether both sides must agree the speed in advance |
| addressing | none — there is only one other end | present — one of several is called | whether many devices can hang off one wire |
| who starts | only one side (controller-target) | anyone (peer, arbitration) | how collisions are prevented |
Table 105.2 — Four axes that separate connections
Figure 105.1 — Sending the same byte serially and in parallel.
Lay the common schemes out along those four axes and this is the result. The sections that follow unpack the rows one at a time.
| Name | Wires | Clock | Address | In a phrase |
|---|---|---|---|---|
| UART (universal asynchronous receiver-transmitter) / RS-232 | serial, 2 strands (+ flow control) | none (asynchronous) | none | the oldest and the simplest |
| parallel port | parallel, 8 strands + control | yes (a strobe) | none | it was fast, then hit the wall of speed |
| I2C | serial, 2 strands | yes (SCL) | yes (7 bits) | many devices on two wires |
| SPI | serial, 4 strands | yes (SCK) | chosen by a wire (CS) | fast and simple, and it eats wires |
| CAN | serial, 2 strands (differential) | none | carried in the message | anyone speaks and arbitration settles it |
| PCIe | serial lanes ×N (differential) | embedded (carried in the coding) | yes (configuration space) | the motorway inside the machine |
| USB | serial, 2 to 4 strands | embedded | yes (the host assigns it) | the host commands everything |
Table 105.3 — The connections covered in this appendix
A common misconception. serial is slow and parallel is fast
The oldest wire — UART and RS-232#
The simplest way to join two machines is this. Send the bits along one wire in turn. There is no clock line either. Instead both sides agree in advance on “so many bits per second” and each counts on its own clock.
The device that handles this is a UART (universal asynchronous receiver-transmitter).
Frames — cutting characters out of an idle wire#
Without a clock, how is “a character starts now” recognised? By an agreed shape.
| Order | Name | Value | What | Without it |
|---|---|---|---|---|
| — | the idle state | 1 (high) | the wire when nothing is being sent | there is no baseline against which a start can be seen |
| 1 | the start bit | 0 (low) | “it begins now” — the receiver syncs its clock here | character boundaries cannot be found |
| 2–9 | 8 data bits | the value | the low-order bit goes first | — |
| (optional) | parity | computed | one bit making the count of ones even (or odd) | a single-bit error goes unnoticed |
| 10 | the stop bit | 1 (high) | “it ends here” — distinguishable from the next start bit | sending back to back loses the boundary |
Table 105.4 — The ten bits of an 8N1 frame
The notation 8N1 is that table abbreviated.
| Position | Meaning | Common values | Note |
|---|---|---|---|
| first digit | data bits | 8, rarely 7 | 7 bits is a trace of the days of ASCII alone |
| middle letter | parity | N (none), E (even), O (odd) | N is today’s default |
| last digit | stop bits | 1, rarely 2 | 2 gives a slow receiver room to breathe |
Table 105.5 — How to read a notation like 8N1
examples-en/apx-links/uart_frame.c
/* UART 프레임을 비트로 짓고, *보율이 어긋난 수신기*로 다시 읽어 본다.
하드웨어는 없다 --- 선 위의 전압을 잘게 썬 배열로 흉내 낸다.
한 비트를 16칸으로 나누어(흔한 UART 가 정말 이렇게 샘플링한다) 시간을 표현한다. */
#include <stdio.h>
#include <stdint.h>
#include <string.h>
#define OS 16u /* oversampling --- 한 비트를 몇 칸으로 나눌까 */
#define MAXW 4096u
/* 8N1: 시작 1 + 자료 8 + 정지 1 = 10비트. 자료는 *낮은 자리부터* 나간다. */
static size_t make_frame(unsigned char *w, uint8_t byte, unsigned parity_bits)
{
size_t n = 0;
for (unsigned i = 0; i < OS * 2; i++) w[n++] = 1; /* 놀고 있을 때는 높다 */
for (unsigned i = 0; i < OS; i++) w[n++] = 0; /* 시작 비트: 떨어뜨린다 */
unsigned ones = 0;
for (int b = 0; b < 8; b++) {
unsigned bit = byte >> b & 1; /* LSB first */
ones += bit;
for (unsigned i = 0; i < OS; i++) w[n++] = (unsigned char)bit;
}
if (parity_bits) { /* 짝수 패리티라면 */
unsigned p = ones & 1; /* 1의 개수를 짝수로 맞춘다 */
for (unsigned i = 0; i < OS; i++) w[n++] = (unsigned char)p;
}
for (unsigned i = 0; i < OS; i++) w[n++] = 1; /* 정지 비트: 다시 높다 */
for (unsigned i = 0; i < OS * 2; i++) w[n++] = 1;
return n;
}
/* 수신기: 시작 비트의 내려감을 찾고, 제 비트 길이로 한가운데를 찍어 읽는다.
rx_bit 이 16 이 아니면 그만큼 보율이 어긋난 것이다. */
static int receive(const unsigned char *w, size_t n, double rx_bit,
uint8_t *out, int *stop_ok)
{
size_t edge = 0;
while (edge < n && w[edge] != 0) edge++; /* 내려가는 자리 */
if (edge >= n) return -1;
double t0 = (double)edge;
uint8_t v = 0;
for (int b = 0; b < 8; b++) {
double at = t0 + rx_bit * (b + 1) + rx_bit / 2.0; /* (b+1)번째 비트의 한가운데 */
size_t idx = (size_t)(at + 0.5);
if (idx >= n) return -1;
v |= (uint8_t)(w[idx] << b);
}
double sat = t0 + rx_bit * 9 + rx_bit / 2.0; /* 정지 비트 자리 */
*stop_ok = (sat < n) && w[(size_t)(sat + 0.5)] == 1;
*out = v;
return 0;
}
static void show_wave(const unsigned char *w, size_t n)
{
printf(" ");
for (size_t i = 0; i < n; i += OS / 2) putchar(w[i] ? '-' : '_');
printf("\n ");
/* 비트 경계에 이름을 붙인다 */
const char *lab[] = { " ", " ", "St", "d0", "d1", "d2", "d3", "d4", "d5", "d6", "d7", "Sp", " ", " " };
for (size_t i = 0, k = 0; i < n; i += OS, k++)
printf("%-*s", (int)(OS / (OS / 2)), k < sizeof lab / sizeof *lab ? lab[k] : " ");
printf("\n");
}
int main(void)
{
unsigned char w[MAXW];
const uint8_t byte = 'K'; /* 0x4B = 0100 1011 */
printf("== the byte to send ==\n");
printf(" '%c' = 0x%02X = binary %c%c%c%c%c%c%c%c (most significant first)\n\n", byte, byte,
"01"[byte >> 7 & 1], "01"[byte >> 6 & 1], "01"[byte >> 5 & 1], "01"[byte >> 4 & 1],
"01"[byte >> 3 & 1], "01"[byte >> 2 & 1], "01"[byte >> 1 & 1], "01"[byte & 1]);
size_t n = make_frame(w, byte, 0);
printf("== the shape on the wire (8N1) --- low=_ high=- ==\n");
show_wave(w, n);
printf(" * data goes least significant bit first, so it looks reversed to the eye.\n\n");
printf("== timing ==\n");
printf(" %-10s %-14s %-14s %s\n", "baud", "one bit", "one frame (10 bits)", "bytes per second");
const long bauds[] = { 300, 9600, 19200, 115200, 921600 };
for (unsigned i = 0; i < sizeof bauds / sizeof *bauds; i++) {
double bit_us = 1e6 / (double)bauds[i];
printf(" %-10ld %-14.3f %-14.1f %.0f\n", bauds[i], bit_us, bit_us * 10,
(double)bauds[i] / 10.0);
}
printf(" (microseconds. 8N1 spends 10 bits on one byte, so 20%% is overhead)\n\n");
printf("== when the receiver's baud rate is off ==\n");
printf(" the receiver samples the middle using its own bit length; the error shifts it.\n\n");
printf(" %-12s %-10s %-8s %-8s %s\n", "rx baud", "error", "value read", "stop bit", "result");
struct { const char *name; double factor; } rx[] = {
{ "9600", 1.00 }, { "9700", 9600.0 / 9700 }, { "9900", 9600.0 / 9900 },
{ "10100", 9600.0 / 10100 }, { "10600", 9600.0 / 10600 }, { "19200", 9600.0 / 19200 },
};
for (unsigned i = 0; i < sizeof rx / sizeof *rx; i++) {
uint8_t got; int stop_ok;
double rx_bit = OS * rx[i].factor;
if (receive(w, n, rx_bit, &got, &stop_ok) != 0) { printf(" %-12s could not read\n", rx[i].name); continue; }
double err = (1.0 / rx[i].factor - 1.0) * 100.0;
printf(" %-12s %+7.1f%% 0x%02X %-8s %s\n", rx[i].name, err, got,
stop_ok ? "ok" : "broken",
got == byte && stop_ok ? "'K' --- correct"
: got == byte ? "value right, frame lost"
: "character corrupted");
}
/* 임계점을 *찾아본다* --- 통설을 옮겨 적는 대신 프로그램이 재게 한다 */
double lo = 0, hi = 0;
for (double e = -20.0; e <= 20.0; e += 0.05) {
double rx_bit = OS / (1.0 + e / 100.0);
uint8_t g; int ok;
int good = receive(w, n, rx_bit, &g, &ok) == 0 && g == byte && ok;
if (good && lo == 0 && hi == 0) lo = e;
if (good) hi = e;
}
printf("\n range of error where the character survives here: %+.1f%% to %+.1f%%\n", lo, hi);
printf(" * the arithmetic agrees. The stop bit is sampled at 9.5 bit times,\n");
printf(" and that instant must fall within that bit (one bit wide), so\n");
printf(" 0.5 / 9.5 = %.1f%% is the margin on one side.\n", 0.5 / 9.5 * 100.0);
printf(" * yet practice uses 2 to 3%% as the rule. This %.1f%% is shared by both ends\n",
0.5 / 9.5 * 100.0);
printf(" (the sender drifts too), oscillators move with temperature and supply,\n");
printf(" and a real receiver samples three points and takes a majority.\n");
printf(" The theoretical limit and the design budget are different numbers.\n");
printf("\n== what parity catches ==\n");
size_t np = make_frame(w, byte, 1);
uint8_t got; int stop_ok;
receive(w, np, OS, &got, &stop_ok);
unsigned ones = 0; for (int b = 0; b < 8; b++) ones += byte >> b & 1;
printf(" '%c' has %u ones -> even parity bit = %u\n", byte, ones, ones & 1);
printf(" one flipped bit changes the parity, so it is caught.\n");
printf(" two flipped bits leave the parity unchanged, so it is not --- parity\n");
printf(" is a device for noticing an error, not for correcting one.\n");
return 0;
}
Output
== the byte to send ==
'K' = 0x4B = binary 01001011 (most significant first)
== the shape on the wire (8N1) --- low=_ high=- ==
----__----__--____--__------
Std0d1d2d3d4d5d6d7Sp
* data goes least significant bit first, so it looks reversed to the eye.
== timing ==
baud one bit one frame (10 bits) bytes per second
300 3333.333 33333.3 30
9600 104.167 1041.7 960
19200 52.083 520.8 1920
115200 8.681 86.8 11520
921600 1.085 10.9 92160
(microseconds. 8N1 spends 10 bits on one byte, so 20% is overhead)
== when the receiver's baud rate is off ==
the receiver samples the middle using its own bit length; the error shifts it.
rx baud error value read stop bit result
9600 +0.0% 0x4B ok 'K' --- correct
9700 +1.0% 0x4B ok 'K' --- correct
9900 +3.1% 0x4B ok 'K' --- correct
10100 +5.2% 0x4B ok 'K' --- correct
10600 +10.4% 0x8B broken character corrupted
19200 +100.0% 0x9E ok character corrupted
range of error where the character survives here: -5.2% to +5.9%
* the arithmetic agrees. The stop bit is sampled at 9.5 bit times,
and that instant must fall within that bit (one bit wide), so
0.5 / 9.5 = 5.3% is the margin on one side.
* yet practice uses 2 to 3% as the rule. This 5.3% is shared by both ends
(the sender drifts too), oscillators move with temperature and supply,
and a real receiver samples three points and takes a majority.
The theoretical limit and the design budget are different numbers.
== what parity catches ==
'K' has 4 ones -> even parity bit = 0
one flipped bit changes the parity, so it is caught.
two flipped bits leave the parity unchanged, so it is not --- parity
is a device for noticing an error, not for correcting one.
The demonstration shows four things.
First, data goes low-order bit first. Which is why reading the shape on the wire makes the binary number look reversed.
Second, 8N1 spends ten bits on one byte. So 9600 baud is 960 bytes per second — 20% of it is overhead holding the frame together.
Third, a baud mismatch accumulates. The clock is synced at the start bit, but by the tenth bit the error has grown nine and a half times. The demonstration searched for the threshold itself and answered about ±5%, and the arithmetic agrees: 0.5 / 9.5 = 5.3%.
Fourth, and yet the rule in practice is “2 to 3%”. That these two numbers differ matters — the theoretical limit is shared between the two ends, oscillators drift with temperature and supply, and a real receiver samples several points and takes a majority. In the words of chapter 53, a contract is not the limit but a budget with margin left.
Q. Why such awkward numbers as 9600, 19200, 115200?
A. The crystal old UART chips used was 1.8432 MHz, and dividing it by integers gives these numbers (1.8432 MHz ÷ 16 ÷ 12 = 9600). What became standard was not “a round decimal” but a number an integer divider can make. Which is why some clocks cannot produce exactly 115200, and the error that follows eats into the budget above.
RS-232 — the same frame, different voltages#
A UART is logic; RS-232 is the electrical specification for putting that logic on a wire. Mixing the two burns hardware often enough to deserve a table.
| TTL/CMOS UART | RS-232 | |
|---|---|---|
| 1 (mark) | 3.3 V or 5 V | −3 V to −15 V |
| 0 (space) | 0 V | +3 V to +15 V |
| logic | high is 1 | inverted — negative is 1 |
| reach | centimetres on a board | tens of metres |
| wired together directly | — | the voltages differ and the chip is damaged — a converter must sit between |
Table 105.6 — TTL-level UART and RS-232
★ The last row is where hardware gets destroyed in practice. A computer’s serial port must not go straight onto a microcontroller’s UART pin — something like a MAX232 has to be in between.
| Name | Direction | What | Today |
|---|---|---|---|
| TxD | out | data being sent | always used |
| RxD | in | data being received | always used |
| GND | — | the reference voltage — without it nothing works | always used |
| RTS / CTS | out / in | “may I send” — hardware flow control | used on fast links |
| DTR / DSR | out / in | “I am powered” — device readiness | a trace of the modem era |
| DCD | in | the modem has a connection | almost never used |
| RI | in | the phone is ringing | almost never used |
Table 105.7 — The signals of RS-232 (on a 9-pin connector)
Flow control — how to say “wait”#
If the receiving side cannot keep up, data simply disappears. A UART has no retransmission.
| Kind | How | Good | Bad |
|---|---|---|---|
| hardware (RTS/CTS) | a separate line is lowered to say “stop” | it does not touch the data, and stops at once | two more wires are needed |
| software (XON/XOFF) | 0x13 and 0x11 are inserted into the data stream | three wires suffice | binary data cannot be sent — those bytes are misread |
Table 105.8 — Two kinds of flow control
Counter-example. sending binary data over an XON/XOFF line
0x13 byte in the middle of the data is read by the other end as “stop”. The transfer freezes then and there, and may never resume. On a line that will carry binary data, use hardware flow control or encode the data as text.The four errors a UART reports#
| Name | When | What it usually means |
|---|---|---|
| framing error | the stop bit position was low | the baud rates differ, or the line was disturbed |
| parity error | the count of ones does not match the agreement | noise — one bit flipped |
| overrun | the next byte arrived before the last was read | my code was late — an interrupt or DMA is needed |
| break | the line stayed low for longer than a frame | a deliberate signal from the other end, or a wire broken to ground |
Table 105.9 — Errors met in a UART’s status register
★ The third row joins on to the end of this appendix. An overrun is a software problem, not a hardware one — when taking an interrupt per byte becomes too much, DMA (direct memory access) is what is needed.
Why parallel disappeared#
Sending eight bits at once looks eight times faster. For a while it was. The Centronics port that connected printers worked that way.
| Group | Lines | What | Direction |
|---|---|---|---|
| data | D0 to D7 | eight bits at once | out |
| control | STROBE | “the data is valid now” — effectively the clock | out |
| status | ACK | “received” | in |
| status | BUSY | “still working, wait” | in |
| status | PAPER OUT, SELECT, ERROR | the printer’s circumstances | in |
| reference | GND ×8 | a ground paired with each data line to reduce noise | — |
Table 105.10 — The signals of a parallel (Centronics) port
Why it vanished despite being fast is in Figure 105.1. The requirement that all eight lines arrive at the same instant becomes the wall.
| Problem | What it is | As speed rises |
|---|---|---|
| skew | lines differ in length and load, so arrival times spread | once one bit’s time is shorter than the spread, they cannot be lined up |
| crosstalk | a change on one line leaks into its neighbour | the faster the change, the worse it is |
| pins and cost | eight lines means eight of everything: connector, board, shielding | expensive, thick and unbendable |
| termination | each line’s impedance must be matched or it reflects | beyond a few hundred MHz it is effectively impossible |
Table 105.11 — Why parallel loses as things get faster
★ So today’s answer is not “parallel was abandoned” but “make one serial line very fast, and bundle several of them”. PCIe’s lanes, USB 3′s several pairs and SATA all have that shape. Lining the lines up is solved by hardware recovering each line separately and then reassembling — possible because each line carries its own clock inside its coding.
Many devices on two wires — I2C#
The scheme for when wires must be saved above all. Two wires (data SDA, clock SCL) carry several devices, chosen by address.
| Name | What | How it is made | Why so |
|---|---|---|---|
| SCL | the clock | the controller drives it | synchronous — no need to agree a speed in advance |
| SDA | data | the two sides use it by turns | half duplex — one direction at a time |
| START | “a transaction begins” | SDA is pulled low while SCL is high | a deliberate violation of “data changes only while SCL is low”, which is what makes it a marker |
| STOP | “it ends” | SDA is released high while SCL is high | likewise |
| repeated START | “do not end, only turn round” | another START without a STOP | no other controller can cut in meanwhile |
| ACK / NACK | “received” / “absent, or enough” | on the ninth clock the receiver pulls SDA low, or does not | possible precisely because a line can only be pulled down |
Table 105.12 — The signals and markers of I2C
examples-en/apx-links/i2c_frame.c
/* I2C 거래 하나를 신호 차례로 짓고, 그 차례를 다시 읽어 해독한다.
선은 둘뿐이다: SCL(클록)과 SDA(자료). 둘 다 「끌어내리기만」 할 수 있다(오픈 드레인). */
#include <stdio.h>
#include <stdint.h>
#include <string.h>
enum { EV_START, EV_BIT, EV_ACK, EV_NACK, EV_RSTART, EV_STOP };
struct ev { int kind; int val; const char *note; };
static struct ev log_[256];
static int n_ev;
static void put(int kind, int val, const char *note)
{ log_[n_ev++] = (struct ev){ kind, val, note }; }
/* 바이트 하나 --- 높은 자리부터 여덟 비트, 그다음 아홉 번째 클록이 ACK 자리 */
static void put_byte(uint8_t b, const char *what, int acked)
{
for (int i = 7; i >= 0; i--) put(EV_BIT, b >> i & 1, i == 7 ? what : NULL);
put(acked ? EV_ACK : EV_NACK, acked ? 0 : 1, NULL);
}
static const char *kind_name(int k)
{
switch (k) {
case EV_START: return "START";
case EV_RSTART: return "repeated START";
case EV_STOP: return "STOP";
case EV_ACK: return "ACK";
case EV_NACK: return "NACK";
default: return "bits";
}
}
int main(void)
{
const uint8_t dev = 0x3C; /* 7비트 장치 주소 */
const uint8_t reg = 0x00, val = 0xAF;
printf("== why the address is confusing ==\n");
printf(" the 7-bit address 0x%02X is shifted one place on the wire (last bit is read/write).\n", dev);
printf(" write: 0x%02X << 1 | 0 = 0x%02X\n", dev, dev << 1);
printf(" read : 0x%02X << 1 | 1 = 0x%02X\n", dev, dev << 1 | 1);
printf(" so datasheets call the same device 0x%02X in one place and 0x%02X in another.\n\n",
dev, dev << 1);
/* ── 쓰기 거래: 장치에게 「레지스터 0 에 0xAF 를 써라」 ── */
put(EV_START, 0, "the controller takes the bus");
put_byte((uint8_t)(dev << 1 | 0), "device address + write", 1);
put_byte(reg, "register number", 1);
put_byte(val, "value to write", 1);
put(EV_STOP, 0, "release the bus");
/* ── 읽기 거래: 반복 START 로 방향만 바꾼다 ── */
put(EV_START, 0, "take it again");
put_byte((uint8_t)(dev << 1 | 0), "device address + write", 1);
put_byte(reg, "say which register to read first", 1);
put(EV_RSTART, 0, "* START again without STOP --- nobody can cut in meanwhile");
put_byte((uint8_t)(dev << 1 | 1), "device address + read", 1);
put_byte(val, "the value the device returned", 0); /* 마지막 바이트는 주인이 NACK 로 끝을 알린다 */
put(EV_STOP, 0, "end");
printf("== the order on the wire ==\n");
int bitpos = 0; uint8_t acc = 0;
for (int i = 0; i < n_ev; i++) {
struct ev *e = &log_[i];
if (e->kind == EV_BIT) {
acc = (uint8_t)(acc << 1 | e->val);
if (++bitpos == 8) {
printf(" %-12s 0x%02X %s\n", "byte", acc,
log_[i - 7].note ? log_[i - 7].note : "");
bitpos = 0; acc = 0;
}
} else {
printf(" %-12s %s%s\n", kind_name(e->kind),
e->kind == EV_ACK ? "the device pulled SDA down = received" :
e->kind == EV_NACK ? "nobody pulled it down = absent, or done" : "",
e->note ? e->note : "");
}
}
printf("\n== what means what ==\n");
printf(" START : SDA falls while SCL is high (deliberately breaking the rule that\n");
printf(" STOP : SDA rises while SCL is high data changes only while SCL is low)\n");
printf(" ACK : on the ninth clock the receiver pulls SDA down\n");
printf(" NACK : nobody pulls it down and the line stays high --- absent and enough look alike\n");
printf("\n== timing ==\n");
int bits = 0, extra = 0;
for (int i = 0; i < n_ev; i++)
(log_[i].kind == EV_BIT) ? bits++ : (log_[i].kind == EV_ACK || log_[i].kind == EV_NACK) ? bits++ : extra++;
printf(" clocks in these two transactions: %d (data and ACK bits), %d markers\n", bits, extra);
printf(" %-14s %-14s %s\n", "speed mode", "clock", "time for these two transactions");
struct { const char *name; double hz; } modes[] = {
{ "standard", 100e3 }, { "fast", 400e3 }, { "fast plus", 1e6 }, { "high speed", 3.4e6 },
};
for (unsigned i = 0; i < sizeof modes / sizeof *modes; i++)
printf(" %-14s %-14.1f %.1f microseconds\n", modes[i].name, modes[i].hz / 1000,
bits / modes[i].hz * 1e6);
printf(" (clock in kHz; markers and wait states are not counted)\n");
printf("\n== why the lines are only ever pulled down ==\n");
printf(" if two devices speak at once, one pushing 5V and one pushing 0V, current\n");
printf(" flows straight through and damages the chips. I2C lets nobody push (open drain)\n");
printf(" and leaves the raising to a resistor. So speaking at once is safe,\n");
printf(" and the property that zero wins gives arbitration and ACK for free.\n");
return 0;
}
Output
== why the address is confusing ==
the 7-bit address 0x3C is shifted one place on the wire (last bit is read/write).
write: 0x3C << 1 | 0 = 0x78
read : 0x3C << 1 | 1 = 0x79
so datasheets call the same device 0x3C in one place and 0x78 in another.
== the order on the wire ==
START the controller takes the bus
byte 0x78 device address + write
ACK the device pulled SDA down = received
byte 0x00 register number
ACK the device pulled SDA down = received
byte 0xAF value to write
ACK the device pulled SDA down = received
STOP release the bus
START take it again
byte 0x78 device address + write
ACK the device pulled SDA down = received
byte 0x00 say which register to read first
ACK the device pulled SDA down = received
repeated START * START again without STOP --- nobody can cut in meanwhile
byte 0x79 device address + read
ACK the device pulled SDA down = received
byte 0xAF the value the device returned
NACK nobody pulled it down = absent, or done
STOP end
== what means what ==
START : SDA falls while SCL is high (deliberately breaking the rule that
STOP : SDA rises while SCL is high data changes only while SCL is low)
ACK : on the ninth clock the receiver pulls SDA down
NACK : nobody pulls it down and the line stays high --- absent and enough look alike
== timing ==
clocks in these two transactions: 63 (data and ACK bits), 5 markers
speed mode clock time for these two transactions
standard 100.0 630.0 microseconds
fast 400.0 157.5 microseconds
fast plus 1000.0 63.0 microseconds
high speed 3400.0 18.5 microseconds
(clock in kHz; markers and wait states are not counted)
== why the lines are only ever pulled down ==
if two devices speak at once, one pushing 5V and one pushing 0V, current
flows straight through and damages the chips. I2C lets nobody push (open drain)
and leaves the raising to a resistor. So speaking at once is safe,
and the property that zero wins gives arbitration and ACK for free.
Three things are immediately useful in practice.
First, why an address is written two ways. A 7-bit address is shifted one place left on the wire with the read/write bit appended. So the same device appears in datasheets as 0x3C and as 0x78. Both are right — one is the address, the other is “the first byte on the wire”.
Second, NACK means two things. “No such device” and “stop sending now” look the same on the wire. The convention is that the controller NACKs the last byte of a read to say it is done.
Third, a design where lines are only ever pulled down makes much of this free. Nobody pushes a voltage up, so two speaking at once damages no chip, and from the property “zero wins” both ACK and arbitration simply fall out. In exchange the pull-up is done by a resistor, so rising is slow, and that sets I2C’s speed ceiling.
| Name | Clock | Bytes per second (roughly) | Note |
|---|---|---|---|
| standard | 100 kHz | about 11,000 | most sensors |
| fast | 400 kHz | about 44,000 | today’s default |
| fast plus | 1 MHz | about 110,000 | the wiring must be short |
| high speed | 3.4 MHz | about 380,000 | needs dedicated circuitry — uncommon |
Table 105.13 — I2C speed modes
Q. I attached several I2C devices and none is found. What do I look at first?
A. Four things in order. ① Are the pull-up resistors there (without them the line never rises)? ② Do the addresses collide (two sensors of the same kind usually do — change one with its address pin)? ③ Is the ground shared? ④ Is the wiring so long, or the device count so high, that the rise time is too slow? Three of the four are electrical problems and only one is software.
Fast on four wires — SPI#
Where I2C gave up speed to save wires, SPI does the reverse: four wires, and fast.
| Name | Other names | What | Direction |
|---|---|---|---|
| SCK | SCLK, CLK | the clock — the controller drives it | controller → device |
| MOSI | SDO, COPI | data the controller sends | controller → device |
| MISO | SDI, CIPO | data the device sends | device → controller |
| CS | SS, NSS, CE | the marker meaning I am speaking to this device (usually active low) | controller → device |
Table 105.14 — The four wires of SPI
★ There is no address. Instead one CS line is run to each device. More devices means more wires — exactly the opposite trade from I2C.
examples-en/apx-links/spi_shift.c
/* SPI 는 「고리 모양으로 이어 붙인 시프트 레지스터 둘」이다.
클록마다 한 비트씩 서로 밀어 넣는다 --- 그래서 주고받기가 *동시에* 일어난다. */
#include <stdio.h>
#include <stdint.h>
static void bits8(uint8_t v, char *out)
{ for (int i = 0; i < 8; i++) out[i] = (char)('0' + (v >> (7 - i) & 1)); out[8] = 0; }
int main(void)
{
uint8_t m = 0xA5; /* 주인이 보낼 값 */
uint8_t s = 0x3C; /* 장치가 보낼 값 (미리 제 레지스터에 넣어 둔다) */
char mb[9], sb[9];
bits8(m, mb); bits8(s, sb);
printf("== at the start ==\n");
printf(" controller register : 0x%02X (%s)\n", m, mb);
printf(" device register : 0x%02X (%s)\n\n", s, sb);
printf("== one bit per clock ==\n");
printf(" %-6s %-6s %-6s %-12s %-12s\n", "clock", "MOSI", "MISO", "controller", "device");
for (int c = 1; c <= 8; c++) {
unsigned mosi = m >> 7 & 1; /* 주인이 내놓는 비트 (높은 자리부터) */
unsigned miso = s >> 7 & 1; /* 장치가 내놓는 비트 */
m = (uint8_t)(m << 1 | miso); /* 서로의 비트를 낮은 자리로 받아 넣는다 */
s = (uint8_t)(s << 1 | mosi);
bits8(m, mb); bits8(s, sb);
printf(" %-6d %-6u %-6u %-12s %-12s\n", c, mosi, miso, mb, sb);
}
printf("\n== after eight clocks ==\n");
printf(" controller register : 0x%02X <- what the device sent\n", m);
printf(" device register : 0x%02X <- what the controller sent\n", s);
printf(" * sending and receiving finished in the same eight clocks. Full duplex is\n");
printf(" free because the two registers form one ring. Even with nothing to receive\n");
printf(" something must be sent (usually 0x00 or 0xFF): a dummy byte.\n");
printf("\n== the four modes --- which edge presents, which samples ==\n");
printf(" %-8s %-6s %-6s %-16s %s\n", "mode", "CPOL", "CPHA", "SCK at rest", "sampling edge");
for (int mode = 0; mode < 4; mode++) {
int cpol = mode >> 1, cpha = mode & 1;
printf(" %-8d %-6d %-6d %-16s %s\n", mode, cpol, cpha,
cpol ? "high" : "low",
cpha == 0 ? (cpol ? "falling (first edge)" : "rising (first edge)")
: (cpol ? "rising (second edge)" : "falling (second edge)"));
}
printf("\n== when the mode is mismatched ==\n");
uint8_t src = 0xA5, wrong = 0, prev = 0;
for (int c = 0; c < 8; c++) {
unsigned bit = src >> (7 - c) & 1;
wrong = (uint8_t)(wrong << 1 | prev); /* 한 모서리 늦게 읽으면 *직전* 비트를 본다 */
prev = bit;
}
bits8(0xA5, mb); bits8(wrong, sb);
printf(" sent 0xA5 (%s)\n", mb);
printf(" read with one edge of error: 0x%02X (%s) <- everything shifted one place\n", wrong, sb);
printf(" * so when SPI gives half-right values, suspect the mode first.\n");
printf("\n== speed ==\n");
printf(" SPI has no start, stop, address or ACK. Exactly eight clocks per byte.\n");
printf(" %-14s %-16s %s\n", "SCK", "bytes per second", "compared");
struct { const char *name; double hz; } sck[] = {
{ "1 MHz", 1e6 }, { "10 MHz", 10e6 }, { "50 MHz", 50e6 },
};
const double i2c_std = 100e3 / 9.0; /* I2C 표준 모드: 바이트마다 9비트 */
for (unsigned i = 0; i < sizeof sck / sizeof *sck; i++)
printf(" %-14s %-16.0f %.0f times I2C standard (100 kHz)\n", sck[i].name, sck[i].hz / 8,
(sck[i].hz / 8) / i2c_std);
printf(" (UART spends 10 bits per byte, I2C 9, SPI 8 --- no overhead)\n");
return 0;
}
Output
== at the start ==
controller register : 0xA5 (10100101)
device register : 0x3C (00111100)
== one bit per clock ==
clock MOSI MISO controller device
1 1 0 01001010 01111001
2 0 0 10010100 11110010
3 1 1 00101001 11100101
4 0 1 01010011 11001010
5 0 1 10100111 10010100
6 1 1 01001111 00101001
7 0 0 10011110 01010010
8 1 0 00111100 10100101
== after eight clocks ==
controller register : 0x3C <- what the device sent
device register : 0xA5 <- what the controller sent
* sending and receiving finished in the same eight clocks. Full duplex is
free because the two registers form one ring. Even with nothing to receive
something must be sent (usually 0x00 or 0xFF): a dummy byte.
== the four modes --- which edge presents, which samples ==
mode CPOL CPHA SCK at rest sampling edge
0 0 0 low rising (first edge)
1 0 1 low falling (second edge)
2 1 0 high falling (first edge)
3 1 1 high rising (second edge)
== when the mode is mismatched ==
sent 0xA5 (10100101)
read with one edge of error: 0x52 (01010010) <- everything shifted one place
* so when SPI gives half-right values, suspect the mode first.
== speed ==
SPI has no start, stop, address or ACK. Exactly eight clocks per byte.
SCK bytes per second compared
1 MHz 125000 11 times I2C standard (100 kHz)
10 MHz 1250000 112 times I2C standard (100 kHz)
50 MHz 6250000 562 times I2C standard (100 kHz)
(UART spends 10 bits per byte, I2C 9, SPI 8 --- no overhead)
The demonstration shows what SPI really is. Two shift registers form a single ring. Push one bit per clock and after eight clocks the two values have swapped places. So “sending and receiving at the same time” is free — and even to receive only, you must send something (a dummy byte).
| Mode | CPOL | CPHA | Clock at rest | Sampling edge |
|---|---|---|---|---|
| 0 | 0 | 0 | low | rising (the first edge) |
| 1 | 0 | 1 | low | falling (the second edge) |
| 2 | 1 | 0 | high | falling (the first edge) |
| 3 | 1 | 1 | high | rising (the second edge) |
Table 105.15 — SPI’s four modes
A common misconception. SPI is a specification, so any device can just be wired up
| I2C | SPI | So | |
|---|---|---|---|
| wires | 2 (unchanged as devices are added) | 3 + 1 per device | many devices favour I2C |
| speed | 100 kHz to 1 MHz | tens of MHz | needing speed favours SPI |
| bits per byte | 9 (the ACK included) | 8 | SPI has no overhead either |
| direction | half duplex | full duplex | needing both at once favours SPI |
| confirmation of receipt | there is an ACK | none | SPI cannot tell that the other end has died |
| specification | written down | custom | with SPI the datasheet must be read |
Table 105.16 — I2C against SPI
On one wire, and with anyone speaking — 1-Wire and CAN#
| Name | Wires | The idea | Where |
|---|---|---|---|
| 1-Wire | 1 (+ ground) | zero and one are told apart by pulse length, and power is stolen from the data line | temperature sensors, identity chips |
| CAN | 2 (differential) | anyone speaks, and on a collision the lower message number wins (arbitration) | cars, industrial machinery |
Table 105.17 — Two unusual schemes
★ CAN’s arbitration is the same idea as I2C’s “zero wins”. When two speak at once the signal of the one sending zero remains on the wire, and the one sending a one sees that the wire differs from what it sent and withdraws. It is not collision detection followed by retransmission: the collision itself is the priority decision — which is why an urgent message is never delayed.
The motorway inside the machine — PCI and PCI Express#
So far these were “wires between chips”. Now the scale changes. Where graphics cards, network cards and NVMe storage attach — the PCI family.
The name says “bus”, but today’s PCIe is really point to point. Old PCI really did have several devices sharing the same lines (so one slow device slowed everything), while PCIe gives each device its own lines with a switch joining them in the middle.
| old PCI | PCIe | So | |
|---|---|---|---|
| connection | several devices share one bus | point to point per device, plus switches | one device cannot hold up the others |
| wires | parallel 32/64 bits + clock | serial differential lanes ×1 to ×16 | exactly the conclusion of Figure 105.1 |
| going faster | raise the clock (33 → 66 MHz) | add lanes or raise the generation | the direction of growth differs |
| interrupts | shared INTA to INTD lines | MSI/MSI-X — sent as a memory write | it becomes data rather than a wire |
Table 105.18 — PCI and PCI Express
| Generation | On the wire | Coding | Effective bandwidth |
|---|---|---|---|
| 1.0 | 2.5 GT/s | 8b/10b (20% lost) | about 250 MB/s |
| 2.0 | 5 GT/s | 8b/10b | about 500 MB/s |
| 3.0 | 8 GT/s | 128b/130b (1.5% lost) | about 985 MB/s |
| 4.0 | 16 GT/s | 128b/130b | about 1.97 GB/s |
| 5.0 | 32 GT/s | 128b/130b | about 3.94 GB/s |
Table 105.19 — PCIe generations and effective speed (per lane)
★ The “coding” column matters for a reason. There is no separate clock line, so the clock must be planted in the data (otherwise a long run of zeros loses the receiver’s timing), and what is planted is the loss. Dropping 8b/10b for 128b/130b at 3.0 is why “1.6 times the speed” became “twice the bandwidth”.
Configuration space — the 256 bytes in which a device introduces itself#
PCI’s best idea is this. Every device carries a business card of the same shape. So the operating system can find out what is plugged in without knowing in advance.
| Offset | Size | Name | What | Typical value |
|---|---|---|---|---|
0x00 | 2 | vendor ID | a number handed out by PCI-SIG | 0x8086 (Intel) |
0x02 | 2 | device ID | a model number chosen by the vendor | — |
0x04 | 2 | command | the switch that turns this device on — I/O, memory, bus master | 0x0007 |
0x06 | 2 | status | errors, and a flag saying “there is a capability list” | 0x0010 |
0x08 | 1 | revision | the revision of the same model | — |
0x09–0x0B | 3 | class code | what kind of device this is — known without a driver | 02 00 00 = Ethernet |
0x0C | 1 | cache line size | a trace of old PCI | 0x10 |
0x0E | 1 | header type | 0 device, 1 bridge, 0x80 means multiple functions | 0x00 |
0x10–0x24 | 24 | BAR 0 to 5 | the address windows this device requests (Table 105.21) | — |
0x2C–0x2E | 4 | subsystem ID | the company and model that built the board | — |
0x30 | 4 | expansion ROM | where firmware on the card lives | — |
0x34 | 1 | capability pointer | the offset of the first item of a linked list | 0x50 |
0x3C | 1 | interrupt line | which IRQ it was wired to in the old scheme | 11 |
0x3D | 1 | interrupt pin | which of INTA to INTD | 1 (A) |
Table 105.20 — The main fields of PCI configuration space (header type 0)
| Bits | Value | Meaning | Note |
|---|---|---|---|
| 0 | 0 | memory space | usually this — a pointer reaches it |
| 0 | 1 | I/O space | x86′s narrow 64 KiB world |
| 2–1 | 00 | 32-bit address | |
| 2–1 | 10 | 64-bit address | the next BAR holds the upper 32 bits — it eats two slots |
| 3 | 1 | prefetchable | reading has no side effect, so caching and burst transfers are allowed |
| rest | address | the base address | the low bits are hardwired to zero for the size |
Table 105.21 — The low bits of a BAR say what kind it is
examples-en/apx-links/pci_config.c
/* PCI 설정 공간(256바이트)을 필드대로 짓고 되읽는다. 그리고 BAR 의 *크기를 알아내는*
고전적인 수법 --- 전부 1을 써 보고 되읽기 --- 을 장치 흉내로 재현한다. */
#include <stdio.h>
#include <stdint.h>
#include <string.h>
static unsigned char cfg[256];
/* 각 BAR 가 실제로 요구하는 크기. 장치가 하드웨어로 정해 두는 값이다. */
static uint32_t bar_size[6] = { 16u << 20, 0, 256u << 20, 0, 0, 256 };
static int bar_is_io[6] = { 0, 0, 0, 0, 0, 1 };
static int bar_is_64[6] = { 0, 0, 1, 0, 0, 0 }; /* BAR2+3 이 짝을 이룬다 */
static uint32_t rd32(unsigned off)
{ uint32_t v = 0; for (int i = 3; i >= 0; i--) v = v << 8 | cfg[off + i]; return v; }
static void wr32(unsigned off, uint32_t v)
{ for (int i = 0; i < 4; i++) cfg[off + i] = (unsigned char)(v >> (8 * i)); }
static uint16_t rd16(unsigned off) { return (uint16_t)(cfg[off] | cfg[off + 1] << 8); }
/* 장치 흉내: BAR 에 값을 쓰면, 크기보다 낮은 비트는 장치가 *0 으로 굳혀* 돌려준다. */
static void bar_write(int i, uint32_t v)
{
unsigned off = 0x10 + 4u * (unsigned)i;
uint32_t low = bar_is_io[i] ? 1u : (uint32_t)(bar_is_64[i] ? 0x4 : 0x0);
uint32_t mask = ~(bar_size[i] - 1);
wr32(off, (v & mask) | low);
}
int main(void)
{
/* ── 머리말 필드 ── */
wr32(0x00, 0x10FA8086u); /* 0x00 제조사 0x8086, 0x02 장치 0x10FA */
wr32(0x04, 0x00100007u); /* 0x04 명령: 메모리·IO·버스마스터 켬 / 0x06 상태 */
wr32(0x08, 0x02000003u); /* 0x08 개정 03, 0x09~0x0B 분류: 02 00 00 = 이더넷 */
wr32(0x0C, 0x00000010u); /* 0x0C 캐시줄 16, 0x0E 머리말 종류 0 */
for (int i = 0; i < 6; i++)
if (bar_size[i])
/* 입출력 공간은 64 KiB 짜리 좁은 세계라 주소도 작다(옛 x86 의 흔적) */
bar_write(i, bar_is_io[i] ? 0xC000u : 0xF0000000u + (uint32_t)i * 0x1000000u);
wr32(0x2C, 0x00108086u); /* 하위 시스템 */
cfg[0x34] = 0x50; /* 능력 목록의 첫 자리 */
cfg[0x3C] = 11; /* 인터럽트 선 (옛 방식) */
cfg[0x3D] = 1; /* 인터럽트 핀 A */
/* 능력 목록: MSI(0x05) → PCIe(0x10) → 끝 */
cfg[0x50] = 0x05; cfg[0x51] = 0x60;
cfg[0x60] = 0x10; cfg[0x61] = 0x00;
printf("== configuration space header (type 0) ==\n");
printf(" %-8s %-6s %-18s %s\n", "offset", "size", "name", "value");
printf(" 0x00 2 vendor ID 0x%04X%s\n", rd16(0x00),
rd16(0x00) == 0x8086 ? " (Intel)" : "");
printf(" 0x02 2 device ID 0x%04X\n", rd16(0x02));
printf(" 0x04 2 command 0x%04X [IO %s · memory %s · bus master %s]\n",
rd16(0x04), rd16(0x04) & 1 ? "on" : "off", rd16(0x04) & 2 ? "on" : "off",
rd16(0x04) & 4 ? "on" : "off");
printf(" 0x06 2 status 0x%04X [capability list %s]\n", rd16(0x06),
rd16(0x06) & 0x10 ? "present" : "absent");
printf(" 0x08 1 revision 0x%02X\n", cfg[0x08]);
printf(" 0x09-0B 3 class %02X %02X %02X = %s\n",
cfg[0x0B], cfg[0x0A], cfg[0x09],
cfg[0x0B] == 0x02 ? "network controller (Ethernet)" : "other");
printf(" 0x0E 1 header type 0x%02X (0=device, 1=bridge)\n", cfg[0x0E]);
printf(" 0x34 1 capability pointer 0x%02X\n", cfg[0x34]);
printf(" 0x3C-3D 2 interrupt line/pin %u / %c\n\n", cfg[0x3C], 'A' + cfg[0x3D] - 1);
printf("== decoding the BARs ==\n");
printf(" %-6s %-14s %-12s %-10s %s\n", "BAR", "raw value", "kind", "note", "base address");
for (int i = 0; i < 6; i++) {
uint32_t v = rd32(0x10 + 4u * (unsigned)i);
if (v == 0) { printf(" BAR%-3d (unused)\n", i); continue; }
if (v & 1) {
printf(" BAR%-3d 0x%08X %-12s %-10s 0x%08X\n", i, v, "I/O space", "---",
v & ~0x3u);
} else {
const char *w = ((v >> 1) & 3) == 2 ? "64-bit" : "32-bit";
printf(" BAR%-3d 0x%08X %-12s %-10s 0x%08X\n", i, v, "memory space",
w, v & ~0xFu);
if (((v >> 1) & 3) == 2) { printf(" BAR%-3d (upper 32 bits of the BAR above)\n", i + 1); i++; }
}
}
printf("\n== how the size is found --- write all ones and read back ==\n");
printf(" a device holds the bits below its requested size at zero.\n");
printf(" so write all ones and the zeros read back tell you the size.\n\n");
for (int i = 0; i < 6; i++) {
if (!bar_size[i]) continue;
unsigned off = 0x10 + 4u * (unsigned)i;
uint32_t saved = rd32(off);
bar_write(i, 0xFFFFFFFFu); /* ① 전부 1을 쓴다 */
uint32_t probe = rd32(off); /* ② 되읽는다 */
uint32_t mask = probe & (bar_is_io[i] ? ~0x3u : ~0xFu);
uint32_t size = ~mask + 1; /* ③ 뒤집고 1을 더하면 크기 */
bar_write(i, saved); /* ④ 원래 값을 되돌려 놓는다 */
printf(" BAR%d: read back 0x%08X -> size %u bytes (%s) %s\n", i, probe, size,
size >= (1u << 20) ? "in MiB" : size >= 1024 ? "in KiB" : "bytes",
size == bar_size[i] ? "matches what the device asked for" : "does not match");
}
printf("\n * these four steps are what real firmware and kernels do. Forget the last one\n");
printf(" (restoring) and the device lands at the wrong address and the machine hangs.\n");
printf("\n== walking the capability list ==\n");
unsigned p = cfg[0x34];
while (p) {
unsigned id = cfg[p], next = cfg[p + 1];
printf(" 0x%02X: capability 0x%02X (%s) -> next 0x%02X\n", p, id,
id == 0x05 ? "MSI --- an interrupt sent as a memory write" :
id == 0x10 ? "PCI Express" : "other", next);
p = next;
}
printf(" * it is a linked list. Zero means the end --- following the chain tells you\n");
printf(" what this device knows how to do.\n");
return 0;
}
Output
== configuration space header (type 0) ==
offset size name value
0x00 2 vendor ID 0x8086 (Intel)
0x02 2 device ID 0x10FA
0x04 2 command 0x0007 [IO on · memory on · bus master on]
0x06 2 status 0x0010 [capability list present]
0x08 1 revision 0x03
0x09-0B 3 class 02 00 00 = network controller (Ethernet)
0x0E 1 header type 0x00 (0=device, 1=bridge)
0x34 1 capability pointer 0x50
0x3C-3D 2 interrupt line/pin 11 / A
== decoding the BARs ==
BAR raw value kind note base address
BAR0 0xF0000000 memory space 32-bit 0xF0000000
BAR1 (unused)
BAR2 0xF0000004 memory space 64-bit 0xF0000000
BAR3 (upper 32 bits of the BAR above)
BAR4 (unused)
BAR5 0x0000C001 I/O space --- 0x0000C000
== how the size is found --- write all ones and read back ==
a device holds the bits below its requested size at zero.
so write all ones and the zeros read back tell you the size.
BAR0: read back 0xFF000000 -> size 16777216 bytes (in MiB) matches what the device asked for
BAR2: read back 0xF0000004 -> size 268435456 bytes (in MiB) matches what the device asked for
BAR5: read back 0xFFFFFF01 -> size 256 bytes (bytes) matches what the device asked for
* these four steps are what real firmware and kernels do. Forget the last one
(restoring) and the device lands at the wrong address and the machine hangs.
== walking the capability list ==
0x50: capability 0x05 (MSI --- an interrupt sent as a memory write) -> next 0x60
0x60: capability 0x10 (PCI Express) -> next 0x00
* it is a linked list. Zero means the end --- following the chain tells you
what this device knows how to do.
The middle of the demonstration is the cleverest part of this specification. There is no field in which to ask a device its size. Instead you write all ones into a BAR and read it back, and the low bits the device holds at zero tell you the size. Four steps — write, read back, invert and add one, restore the original value — are the heart of enumeration, and leaving out the last step hangs the machine.
Q. What is lspci actually showing?
A. It reads this business card (the configuration space) and renders it in human words. The vendor and device IDs are only numbers; the names are looked up in a list file called pci.ids. Which is why a new device in an old system shows up as “Unknown device 8086:10fa” — the device is not broken, the list of names is stale.
The motorway going out — USB#
USB’s design principle is one sentence. The host commands everything. A device cannot speak on its own; it answers only when the host asks.
| Name | Generation | On the wire | Used for |
|---|---|---|---|
| Low Speed | 1.x | 1.5 Mbit/s | mice and keyboards |
| Full Speed | 1.x | 12 Mbit/s | audio and simple devices |
| High Speed | 2.0 | 480 Mbit/s | storage and cameras |
| SuperSpeed | 3.x | 5 to 20 Gbit/s | external SSDs and displays |
Table 105.22 — USB speed grades
| Name | What it guarantees | Used for | Note |
|---|---|---|---|
| control | delivery | recognising and configuring a device | every device must have it |
| bulk | delivery, but not when | storage and printers | it uses whatever bandwidth is left |
| interrupt | delivery within a set period | mice and keyboards | despite the name it is not an interrupt — the host asks periodically |
| isochronous | timing guaranteed, delivery not | audio and video | late data is useless, so it is not retransmitted |
Table 105.23 — USB’s four transfer types
★ The third row is the most confusing word in this whole appendix. USB’s “interrupt transfer” is not a device interrupting. The host asks “anything to say?” every millisecond or so, and the device answers only then. The real interrupt is the IRQ of the next section.
A common misconception. plug in a USB device and the device speaks to the computer
Interruption — IRQs#
So far it was “how things are moved”. Now it is “who speaks when”. The line by which a device tells the CPU “something happened to me” is the IRQ (interrupt request).
Where chapter 81 and the appendix on C without an operating system covered what may be done inside a handler, here we look at the stage before it — the wire and its value.
| edge | level | So | |
|---|---|---|---|
| what is the signal | the instant the line changes | while the line is held low (or high) | |
| can it be missed | yes — miss the instant and it is gone | no — it is still there | level is safer |
| how it is cleared | it clears itself | a device register must be read to clear it | not clearing it raises it endlessly |
| sharing between devices | hard — two at once look like one | easy — ask who is still holding it | a shared IRQ is properly level triggered |
Table 105.24 — Edge triggering and level triggering
★ The third row is the trap seen in the appendix on C without an operating system. With level triggering, a handler that does not read the device’s status register leaves the request asserted, and the program spins inside the handler forever. And the word that stops the compiler removing that read is volatile.
| Scheme | How | Device count | Problem |
|---|---|---|---|
| a dedicated line | one IRQ line per device | as many as there are lines | pins run out |
| a shared line | several devices share one line | several | everyone must be asked who raised it |
| MSI | an interrupt becomes writing a value to an agreed address | effectively unlimited | no wire, so no sharing problem |
| MSI-X | MSI extended, a different number per queue | thousands | each queue can be routed to a different core |
Table 105.25 — How interrupt delivery changed
Q. How can “writing a value to memory” replace an interrupt?
A. PCIe already has a path that carries data. So agree that when a device writes a set value to “a special address”, the interrupt controller catches that write and wakes the CPU. The real gain is not saving a wire but being able to number each interrupt — so “which queue filled up” is known without asking.
examples-en/apx-links/signal_cost.c
/* 「끼어들기에는 값이 있다」를 이 기계에서 *실제로 재어* 본다.
★ 주의: 이것은 하드웨어 인터럽트가 아니라 운영체제의 신호다. 자릿수의 감각을 얻는
용도이고, 하드웨어 IRQ 지연과 같은 수가 아니다(그쪽이 대개 더 짧다). */
#define _POSIX_C_SOURCE 200809L
#include <stdio.h>
#include <signal.h>
#include <time.h>
#include <stdint.h>
static volatile sig_atomic_t hits;
static void handler(int sig) { (void)sig; hits++; }
static double now_ns(void)
{
struct timespec ts;
clock_gettime(CLOCK_MONOTONIC, &ts);
return (double)ts.tv_sec * 1e9 + (double)ts.tv_nsec;
}
static volatile int sink;
static void plain(void) { sink++; }
static void (*volatile plain_p)(void) = plain;
int main(void)
{
struct sigaction sa = { 0 };
sa.sa_handler = handler;
sigaction(SIGUSR1, &sa, NULL);
const long N = 200000;
/* 몸풀기 --- 첫 몇 번은 캐시가 차가워 느리다 */
for (long i = 0; i < 1000; i++) raise(SIGUSR1);
double t0 = now_ns();
for (long i = 0; i < N; i++) raise(SIGUSR1);
double t1 = now_ns();
double per_signal = (t1 - t0) / (double)N;
t0 = now_ns();
for (long i = 0; i < N; i++) plain_p();
t1 = now_ns();
double per_call = (t1 - t0) / (double)N;
printf("== measured on this machine (mean of %ld) ==\n", N);
printf(" one ordinary function call : %8.1f ns\n", per_call);
printf(" raising and handling one signal: %8.1f ns\n", per_signal);
printf(" ratio : %8.1f times\n\n", per_signal / per_call);
printf(" the handler ran %lld times (a check that none were missed)\n\n",
(long long)hits);
printf("== what this number tells you ==\n");
printf(" that one interruption is not free, and that it costs far more than an\n");
printf(" ordinary function call --- because each interruption must put the state of\n");
printf(" the work somewhere, go to the handler, and take it back out again.\n\n");
printf(" from it, how many per second can be borne:\n");
printf(" %-16s %-16s %s\n", "events/s", "CPU used", "note");
const long rates[] = { 1000, 10000, 100000, 1000000 };
for (unsigned i = 0; i < sizeof rates / sizeof *rates; i++) {
double busy = per_signal * (double)rates[i] / 1e9 * 100.0;
char pct[16]; snprintf(pct, sizeof pct, "%.2f%%", busy);
printf(" %-16ld %-16s %s\n", rates[i], pct,
busy > 100 ? "impossible --- it would do nothing but handle them"
: busy > 50 ? "dangerous --- no room for other work"
: busy > 5 ? "workable, but the margin is thin" : "comfortable");
}
printf("\n * again, this is the cost of an operating system signal. A hardware interrupt\n");
printf(" is usually far cheaper (a few hundred nanoseconds). But the *shape* of the\n");
printf(" arithmetic is the same --- learning that shape is the point of this demonstration.\n");
return 0;
}
Output
== measured on this machine (mean of 200000) ==
one ordinary function call : 1.9 ns
raising and handling one signal: 2122.5 ns
ratio : 1139.0 times
the handler ran 201000 times (a check that none were missed)
== what this number tells you ==
that one interruption is not free, and that it costs far more than an
ordinary function call --- because each interruption must put the state of
the work somewhere, go to the handler, and take it back out again.
from it, how many per second can be borne:
events/s CPU used note
1000 0.21% comfortable
10000 2.12% comfortable
100000 21.22% workable, but the margin is thin
1000000 212.25% impossible --- it would do nothing but handle them
* again, this is the cost of an operating system signal. A hardware interrupt
is usually far cheaper (a few hundred nanoseconds). But the *shape* of the
arithmetic is the same --- learning that shape is the point of this demonstration.
★ The demonstration above measures operating system signals, not hardware interrupts (and says so itself). A hardware IRQ is usually far cheaper. The number is still useful because the shape of the arithmetic is the same — “cost per event × events per second = CPU occupied”.
Moving it instead — DMA#
If you cannot be interrupted per byte, what then? The answer is “take the CPU out”.
Figure 105.2 — The path where the CPU carries the data, and the path where DMA does.
| Mode | How | Trade |
|---|---|---|
| single transfer | one word at a time, returning the bus to the CPU in between | the CPU is blocked less, and it is slower |
| burst | a set number in one go | fast, and the CPU waits meanwhile |
| scatter-gather | given a list of “4 KiB here, 8 KiB there” it carries them itself | scattered memory in one go — the list itself becomes a data structure |
Table 105.26 — How DMA uses the bus
| Problem | What it is | How it is solved |
|---|---|---|
| cache coherence | DMA writes to memory while the CPU looks at the cache | invalidate before reading, flush before sending. Or an uncached region |
| kinds of address | the address the device sees may differ from the one the CPU sees | the driver converts to a “device address” and passes that |
| security and isolation | a device can read and write any memory | an IOMMU limits what each device can see |
Table 105.27 — Three things that catch you with DMA
★ The first row joins exactly on to the volatile story in the appendix on C without an operating system. volatile restrains the compiler and does nothing to the cache. Different layers need different devices.
Counter-example. putting a DMA buffer on the stack
The three compared — polling, interrupts and DMA#
The arithmetic that amounts to this appendix’s conclusion.
examples-en/apx-links/link_budget.c
/* 같은 일을 폴링·인터럽트·DMA 로 할 때 무엇이 얼마나 드는가 --- 예산 계산.
하드웨어가 없으므로 이것은 *계산*이다. 값(비용)은 흔히 인용되는 자릿수를 가정으로
두고, 그 가정을 화면에 함께 적는다 --- 가정을 감추면 계산이 아니라 주장이 된다. */
#include <stdio.h>
/* 가정 --- 요즘 흔한 마이크로컨트롤러/PC 급의 어림값 */
#define IRQ_COST_US 1.0 /* 인터럽트 한 번 처리에 드는 시간 */
#define POLL_COST_US 0.2 /* 폴링 한 번(레지스터 읽고 판정)에 드는 시간 */
struct link { const char *name; double bytes_per_sec; };
static void budget(struct link L, double dma_block)
{
double irq_rate = L.bytes_per_sec; /* 바이트마다 끼어들기 */
double irq_busy = irq_rate * IRQ_COST_US / 1e6 * 100.0;
double dma_rate = L.bytes_per_sec / dma_block; /* 덩어리마다 한 번 */
double dma_busy = dma_rate * IRQ_COST_US / 1e6 * 100.0;
char a[24], b[24];
snprintf(a, sizeof a, "%.1f%%", irq_busy);
snprintf(b, sizeof b, "%.3f%%", dma_busy);
printf(" %-22s %12.0f %10s %12.0f %10s\n", L.name, irq_rate, a, dma_rate, b);
}
int main(void)
{
printf("== assumptions (the values used) ==\n");
printf(" handling one interrupt : %.1f microseconds\n", IRQ_COST_US);
printf(" one poll (read and judge) : %.1f microseconds\n", POLL_COST_US);
printf(" DMA batch size : 256 bytes\n");
printf(" * all three differ by machine. Change them and the conclusion changes --- that is the point.\n\n");
printf("== an interrupt per byte vs DMA in 256-byte batches ==\n");
printf(" %-22s %12s %10s %12s %10s\n", "link", "IRQ/s", "CPU", "DMA IRQ/s", "CPU");
struct link links[] = {
{ "UART 9600 8N1", 960 },
{ "UART 115200 8N1", 11520 },
{ "I2C 400 kHz", 44000 },
{ "SPI 10 MHz", 1250000 },
{ "USB 2.0 (effective)", 30000000 },
{ "1 Gbit Ethernet", 118000000 },
};
for (unsigned i = 0; i < sizeof links / sizeof *links; i++) budget(links[i], 256);
printf("\n * reading the table downwards shows when DMA becomes necessary.\n");
printf(" on a slow link an interrupt per byte is no trouble; past some point\n");
printf(" that scheme leaves the machine doing nothing else.\n");
printf("\n== where the crossover is (assuming an interrupt per byte) ==\n");
printf(" %-14s %-16s %s\n", "CPU used", "events/s", "the data rate that means");
for (double busy = 5; busy <= 100; busy *= 2) {
double rate = busy / 100.0 * 1e6 / IRQ_COST_US; /* 사건/초 */
char pct[16]; snprintf(pct, sizeof pct, "%.0f%%", busy);
printf(" %-14s %-16.0f %.1f KiB/s\n", pct, rate, rate / 1024.0);
}
/* 한글은 한 글자가 두 칸이라 %-Ns 로는 안 맞는다 --- 줄마다 손으로 적는다 */
printf("\n== the three schemes compared ==\n");
printf(" polling : the CPU keeps asking. latency = about half the period.\n");
printf(" suits very frequent events, or very simple places.\n");
printf(" interrupt : it wakes only when there is work. latency = the cost of waking.\n");
printf(" suits events that come now and then.\n");
printf(" DMA : it wakes only when a batch ends. latency = until the batch fills.\n");
printf(" suits data that is large and steady.\n");
printf("\n== there are places where polling is better ==\n");
double poll_hz[] = { 1e3, 1e4, 1e5, 1e6 };
printf(" %-16s %-14s %s\n", "poll rate", "CPU used", "mean latency");
for (unsigned i = 0; i < 4; i++) {
char pct[16]; snprintf(pct, sizeof pct, "%.2f%%", poll_hz[i] * POLL_COST_US / 1e6 * 100.0);
printf(" %-11.0f /s %-14s %.1f microseconds\n", poll_hz[i], pct, 1e6 / poll_hz[i] / 2);
}
printf(" * when events are very frequent polling wins --- there is always work, so\n");
printf(" there is no reason to pay the cost of waking each time. That is why fast\n");
printf(" network drivers switch to polling under load (Linux NAPI).\n");
return 0;
}
Output
== assumptions (the values used) ==
handling one interrupt : 1.0 microseconds
one poll (read and judge) : 0.2 microseconds
DMA batch size : 256 bytes
* all three differ by machine. Change them and the conclusion changes --- that is the point.
== an interrupt per byte vs DMA in 256-byte batches ==
link IRQ/s CPU DMA IRQ/s CPU
UART 9600 8N1 960 0.1% 4 0.000%
UART 115200 8N1 11520 1.2% 45 0.005%
I2C 400 kHz 44000 4.4% 172 0.017%
SPI 10 MHz 1250000 125.0% 4883 0.488%
USB 2.0 (effective) 30000000 3000.0% 117188 11.719%
1 Gbit Ethernet 118000000 11800.0% 460938 46.094%
* reading the table downwards shows when DMA becomes necessary.
on a slow link an interrupt per byte is no trouble; past some point
that scheme leaves the machine doing nothing else.
== where the crossover is (assuming an interrupt per byte) ==
CPU used events/s the data rate that means
5% 50000 48.8 KiB/s
10% 100000 97.7 KiB/s
20% 200000 195.3 KiB/s
40% 400000 390.6 KiB/s
80% 800000 781.2 KiB/s
== the three schemes compared ==
polling : the CPU keeps asking. latency = about half the period.
suits very frequent events, or very simple places.
interrupt : it wakes only when there is work. latency = the cost of waking.
suits events that come now and then.
DMA : it wakes only when a batch ends. latency = until the batch fills.
suits data that is large and steady.
== there are places where polling is better ==
poll rate CPU used mean latency
1000 /s 0.02% 500.0 microseconds
10000 /s 0.20% 50.0 microseconds
100000 /s 2.00% 5.0 microseconds
1000000 /s 20.00% 0.5 microseconds
* when events are very frequent polling wins --- there is always work, so
there is no reason to pay the cost of waking each time. That is why fast
network drivers switch to polling under load (Linux NAPI).
Three things to note.
First, there is no “good scheme”, only “the scheme that fits this speed”. A 9600 baud UART is only 960 bytes per second, so even an interrupt per byte takes 0.1% of the CPU. Use the same scheme on 1 Gbit Ethernet and the CPU is 118 times short.
Second, DMA does not remove “the moving”; it reduces “how often you are interrupted”. Batch 256 bytes and the events drop to a 256th. In exchange you must wait for the batch to fill, so latency grows — which is why data with a short deadline gets small batches.
Third, when events are very frequent, polling wins. There is always work anyway, so there is no reason to pay the cost of waking each time. Linux network drivers turning interrupts off under load and switching to polling (NAPI) is exactly that. The three are not a ladder but places to move between as the load changes.
| polling | interrupts | DMA | |
|---|---|---|---|
| what the CPU does | keeps asking | wakes only when there is work | only starts and finishes |
| latency | about half the polling period | the cost of waking | until the batch fills |
| when events are rare | wasteful | right | excessive |
| when events are frequent | right | the machine does nothing else | right |
| when the data is large | every byte goes through the CPU | likewise | right — it does not go through the CPU |
Table 105.28 — The three at a glance
What to take from this#
Recap
- Four questions settle a connection — how many wires, a clock, addressing, who starts.
- The faster it goes the more parallel loses. Today’s answer is one very fast serial line, bundled several times over.
- A UART cuts characters out with an agreed shape and no clock. The price is a baud error budget, and the theoretical limit (about 5%) and the design rule (2 to 3%) are different numbers.
- I2C carries many devices on two wires — a design that only ever pulls down gets ACK and arbitration for free.
- SPI is a ring of two shift registers. So full duplex is free, and a mode mismatch shifts every value by one place.
- PCI’s idea is that every device carries a business card of the same shape. Even the size is discovered by “writing ones and reading back”.
- USB has the host command everything. Its “interrupt transfer” is not an interrupt but a periodic question.
- Polling, interrupts and DMA are not a ladder but three places chosen by load. The choosing is arithmetic, not taste — cost per event × events per second.