Proven C Book←↑→

Appendix N — How a disk is divided

Using a computer you meet these words. “Partition the disk”, “format it”, “the MBR is damaged”, “you need to convert to GPT”. You half know what they mean, and usually go by without knowing what is actually written on the disk, and how.

This appendix opens those bytes. And by this book’s discipline, what the tables describe is actually built by an example and read back as evidence.

Platform note. what this appendix rests on, and its limits

The authorities for these structures are their specifications — for the MBR, convention and each operating system’s documentation; for GPT, the UEFI specification; for FAT, Microsoft’s FAT specification. What is written here belongs to their opening pages, and building real tools means reading the specifications.

This machine’s disk was not read. The discipline is to leave nobody’s machine circumstances in the document. Instead the examples build the bytes to the specification and decode them again.

Words to know first#

Only these words are used later. There is no need to memorise them now — come back to the table.

WordMeaningBy analogy
sectorthe smallest cell a disk reads and writes; usually 512 bytesone line of a notebook
LBAthe number given to a sector, from 0 (logical block addressing)the line number
CHSthe old way of naming a place by cylinder, head and sector“volume, page, line”
partitionone piece the disk is divided into. Merely “from this sector, this many”a divider drawn in the notebook
partition tablethe list of those dividers, written at the front of the diskthe contents page
volumewhat the operating system treats as one storage space“the D drive”
filesystemhow files and folders are managed inside a partition (FAT32, ext4 …)the tidying rules within a section
formatwriting a filesystem’s basic structures afresh inside a partitionclearing a section and ruling a new form
mountattaching that filesystem somewhere in the operating systemlaying the section open on the desk

Table 105.1 — The words used in this appendix

A common misconception. formatting erases the data

Usually it does not. A quick format merely rewrites the filesystem’s management structures (the table of what is where); the bytes of the data area remain. That is where recovery tools live. Which also means: do not hand a disk to someone else believing it has been wiped.

A disk is a row of numbered cells#

What the machine knows is very simple. A disk is a row of sectors, and each sector has a number from 0. “Read sector 3”, “write these 512 bytes to sector 7” — that is effectively all the device knows.

Partitions and filesystems are laid on that row by convention. The disk does not know how it is divided.

disk-layers

Figure 105.1 — Three layers on one disk — the row of sectors, the partition table, the filesystem.

Places used to be named by CHS: cylinder, head, sector. Names from when a disk really was several platters. Now everything is LBA — just a number.

CalledPhysical sectorLogical sectorWhat changes
512n512 bytes512 bytesold disks. The two agree
512e4096 bytes512 bytesinside it is 4 KiB while pretending to be 512 — misalignment makes it slow
4Kn4096 bytes4096 bytes4 KiB with nothing hidden. Some older software cannot read it

Table 105.2 — Three kinds of sector size

Q. Why is misalignment slow on 512e?

A. The disk really works in 4 KiB chunks, so if software writes 512 bytes at a misaligned place, the device must read 4 KiB, change 512 bytes inside it, and write 4 KiB back (read-modify-write). One write becomes three pieces of work. Today’s convention of starting partitions at sector 2048 (= 1 MiB) exists to avoid this.

Why divide at all#

Can a disk not be used whole? Sometimes it can. But there are reasons to divide.

MBR — everything written into the first 512 bytes#

The oldest scheme. One sector, sector 0 of the disk, holds both the boot code and the partition table.

OffsetSizeNameWhatTypical value
0x000446bootstrap codethe first code the CPU runs when booting. Only the first piece of a bootloader fitsmachine instructions
0x1B84disk signaturethe number by which Windows recognises the disk (absent in the oldest MBRs)four arbitrary bytes
0x1BC2reservedusually zero00 00
0x1BE16partition entry 1see Table 105.4 below—
0x1CE16partition entry 2likewise—
0x1DE16partition entry 3likewise—
0x1EE16partition entry 4likewise—
0x1FE2boot signaturewithout it the BIOS does not accept this as a boot sector55 AA

Table 105.3 — The whole layout of the MBR’s 512 bytes

★ The first and last rows tell the character of this scheme entirely. The boot code and the layout of the disk share the same 512 bytes. Which is why the code gets 446 bytes and there are only four partitions.

The 16-byte partition entry#

OffsetSizeNameWhatTypical value
+01boot flag0x80 means “boot here”, 0x00 means not. Anything else is wrong80 or 00
+13starting CHSthe old start position. Today it is not readFE FF FF
+41partition typeone byte hinting what is inside (Table 105.6)83, 07, 0C …
+53ending CHSthe old end position. Also unreadFE FF FF
+84starting LBAthe value actually used. The sector this partition starts at2048
+124sector countthe value actually used. The length of this partition2097152 (= 1 GiB)

Table 105.4 — The 16 bytes of an MBR partition entry

How the values pack into the three CHS bytes is worth recording too. This cramping is the source of the old limits.

ByteBitsWhatRange
+07–0head0–255
+17–6the top 2 bits of the cylinderthe cylinder is 10 bits in all
+15–0sector1–63 — there is no 0
+27–0the low 8 bits of the cylinder0–1023

Table 105.5 — Bit layout of the three CHS bytes

examples-en/apx-disk/mbr_detail.c

/* MBR 을 필드 단위로 짓고 되읽는다 --- 표에 적은 오프셋이 진짜인지 코드가 증언한다.
   주의: 디스크를 건드리지 않는다. 기억 속 배열만 채우고 다시 읽어 화면에 찍는다. */
#include <stdio.h>
#include <stdint.h>
#include <string.h>

#define SEC 512u

static void put32(unsigned char *p, uint32_t v)
{ p[0] = v & 0xff; p[1] = v >> 8 & 0xff; p[2] = v >> 16 & 0xff; p[3] = v >> 24; }
static uint32_t get32(const unsigned char *p)
{ return (uint32_t)p[0] | (uint32_t)p[1] << 8 | (uint32_t)p[2] << 16 | (uint32_t)p[3] << 24; }

/* CHS 는 3바이트에 실린다: 머리 8비트, 실린더 10비트, 섹터 6비트.
   실린더의 위 두 비트가 섹터 바이트의 위 두 비트로 올라간다 --- 그 비좁음이 한계를 만든다. */
static void put_chs(unsigned char *p, uint32_t cyl, uint32_t head, uint32_t sect)
{
    if (cyl > 1023) { p[0] = 0xfe; p[1] = 0xff; p[2] = 0xff; return; }  /* 못 적으면 관용값 */
    p[0] = (unsigned char)head;
    p[1] = (unsigned char)(((cyl >> 2) & 0xc0) | (sect & 0x3f));
    p[2] = (unsigned char)(cyl & 0xff);
}
static void get_chs(const unsigned char *p, unsigned *cyl, unsigned *head, unsigned *sect)
{ *head = p[0]; *sect = p[1] & 0x3f; *cyl = ((unsigned)(p[1] & 0xc0) << 2) | p[2]; }

static void put_part(unsigned char *e, uint8_t boot, uint8_t type,
                     uint32_t lba, uint32_t count)
{
    e[0] = boot;
    put_chs(e + 1, lba / (255 * 63), (lba / 63) % 255, lba % 63 + 1);
    e[4] = type;
    put_chs(e + 5, (lba + count - 1) / (255 * 63), ((lba + count - 1) / 63) % 255,
            (lba + count - 1) % 63 + 1);
    put32(e + 8, lba);
    put32(e + 12, count);
}

static const char *type_name(uint8_t t)
{
    switch (t) {
    case 0x00: return "empty";
    case 0x05: return "extended (CHS)";
    case 0x07: return "NTFS / exFAT";
    case 0x0b: return "FAT32 (CHS)";
    case 0x0c: return "FAT32 (LBA)";
    case 0x0e: return "FAT16 (LBA)";
    case 0x0f: return "extended (LBA)";
    case 0x82: return "Linux swap";
    case 0x83: return "Linux";
    case 0xee: return "GPT protective MBR";
    case 0xef: return "EFI system partition";
    default:   return "other";
    }
}

static void show_entry(const char *tag, const unsigned char *e, uint32_t base)
{
    unsigned c0, h0, s0, c1, h1, s1;
    get_chs(e + 1, &c0, &h0, &s0);
    get_chs(e + 5, &c1, &h1, &s1);
    uint32_t rel = get32(e + 8), cnt = get32(e + 12);
    if (e[4] == 0 && cnt == 0) { printf("  %-10s (empty)\n", tag); return; }
    printf("  %-10s bootable=%s type=0x%02x (%s)\n", tag, e[0] == 0x80 ? "yes" : "no",
           e[4], type_name(e[4]));
    printf("             start LBA=%u (relative) -> %u (absolute)  size=%u sectors = %.1f MiB\n",
           rel, base + rel, cnt, cnt * (double)SEC / (1024 * 1024));
    printf("             CHS start=(%u,%u,%u) end=(%u,%u,%u)%s\n",
           c0, h0, s0, c1, h1, s1,
           (e[1] == 0xfe && e[2] == 0xff) ? "  <- beyond what CHS can express (the idiomatic value)" : "");
}

int main(void)
{
    unsigned char mbr[SEC] = { 0 };

    /* ── 주 파티션 넷 중 셋 + 확장 하나 ─────────────────────────── */
    put_part(mbr + 0x1be, 0x80, 0x0c, 2048,     204800);      /* FAT32(LBA) 100 MiB, 부팅 */
    put_part(mbr + 0x1ce, 0x00, 0x83, 206848,   2097152);     /* 리눅스 1 GiB */
    put_part(mbr + 0x1de, 0x00, 0x82, 2304000,  262144);      /* 스왑 128 MiB */
    put_part(mbr + 0x1ee, 0x00, 0x0f, 2566144,  4194304);     /* 확장(LBA) 2 GiB */
    mbr[510] = 0x55; mbr[511] = 0xaa;

    printf("== the MBR partition table (LBA 0) ==\n");
    for (int i = 0; i < 4; i++) {
        char tag[16]; snprintf(tag, sizeof tag, "entry %d", i + 1);
        show_entry(tag, mbr + 0x1be + 16 * i, 0);
    }
    printf("  signature 0x%02x%02x --- %s\n\n", mbr[510], mbr[511],
           (mbr[510] == 0x55 && mbr[511] == 0xaa) ? "accepted as a boot sector" : "not accepted");

    /* ── 확장 파티션 안의 EBR 사슬 ──────────────────────────────
       규칙이 둘이고, 둘의 기준이 *다르다*:
         1번 항목 = 이 EBR 바로 뒤의 논리 파티션 → 이 EBR 기준의 상대 LBA
         2번 항목 = 다음 EBR 의 자리          → 확장 파티션 시작 기준의 상대 LBA
       이 어긋남이 EBR 을 손으로 읽을 때 가장 많이 틀리는 자리다. */
    const uint32_t ext_start = 2566144;
    unsigned char ebr1[SEC] = { 0 }, ebr2[SEC] = { 0 };

    put_part(ebr1 + 0x1be, 0x00, 0x83, 2048, 1048576);        /* 논리 1: 512 MiB */
    put_part(ebr1 + 0x1ce, 0x00, 0x0f, 1050624, 2097152);     /* 다음 EBR: 확장 시작 기준 */
    ebr1[510] = 0x55; ebr1[511] = 0xaa;

    put_part(ebr2 + 0x1be, 0x00, 0x07, 2048, 2095104);        /* 논리 2: NTFS */
    ebr2[510] = 0x55; ebr2[511] = 0xaa;                       /* 2번 항목 비었다 = 사슬 끝 */

    printf("== the EBR chain inside the extended partition (extended start LBA=%u) ==\n", ext_start);
    uint32_t ebr_lba = ext_start;
    const unsigned char *chain[] = { ebr1, ebr2 };
    for (int i = 0; i < 2; i++) {
        printf("\n  [EBR %d] absolute LBA of this EBR = %u\n", i + 1, ebr_lba);
        show_entry("logical", chain[i] + 0x1be, ebr_lba);        /* 기준: 이 EBR */
        show_entry("next EBR", chain[i] + 0x1ce, ext_start);   /* 기준: 확장 시작 */
        uint32_t next = get32(chain[i] + 0x1ce + 8);
        if (next == 0) { printf("  -> end of the chain\n"); break; }
        ebr_lba = ext_start + next;
    }

    /* ── CHS 의 한계 ─────────────────────────────────────────── */
    printf("\n== the limit of what CHS can express ==\n");
    unsigned long long chs_max = 1024ull * 255 * 63 * SEC;
    printf("  1024 cylinders x 255 heads x 63 sectors x %u bytes = %llu bytes = %.1f GB\n",
           SEC, chs_max, chs_max / 1e9);
    printf("  beyond that the LBA field (4 bytes) takes over: 2^32 sectors x %u = %.1f TB\n",
           SEC, 4294967296.0 * SEC / 1e12);
    printf("  -> the MBR 2 TiB limit comes from here (the sector number is 32 bits).\n");
    return 0;
}

Output

== the MBR partition table (LBA 0) ==
  entry 1    bootable=yes type=0x0c (FAT32 (LBA))
             start LBA=2048 (relative) -> 2048 (absolute)  size=204800 sectors = 100.0 MiB
             CHS start=(0,32,33) end=(12,223,19)
  entry 2    bootable=no type=0x83 (Linux)
             start LBA=206848 (relative) -> 206848 (absolute)  size=2097152 sectors = 1024.0 MiB
             CHS start=(12,223,20) end=(143,106,27)
  entry 3    bootable=no type=0x82 (Linux swap)
             start LBA=2304000 (relative) -> 2304000 (absolute)  size=262144 sectors = 128.0 MiB
             CHS start=(143,106,28) end=(159,187,28)
  entry 4    bootable=no type=0x0f (extended (LBA))
             start LBA=2566144 (relative) -> 2566144 (absolute)  size=4194304 sectors = 2048.0 MiB
             CHS start=(159,187,29) end=(420,208,44)
  signature 0x55aa --- accepted as a boot sector

== the EBR chain inside the extended partition (extended start LBA=2566144) ==

  [EBR 1] absolute LBA of this EBR = 2566144
  logical    bootable=no type=0x83 (Linux)
             start LBA=2048 (relative) -> 2568192 (absolute)  size=1048576 sectors = 512.0 MiB
             CHS start=(0,32,33) end=(65,101,36)
  next EBR   bootable=no type=0x0f (extended (LBA))
             start LBA=1050624 (relative) -> 3616768 (absolute)  size=2097152 sectors = 1024.0 MiB
             CHS start=(65,101,37) end=(195,239,44)

  [EBR 2] absolute LBA of this EBR = 3616768
  logical    bootable=no type=0x07 (NTFS / exFAT)
             start LBA=2048 (relative) -> 3618816 (absolute)  size=2095104 sectors = 1023.0 MiB
             CHS start=(0,32,33) end=(130,138,8)
  next EBR   (empty)
  -> end of the chain

== the limit of what CHS can express ==
  1024 cylinders x 255 heads x 63 sectors x 512 bytes = 8422686720 bytes = 8.4 GB
  beyond that the LBA field (4 bytes) takes over: 2^32 sectors x 512 = 2.2 TB
  -> the MBR 2 TiB limit comes from here (the sector number is 32 bits).

The demonstration built exactly what the table describes and read it back. Three things to note.

First, CHS is already abandoned. Large values are filled with the idiomatic FE FF FF, meaning “a size CHS cannot express”, and the reading side ignores it.

Second, the arithmetic shows where the limits come from. 1024 cylinders × 255 heads × 63 sectors × 512 bytes = 8.4 GB. And since the LBA field is four bytes, 232 sectors × 512 bytes ≈ 2.2 TB — the MBR 2 TiB limit people speak of.

Third, the type byte is only a hint. The values in the table below are convention, “usually this”; what is really inside is known only by opening it.

ValueMeaningNote
0x00emptythe entry is unused
0x05extended (CHS)the old way; only below 8.4 GB
0x07NTFS / exFATboth use the same value — only opening it tells
0x0BFAT32 (CHS)
0x0CFAT32 (LBA)today’s FAT32 is usually this
0x0EFAT16 (LBA)
0x0Fextended (LBA)today’s extended partitions are this
0x82Linux swap
0x83Linuxext4, xfs, everything takes this value
0xEEGPT protective MBR“this disk is GPT”
0xEFEFI system partitionwhen an ESP is used on an MBR disk

Table 105.6 — Partition type bytes commonly seen

When four partitions are not enough — extended partitions and the EBR chain#

With only four entries there can be no fifth partition. The workaround devised was the extended partition. One entry is used as a marker (0x05/0x0F) saying “there are more divisions inside”, and inside it lives a linked list.

ebr-chain

Figure 105.2 — The EBR chain — two entries of one table with different bases.

Before each logical partition sits one sector called an EBR (extended boot record), and only two entries of its partition table are used.

EntryWhat it points atWhat the relative LBA is relative toType byte
1 (0x1BE)the logical partition immediately behind itthis EBR itselfthe real type (0x83 and so on)
2 (0x1CE)the next EBRthe start of the extended partition0x05 or 0x0F
3 and 4unused—0x00

Table 105.7 — The two EBR entries — their bases differ

★ The third column of Table 105.7 is the trap. Two entries of the same table with different bases. The latter part of the demonstration above walks that chain and computes the absolute LBAs — built deliberately, because it is the place most often misread by hand.

Counter-example. deleting an EBR from the middle of the chain

Logical partitions are a linked list. Delete an EBR in the middle and every logical partition behind it disappears at once — the data is still there, but the way to reach it is cut. Which is also why GPT abandoned this structure for an array.

GPT — rebuilt out of an array, a check and a copy#

GPT (the GUID Partition Table) is the same job designed again. What changed, in one line: what was crammed into one sector was spread over an array of sectors, check values were attached so damage can be seen, and the whole thing was duplicated.

MBRGPT
where the layout livesone sector, number 0sector 1 (the header) + an array from sector 2
number of partitions4 (worked around with extended)the header decides — usually 128
partition size limit2 TiB (32-bit sector numbers)effectively none (64-bit)
partition namesnoneyes — 36 characters (UTF-16)
type identificationa one-byte hinta 16-byte GUID
damage detectionnone (only two bytes, 55 AA)two CRC32s — the header and the entry array
copiesnonea second set at the end of the disk
protection from old tools—a protective MBR (type 0xEE)

Table 105.8 — MBR and GPT

The protective MBR — sector 0 is still an MBR#

Sector 0 of a GPT disk still holds an MBR. Only one partition entry, of type 0xEE, covers the whole disk. It is bait, to stop an old tool that does not know GPT from seeing an empty disk and overwriting it. The size field holds the whole disk (or the largest value 32 bits can express).

The 92-byte GPT header#

OffsetSizeNameWhatTypical value
08signaturethe eight characters EFI PART45 46 49 20 50 41 52 54
84revisionthe specification version00 00 01 00 (= 1.0)
124header sizethe range over which the CRC is computed92
164header CRC32the hash of the header with this field zeroedcomputed
204reservedmust be zero00 00 00 00
248LBA of this headerwhere it is itself1
328LBA of the alternate headerwhere the backup isthe last sector
408first usable LBAthe earliest a partition may start34
488last usable LBAthe latest a partition may endend − 33
5616disk GUIDthe unique number of this disk16 bytes
728LBA of the entry arraythe sector where the entries begin2
804number of entrieshow many slots the array has128
844size of one entryusually 128 bytes128
884entry array CRC32the hash of the whole array (count × size)computed

Table 105.9 — Every field of the GPT header (LBA 1, 92 bytes)

★ Look at offset 72. While writing this appendix it was written as 64 by mistake, which overwrote the tail of the disk GUID and made the output strange. That a mismatch between table and code shows itself is how this appendix verifies things.

The 128-byte GPT partition entry#

OffsetSizeNameWhatTypical value
016type GUID“what this partition is for” (Table 105.13)C12A7328-… for an ESP
1616unique GUIDthe label of this one partition — cloning it causes a collisiondifferent for each
328first LBAthe starting sector2048
408last LBAthe ending sector — this sector is included206847
488attribute bitsTable 105.11usually 0
5672nameup to 36 characters in UTF-16LE“EFI System”

Table 105.10 — Every field of a GPT partition entry (128 bytes)

BitMeaningWho reads it
0a system partition — do not touchpartitioning tools
1the firmware ignores itUEFI firmware
2bootable by legacy BIOSlegacy booting
60read onlyWindows
62hiddenWindows
63do not automount (no drive letter)Windows

Table 105.11 — GPT attribute bits worth knowing

A GUID is written one way and stored another#

This is the first trap when reading GPT by hand.

GroupSizeStorage orderExample (C12A7328-F81F-11D2-BA4B-00A0C93EC93B)
14 byteslittle-endian — reversed28 73 2A C1
22 byteslittle-endian — reversed1F F8
32 byteslittle-endian — reversedD2 11
42 bytesas writtenBA 4B
56 bytesas written00 A0 C9 3E C9 3B

Table 105.12 — The five groups of a GUID and their storage order

examples-en/apx-disk/gpt_detail.c

/* GPT 를 필드 단위로 짓고 되읽는다 --- 헤더 92바이트, 항목 128바이트, CRC 둘.
   주의: 디스크를 건드리지 않는다. 기억 속 배열만 채운다. */
#include <stdio.h>
#include <stdint.h>
#include <string.h>

#define SEC        512u
#define ENTRIES    128u          /* 표준이 권하는 최소 개수 */
#define ENTRY_SZ   128u
#define DISK_SECS  4194304u      /* 2 GiB 짜리 디스크라고 하자 */

static uint32_t crc32(const void *buf, size_t n)
{
    const unsigned char *p = buf;
    uint32_t c = 0xFFFFFFFFu;
    for (size_t i = 0; i < n; i++) {
        c ^= p[i];
        for (int k = 0; k < 8; k++) c = (c >> 1) ^ (0xEDB88320u & (uint32_t)-(int32_t)(c & 1));
    }
    return c ^ 0xFFFFFFFFu;
}

static void put32(unsigned char *p, uint32_t v)
{ for (int i = 0; i < 4; i++) p[i] = (unsigned char)(v >> (8 * i)); }
static void put64(unsigned char *p, uint64_t v)
{ for (int i = 0; i < 8; i++) p[i] = (unsigned char)(v >> (8 * i)); }
static uint32_t get32(const unsigned char *p)
{ uint32_t v = 0; for (int i = 3; i >= 0; i--) v = v << 8 | p[i]; return v; }
static uint64_t get64(const unsigned char *p)
{ uint64_t v = 0; for (int i = 7; i >= 0; i--) v = v << 8 | p[i]; return v; }

/* ★ GUID 의 함정: 글로 쓸 때와 디스크에 놓일 때의 바이트 차례가 *다르다*.
   앞의 세 마디(4·2·2바이트)는 작은 끝으로 뒤집혀 저장되고, 뒤의 두 마디(2·6바이트)는
   글자 순서 그대로다. 그래서 16진수 덤프와 문서의 GUID 가 달라 보인다. */
static void guid_parse(unsigned char *out, const char *s)
{
    unsigned b[16]; int n = 0;
    for (const char *p = s; *p && n < 16; ) {
        if (*p == '-') { p++; continue; }
        unsigned hi, lo;
        sscanf(p, "%1x%1x", &hi, &lo);
        b[n++] = hi << 4 | lo; p += 2;
    }
    out[0] = (unsigned char)b[3]; out[1] = (unsigned char)b[2];   /* 첫 마디 뒤집기 */
    out[2] = (unsigned char)b[1]; out[3] = (unsigned char)b[0];
    out[4] = (unsigned char)b[5]; out[5] = (unsigned char)b[4];   /* 둘째 마디 */
    out[6] = (unsigned char)b[7]; out[7] = (unsigned char)b[6];   /* 셋째 마디 */
    for (int i = 8; i < 16; i++) out[i] = (unsigned char)b[i];    /* 나머지는 그대로 */
}
static void guid_text(const unsigned char *g, char *out)
{
    sprintf(out, "%02X%02X%02X%02X-%02X%02X-%02X%02X-%02X%02X-%02X%02X%02X%02X%02X%02X",
            g[3], g[2], g[1], g[0], g[5], g[4], g[7], g[6],
            g[8], g[9], g[10], g[11], g[12], g[13], g[14], g[15]);
}
static void guid_bytes(const unsigned char *g, char *out)
{ for (int i = 0; i < 16; i++) sprintf(out + i * 3, "%02x ", g[i]); }

/* 이름은 UTF-16LE 36글자 자리(72바이트)에 들어간다 --- ASCII 만 쓴다면 이렇게 */
static void put_name(unsigned char *p, const char *ascii)
{ for (int i = 0; ascii[i]; i++) { p[i * 2] = (unsigned char)ascii[i]; p[i * 2 + 1] = 0; } }

#define ESP   "C12A7328-F81F-11D2-BA4B-00A0C93EC93B"
#define LINUX "0FC63DAF-8483-4772-8E79-3D69D8477DE4"
#define SWAP  "0657FD6D-A4AB-43C4-84E5-0933C84B4F4F"

int main(void)
{
    static unsigned char ents[ENTRIES * ENTRY_SZ];   /* = 16384바이트 = 32섹터 */
    unsigned char hdr[92] = { 0 };
    char t1[64], t2[64];

    /* ── 항목 셋 ─────────────────────────────────────────────── */
    struct { const char *type, *name; uint64_t first, last; uint64_t attr; } part[] = {
        { ESP,   "EFI System",  2048,   206847,  0 },
        { LINUX, "root",        206848, 3358719, 0 },
        { SWAP,  "swap",        3358720, 4194270, 1ull << 63 },  /* 63: 자동 마운트 금지 */
    };
    for (unsigned i = 0; i < sizeof part / sizeof *part; i++) {
        unsigned char *e = ents + i * ENTRY_SZ;
        guid_parse(e, part[i].type);                       /* 0  : 종류 GUID */
        char uniq[40];
        sprintf(uniq, "12345678-1234-5678-9ABC-DEF01234567%X", i);   /* 파티션마다 달라야 한다 */
        guid_parse(e + 16, uniq);                          /* 16 : 이 파티션의 고유 GUID */
        put64(e + 32, part[i].first);                      /* 32 : 첫 LBA */
        put64(e + 40, part[i].last);                       /* 40 : 마지막 LBA(포함) */
        put64(e + 48, part[i].attr);                       /* 48 : 속성 비트 */
        put_name(e + 56, part[i].name);                    /* 56 : 이름 UTF-16LE 72바이트 */
    }

    /* ── 헤더 ────────────────────────────────────────────────── */
    uint64_t backup_lba = DISK_SECS - 1;
    uint64_t ents_sectors = (ENTRIES * ENTRY_SZ + SEC - 1) / SEC;   /* 32 */
    memcpy(hdr, "EFI PART", 8);            /* 0  : 서명 */
    put32(hdr + 8, 0x00010000u);           /* 8  : 개정 1.0 */
    put32(hdr + 12, 92);                   /* 12 : 헤더 크기 */
    put32(hdr + 16, 0);                    /* 16 : 헤더 CRC --- 계산 전에는 0 */
    put32(hdr + 20, 0);                    /* 20 : 예약 */
    put64(hdr + 24, 1);                    /* 24 : 이 헤더의 LBA */
    put64(hdr + 32, backup_lba);           /* 32 : 짝 헤더의 LBA */
    put64(hdr + 40, 2 + ents_sectors);     /* 40 : 쓸 수 있는 첫 LBA = 34 */
    put64(hdr + 48, backup_lba - ents_sectors - 1); /* 48 : 쓸 수 있는 마지막 LBA */
    guid_parse(hdr + 56, "01234567-89AB-CDEF-0123-456789ABCDEF");  /* 56 : 디스크 GUID */
    put64(hdr + 72, 2);                    /* 72 : 항목 배열의 LBA */
    put32(hdr + 80, ENTRIES);              /* 80 : 항목 개수 */
    put32(hdr + 84, ENTRY_SZ);             /* 84 : 항목 하나의 크기 */
    put32(hdr + 88, crc32(ents, ENTRIES * ENTRY_SZ));   /* 88 : 항목 배열 전체의 CRC */
    put32(hdr + 16, crc32(hdr, 92));       /* 마지막에 헤더 자신의 CRC */

    printf("== the GPT header (LBA 1, 92 bytes) ==\n");
    printf("  signature        : %.8s\n", hdr);
    printf("  revision         : %u.%u\n", get32(hdr + 8) >> 16, get32(hdr + 8) & 0xffff);
    printf("  header size      : %u bytes (the sector is 512; the header uses only 92)\n", get32(hdr + 12));
    printf("  header CRC32     : 0x%08x\n", get32(hdr + 16));
    printf("  this / alternate : LBA %llu / LBA %llu\n",
           (unsigned long long)get64(hdr + 24), (unsigned long long)get64(hdr + 32));
    printf("  usable range     : LBA %llu - %llu\n",
           (unsigned long long)get64(hdr + 40), (unsigned long long)get64(hdr + 48));
    guid_text(hdr + 56, t1);
    printf("  disk GUID        : %s\n", t1);
    printf("  entry array      : from LBA %llu, %u x %u bytes = %u sectors\n",
           (unsigned long long)get64(hdr + 72), get32(hdr + 80), get32(hdr + 84),
           (unsigned)ents_sectors);
    printf("  entry array CRC32: 0x%08x\n\n", get32(hdr + 88));

    printf("== checking it ==\n");
    unsigned char probe[92]; memcpy(probe, hdr, 92); put32(probe + 16, 0);
    printf("  header CRC recomputed (that field zeroed) : 0x%08x -> %s\n",
           crc32(probe, 92), crc32(probe, 92) == get32(hdr + 16) ? "matches" : "does not match");
    printf("  entry array CRC recomputed               : 0x%08x -> %s\n\n",
           crc32(ents, ENTRIES * ENTRY_SZ),
           crc32(ents, ENTRIES * ENTRY_SZ) == get32(hdr + 88) ? "matches" : "does not match");

    printf("== entries (128 bytes each) ==\n");
    for (unsigned i = 0; i < 3; i++) {
        const unsigned char *e = ents + i * ENTRY_SZ;
        guid_text(e, t1); guid_bytes(e, t2);
        uint64_t f = get64(e + 32), l = get64(e + 40), a = get64(e + 48);
        char name[40] = { 0 };
        for (int k = 0; k < 36 && e[56 + k * 2]; k++) name[k] = (char)e[56 + k * 2];
        printf("  [%u] name \"%s\"\n", i + 1, name);
        printf("      type GUID (as text) : %s\n", t1);
        printf("      type GUID (byte order): %s\n", t2);
        printf("      LBA %llu ~ %llu  = %.1f MiB\n", (unsigned long long)f,
               (unsigned long long)l, (l - f + 1) * (double)SEC / (1024 * 1024));
        printf("      attributes 0x%016llx%s\n", (unsigned long long)a,
               a & (1ull << 63) ? "  (bit 63 = do not automount)" : "");
    }

    printf("\n== where the alternate (backup) GPT is ==\n");
    printf("  disk %u sectors = %.1f GiB\n", DISK_SECS, DISK_SECS * (double)SEC / (1 << 30));
    printf("  alternate header: the last sector, LBA %llu\n", (unsigned long long)backup_lba);
    printf("  alternate array : LBA %llu - %llu (%u sectors just before the header)\n",
           (unsigned long long)(backup_lba - ents_sectors),
           (unsigned long long)(backup_lba - 1), (unsigned)ents_sectors);
    printf("  -> damage at the front is repaired from the back, and the back from the front.\n");
    return 0;
}

Output

== the GPT header (LBA 1, 92 bytes) ==
  signature        : EFI PART
  revision         : 1.0
  header size      : 92 bytes (the sector is 512; the header uses only 92)
  header CRC32     : 0x3e0d10e0
  this / alternate : LBA 1 / LBA 4194303
  usable range     : LBA 34 - 4194270
  disk GUID        : 01234567-89AB-CDEF-0123-456789ABCDEF
  entry array      : from LBA 2, 128 x 128 bytes = 32 sectors
  entry array CRC32: 0xb43cd5f9

== checking it ==
  header CRC recomputed (that field zeroed) : 0x3e0d10e0 -> matches
  entry array CRC recomputed               : 0xb43cd5f9 -> matches

== entries (128 bytes each) ==
  [1] name "EFI System"
      type GUID (as text) : C12A7328-F81F-11D2-BA4B-00A0C93EC93B
      type GUID (byte order): 28 73 2a c1 1f f8 d2 11 ba 4b 00 a0 c9 3e c9 3b 
      LBA 2048 ~ 206847  = 100.0 MiB
      attributes 0x0000000000000000
  [2] name "root"
      type GUID (as text) : 0FC63DAF-8483-4772-8E79-3D69D8477DE4
      type GUID (byte order): af 3d c6 0f 83 84 72 47 8e 79 3d 69 d8 47 7d e4 
      LBA 206848 ~ 3358719  = 1539.0 MiB
      attributes 0x0000000000000000
  [3] name "swap"
      type GUID (as text) : 0657FD6D-A4AB-43C4-84E5-0933C84B4F4F
      type GUID (byte order): 6d fd 57 06 ab a4 c4 43 84 e5 09 33 c8 4b 4f 4f 
      LBA 3358720 ~ 4194270  = 408.0 MiB
      attributes 0x8000000000000000  (bit 63 = do not automount)

== where the alternate (backup) GPT is ==
  disk 4194304 sectors = 2.0 GiB
  alternate header: the last sector, LBA 4194303
  alternate array : LBA 4194271 - 4194302 (32 sectors just before the header)
  -> damage at the front is repaired from the back, and the back from the front.

The demonstration gives evidence for three things. The header CRC is computed with that field zeroed (otherwise its own value would enter its own hash and the computation would not stand). The entry array CRC covers count × size in full — including unused slots. And the alternate GPT is at the very end of the disk, so damage at the front can be repaired from the back.

GUIDWhatNote
C12A7328-F81F-11D2-BA4B-00A0C93EC93BEFI system partition (ESP)formatted as FAT. The firmware reads .efi from here — see the appendix on booting
21686148-6449-6E6F-744E-656564454649BIOS boot partitionwhere GRUB puts its body when a GPT disk boots from a legacy BIOS
0FC63DAF-8483-4772-8E79-3D69D8477DE4Linux filesystem
0657FD6D-A4AB-43C4-84E5-0933C84B4F4FLinux swap
E6D6D379-F507-44C2-A23C-238F2A3DF928Linux LVM
EBD0A0A2-B9E5-4433-87C0-68B6B72699C7Microsoft basic dataNTFS, exFAT and FAT all take this value
DE94BBA4-06D1-4D40-A16A-BFD50179D6ACWindows recovery environment

Table 105.13 — Type GUIDs commonly seen

Alignment — why exactly sector 2048#

Today’s tools start the first partition at LBA 2048. 2048 × 512 bytes = 1 MiB.

WhatChunk sizeIf it does not fit
512e / 4Kn disks4 KiBevery write gains a read-modify-write
an SSD’s erase unithundreds of KiB to several MiBmore erasing and rewriting, cutting life and speed
RAID stripes64 KiB to 1 MiBone write spans two devices
virtual disk blocksaround 1 MiBthe same problem again, one layer up on the host

Table 105.14 — What 1 MiB alignment lines up with

★ 1 MiB is a multiple of every chunk size above, so lining up with one lines up with all. Compare the old tools that started at sector 63 (“just past the first track” of the CHS era) — 63 is not a multiple of 4 KiB, so it was always misaligned.

What is inside a partition — a filesystem’s first sector#

A partition is only “from here, this many”, and what goes on inside is the filesystem’s business. A filesystem usually writes its own description into its first sector (or first block).

FilesystemNameWhereHow it is recognised
FAT12/16/32BPB (the boot sector)sector 0 of the partition55 AA at the end, and the FAT32 label
exFATboot sectorsector 0 of the partitionEXFAT at offset 3
NTFSboot sectorsector 0 of the partitionNTFS at offset 3
ext2/3/4the superblock1024 bytes past the start of the partition53 EF at superblock offset 0x38

Table 105.15 — Where a filesystem’s self-description lives

FAT32 — the easiest example to read#

FAT is old and simple, which makes it a good way to learn what a filesystem must record.

OffsetSizeNameWhatTypical value
03jump instructionit jumps over to the code when bootingEB 58 90
38OEM namecharacters. It means nothingMSWIN4.1
112sector sizein bytes512
131sectors per clusterthe unit in which file space is handed out8 (= 4 KiB)
142reserved sectorsspace before the FAT (the boot sector included)32
161number of FATshow many copies2
172root entry count0 on FAT32 (the root lives in clusters too)0
192total sectors (16-bit)for small volumes; zero if unused0
211media typean old traceF8 (fixed disk)
222FAT size (16-bit)0 on FAT320
284hidden sectorswhere this partition starts on the disk2048
324total sectors (32-bit)the size of this volume2097152
364FAT size (32-bit)sectors in one FAT — a computed value2046
444root clusterthe cluster the root directory starts at2
482FSInfo sectorwhere the free cluster count is cached1
502backup boot sectorwhere a copy sits in case the boot sector is damaged6
661extended signature0x29 means the next three are valid29
674volume serial numberdecided at format timearbitrary
7111volume labelcharactersNO NAME
828filesystem labelnot to be trusted — it is only charactersFAT32
5102signature55 AA55 AA

Table 105.16 — The main fields of the FAT32 boot sector (BPB)

examples-en/apx-disk/fat32_bpb.c

/* 파티션 *안*의 첫 섹터 --- FAT32 의 BPB 를 짓고 되읽어, 「클러스터 번호 → 섹터 번호」
   산수를 실제로 해 본다. 그리고 FAT 사슬을 하나 따라간다.
   주의: 디스크를 건드리지 않는다. */
#include <stdio.h>
#include <stdint.h>
#include <string.h>

#define SEC 512u

static void put16(unsigned char *p, uint16_t v) { p[0] = v & 0xff; p[1] = v >> 8; }
static void put32(unsigned char *p, uint32_t v)
{ for (int i = 0; i < 4; i++) p[i] = (unsigned char)(v >> (8 * i)); }
static uint16_t get16(const unsigned char *p) { return (uint16_t)(p[0] | p[1] << 8); }
static uint32_t get32(const unsigned char *p)
{ uint32_t v = 0; for (int i = 3; i >= 0; i--) v = v << 8 | p[i]; return v; }

int main(void)
{
    unsigned char bs[SEC] = { 0 };
    const uint32_t part_lba = 2048;        /* 이 파티션이 디스크에서 시작하는 자리 */
    const uint32_t tot_sec  = 2097152;     /* 1 GiB */
    const uint32_t spc_want = 8;           /* 클러스터 = 8섹터 = 4 KiB */
    const uint32_t rsvd_want = 32, nfat_want = 2;

    /* FAT 한 벌의 크기는 마음대로 정하는 값이 아니라 *계산해서 나오는* 값이다.
       규격이 주는 어림 공식(FAT32): 자료가 될 섹터를 「클러스터 하나가 먹는 섹터 +
       그 클러스터를 가리키는 FAT 항목이 먹는 자리」로 나눈다. */
    uint32_t t1 = tot_sec - rsvd_want;
    uint32_t t2 = (256 * spc_want + nfat_want) / 2;
    const uint32_t fat_sz = (t1 + t2 - 1) / t2;

    /* ── BPB 를 규격대로 채운다 ─────────────────────────────── */
    bs[0] = 0xeb; bs[1] = 0x58; bs[2] = 0x90;      /* 0  : 점프 명령(3바이트) */
    memcpy(bs + 3, "MSWIN4.1", 8);                 /* 3  : 만든 곳 이름(8) */
    put16(bs + 11, 512);                           /* 11 : 섹터 하나의 바이트 수 */
    bs[13] = (unsigned char)spc_want;              /* 13 : 클러스터 하나의 섹터 수 */
    put16(bs + 14, (uint16_t)rsvd_want);           /* 14 : 예약 섹터 수(FAT 앞) */
    bs[16] = (unsigned char)nfat_want;             /* 16 : FAT 벌 수(대개 2 --- 사본) */
    put16(bs + 17, 0);                             /* 17 : 루트 항목 수 --- FAT32 는 0 */
    put16(bs + 19, 0);                             /* 19 : 총 섹터(16비트) --- 안 쓰면 0 */
    bs[21] = 0xf8;                                 /* 21 : 매체 종류(고정 디스크) */
    put16(bs + 22, 0);                             /* 22 : FAT 크기(16비트) --- FAT32 는 0 */
    put16(bs + 24, 63);                            /* 24 : 트랙당 섹터(옛 CHS 흔적) */
    put16(bs + 26, 255);                           /* 26 : 머리 수(옛 CHS 흔적) */
    put32(bs + 28, part_lba);                      /* 28 : 이 파티션 앞의 숨은 섹터 수 */
    put32(bs + 32, tot_sec);                       /* 32 : 총 섹터(32비트) */
    put32(bs + 36, fat_sz);                        /* 36 : FAT 한 벌의 섹터 수(계산값) */
    put16(bs + 40, 0);                             /* 40 : FAT 미러링 표시 */
    put16(bs + 42, 0);                             /* 42 : 파일 시스템 판 번호 */
    put32(bs + 44, 2);                             /* 44 : 루트 디렉터리의 클러스터 번호 */
    put16(bs + 48, 1);                             /* 48 : FSInfo 섹터 */
    put16(bs + 50, 6);                             /* 50 : 부트 섹터 사본의 자리 */
    bs[64] = 0x80;                                 /* 64 : BIOS 드라이브 번호 */
    bs[66] = 0x29;                                 /* 66 : 확장 서명 --- 아래 셋이 있다는 표시 */
    put32(bs + 67, 0x1234ABCDu);                   /* 67 : 볼륨 일련번호 */
    memcpy(bs + 71, "NO NAME    ", 11);            /* 71 : 볼륨 이름(11) */
    memcpy(bs + 82, "FAT32   ", 8);                /* 82 : 파일 시스템 이름표(믿지 말 것) */
    bs[510] = 0x55; bs[511] = 0xaa;                /* 510: 서명 */

    /* ── 되읽는다 ───────────────────────────────────────────── */
    uint32_t bps   = get16(bs + 11);
    uint32_t spc   = bs[13];
    uint32_t rsvd  = get16(bs + 14);
    uint32_t nfat  = bs[16];
    uint32_t fatsz = get32(bs + 36);
    uint32_t tot   = get32(bs + 32);
    uint32_t root  = get32(bs + 44);

    printf("== the FAT32 BPB (first sector of the partition) ==\n");
    printf("  sector size       : %u bytes\n", bps);
    printf("  cluster size      : %u sectors = %u bytes (%u KiB)\n", spc, spc * bps, spc * bps / 1024);
    printf("  reserved sectors  : %u\n", rsvd);
    printf("  FAT copies / size : %u x %u sectors\n", nfat, fatsz);
    printf("  total sectors     : %u = %.1f MiB\n", tot, tot * (double)bps / (1024 * 1024));
    printf("  root cluster      : %u\n", root);
    printf("  volume label      : \"%.11s\", filesystem label \"%.8s\"\n",
           bs + 71, bs + 82);
    printf("  hidden sectors    : %u  <- where this partition starts on the disk\n\n",
           get32(bs + 28));

    /* ── 자리 계산 ──────────────────────────────────────────── */
    uint32_t first_data = rsvd + nfat * fatsz;         /* FAT32 는 루트 디렉터리 자리가 따로 없다 */
    uint32_t clusters   = (tot - first_data) / spc;

    printf("== how the partition divides up ==\n");
    printf("  reserved area : sectors 0 - %u  (boot sector, FSInfo, copies)\n", rsvd - 1);
    printf("  FAT copy 1    : sectors %u - %u\n", rsvd, rsvd + fatsz - 1);
    printf("  FAT copy 2    : sectors %u - %u  <- a duplicate of the same content\n",
           rsvd + fatsz, rsvd + 2 * fatsz - 1);
    printf("  data area     : from sector %u  (cluster numbers start here)\n", first_data);
    printf("  clusters      : %u --- FAT32 needs at least 65525 -> %s\n\n",
           clusters, clusters >= 65525 ? "it is FAT32" : "cannot be FAT32");

    printf("== changing the cluster size on the same volume ==\n");
    /* 한글은 한 글자가 두 칸이라 %-10s 로는 안 맞는다 --- 머리글은 손으로 맞춘다 */
    printf("  cluster   clusters       one FAT      usable as FAT32\n");
    for (uint32_t sc = 1; sc <= 64; sc *= 2) {
        uint32_t a  = tot - rsvd;
        uint32_t b  = (256 * sc + nfat) / 2;
        uint32_t fz = (a + b - 1) / b;                 /* 규격의 어림 공식 */
        uint32_t cl = (tot - (rsvd + nfat * fz)) / sc; /* 실제로 남는 클러스터 수 */
        char csz[16];
        snprintf(csz, sizeof csz, "%u KiB", sc * bps / 1024 ? sc * bps / 1024 : 0);
        if (sc * bps < 1024) snprintf(csz, sizeof csz, "%u B", sc * bps);
        char fsz[16]; snprintf(fsz, sizeof fsz, "%u sectors", fz);
        printf("  %-10s %-14u %-12s %s\n", csz, cl, fsz,
               cl >= 65525 ? "yes" : "no --- too few clusters");
    }
    printf("\n  -> this is why a small volume ends up FAT16. Larger clusters mean fewer\n");
    printf("     of them to manage, and the count falls below 65525.\n\n");

    printf("== cluster number -> sector number ==\n");
    printf("  formula: start of data area + (N - 2) x sectors per cluster\n");
    printf("        (why subtract 2: 0 and 1 are used as labels and have no real place)\n\n");
    for (uint32_t n = 2; n <= 5; n++) {
        uint32_t rel = first_data + (n - 2) * spc;
        printf("  cluster %-3u -> sector %-6u in the partition -> absolute LBA %-8u%s\n",
               n, rel, part_lba + rel, n == root ? "  <- the root directory" : "");
    }

    /* ── FAT 사슬 따라가기 ──────────────────────────────────── */
    static unsigned char fat[SEC * 4];      /* FAT 앞 네 섹터만 흉내 낸다 */
    put32(fat + 3 * 4, 4);                  /* 3번 다음은 4번 */
    put32(fat + 4 * 4, 7);                  /* 4번 다음은 7번 */
    put32(fat + 7 * 4, 0x0FFFFFFFu);        /* 7번이 마지막 */

    printf("\n== when one file is scattered --- the FAT chain ==\n");
    printf("  the FAT is an array of cluster number -> next cluster number (4 bytes each).\n");
    printf("  if a file starts at cluster 3:\n\n");
    uint32_t n = 3;
    for (int step = 0; step < 8; step++) {
        uint32_t next = get32(fat + n * 4) & 0x0FFFFFFFu;   /* 위 4비트는 예약 */
        uint32_t lba  = part_lba + first_data + (n - 2) * spc;
        printf("    cluster %-3u (LBA %-8u, %u KiB)", n, lba, spc * bps / 1024);
        if (next >= 0x0FFFFFF8u) { printf(" -> end of chain\n"); break; }
        if (next == 0x0FFFFFF7u) { printf(" -> bad cluster\n"); break; }
        printf(" -> next is %u\n", next);
        n = next;
    }
    printf("\n  so a file that is not contiguous can still be read. And damage to one FAT\n");
    printf("  removes the way to find the rest of a file --- which is why there are two copies.\n");
    return 0;
}

Output

== the FAT32 BPB (first sector of the partition) ==
  sector size       : 512 bytes
  cluster size      : 8 sectors = 4096 bytes (4 KiB)
  reserved sectors  : 32
  FAT copies / size : 2 x 2046 sectors
  total sectors     : 2097152 = 1024.0 MiB
  root cluster      : 2
  volume label      : "NO NAME    ", filesystem label "FAT32   "
  hidden sectors    : 2048  <- where this partition starts on the disk

== how the partition divides up ==
  reserved area : sectors 0 - 31  (boot sector, FSInfo, copies)
  FAT copy 1    : sectors 32 - 2077
  FAT copy 2    : sectors 2078 - 4123  <- a duplicate of the same content
  data area     : from sector 4124  (cluster numbers start here)
  clusters      : 261628 --- FAT32 needs at least 65525 -> it is FAT32

== changing the cluster size on the same volume ==
  cluster   clusters       one FAT      usable as FAT32
  512 B      2064606        16257 sectors yes
  1 KiB      1040400        8160 sectors yes
  2 KiB      522236         4088 sectors yes
  4 KiB      261628         2046 sectors yes
  8 KiB      130942         1024 sectors yes
  16 KiB     65503          512 sectors  no --- too few clusters
  32 KiB     32759          256 sectors  no --- too few clusters

  -> this is why a small volume ends up FAT16. Larger clusters mean fewer
     of them to manage, and the count falls below 65525.

== cluster number -> sector number ==
  formula: start of data area + (N - 2) x sectors per cluster
        (why subtract 2: 0 and 1 are used as labels and have no real place)

  cluster 2   -> sector 4124   in the partition -> absolute LBA 6172      <- the root directory
  cluster 3   -> sector 4132   in the partition -> absolute LBA 6180    
  cluster 4   -> sector 4140   in the partition -> absolute LBA 6188    
  cluster 5   -> sector 4148   in the partition -> absolute LBA 6196    

== when one file is scattered --- the FAT chain ==
  the FAT is an array of cluster number -> next cluster number (4 bytes each).
  if a file starts at cluster 3:

    cluster 3   (LBA 6180    , 4 KiB) -> next is 4
    cluster 4   (LBA 6188    , 4 KiB) -> next is 7
    cluster 7   (LBA 6212    , 4 KiB) -> end of chain

  so a file that is not contiguous can still be read. And damage to one FAT
  removes the way to find the rest of a file --- which is why there are two copies.

Three things to learn here.

First, the arithmetic of places is only addition. Start of the data area = reserved sectors + (number of FATs × FAT size). And the sector of cluster N = start of data area + (N − 2) × sectors per cluster. Two is subtracted because clusters 0 and 1 are used as labels and have no real place.

Second, the FAT is an array of “the next cluster number”. However scattered a file is across the disk, recording the next slot’s number in each cluster lets it be read in order. So damage to the FAT removes the way to reach the rest of a file while the data is still there — which is why there are two copies.

ValueMeaningNote
0x0000000a free clusteravailable
0x0000002 to 0xFFFFFEFthe next cluster numberthe chain continues
0xFFFFFF7a bad clusternot used
0xFFFFFF8 and aboveend of chainthe file’s last cluster

Table 105.17 — FAT32 entry values (only the low 28 bits are used)

Third, the cluster size even settles which filesystem it can be. As the demonstration’s table shows, on the same 1 GiB volume, raising the cluster to 16 KiB drops the cluster count below 65,525 and it cannot be FAT32. That is where a small USB stick formatted as FAT16 comes from.

How ext4 and NTFS differ#

OffsetSizeNameWhatTypical value
0x004total inodesone inode per file — fixed at format timecomputed
0x044total blocksthe volume size in blockscomputed
0x184block size exponentblock size = 1024 << this value2 (= 4 KiB)
0x204blocks per groupthe size of a block group32768
0x284inodes per grouplikewisecomputed
0x382magic53 EF — how ext is recognised53 EF
0x3A2statewhether it was unmounted cleanly1
0x582inode sizeusually 256 bytes256
0x604feature flags (compatible)the journal and so on—
0x644feature flags (incompatible)features that forbid mounting if unknown—
0x6816UUIDthe volume’s unique number — what /etc/fstab uses16 bytes
0x7816volume namecharacters—

Table 105.18 — The main fields of the ext4 superblock (start of partition + 1024 bytes)

Where FAT is “an array of next-slot numbers”, ext4 centres on the inode. One file holds one inode, in which its size, permissions, times and where its data is are recorded. And places are recorded not one number at a time but as extents (contiguous runs), which expresses a large file briefly.

OffsetSizeNameWhatTypical value
38labelrecognised by NTFSNTFS
112sector size512
131sectors per cluster8
284hidden sectorswhere the partition starts2048
408total sectors64-bit — hence large volumescomputed
488cluster of $MFTwhere the list of everything beginscomputed
568cluster of $MFTMirra copy of that listcomputed
641MFT record sizenegative means 2|value| bytes (usually 1024)F6
5102signature55 AA

Table 105.19 — The main fields of the NTFS boot sector

In NTFS everything is a file — even the list of files ($MFT) is a file, and the boot sector is a file called $Boot. Record only where that list is in the boot sector, and everything else follows the list.

NameSpace managementJournalFile size limitMostly where
FAT32FAT chainsnoneunder 4 GiBUSB, SD, ESP — readable everywhere
exFATFAT chains (64-bit)noneeffectively nonelarge SD cards, cameras
NTFSMFT + extentsmetadatavery largeWindows
ext4inodes + extentsyes (optional)around 16 TiBLinux
btrfs, ZFS, APFScopy-on-write + checksumsthe structure itself replaces itvery largewhere snapshots and integrity are needed

Table 105.20 — Filesystems commonly met

★ The “journal” column is what divides things most in practice. The FAT family, with no journal, can be left with the filesystem itself inconsistent if power is lost during a write. That is why “safely remove” exists before pulling out a USB stick.

The physics of the medium — why random writes are slow#

So far we read the layout. Now the thing underneath, where the bytes actually live. The same word “storage” covers a spinning platter and a semiconductor, and their characters differ enormously — a difference that settles the design of filesystems and databases.

The spinning platter — the time to reach a place#

One read on a hard disk divides into three.

WhatWhat is waited forTypical valueCan it be reduced
seekthe head moving to that track5–10 msreading nearby places reduces it
rotational latencythe platter turning until that sector arrivesat 7200 rpm, 4.2 ms on averagehard to reduce
transferactually moving the bytestens of microsecondsvery small if sequential

Table 105.21 — The time for one read on a hard disk

★ The first two are the cost of finding a place and the third is the cost of the work, and the first two are about a hundred times the third. That is where “random access is slow” comes from — reading 4 KiB or 1 MiB, the cost of finding the place once is the same.

Q. Why is the average rotational latency 4.2 ms at 7200 rpm?

A. 7200 per minute is 607200=8.33 ms per turn. On average the wanted sector arrives half a turn later, so half of that: 4.17 ms. The rotation speed alone settles the latency — software can do nothing about it.

The semiconductor — writing and erasing in different units#

An SSD has no moving parts, so there is no seek. In its place another property appears: it cannot be overwritten where it stands.

ssd-erase

Figure 105.3 — An SSD’s pages and blocks — where write amplification comes from.

UnitSize (roughly)Used forConstraint
cell1 to 4 bitsthe smallest thing holding a valueit wears a little with every write
page4 to 16 KiBthe unit of reading and writingonly a free page can be written
blockseveral MiB (hundreds of pages)the unit of erasingerasing means the whole block

Table 105.22 — Three units inside an SSD

Every property of an SSD follows from this.

WhatWhat it isWhat the programmer feels
the FTL (mapping layer)the device keeps a “logical address → real page” table itselfwriting the same address lands somewhere different each time
write amplificationchanging 4 KiB rewrites and erases a blocksmall random writes are unusually expensive
garbage collectiontidying blocks full of dead pages into free oneswrites suddenly slow down now and then
TRIMthe filesystem saying “this page is no longer used”after deleting, speed returns
wear levellingrotating use so the same block does not wear out“with no free space the life shortens”
over-provisioningspare capacity invisible to the userthe real chips are larger than the advertised size

Table 105.23 — What “it cannot be overwritten” produces

A common misconception. on an SSD, random access is as fast as sequential

For reading, nearly so — there is no seek. For writing it is different. Small random writes invite write amplification and garbage collection, and latency spikes then. Which is why databases and filesystems still work hard to gather writes and put them down sequentially even on an SSD (log-structured designs).

What was measured on this machine#

examples-en/apx-disk/storage_cost.c

/* 저장 장치는 얼마나 느린가 --- 그리고 「느리다」의 얼굴이 몇 가지인가.
   임시 파일 하나를 만들어 재고, 끝나면 지운다. 장치를 직접 만지지 않는다. */
#define _POSIX_C_SOURCE 200809L
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <string.h>
#include <time.h>
#include <fcntl.h>
#include <unistd.h>

static double ns(void)
{
    struct timespec ts;
    clock_gettime(CLOCK_MONOTONIC, &ts);
    return (double)ts.tv_sec * 1e9 + (double)ts.tv_nsec;
}
static int cmp_d(const void *a, const void *b)
{ double x = *(const double *)a, y = *(const double *)b; return x < y ? -1 : x > y; }

static uint64_t st = 0xDEADBEEFCAFEBABEull;
static uint64_t rnd(void) { st ^= st << 13; st ^= st >> 7; st ^= st << 17; return st; }

#define MiB (1024u * 1024u)
#define FILE_MiB 64u
#define BLK 4096u

int main(void)
{
    const char *path = "./_storage_probe.tmp";
    unsigned char *buf = malloc(MiB);
    memset(buf, 0xA5, MiB);

    printf("== what is being measured ==\n");
    printf("  measured with one %u MiB temporary file. This is not the figure of one device\n", FILE_MiB);
    printf("  but of the filesystem, the cache and the device together --- the whole stack.\n\n");

    /* ── ① 차례로 쓰기 ── */
    int fd = open(path, O_CREAT | O_TRUNC | O_WRONLY, 0600);
    if (fd < 0) { perror("open"); return 1; }
    double t0 = ns();
    for (unsigned i = 0; i < FILE_MiB; i++)
        if (write(fd, buf, MiB) != (ssize_t)MiB) { perror("write"); return 1; }
    double t1 = ns();
    double write_mbs = FILE_MiB / ((t1 - t0) / 1e9);
    printf("== 1. sequential writing ==\n");
    printf("  writing %u MiB (into cache) : %8.1f MiB/s\n", FILE_MiB, write_mbs);

    /* ── ② 정말 디스크에 닿게 하기 --- fsync ── */
    t0 = ns(); fsync(fd); t1 = ns();
    double fsync_all_ms = (t1 - t0) / 1e6;
    printf("  one fsync after it          : %8.1f ms  <- this much had not reached the disk\n",
           fsync_all_ms);
    close(fd);

    /* 작은 쓰기 + fsync 를 되풀이 --- 데이터베이스가 하는 일 */
    fd = open(path, O_WRONLY);
    double s[9];
    for (int r = 0; r < 9; r++) {
        t0 = ns();
        for (int i = 0; i < 20; i++) {
            if (write(fd, buf, BLK) != (ssize_t)BLK) { perror("write"); return 1; }
            fsync(fd);
        }
        t1 = ns();
        s[r] = (t1 - t0) / 20.0 / 1e6;         /* fsync 한 번당 밀리초 */
    }
    qsort(s, 9, sizeof *s, cmp_d);
    double fsync_ms = s[4];
    close(fd);
    printf("  4 KiB write + fsync         : %8.3f ms each -> about %.0f per second\n",
           fsync_ms, 1000.0 / fsync_ms);
    printf("  * that is the cost of one database commit. Which is why designs gather several\n");
    printf("    commits and flush them together (group commit).\n");

    /* ── ③ 캐시가 있을 때와 없을 때 ── */
    printf("\n== 2. reading --- with the cache, and after asking to drop it ==\n");
    fd = open(path, O_RDONLY);
    double seq_cached = 0, seq_cold = 0, rnd_cached = 0, rnd_cold = 0;

    for (int cold = 0; cold < 2; cold++) {
        if (cold) posix_fadvise(fd, 0, 0, POSIX_FADV_DONTNEED);   /* 이 파일의 캐시만 버린다 */
        lseek(fd, 0, SEEK_SET);
        t0 = ns();
        for (unsigned i = 0; i < FILE_MiB; i++)
            if (read(fd, buf, MiB) != (ssize_t)MiB) { perror("read"); return 1; }
        t1 = ns();
        double mbs = FILE_MiB / ((t1 - t0) / 1e9);
        if (cold) seq_cold = mbs; else seq_cached = mbs;
    }
    printf("  sequential read (cached)  : %8.1f MiB/s\n", seq_cached);
    printf("  sequential read (dropped) : %8.1f MiB/s  -> a factor of %.1f\n",
           seq_cold, seq_cached / seq_cold);

    /* 무작위 4 KiB 읽기 */
    for (int cold = 0; cold < 2; cold++) {
        if (cold) posix_fadvise(fd, 0, 0, POSIX_FADV_DONTNEED);
        const int NREAD = 2000;
        t0 = ns();
        for (int i = 0; i < NREAD; i++) {
            off_t off = (off_t)((rnd() % (FILE_MiB * MiB / BLK)) * BLK);
            if (pread(fd, buf, BLK, off) != (ssize_t)BLK) { perror("pread"); return 1; }
        }
        t1 = ns();
        double us = (t1 - t0) / NREAD / 1000.0;
        if (cold) rnd_cold = us; else rnd_cached = us;
    }
    printf("  random 4 KiB (cached)     : %8.1f us each -> %.0f per second\n",
           rnd_cached, 1e6 / rnd_cached);
    printf("  random 4 KiB (dropped)    : %8.1f us each -> %.0f per second\n",
           rnd_cold, 1e6 / rnd_cold);

    /* ★ 부탁이 먹혔는지 확인한다. 두 값이 비슷하면 캐시가 그대로 남아 있었던 것이다. */
    double ratio = seq_cached / seq_cold;
    if (ratio < 1.2 && ratio > 0.8) {
        printf("\n  * the two are nearly equal --- the request to drop the cache was not honoured.\n");
        printf("    This filesystem keeps its own cache, which `posix_fadvise` does not\n");
        printf("    empty. So read the numbers above not as the speed of the device but as\n");
        printf("    the speed of reading a file that is already in memory.\n");
        printf("    (%.0f MiB/s is the speed of memory, not of storage.)\n", seq_cold);
        printf("    Really measuring the device needs bypassing the cache (O_DIRECT) or the\n");
        printf("    privilege to drop the system cache, and this environment has neither ---\n");
        printf("    so what could not be measured is left unmeasured.\n");
    }
    close(fd);
    unlink(path);

    /* ── ④ 사다리에 얹어 보기 ── */
    printf("\n== 3. placed on the ladder from earlier ==\n");
    printf("  %-28s %14s %s\n", "what", "each", "against L1 (1.4 ns)");
    struct { const char *name; double v_ns; } rows[] = {
        { "L1 cache read",           1.4 },
        { "main memory read",            75.0 },
        { "random 4 KiB (from cache)", rnd_cached * 1000.0 },
        { "4 KiB write + fsync",     fsync_ms * 1e6 },
    };
    for (unsigned i = 0; i < sizeof rows / sizeof *rows; i++) {
        char v[32];
        if (rows[i].v_ns < 1e3)      snprintf(v, sizeof v, "%.1f ns", rows[i].v_ns);
        else if (rows[i].v_ns < 1e6) snprintf(v, sizeof v, "%.1f us", rows[i].v_ns / 1e3);
        else                         snprintf(v, sizeof v, "%.2f ms", rows[i].v_ns / 1e6);
        printf("  %-28s %14s %12.0f x\n", rows[i].name, v, rows[i].v_ns / 1.4);
    }
    printf("\n  * the ladder does not end at the cache. Several more orders of magnitude lie below.\n");
    printf("    And on those lower rungs, reading in order matters far more.\n");

    free(buf);
    return 0;
}

Output

== what is being measured ==
  measured with one 64 MiB temporary file. This is not the figure of one device
  but of the filesystem, the cache and the device together --- the whole stack.

== 1. sequential writing ==
  writing 64 MiB (into cache) :   6362.3 MiB/s
  one fsync after it          :     49.2 ms  <- this much had not reached the disk
  4 KiB write + fsync         :    0.261 ms each -> about 3826 per second
  * that is the cost of one database commit. Which is why designs gather several
    commits and flush them together (group commit).

== 2. reading --- with the cache, and after asking to drop it ==
  sequential read (cached)  :  10303.7 MiB/s
  sequential read (dropped) :  10015.2 MiB/s  -> a factor of 1.0
  random 4 KiB (cached)     :      2.9 us each -> 349056 per second
  random 4 KiB (dropped)    :      2.7 us each -> 365889 per second

  * the two are nearly equal --- the request to drop the cache was not honoured.
    This filesystem keeps its own cache, which `posix_fadvise` does not
    empty. So read the numbers above not as the speed of the device but as
    the speed of reading a file that is already in memory.
    (10015 MiB/s is the speed of memory, not of storage.)
    Really measuring the device needs bypassing the cache (O_DIRECT) or the
    privilege to drop the system cache, and this environment has neither ---
    so what could not be measured is left unmeasured.

== 3. placed on the ladder from earlier ==
  what                                   each against L1 (1.4 ns)
  L1 cache read                        1.4 ns            1 x
  main memory read                    75.0 ns           54 x
  random 4 KiB (from cache)            2.9 us         2046 x
  4 KiB write + fsync                261.4 us       186693 x

  * the ladder does not end at the cache. Several more orders of magnitude lie below.
    And on those lower rungs, reading in order matters far more.

★ The most important thing in this measurement’s conclusion is not what was measured but what was not. The request to drop the cache (posix_fadvise) is not honoured by this filesystem, so “sequential read” came out at 11 GiB/s — which is the speed of memory, not of storage. The demonstration checks that itself and says so.

In practice. meaning to measure the device and measuring the cache

This is the commonest mistake in storage measurement. The filesystem cache sits in between, so measuring without preparation gives the speed of memory. Measuring properly means bypassing the cache (O_DIRECT) or emptying the whole system’s cache, and both need privilege.

So this appendix leaves it as “the device could not be isolated in this environment”. What could be isolated is clear enough — the cost of fsync, because that is a cost no cache can hide.

fsync — the cost no cache can hide#

A write usually reaches only the cache. Lose power and it is gone. To be promised “it really is stored” you must call fsync, and that cost is of another magnitude.

WhatEachAgainst L1What spends the time
L1 cache readabout 1.4 ns1×—
main memory readabout 75 nsabout 54×the distance to memory
reading 4 KiB from a file (cached)about 2.3 µsabout 1,600×the system call and the copy
4 KiB write + fsyncabout 0.3 msabout 200,000×it really goes down to the device

Table 105.24 — The lower rungs of the ladder (measured on this machine)

★ The last row explains database design entirely. At 0.3 ms per commit, three thousand per second is the ceiling. So databases gather several commits and flush them together (group commit), and write first to a log that is only ever appended sequentially (WAL). The PostgreSQL story in chapter 85 looks like this from this layer.

SituationWhat to doWhy
small writes, very frequentgather and write oncethe cost of one fsync barely depends on size
you must promise “it is saved”call fsync and budget for its costwritten to the cache is not yet saved
many random readsfine on an SSD; rearrange the layout on a platterthe seek cost differs a hundredfold
keeping a logan append-only filesequential writing is the cheapest on any medium
measuring the devicebypass or empty the cache firstotherwise you measure memory

Table 105.25 — Design guidance that comes from this layer

What breaks where#

The symptom alone can tell which layer was damaged. That is this appendix’s practical use.

SymptomLayer damagedThe dataWhat to look at first
it will not boot, but every file appears when plugged into another machinethe bootstrap code (446 bytes)intactreinstall the bootloader
the disk is seen, but it asks “initialise this disk?”the partition tableusually intactwrite nothing. The alternate GPT, or a partition recovery tool
the partition is seen, but it asks “format this?”the filesystem’s first sectorusually intactthe backup boot sector (sector 6 for FAT), or a superblock copy
the file list appears but the contents are corruptthe FAT, MFT or similar structurepartlya checking tool (fsck, chkdsk) from a copy
only certain files cannot be readsectors in the data areaonly those filesthe health of the disk itself (SMART)

Table 105.26 — Symptom → which layer was damaged

In practice. saved by not clicking "initialise"

A disk whose partition table alone is damaged still holds all of its data. Find out only where the partitions began and everything can be recovered, and on GPT the alternate header at the end of the disk holds that answer. The most dangerous action at that moment is accepting the offer to “fix it” and writing a new partition table — which can overwrite the alternate too. Imaging the whole thing before touching anything is the order to work in.

What to take from this#

Recap

  • All a disk knows is numbered sectors. Partitions and filesystems are conventions above them.
  • The MBR put boot code and layout into one 512-byte sector — hence 446 bytes of code, four partitions and the 2 TiB limit.
  • The extended partition invented to pass four is a linked list, and its two entries have different bases, which confuses people.
  • GPT rebuilt the same job out of an array, a CRC and a copy. And it left a protective MBR in sector 0 to hold off old tools.
  • A GUID’s byte order when stored differs from how it is written.
  • Inside a partition is the filesystem’s world. FAT is an array of next-slot numbers, ext4 centres on inodes, and in NTFS everything is a file.
  • Because the layers are distinct, the symptom tells which layer was damaged. And when the damaged layer is the layout, the data is usually still there — at which point the most dangerous thing is a hasty write.