31 Compound File Binary (CFB)

An .msi file is a Compound File Binary file ("structured storage"): a small file system of named streams inside one file. The format is specified in [MS-CFB]; this page is the part an MSI writer needs, plus what Windows' own writer does.

31.1 Sectors#

• The file is a sequence of fixed-size sectors. Version 3 uses 512-byte sectors (sector shift 9), version 4 uses 4096-byte sectors (shift 12). [spec]
• The header is at offset 0. Sector number N starts at file offset (N + 1) * sector_size in both versions; in version 4 the header therefore occupies a whole 4096-byte sector whose bytes after the header are zero. [spec]
• msi.dll writes version 4 for every new database, whatever its size (20 KB to 10 MB were tried), with minor version 0x003E. Either version is accepted when installing; rubrapack writes version 4 by default and can write version 3. [observed]

Special sector numbers in chains and tables: [spec]

31.2 Header (first 512 bytes) [spec]#

31.3 FAT, DIFAT, mini FAT [spec]#

• The FAT is an array of u32, one entry per sector, giving the next sector of the chain the sector belongs to. FAT sectors themselves are marked FFFFFFFD.
• The FAT's own sector numbers are listed in the DIFAT: the first 109 in the header, the rest in DIFAT sectors, each holding sector_size / 4 - 1 numbers and, in its last u32, the next DIFAT sector (FFFFFFFE at the end). DIFAT sectors are marked FFFFFFFC in the FAT.
• With 4096-byte sectors, 109 FAT sectors already cover about 457 MB, so real MSI files in version 4 have no DIFAT sectors. With 512-byte sectors DIFAT starts at about 7 MB.
• Streams smaller than the cutoff (4096 bytes) live in the mini stream, in 64-byte mini sectors chained through the mini FAT (same layout as the FAT). The mini stream is itself stored as the root directory entry's own regular stream.

The FAT must cover its own sectors and the DIFAT's; a writer computes the counts by iterating until they stop changing.

31.4 Directory [spec]#

The directory is a chain of sectors holding 128-byte entries. Entry 0 is the root.

• The root entry is named Root Entry; its starting sector and size describe the mini stream.
• The root entry of an MSI carries the CLSID {000C1084-0000-0000-C000-000000000046}, stored as bytes 84 10 0C 00 00 00 00 00 C0 00 00 00 00 00 00 46. [spec] [observed]
• Unused entries are all zero except the three link fields, which are FFFFFFFF. [spec]
• The children of a storage form a red-black tree ordered first by name length, then by comparing the upper-cased UTF-16 names. [MS-CFB] allows the simplest valid tree: every node black, the tree a plain balanced binary search tree. rubrapack builds that - a balanced tree over the sorted names, all nodes black - and msi.dll opens it. [spec] [observed]
• Times may be zero. Writing zero times (and no other clock input) is what makes a CFB writer deterministic. [spec]

31.5 A simple, deterministic writer#

rubrapack lays the file out in this order, all runs contiguous:

header | FAT sectors | DIFAT sectors | directory | mini FAT | mini stream | large streams in order

1. Sort stream sizes into "mini" (< 4096) and "regular".
2. Mini streams are packed back to back in 64-byte units; the mini stream length is the sum.
3. Count directory, mini FAT and mini stream sectors, then FAT and DIFAT sectors (iterate).
4. Fill FAT chains for each contiguous run, write the directory tree, mini FAT, FAT, DIFAT, header.

31.6 Reading safely#

A reader must refuse, rather than follow: chains that loop (a chain never reaching FFFFFFFE within the number of sectors in the file), sector numbers beyond the file, a chain shorter than the stream size needs, directory links out of range, sibling trees that visit a node twice, and counts beyond sane limits. None of this costs much, and a crafted .msi otherwise hangs or crashes the reader.

31.7 Worked example: the tutorial's hello.msi#

The first package of the tutorial (chapter 2), built with --reproducible, is 139264 bytes: the header sector and 33 sectors of 4096 bytes. Its header:

The sectors, and the FAT entry of each (the next sector of its chain):

The root directory entry, the first 128 bytes of sector 1:

The root's size is the mini stream's: 8704 bytes, holding 21 of the 23 streams in 64-byte mini sectors; 2 streams are large enough for regular sectors. The MSI database inside the compound file continues with the streams' names and contents.