One tar Command, Eight Extractions: The 512 Bytes POSIX Left Unspecified

How many files should come out of an archive that was built from three? Every tool you have ever used will answer three, and for any normal archive they would be right. On November 7, Harmen Stoppels published a ten-kilobyte tar file, built with a single GNU tar invocation and no hand-assembled bytes, for which the honest answer depends entirely on which program you ask. GNU tar returns three files. The tar that ships with macOS returns two, one of them silently corrupted. 7-Zip returns two and never mentions the third. R returns three, one padded with 512 zero bytes it pulled from the archive's end marker. .NET, Julia, and uv's new tar-codec refuse to return anything at all. Eight distinct outcomes from one input, with no implementation bug in the usual sense: just a gap in the POSIX text that two careful reader-writers can close in different directions. This article reconstructs the archive, reruns the experiment on a real Windows machine with GNU tar 1.35, bsdtar 3.8.4, and Python 3.14.9, reads the two sentences of the standard that cause the split, and works out what a practicing engineer should change.

The one-line construction

The archive is built from three ordinary files with three commands:

printf hello > file
touch empty
tar --format=pax --pax-option='delete=?time,size:=512' \
    --transform='s,^\./file,café,' \
    -cf hardest.tar file ./file empty

Each flag does one piece of work. The operand list names file twice, once plainly and once as ./file. GNU tar notices both paths resolve to the same inode and stores the second entry as a hard link, typeflag 1, pointing back at file. The --transform expression renames that second entry to café, and the non-ASCII name forces tar to write a pax extended header, the x block, because a plain ustar header cannot carry the UTF-8 name. Finally, --pax-option edits the extended records: delete=?time strips the atime, mtime, and ctime records that pax would otherwise emit, and size:=512 overrides the size the link would normally declare, which is zero.

The result is an archive that every tool agrees is syntactically valid. Nothing in it is truncated, misaligned, or checksummed wrong. The disagreement is entirely about what the bytes mean.

Why 512 is the loaded number

Tar is a blocked format: every header occupies exactly one 512-byte block, and every file payload is padded to a block boundary. A hard link normally declares size zero and carries no payload at all, because its content is defined to be whatever the link target holds. Setting the link's size to 512 tells a reader that exactly one payload block follows the link header. In the actual byte stream, the block that follows is the header of empty, the third member. The same 512 bytes are therefore the link's payload under one reading and a member header under the other, and both readings parse cleanly.

The size is chosen with precision. A smaller value would leave the parser resynchronized in the middle of the next header and produce an obvious error. A larger value would swallow more structure than one header. The value 512 is exactly one block, so the two interpretations diverge without either one tripping a format error. On the machine used for this article, the constructed archive is 10,240 bytes, and the size=512 record sits at offset 0x611 inside the pax data block, exactly where the construction puts it.

Two sentences of the standard, pulling apart

The POSIX.1-2024 specification for pax contains both halves of the contradiction. In the extended description of the pax format, the standard says that data blocks for files of typeflag 1, the hard link, may be included. In the description of the older ustar format, it says the opposite for its own world: no data logical records are stored for types 1, 2, or 5. The archive announces itself as pax through its x extended header, so the permissive sentence governs the bytes. Yet even under pax, the standard never says what such a payload means. Does it replace the contents of the link target? Attach new contents to the link itself? Is it simply to be ignored? The test-case page for this archive, F35 in Stoppels' tar-minefield suite, states the gap plainly: pax permits the data and does not say what the data does.

Every implementor has had to pick a branch, and the branches were picked independently, in different languages, over two decades. That is how one sentence left unspecified turns into eight observable behaviors.

Three tools, three answers, one Windows box

The experiment is cheap to repeat, and this article did repeat it. On a Windows 11 machine, the archive was built with the GNU tar 1.35 that ships with the Git for Windows toolchain, then extracted with that same GNU tar, with the bsdtar 3.8.4 that Windows installs as the system tar, and with the tarfile module of Python 3.14.9:

mkdir out_gnu out_bsd out_py
tar --force-local -xf hardest.tar -C out_gnu
"C:/Windows/System32/tar.exe" -xf hardest.tar -C out_bsd

GNU tar's extraction is the polite outcome: three files on disk, file holding the five bytes of hello, café as a real hard link to it, and empty at zero bytes. The oddity is in the listing. Asking the same binary for a table of contents with tar -tf shows only two members, file and café, with the link displayed at size 512. One tool, two answers: the listing path treats the payload as belonging to the link and skips the third member, while the extraction path ignores the size and restores all three.

bsdtar lands on a different branch entirely. It honors the declared size, reads the next block as the link's payload, and writes those 512 bytes into café. Because café is a hard link to file, both names now read the same captured block, which is the header of empty, beginning with the name empty followed by zeros. The member empty itself never appears. Two files come out, and the file you archived with five bytes now reads back 512 bytes of someone else's header.

Python shows the third personality:

import tarfile
with tarfile.open("hardest.tar") as t:
    for m in t.getmembers():
        print(m.name, m.type, m.size, m.linkname)
    t.extractall("out_py", filter="data")

The metadata pass reports all three members and honestly shows the link's declared size of 512, yet the extraction ignores that size and materializes file, café, and empty exactly as archived. One machine, three tools, three different trees, all from exit-code-zero runs.

The full scoreboard

The reproduction above covers three of the eight behaviors. The original post surveyed the current releases of the major implementations and sorted the rest. GNU tar, BusyBox, Python's tarfile, Go's archive/tar, and Perl's Archive::Tar form the group that ignores the size and extracts all three files. bsdtar writes the payload into the link, as seen here. 7-Zip, node-tar, tar-fs, and both Rust implementations, tar and astral-tokio-tar, agree the 512 bytes are the link's payload but skip over them instead of writing, so empty silently vanishes. Java's Commons Compress keeps file intact and turns café into a separate regular file containing the captured header. PHP's PharData makes café a separate empty file and also extracts empty. RubyGems declines to extract café at all and leaves a stray PaxHeaders directory behind. R's internal untar applies the size to the wrong member and delivers empty padded with 512 zero bytes read from the end-of-archive marker. Haskell's tar matches GNU tar's tree but decodes the name incorrectly, writing café as mojibake on disk. And .NET, Julia's Tar.jl, and uv's tar-codec reject the archive outright.

The parsers that refuse, and when refusal is right

The rejecting implementations fail with precise errors. Tar-codec reports that a member of type HardLink cannot carry payload size 512, and Julia's parser reports a hard link with non-zero size. Strictly speaking, refusal is not what POSIX asks for, since the pax text permits the payload. Stoppels argues it is still the best available answer, and the discussion around the post explains why: tar serves two masters with opposite threat models. Backup and recovery want best-effort extraction of whatever can be salvaged. Software distribution, where the archive is untrusted input from a registry or a user upload, wants fail-closed parsing, because a format disagreement between the tool that scans an archive and the tool that extracts it is exactly where supply-chain surprises live. One engineer building SBOM tooling noted in the thread that automated extractors now have to decide which tree is the real one before they can inventory it. Uv's tar-codec, written to install packages, sits at the strict end; libarchive, written to open anything, sits at the permissive end. Neither is wrong. They implement different contracts under the same three-letter name.

A minefield with a map

This archive is not a one-off party trick. It is case F35, pax-size-on-hardlink, in tar-minefield, a public test suite Stoppels has been building since late October that now holds 134 ambiguous archives, each with an annotated block map, a generator script, an expected tree, and a per-tool results table. The suite also measures how exotic each construct is in the wild: across 82,586 distinct release tarballs scanned from Spack, nixpkgs, Guix, PyPI, crates.io, RubyGems, and live-bootstrap, exactly zero carry a pax size on a hard link. The construct is a lab instrument, not an exploit circulating in release tarballs. Its job is to surface parser disagreement before a real archive does it for you.

What this changes for your pipeline

The practical consequences are concrete. If your system extracts archives it did not create, such as CI artifacts, user uploads, or registry packages, choose a parser that rejects ambiguity rather than one that guesses politely, and record the extractor's name and version next to the artifact so the tree is reproducible. If you build recovery or forensics tooling, permissive parsing with loud warnings is the right contract instead. Never use a second tool's output as the correctness check for a first tool without expecting disagreement; bsdtar exited successfully while delivering a corrupted file, so compare extracted trees by content hash rather than by exit code. Python users get a reasonable default already, since tarfile's data filter handles links conservatively, but the metadata pass exposes the declared size on the link, which makes this family of ambiguity detectable with a three-line check.

The 512 bytes inside hardest.tar are not a bug, a vulnerability, or a malformed file. They are a precise measurement of how much room a written standard leaves for interpretation, and eight mainstream tools filled that room with eight different directory trees. The next time two machines disagree about what an archive contained, the first suspect is neither the archive nor the tools, but the sentence nobody finished writing.

Originally published on Dispatch.