Sparkles

Appearance

Polyglot craft — Corkami and PoC||GTFO (format superposition as a discipline) ​

The deliberate-polyglot tradition: a body of proofs-of-concept, a taxonomy, and finally a generator (mitra) that together answer the question which formats compose, and why — by building files that are simultaneously a PDF, a ZIP, a TAR, an ISO, an iNES ROM, and a boot sector, and then writing down the structural rule that made each composition possible.

FieldValue
KindCraft tradition + PoC corpus + a polyglot generator (mitra) + a document series (PoC||GTFO)
LanguagePython (mitra, mocky), NASM/x86, LaTeX (pdflatex), and the file formats themselves
LicenseMIT (corkami/mitra, corkami/pocs mini corpus); PoC||GTFO issues released as free PDFs
Repositorycorkami/mitra · corkami/pocs · corkami/docs · angea/pocorgtfo
Documentation"Abusing file formats" (= PoC||GTFO 7:6) · mitra/README.md · PDF tricks
First releaseCorkami PoCs from c. 2009; PoC||GTFO 0x00 on 2013-08-05; mitra 0.3 dated 2023-03-25 in mitra.py
Axis profileMultiplicity 3 / Reflexivity 1 / Closure 1 / Mutability 0
Index anchoringMixed by construction — a footer-anchored host (ZIP EOCD, PDF startxref) wrapping header-anchored and offset-anchored guests
Dispatch ownerConsumer sniffing (libmagic / TrID / the renderer's own recovery heuristics); occasionally the shell, via a renamed extension

Latest revision surveyed: corkami/mitra at 95e1d2a7, corkami/pocs at 6d277c83, corkami/docs at fd339bf6, angea/pocorgtfo at 933c020f (2024-02-11). Platform: format-level, so none — the artifacts are byte streams; the verification below was run on Linux with file 5.x, Info-ZIP unzip, GNU tar and Python 3.

Overview ​

What it solves ​

This subject is the catalog's control group for thesis 5 — that portability has migrated from the format to the access layer. Polyglot craft is the old strategy taken to its limit: reach is bought by satisfying every parser's grammar at once, in one immutable byte stream, with no substrate, no loader cooperation, and no runtime. redbean/APE is the engineering-grade descendant of exactly this technique; everything APE does to a PE/ELF/Mach-O/ZIP/shell-script header, Corkami had already done to a PDF/JPEG/ZIP/TAR/ISO header, first as art and then as a security argument.

Concretely, the tradition solves three separate problems that the rest of this catalog keeps colliding with:

  1. A taxonomy of composability. Which pairs of formats can share a byte stream, and — the useful part — from what structural property of each format does that follow? mitra is that taxonomy compiled into code: 48 format parsers, each declaring a handful of tolerance flags and offsets, plus a driver that mechanically tries four layouts on every ordered pair. Its README.md publishes the resulting matrix: 288 format combinations over the surveyed set. This is the open question in cluster A of the source outline, already answered for a concrete corpus.
  2. A demonstration argument. A bad format design is not a vulnerability, so it is never fixed. Albertini's response is to make bad design cheap to exploit and therefore visible.
  3. A publication vehicle. PoC||GTFO ships each issue as one file that is at once a readable journal, an archive of the code its articles describe, and a running joke about the file's own type — which is the autological move this catalog is named for.

Design philosophy ​

The opening of "Abusing file formats" states the position that the whole tradition rests on:

"First, you must realize that a file has no intrinsic meaning. The meaning of a file - its type, its validity, its contents - can be different for each parser or interpreter."

And immediately after, the vocabulary the rest of this page uses:

"A polyglot is a file that has different types simultaneously, which may bypass filters and avoid security counter-measures. A multiple-personality (let's call them 'multi') file is one that is interpreted differently depending on the parser. These files may look innocent (or corrupted) to one interpreter, malicious to another. A chimera is a polyglot where the same data is interpreted as different types, which is a more advanced kind of filter bypass."

Three terms, three different claims, and it is worth keeping them apart because the catalog's axes score them differently:

TermClaimAxis consequence
PolyglotOne stream, n types, all valid simultaneouslyMultiplicity n; parsers agree that the file is also their type
Multi (formerly schizophrenic)One stream, one nominal type, different content per parserMultiplicity 1, but a parser differential — the interesting failure mode
ChimeraPolyglot where the same bytes are the payload of several typesMultiplicity n with no duplication — the dedup story of cluster E
Mock (mocky.py)One valid type, plus foreign magic planted where sniffers lookMultiplicity 1 claimed as n — attacks the dispatcher, not the parsers

The schizophrenic label is Albertini's 2014 talk title (slides, slides/1406-SchizophrenicFiles.pdf in corkami/docs); the current corpus consistently renames it to multi, and this page follows the current usage while noting the older term because most citations of the work still use it.

The engineering advice the tradition distils out of ten years of PoCs is stated as a short list of format-design rules in mitra/README.md:

"Enforcing a magic at offset zero should be standard. Starting at offset zero and not enforcing a magic at zero is still exploitable (PS, MP4). Starting at any offset makes polyglots trivial. Enforcing a footer (like XZ, ID3v1) is a great way to check if a file isn't truncated, and prevents 'naturally' appended data. Most formats have a way to store parasite data, except very simple ones."

Those four sentences are the generalizable rule set this page unpacks. They also predict, correctly, which two formats in mitra's whole table combine with nothing.

How it works ​

The four layouts ​

mitra enumerates exactly four ways two files can share a stream. The names are the tool's own, and its output filenames encode which was used (mitra.py):

LayoutFilename tagShapeRequires
StackS(off)-A-BA then B appendedA.bAppData and B.start_o > len(A)
CavityC(off)-A_BA written into B's leading dead spaceA.bAppData and B.precav_s >= len(A)
ParasiteP(off)-A[B]B wrapped in a comment/extension chunk of AA.bParasite, A.parasite_o <= B.start_o + B.precav_s, A.parasite_s >= len(B)
ZipperZ(o-o-o)-A^Bthe two files interleave, each hiding the other's spans as commentsA.bZipper and both bParasite — only TAR and DICOM host it today

The predicates are literal source. isStackOk in mitra.py is the entire "suffix-tolerant ∘ prefix-tolerant" rule of the source outline, written as four lines of Python:

python
# mitra.py — isStackOk (abridged)
if not ftype1.bAppData:      # host must tolerate appended data
    result = False
if ftype2.start_o == 0:      # guest must tolerate a non-zero start offset
    return False
elif len(ftype1.data) >= ftype2.start_o:
    result = False           # ... and the host must fit inside that tolerance

start_o is the guest's maximum tolerated start offset, not its actual one — the number of bytes of foreign prefix it will forgive. That single field is the whole prefix-tolerance axis, and its values are startlingly concrete:

Formatstart_oSource
ZIP / RAR / 7z / ARJ4 MiBself.start_o = 4*1024*1024 # no actual downward limit (parsers/zip_.py)
PDF1016self.start_o = 1024 - 8 (parsers/pdf.py)
ISO 96600x8000as a cavity, precav_s = 0x8000 (parsers/iso.py)
DICOM128the DICM magic sits after a 128-byte preamble
TAR0 + slackmagic \0ustar at MAGIC_o = 0x100 (parsers/tar.py)
Everything else0magic enforced at offset zero

1024 - 8 is not folklore; it is the arithmetic of "the whole %PDF-1.x signature has to be present in the first kilobyte", and it is directly observable. pocs/pdf/1016garbage.pdf is 1648 bytes of which the first 1016 are junk:

text
$ od -A d -c pocs/pdf/1016garbage.pdf | head -1
0000000 001   P   K 003 004  \0   J   F   i   f   R   a   r   ! 032  \a
$ od -A d -c pocs/pdf/1016garbage.pdf | sed -n '5p'
0001008 360 361 362 363 364 365 366 367   %   P   D   F   -   1   .   4

The signature lands at byte 1016 exactly, so the eight-byte magic ends at 1024. The junk is not random: it opens with PK\3\4, JFif, Rar!\x1a\x07, <html>, Acsp and PK\5\6 — a deliberate demonstration that the tolerated prefix is a free-fire zone for other formats' magic.

The format model: tolerance as data ​

Every mitra parser subclasses FType (parsers/__init__.py), whose constructor is the taxonomy — eleven fields, of which three are booleans and the rest are offsets or sizes:

python
# mitra/parsers/__init__.py — FType.__init__ (abridged, comments verbatim)
self.cut = None         # minimal cut generic to that format
self.prewrap = 0        # [minimal] size of data to be added before the parasite
self.postwrap = 0       # [minimal] size of data to be added after the parasite
self.start_o = 0        # where the format should start in the file
self.bAppData = True    # does it tolerate appended data - quite common
self.bParasite = False  # does it tolerate any parasite - quite common
self.parasite_o = None  # min offset of a parasite (=cut + prewrap ?)
self.parasite_s = None  # max size of a parasite
self.precav_o = 0       # (fixed) offset of a pre-cavity
self.precav_s = 0       # (max) size of pre-cavity

and four methods — identify, getCut, wrap, fixformat — which the README reduces to a single sentence:

"For example, in a chunk-based format, just find where to cut the file, then wrap foreign data in a new chunk and insert the chunk. So you just need to teach Mitra how to identify the type, where to cut, and how to wrap."

PNG is the canonical instance and is 30 lines total (parsers/png.py): cut at 8 (immediately after the signature, before IHDR), prewrap = 2*4 (a 4-byte length and a 4-byte type), postwrap = 4 (the CRC), and:

python
# mitra/parsers/png.py
def wrap(self, data, type_=b"cOMM"):
    return b"".join([
        int4b(len(data)), type_, data,
        int4b(binascii.crc32(type_ + data) % 0x100000000)
    ])

cOMM is chosen for its capitalization, not its spelling: in the PNG chunk grammar the case of each of the four letters carries a property bit. The lowercase first letter makes the chunk ancillary — a decoder that does not recognise it must skip it rather than fail — and the uppercase fourth letter makes it unsafe to copy, which is honest, since an editor must not carry the parasite across a re-encode. (The uppercase second letter claims a public, registered type, which cOMM of course is not; nothing enforces that bit.) PNG's forward-compatibility rule is therefore the parasite mechanism, and it is a rule the format got right for maintainability and cannot revoke without breaking every decoder.

WASM is structurally identical and even shorter: section id 0 is the custom section, so the wrapper is \0 + LEB128 length + a name-length-prefixed blob (parsers/wasm.py), and bAppData = True # via Wrappending — appended data is legal because you can always append another custom section. A format designed in 2015 with an explicit extension point inherits the same composability as PNG from 1996. That is not an accident of either spec; it is what "tolerate unknown chunks" means.

The host/guest asymmetry ​

Composition is not symmetric, and mitra's matrix is deliberately a full square rather than a triangle. S(x)-PDF-ZIP and S(x)-ZIP-PDF are different files with different failure modes. The four ZIP-family formats and PDF/ISO/DICOM/TAR each combine with 30–41 of the surveyed formats, while ELF, PNG, GIF, PE, Java and BPG each combine with 6–8 — and every one of those 6–8 is "as a host to one of the eight promiscuous guests", never as a guest themselves, because their magic is enforced at zero.

Two formats in the whole table compose with nothing: XZ and ID3v1. The reason is one line each. parsers/xz.py:

python
self.bAppData = False  # Required matching footer
self.bParasite = False # No known strategy

and parsers/id3v1.py:

python
self.bAppData = False # it's a footer

A mandatory footer is the only structural feature in this corpus that defeats every layout at once. It kills stacking (nothing may follow), it kills being stacked onto (start_o == 0), and if the format also has no comment/extension chunk it kills parasitism. This is the single most transferable finding on the page, and it is the exact inverse of the property that makes ZIP the universal suffix parasite — see ZIP parasitism and footer-indexed formats.

Format identity and multiplicity ​

What the bytes are ​

There is no one artifact here, so multiplicity is measured per specimen. The PoC||GTFO series is the best-documented run, because its README.md records, per issue, exactly what the released file simultaneously is. Reproduced and, where marked ✓, verified first-hand against the release blobs at angea/pocorgtfo@933c020f:

IssueDateSimultaneouslyVerified
0x002013-08-05PDF only—
0x012013-10-06ZIP, PDF✓ 6 files; EOCD 63 bytes from EOF (release)
0x022013-12-28MBR, ZIP, PDF — "This OS is also a PDF" (article)—
0x032014-03-02JPG, AES(PNG), ZIP, AFSK audio, PDF✓ FF D8 at 0, %PDF at 25 inside a FF FE comment (release)
0x042014-06-27TrueCrypt volume, ZIP, PDF—
0x052014-08-10ISO, SWF, ZIP, PDF—
0x062014-11-25TAR, ZIP, PDF (article)✓ ustar at 257; first TAR member is named %PDF-1.5 (release)
0x072015-03-19BPG, HTML, ZIP, PDF — the "Funky Files" issue—
0x082015-06-20Shell script, ZIP, PDF—
0x092015-09-14WavPack, ZIP, PDF—
0x102016-01-16LSMV (a TAS movie), ZIP, PDF—
0x112016-03-17Ruby, HTML, ZIP, PDF—
0x122016-06-18APK, ZIP, PDF—
0x132016-10-04PostScript, ZIP, PDF—
0x142017-03-20iNES ROM, ZIP, PDF — plus MD5 hashquines and a collision—
0x152017-06-17ILDA (laser-projector frames), ZIP, PDF—
0x162017-10-20Bash (also Python, WebIDE), ZIP, PDF—
0x172017-12-30Apollo Guidance Computer source, ZIP, PDF—
0x182018-06-26HTML, PDF, ZIP — with a SHA-1 collision—
0x192019-03-27HTML, PDF, ZIP — with an MD5 pileup across PE/PDF/PNG/MP4—
0x202020-01-21PDF, ZIP — signed—
0x212022-02-12PCAPNG, PDF, ZIP—
0x222024-02-12ISO, ZIP, PDF + mocks: TAR, DICOM, XMS, PIF…✓ DICM at 128, CD001 at 32769, EOCD comment covers the PDF trailer (release)

Two structural constants across twenty-three issues are worth naming. ZIP appears in every polyglot issue from 0x01 onward — it is the invariant host, for reasons developed under Index anchoring. And PDF is always the presentation type, because PDF is the only widely-deployed format that tolerates a kilobyte of leading garbage, tolerates arbitrary appended data, and is parsed bottom-up, so it can be the guest of a header-anchored format and the host of a footer-anchored one at the same time.

The prefix/suffix partial order ​

The source outline asks for "a partial order of prefix-tolerant / suffix-tolerant / neither" that would predict polyglots. mitra's field set delivers it, with one refinement: the order is over four properties, not two, and the fourth (interior parasitism) is what rescues formats that are strict at both ends.

PropertyFieldFormats in the surveyed set
Prefix-tolerant (scanned magic)start_o > 0ZIP, 7z, ARJ, RAR (4 MiB); PDF (1016); TAR, DICOM, ISO (fixed offsets)
Prefix-tolerant (cavity)precav_s > 0ISO (0x8000), PDF-as-cavity (1018, parsers/pdfc.py)
Suffix-tolerantbAppDataalmost everything — the default in FType is True
Interior-tolerantbParasiteevery chunked format; PNG, WASM, JPEG, JP2, RIFF, ELF, PE, NES, GZIP, PostScript
None of the above—XZ, ID3v1

The predictive rule that falls out: a pair composes iff one member is interior- or prefix-tolerant at a distance that exceeds the other member's size, or one is suffix-tolerant and the other prefix-tolerant. Everything else in the matrix is bookkeeping about how much space, which is why the tool records the swap offsets in the output filename: Z(80-162-286)-DICOM^TIFF.…dcm.tif.

Note the deliberate line this catalog draws: a mock is not a polyglot. mocky.py plants foreign magic in a valid PDF's slack until file --keep-going --raw reports fourteen types for a file that is only a PDF. That is Multiplicity 1 attacking the dispatcher, and it belongs in threat model and parser differentials, not here — but issue 0x22 ships both in one file, which is why the file output on it is so long.

Chimeras: multiplicity without duplication ​

The chimera is the case where multiplicity costs no bytes. The worked example in "Abusing file formats" is the JPG/PDF/ZIP chimera, whose byte map the article prints in full; the load-bearing observation is that all three formats store JPEG data uncompressed and contiguously, so one copy serves all three:

  • The ZIP's local file header is followed immediately by stored file data — and by a duplicate of the filename, which the PDF's endstream keyword is made to occupy.
  • The PDF stores an image XObject as a raw stream of exactly those bytes.
  • The JPEG is those bytes, reached from offset 0 by way of a second JFIF header planted before them.

"Even better, we only have one copy of the image data; this copy is reused by each of the forms of the chimera."

The same technique in a different pair is pocs/poly/zgip, a ZIP/GZIP chimera whose Deflate stream is shared: "Just to prove that while Zip and Gzip can use the same compression algorithm, neither is an encapsulation of the other." This is the sharing/duplication question of cluster E answered at the format level rather than the store level — see content-addressed chunking for the same idea when the sharing is between artifacts instead of within one.

Index anchoring and random access ​

Why ZIP is always the host ​

ZIP's index is the End of Central Directory record, found by scanning backwards from EOF for PK\x05\x06 within a 64 KiB window (the maximum comment length). Nothing about that procedure references offset 0. "Abusing file formats" states the historical reason, which is not the one usually given:

"ZIP doesn't require magic at offset zero, and like PDF it's parsed from the bottom up. In this case, it's not to allow for incremental updates; rather, it's to limit those time-consuming floppy swaps when a multi-volume archive is created on the fly, on external storage."

The consequence, measured on the real releases:

FileSizefirst PK\x03\x04EOCD offsetbytes after EOCDEOCD.cd_off fieldresolves how
pocorgtfo01.pdf3 790 4383 505 1493 790 37563284 738relative — +3 505 149 lands on PK\x01\x02
pocorgtfo06.pdf101 508 87810 672 929101 508 8146490 824 610relative — +10 672 929 lands on PK\x01\x02
pocorgtfo22.pdf53 215 8889 60053 215 81078 (56 in-comment)53 214 138absolute — already correct

Both behaviours are legal-ish and both work, which is exactly the problem. Info-ZIP prints warning: 3505149 extra bytes at beginning or within zipfile (attempting to process anyway) and self-corrects by taking the delta between the EOCD's claimed central-directory offset and where it actually found it. Issue 0x22 instead pre-fixes the pointers, so no warning appears — the fixup Albertini describes under "Fixing Absolute Pointers" and that zip -A performs. A ZIP index is therefore only conditionally random-access: the offsets are relative to a base the format never records. Consumers reconstruct that base by search. That is the mechanism by which the same archive is simultaneously "valid" and "damaged", and it is the seam every parser differential in the ZIP ecosystem is built on.

PDF's two anchors, and Julia Wolf's trick ​

PDF is the rare format that is anchored at both ends: a header signature that must appear within the first kilobyte, and a startxref footer giving the byte offset of the cross-reference table. Because the footer is authoritative, PDF supports incremental update, and because the header is merely "within 1 KiB", it is prefix-tolerant.

The canonical way to nest a ZIP inside a PDF exploits the space between those anchors. From "Abusing file formats":

"A good way to embed a ZIP in a PDF, as Julia Wolf showed us with napkins in PoC||GTFO 1:05, is to create a fake stream object after the xref, where the trailer object is present, before the startxref pointer."

with the layout given as PDF signature → objects → cross-reference table → (extra stream object containing the ZIP) → trailer → startxref. The reason is quantitative, not aesthetic: the EOCD must be within 64 KiB of EOF, and the cross-reference table grows linearly with the object count, so a ZIP placed among the normal objects gets pushed out of the window on any large document.

pocorgtfo01.pdf is that layout, verifiable byte-for-byte:

text
startxref → 3 504 383  →  "169 0 obj <</Type /XRef /Index [0 170] /Size 170 …"
3 505 127                  "999 0 obj\n<<>>\nstream\n"      ← the fake object
3 505 149                  PK 03 04 …                        ← the ZIP begins
3 790 375                  PK 05 06 (comment length 0)
3 790 438 (EOF)            "endstream\nendobj\nstartxref\n3504383\n%%EOF\n"

Object 999 is declared after the object that startxref points at, so no PDF pointer needs adjusting and the xref stream stays byte-identical to what a normal producer emits. mutool clean will happily renormalize such a file — the tradition's advice is to run it, because "it modifies very little, yet rebuilds the XREF table and adjusts objects lengths, which turns your hand-made tolerated PDF into one that looks perfectly standard."

pocorgtfo22.pdf uses the complementary trick: the EOCD really is last, and its 56-byte comment field swallows the PDF's own trailer —

text
PK 05 06 … comment_len = 56
comment = "\0\0\0\0" "endstream\nendobj\nstartxref\nstartxref\n37074454\n%%EOF\n"

— with four leading NUL bytes so that a ZIP tool displaying the comment as a C string shows an empty one. The archive's index is the last structure in the file and the PDF's %%EOF is the last structure in the file, because one is nested in the other's variable-length tail.

Offset-anchored guests ​

The third anchoring style — magic at a fixed non-zero offset — is the reason a single file can carry so many claimed types at once. pocorgtfo22.pdf, read directly:

OffsetBytesClaimed format
0%PDF-1.5PDF header
128DICMDICOM (128-byte preamble by spec)
9 600PK\x03\x04ZIP local file header
32 769CD001ISO 9660 Primary Volume Descriptor
53 215 810PK\x05\x06ZIP EOCD

A format that puts its magic at a fixed offset has donated every byte before that offset to whoever wants it. DICOM donates 128 bytes; ISO 9660 donates 32 KiB; TAR donates 257 bytes of which the first 100 are a filename field — which is precisely why pocorgtfo06.pdf's first TAR member is named %PDF-1.5:

text
$ tar -tvf pocorgtfo06.pdf | head -1
-rw-r--r-- Manul/Laphroaig   0 2014-10-06 22:33 %PDF-1.5

A zero-length file whose name is the other format's signature, with \0ustar \0 at 257 and a valid header checksum. The TAR half is not hiding in the PDF; the PDF's identity is a field of the TAR half. That is the article's own title — "This TAR archive is a PDF!" — meant literally.

Cost of a partial read ​

For the catalog's comparison purposes: a polyglot has no unified index, so the cost of a ranged read is whatever the chosen parse costs, times nothing shared. Fetching pocorgtfo22.pdf's ZIP listing needs the last 64 KiB plus the central directory; fetching its first PDF page needs the last few KiB (for startxref) plus the object graph reachable from /Root; fetching the ISO's volume descriptor needs bytes 32 769..32 774. Three parses, three access patterns, zero shared machinery — the opposite of the single b-tree story in SQLite as an application file format. Whether any of those parses survives HTTP range access is exactly the question in range-request access; the answer for ZIP-in-PDF is yes for ZIP, awkwardly for PDF, and it is awkward because the base-offset reconstruction above requires locating PK\x03\x04 by scan.

Reflexivity and query surface ​

This is the axis on which polyglot craft scores lowest, and the absence is a finding.

A polyglot carries no schema, no manifest, no self-description. Nothing in pocorgtfo22.pdf announces "I am also an ISO". There is no equivalent of SELF's sqlite_schema, no sqlelf virtual table, no Wasm component type export. The artifact's multiplicity is implicit — recoverable only by trying every parser you own. That is the definition of the problem, not a shortcoming of the implementations: the entire craft depends on each parser believing the file belongs to it and to nobody else.

What interrogation surface exists is entirely out-of-band and heuristic, and the tradition's own tooling makes the point:

text
$ file pocorgtfo22.pdf
pocorgtfo22.pdf: tar archive

$ file --keep-going --raw pocorgtfo22.pdf
pocorgtfo22.pdf: tar archive
- DR-DOS executable (COM)
- Windows Program Information File for 145
- PDF document, version 1.5
- ISO 9660 CD-ROM filesystem data (DOS/MBR boot sector) 'CDROM'
- DICOM medical imaging data
- Nintendo DS ROM image: "%PDF-1.5"
…

Two things follow, and both are load-bearing for the catalog:

  1. Default sniffing reports one type, and it is the wrong one. file without --keep-going stops at the first match, and its ordering is by internal magic-entry strength, not by what the file "is". A 53 MB journal identifies as tar archive. The mitra README's own demonstration is blunter: a plain PDF, after mocky.py --combine, still passes pdftotext and pdfinfo cleanly while file calls it a TAR. Content sniffing is not a query surface; it is a guess with a stable tie-break.
  2. Multiplicity is only ever a lower bound. file --keep-going finds magic; it does not validate, as the README says explicitly: "(it does not validate the formats, but at least gives you some information)". A reported type may be a mock; an unreported type may be a real, valid parse whose magic libmagic has no entry for.

The nearest thing to genuine self-interrogation in the corpus is the PDFLaTeX quine — a file that is simultaneously its own TeX source and the PDF that compiling that source produces (pocs/pdf/quine.pdf; the technique, \pdfcompresslevel=0 plus \immediate\pdfobj stream file {…}, is in "Abusing file formats"). That is reflexivity in the autological sense this catalog cares about — the artifact contains a complete description of how to rebuild itself — but it is still not queryable: there is no language in which to ask it a question, only a compiler that reproduces it. Score: 1, incidental.

The contrast to draw explicitly, because it is thesis 2 of the source outline: PDF, ZIP and TAR are all formats without a carried schema, and all three have accreted exactly the conventions that thesis predicts — libmagic entries, zip -A fixups, "attempting to process anyway" recovery, per-reader tolerance of a truncated %PDF-1. signature. The polyglot exists in the gap between the spec and the accreted conventions. A format that carried its own schema would not have that gap in the same shape.

Closure, dedup, and size model ​

What travels ​

PoC||GTFO's ZIP half is a real closure claim, if a modest one: each issue carries the code, patches, ROMs and tarballs its own articles discuss — the "feelies". This is the one axis where the artifact is doing something redbean also does, and doing it first. Verified contents of pocorgtfo03.pdf's archive (15 members, 14.8 MB):

text
alexander.txt  bochs-2.6.2.patch  bochs-20140203.patch  defusing.zip
despair.txt    lasta.txt          lastq.txt             netwatch-337f8b1.tar.gz
nokiacipher.png  packed  saucers.txt  tamadec.txt  tetranglix.tar.bz2
pocorgtfo02.pdf  pocorgtfo03-encrypt.py

Note pocorgtfo02.pdf — the previous issue, carried inside this one, itself a ZIP/MBR/PDF polyglot. And pocorgtfo03-encrypt.py, the script that produces this issue's AngeCryption layer: the artifact carries its own construction recipe, which is the closest the corpus comes to the reproducibility story in embedded provenance.

But this is payload bundling, not dependency closure. Nothing in the file lets a reader resolve what netwatch-337f8b1.tar.gz needs to build, and nothing enumerates the transitive set. There is no equivalent of a Nix closure or a DT_NEEDED graph. Score: 1.

Size model ​

Polyglot size is additive minus overlap, and the overlap term is the whole craft:

LayoutSize costExample
Stacklen(A) + len(B) exactlyany ZIP appended to anything
Cavitymax(len(A), len(B)) when A fits the cavityISO's 32 KiB system area absorbs A for free
Parasitelen(A) + len(B) + wrap (wrap ≈ 12 for PNG)PNG cOMM: 4 length + 4 type + 4 CRC
Chimeramax(len(A), len(B)) — payload sharedJPG/PDF/ZIP: one copy of the image; ZIP/GZIP: one Deflate stream
Near-polyglot (--overlap)as chimera, plus the recovered bytes in the filenameO(5-204){424D4E0100}.bmp.jpg — 5 overwritten bytes recorded out-of-band

The chimera row is the only one that beats concatenation, and it beats it only when the two formats agree on a payload encoding. Albertini's list of what makes that possible is short and worth reading as a compatibility table: uncompressed JPEG data (JPG/PDF/ZIP), raw Deflate (ZIP/GZIP/PNG), and — with ascii-zip-style Huffman abuse — a Deflate stream that is also printable ASCII, which is how Rosetta Flash worked.

The hard size constraint the tradition keeps hitting is the opposite one — formats with a required exact size. From "Abusing file formats":

"It's common that ROM and disk images require a specific rounded size, and there is often no workaround to this. You can merge a PDF and an Apple II floppy image, but only if the PDF fits in the 143,360 byte disk image."

The workaround used for pocs/poly/Apple2PDF was to move up to a 2 MB hard-disk image; the workaround in pocs/poly/SnesMd (a Super NES + Sega Megadrive + PDF triple) was to put the PDF at the bottom of the ROM rather than after it, "because the exact rom size is critical for SMC". A size-exact format is prefix- and suffix-hostile even when it is interior-tolerant — a fifth column the FType model does not have a field for, and the one gap in the taxonomy this survey found.

Mutability, dispatch, and trust ​

Mutability: zero, by construction ​

A polyglot is the most brittle artifact in this catalog. Its correctness is a conjunction of n invariants, several of which are checksums over overlapping ranges: PNG per-chunk CRC32, ZIP per-member CRC32, TAR's header checksum, PDF's startxref offset and per-stream /Length, the iNES trainer flag, MBR's 0xAA55. Change one byte in the shared region and you must recompute every dependent field in every format — which is why mitra parsers carry a fixformat hook and why parsers/pdf.py contains a full xref/startxref rewriter whose comment reads "dumb [start]xref fix: fixes old-school xref with no holes, with hardcoded \n".

The artifact is therefore not a state store, not transactional, and not incrementally editable in any sense the SELF/selfdb line means. Score: 0. PDF's incremental-update feature is the one exception in principle — appending a new xref section is legal — but doing so in a polyglot moves EOF and so invalidates the ZIP's 64 KiB EOCD window and any footer-anchored guest. Editing collapses the superposition.

There is one striking mutability-adjacent exhibit: the MD5 hashquines in issue 0x14, files that display their own MD5 hash. That is self-reference without self-modification — the fixed point is found by collision search at build time, not maintained at runtime — and it belongs with the measurement discussion of what "the artifact knows about itself" can be made to mean without a query engine.

Dispatch: the consumer, and that is the vulnerability ​

For every other subject in this catalog, dispatch is owned by a named component: the kernel for binfmt_misc, the loader for ld.so, the shell for a shebang. For a polyglot, dispatch is owned by whichever consumer happens to open the file, using whichever of three incompatible mechanisms it prefers:

MechanismUsed byWhat the polyglot does to it
Filename extensionshells, desktop environments, most upload pathspocorgtfo07.pdf → rename to .html and it is a working web page
Magic sniffingfile/libmagic, TrID, AV, IDS, browsersreports one type; a mock makes it report the attacker's chosen type
The parser's own recoveryPDF readers, ZIP tools, media playersaccepts the file that the sniffer said was something else

The security consequence is the tradition's whole argument, and it is stated plainly:

"Testing various polyglots on Encase showed that nearly all of them were reported as a single file type, with no warnings whatsoever."

and, on the ZIP-scanning divergence that breaks signatures:

"This is likely why some modern tools take a different approach, ignoring the official structure of a ZIP. These extractors start at offset zero and look for a sequence of Local File Headers. … Sadly, doing this differently makes ZIP multi possible, which can be critical as it can break signatures and the complete chain of trust of a standard system."

That sentence is the bridge to parser differentials and to the JAR/APK signing problem in ZIP parasitism: a verifier that enumerates members top-down and an extractor that enumerates them from the central directory can disagree about which bytes were signed.

Trust: blacklists, and why they lose ​

The one deployed mitigation the corpus documents is Adobe's, and it is instructive precisely because it half-worked:

"For security reasons, Adobe Reader, the standard PDF reader, has blacklisted known magic signatures such as PNG or PE since version 10.1.5. It is thus not possible anymore to have a valid polyglot that would open in Adobe Reader as PDF. This is a good security measure even if it breaks compatibility with older releases of PoC||GTFO."

The immediate bypass is a lesson about what a magic number actually is:

"However, it's critical to blacklist the actual signature as opposed to what is commonly appearing in files. JFIF files typically start with the signature, SOI, and an APP0 segment, which make the file start with FF D8 FF E0. However, the signature itself is only FF D8, which can lead to a blacklist bypass by using a different segment or different marker right after the signature."

pocorgtfo03.pdf is that bypass, and its first 30 bytes still show it:

text
FF D8                                   JPEG SOI — the entire real magic
00 00 00 10 4A 46 49 46 …               "JFIF" text, but NOT preceded by FF E0
FF FE 00 22                             COM segment, length 0x22
0A 25 50 44 46 2D 31 2E 35 0A           "\n%PDF-1.5\n"  ← at offset 25
39 39 39 20 30 20 6F 62 6A …            "999 0 obj\n<<>>\nstream\n"

The JPEG decoder skips from FF D8 to the next marker and finds the comment; the PDF reader finds %PDF-1.5 inside that comment, comfortably within its 1 KiB budget; the blacklist, which was looking for FF D8 FF E0, sees nothing. (Adobe subsequently fixed the JFIF signature check, and the README records that pocorgtfo03.pdf has not opened in Adobe Reader since March 2014 — the mitigation eventually landed, three releases and one bypass later.)

The structural recommendation the tradition offers instead of blacklisting is a whitelist tightening, and it is the single most actionable line in the corpus:

"Requiring the PDF signature to appear earlier in the file - even just in the first 64 bytes instead of a whole kilobyte - would proactively prevent a lot of polyglot types, as most recent formats are dense at the start of the file."

That is a quantitative trust claim — reduce start_o from 1016 to 56 and the set of formats that fit in the prefix collapses to almost nothing — and it is the kind of claim the threat model page can test directly against the mitra matrix.

Finally, on signing: issue 0x20 is "polyglot: PDF, ZIP — Signed", which is the corpus quietly conceding the point. A digitally signed PDF fixes a byte range; once the artifact is signed, the polyglot is frozen, and every technique above becomes a build-time-only affordance. The general problem — signing an artifact whose bytes several verifiers disagree about the extent of — is the subject of embedded provenance, and it has no answer here.

Strengths ​

  • The taxonomy is executable. mitra is not a description of which formats compose; it is a program that tries, and its 288-combination matrix is regenerable. A claim in this area is falsifiable in one command.
  • Zero infrastructure. No loader, no kernel registration, no runtime, no substrate. A polyglot works on any system that has any of its parsers, including systems built decades apart — pocs/poly/SnesMd targets a 1988 console, a 1990 console and a PDF reader from the same 512 KiB.
  • Chimeras achieve multiplicity with no size penalty, which no other technique in this catalog does — the payload is shared rather than duplicated.
  • The corpus is a real regression suite for parsers. The mini collection (pocs/mini/README.md) — a valid, minimal, self-describing file for each of ~60 formats, with rules ("should be fully valid… shows what they are… should be made as small as possible") — is directly reusable as fixtures, and the pdf/ PoCs form a compatibility matrix across readers.
  • The design advice generalizes. "Magic at zero, mandatory footer, an explicit comment chunk instead of tolerated garbage" is four rules that a new format can adopt for free and that measurably shrink its composability surface.

Weaknesses ​

  • No reflexivity whatsoever. The artifact cannot be asked what it is. Every downstream consumer re-derives multiplicity by brute force, and the answer is a lower bound.
  • Fragility is multiplicative. n formats mean n sets of checksums and offsets over overlapping ranges; the file is effectively immutable after construction, and normalization tools routinely destroy it ("the library — too smart for its britches — removed your dummy chunk, recompressed your intentionally uncompressed data…").
  • Compatibility is a heisenbug problem, in the tradition's own words: "a single font in a PDF might become corrupted. One image — and only one image! — might go missing." The boundary that matters is not valid/invalid but "good enough" versus "let's try to recover".
  • Reach is per-parser and decays. Every specimen in the corpus is a bet on specific implementations; Adobe's blacklist retired issue 0x03 and 0x05 from the flagship reader within months of each.
  • It is fundamentally the wrong tool for deployment. Albertini says so himself: "Archiving files together is much more natural than making a polyglot file. Although opening a polyglot file may be transparent for the targeted software, it's not a natural action for user."

Key design decisions and trade-offs ​

DecisionRationaleTrade-off
Model a format as five booleans + four offsets (FType)Composability is decidable from tolerance properties alone; no need to model semanticsMisses size-exact formats (ROM/disk images), which the model calls composable and reality does not
Enumerate four layouts (Stack / Cavity / Parasite / Zipper) rather than searchEach maps to one structural property, so a hit is explained, not just foundLayouts outside the four (interleaved chimeras, encoding-level sharing) still need hand-crafting
ZIP as the invariant host of every PoC||GTFO issueBackwards-scanned EOCD + 64 KiB comment makes it the universal suffix parasiteInherits ZIP's relative-offset ambiguity — "damaged" warnings, and a signing surface
Put the ZIP in a fake PDF object after the xrefKeeps EOCD inside the 64 KiB window regardless of object count; no PDF pointer needs fixingRelies on readers tolerating an undeclared object between xref and trailer — legal by omission, not by statement
PDF as the presentation type everywhereThe only common format that is prefix-tolerant (1 KiB), suffix-tolerant, and footer-authoritative at onceThat very tolerance is what got PDF a magic blacklist; the technique attracted the mitigation
Chimera (shared payload) over concatenation where encodings agreeMultiplicity at max(len) instead of sum(len)Requires both formats to store the payload uncompressed or with the same codec; brittle to any re-encode
Publish mocky.py (mocks) alongside real polyglotsSeparates "attacks the parsers" from "attacks the dispatcher" — different mitigationsMuddies type-counting: file --keep-going output for issue 0x22 lists types the file is not
Record swap offsets in the output filenameThe metadata has nowhere else to live — the artifact carries no self-descriptionOut-of-band metadata: rename the file and the provenance is gone. The exact gap SELF closes with a table

Where this sits in the catalog ​

  • Against thesis 1 ("every binary format eventually reimplements a database, badly"). Supported, with a twist: PDF's xref is a hand-maintained primary-key index over objects, and mitra's parsers/pdf.py has to reimplement it ("only very standard object declarations") to insert one payload. ZIP goes further and stores two copies of the same relation — filenames appear in both the local file headers and the central directory — which is a denormalized index maintained by hand, and Albertini's closing section asks for exactly the schema fix a database person would: "One that doesn't duplicate file names between Central Directory and Local File Headers?"
  • For thesis 2 ("self-description is what makes a format survivable"). Strongly supported in the negative. Every format in the corpus that lacks a carried schema has accreted a recovery convention (file's heuristics, zip -A, "attempting to process anyway", truncated-signature tolerance), and the polyglot lives in the gap between spec and convention. XZ and ID3v1, the two formats that compose with nothing, are also the two that enforce a structural invariant end-to-end.
  • Against thesis 3 ("the container is a tax"). Complicated. The chimera is the counter-example: when host and guest agree on an encoding, the container costs zero bytes. The tax is real for stacking and parasitism and is exactly quantifiable — 12 bytes per PNG chunk, 2 + 2 per JPEG segment, ~30 + filename per ZIP member.
  • For thesis 5 ("portability migrated from the format to the access layer"). This page is the strongest evidence for the thesis, precisely by exhausting the alternative. Twenty-three issues of format-level cleverness produced artifacts whose reach shrinks as parsers tighten, that cannot be edited, and whose type nobody can query. See the comparison table for the side-by-side against APE and SELF.

Sources ​