Sparkles

Appearance

Image-based systems (Smalltalk, Lisp, Emacs, CRIU, Erlang/OTP) ​

Programs that carry their own state: the deepest prior art for the Mutability axis, and the tradition in which every participant eventually collided with the same wall — a memory image and a demand-paged mmap-ed file want incompatible things.

FieldValue
KindRuntime state snapshots (object memory / heap / process) and one live-mutation system
LanguageSmalltalk + generated C (OpenSmalltalk/Spur) · C (Emacs pdumper, SBCL runtime, CRIU) · Common Lisp (SBCL save-lisp-and-die) · C + Erlang (ERTS, SASL release_handler)
LicenseOpenSmalltalk-VM MIT · GNU Emacs GPL-3.0-or-later · SBCL public domain + BSD-style · CRIU GPL-2.0 (library MIT) · Erlang/OTP Apache-2.0
RepositoryOpenSmalltalk/opensmalltalk-vm · emacs-mirror/emacs · sbcl/sbcl · checkpoint-restore/criu · erlang/otp
Documentationsqueak.org/documentation · Emacs Lisp Reference, "Building Emacs" · SBCL manual, save-lisp-and-die · criu(1) · Erlang Reference Manual, "Compilation and Code Loading"
First releaseSmalltalk-80 images 1980 · Emacs unexec 1982 (Spencer W. Thomas, unexec.c header) · SBCL 1999 (from CMUCL) · Erlang/OTP releases 1996 · CRIU 2012 · Emacs pdumper in Emacs 27.1 (2020)
Axis profileMultiplicity 1 / Reflexivity 2 / Closure 2 / Mutability 3
Index anchoringheader — a magic-plus-version record at offset 0 naming section extents (Squeak, pdmp, SBCL core); CRIU is the exception, and is out-of-band
Dispatch ownerloader — the runtime binary, not the kernel: an image is data that a specific interpreter opens. Only SBCL's :executable t and Emacs's dumped emacs are binfmt_elf files in their own right

Revisions surveyed: emacs-mirror/emacs @ b28750b8 (2026-08-26, configure.ac says 32.0.50) · checkpoint-restore/criu @ 71285b3e (2026-08-23, Makefile.versions 4.2.1) · erlang/otp @ 15f55651 (2026-08-26, OTP_VERSION 30.0-rc0) · sbcl/sbcl @ 6b0c5fe4 (2026-08-26, NEWS 2.6.7-dev) · OpenSmalltalk/opensmalltalk-vm @ fb2be466 (2026-08-21). Measured artifact: the emacs-30.2 portable dump from this machine's Nix store. Platform: POSIX throughout; CRIU is Linux-only.

NOTE

Five systems on one page because they answer the same question — "what does it mean for a program to carry its own state?" — and because their disagreements are the finding. The Lisp-machine lineage is treated here only as the maximal case and is developed in single-level store; SELF/selfdb is the modern descendant and owns its own page. The umbrella is Autological Artifacts.

Overview ​

What it solves ​

Start-up is the cheap answer and the wrong one. The expensive part of starting a large interactive system is not parsing its code but reconstructing its object graph: interning symbols, building method dictionaries, wiring class hierarchies, evaluating top-level forms whose results other top-level forms depend on. Doing that from source at every launch costs seconds. Doing it once and writing the resulting heap to a file costs a read or an mmap.

That is the mechanical justification, and every system here started from it. But the systems divide sharply on what they then did with the artifact:

SystemWhat is in the artifactWritten byCan the running program rewrite it?
Smalltalk-80 / Squeak / PharoThe entire object memory, including the IDE and the developer's unsaved workthe image itself, on snapshotYes — this is the normal way to work
Lisp images (SBCL save-lisp-and-die)The whole Lisp heap, plus optionally the runtimethe image itself, then it exitsYes, but the process must die doing it
Emacs unexec (1982–2024)The running process's data segment, re-emitted as an executablethe running processYes — it rewrote its own a.out/ELF
Emacs pdumper (2020–)The Lisp heap only, as an explicit serialization plus relocation tablesa -batch build stepNo — the dump is read-only input to a fixed binary
CRIUAn arbitrary process tree's memory, fds, sockets, namespacesan external privileged observerNot by the process; by crit encode, yes
Erlang/OTP hot code loadingNothing on disk — the mutation is to the live VM's code tablethe running node, transactionallyYes, in production, module by module

Two of those rows are the interesting ones. Erlang is the only system in this catalog that mutates a running program in production, under a defined rule; and Emacs is the only one that had a fully autological artifact — a file that was simultaneously the program, its state, and its own producer — and then deliberately gave it up. The unexec → pdumper transition is the single most instructive story on this page, because it is a controlled experiment in which one variable changed (the operating system stopped tolerating a hand-relocated process image) and the artifact's autology was the thing that had to go.

Design philosophy ​

The founding statement is the header comment of Emacs's unexec.c, and it is worth reading as written, because the whole design fits in one sentence and one date (src/unexelf.c, at the commit before its removal):

text
 * unexec.c - Convert a running program into an a.out file.
 *
 * Author:	Spencer W. Thomas
 *		Computer Science Dept.
 *		University of Utah
 * Date:	Tue Mar  2 1982
 * Modified heavily since then.
 *
 * Synopsis:
 *	unexec (const char *new_name, const char *old_name);
 *
 * Takes a snapshot of the program and makes an a.out format file in the
 * file named by the string argument new_name.

"Convert a running program into an a.out file" is the autological claim in six words. The counter-statement is Emacs's own, in the release notes that retired it (etc/NEWS.27):

"Emacs now uses a 'portable dumper' instead of unexec. This improves compatibility with memory allocation on modern systems, and in particular better supports the Address Space Layout Randomization (ASLR) feature, a security technique used by most modern operating systems."

And the manual states the same thing as a property of the replacement rather than a defect of the original (doc/lispref/internals.texi):

"This method is the most preferred one, as it does not require Emacs to employ any special techniques of memory allocation, which might get in the way of various memory-layout techniques used by modern systems to enhance security and privacy."

The Lisp world reached the identical conclusion by a different road and says so with less ceremony (SBCL manual, save-lisp-and-die):

"There is absolutely no binary compatibility of core images between different runtime support programs. Even runtimes built from the same sources at different times are treated as incompatible for this purpose. … This isn't because we like it this way, but just because there don't seem to be good quick fixes for either limitation and no one has been sufficiently motivated to do lengthy fixes."

Erlang's philosophy is the odd one out, and it is stated as a language-level guarantee rather than a build trick (system/doc/reference_manual/code_loading.md):

"The code of a module can exist in two variants in a system: current and old. … Both old and current code are valid, and can be evaluated concurrently. Fully qualified function calls always refer to current code. Old code can still be evaluated because of processes lingering in the old code."

Two versions, no more; a third load purges the oldest and kills whatever is still executing it. That is a garbage-collection rule for code, and it is the only mechanism in this survey that answers the catalog's open question about running a program out of a store somebody is concurrently rewriting.

How it works ​

Squeak/Pharo: the whole object memory, read and swizzled ​

A Squeak image is a header followed by the heap. readImageFromFileHeapSizeStartingAt reads a 4-byte imageFormatVersion (68021 for 64-bit Spur), a headerSize, a dataSize, the oldImageBaseAddress the heap was written at, and the specialObjectsOop root (src/spur64.stack/interp.c). It then allocates a heap and reads the segments in:

c
/* src/spur64.stack/interp.c — SpurSegmentManager>>readHeapFromImageFile:dataBytes: */
(segInfo->swizzle = newBase - oldBase);
bytesRead = sqImageFileRead(pointerForOop(newBase), sizeof(char), nextSegmentSize, f);

The destination is not a mapping of the image file. sqAllocateMemorySegmentOfSizeAboveAllocatedSizeInto obtains it with an anonymous mmap and retries at successively higher addresses until the kernel gives it something near where Spur wants it (platforms/unix/vm/sqUnixSpurMemory.c):

c
alloc = mmap(address, bytes, PROT_READ | PROT_WRITE /*| PROT_EXEC*/,
             MAP_FLAGS, -1, 0);

Then bytesToShift = oldSpaceStart - oldImageBaseAddress and initializeObjectMemory(bytesToShift) walks the heap applying swizzleObj to every field of every object, plus the special-objects array, the free lists, and the mark/weakling/mourn object stacks. The image is a heap dump with a base address, and loading it is a full pointer-rewrite pass. Nothing is file-backed, nothing is shared, and nothing is demand-paged.

The image is also famously not one file in practice. The .image needs its .changes beside it, with the same basename, and its .sources in the same folder, or method sources degrade to decompiled t1, t2 temporaries — the FAQ's first two entries are both about exactly that (squeak.org/documentation). The object memory is complete; the human-readable half of the artifact is out-of-band.

That second file is more interesting than the FAQ makes it sound. The .changes file is an append-only log of every source-level change made in the image — the FAQ confirms only that it must be co-located and is write-locked to a single running VM, and the append-only-log characterization here is from the ecosystem's usage rather than from a cited primary source — and it is what makes the working model survivable: a snapshot is a whole-heap write, so a crash between snapshots would otherwise lose everything since the last one, and the log is what a recovery replays. In this catalog's vocabulary the pair is a write-ahead log next to a page store, invented for the same reason SQLite has one and reached by the same route — a format whose only durable operation is "rewrite the whole thing" needs a journal beside it. It is also the reason the Smalltalk image is not, strictly, an autological artifact of the redbean kind: the file that is the program is not the file that records what changed in it.

Emacs: unexec, and why it had to go ​

unexec re-emitted the running process. The ELF variant read the original executable, enlarged .data to cover everything the process had allocated since start-up, rewrote the section and program headers, and wrote a new executable — explicitly avoiding mmap on the way ("We do not use mmap because that fails with NFS. Instead we read the whole file, modify it, and write it out.", src/unexelf.c). Emacs's own allocator had to cooperate: sheap.c provided a static heap so that early allocations landed somewhere unexec could describe. The whole arrangement depends on three assumptions that stopped holding — that the process's data lives at a fixed, known address; that malloc's internal state is safe to freeze and thaw; and that the loader will place the resulting file where the dumped pointers expect. ASLR breaks the first, modern malloc implementations the second, and PIE the third.

The replacement inverts the direction. pdumper does not dump the process; it serializes the Lisp heap and emits a relocation program for reconstructing it anywhere. The format's own comment states the contract (src/pdumper.c):

"An Emacs dump file contains the contents of the Lisp heap. On startup, Emacs can start faster by mapping a dump file into memory and using the objects contained inside it instead of performing initialization from scratch. The dump file can be loaded at arbitrary locations in memory, so it includes a table of relocations that let Emacs adjust the pointers embedded in the dump file to account for the location where it was actually loaded."

The dump is cut into three regions named in the header — DS_HOT, DS_DISCARDABLE, DS_COLD — and pdumper_load maps them contiguously from the same fd:

c
/* src/pdumper.c — pdumper_load() */
sections[DS_COLD].spec = (struct dump_memory_map_spec)
  { .fd = dump_fd,
    .size = dump_size - header->cold_start,
    .offset = header->cold_start,
    .protection = DUMP_MEMORY_ACCESS_READWRITE, };
if (!dump_mmap_contiguous (sections, countof (sections)))
  goto out;

DUMP_MEMORY_ACCESS_READWRITE resolves to MAP_PRIVATE in dump_map_file_posix, so the dump is file-backed and demand-paged — until a relocation writes to a page, which copies it. Two relocation kinds run: dump_relocs fix pointers inside the dump for the load address, and emacs_relocs write into the Emacs binary's own data segment, in six flavours (RELOC_EMACS_COPY_FROM_DUMP, RELOC_EMACS_IMMEDIATE, RELOC_EMACS_DUMP_PTR_RAW, …). The discardable region is MADV_DONTNEED-ed once its contents have been copied into Emacs.

Two consequences make Emacs the clearest case in the catalog. First, the artifact split in two: an ordinary PIE ELF executable and a separate .pdmp data file, located at run time via --dump-file or a default path. Second, the two halves are welded by a hash. make-fingerprint computes a digest of the freshly linked temacs and patches it into the binary in place (lib-src/make-fingerprint.c); pdumper_load compares it against the dump's copy and refuses on mismatch. On this machine the fingerprint is also the dump's filename: emacs-31c0351ac3ea39ff514f4b80c1a7bd95f4ab79ca561e6b09bc2e01611c300331.pdmp. That is content addressing arrived at independently, for the same reason build-ids exist.

Dumping is now a build step, not a user gesture: Fdump_emacs_portable refuses outright in an interactive session (src/pdumper.c):

c
if (! noninteractive)
  error ("Dumping Emacs currently works only in batch mode.  "
         "If you'd like it to work interactively, please consider "
         "contributing a patch to Emacs.");

The old regime's fingerprints are still visible in the new code. Fdump_emacs_portable binds Vpurify_flag to nil for the duration of the dump — purify being the pre-pdumper mechanism that copied objects into a separate "pure" space so unexec could mark them read-only — and, under REL_ALLOC, calls r_alloc_inhibit_buffer_relocation (1) around the whole operation, because Emacs's own relocating buffer allocator would otherwise move data out from under the serializer. Both are load-bearing only because Emacs still contains allocators built for the world in which the process image was the artifact.

unexec was deprecated in Emacs 27.1 and deleted in Emacs 31: commit 7ce34a3b (2024-12-12, Pip Cet) removed sheap.c, unexec.h and the eight per-platform unex*.c files, 4,666 lines in total; etc/NEWS.31 records it as "The traditional unexec dumper, deprecated since Emacs 27, has been removed." (etc/NEWS.31).

SBCL: a core file, mmap-ed, with a read-only space ​

save-lisp-and-die writes a core: "enough information to restart a Lisp process later in the same state … Only global state is preserved: the stack is unwound in the process." (SBCL manual). coreparse.c walks a directory of spaces (READ_ONLY, STATIC, PERMGEN, IMMOBILE_FIXEDOBJ, IMMOBILE_TEXT, DYNAMIC) and maps each one, and the mapping flags are the whole story (src/runtime/os-common.c):

c
/* src/runtime/os-common.c — load_core_bytes() */
int protection = 0, sharing = MAP_PRIVATE;
protection = (addr ? (is_readonly_space ? OS_VM_PROT_READ : OS_VM_PROT_ALL)
              : OS_VM_PROT_READ | OS_VM_PROT_WRITE);
if (is_readonly_space) sharing = MAP_SHARED;

Read-only space is MAP_SHARED | PROT_READ straight off the core file: file-backed, demand-paged, and shared between every process running that core. Everything else is MAP_PRIVATE. And what goes into read-only space is exactly the :purify argument's job — "If true (the default), then some objects in the restarted core will be memory-mapped as read-only. Among those objects are numeric vectors that were determined to be compile-time constants, and any immutable values according to the language specification such as symbol names." :purify is not a garbage-collection tuning knob; it is a page-sharing knob, and it is on by default.

ASLR does to SBCL what it did to Emacs. When the runtime is built with LISP_FEATURE_ASLR, coreparse.c throws away the core's preferred address before mapping and lets the OS choose (src/runtime/coreparse.c):

c
/* Try to map at address requested by the core file unless ASLR. */
#ifdef LISP_FEATURE_ASLR
    addr = 0;
#endif

and relocate_heap then adjusts every affected word, including each entry of the linkage space. Same problem, same answer, arrived at independently of Emacs.

CRIU: the image that refuses to contain the program ​

CRIU checkpoints an arbitrary process tree from outside. It parses /proc/$pid/smaps and /proc/$pid/map_files/, injects parasite code into the target so that the target's own address space can be read from inside it, and moves pages out with vmsplice(SPLICE_F_GIFT) into a pipe that the dumper splices into pages-$id.img (memory-dumping-and-restoring.md). The metadata is a directory of Protocol Buffers images — mm-$id.img holds an mm_entry with the auxv and a repeated vma_entry, and pagemap-$id.img holds the pagemap_entry records that say where each range's bytes are.

The decision that matters for this catalog is what CRIU refuses to copy:

"Unchanged File Pages: Read-only, file-backed pages (like library code) that have not been modified are not dumped. CRIU simply records the file and offset to re-map them during restoration." — memory-dumping-and-restoring.md

Restore re-mmaps those VMAs from the original files on the destination host, which is why a restored process keeps file-backed, shared, demand-paged text. It is also why the artifact is not closed over anything: it names libc.so.6 and trusts the destination to have the same one. The guard is a build-id check (validate-files-on-restore.md, --file-validation buildid, the default per criu(1)): at most BUILD_ID_MAP_SIZE = 1048576 bytes of each ELF are mapped to extract NT_GNU_BUILD_ID (criu/files-reg.c), and a mismatch aborts the restore. Emacs's fingerprint, SBCL's runtime-incompatibility rule, and CRIU's build-id are three independent inventions of the same constraint.

Restoring also reproduces, at the level of a whole process, the manoeuvre SELF's native.c performs for one program: the restorer must vacate its own address space while still executing. CRIU's answer is a freestanding PIE blob relocated into a hole that neither CRIU's temporary mappings nor the target's final layout occupies — restorer_get_vma_hint in criu/cr-restore.c walks the two VMA lists in lockstep to find it — after which the blob unmaps CRIU, rebuilds the target's mappings, and finishes with sigreturn off a hand-built signal frame (restorer-context.md, stages-of-restoring.md).

Erlang/OTP: mutate the running program, under a rule ​

ERTS never writes an image. Its equivalent of "the artifact carries its state" is that the node carries it, and code is replaced underneath it. The mechanism is a small MVCC scheme over the code tables (erts/emulator/beam/code_ix.h):

"The basic idea is to maintain several 'logical copies' of the code. These copies are identified by a global 'code index', an integer of 0, 1 or 2. … The current 'active' code index is used to access the current running code. The 'staging' code index is used by the process that performs a code change operation. When a code change operation completes successfully, the staging code index becomes the new active code index. The third code index is not explicitly used. It can be thought of as the 'previous active' or the 'next staging' index."

Three indices, one active, one staging, one quarantined until every scheduler has been observed to have moved past it — this is a commit protocol, and its purpose is stated as "allowing executing Erlang processes to access the code without any locks or other expensive memory barriers." Above it sits the two-version rule quoted earlier, and above that sits release_handler, which drives an upgrade from a relup file generated from per-application .appup files. A stateful upgrade is a defined dance (release_handling.md):

"Suspend the processes using the module … Ask them to transform the internal state format and switch to the new version of the module. Remove the old version. Resume the processes. This is called synchronized code replacement"

executed via sys:suspend/1,2, sys:change_code/4,5 and sys:resume/1,2, with the affected processes discovered by walking each application's supervision tree and reading the Modules field of every child specification. The documentation is candid that this leaks: "During release handling, non-affected processes continue normal execution. This can lead to time-outs or other problems. For example, new processes created in the time window between suspending processes using a certain module, and loading a new version of this module, can execute old code." A transaction with a defined isolation level, and the level is not serializable.

The part of that machinery which speaks directly to this catalog is what happens to the data the purged code owned. Compiled BEAM modules carry literal areas — constant terms shared by reference with every process that touched them — so purging a module's code cannot simply free them: an arbitrary number of process heaps may hold pointers in. ERTS's answer is a dedicated erts_literal_area_collector process running a protocol that reads like a distributed two-phase commit (erts/emulator/beam/beam_bif_load.c):

"The literal area collector process sends copy-literals requests to all processes in the system. Processes inspects their heap for literals in the area, if such are found do a literal-gc to make copies on the heap of all those literals, and then send replies to the literal area collector process. … When all processes has responded, the literal area collector process calls erts_internal:release_literal_area_switch() again in order to switch to the next area."

Between phases the collector suspends itself waiting on thread progress, twice, so that both the publication of the area and the subsequent change of the blocking counters are guaranteed visible to every scheduler; terminating processes bump a counter to hold the release open until they are done. This is a mark-and-sweep over a store that is being executed from, with no stop-the-world, and the catalog asks for exactly that in its closure cluster's open question. It exists, it ships, and it is roughly four hundred lines of C plus one system process.

Format identity and multiplicity ​

Multiplicity = 1, and deliberately so. Every artifact here is one format with one parse. A Squeak image is identified by a 4-byte format version at offset 0 (68021, 6521, and their byte-swapped forms — the reader tries both and sets swapBytes); an Emacs dump begins with the 16-byte literal DUMPEDGNUEMACS\0\0; an SBCL core begins with a core magic and a directory of spaces; a CRIU image file begins with a magic and holds length-prefixed protobufs. None of them is polyglot, none is prefix- or suffix-tolerant in the ZIP sense, and none wants to be: a heap image has to be read in full by exactly one program that already knows its layout.

The two places multiplicity does appear are instructive because they appear outside the image:

  • SBCL :executable t concatenates the runtime and the core into one file, and :executable :elf-object (marked experimental in the manual) wraps a core in a .o for further linking. This is a chimera in the concepts sense — runtime in front, core behind, neither reinterpreting the other's bytes — and it is the only route by which any artifact on this page becomes a binfmt_elf file the kernel will execute directly.
  • Erlang's escript is the ZIP story exactly. An escript may be source, a .beam file, or "an entire Erlang archive", and the header is optional: "you can make an archive file executable by prepending the file with the lines starting with #! and %%!" (erts/doc/references/escript_cmd.md). escript factorial.zip 5 runs a bare ZIP with no header at all. escript:extract/2 hands back the archive as a binary you can write out and open with zip:foldl/4 (lib/stdlib/src/escript.erl). That is prefix-tolerance exploited precisely as ZIP parasitism predicts — and note where it sits: in the distribution format, never in the image.

Emacs is the case where multiplicity was lost as a direct result of the ASLR retreat. An unexec-ed Emacs was one file that was simultaneously a valid ELF executable, the Lisp heap, and the output of its own execution. A pdumper Emacs is two files, one of which (emacs) is a perfectly ordinary PIE with nothing autological about it, and the other of which (.pdmp) is inert data keyed to that binary by a hash. The catalog's headline property was traded away for compatibility with the platform's memory model, and that trade is the page's central evidence.

Index anchoring and random access ​

Header-anchored, and then nothing. These formats put a locator at offset 0 and are otherwise designed to be consumed whole. There is no partial read, because a heap has no meaningful proper subset: a live object graph is closed under reference, so reading "just the part you need" is undecidable without first reading the pointers, which are the thing you were trying not to read.

The decoded header of a real dump makes the shape concrete. Parsing emacs-31c0351a….pdmp (Emacs 30.2, 17,719,896 bytes) against struct dump_header gives:

FieldValueMeaning
magicDUMPEDGNUEMACS14 bytes plus two NULs
fingerprint31c0351a…033132 bytes; also the file's own name
dump_relocs[EARLY_RELOCS]@14,216,384 × 591,692pointer fixups inside the dump
dump_relocs[LATE_RELOCS]@16,583,152 × 92after the load hooks run
dump_relocs[VERY_LATE_RELOCS]@16,583,520 × 6,056last, with the allocator available
object_starts@16,607,744 × 264,458one entry per dumped object, for conservative GC
emacs_relocs@17,665,576 × 3,395writes into the Emacs binary's data segment
discardable_start10,150,088end of the hot region
cold_start10,420,224start of the never-relocated region

The parse checks out arithmetically, which is worth stating because it is what makes the numbers usable: the relocation tables are 4 bytes per entry ((16,583,152 − 14,216,384) / 591,692 = 4.00), emacs_reloc is 16, and 17,665,576 + 3,395 × 16 = 17,719,896 — exactly the file size. So the layout is:

  • hot, 0 – 10,150,088 (9.68 MiB) — mapped MAP_PRIVATE, and the target of 591,692 relocations;
  • discardable, 10,150,088 – 10,420,224 (264 KiB) — copied into Emacs, then MADV_DONTNEED;
  • cold, 10,420,224 – 17,719,896 (6.96 MiB), of which 3.34 MiB is the relocation and object-start tables themselves (they live in the cold section by construction) and 3.62 MiB is cold Lisp data.

Random access is therefore possible in exactly one direction and nobody uses it: you could mmap the cold region and read a string without touching the hot region, but no tool does, because there is no index from name to offset — only from offset to relocation. Compare SELF, where the identical data is a b-tree and SELECT soname FROM ldd touches three pages of a 96 MiB file. A heap image is the purest illustration of thesis 1: it has an index (object_starts, sorted, binary-searched — the comment says so: "We need an explicit end indicator (as opposed to a special sentinel) so we can efficiently binary search over the relocation entries"), a string region, and a foreign-key discipline, all hand-rolled, none queryable.

CRIU is the exception and proves the rule. Its state is a directory, not a file: inventory.img, pstree.img, core-$pid.img, mm-$pid.img, pagemap-$pid.img, pages-$id.img, one per concern. That is out-of-band indexing taken to its conclusion, and it buys exactly what out-of-band indexing always buys — you can read one part without the others. crit x <dir> mems explores memory maps without decoding sockets; pagemap_blocks.total_payload_size exists specifically so a reader can "advance the read offset past a whole entry" in O(1) instead of summing per-block sizes (images/pagemap.proto). It also buys the standard failure mode: the image set is only meaningful next to the filesystem it was taken against.

Reflexivity and query surface ​

Reflexivity = 2, and the two sub-questions of the axis come apart harder here than anywhere else in the catalog.

Self-interrogation is maximal. A Smalltalk image is the limiting case: the compiler, the debugger, the inspector and the class library are objects in the heap being inspected, so thisContext, Object>>inspect and the browser are not tools pointed at the program, they are the program pointing at itself. Nothing in this survey — nothing in the catalog — scores higher on the "can it interrogate itself while running" half. Emacs is nearly as strong for the same reason (C-h f, describe-variable, and the byte-compiler are Lisp objects in the dumped heap), and SBCL likewise via describe, sb-introspect and the debugger. pdumper even exposes its own load as data: (pdumper-stats) returns an alist of dumped-with-pdumper, load-time and dump-file-name (doc/lispref/internals.texi).

Interrogation of the artifact at rest is close to absent, and this is the axis's other half failing. There is no readelf for a .pdmp, no sqlite3 for a .image, no general question-asking surface over a core file. The only ways to ask a Squeak image a question are to run it or to reimplement Spur's object format. The asymmetry is total: these are the most self-aware artifacts in the catalog while executing and the most opaque while sitting on disk. That is the precise gap sqlelf and SELF set out to close for ELF, and it is why a heap image never developed the tooling ecosystem a self-describing format gets for free.

Three partial exceptions are worth naming:

  • crit is a real query surface for a CRIU image, and the only one on this page. crit decode converts a binary image to JSON, crit show pretty-prints it, crit info summarizes, and crit x <dir> {ps,fds,mems} answers the three questions people actually ask (Documentation/crit.txt). It works because CRIU's images are protobufs with a published schema — that is thesis 2 doing its job. Crucially crit also has encode: JSON back to binary, so a checkpoint can be edited between dump and restore.
  • Erlang's runtime is queryable by design: code:all_loaded/0, code:which/1, erlang:check_process_code/2, sys:get_state/1. The release handler's own upgrade planning is a query — it discovers which processes use a module by walking supervision trees and reading child specifications.
  • pdumper's object_starts table is a genuine on-disk index of object identity, added not for tooling but because conservative GC needs pdumper_object_type to work on a pointer into the dump. An index that exists only to serve the collector, and that no user-facing tool reads, is a compact statement of where these formats put their effort.

Closure, dedup, and size model ​

Closure = 2 — designed-in, but bounded, and each system draws the boundary at a different place.

A Smalltalk image is maximally closed over everything in Smalltalk and not at all closed over the VM: the image is unrunnable without a matching interpreter, and the interpreter is a separately distributed native binary per platform. That split is the entire portability strategy — one image, N VMs — and it is thesis 5 in a system that predates the thesis by three decades. It is also incomplete, since the .changes and .sources files must travel alongside for the sources to be readable at all.

Emacs, post-pdumper, is closed over its Lisp but split across two files welded by a hash. SBCL with :executable t is the only configuration here that produces a genuinely single-file closed artifact — runtime plus heap — and it pays for it in size. CRIU is the anti-closure case: it deliberately does not carry the file-backed pages it references, and validates rather than contains.

Numbers ​

Measured on this machine, from the Nix store (emacs-30.2):

ArtifactBytesNote
bin/emacs-30.2 (PIE ELF)10,186,640ordinary executable, nothing dumped into it
emacs-31c0351a….pdmp17,719,896the Lisp heap plus relocation tables
— of which relocation/index tables3,503,51219.8% of the dump is metadata about the dump
— of which hot (relocated) region10,150,08857.3%, private-dirty after load
— of which cold (mappable) data3,796,16021.4%, the only part that can genuinely be shared
combined artifact27,906,536two files, one hash binding them

Two readings follow. First, the metadata tax is large and structural: nearly a fifth of an Emacs dump is relocation bookkeeping, which exists solely because the image may land at an arbitrary address. pdumper.c's own TODO list names the escape — "Preferred base address for relocation-free non-PIC startup" — and it is the one item on that list that ASLR forbids. Second, the shareable fraction is small. Only the cold region is untouched by relocation, and it is 21% of the dump.

Deduplication is where CRIU is far ahead of everyone, because it is the only system that expects to write the same image repeatedly. pagemap_entry.in_parent marks a page as identical to the parent snapshot's, detected via the kernel's soft-dirty bit during pre-dump, so iterative checkpoints store only deltas; --auto-dedup goes further and fallocate(FALLOC_FL_PUNCH_HOLE)s the image files as it reads them during restore, freeing disk blocks the instant their contents are in RAM (memory-images-deduplication.md). Compare content-addressed chunking, which solves the same problem with hashes instead of with a chain of parent images and a kernel bit; CRIU's version is cheaper and strictly less general, since the chain only dedups against its own ancestry.

Erlang's size model is the outlier because there is no image: an OTP release is a directory of .beam files and applications, and the live cost is bounded by the two-version rule — at most two copies of any module's code, plus its literal areas, at any instant. That is a size guarantee expressed as a concurrency rule, which no other system here attempts.

Mutability, dispatch, and trust ​

Mutability = 3, and this page is where the axis was invented. But the systems split three ways on who does the mutating, and the split predicts everything else about them.

The program rewrites its own file (Smalltalk, unexec). Squeak's snapshot writes the entire object memory back over the .image, including the state of every open window and the developer's half-finished edit; that is the normal working mode, not a recovery feature. unexec did the same thing to an ELF executable. Both are the full autological property, and both are the configurations that could not survive contact with a modern memory model.

A build step writes it; the program only reads (pdumper, save-lisp-and-die). Emacs's dump is generated in -batch and is thereafter immutable input. SBCL's core is written by a process that must then die ("It corrupts the current Lisp image enough that the current process needs to be killed afterwards" — the manual's own words, and the recommended workaround is to fork a child to do the saving). The self-mutation is gone; what remains is a fast start.

Nobody writes a file at all (Erlang). The mutation is to live memory, transactionally, with the three-index commit protocol and the two-version rule doing the work a filesystem would otherwise do.

mmap is the load-bearing constraint, five times over ​

This section is the page's contribution to thesis 4, and the evidence is unusually clean because five independent teams hit the same wall and their fixes converge:

SystemWhere the image landsShared between processes?Demand-paged?
Squeak/Spuranonymous mmap, then read into it, then swizzle every pointerNoNo
Emacs unexecthe ELF's own .data, mapped by binfmt_elfUntil first write — then COW, and it wrote a lotYes
Emacs pdumperMAP_PRIVATE file-backed, three regionsCold region yes; hot region noYes
SBCLMAP_SHARED for read-only space, MAP_PRIVATE elsewhereRead-only space yes — that is what :purify buysYes
CRIUoriginal files re-mmaped; anonymous VMAs refilled from pages-*.imgYes for file-backed — the point of not dumping themYes
SELFMAP_PRIVATE|MAP_ANONYMOUS, then memcpy out of b-tree pagesNoNo

The convergence is the finding: every system that recovered any sharing did it by carving out a region that is never relocated and mapping that region directly off the file. Emacs calls it the cold section and describes it in exactly those terms (src/pdumper.c):

"Start of the region that does not require relocations and that we expect never to be modified. This region can be memory-mapped directly from the backing dump file with the reasonable expectation of taking few copy-on-write faults."

SBCL calls it read-only space and gates it on :purify. CRIU calls it "unchanged file pages" and simply declines to dump them. Three names for one idea, invented separately, all of them a partition of the image into a relocated part and a mappable part — and in every case the mappable part is the minority. Emacs's is 21% of the dump. Squeak, which never partitioned, gets zero.

The numbers sharpen the point. The emacs-30.2 hot region is 10,150,088 bytes — 2,478 pages of 4 KiB — and carries 591,692 early relocations, roughly 239 relocations per page. There is no plausible distribution under which a meaningful number of those pages escape being written, so essentially the whole hot region is private-dirty in every Emacs process on the machine. Ten megabytes per process, unshared, is the price of ASLR compatibility, and it is invisible in start-up benchmarks because it shows up as RSS rather than latency — the same measurement trap the SELF page identifies and the measurement page owns.

WARNING

The 239-relocations-per-page figure is a density, derived from the decoded header of one dump, not a measurement of dirtied pages. Relocation entries are sorted by dump offset and could in principle cluster; nothing in the sources states their distribution, and the honest statement is "the mean is 239 per page, so a page escaping all of them would be a considerable outlier". The direct experiment is the one the measurement page specifies for SELF — PSS and USS sampled across N concurrent processes — and it has not been run for Emacs here. Any claim about how much of a pdumper image is actually shared should come from /proc/*/smaps_rollup, not from this table.

The counterfactual is what makes this thesis-4 evidence rather than an anecdote. unexec did preserve sharing and demand paging perfectly, because the artifact was an ELF file and binfmt_elf mapped it like any other. Emacs gave that up knowingly, and the reason given is not performance but that the technique "might get in the way of various memory-layout techniques used by modern systems to enhance security and privacy". The platform's threat model, not the format, is what killed the autological artifact. Any proposal of the "the executable is X" family — SELF included — is proposing to re-enter a fight the incumbents already lost, and the only two moves known to work are SBCL's (partition, and map the immutable part shared) and CRIU's (don't carry the pages at all; reference and validate them).

Dispatch ​

The dispatcher is the loader, in the narrow sense that the artifact is data and a specific runtime binary opens it. No image on this page is matched by the kernel on its magic; there is no binfmt_misc registration for .image, .pdmp or .core, and none of these projects has asked for one. The exceptions are the two configurations that stop being images: save-lisp-and-die :executable t produces an ELF the kernel executes directly, and an escript is dispatched by the shell, via #!/usr/bin/env escript. Compare SELF, which is a database that the kernel is taught to execute via binfmt_misc — precisely the move this tradition never made, and a plausible reason it never had to solve the identity problems that come with it.

Trust ​

The trust story is dominated by one fact: an image is a heap, and a heap contains function pointers. Loading a hostile .pdmp or .core into a matching runtime is arbitrary code execution by construction, with no parsing bug required — the pointers are the payload. The mitigations are all identity checks rather than validation:

  • Emacs compares a SHA-256-class fingerprint of the exact temacs and refuses on mismatch; the check is a build-identity check, not an authenticity check, and there is no signature anywhere in the format.
  • SBCL declares runtime/core compatibility to be exact and time-sensitive: even same-source rebuilds are incompatible.
  • CRIU validates every file-backed mapping's build-id before restoring, and the documentation is explicit that this is a security control: "This prevents 'library injection' scenarios where an attacker might try to force a restored process to run against malicious versions of its original dependencies." (validate-files-on-restore.md).

CRIU additionally carries a real privilege surface. Checkpointing needs CAP_SYS_ADMIN or, since Linux 5.9, CAP_CHECKPOINT_RESTORE, plus CAP_SYS_PTRACE or a permissive yama/ptrace_scope, and the manual is blunt that "Running criu as non-root has many limitations" (criu(1)). A tool that can read any process's memory and reconstruct it with restored credentials is a capability that belongs in the threat-model page alongside binfmt_misc registration.

Signing is unsolved here in the same way it is unsolved for SELF, and for a sharper reason: a Squeak image's bytes change every time the developer saves, so any whole-file attestation is invalidated by normal use. Emacs sidestepped it by making the mutable half of the artifact disappear — a .pdmp is written once by a build and can be signed like any other build output, which is a real if unglamorous argument for the split. See embedded provenance.

The Lisp machines and Genera, briefly ​

The maximal case, and the one the current wave under-cites. The Symbolics 3600's architecture put type tags in hardware — "Tagged architecture: run-time data-type checking with no overhead" (3600 Technical Summary, February 1983) — with a demand-paged virtual memory in which the distinction between "program", "data" and "the running system" simply did not exist as a representational matter. Everything was an object with a tag; the compiler, the editor, the network stack and the paging system were inspectable objects in one address space; and the persistent artifact was a world load, the whole of that address space written out. Genera is the endpoint of the trajectory this page describes: maximal reflexivity, maximal mutability, an artifact that is unambiguously the program and its state and its own development environment.

It did not survive, and the reasons are the ones this page has been accumulating. The tagged-memory requirement made the system a hardware proposition at exactly the moment stock RISC and x86 parts were pulling away on price and performance, so the substrate stopped being available. The single-address-space model — one protection domain, everything mutable, everything reachable — is orthogonal to the process isolation, W^X, and address-space randomization that commodity operating systems spent the following two decades making mandatory; it is the same collision that killed unexec, arriving earlier and at a larger scale. And the world load has every property catalogued above: no partial read, no signature, no sharing between world loads, and no query surface except by booting it.

The lineage's real successor is not a file format but an architecture argument, and it belongs to single-level store — IBM i/OS400, Multics, and the commercial system that actually shipped "the OS is a database". This page's contribution is the narrower observation that the failure mode was identical at both scales: the reflexive artifact and the platform's memory protection model are in direct competition, and the platform wins.

Strengths ​

  • Start-up cost is paid once. The mechanical justification is real and undefeated. Emacs, SBCL and Squeak all start in milliseconds with tens of megabytes of pre-built object graph available.
  • Self-interrogation at runtime is unmatched. A Smalltalk image is the strongest example in the entire catalog of a program that can ask questions about itself, because the questioner and the subject are the same heap.
  • The image is the unit of reproducibility. Whatever state you saved is the state you get. No configuration drift, no re-derivation, no "works on my machine" from a differently ordered load path.
  • Erlang's two-version rule is a genuine answer to a hard question. Concurrent mutation of the code a running system is executing, with bounded memory, no locks on the read path, and a defined purge semantics. The catalog asks what a concurrent GC over a store somebody is executing from looks like; beam_bif_load.c contains one, complete with a dedicated erts_literal_area_collector process, copy-literals requests to every process in the system, and thread-progress barriers.
  • CRIU works on programs that were not designed for it. Every other system here required its runtime to be built around the image. CRIU checkpoints arbitrary unmodified processes, and preserves fork-derived COW sharing across a restore by re-forking and comparing rather than by filling each VMA independently (copy-on-write-memory.md).
  • crit proves a memory image can have a query surface. Binary → JSON → binary, because the schema is published. The only artifact on this page you can inspect and edit without running it.

Weaknesses ​

  • Page sharing and demand paging are lost or heavily degraded in every system that puts the heap in the artifact. Squeak loses both outright; Emacs keeps demand paging and loses sharing for 57% of its dump; SBCL keeps both only for the :purify-ed read-only space.
  • The artifact is welded to one exact binary. Emacs's fingerprint, SBCL's "even runtimes built from the same sources at different times are incompatible", CRIU's build-id validation. An image is not portable; a pair is.
  • Nothing at rest is queryable. No index from name to offset, no schema, no tooling that is not the runtime itself. This is the gap that motivated sqlelf and SELF.
  • Relocation metadata is a fifth of the artifact in the one dump measured here, and exists only because the load address is not knowable in advance.
  • Signing is incompatible with the working model wherever the program writes its own image, and this is the reason Emacs's split is defensible rather than merely a retreat.
  • The Smalltalk image is not one file. .image plus .changes plus .sources, same basename, same directory, or method sources decompile to t1, t2.
  • Erlang's synchronized code replacement is not serializable, and the documentation says so: processes created in the window between suspension and load run old code.
  • CRIU needs CAP_CHECKPOINT_RESTORE at minimum, and cannot follow a process onto a host whose libraries differ — the build-id check turns a subtle corruption into a hard failure, which is the right trade and still a hard failure.
  • unexec's removal deleted a capability, not just an implementation. Nothing in Emacs 31 can turn a running Emacs into an executable, and the pdumper replacement is -batch-only by explicit error.

Key design decisions and trade-offs ​

DecisionRationaleTrade-off
Emacs: replace unexec with an explicit heap serializerASLR, PIE and modern malloc made "re-emit the running process" unmaintainable; the manual frames it as not obstructing the platform's security techniquesThe artifact stops being autological: two files, plus a fingerprint to bind them, plus 3.3 MiB of relocation tables
Emacs: three regions (hot / discardable / cold)Only the never-relocated region can be mapped off the file with few COW faults — sharing is recovered where it can beThe mappable region is 21% of the dump; the other 79% is private-dirty per process
Emacs: fingerprint the exact temacs and refuse on mismatchA dump is raw pointers into a specific binary's data segment; a near-match is a silent corruptionThe dump has no independent existence, cannot be shared between builds, and is not signable as a standalone artifact
Emacs: dumping is -batch-onlyA dump taken from a live session would capture arbitrary user state and running threadsThe user-facing capability that made unexec interesting is gone; pdumper is a build tool
SBCL: :purify defaults to true, read-only space is MAP_SHAREDImmutable objects (constant vectors, symbol names) can be mapped straight off the core and shared across processesRequires classifying the heap into mutable and immutable at save time; anything misclassified is a correctness bug
SBCL: cores are runtime-specific, deliberately, including across rebuildsThe alternative is a versioned external representation nobody was motivated to build ("no good quick fixes")Zero portability; distributing a core means distributing the runtime it was made with
Squeak: read the heap into anonymous memory and swizzle every pointerSimplest possible loader; works identically on every platform the VM is ported toNo file backing at all, so no sharing, no demand paging, and load time proportional to heap size
Squeak: image portable across VMs, VM native per platformOne artifact, N interpreters — portability moved to the substrate three decades before the catalog named the patternThe image alone is inert; and .changes/.sources must travel with it for sources to survive
CRIU: do not dump clean file-backed pagesThey can be re-mapped from the original files on restore, preserving sharing and demand paging and shrinking the imageThe image is not closed: restoring requires the same files on the destination, enforced only by a build-id check
CRIU: images are schema'd protobufs in a directoryGives crit decode/encode, partial reads, and per-concern evolution — self-description buying toolingNot one file; the "artifact" is a directory whose members are only meaningful together
CRIU: in_parent + soft-dirty + --auto-dedup hole punchingIterative checkpoints store deltas, and restore frees blocks as it consumes themDedup is only against the image's own ancestry chain, never content-global
Erlang: exactly two live versions of a module, third load purgesBounds memory and makes the lifetime of "old code" decidable; processes migrate by making a fully qualified callA process that never makes a qualified call is killed at the third load; the rule is a hard deadline, not a negotiation
Erlang: three code indices, one active, one staging, one quarantinedReaders need no locks or barriers; the switch is a single publicationA code change must wait for thread progress before a released index can be reused
Erlang: upgrades described by .appup/relup and executed by release_handlerState migration (code_change/3) is explicit and scriptable, with rollback by reboot into the old releaseNot serializable — processes created mid-upgrade can run old code; the docs recommend small, backward-compatible steps

Sources ​