sparkles:dmd-fmt — Testing & Decision Documentation
Status: specified, not implemented · Date: 2026-08-25 · Scope: how the formatter's behaviour is pinned by tests and published to users · Requirement IDs: TST*
The decision inventory catalogues 160 formatting decisions, of which 132 are scored Adopt/Adapt/Opt-in for D, several are configurable, and D9 adds a second tier of opt-in rewrites on top. That is far past the size where ad-hoc unit tests hold: without a plan, the suite becomes a pile of string literals nobody can audit for coverage, and the user-facing documentation drifts from the implementation within one release.
This spec fixes one artifact that serves three jobs — test fixture, rule specification, and the published documentation page — plus the gates that keep the three from separating. The survey behind it is How formatters test themselves; the two techniques it takes wholesale are dart_style's multi-case fixture files and rustfmt's Configurations.md-must-cover-every-option gate.
The artifact is a markdown page with VitePress code groups. There is no bespoke fixture syntax and no generation step: the page a user reads is the file the test runner executes. The repo already has the infrastructure this needs — ::: code-group blocks with [Title]-labelled fences are used across the docs, <!-- … --> directives already carry machine-readable metadata for the markdown example verifier, and the sidebar/staleness gates already run over docs/.
The artifact: a page that is also a fixture
TST1 — Fixture format: a documentation page
A fixture file is a markdown page. Each decision is a section; each section carries one or more cases, and a case is a ::: code-group whose first fence is the input and whose remaining fences are the expectations:
### Blank-line runs collapse {#p19}
<!-- fmt id=P19 -->
A run of blank lines collapses to at most `max_blank_lines` (default 2). Blank lines are never
inserted, and never removed entirely — the author's paragraphing survives, bounded.
::: code-group
```d [Before]
void a() {}
void b() {}
```
```d [After]
void a() {}
void b() {}
```
:::<!-- fmt … -->carries the machine-readable metadata and renders as nothing. Keys:id=P19(one or more decision IDs),width=60(print width for this case),variants=<key>,verifier=<name>(TST11),expect=fail(TST10). The precedent is the repo's existing<!-- md-example-expected -->and<!-- md-example-skip -->directives.- The first fence in the group is the input, conventionally titled
[Before]. - Every later fence is an expectation.
[After]means "with this section's configuration"; a title of the form[brace_style=allman]means "with these keys merged over it", and a case with two or more such fences is a variant case — the reader sees one tab per option value, and the runner checks each. - Prose is free. Everything outside the code groups is ordinary documentation, written for a reader rather than for the harness.
Print width deserves a note. dart_style and prettier both draw a visible column ruler in their fixtures, which is genuinely useful and does not survive contact with a syntax-highlighted code fence. The replacement is honesty in prose: a case whose behaviour depends on the wrap column states it (width=60 in the directive, "at a print width of 60" in the sentence), and the inputs are written short enough that the reader can see the column without counting to 120.
Validated, not just proposed. The format above is what docs/libs/dmd-fmt/reference/decisions/layout.md already uses, and a ~130-line prototype parser calling formatText directly runs its seven expectations against the real formatter. Nothing in TST1 needs a markdown library, a preprocessor, or a change to the docs build: the page renders as-is with the ::: code-group support this tree already uses everywhere.
The prose above each group is the rule statement. It is what a user reads and what a reviewer checks the behaviour against; there is no separate test name to drift from it.
TST16 — The input must be misformatted
A case's Before block must differ from its After block. A fixture whose input is already its own output demonstrates nothing: the reader sees two identical blocks and has to take the prose's word for what the rule does — which is the drift an executed page exists to prevent, reintroduced in the one place that was supposed to close it. Break the input on purpose: wrong indentation, joined lines, doubled spaces, a blank run where none belongs.
Published cases carry a second obligation: misformat them the way a person would. A reader learns more from an input that looks like a plausible mistake — an attribute indented as though it were a continuation, a nested body under-indented — than from randomised noise, and the rule reads as something that helps rather than something that tidies.
The exception is a case whose subject is that nothing changes: a verbatim region, a suppression range, an already-canonical form the formatter must not touch. Those declare fixedpoint in the directive. That is a claim about the rule, not an excuse for the fixture, and the runner enforces the distinction — an unmarked no-op case fails, naming both remedies.
The check is cheap and it caught nineteen existing cases the day it was written, four of them on the published page.
TST2 — Decision tags and the coverage gate
Every case carries zero or more decision IDs in its <!-- fmt id=… --> directive, drawn from the inventory. Two gates run in CI:
- Every decision scored Adopt, Adapt or Opt-in has at least one case. A decision with no case fails the build, naming the ID — rustfmt's
does not have a configuration guidepanic, applied to behaviour rather than to options. - Every configuration key has at least one
variants=<key>case, covering every value the key accepts — so the docs cannot document an option without showing what it does.
Decisions scored Have, N/A, Reject, Codemod or Oracle are exempt, and the exemption list is data: the gate reads the inventory's verdict column, so re-scoring a decision automatically demands a fixture.
Coverage that does not exist yet is a ledger, not a silence. 127 decisions oblige a case and most are not implemented, so libs/dmd-fmt/test/cases/uncovered.txt lists the ones still waiting for one — and the gate reads it in both directions: a decision that is neither covered nor listed fails, and a listed decision that has gained a case fails too, naming the line to delete. The count only moves down, and it moves down in a diff. A third check catches the typo in the other direction: a case whose id= names neither an inventory row nor a decision the record defines (M4, D5) is dangling.
Where the gate runs, and why not in ci. An earlier draft of this spec promised ci --check-fmt-decisions. It lives in libs/dmd-fmt instead, as checkDecisionCoverage plus the unittest that asserts on it, because the check needs the case parser and giving apps/ci a dependency on sparkles:dmd-fmt would drag the DMD frontend into the CI helper's build closure — paid on every CI run, for a check dub test :dmd-fmt (and so ci --test) already performs.
Deferred: gate 2 above, the per-value variant coverage. It needs a machine-readable list of each key's admissible values, which FormatConfig does not carry yet; until it does, a variants= case is checked for naming a real key and nothing more.
TST3 — Two locations, one format
Fixtures torture the implementation; documentation should not. Rather than flag cases, the distinction is which file a case lives in — same syntax, same parser, same blessing path:
| Location | Published | Contents |
|---|---|---|
docs/libs/dmd-fmt/reference/decisions/<area>.md | ✅ yes | The presentable case for each decision: plausible code, a sentence a user wants. |
libs/dmd-fmt/test/cases/<area>.md | ❌ no | Everything hostile: pathological nesting, comment placement, degenerate input. |
The unpublished tree needs no exclusion list: VitePress builds docs/ and nothing else, so a markdown file under libs/ is never a page. That is the reason to put the hostile cases there rather than under docs/ behind a srcExclude entry — one fewer list to keep in step, and the location itself says which tree a case belongs to. Every decision needs ≥1 case anywhere; every user-visible decision needs ≥1 case in the published tree.
Running a case
TST4 — The per-case pipeline
For each case, for each variant, in order — every step a separate failure naming the page, the heading, and the line of the offending fence:
- Format the input with the merged configuration; compare byte-for-byte with the expectation.
- Verify the formatting with
verifyFormat— tier-3 token equality plus the DDoc-attachment check for the layout tier; for a rewrite-tier case, the rewrite's own verifier (TST11) instead. - Converge —
checkConvergence, bounded iteration to a fixed point with per-step verification. - Idempotence on the golden —
format(expected) == expected. This is the check gofmt runs as a second pass, and it catches the class of bug where the engine produces a stable-looking output it would not itself produce.
TST5 — Blessing
SPARKLES_UPDATE_GOLDENS=1 dub test :dmd-fmt rewrites the expectation fences in place, in the markdown, preserving prose, directives, fence titles and group structure. The repo already uses this environment variable for the markdown golden suite; reusing it is deliberate.
Blessing is loud: it names every expectation it rewrote, on stderr. A silent rewrite is how a stale binary enshrines its own bug in a committed golden — which happened once during this work, and was caught only because the file was already under version control.
The update path never touches an input fence, never reorders or deletes a case, and never edits prose — so a blessed diff reads as "what changed in the output", and a prose statement that the new output contradicts stays in the diff for a reviewer to notice. That is a feature: the rule sentence and the golden are in the same diff hunk.
Beyond fixtures
TST6 — The perturbation oracle
The layout tier is defined to normalize horizontal whitespace, recompute indentation and collapse blank runs. That definition is an equality oracle needing no expected file: perturb only those dimensions and the output must be byte-identical.
perturb(s): randomize leading indentation width and style,
append trailing spaces to random lines,
lengthen blank-line runs,
swap space/tab in leading whitespace
assert: format(perturb(s)) == format(s) for every corpus fileThis is the layout-preserving formatter's replacement for clang-format's messUp, which is unavailable to us because it destroys the author's line breaks — which are input, not noise. Seeded and deterministic, so a failure reproduces; run over the whole corpus, so coverage comes free of authoring cost.
TST7 — Robustness inputs
Inputs that carry no expected output, only the requirement that the formatter does not crash, converges, and passes its verifier: messUp-style whole-construct-on-one-line collapse, truncation at every token boundary of a corpus file, unbalanced brackets, unterminated literals, and files that do not parse at all (the bracket-only degraded path). Failures are recorded as crash reproductions, never as expected output.
TST8 — Corpus tests
Already delivered by M8 and retained: the repo's own library trees, the pinned frontend's expressionsem.d (~20 kLOC), and any corpus reachable through $SPARKLES_FLAKE_INPUT_*. The stability triad runs over all of them. TST6's perturbation and TST7's robustness inputs are driven from the same corpus.
TST9 — The dfmt differential, as disappearing snapshots
M8's similarity index (mean 0.927, tripwire floor 0.85) is a number, and a number is hard to review. Alongside it, adopt ruff's model: for each corpus file, if dmd-fmt's output equals dfmt's, no artifact exists; where they differ, a snapshot holding the input, a unified diff labelled dfmt/dmd-fmt, and both outputs is written to libs/dmd-fmt/test/differential/. Convergence then shows up in review as files disappearing, and every divergence is a reviewable diff instead of a decimal place. The index stays as the ratchet; the snapshots are what a human reads.
TST10 — The known-failure ratchet
Cases that are expected to fail — an unimplemented decision whose case is already written, a convergence failure under a specific option — are marked expect=fail in the directive and listed in the registry, not left as a comment. A published page never carries one: a expect=fail case lives in the unpublished tree, because a documentation page must not show output the formatter does not produce. The harness asserts they still fail, and reports "…now passes, remove it from the registry" when they do not. Prettier's unstableTests list, and the reason its suite cannot silently loosen.
TST11 — Rewrite-tier verifiers
D9 requires each opt-in rewrite to ship with its own verifier. A rewrite-tier case must therefore declare it: verifier=literal_forms in the case's directive selects the check TST4 step 2 runs instead of tier-3 equality. The verifier registry is part of the rewrite's implementation, and a rewrite with no registered verifier cannot have fixtures — which is the mechanism that keeps the D9 rule from being aspirational.
Publishing the decisions
TST12 — No generator, two gates
Because the published page is the fixture, there is nothing to generate and nothing to keep in sync. What remains is two gates, both ci subcommands in the shape of --check-docs-sidebar:
- The coverage gate — every decision scored Adopt/Adapt/Opt-in has a case or a ledger entry, and every case's
id=names a real decision (TST2). Coverage in both directions, and it runs insidedub test :dmd-fmtrather than as acisubcommand, for the reason TST2 records. - The runner itself is the staleness gate.
dub test :dmd-fmtexecutes the published pages; a documented example that stops being true fails the build. rustfmt'sConfigurations.mdguarantee, obtained without rustfmt's markdown-scraping test, because the page is the fixture rather than a copy of one.
This is the whole argument for the markdown format over a bespoke one: a generated page can be stale, and a page that is executed cannot.
TST13 — Page shape
One page per inventory area (§A…§J), each decision an ### section with an explicit anchor ({#p19}) so /libs/dmd-fmt/reference/decisions/expressions#p90 is a durable deep link. A section carries the rule sentence, the option and default when the decision is configurable, a link to the inventory row for provenance, and its case.
Configurable decisions render as tabs, not as prose. A variants= case becomes one tab per option value, so the effect of brace_style is one click apart:
<!-- fmt id=P33 variants=brace_style -->
::: code-group
```d [Before]
void f() { return; }
```
```d [brace_style=allman]
void f()
{
return;
}
```
```d [brace_style=otbs]
void f() {
return;
}
```
:::No theme component is needed — ::: code-group is stock VitePress and already used across this docs tree. An index page carries the full 160-row table (ID, area, rule, verdict, option, default), which is the map users navigate before the per-area pages.
TST15 — Failure reports read like the code they are about
A failing case prints three panes — the expectation, what the formatter produced, then the diff between them — each in its own titled box (sparkles:ui's drawBox).
Every box is the same width, the widest any of the three needs, whether that comes from a line of code, a title or a footer path. Three panes at three widths is three left edges and three right edges for the eye to track, and the whole point of the report is comparing two near-identical texts: two columns of indentation has to be the most obvious difference on screen, not the second-most after a ragged frame. Equal widths also make expectation and actual line up character-for-character, so a change of length shows as one box's text running past where the other's stopped.
Equal height too, for the pair that shares a row: the taller of expectation and actual sets it and the shorter is padded, so both boxes close on the same line. The pair is read as one picture, and two bottom borders at different heights leave the eye resolving which one it just crossed. The padding comes from hue --height, so those rows are painted like the rest — padding the box ourselves would leave a blank strip inside a filled pane. The diff box is excluded: it is under them, not beside them, and stretching it to match would only add empty rows to the pane that already says the most.
Two columns, never three. Expectation and actual go side by side, because they are the pair being compared; the diff goes underneath at the same width. The diff is a different kind of thing — it is about the pair rather than a member of it — and reading it wants the eye moving down a column of -/+ markers, not across a third pane whose rows do not correspond to the two beside it. Below 2 × boxWidth + gap there is no room for even that, and all three stack.
Both sides are also written to disk, under .result/dmd-fmt-cases/ at the repository root (.result/ is already git-ignored), named after the case that produced them — declarations-9-P22-After.expected.d and .actual.d. Each box's footer names its file relative to the working directory. A report is a snapshot; the artifacts are the thing itself, and a whitespace defect is often faster to settle with diff, cmp, xxd or an editor than by reading two columns of near-identical D.
The directory is cleared once at the top of a sweep, not per case: what survives a run is exactly what that run produced. A stale artifact is worse than no artifact, because it looks current — the same failure mode as the stale golden that motivated loud blessing in TST5.
A footer is as wide as the path it names, which is usually wider than the D above it, so drawing the paths in the borders can widen every box past what the terminal holds. When it would, the panes are drawn bare and the paths go on their own lines underneath — the same information, and the columns kept. The layout picks whichever leaves more panes side by side, preferring footers when the outcome is equal.
When hue is on PATH and the output takes colour, all three panes go through it. The formatter's own repository ships a syntax-highlighting viewer and a diff view; a formatting failure is exactly the thing they are good at, and reading one as an undifferentiated wall of text is a self-inflicted wound.
Five hue flags make it usable as a pane rather than as a page:
| Flag | Why |
|---|---|
--diff-show-formatting | A formatter's diff is nearly always whitespace-only, and hue folds such a hunk to a one-line badge — right for review, where nobody wants to read re-indentation, and wrong here, where the whitespace is the subject. A one-shot render has no keystroke to expand the fold with. |
--no-diff-chrome | The file header, the @@ bands and the ⋯ N unchanged lines trailers name a file and a region the box's own title already names, in a report where both sides are two temporary files. |
--no-line-numbers | Same: the gutter numbers lines of a scratch file. |
--width | A pipe has no size to ask for, so hue would otherwise lay out at its 80-column fallback no matter how wide the pane it is going into is. |
--diff-context=<lines> | Passed the file's own line count, so the diff pane spans exactly what the other two panes span. hue's default window is three lines, and with the bands suppressed a truncated pane is indistinguishable from a complete one — the reason this flag is not optional here. |
All three panes render at --background=full — every cell painted, the theme's page colour behind the rest. That became the right answer once the panes were framed: a box gives the block an edge, so a filled interior reads as an editor pane, where unframed the same fill had nothing to stop it and no shape to be.
Mind that flag. A diff's row tints and its delta-style intra-line emphasis are backgrounds — four of them: added row, removed row, added emphasis, removed emphasis. They are span backgrounds, so they survive full and spans alike (the tint counts are identical either way), but no-background means BgEmit.none, foreground only, and erases them outright. That is how the diff pane first shipped monochrome, distinguishable from the code panes only by its -/+ markers. spans is the third option: it paints what a span asked for and nothing else, which suits a diff but leaves a code pane with the page colour on its indentation and nowhere else.
Everything degrades in one step: no hue, no colour, an older hue that rejects one of the flags, or any error invoking it falls back to plain text with a unified diff computed by the same sparkles:diff engine the edit emitter uses. The box-and-column layout is unchanged either way — only the pane bodies lose their colour. A report that cannot be produced is worse than a plain one.
TST14 — What lives outside the harness
Prose, entirely. The harness owns the fences inside ::: code-group blocks that carry an <!-- fmt … --> directive; every other byte of the page — orientation, rationale, links, the index table — is hand-written and never rewritten. A code block that is not part of a marked group is ordinary documentation, ignored by the runner (and, since it carries no dub recipe, already ignored by the existing markdown example verifier).
Layout
libs/dmd-fmt/
├── src/sparkles/dmd_fmt/
│ └── cases.d # markdown case parser + runner + blesser (TST1, TST4, TST5)
└── test/
├── cases/<area>.md # unpublished, hostile cases (TST3)
├── differential/*.snap # dfmt divergences only (TST9)
└── known-failures.txt # the ratchet registry (TST10)
apps/ci/
└── src/fmt_decisions.d # --check-fmt-decisions (TST12)
docs/libs/dmd-fmt/
├── index.md
└── reference/decisions/
├── index.md # the 160-row map
└── <area>.md # published cases — read as docs, executed as testsDelivery
| Phase | Contents | Gate |
|---|---|---|
| P1 | TST1/TST4/TST5 — the markdown case parser, the runner, blessing | The existing golden tests move into .md cases |
| P2 | TST2/TST3/TST12 — decision ids, both coverage gates, the two trees | Every shipped decision has a case |
| P3 | TST13 — the published area pages and the index | The decision pages ship with the docs site |
| P4 | TST6/TST7 — perturbation oracle and robustness inputs | Clean over the whole corpus |
| P5 | TST9/TST10 — differential snapshots, the known-failure registry | The similarity index gets a reviewable companion |
| P6 | TST11 — rewrite verifier registry | Lands with the first opt-in rewrite (P154) |
P1–P3 are the sequence that matters: they are what turn 160 catalogued decisions into 160 executed, published, non-rotting statements about behaviour. P4–P6 are depth.
Traceability
- Research: How formatters test themselves · Verification · The decision inventory
- Spec: the decision record (
D8verifier,D9two-tier scope) · the codemod roadmap - Code:
libs/dmd-fmt/src/sparkles/dmd_fmt/verify.d(the checks TST4 composes),differential.d(TST8/TST9)