The D parsing landscape

The survey's inward turn: what the D ecosystem itself offers for parsing, mapped onto the taxonomy the cross-ecosystem deep-dives established, and read against Sparkles' allocation-conscious (@nogc/@safe) needs. Where the other pages ask "how does Rust/Haskell/C++ do it," this one asks "what can a D project reach for today, and what is missing" — the evidence base for the Sparkles parsing proposal. Every project below is grounded against a locally pinned checkout under $REPOS/dlang/ ($REPOS = /home/petar/code/repos; per-claim verification lives in the survey's internal grounding tree, which the published pages do not link).

Last reviewed: July 3, 2026

---

The landscape at a glance

D's parsing story has four strong pillars and one conspicuous gap. The pillars: a compile-time PEG generator (Pegged), a production hand-written front-end for D itself (three of them — libdparse, dmd, sdc), a world-class @nogc/SIMD serialization stack (mir-ion/asdf), and solid range-based structured-data parsers (dxml, D-YAML, sdlite). The gap: no modern, maintained, @nogc zero-copy ordered-choice combinator (nom/winnow style) and no recovering parser (chumsky-style) — the two designs the survey identifies as the Sparkles design center.

Project	Category (survey link)	Grammar / approach	Compile-time?	@nogc / zero-copy	Maturity (dub score)
Pegged	PEG generator (cf. pest)	PEG string → parser via mixin; ParseTree	Yes (CTFE)	✗ (GC ParseTree)	Flagship, 4.1
libdparse	Hand-written RD + lexer-gen	vendored std.experimental.lexer + RD → AST	tables at CT	perf-tuned (pools), not @nogc	Backbone, 4.9
dmd	Hand-written RD	reference D lexer + RD parser → AST	no	core.stdc/betterC-leaning	Reference
sdc	Hand-written RD	independent RD over a TokenRange; ambiguous.d	no	perf-oriented	Independent
pry	Parser combinator	Stream-over-ranges combinators; TLV	CT-specialized	Stream/zero-copy-leaning	Stale-ish, 2.1
mir-ion	SIMD / data-parallel	Ion/JSON/YAML/MsgPack (de)serialization engine	CT introspection	@nogc, SIMD, zero-copy	Active, 3.3
asdf	SIMD / data-parallel	fast JSON + compact IR; SSE4.2	CT introspection	cache-oriented, low-alloc	Active, 4.0
JSONiopipe	Streaming (Angstrom-like)	pull tokenizer over an iopipe char stream	no	streaming, no required DOM	Niche
std.json	Built-in DOM (baseline)	GC JSONValue DOM	no	✗ (GC; RED at-scale warning)	Stdlib
dxml	Pull (StAX)	range-based StAX pull + optional DOM	no	@safe, slicing, some @nogc	Active, 4.6
D-YAML	Hand-written (spec-heavy)	PyYAML-derived YAML 1.1 parser	no	slicing input reuse	Active, 4.9
sdlite	Range-based (config)	SDLang parser/generator; pool allocator	no	pooled GC, low overhead	Active, 3.8

_{dub scores are the registry's 0–5 popularity/quality signal at review time; "compile-time?" distinguishes parsing at CT (Pegged) from tables/reflection generated at CT (libdparse, mir). Per-cell sources are recorded in the survey's internal grounding tree.}

---

Compile-time PEG generation — Pegged

Pegged (Philippe Sigaud; Boost) is D's flagship parser generator and the ecosystem's headline demonstration of metaprogramming as a parser generator — the D answer to pest/ANTLR, but with a twist no runtime generator has: it can run the parser at compile time. You write a PEG as a string, pass it to grammar, and mixin the result:

import pegged.grammar;

mixin(grammar(`
Arithmetic:
    Term     < Factor (Add / Sub)*
    Add      < "+" Factor
    Factor   < Primary (Mul / Div)*
    Primary  < Parens / Number / Variable
    Number   < ~([0-9]+)
    Variable <- identifier
`));

The generated Arithmetic parser then works both ways (README.md):

enum parseTree1 = Arithmetic("1 + 2 - (3*x-5)*6");   // Parsing at compile-time
auto parseTree2 = Arithmetic(" 0 + 123 - 456 ");     // ...and at runtime too

It implements the full PEG operator set (ordered choice /, */+/?, syntactic predicates &/!), supports semantic actions and a dynamic/ runtime-grammar variant, and generates packrat-style memoizing recursive-descent parsers. The cost is the one the PEG/packrat theory predicts and that matters for Sparkles: the output is a GC-allocated ParseTree — Pegged is not @nogc (the latest commit is literally "use GC allocated slice instead of always alloca on stack"). It is the right tool for a throwaway DSL or a compile-time grammar, the wrong tool for an allocation-bounded hot path. The historical alternative, ctpg (youkei's Compile-Time Parser Generator), predates Pegged with the same CTFE idea but is effectively abandoned.

---

The D-language front-ends — three hand-written recursive-descent parsers

The most instructive data point in the D landscape is that the D language itself is parsed, in production, by three independent hand-written recursive-descent parsers — not one of them generated. This is the same lesson the top-down page draws from GCC/Clang/rustc: nobody generates a production language parser.

• libdparse (dlang-community; dub score 4.9, the highest in the space) — "Library for lexing and parsing D source code" (README.md). It is the ecosystem's backbone: DCD, D-Scanner, dfmt, dfix all parse D through it. Two-stage: a lexer built on a compile-time lexer generator feeding a hand-written RD parser (src/dparse/parser.d) that builds an AST. Its lexer is where std.experimental.lexer actually lives — the generator was removed from Phobos and the canonical copy is now vendored at src/std/experimental/lexer.d, documented as "a range-based compile-time lexer generator" driven by staticTokens/dynamicTokens/tokenHandlers declarations. The lexer uses SSE4.2 (core.cpuid : sse42) and a rollback/stack-buffer allocator for speed — perf-conscious, though not a blanket @nogc.
• dmd (Walter Bright / D Language Foundation; Boost) — the reference implementation. compiler/src/dmd/lexer.d "converts source code into lexical tokens" and compiler/src/dmd/parse.d "takes a token stream from the lexer, and parses it into an abstract syntax tree" (spec-linked to dlang.org/spec/{lex,grammar}.html). It leans on core.stdc and is written in a betterC-friendly style — the authoritative D grammar, hand-written throughout.
• sdc (Amaury "deadalnix" Séchet; MIT; built on libd) — an independent from-scratch D compiler. Its parser (src/d/parser/) is RD over a TokenRange, and its standout file ambiguous.d confronts head-on the thing that makes D genuinely hard to parse — the type/expression/identifier ambiguity. parseAmbiguous is documented as "Branch to the right code depending if we have a type, an expression or an identifier" and drives a mode-parameterized disambiguation, a concrete worked example of the general-parsing problem of local ambiguity handled inside a deterministic RD parser.

For Sparkles the takeaway is direct: the hand-written scannerless RD the repo already uses for version schemes is exactly what D's own toolchain does for a much harder grammar. The proposal builds on that grain, not against it.

---

Parser combinators — pry (and the gap)

pry (Dmitry Olshansky — the std.regex author; BSL-1.0) is D's most serious parser-combinator library and the closest local analogue to nom/winnow. It is explicitly pragmatic: "the focus of development is pragmatic qualities such as achieving performance on par with handwritten parsers" (README.md), with generic input via a thin Stream wrapper over D ranges, compile-time-specialized building blocks ("one of a set of values", "given value"), and support for TLV (type-length-value) binary formats. Architecturally it is the D design the survey most wants for an @nogc combinator — but its last release is 2024 (v0.7.0), it is small, and it never grew the zero-copy/@nogc-by-construction discipline of nom/flatparse. Other combinator attempts (sdpc, parsed, pushdown) are one-off and low-signal. This is the gap: D has no maintained, @nogc, zero-copy ordered-choice combinator, and none with chumsky-style error recovery.

---

High-performance JSON & serialization — the mir stack, asdf, JSONiopipe

D's answer to simdjson + serde is the mir stack, and it is genuinely world-class — but it is a serialization framework, not a general parsing toolkit. mir-algorithm ships the @nogc substrate: mir.serde "implements common de/serialization routines" and mir.parse is billed as "@nogc and nothrow Parsing Utilities" (Apache-2.0, Ilia Ki / Symmetry Investments). The actual parsers sit on top:

• mir-ion — a "serialization engine [that] supports Text and binary Ion, JSON, MsgPack, YAML" and more (README.md). Amazon Ion is the core model; JSON/YAML/CSV are frontends. SIMD-optimized, @nogc-friendly, and driven by heavy compile-time introspection-based (de)serialization — the D design that most resembles simdjson (SIMD) fused with serde (derive-based mapping). It is the high-performance successor to asdf.
• asdf — the original libmir fast JSON: "a cache oriented string based JSON representation … specially geared towards transforming high volumes of JSON dataframes" (README.md), SSE4.2-accelerated, built at Tamedia for click-stream processing. Its "Simple Document Format" is a compact binary intermediate — a simdjson-tape-adjacent idea.
• JSONiopipe (Steven Schveighoffer) — the streaming outlier, the D cousin of Angstrom/attoparsec: "a streaming parser, which can be applied to any iopipe of character type … the entire data does not need to be held in memory for parsing" and "there is no required intermediate format" (README.md), with an optional DOM. Pull/lazy, low-allocation, JSON5-capable.

The stdlib baseline is std.json, and it is honest about its limits — the module header carries a red warning that its GC JSONValue DOM, "at the range of hundreds of megabytes … is known to cause and exacerbate GC problems … try replacing it with a stream parser" (std/json.d). It is the correctness-first baseline every faster D JSON parser is measured against. (hipjson is a newer SIMD-DOM attempt; niche.)

---

Structured-data & config parsers

• dxml (Jonathan M Davis) — "a library … for parsing XML 1.0 [whose] parser is a range-based StAX parser, but … also has support for generating a DOM" (README.md). @safe, pure-D, slicing over the input; the clean pull-parser design point in D.
• D-YAML (dlang-community; dub score 4.9) — the canonical D YAML, PyYAML-derived (~YAML 1.1), a spec-heavy hand-written parser with slicing input reuse.
• sdlite (Sönke Ludwig) — "a lightweight SDLang parser/generator … providing a range based API. While the parser still uses the GC … it uses a very efficient pool based allocation scheme that has very low computation and memory overhead" (README.md). The modern SDLang parser (SDLang is dub's own recipe format; dub bundles its own copy). Supersedes the older sdlang-d.

A few adjacent points worth a line: d_tree_sitter provides D FFI bindings to the tree-sitter C runtime — the D path to incremental parsing without reimplementing it; httparsed is a @nogc/betterC HTTP/1 request-response parser (picohttpparser-style), a nice small proof that @nogc parsing is idiomatic in D; and cerealed is a compile-time-introspection binary serialization library (the D cereal), parsing-adjacent but not a grammar parser.

---

The in-tree Sparkles parsers — the baseline

Sparkles already hand-parses several small languages, and they set the idiom the proposal must fit:

• Version schemes (sparkles.versions.parsing + schemes/) — hand-written scannerless recursive descent over a scope const(char)[] cursor. The engine parseSemVerShaped walks a scheme's components with static foreach (design-by-introspection over the scheme struct), advancing the slice and a byte offset in lockstep; the node-semver range grammar is a second, deeper RD layer. Entry points are @safe pure nothrow @nogc, and the result type is Expected!(T, ParseError, NoGcHook) — no exceptions, structured (code, offset) errors, fail-fast.
• The @nogc text substrate (sparkles.base.text) — readers.d provides zero-copy, cursor-advancing primitives (readInteger, skipSpaces, tryConsume, readUntil) that take ref scope const(char)[] and advance only on success, returning slices into the input; writers.d mirrors them for output ranges (SmallBuffer); errors.d defines the shared ParseError/ParseErrorCode/Expected vocabulary. Documented as "mechanism, not policy" — the ideal foundation for a combinator layer.
• CLI arguments (sparkles.core_cli.args) — the pragmatic exception: a compile-time UDA (@CliOption) wrapper around Phobos std.getopt, which uses GC + exceptions. Explicitly not the @nogc/Expected idiom; a candidate to migrate onto the proposed toolkit.
• Terminal VT sequences (sparkles:ghostty) — delegated to the C libghostty-vt engine via ImportC; Sparkles owns no D-side VT state machine.

---

Synthesis — the gap Sparkles fills

Map the D landscape onto the survey's taxonomy and the shape is clear:

Survey category	D's strongest option	Fit for an @nogc Sparkles parser
PEG generator	Pegged (compile-time)	✗ GC ParseTree; great for DSLs, wrong for hot paths
Hand-written RD	dmd / libdparse / sdc	✓ the grain Sparkles already follows (version schemes)
Combinator	pry (stale)	✗ no maintained @nogc, zero-copy combinator exists
SIMD / high-perf	mir-ion/asdf	✓ but serialization, not general parsing; heavyweight dep
Streaming (Angstrom)	JSONiopipe	~ iopipe-coupled; JSON-specific
Recovering	— (none)	✗ no D parser offers chumsky-style recovery

D has a strong compile-time generator (but GC-bound), a proven hand-written-RD tradition (D-specific), and a world-class @nogc/SIMD serialization stack (mir) — yet nothing in the middle: no small, maintained, @nogc, zero-copy ordered-choice combinator over const(char)[] slices, and no error-recovering parser. That middle is exactly where Sparkles' needs sit (version constraints, config, CLI, small DSLs), exactly what the existing sparkles.base.text readers are shaped to support, and exactly the design center the comparison identifies: zero-copy scannerless RD in the nom/winnow mold, zero-allocation validators (flatparse's proof), a Pratt loop for constraint grammars, and Expected!(T, E) results. The Sparkles parsing proposal cashes this in.

---

Sources

Every project characterization is grounded against a locally pinned checkout under $REPOS/dlang/ (pinned SHAs and per-claim verification are recorded in the survey's internal grounding tree, docs/research/parsing/grounding/, which the published pages do not link). Primary artifacts: the project READMEs and source trees named inline (Pegged README.md; libdparse README.md

• vendored src/std/experimental/lexer.d; dmd compiler/src/dmd/{lexer,parse}.d; sdc src/d/parser/ambiguous.d; pry README.md; mir-algorithm source/mir/{serde,parse}.d; mir-ion & asdf README.md; JSONiopipe README.md; Phobos std/json.d; dxml/D-YAML/sdlite README.md), plus the in-tree Sparkles sources.