sparkles:test-runner — Measurement-layer specification

Abstract

sparkles:test-runner runs a D package's unit tests in parallel and carries them into places ordinary unit tests cannot reach: compile-time evaluation, programs built without the D runtime, WebAssembly, and measurement. Its measurement layer turns a test into a benchmark or a measured workload and reports time alongside hardware and operating-system counters. Its rule is that a reported number is either a measurement or labeled as something less. A benchmark's clock runs only while no counter is enabled, so counting cannot distort a reported duration. Every measurement source is probed on the host that runs it: a missing capability yields a named absence with its reason, an estimate is marked as one, and nothing unmeasured is shown as zero.

Introduction

D compiles a module's unittest blocks into the program itself, and the language's built-in runner executes them one after another, with no way to name, filter, or parallelize them. A library also needs some of its tests to run where the ordinary test build never goes: during compilation, without the D runtime, or compiled to WebAssembly. And the test block, sitting next to the code it exercises, is the natural home for a benchmark of that code.

Measurement is the hard part. The counters that explain a timing — retired instructions, cache misses, system calls, time spent waiting for the CPU or the disk — differ by operating system and processor, often need privileges, and perturb the very work they observe. A processor with too few counters time-shares them and returns scaled estimates; a counter that never ran reads as zero. Harnesses that leave these facts implicit either fail on a host that lacks a counter, or print a guess that looks exactly like a measurement. Some work, such as a build or a burst of file I/O, cannot be repeated millions of times to average out noise at all.

The runner answers with two measurement models, and a test opts into exactly one of them by attribute. A benchmark runs its body many times and reports per-iteration statistics. It times and counts in separate passes, so no counter is read while the clock runs. A workload runs its body once, or a few times, and reports what changed across that single window, including how the elapsed time divides into time on the CPU and time off it. Because a workload cannot be re-run, it reads every source at the window's two edges and discloses, rather than subtracts, the small, deterministic overhead those reads add. It may also ask for the files it reads to start cold or warm in the operating system's page cache; the runner prepares only those files, then verifies which state it achieved and records both.

Two rules hold for both models. Every metric carries a class: quantitative metrics barely perturb the work and are the only ones a reported result may rest on, while diagnostic ones explain a result and are rendered apart from it. And every measurement source, such as the kernel's performance-counter interface, opens through a run-time probe, so a missing capability becomes a reported absence with its reason. All sources render through one metric catalog, so a new source appears in tables, column filters, and machine-readable output without changes to any of them.

This document specifies the whole runner at the contract level and the measurement layer in depth. Test discovery and the compile-time, runtime-free, and WebAssembly modes appear only as far as their package layout and command-line contract; the library docs describe their behavior and carry the tutorials and how-to guides. The runner ships as a thin shim compiled into every test binary plus a prebuilt implementation library, sparkles:test-runner-impl. Its floor is the timing and resource counters an operating system grants every process, and it needs no privileges; every source beyond it is optional and degrades on its own. C libraries enter only as soft dependencies, used when present and reported absent otherwise. The runner never evicts system-wide caches: cache control touches only the files a test names.

Section 1 states the two models and the rules they share as the contract at a glance; §2–3 give the package layout and the attributes and in-body calls a test uses; §4 fixes the measurement protocol for both models. Sections 5–6 define the metric catalog and the backend contract, including capabilities and the rules for degrading. Sections 7–8 pin the command line and the output surfaces, and §9 the per-platform floors and privilege rules. Passages marked target belong to the contract but are delivered by the PLAN.md milestone they name; PLAN.md also holds the delivery order. Unresolved behavioral questions live in open-issues.md. The evidence for the design is the CPU-PMU research catalog, in particular its audit baseline and backend proposal.

1. Overview

sparkles:test-runner is a general-purpose unittest runner (parallel runtime tests plus @ctfe, @betterC, and @wasm modes) with a benchmark-measurement layer under --bench.

The measurement layer has two measurement models:

• @benchmark — per-iteration statistics: the timed body runs many times; the runner reports median / median-absolute-deviation / min / max nanoseconds per iteration, plus per-iteration counter averages.
• @workload — window statistics: the body runs once (or a few reps); the runner reports counter deltas and integrals across the window, including a wall-clock decomposition into on-CPU and attributable off-CPU time.

Three invariants shape everything:

1. Counting is separated from timing. The timing pass runs with no counters enabled; counters bracket a separate pass. No repaint, GC, or counter I/O ever runs concurrently with a timed body.
2. Every metric is classed quantitative (near-zero perturbation — the only class a reported or gated number may read) or diagnostic (perturbs; explains a result; rendered separately, never blended into a headline).
3. Capability is a runtime probe result. Every acquisition source opens through a probe handshake and degrades to reported absence, never fails the run and never fabricates numbers — absences are enumerated per capability with a reason (§6.2), not just per-tier status strings.

2. Package and module layout

The runner is two dub packages:

• sparkles:test-runner (libs/test-runner) — a thin sourceLibrary shim (compile-time discovery + registration) compiled into each test binary.
• sparkles:test-runner-impl (libs/test-runner-impl) — the prebuilt implementation library, linked across an extern(C) seam.

Measurement modules (all under libs/test-runner-impl/src/sparkles/test_runner/):

Module	Role
bench.d	protocol driver, benchIter/benchCase/blackBox, BenchStats
perf.d	hardware-counter tier (perf_event group)
perf_group.d	shared counting bracket + multiplex-delta scaling
tier0.d	no-privilege tier (getrusage + /proc/self/io)
syscalls.d	syscall-tracepoint tier
metrics.d	the metric catalog seam (§5)
capability.d	the capability seam: flags, reports, backend trait, host probes (§6.2)
raw.d	raw hardware-event tier (raw:r<hex> selectors)
event_naming.d	symbolic event names via soft libpfm4 (pfm:<name>)
rdpmc.d	user-space counter reads (the selfMonitoring primitive)
workload.d	the @workload window model: wall source, decomposition, driver
psi.d	PSI stall integrals (/proc/pressure parser + diagnostic source)
cache_regime.d	page-cache regime control: fs probes, residency, the stamp policy
bench_json.d	the --bench-json emitter (§8.3)
reporting.d	tables, live displays, progress
skip.d	skipTest

Planned modules (targets): provenance.d/cgroup.d (M7–M8), histogram.d (B5), sampling.d/symbolize.d (B6), offcpu.d (M9), loadgen.d (M10), plus per-OS backend variants inside the existing modules (B3/B4).

3. Attribute and in-body surface

Marker UDAs live in sparkles.test_runner.attributes and must be imported unconditionally (not under version (unittest)).

• @benchmark — the test is skipped by normal runs and measured by --bench. @benchmark(iterations: N) pins the iteration count instead of auto-scaling: per sample for batched timing; a per-call benchCase runs exactly N timed calls, one sample each.
• benchIter(scope void delegate()) — measure only the closure; the rest of the body is setup. Outside --bench the closure runs exactly once.
• benchCase(name:, labels:, timed:, after:, setup:, teardown:, metrics:) — register one row of a matrix benchmark. Under --bench the case is registered and measured after the body returns (deferred execution: varying state must be captured by value); outside --bench it runs once immediately. timed's result flows to after, which runs untimed to verify/release it; a throwing after or an Expected error isolates into a single in-table error row while the rest of the matrix measures. A soft error returned on the inert (non---bench) path is re-raised as an exception. Label keys/values and name must not contain \x1f (the group separator) — violation is a registration-time error.
• blackBox(x) — identity the optimizer cannot see through; route measured inputs and results through it.
• skipTest(reason) (sparkles.test_runner.skip, @safe pure nothrow @nogc) — skip the enclosing test at runtime: a yellow ⊘ line plus an N skipped summary segment; never fails the run. Under --bench a case-level skip renders as a yellow row.
• Client metrics — Metric(Unit unit, double amount, Mode mode): rate reports amount ÷ iteration-time as <unit>/s; level reports the per-iteration amount as-is. Unit is an open-basis symbol ("B", "req", …); see §5.3.
• @workload / @workload(reps: N) — window-model marker (reps clamps to ≥ 1; combining with @benchmark is a discovery-time error — the models are exclusive). The in-body workloadWindow(dg) / workloadWindow(name, dg) primitive measures only the closure (× reps) as one window; without any call the whole body is the window. Outside a workload measurement the closure runs exactly once, inertly. Unnamed windows take the test name, then #2, #3; named ones <test>/<name>. @workload(regime: CacheRegime.cold) requests a page-cache regime for the files the body names via workloadFiles(paths...) / workloadFiles(regime, paths...) (per-call override): the call preps the files NOW (cold = fdatasync + fadvise-evict; warm = read-through preload; steadyState = nothing), verifies residency (mmap + mincore), and stamps the NEXT measured window with requested-vs-effective (CacheRegimeStamp) — consume-once, because the verified state is stale the moment a window has run against it: later windows are unstamped until their own workloadFiles call, and a later call before any window REPLACES the pending stamp. Outside a workload measurement the call does NOTHING (no eviction, no probes — cache vandalism in an ordinary run); inside an open window or on a whole-body repetition after the first, prep is SKIPPED and the window's note says so.

4. Measurement protocol

Normative for @benchmark:

1. Auto-scaling — the per-sample iteration count doubles until one sample takes at least BenchConfig.minSampleTime (default 5 ms; --bench-min-time=MS overrides), then sampleCount (32) samples are collected. Timing uses raw MonoTime ticks (no hectonanosecond quantization).
2. Statistics — median, median absolute deviation, min, max ns/iter. No mean is reported.
3. Per-call vs batched — benchCase times each call individually (one sample per call; --bench-min-time is the minimum total measured time, samples accumulate past 32 until met); benchIter/whole-body timing is batched (--bench-min-time is the per-sample auto-scale target). @benchmark(iterations: N) pins both shapes and makes the budget inert.
4. Counting passes — after timing, each available source re-runs the body under its counters, capped at perfMaxIters (100 000) iterations, with the untimed release/verify hook outside the enabled window. Counter values project to per-iteration doubles, unrounded.
5. Serial execution — benchmarks never run in the test thread pool. Registered cases are scheduled grouped by their streaming key (the --group-by group, else the source test's qualified name).
6. Assert-enabled builds warn — --bench on a debug/assert build prints a stderr warning; real numbers require an optimized unittest build type.

Thread coverage is inherit-shaped: counters follow threads spawned after the source opens; pre-existing threads and short-lived children are blind spots (see open-issues.md § O3).

For @workload: the driver phase model is setup → snapshot-before all sources → body × reps → snapshot-after → assemble deltas. Recorded judgment (M6): the originally planned distinct regime-prep and post-body residency-verify phases dissolved into the in-body workloadFiles call site — prep AND verification happen there, and the whole-body candidate window's edges are re-opened after prep (prep is setup, retroactively; a second call restarts the window again and the fallback row discloses that only work after the last call is measured), so a post-body probe phase would either pollute windows or describe a candidate that in-body windows discard. The same judgment records that /proc/sys/vm/drop_caches is never used: it is a system-global, root-only sledgehammer that evicts every other process's state — per-file posix_fadvise + verification + downgrade-note is the honest scope, and M8's cgroup memory.max is the scoped successor if stronger eviction is ever needed. Measurement is a single pass — sources are read cumulatively at the window edges (GroupSnapshot, no per-iteration bracket, no RESET), so a whole-body candidate window and in-body windows overlap freely and the body is never re-run for counting (it may be expensive or non-idempotent). Edge nesting, outer → inner: psi, wall clock, wall source, syscalls, raw, tier-0, perf. The nesting means an outer source's window contains the inner sources' edge reads — a small, deterministic apparatus floor (≈2 syscr in tier-0 with perf open; ≈a dozen syscalls in the syscall total), disclosed in the how-to and never netted out (subtracting an estimate would fabricate). Psi is outermost — its file reads sit outside every other window, including the wall clock and rusage, so the decomposition carries zero psi apparatus (a system-wide µs-resolution integral's own window being ~20 µs wider than the wall clock is immaterial). A window across which a group's enabled time never advanced reads nan, never zeros. The wall-clock decomposition reports onCpuUser/onCpuKernel (rusage), offCpuRunqueue (schedstat), offCpuDisk (cgroup-scoped PSI — target — M8), and a clamped offCpuOther residual — only runqueue is a true per-cause duration today; lock/sleep/disk time is never fabricated, and every unattributable component is nan plus a note, with its time left in the residual. On Linux the decomposition is thread-scoped (RUSAGE_THREAD plus /proc/thread-self/schedstat — the only scoping under which wall = onCpu + runqueue + other is arithmetically meaningful); CPU burned by other threads is disclosed via the process-wide reading. Elsewhere it degrades to process scope (POSIX) or wall-only. The thread-coverage caveat above applies to windows identically.

Recorded judgment (M5): PSI stall integrals are diagnostics, not attribution. /proc/pressure is system-wide, so a window delta of its monotonic total=<µs> accumulators states "the system accumulated this much stall concurrently with the window" — it cannot be assigned to the measured thread without fabricating (on a host with background IO, a pure CPU spin's window shows tens of ms of concurrent io stall the thread never waited on). The original M5 target ("feeds offCpuDisk") was therefore retargeted: windows carry the raw system-wide deltas as a psi diagnostic (the io-stall table column, placed after other, and the psi JSON object with scope: "system"), and offCpuDisk stays nan until M8's per-cgroup *.pressure (same file shape, same parser) makes the attribution real. Considered and rejected: capping the integral by the thread's off-CPU time (reports coincidence as attribution exactly when the cap doesn't engage, and fails the shipped sleep-honesty test on a busy host); attributing only provable zeros (flaps baselines between 0 and null with background load).

5. The metric catalog

5.1 Types

Everything downstream renders through two types (metrics.d):

enum MetricClass  { quantitative, diagnostic }
enum MetricFormat { ratio, count, percent }

struct MetricCell        // one rendered value
{
    string name;         // stable id: "ipc", "instr", "B/s", "syscalls:read"
    string header;       // column label
    double value = double.nan;  // nan renders as an em dash
    MetricFormat format;
    MetricClass cls;
}

struct MetricDescriptor  // one catalog entry
{
    string name;
    string header;
    MetricFormat format;
    MetricClass cls;
    string source;       // "client" | "perf" | "tier0" | "syscall" | …
    bool available;      // producible on this run
    bool isDefault;      // shown without a --metrics filter
}

5.2 Contract

• Each source contributes a projection pair — XCells(in XStats) for row cells and XFamily(bool available) for descriptors. Adding a source is one Nullable!XStats field on BenchStats plus one line in each of open/close/countInto, plus the cells/family pair. Table rendering, --metrics filtering, --list-metrics, and --bench-json then work unchanged.
• --metrics=LIST selects columns by exact name, comma list, *-glob, all, or ?/help (print the catalog). Selection transitively opens the sources it needs (naming a perf metric opens the perf pass; naming syscalls/syscalls:<name> opens the tracepoint pass). A selector that matches nothing warns on stderr — selectors are never silently dropped.
• Client metric names that shadow built-ins get a user. prefix.
• With no filter, the default column set is stable across releases within a schema version (regression guard: --perf output is byte-compatible).

5.3 Units seam

Unit.symbol is a label, not semantics (the open-basis "mint-by-name" identity); Metric.Mode is a two-valued stand-in for a time-exponent dimension (rate = unit·s⁻¹). All unit/rate/format semantics live behind one seam — the scaled/fixed formatters — so the future sparkles.quantities binding is a localized swap (see open-issues.md § O6).

6. The backend contract

6.1 The source shape (shipped)

Every acquisition source implements, on all platforms:

tryOpen(...)      // probe handshake; may calibrate (arity varies per tier)
available()       // bool: producible on this run
status()          // human reason when unavailable
capabilities()    // CapabilityReport: flags + reasoned absences
count(...)        // bracket a counting pass; fill the row's XStats
close()

with a version (linux) (or per-OS) real body and an identical-surface stub elsewhere. Two source shapes exist: bracketed (ioctl ENABLE → body → DISABLE: perf, syscalls, raw) and snapshot/delta (a reading pair around the body: tier-0; window-friendly for M4+).

Batching. A bracket is not free — an ioctl ENABLE/DISABLE pair costs ~2.2 µs (~3 300 retired instructions), and a tier-0 snapshot pair two /proc reads — so a bracket per iteration makes a nanosecond-scale body's counters report the apparatus instead of the body. count therefore takes a batch: it brackets batch iterations together, running exactly iterationstimed() calls either way, so the divisor is unchanged and the bracket's cost amortizes by batch. Batching reorders between() to after its batch, so it is used only for batched rows (benchIter/whole-body, where between is a no-op); per-call benchCase rows keep batch == 1, which is exactly the original per-iteration bracket. The runner auto-sizes batch from the timing pass so one bracket spans ~1 ms (--perf-batch=N pins it; 1 disables). Measured effect, three ~10–14 ns codec rows: instr/iter 3.52k/3.55k/3.57k (indistinguishable — all bracket) → 233/266/284 (separated, ordered with their timings, self-consistent at IPC ≈ 4.4), with the timing pass unchanged.

6.2 The capability model

Backends advertise what this host, this run, can measure (capability.d):

enum Capability : uint
{
    none, counting, countingRaw, countingScaled, selfMonitoring,
    ipSampling, preciseMemory, symbolization, eventTracing,
    numaAttribution, eventNaming,   // one flag per survey concern
}

struct CapabilityAbsence { Capability capability; string reason; }

struct CapabilityReport
{
    Capability available;                // OR of the present flags
    const(CapabilityAbsence)[] absences; // reasoned, declaration order
}

capabilities() is a const observer read after the open handshake — a deliberate deviation from the research sketch's "tryOpen returns a report": the real tryOpens have divergent arities and every call site depends on them returning the group, so construction stays per-tier and the report is a separate query. Absences are an ordered array of pairs, not a reason map — deterministic render order, nothrow-friendly, and static immutable- bindable for the strict stub attribute blocks.

The DbI isCounterBackend trait names the required instance surface (available/status/capabilities/close/count); optional primitives (hasSnapshot, hasNamedColumns; later: precise sampling, page classification, name resolution) unlock optional capabilities by presence. Each tier module compile-validates the trait against whichever body — real or stub — the platform built.

A capability that hardware supports but no backend delivers yet stays absent, with the host finding carried in the reason (e.g. preciseMemory — hardware present (ibs_op PMU) — data-source sampling lands in B5). Concerns no backend owns yet report harness-level, so the vocabulary is complete from the start.

--list-metrics renders a per-backend capability block; the bench header prints one line per absent-but-requested capability, re-derived from the same reports. Workload-track sources (PSI, cgroup, cache regime) adopt the same report when they land — one absence vocabulary program-wide.

Recorded judgment (B3): the darwin fixed counters stay diagnostic class — the catalog entries (ipc, instr, cycles) cannot change class per OS without breaking the schema-stability contract (§5.2), and process-wide counters polluted by concurrent threads are precisely not safe for reported/gated numbers. The process-wide scope itself is a permanent degraded() condition, disclosed by the bench header once per run (with the P/E-core aggregation suffix on heterogeneous hosts). Two further honesty bounds: the darwin counters are calibrated-bracket, not ioctl-excluded — they free-run, so each bracket's own proc_pid_rusage syscalls are counted and removed by subtracting a median empty-bracket cost re-measured at the start of every counting pass (the cost scales with the process's live thread count) — treat per-iteration differences of a few tens of instructions as calibration residue, not signal; and the catalog only advertises the fixed-backed columns (perfFamily(fixedOnly:) — the configurable-event columns are absent, not permanently-em-dash "available").

The workload wall source (WallSource: rusage + schedstat) reports through the same vocabulary, appearing in the --list-metrics block as wall (inserted at render time — it has no counting pass, so it is deliberately not in CounterGroups' fixed bench bundle). One recorded vocabulary judgment: schedstat unreadability is not a CapabilityAbsence — the report cannot hold counting as both present (rusage) and absent (schedstat); the narrower fact surfaces via status(), one stderr disclosure per run, and a per-window note.

6.3 Degradation rules (normative)

• Absence is reported, never fatal. An unavailable source yields omitted columns plus a reasoned capability entry (§6.2).
• "Not counted" is not zero. A counter group with time_running == 0 (never scheduled) reports its cells unavailable (nan → em dash) — never 0, never a scaled estimate.
• Exact by intent; every estimate is labeled. The default counting group is calibrated at open and shrunk (the LLC pair drops first) to avoid multiplexing; opt-in --perf-scaled keeps the full multiplexing group instead. In either mode the label keys off the truth of each pass: every cell whose pass was scaled (running/enabled < 1 — including ambient PMU contention the open-time calibration could not foresee) renders with a ≈ prefix and is named in --bench-json's per-row estimatedMetrics. A multiplexed pass with under a millisecond of PMU time renders unavailable, never as a number (a 0.58 ms slice measured a 5.7× scale error).
• Group-refused degrades at open. A platform that refuses an unplaceable group outright (RISC-V SBI) fails perf_event_open and reports the standard open-failure absence — it is never multiplex-scaled.
• Privilege gates are independent axes. perf_event_paranoid, tracefs file permissions, and per-field gates (e.g. physical addresses) are probed separately; each degrades on its own.
• Degraded-but-available modes are disclosed. A perf group that opened but is not in its clean default state — user-only fallback, dropped LLC pair, scaled mode — prints its status once in the bench header; em-dash columns alone never stand in for the reason.

7. CLI contract

The authoritative option list lives in the CLI reference; this section pins the contracts.

• Mode exclusivity — --bench, --ctfe-trace, and --better-c/--wasm are mutually exclusive (hard error); --better-c with --wasm is one extraction family; --list/--list-metrics are queries that win over any mode.
• Readable failures — malformed options, unknown flags, and stray positionals produce one-line errors, never stack traces.
• Warn, don't drop — unknown --metrics, --sort-by, and --syscalls selectors warn on stderr; the run proceeds with the remainder.
• Selector equivalences — sc:<name> ≡ syscalls:<name> everywhere a metric name is accepted.
• Sorting — --sort-by=KEY orders ascending within each group; error rows always sort last under every order; default is median/iter.
• Grouping — --group-by=KEYS streams one table per label-key group; =all uses every label key; =list prints the keys and exits (reporting registration failures with a non-zero exit).
• Exit status — 0 iff everything passed or was skipped (skipTest or a toolchain-missing mode); non-zero otherwise. A --bench-json write failure fails the run.
• Hardware-event selectors — --metrics accepts raw selectors (raw:r<hex>, the perf tool's rNNNN notation) and, when event naming is available (soft libpfm4), symbolic names (pfm:<name> with umask and :u/:k modifier grammar); both become diagnostic columns riding their own counter group, so the default group's exactness is never perturbed. A failed name resolution warns and drops the column. --perf-scaled opts into labeled multiplex estimates (§6.3).
• Counting-pass pinning — --perf-iters=N pins the counting-pass iteration count (default: the timing pass's count, capped), making per-pass counter totals and amortized one-time costs reproducible across runs; the effective count is auditable per row (§8.3).
• (target — B6) --bench-profile enables the sampling pass.

8. Output surfaces

8.1 Tables

• The fixed value columns are n (samples, or samples×iterations when a case batches), median/iter (with the ±median-absolute-deviation folded into the same cell), min, and max, followed by the metric columns — retired instructions first, as the exact, host-stable anchor (§6.3). n and the merged median/iter are right-aligned (a trailing ±dev leaves no shared dot); min, max, and the metric columns align on the decimal point. Consecutive streamed tables share their column geometry (floors only widen during a run). Grouped tables carry benchmark: <group> in the top border over an implementation column.
• Unavailable cells render as an em dash. Multiplex-scaled estimates carry a ≈ prefix (§6.3). A run containing an error or skip row grows a trailing notes column — the first line of the message renders there, wrapped to a fixed cap, leaving the row's numeric cells em dashes (full traces print to the console); error rows sort last. An all-green run has no notes column.
• Diagnostic-class output beyond columns (profiles, histograms — targets B5, B6, M9) renders as labeled blocks below the numeric table, never as throughput-lookalike columns.

8.2 Live displays

One suppression policy for all three live displays (runtime progress line, bench table ticker on stdout, bench stderr spinner): suppressed when piped, under --no-colors, $NO_COLOR, or TERM=dumb. Repaints are bracketed in DEC-2026 synchronized output and happen only at case boundaries — no painter thread. Piped output is byte-stable and prints each table once.

8.3 The --bench-json document

One deterministic JSON document, {schema: 2, meta, columns, rows}:

• meta — {date, hostname, os, arch, compiler, cpu, minSampleTimeMs, sampleCount}: host/toolchain provenance plus the run's effective knobs (baselines are self-describing), and — when a suite registered any via benchProvenance — a provenance array of suite-controlled facts that shape the numbers (allocator regime, codegen configuration).
• columns — the available catalog descriptors {name, header, format, class, source}; metrics keys in rows match --list-metrics names.
• rows — measurement order, unaffected by --sort-by/--group-by (group dimensions travel in each row's sorted labels): {name, labels, iterations, samples, medianNs, deviationNs, minNs, maxNs, metrics, error}. Error rows keep labels and error with null timing fields; nan cells are null. A row whose counters were multiplex-scaled additionally carries estimatedMetrics, the array of metrics keys holding estimates (absent = every metric exact), and a row that ran a counting pass carries its effective countIterations (absent = no pass ran) — the schema-2 additions.
• windows — present only when @workload tests ran (a windowless document is byte-identical to the pre-window shape): one object per measured window in measurement order — {name, reps, wallNs, scope, onCpuUserNs, onCpuKernelNs, offCpuRunqueueNs, offCpuDiskNs, offCpuOtherNs} (null = unattributable, exactly the table's em dash; offCpuDiskNs is always null until M8 — see the §4 recorded judgment), one nested totals object per attached source (perf, tier0, syscalls, raw), an optional psi object — {scope: "system", ioSomeNs, ioFullNs, memSomeNs, memFullNs, cpuSomeNs}, the system-wide stall deltas, omitted when PSI is unavailable — an optional regime object — {requested, effective, residentBefore, residentAfter, note?}, the page-cache regime workloadFiles established and verified for this window, omitted when no prep preceded it and KEPT on error/skip windows (the stamp predates the window) — an optional note, and error/skipped mirroring the row shapes. Window values are totals with their own field names, deliberately never the per-iteration metrics catalog keys (the misrepresentation open-issue O7 guarded against; resolved as its option A, the anticipated bump absorbed into the never-released schema 2).
• Number policy: nan/infinity → null; integral values below 2⁵³ print as integers; others to 6 significant digits. Output is byte-deterministic for committing.

9. Portability and privilege

Per-OS floors, with shipped-vs-target markers. The full evidence base is the research capability matrix.

Platform	Floor (unprivileged)	Beyond the floor	Status
Linux x86_64	tier-0 (getrusage + /proc/self/io); perf counting at paranoid ≤ 2*	tracepoints (tracefs, usually root); precise memory (B5); sampling (B6)	shipped / targets
macOS (Apple Silicon)	proc_pid_rusage → true instructions/cycles/IPC, process-scope**	kpc: root-or-blessed-pid + kernel allowlist, single-owner EBUSY; sampling via xctrace only; no DTrace cpc provider	shipped
Windows	CycleTime via thread profiling (driver-free)	ETW PMC counting (admin + SeSystemProfilePrivilege, 3–4 PMC hard budget, no multiplex scaling); public precise sampling: absent	target — B4 (blocked: no hardware bed)
ARM-Linux	generic events port as-is (PMUv3)	big.LITTLE: events must open on the pinned core's PMU (wrong cluster counts silent zero); SPE/BRBE gated	target — M11 (source-verified only)
RISC-V	counting always (SBI-mediated)	sampling iff Sscofpmf; exclude_* iff Sscofpmf; precise/data-source: permanently absent; branch records: no kernel consumer	target — M11 (capability subset by construction)
Android (x86_64, arm64-v8a)	unittests in-process, through the same runner (buildDAndroidTests); plain-text reports	perf counters, tracepoints and PSI only where the device exposes them; no @ctfe probe (no compiler on the device)	shipped (x86_64 emulator); arm64 built, not yet run

* All hardware verification to date ran at perf_event_paranoid = −1; behavior at stricter levels is literature-derived until probed (open-issues.md § O1).

** The macOS floor is perf.d's darwin body: the same PerfGroup surface backed by the XNU monotonic fixed counters (user+kernel, all threads, free-running — enable/disable degrade to a window-arming latch), verified live on the T6041 bed; a darwin tier-0 body adds the getrusage fault/context-switch fields plus ri_diskio_* byte counters. Virtualization.framework guests (GitHub's macOS runners) read flat fixed counters — the open probe degrades to a reasoned absence there, so CI exercises the degrade path and real hardware the live path.

Normative portability rules:

• Never assume 4 KiB pages / 64 B cache lines (Apple Silicon: 16 KiB / 128 B).
• Topology and storage provenance are re-probed per boot (BIOS NUMA modes re-scope nodes and uncore counters).
• The event-name vocabulary is harness-owned: no naming layer spans operating systems (libpfm4/LIKWID are Linux-only; kpep and ETW profile-sources are OS-local). Naming data (kernel pmu-events, kpep plists, vendor JSON) is harvested offline, never a runtime dependency.
• C libraries are soft dependencies at most (libpfm4 for naming, libdw for symbolization): absence degrades to an advertised missing capability. libnuma is never linked — page→node classification uses the raw get_mempolicy/move_pages syscalls.
• Environment-dependent tests use skipTest(reason), so a degraded host skips visibly and never fails.