The UI backend seam

Status: complete — 38 of 38 subjects surveyed. The synthesis in comparison.md is written against all of them and reports findings F1–F12; the shared vocabulary is concepts.md. Last reviewed: August 23, 2026

sparkles:ui draws through one seam, the Design-by-Introspection concept isCanvas!T in libs/ui/src/sparkles/ui/canvas.d. Three backends implement it today — a terminal cell grid, raylib, and now Skia — and writing the third one produced a list of places where the seam forces a backend to lie. That list is canvas-seam-friction.md: eight entries, each recorded while implementing rather than reasoned about afterwards.

This survey exists to answer whether those eight are a design that can be repaired in place or a design that should be replaced, and to do it against evidence from systems that solved the same problem rather than against taste.

The question this survey answers

How should a UI toolkit define its renderer seam when the targets genuinely disagree about what a pixel is?

That qualifier is the whole difficulty, and it is worth stating plainly because it makes most of the obvious prior art inapplicable. sparkles:ui must serve:

a terminal, where the smallest addressable unit is a character cell, text advance is a Unicode property, and there is no such thing as a hairline; and
a GPU surface, where the unit is a device pixel, text advance is a property of a shaped font, and a hairline is ordinary.

Almost every toolkit surveyed below picks one of those and designs freely. The few that span a comparable range are the ones to read closely — and the survey's sharpest negative result is that no surveyed subject spans what sparkles:ui spans. Slint comes closest and its range is small-GPU to large-GPU; imtui is the one project that retargeted a geometry seam to character cells, and it paid with a permanent fork of the widget layer.

What each subject must answer

Eight questions, one per friction entry, so that every subject is read for the same thing and the synthesis can compare like with like. A subject that does not have an answer to a question is a finding, not a gap in the notes.

Every question is answered across all 38 subjects in comparison.md, which groups the subjects by distinct answer rather than listing them — the size of each group is itself evidence.

#	Question	Friction entry it tests	Verdict in `comparison.md`
Q1	What unit does text measurement return, and can a backend answer in its own unit?	§1 `measure` is denominated in cells	Confirmed, and enlarged (`F1`, `F2`)
Q2	Is the backend contract stated in one place, or discovered by the caller probing for optional methods?	§2 five methods, eight kinds	Confirmed, reframed (`F5`, `F11`)
Q3	Do semantic widgets (a scrollbar, a focus ring) reach the backend as themselves, or as primitives?	§3 `scrollbar` is a widget concept in the drawing seam	Half-refuted, half-confirmed (`F4`)
Q4	How is a draw command encoded, and what does the encoding charge the operations that do not need the widest payload?	§4 the encoding is neither `@safe` nor variable-width	Reification affirmed; the encoding is a live trade (`F3`)
Q5	How is sub-unit placement expressed when the toolkit's unit is coarser than the device's?	§5 sub-cell placement as a compass direction	Reframed, not refuted (`F6`)
Q6	Does a command carry resolved appearance, semantic role, or both — and who re-resolves?	§6 a resolved appearance and a semantic role on every drawing op	Confirmed, with a fork (`F9`)
Q7	Who owns a command's payload (text, images), and can a command outlive the frame that made it?	§7 `DrawOp.text` is borrowed, and the borrow is not expressible	Confirmed, unanimous 38/38 (`F8`)
Q8	Can a backend ask the scene its extent before allocating a surface?	§8 no extent query	Confirmed, and split three ways (`F7`)

NOTE

The verdict column summarises; it does not substitute. Each verdict turns on a distinction the one-word label cannot carry — §1 is confirmed but relocation alone fixes nothing, §3 is right about the layering smell and wrong about which half is the smell. Read comparison.md before acting on any row.

Master catalog

Thirty-eight subjects. The four that opened the survey (Slint, Qt QPaintEngine, Notcurses, egui) plus 34 added in the completing pass. Ordered as a curriculum rather than alphabetically: the ones that bound the space first, then the retained trees and reified streams, then capability, measurement, cell targets, and finally the functional-programming end that shows what the questions look like when they dissolve.

Target class is the coarse fact that decides whether a subject's answer transfers: terminal (character cells), GPU (device pixels, shaders), vector device (a resolution-independent 2-D imaging model — printers, PDF, SVG, CPU raster), cross-target (more than one class behind one design), and none for the subjects that name no device at all.

Subject	Category	Language / ecosystem	Target class	What it teaches
Slint	spans the range	Rust (C++/JS/Python APIs)	cross-target	One semantic `ItemRenderer` does span an MCU software renderer and Skia — and it is semantic because it has to.
Qt `QPaintEngine`	virtual paint device	C++	vector device	The declared feature set, and framework emulation only when the engine declines. Measurement was never on the painter.
Qt Quick scene graph	retained render tree	C++	GPU	Qt's own replacement deleted `PaintEngineFeature`; geometry-plus-shader is portable only among like backends.
Notcurses	cell-only, sub-cell	C	terminal	Name a fidelity ladder that auto-degrades, not a sub-cell position — and width is a free function, not a method.
egui / epaint	backend gets geometry	Rust	GPU	The far end: the backend receives triangles. Nothing left to negotiate, and no cell target reachable.
GTK4 / GSK	retained render tree	C (GObject)	GPU + CPU + browser	The seam can be a data structure, not a painter: 37 node kinds, each shipping its own Cairo fallback.
Flutter engine	retained scene	C++ engine, Dart API	GPU	68 exactly-sized op structs in a byte arena — reified and comparable without being a tagged record; no string crosses.
Zed GPUI	primitive scene	Rust	GPU	When every target shares a rendering model you can pick primitives that never need degrading — and own every payload.
WebRender	display list → batching	Rust	GPU	The only seam that is a serialized value, and the only one that writes down what its vocabulary refuses to carry.
Vello	encoding, not commands	Rust	GPU (+ CPU raster)	The consumer dictates the encoding: parallel streams for the shader, a plain enum for the sequential renderer.
Ratatui	cell-only	Rust	terminal	A seam that is the result buys diffing, read-back composition and value goldens — and cannot reach a GPU.
Cairo and Direct2D	device abstraction	C · C++/COM	vector device	The prior generation, and where it leaked: every GPU backend Cairo ever had is now deprecated or removed.
Skia `SkPicture`	reified command stream	C++	GPU + vector device	Three seams of decreasing width, and the whole optional-capability contract in language constructs the compiler checks.
Chromium `cc::PaintOpBuffer`	reified command stream	C++20	GPU (across a process)	What a reified stream converges on once it must survive serialization: tag plus per-op struct, tiered payload ownership.
Avalonia	multi-backend toolkit seam	C# (.NET)	cross-target	Optionality is not one bucket: capability is declared at four distinct scopes, and every node carries `Bounds`.
Iced	capability by trait decomposition	Rust	GPU	Capabilities as sub-traits a widget demands by name — and the price of a backend-chosen measurement type.
wgpu	formal capability negotiation	Rust	GPU	Refusability is a type, not a value; and a grant closed above turns cross-backend parity into a consequence.
SDL_Renderer	minimal device abstraction	C99	GPU	A written mandatory floor plus one universal lowering primitive — and fidelity exposed as named user policy.
Java2D	framework-side emulation	Java (+ C/Objective-C loops)	vector device	Make the floor the superclass and a backend cannot forget it. Appearance lives in pipeline state, not on the command.
Godot `TextServer`	measurement as a service	C++17	GPU	`F1` in its strongest form: measurement is a named, swappable, capability-declaring peer; the drawing seam has no text.
Parley and Xilem	greenfield split of text from scene	Rust	GPU	Shaping as a borrowed context, a ten-method sink, and appearance as a late-bound index into a per-frame palette.
piet	abandoned multi-backend 2D seam	Rust	vector device	The survey's signed post-mortem: a seam defined as the intersection of its backends can be neither consistent nor grown.
Pango and HarfBuzz	measurement as a separate layer	C (GObject) · C++ with a C ABI	vector device	Measurement is device-parameterised: hinting and resolution are declared inputs that change the numbers.
imtui	cross-target natural experiment	C++	cross-target	Retargeting a geometry seam to cells fails upward: with no semantics, degradation has nowhere to live but the widget.
Mosaic	cross-target, shared layer above	Kotlin (Multiplatform)	terminal	The portable artifact is the retained node tree, reused verbatim through a four-method `Applier`. The painter is two ops.
Textual	cell toolkit with a second target	Python	terminal (+ web, pixel)	Extra targets were cheap because there is no drawing seam — only a reified, immutable, self-measuring result.
libvaxis	cell-only with sub-cell escapes	Zig	terminal	Publish the pixel size of a cell and negotiate the rest; `Widget.draw` returns a size — the cheapest extent mechanism.
Haskell diagrams	typed backend seam	Haskell	vector device	Capability as a type-class instance makes omission a compile error — and leaves the silent degrade one line away.
OCaml Vg	documented renderer contract	OCaml	vector device	The renderer contract as a written artifact, per backend, in the seam's own vocabulary — and entirely unchecked.
Monomer	record-of-functions seam	Haskell (+ a C shim)	GPU	A value seam is perfectly self-describing and perfectly inflexible: 45 mandatory fields, no optional primitive.
Doodle	tagless-final algebra	Scala 3	cross-target	The placement rule the friction log was groping for: semantic and unserviceable by some backends ⇒ its own capability.
Vty `Image`	image algebra (cell)	Haskell (+ a C width table)	terminal	Extent for free, cached by every constructor — and one process-global width table so two consumers cannot disagree.
Notty	image algebra with stated laws	OCaml	terminal	Invariants maintained at construction remove the need for backend queries entirely; twelve total closures suffice.
Gloss `Picture`	pure sum-type scene	Haskell	GPU (OpenGL)	Sum-type encoding and tree-vs-flat are independent axes; a tree keeps the scale factor a flat list has thrown away.
Functional images	the theoretical extreme	Henderson 1982 · Elliott's `Pan`	none	Six of the eight questions dissolve — and why they dissolve is the boundary marker, not a route.
elm-ui	purity as a forcing function	Elm 0.19	browser DOM	Exactly which layout decisions you lose when nothing can measure text: ellipsis, conditional wrap, content-sized things.
Racket `dc<%>` and pict	device abstraction under an FP layer	Racket	vector device	Measurement on the painter, priced honestly — plus a recorder generated from the seam, and three distinct extent queries.
elm-canvas	commands as pure data	Elm 0.19 (+ ~90 lines of JS)	browser canvas	Reifying commands buys nothing on its own if the actual seam is still a stringly-typed `{type, name, args}`.

Taxonomies

The same 38 subjects, re-cut one axis at a time. These tables are the umbrella's real payload: they are where you look to find the subjects that answered a question the way you are considering answering it. Membership follows comparison.md's per-question tables; a subject that gives two answers appears twice, and where that happens it is called out.

By who measures text (`Q1`)

Who answers	Subjects
Nobody — unshaped text cannot cross the seam	Flutter, WebRender, Chromium, Qt Quick, GSK, Avalonia, Vello, egui
A free function or fixed table above the seam	Ratatui, Textual, Mosaic, Vty, Notty, Notcurses
A peer service with its own contract	Godot `TextServer`, Pango, Parley, Qt `QPaintEngine`, Java2D, Skia, Cairo, GPUI, Monomer
A type the backend chooses	Slint, Doodle, Iced, piet
Removed from the library entirely	Vg, Gloss, diagrams, elm-ui, elm-canvas, SDL
The painter — the only dissent	Racket `dc<%>`, libvaxis (wrapped extent only), imtui (by redefining the unit)
Does not arise — no text in the seam	wgpu, functional images

35 of 38 keep measurement off the painter, and the one outright dissenter prices it. Only Slint and Doodle make the returned unit backend-chosen.

By command shape (`Q4`)

The five families the friction log cares about — no reified command, an instruction stream, a reified result, geometry, an encoding — split into eight distinct answers once the survey is read.

Shape	Subjects
Sum type / tagged union, per-variant payload	egui, GPUI, WebRender, SDL, Cairo, Vg, Gloss, Vty, Notty, Doodle, elm-canvas, diagrams
Per-op struct in variable-stride storage	Flutter, Chromium, Skia, Godot
Open class hierarchy	GSK, Avalonia, Qt Quick
Parallel streams, or tag plus index into arenas	Vello, Masonry/imaging
The result, not the instructions	Ratatui, Textual, Mosaic, Notcurses, imtui, Vty
Generated from the method set	Racket `record-dc%`
Nothing reified	Slint, Qt `QPaintEngine`, Java2D, Pango, piet, Iced, elm-ui, functional images
Reified but unusable	Monomer, imtui, elm-canvas

Three subjects appear twice by design: Vty is a sum-type image algebra that lowers to SpanOp rows, and imtui and elm-canvas each reify and then cash nothing. wgpu has no 2-D command vocabulary and is not placed here.

sparkles:ui sits in the first row: DrawOp is a closed sum over eight per-kind payloads, and its size budget is governed by the widest of them. The three largest reifying subjects sit in the second — Flutter, Chromium and Skia each encode a per-op struct at variable stride, so an operation is charged for its own fields and nothing else.

comparison.md's F3 holds the two halves of the question apart. Reifying the stream at all is what buys recording, replay, culling and comparison, and every subject in the first four rows is evidence for it; the survey affirms that much. How the stream is encoded is the part that stays open: a closed sum eliminates illegal field combinations and leaves each operation an independently comparable value, while variable stride prices each operation at its own width. Four subjects — diagrams, SDL, Masonry/imaging and elm-canvas — press a third axis under the same heading: how many arms a sum should carry, and what a sum guarantees when an illegal state can still be spelled inside one of them.

By how capability is declared (`Q2`)

Declaration	Subjects
Nothing to declare — the contract is total	Flutter (49 pure virtuals), Chromium (45), Avalonia (27), Monomer (45 record fields), Notty (12 closures), GPUI, Gloss, Godot (render seam), egui
A declared feature set, as data	Qt `QPaintEngine`, wgpu, SDL, WebRender, libvaxis, Mosaic, Vty, Avalonia
Type-level — encoded in the language, no capability data	Skia, Java2D, Ratatui, Masonry/imaging, Doodle, diagrams, Iced, Slint, Pango, Racket
A request that fails, per call and argument-dependent	Cairo, SDL, Vg, libvaxis, Ratatui (typed `Result` per method), Masonry/imaging (deferred, stream-scoped)
A runtime probe of the live target	Notcurses (blitter ladder), libvaxis (negotiated `Capabilities` + override env vars), Vty (width table built by interrogating the terminal), Mosaic, Godot `TextServer` (`has_feature`)
Prose only	Vg, piet, Qt Quick
Nothing stated	imtui, elm-ui, Textual, functional images, Vello, elm-canvas

IMPORTANT

Subjects appear in more than one row on purpose. Qt declares a feature set and probes it; libvaxis and Vty both probe the target and publish the result as data; SDL splits a hard-coded floor from a per-domain query and refuses per call. The rows are mechanisms, not camps. GSK belongs to none of them: it substitutes observability (a pink checkerboard under GSK_DEBUG=cairo) for refusal.

By who degrades (`Q3`, and `F4`)

Seven camps, plus one project where degradation escaped the seam altogether.

Where the lowering lives	Subjects
In the backend	Slint, Doodle, Skia, Flutter, Vello, diagrams, Vg
In the framework, once	Qt, Avalonia, Godot, SDL, Java2D, Cairo, Pango, Racket, Notcurses
In the node kind — the fallback travels with it	GSK
Published by the framework, called at the backend's discretion	piet
In the producer, before the stream exists	WebRender
In the widget, before any target exists	Ratatui, Textual, libvaxis, Monomer, Gloss, Mosaic, Iced, egui
Nobody — nothing needs to	GPUI, Notty, Vty, functional images, elm-canvas
Nowhere — unsupported content is silently dropped	Qt Quick (the software adaptation's `// We dont know, so skip`), elm-ui (the browser sizes the thumb)
Upward, into a permanent fork of the widget layer	imtui

Refusal is a separate axis from lowering: Ratatui, Masonry/imaging and Vg report rather than degrade, each at a different granularity (per call, per stream, per offending value). Chromium inverts the direction entirely — the stream declares its own needs and nobody probes anybody — and wgpu has no widgets, so the question does not arise for it.

By whether the scene knows its own extent (`Q8`)

Answer	Subjects
Yes — cached on every node, maintained at construction	GSK, Vty, Notty, Racket pict, Avalonia, diagrams
Yes — accumulated during recording	Flutter, Cairo, Godot, Chromium
Yes — layout already knew it	Doodle, Mosaic, libvaxis, Textual, Ratatui, Monomer, Pango (per layout)
Deliberately refused	Skia, GPUI, WebRender, Vello, Vg, Masonry/imaging
*The extent is an input* to the scene**	Vg, functional images, elm-canvas
Only the surface declares it	Qt `QPaintEngine`, Notcurses, SDL, Java2D, Iced, Gloss, imtui, egui, elm-ui, piet

Slint, Qt Quick and wgpu give no clear answer and are not placed. Twelve subjects publish scene extent, and the live axis is maintained-at-construction versus derived-by-scan: nobody who publishes it derives it by scanning.

I want the conclusions. comparison.md — findings F1–F12, then the verdict table on the eight friction entries, then ten ordered recommendations. Read the verdict table first; it is the index into the rest.

A word in a subject file does not mean what I expect.concepts.md — six clusters pinning "display list", "scene", "render node", "command buffer", "backend", "device", "lowering", "degradation", "advance", "cell width", "feature", "limit" and the rest, each grounded in at least two subjects, with a closing section on what these words mean in sparkles:ui today.

I am writing the sparkles:ui proposal this informs. Ten subjects move a decision; the rest are corroboration. Read them in this order, against comparison.md's recommendations:

elm-ui — why measure can be relocated but not deleted, stated as the exact list of layout decisions that need an answer.
Godot TextServer and Pango and HarfBuzz — what replaces it: a named, capability-declaring measurement peer, and the fact that its answers are device-parameterised.
Ratatui — the failure to avoid (two width functions that disagree) and Vty's remedy (one authoritative table).
Skia SkPicture and Java2D — the whole optional-capability contract expressed in language constructs, and the unforgettable floor.
Chromium cc::PaintOpBuffer and Flutter — how a reified stream stays comparable and replayable while each operation pays only for its own fields, read at the two scales where that trade bites hardest.
GSK — the admission test for a semantic operation, and the node kind that carries its own fallback.
libvaxis — the cheapest answers in the survey to both §5 (publish the cell's pixel size) and §8 (make paint return a size).
imtui — the counterfactual: what happens if the semantics are removed instead of the derived geometry.
piet and Cairo — the two failure post-mortems, and the reason neither failure was about mechanism.
Textual — the accumulator that lets two ops touching one cell agree, which RuleEdge's edge-per-op cannot.

I work on the terminal arm. Ratatui, Textual, libvaxis, Vty, Notty, Notcurses, Mosaic — then imtui for what a GPU-shaped seam does to a cell target.

I work on the GPU arm. Flutter, Chromium, Skia, WebRender, Vello, GPUI — then wgpu for the capability model none of them have.

I want the theory, not the engineering.Functional images, Notty, Gloss, Doodle, diagrams, Vg — the questions dissolve in order, and the order is informative.

Deliberately not surveyed

The exclusions are as load-bearing as the inclusions, so they are recorded with their reasons.

React/DOM-style retained trees where the "backend" is a browser. The DOM answers Q2, Q5 and Q8 on the toolkit's behalf, so a survey entry would report the browser's design, not the toolkit's. Two representatives were kept anyway, and only because each prices the exclusion: elm-ui shows exactly which layout decisions are lost when nothing can measure text, and elm-canvas shows a reified command list dissolving into a stringly-typed record at the actual seam.
Reflex and the FRP web libraries. Considered, and dismissed for the same reason: the seam is the DOM. An FRP layer changes how the tree is built over time, which is a question above this survey's; below it, every one of these libraries hands a browser the same nodes elm-ui does. The two Elm subjects already carry whatever this family had to teach about purity as a constraint.
Game engines whose text story is a texture atlas with no shaping. They have no Q1 to answer and their Q5 is "render at a higher resolution". Godot is the deliberate exception and was admitted for its TextServer — a measurement seam with a live capability query — not for its renderer.
GPU APIs below the 2-D level. wgpu is in the catalog solely as the field's most careful capability model; Vulkan, Metal and D3D were not surveyed, because "what a pixel is" is settled before their seam begins.
Immediate-mode GPU toolkits beyond egui and Dear ImGui. The design is well-represented by egui at the geometry extreme, and the one question worth asking of the family — what happens when it is retargeted to cells — is answered by imtui, which forks Dear ImGui itself.

What the survey must produce

comparison.md — done. The capstone, rewritten against all 38 subjects. Findings F1–F12, a verdict per friction entry, a partial answer to the open question below, and ten ordered recommendations.
concepts.md — done. The shared vocabulary, because the field does not agree on terms: "display list", "scene", "render node" and "command buffer" name at least three different artifacts each across this catalog, and the synthesis is unreadable until they are pinned.
A proposal in docs/specs/ui/ — outstanding. Research states what others did; the proposal states what sparkles:ui will do, with requirement IDs. comparison.md's recommendations are input to it, not a substitute for it: they are ordered so that each is cheap and each unblocks the next, but none of them is a decision until the proposal makes it one.

Constraints the proposal must respect

Recorded from canvas-seam-friction.md's "what did not cause friction" section so the proposal does not spend its budget re-litigating things that work. All four are carried by the evidence across the 38 subjects.

Structural typing over an interface. Attribute inference is load-bearing: a @system GPU canvas and a @safe @nogc recorder satisfy one seam with neither lying. Any replacement keeps this. Strengthened: F5 finds that optional capability needs no capability data at all — Skia encodes floor, negotiable and refusal in ordinary language constructs, and D has all three. Sharpened by F11: the concept, the op kinds, the payloads and the interpreter's probes should be generated from one declaration, so the four artifacts cannot drift again.
Cell-space layout. The toolkit is terminal-first; Q1 is about what measure returns, not about laying out in pixels. Sharpened: F2 finds relocating measure necessary and nowhere near sufficient — unit, oracle, return shape, device parameters and the identity of the measured artifact are five further decisions — and elm-ui proves it can be relocated but not deleted, because neither backend contains a line breaker.
A canonical conforming backend that is also the test seam.RecordingCanvas caught real mistakes and pays for itself. Strengthened by F12: the op stream's value is as the cross-target parity oracle — the artifact that proves the cell grid and the image agree — rather than as the golden itself. Two further refinements from the survey: Avalonia's headless backend deliberately declines optional capabilities Skia grants, so the fallback path is exercised on every run; and Vg argues for generating the written contract from a conformance run rather than writing prose that rots.
The optional-primitive bargain. Probing for pushClip and degrading without it is right; Q2 is that the contract is unstated, not that optionality is wrong. Strengthened: F5 upgrades this from acceptable to the field's ordinary answer — it is simply usually written down, scoped to a domain rather than a global probe, and consumed at the lowering step rather than at each call site.

The open question

Whether the terminal and GPU targets should share one seam at all. The alternative — two seams with a shared layout vocabulary above them — was recorded here as something the survey might not settle.

comparison.md settles the part that matters, and its verdict is: one shared vocabulary above the painters, per-target painters below, and the reified op stream retained as the cross-target parity artifact rather than as the portability abstraction. The evidence against one drawing seam spanning both classes includes failures rather than only designs — imtui's upward fork, Cairo's deprecated GPU backends, piet's signed post-mortem, Qt Quick's silently skipped custom nodes — while the evidence for a shared layer (Mosaic, Textual, Ratatui, Doodle) is consistently about a layer above drawing. That is not a rejection of isCanvas; it is a statement about what the seam should be allowed to grow into, and every friction entry that is a genuine defect is fixable without answering it.

What remains open is a measurement rather than more reading: enumerate every use of every OpKind in sparkles:ui's widget set and ask whether the widget could emit cells and rects instead. Read comparison.md's closing section for the two cases that stay open under that test.

Sources

Every subject file in this directory carries its own citations, each pinned to a 40-character commit SHA per Writing Research Docs; the comparison and concepts pages cite the subject files rather than restating the primary sources. The friction entries under test are canvas-seam-friction.md, and the seam itself is libs/ui/src/sparkles/ui/canvas.d.

Foundations (Theory)

Syntax Highlighting

io_uring Reference

Foundations (Theory)

Manim Community (Python)