#!/usr/bin/env dub
/+ dub.sdl:
name "io_uring_recv_zc"
dependency "during" version="~>0.5.0"
platforms "linux"
targetPath "build"
+/
/**
* `io_uring` — zero-copy receive (`IORING_OP_RECV_ZC` + `IORING_REGISTER_ZCRX_IFQ`, Linux 6.15).
*
* Zero-copy receive (ZCRX) lets the kernel DMA incoming packet payloads straight
* into a userspace memory area pinned to a NIC hardware receive queue, then hand
* the ring a CQE pointing at the data — no `copy_to_user`. To use it you first
* `IORING_REGISTER_ZCRX_IFQ` an `io_uring_zcrx_ifq_reg` describing the netdev
* (`if_idx`) and its RX queue (`if_rxq`) plus a buffer area; afterwards
* `IORING_OP_RECV_ZC` (`prepRecvZc`) receives into that registered ifq, indexed
* by `zcrx_ifq_idx`.
*
* This requires a NIC with a configured ZC RX queue (header-data split, a steered
* RSS context, etc.) — hardware that a typical host or CI runner does NOT have.
* So this example is EXPECTED to take the SKIP path almost everywhere: we attempt
* `registerIfq` with a minimal descriptor and, on the inevitable failure
* (`-EINVAL` / `-EOPNOTSUPP` / `-ENODEV` / `-EPERM` / …), print a `SKIP:` line and
* exit 0. It demonstrates the *shape* of the ZCRX setup call rather than a live
* transfer.
*
* Companion to the io_uring chronology:
* see docs/research/async-io/io-uring/timeline.md
* § "6.15 — Zero-copy receive, epoll-wait, vectored fixed, query".
*
* Run with: `dub run --single recv-zc.d`
*
* Portability: green on any kernel. Old kernels without io_uring (or without the
* RECV_ZC op / ZCRX register opcode) and hosts lacking a ZCRX-capable NIC all take
* the SKIP path and exit 0.
*/
module (module) io_uring_recv_zcio_uring — zero-copy receive (IORING_OP_RECV_ZC + IORING_REGISTER_ZCRX_IFQ, Linux 6.15).
Zero-copy receive (ZCRX) lets the kernel DMA incoming packet payloads straight
into a userspace memory area pinned to a NIC hardware receive queue, then hand
the ring a CQE pointing at the data — no copy_to_user. To use it you first
IORING_REGISTER_ZCRX_IFQ an io_uring_zcrx_ifq_reg describing the netdev
(if_idx) and its RX queue (if_rxq) plus a buffer area; afterwards
IORING_OP_RECV_ZC (prepRecvZc) receives into that registered ifq, indexed
by zcrx_ifq_idx.
This requires a NIC with a configured ZC RX queue (header-data split, a steered
RSS context, etc.) — hardware that a typical host or CI runner does NOT have.
So this example is EXPECTED to take the SKIP path almost everywhere: we attempt
registerIfq with a minimal descriptor and, on the inevitable failure
(-EINVAL / -EOPNOTSUPP / -ENODEV / -EPERM / …), print a SKIP: line and
exit 0. It demonstrates the shape of the ZCRX setup call rather than a live
transfer.
Companion to the io_uring chronology:
see docs/research/async-io/io-uring/timeline.md
§ "6.15 — Zero-copy receive, epoll-wait, vectored fixed, query".
Run with: dub run --single recv-zc.d
Portability
green on any kernel. Old kernels without io_uring (or without the
RECV_ZC op / ZCRX register opcode) and hosts lacking a ZCRX-capable NIC all take
the SKIP path and exit 0.
io_uring_recv_zc;
import (module) duringSimple idiomatic dlang wrapper around linux io_uring
(see: https://kernel.dk/io_uring.pdf) asynchronous API.
during;
import (package) corecore.(package) core.syssys.(package) core.sys.posixposix.(package) core.sys.posix.syssys.(module) core.sys.posix.sys.socketD header file for POSIX.
socket : (alias enum value) io_uring_recv_zc.AF_INET = core.sys.posix.sys.socket.AF_INET = 2AF_INET, (alias enum value) io_uring_recv_zc.SOCK_STREAM = core.sys.posix.sys.socket.SOCK_STREAM = 1SOCK_STREAM, (alias) io_uring_recv_zc.socket = int core.sys.posix.sys.socket.socket(int, int, int) nothrow @nogc @safesocket;
import (package) corecore.(package) core.syssys.(package) core.sys.posixposix.(module) core.sys.posix.unistdD header file for POSIX.
unistd : (alias) io_uring_recv_zc.close = int core.sys.posix.unistd.close(int) nothrow @nogc @trustedclose;
import (package) stdstd.(module) std.stdioCategory Symbols File handles _popen File isFileHandle openNetwork stderr stdin stdout Reading chunks lines readf readfln readln Writing toFile write writef writefln writeln Misc KeepTerminator LockType StdioException
Standard I/O functions that extend core.stdc.stdio. core.stdc.stdio
is publically imported when importing std.stdio.
There are three layers of I/O:
The lowest layer is the operating system layer. The two main schemes are Windows and Posix.
C's stdio.h which unifies the two operating system schemes.
std.stdio, this module, unifies the various stdio.h implementations into
a high level package for D programs.
Source
std/stdio.d
stdio : (alias template) io_uring_recv_zc.writefln = std.stdio.writefln(alias fmt, A...)(A args) if (isSomeString!(typeof(fmt)))Equivalent to writef(fmt, args, '\n').
writefln, stderr;
int int D main()main()
{
(struct) during.UringMain entry point to work with io_uring.
It hides SubmissionQueue and CompletionQueue behind standard range interface.
We put in SubmissionEntry entries and take out CompletionEntry entries.
Use predefined prepXX methods to fill required fields of SubmissionEntry before put or during putWith.
Note
prepXX functions doesn't touch previous entry state, just fills in operation properties. This is because for
less error prone interface it is cleared automatically when prepared using putWith. So when using on own SubmissionEntry
(outside submission queue), that would be added to the submission queue using put, be sure its cleared if it's
reused for multiple operations.
Uring (local variable) during.Uring ioio;
const (local variable) const(int) setupRetsetupRet = (local variable) during.Uring ioio.int during.setup(ref during.Uring uring, uint entries = 128u, during.io_uring.SetupFlags flags = SetupFlags.NONE) nothrow @nogc @safeSetup new instance of io_uring into provided Uring structure.
setup(8);
if ((local variable) const(int) setupRetsetupRet < 0)
{
void std.stdio.writefln!(char, const(int))(in char[] fmt, const(int) __param_1) @safeEquivalent to writef(fmt, args, '\n').
writefln("SKIP: io_uring_setup failed (errno %d) — io_uring unavailable on this host", -(local variable) const(int) setupRetsetupRet);
return 0;
}
// First gate: does this kernel even know IORING_OP_RECV_ZC (op 58, since 6.15)?
// Probing avoids issuing a register call the kernel can't understand.
auto (local variable) during.Probe pp = (local variable) during.Uring ioio.during.Probe during.Uring.probe() nothrow @nogc @safeProbes supported operations
probe();
if (!cast(bool)(local variable) during.Probe pp)
{
void std.stdio.writefln!(char, int)(in char[] fmt, int __param_1) @safeEquivalent to writef(fmt, args, '\n').
writefln("SKIP: io_uring_probe failed (errno %d) — cannot determine RECV_ZC support", -(local variable) during.Probe pp.int during.Probe.error() const pure nothrow @nogc @property @safeError code when we fail to get Probe.
error);
return 0;
}
if (!(local variable) during.Probe pp.bool during.Probe.isSupported(during.io_uring.Operation op) const pure nothrow @nogc @safeIs operation supported?
isSupported((enum) during.io_uring.OperationDescribes the operation to be performed
Operation.(enum value) during.io_uring.Operation.RECV_ZC = cast(ubyte)58uIORING_OP_RECV_ZC - zero-copy receive (requires REGISTER_ZCRX_IFQ)
RECV_ZC))
{
void std.stdio.writefln!char(in char[] fmt) @safeEquivalent to writef(fmt, args, '\n').
writefln("SKIP: IORING_OP_RECV_ZC unsupported on this kernel — needs Linux 6.15+");
return 0;
}
// Second gate: actually try to register a zero-copy RX interface queue.
//
// A *working* registration needs a real netdev index (`if_idx`), one of its
// hardware RX queues (`if_rxq`) put into zero-copy mode, plus an `area_ptr`
// describing a pinned userspace buffer area and a `region_ptr` for the ifq's
// shared refill/completion region. We deliberately pass a minimal descriptor
// (loopback-ish if_idx=1, rq_entries a power of two, no area/region) so we can
// show the call site; on a host without ZCRX hardware this fails, which is the
// expected outcome here.
(struct) during.io_uring.io_uring_zcrx_ifq_regArgument for IORING_REGISTER_ZCRX_IFQ. Pins a NIC hardware receive queue (if_idx/if_rxq)
to this ring for zerocopy receive. area_ptr and region_ptr point at the userspace
buffer area and the io_uring memory region respectively.
Note
Available from Linux 6.11. Functionally usable only on hosts with a supported NIC.
io_uring_zcrx_ifq_reg (local variable) during.io_uring.io_uring_zcrx_ifq_reg regreg;
(local variable) during.io_uring.io_uring_zcrx_ifq_reg regreg.(field) uint during.io_uring.io_uring_zcrx_ifq_reg.if_idxif_idx = 1; // would be a real NIC ifindex (loopback is index 1)
(local variable) during.io_uring.io_uring_zcrx_ifq_reg regreg.(field) uint during.io_uring.io_uring_zcrx_ifq_reg.if_rxqif_rxq = 0; // hardware RX queue id configured for zero-copy
(local variable) during.io_uring.io_uring_zcrx_ifq_reg regreg.(field) uint during.io_uring.io_uring_zcrx_ifq_reg.rq_entriesrq_entries = 256; // refill-queue depth (power of two)
(local variable) during.io_uring.io_uring_zcrx_ifq_reg regreg.(field) uint during.io_uring.io_uring_zcrx_ifq_reg.flagsflags = 0;
(local variable) during.io_uring.io_uring_zcrx_ifq_reg regreg.(field) ulong during.io_uring.io_uring_zcrx_ifq_reg.area_ptrpointer to io_uring_zcrx_area_reg
area_ptr = 0; // -> io_uring_zcrx_area_reg (pinned buffer area)
(local variable) during.io_uring.io_uring_zcrx_ifq_reg regreg.(field) ulong during.io_uring.io_uring_zcrx_ifq_reg.region_ptrpointer to io_uring_region_desc
region_ptr = 0; // -> io_uring_region_desc (shared ifq region)
const (local variable) const(int) regRetregRet = (local variable) during.Uring ioio.int during.Uring.registerIfq(ref scope during.io_uring.io_uring_zcrx_ifq_reg reg) nothrow @nogc @trustedRegister a NIC hardware receive queue for zerocopy receive. Requires a supported NIC
and is hence environment-dependent — the SQE-side counterpart is IORING_OP_RECV_ZC.
Note
Available from Linux 6.11
registerIfq((local variable) during.io_uring.io_uring_zcrx_ifq_reg regreg);
if ((local variable) const(int) regRetregRet < 0)
{
// Any of these is expected on a host without a ZCRX-capable NIC:
// -EINVAL/-EOPNOTSUPP: kernel/driver can't honor this ZCRX request,
// -ENODEV: no such netdev / RX queue,
// -EPERM: insufficient privilege to pin the queue.
void std.stdio.writefln!(char, const(int))(in char[] fmt, const(int) __param_1) @safeEquivalent to writef(fmt, args, '\n').
writefln("SKIP: IORING_REGISTER_ZCRX_IFQ failed (errno %d) — "
~ "zero-copy receive needs a NIC with a configured ZC RX queue "
~ "(header/data split + steered RSS), not available on this host", -(local variable) const(int) regRetregRet);
return 0;
}
// --- Reached only on a genuinely ZCRX-capable host (not this 6.18 box). ---
// The ifq is now registered at index 0 of the ring's zcrx table. Exercise the
// SQE-side counterpart for real: open a loopback socket and post an actual
// IORING_OP_RECV_ZC against ifq index 0. On a live ZCRX setup payloads would
// DMA into the registered area and surface in auxiliary CQEs.
const (local variable) const(int) socksock = int core.sys.posix.sys.socket.socket(int, int, int) nothrow @nogc @safesocket((enum value) core.sys.posix.sys.socket.AF_INET = 2AF_INET, (enum value) core.sys.posix.sys.socket.SOCK_STREAM = 1SOCK_STREAM, 0);
if ((local variable) const(int) socksock < 0)
{
stderr.std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @systemwritefln("socket() failed");
return 1;
}
scope (exit) int core.sys.posix.unistd.close(int) nothrow @nogc @trustedclose((local variable) const(int) socksock);
enum ulong (constant) ulong io_uring_recv_zc.main.cookie = 46202862LUcookie = 0x2C0FFEE;
// Pass `sock` as an explicit arg (not a capture) so `putWith` stays @nogc.
// len=0 lets the kernel size the receive; ifqIdx=0 selects the ifq we registered.
(local variable) during.Uring ioio.putWith!((ref SubmissionEntry e, int recvFd) {
e.prepRecvZc(recvFd, /*len*/ 0, MsgFlags.NONE, /*ifqIdx*/ 0);
e.user_data = cookie;
})(during.Uring during.Uring.putWith!(function (ref during.io_uring.SubmissionEntry e, int recvFd) nothrow @nogc @safe
{
prepRecvZc(e, recvFd, 0u, MsgFlags.NONE, 0u);
e.user_data = 46202862LU;
}
, const(int))(ref const(int) __param_0) nothrow @nogc return ref @safeAdds new entry to the SubmissionQueue.
Note that this just adds entry to the queue and doesn't advance the tail
marker kernel sees. For that finishSq() is needed to be called next.
Also note that to actually enter new entries to kernel,
it's needed to call submit().
sock);
const (local variable) const(int) submittedsubmitted = (local variable) during.Uring ioio.int during.Uring.submit(uint want) nothrow @nogc @safeSubmits qued SubmissionEntry to be processed by kernel.
submit(0); // flush the SQ; don't block waiting for traffic
if ((local variable) const(int) submittedsubmitted < 0)
{
stderr.std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @systemwritefln("submit failed: errno %d", -(local variable) const(int) submittedsubmitted);
return 1;
}
// The ifq registration is torn down when the ring is closed (Uring's destructor),
// so there's nothing more to clean up here.
void std.stdio.writefln!char(in char[] fmt) @safeEquivalent to writef(fmt, args, '\n').
writefln("ok: registered a zero-copy RX interface queue (IORING_REGISTER_ZCRX_IFQ) "
~ "and posted IORING_OP_RECV_ZC — host has ZCRX-capable hardware");
return 0;
}