#!/usr/bin/env dub
/+ dub.sdl:
name "io_uring_sqpoll"
dependency "during" version="~>0.5.0"
platforms "linux"
targetPath "build"
+/
/**
* `io_uring` — kernel-side submission polling (`IORING_SETUP_SQPOLL`, Linux 5.1).
*
* With `SQPOLL` the kernel spawns a side thread that *polls* the submission
* queue tail. Once that thread is running and awake, userspace can hand it work
* simply by writing an SQE and advancing the shared SQ tail — needing **no
* `io_uring_enter` syscall at all** to submit. That syscall-free submission is
* the whole point of SQPOLL, and this program demonstrates it directly in two
* phases:
*
* Phase 1 — prime the poll thread with one ordinary `submit`+`wait`. A freshly
* created SQPOLL ring's poll thread does not pick up work until it
* has been woken at least once, so this first `io_uring_enter`
* (which carries `IORING_ENTER_SQ_WAKEUP` when needed) gets it
* spinning. We confirm the NOP round-trips.
*
* Phase 2 — THE PAYOFF: stage a second NOP and publish it with `Uring.flush()`,
* which only advances the shared SQ tail pointer in user memory and
* issues *no* syscall whatsoever. The now-awake poll thread observes
* the new tail and submits the entry on its own; we observe the
* completion by busy-polling the completion queue (so we never block
* on a syscall either). This is genuine, syscall-free submission.
*
* The tradeoff: the kernel thread burns a CPU while it spins, parking only after
* `sq_thread_idle` ms of inactivity. Once parked it raises `IORING_SQ_NEED_WAKEUP`
* and the next submission must ring a doorbell (`io_uring_enter` with
* `IORING_ENTER_SQ_WAKEUP`) to wake it — so SQPOLL trades CPU for latency: a win
* for a saturated I/O path, wasteful for a bursty one. We pick a large
* `sq_thread_idle` so the thread stays awake across phase 2's short window.
*
* A `NOP` touches no file descriptor, so this needs no registered-files table.
*
* Companion to the io_uring chronology:
* see docs/research/async-io/io-uring/timeline.md § "5.1 — The introduction".
*
* Run with: `dub run --single sqpoll.d`
*
* Portability: SQPOLL has historically required privilege (`CAP_SYS_ADMIN` /
* `CAP_SYS_NICE`); on a stricter kernel `setup` returns `-EPERM`, and a host
* without io_uring (or this flag) returns `-ENOSYS`/`-EINVAL`/`-EOPNOTSUPP`. In
* every such case the program prints a `SKIP:` line and exits 0 so it stays
* green in CI regardless of host kernel/privileges.
*/
module (module) io_uring_sqpollio_uring — kernel-side submission polling (IORING_SETUP_SQPOLL, Linux 5.1).
With SQPOLL the kernel spawns a side thread that polls the submission
queue tail. Once that thread is running and awake, userspace can hand it work
simply by writing an SQE and advancing the shared SQ tail — needing no
io_uring_enter syscall at all to submit. That syscall-free submission is
the whole point of SQPOLL, and this program demonstrates it directly in two
phases:
Phase 1 — prime the poll thread with one ordinary submit+wait. A freshly
created SQPOLL ring's poll thread does not pick up work until it
has been woken at least once, so this first io_uring_enter
(which carries IORING_ENTER_SQ_WAKEUP when needed) gets it
spinning. We confirm the NOP round-trips.
Phase 2 — THE PAYOFF: stage a second NOP and publish it with Uring.flush(),
which only advances the shared SQ tail pointer in user memory and
issues no syscall whatsoever. The now-awake poll thread observes
the new tail and submits the entry on its own; we observe the
completion by busy-polling the completion queue (so we never block
on a syscall either). This is genuine, syscall-free submission.
The tradeoff: the kernel thread burns a CPU while it spins, parking only after
sq_thread_idle ms of inactivity. Once parked it raises IORING_SQ_NEED_WAKEUP
and the next submission must ring a doorbell (io_uring_enter with
IORING_ENTER_SQ_WAKEUP) to wake it — so SQPOLL trades CPU for latency: a win
for a saturated I/O path, wasteful for a bursty one. We pick a large
sq_thread_idle so the thread stays awake across phase 2's short window.
A NOP touches no file descriptor, so this needs no registered-files table.
Companion to the io_uring chronology:
see docs/research/async-io/io-uring/timeline.md § "5.1 — The introduction".
Run with: dub run --single sqpoll.d
Portability
SQPOLL has historically required privilege (CAP_SYS_ADMIN /
CAP_SYS_NICE); on a stricter kernel setup returns -EPERM, and a host
without io_uring (or this flag) returns -ENOSYS/-EINVAL/-EOPNOTSUPP. In
every such case the program prints a SKIP: line and exits 0 so it stays
green in CI regardless of host kernel/privileges.
io_uring_sqpoll;
import (module) duringSimple idiomatic dlang wrapper around linux io_uring
(see: https://kernel.dk/io_uring.pdf) asynchronous API.
during;
import (package) corecore.(package) core.syssys.(package) core.sys.linuxlinux.(module) core.sys.linux.errnoD header file for GNU/Linux
errno : (alias constant) io_uring_sqpoll.EPERM = int core.stdc.errno.EPERM = 1EPERM, (alias constant) io_uring_sqpoll.EINVAL = int core.stdc.errno.EINVAL = 22EINVAL, (alias constant) io_uring_sqpoll.ENOSYS = int core.stdc.errno.ENOSYS = 38ENOSYS, (alias constant) io_uring_sqpoll.EOPNOTSUPP = int core.stdc.errno.EOPNOTSUPP = 95EOPNOTSUPP;
import (package) corecore.(module) core.threadThe thread module provides support for thread creation and management.
Source
core/thread/package.d
thread : (class) core.thread.osthread.ThreadThis class encapsulates all threading functionality for the D
programming language. As thread manipulation is a required facility
for garbage collection, all user threads should derive from this
class, and instances of this class should never be explicitly deleted.
A new thread may be created using either derivation or composition, as
in the following example.
Thread;
import (package) corecore.(module) core.timeModule containing core time functionality, such as Duration (which
represents a duration of time) or MonoTime (which represents a
timestamp of the system's monotonic clock).
Various functions take a string (or strings) to represent a unit of time
(e.g. convert!("days", "hours")(numDays)). The valid strings to use
with such functions are "years", "months", "weeks", "days", "hours",
"minutes", "seconds", "msecs" (milliseconds), "usecs" (microseconds),
"hnsecs" (hecto-nanoseconds - i.e. 100 ns) or some subset thereof. There
are a few functions that also allow "nsecs", but very little actually
has precision greater than hnsecs.
Symbol Description Types Duration Represents a duration of time of weeks or less (kept internally as hnsecs). (e.g. 22 days or 700 seconds). TickDuration DEPRECATED Represents a duration of time in system clock ticks, using the highest precision that the system provides. MonoTime Represents a monotonic timestamp in system clock ticks, using the highest precision that the system provides. Functions convert Generic way of converting between two time units. dur Allows constructing a Duration from the given time units with the given length. weeks days hours
minutes seconds msecs
usecs hnsecs nsecs |
Convenience aliases for dur. |
| abs | Returns the absolute value of a duration. |
From Duration
From TickDuration
From units
To Duration tickDuration.to, std,conv!Duration() dur!"msecs"(5) or 5.msecs()
| To TickDuration |
duration.to, std,conv!TickDuration() |
|
TickDuration.from!"msecs"(msecs) |
| To units |
duration.total!"days" |
tickDuration.msecs |
convert!("days", "msecs")(msecs) |
Source
core/time.d
time : msecs;
import (package) stdstd.(module) std.stdioCategory Symbols File handles _popen File isFileHandle openNetwork stderr stdin stdout Reading chunks lines readf readfln readln Writing toFile write writef writefln writeln Misc KeepTerminator LockType StdioException
Standard I/O functions that extend core.stdc.stdio. core.stdc.stdio
is publically imported when importing std.stdio.
There are three layers of I/O:
The lowest layer is the operating system layer. The two main schemes are Windows and Posix.
C's stdio.h which unifies the two operating system schemes.
std.stdio, this module, unifies the various stdio.h implementations into
a high level package for D programs.
Source
std/stdio.d
stdio : (alias template) io_uring_sqpoll.writefln = std.stdio.writefln(alias fmt, A...)(A args) if (isSomeString!(typeof(fmt)))Equivalent to writef(fmt, args, '\n').
writefln, stderr;
int int D main()main()
{
enum ulong (constant) ulong io_uring_sqpoll.main.primeCookie = 364561LUprimeCookie = 0x5_9011UL; // phase 1 correlation cookie
enum ulong (constant) ulong io_uring_sqpoll.main.pollCookie = 364562LUpollCookie = 0x5_9012UL; // phase 2 (syscall-free) correlation cookie
(struct) during.UringMain entry point to work with io_uring.
It hides SubmissionQueue and CompletionQueue behind standard range interface.
We put in SubmissionEntry entries and take out CompletionEntry entries.
Use predefined prepXX methods to fill required fields of SubmissionEntry before put or during putWith.
Note
prepXX functions doesn't touch previous entry state, just fills in operation properties. This is because for
less error prone interface it is cleared automatically when prepared using putWith. So when using on own SubmissionEntry
(outside submission queue), that would be added to the submission queue using put, be sure its cleared if it's
reused for multiple operations.
Uring (local variable) during.Uring ioio;
// Request SQPOLL via the SetupParameters overload so we can tune the poll
// thread's idle timeout. `sq_thread_idle` (ms) is how long the kernel poll
// thread stays awake before parking and raising NEED_WAKEUP. We pick a value
// far larger than this program's runtime, guaranteeing the thread is still
// awake during phase 2 — so that submission stays purely syscall-free. We
// deliberately omit SQ_AFF (CPU pinning), which would add a needless point of
// failure on constrained CI hosts.
(struct) during.io_uring.SetupParametersPassed in for io_uring_setup(2). Copied back with updated info on success.
C API: struct io_uring_params
SetupParameters (local variable) during.io_uring.SetupParameters paramsparams;
(local variable) during.io_uring.SetupParameters paramsparams.(field) during.io_uring.SetupFlags during.io_uring.SetupParameters.flags(input)
flags = (enum) during.io_uring.SetupFlagsio_uring_setup() flags
SetupFlags.(enum value) during.io_uring.SetupFlags.SQPOLL = 2uIORING_SETUP_SQPOLL
When this flag is specified, a kernel thread is created to perform submission queue polling.
An io_uring instance configured in this way enables an application to issue I/O without ever
context switching into the kernel.
By using the submission queue to fill in new submission queue entries and watching for
completions on the completion queue, the application can submit and reap I/Os without doing
a single system call.
If the kernel thread is idle for more than sq_thread_idle microseconds, it will set the
IORING_SQ_NEED_WAKEUP bit in the flags field of the struct io_sq_ring. When this happens,
the application must call io_uring_enter(2) to wake the kernel thread. If I/O is kept busy,
the kernel thread will never sleep. An application making use of this feature will need to
guard the io_uring_enter(2) call with the following code sequence:
// Ensure that the wakeup flag is read after the tail pointer has been written.
smp_mb();
if (*sq_ring->flags & IORING_SQ_NEED_WAKEUP)
io_uring_enter(fd, 0, 0, IORING_ENTER_SQ_WAKEUP);
where sq_ring is a submission queue ring setup using the struct io_sqring_offsets described below.
To successfully use this feature, the application must register a set of files to be used for
IO through io_uring_register(2) using the IORING_REGISTER_FILES opcode. Failure to do so will
result in submitted IO being errored with EBADF.```
SQPOLL;
(local variable) during.io_uring.SetupParameters paramsparams.(field) uint during.io_uring.SetupParameters.sq_thread_idle(input) used if SQPOLL flag is active; timeout in milliseconds
until kernel poll thread goes to sleep.
sq_thread_idle = 10_000; // 10 s — keeps the poll thread awake for the run
const (local variable) const(int) setupRetsetupRet = (local variable) during.Uring ioio.int during.setup(ref during.Uring uring, uint entries, ref const(during.io_uring.SetupParameters) params) nothrow @nogc @safeSetup new instance of io_uring into provided Uring structure.
setup(8, (local variable) during.io_uring.SetupParameters paramsparams);
if ((local variable) const(int) setupRetsetupRet < 0)
{
// -EPERM: SQPOLL denied (needs privilege). -EINVAL/-ENOSYS/-EOPNOTSUPP:
// io_uring or this specific setup flag is unavailable. All are "not
// supported here", so SKIP and exit 0 rather than fail.
const (local variable) const(int) ee = -(local variable) const(int) setupRetsetupRet;
if ((local variable) const(int) ee == (constant) int core.stdc.errno.EPERM = 1EPERM)
void std.stdio.writefln!char(in char[] fmt) @safeEquivalent to writef(fmt, args, '\n').
writefln("SKIP: IORING_SETUP_SQPOLL denied (EPERM) — needs CAP_SYS_ADMIN/CAP_SYS_NICE");
else if ((local variable) const(int) ee == (constant) int core.stdc.errno.EINVAL = 22EINVAL || (local variable) const(int) ee == (constant) int core.stdc.errno.ENOSYS = 38ENOSYS || (local variable) const(int) ee == (constant) int core.stdc.errno.EOPNOTSUPP = 95EOPNOTSUPP)
void std.stdio.writefln!(char, const(int))(in char[] fmt, const(int) __param_1) @safeEquivalent to writef(fmt, args, '\n').
writefln("SKIP: IORING_SETUP_SQPOLL unsupported (errno %d) — older kernel or no io_uring", (local variable) const(int) ee);
else
void std.stdio.writefln!(char, const(int))(in char[] fmt, const(int) __param_1) @safeEquivalent to writef(fmt, args, '\n').
writefln("SKIP: SQPOLL setup failed (errno %d) — io_uring unavailable on this host", (local variable) const(int) ee);
return 0;
}
// ---- Phase 1: prime the poll thread ------------------------------------
// Stage a NOP and submit-and-wait normally. This single `io_uring_enter`
// wakes the freshly created poll thread (passing SQ_WAKEUP if it was parked).
(local variable) during.Uring ioio.putWith!((ref SubmissionEntry e) {
e.prepNop();
e.user_data = primeCookie;
})();
const (local variable) const(int) primedprimed = (local variable) during.Uring ioio.int during.Uring.submit(uint want) nothrow @nogc @safeSubmits qued SubmissionEntry to be processed by kernel.
submit(1); // submit + wait for 1 completion
if ((local variable) const(int) primedprimed < 0)
{
stderr.std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @systemwritefln("phase 1 submit failed: errno %d", -(local variable) const(int) primedprimed);
return 1;
}
(local variable) during.Uring ioio.int during.Uring.wait(uint want = 1u) nothrow @nogcSimmilar to submit but with this method we just wait for required number
of CompletionEntries.
wait(1);
{
const (local variable) const(int) resres = (local variable) during.Uring ioio.during.io_uring.CompletionEntry during.Uring.front() pure nothrow @nogc return ref @safeGet first CompletionEntry from cq ring
front.(field) int during.io_uring.CompletionEntry.resresult code for this event
res;
const (local variable) const(ulong) echoedechoed = (local variable) during.Uring ioio.during.io_uring.CompletionEntry during.Uring.front() pure nothrow @nogc return ref @safeGet first CompletionEntry from cq ring
front.(field) ulong during.io_uring.CompletionEntry.user_datasqe->data submission passed back
user_data;
(local variable) during.Uring ioio.void during.Uring.popFront() pure nothrow @nogc @safeMove to next CompletionEntry
popFront();
if ((local variable) const(int) resres < 0)
{
stderr.std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @systemwritefln("phase 1 NOP completed with error: errno %d", -(local variable) const(int) resres);
return 1;
}
if ((local variable) const(ulong) echoedechoed != (constant) ulong io_uring_sqpoll.main.primeCookie = 364561LUprimeCookie)
{
stderr.std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @systemwritefln("phase 1 user_data mismatch: expected 0x%X, got 0x%X", (constant) ulong io_uring_sqpoll.main.primeCookie = 364561LUprimeCookie, (local variable) const(ulong) echoedechoed);
return 1;
}
}
// ---- Phase 2: syscall-free submission ----------------------------------
// The poll thread is now spinning. Stage a second NOP and publish it with
// `flush()` — which only advances the shared SQ tail pointer in user memory.
// There is NO io_uring_enter syscall here; the kernel poll thread picks the
// entry up on its own. We then busy-poll the completion queue (no blocking
// syscall) so the entire phase-2 round-trip happens without entering the
// kernel from userspace at all.
(local variable) during.Uring ioio.putWith!((ref SubmissionEntry e) {
e.prepNop();
e.user_data = pollCookie;
})();
(local variable) during.Uring ioio.void during.Uring.flush() nothrow @nogc @safeFlushes submission queue index to the kernel.
Doesn't call any syscall, it just advances the SQE queue for kernel.
This can be used with IORING_SETUP_SQPOLL when kernel polls the submission queue.
flush(); // <-- the syscall-free publish
bool (local variable) bool gotgot;
// Hard bound: 400 * 1 ms = 400 ms cap (well under the 2 s budget). On an awake
// poll thread the NOP completes in well under a millisecond.
foreach ((local variable) int __; 0 .. 400)
{
if ((local variable) during.Uring ioio.ulong during.Uring.length() const pure nothrow @nogc @safeNumber of entries in completion queue
length > 0) { (local variable) bool gotgot = true; break; }
(class) core.thread.osthread.ThreadThis class encapsulates all threading functionality for the D
programming language. As thread manipulation is a required facility
for garbage collection, all user threads should derive from this
class, and instances of this class should never be explicitly deleted.
A new thread may be created using either derivation or composition, as
in the following example.
Thread.void core.thread.osthread.Thread.sleep(core.time.Duration val) nothrow @nogc @trustedSuspends the calling thread for at least the supplied period. This may
result in multiple OS calls if period is greater than the maximum sleep
duration supported by the operating system.
In
period must be non-negative.
Example
Thread.sleep( dur!("msecs")( 50 ) ); // sleep for 50 milliseconds
Thread.sleep( dur!("seconds")( 5 ) ); // sleep for 5 seconds
sleep(1.msecs);
}
if (!(local variable) bool gotgot)
{
// The awake poll thread should have submitted+completed our NOP without a
// syscall. Failing to within a generous bound is a genuine SQPOLL failure.
stderr.std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @systemwritefln("SQPOLL poll thread did not pick up the flush()-only submission within the budget");
return 1;
}
const (local variable) const(int) resres = (local variable) during.Uring ioio.during.io_uring.CompletionEntry during.Uring.front() pure nothrow @nogc return ref @safeGet first CompletionEntry from cq ring
front.(field) int during.io_uring.CompletionEntry.resresult code for this event
res;
const (local variable) const(ulong) echoedechoed = (local variable) during.Uring ioio.during.io_uring.CompletionEntry during.Uring.front() pure nothrow @nogc return ref @safeGet first CompletionEntry from cq ring
front.(field) ulong during.io_uring.CompletionEntry.user_datasqe->data submission passed back
user_data;
(local variable) during.Uring ioio.void during.Uring.popFront() pure nothrow @nogc @safeMove to next CompletionEntry
popFront();
if ((local variable) const(int) resres < 0)
{
stderr.std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @systemwritefln("phase 2 NOP completed with error: errno %d", -(local variable) const(int) resres);
return 1;
}
if ((local variable) const(ulong) echoedechoed != (constant) ulong io_uring_sqpoll.main.pollCookie = 364562LUpollCookie)
{
stderr.std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @systemwritefln("phase 2 user_data mismatch: expected 0x%X, got 0x%X", (constant) ulong io_uring_sqpoll.main.pollCookie = 364562LUpollCookie, (local variable) const(ulong) echoedechoed);
return 1;
}
void std.stdio.writefln!(char, const(int), const(ulong))(in char[] fmt, const(int) __param_1, const(ulong) __param_2) @safeEquivalent to writef(fmt, args, '\n').
writefln("ok: SQPOLL kernel thread completed a NOP submitted via a syscall-free flush() "
~ "(res=%d); user_data 0x%X round-tripped", (local variable) const(int) resres, (local variable) const(ulong) echoedechoed);
return 0;
}