#!/usr/bin/env dub
/+ dub.sdl:
name "io_uring_timeout"
dependency "during" version="~>0.5.0"
platforms "linux"
targetPath "build"
+/
/**
* `io_uring` timeouts — standalone `IORING_OP_TIMEOUT` (Linux 5.4) and a chained
* `IORING_OP_LINK_TIMEOUT` (Linux 5.5).
*
* Before timeouts, `io_uring` had no in-kernel notion of "give up after N
* nanoseconds": you either blocked in `io_uring_enter` or polled. 5.4 added a
* first-class TIMEOUT op (and the single-`mmap` setup); 5.5 added LINK_TIMEOUT,
* a timeout *attached* to the preceding linked SQE that cancels it when it fires.
*
* Part A — standalone TIMEOUT: arm a ~30ms relative timeout (`count = 0`, so it
* expires on time rather than after a number of completions) and confirm the CQE
* reports `-ETIME`.
*
* Part B — LINK_TIMEOUT: arm a `POLL_ADD` on the read end of a pipe that never
* becomes readable (there is no writer), flagged `IO_LINK` so the *next* SQE is
* linked to it. That next SQE is a `LINK_TIMEOUT` of ~30ms. When the timeout
* fires it cancels the still-pending poll: the poll CQE comes back `-ECANCELED`
* and the link-timeout CQE reports `-ETIME` (or `0` on some kernels).
*
* Companion to the io_uring chronology:
* see docs/research/async-io/io-uring/timeline.md § "5.4 — Timeouts and single mmap (November 2019)".
*
* Run with: `dub run --single timeout-link-timeout.d`
*
* Portability: if the running kernel has no `io_uring` (too old, or blocked by a
* seccomp/container policy), or if the TIMEOUT/LINK_TIMEOUT ops are unsupported,
* the program prints a `SKIP:` line and exits 0 so it stays green in CI.
*/
module (module) io_uring_timeoutio_uring timeouts — standalone IORING_OP_TIMEOUT (Linux 5.4) and a chained
IORING_OP_LINK_TIMEOUT (Linux 5.5).
Before timeouts, io_uring had no in-kernel notion of "give up after N
nanoseconds": you either blocked in io_uring_enter or polled. 5.4 added a
first-class TIMEOUT op (and the single-mmap setup); 5.5 added LINK_TIMEOUT,
a timeout attached to the preceding linked SQE that cancels it when it fires.
Part A — standalone TIMEOUT: arm a ~30ms relative timeout (count = 0, so it
expires on time rather than after a number of completions) and confirm the CQE
reports -ETIME.
Part B — LINK_TIMEOUT: arm a POLL_ADD on the read end of a pipe that never
becomes readable (there is no writer), flagged IO_LINK so the next SQE is
linked to it. That next SQE is a LINK_TIMEOUT of ~30ms. When the timeout
fires it cancels the still-pending poll: the poll CQE comes back -ECANCELED
and the link-timeout CQE reports -ETIME (or 0 on some kernels).
Companion to the io_uring chronology:
see docs/research/async-io/io-uring/timeline.md § "5.4 — Timeouts and single mmap (November 2019)".
Run with: dub run --single timeout-link-timeout.d
Portability
if the running kernel has no io_uring (too old, or blocked by a
seccomp/container policy), or if the TIMEOUT/LINK_TIMEOUT ops are unsupported,
the program prints a SKIP: line and exits 0 so it stays green in CI.
io_uring_timeout;
import (module) duringSimple idiomatic dlang wrapper around linux io_uring
(see: https://kernel.dk/io_uring.pdf) asynchronous API.
during;
import (package) corecore.(package) core.syssys.(package) core.sys.linuxlinux.(module) core.sys.linux.errnoD header file for GNU/Linux
errno : (alias constant) io_uring_timeout.ETIME = int core.stdc.errno.ETIME = 62ETIME, (alias constant) io_uring_timeout.ECANCELED = int core.stdc.errno.ECANCELED = 125ECANCELED;
import (package) corecore.(package) core.syssys.(package) core.sys.posixposix.(module) core.sys.posix.unistdD header file for POSIX.
unistd : (alias) io_uring_timeout.close = int core.sys.posix.unistd.close(int) nothrow @nogc @trustedclose, (alias) io_uring_timeout.pipe = int core.sys.posix.unistd.pipe(ref int[2]) nothrow @nogc @trustedpipe;
import (package) stdstd.(module) std.stdioCategory Symbols File handles _popen File isFileHandle openNetwork stderr stdin stdout Reading chunks lines readf readfln readln Writing toFile write writef writefln writeln Misc KeepTerminator LockType StdioException
Standard I/O functions that extend core.stdc.stdio. core.stdc.stdio
is publically imported when importing std.stdio.
There are three layers of I/O:
The lowest layer is the operating system layer. The two main schemes are Windows and Posix.
C's stdio.h which unifies the two operating system schemes.
std.stdio, this module, unifies the various stdio.h implementations into
a high level package for D programs.
Source
std/stdio.d
stdio : (alias template) io_uring_timeout.writefln = std.stdio.writefln(alias fmt, A...)(A args) if (isSomeString!(typeof(fmt)))Equivalent to writef(fmt, args, '\n').
writefln, stderr;
int int D main()main()
{
(struct) during.UringMain entry point to work with io_uring.
It hides SubmissionQueue and CompletionQueue behind standard range interface.
We put in SubmissionEntry entries and take out CompletionEntry entries.
Use predefined prepXX methods to fill required fields of SubmissionEntry before put or during putWith.
Note
prepXX functions doesn't touch previous entry state, just fills in operation properties. This is because for
less error prone interface it is cleared automatically when prepared using putWith. So when using on own SubmissionEntry
(outside submission queue), that would be added to the submission queue using put, be sure its cleared if it's
reused for multiple operations.
Uring (local variable) during.Uring ioio;
const (local variable) const(int) setupRetsetupRet = (local variable) during.Uring ioio.int during.setup(ref during.Uring uring, uint entries = 128u, during.io_uring.SetupFlags flags = SetupFlags.NONE) nothrow @nogc @safeSetup new instance of io_uring into provided Uring structure.
setup(8);
if ((local variable) const(int) setupRetsetupRet < 0)
{
void std.stdio.writefln!(char, const(int))(in char[] fmt, const(int) __param_1) @safeEquivalent to writef(fmt, args, '\n').
writefln("SKIP: io_uring_setup failed (errno %d) — io_uring unavailable on this host", -(local variable) const(int) setupRetsetupRet);
return 0;
}
// TIMEOUT (5.4) and LINK_TIMEOUT (5.5) are old enough that almost every
// io_uring-capable kernel has them, but probe defensively so CI on the
// oldest hosts still degrades to a SKIP rather than a hard failure.
auto (local variable) during.Probe probeprobe = (local variable) during.Uring ioio.during.Probe during.Uring.probe() nothrow @nogc @safeProbes supported operations
probe();
if (cast(bool) (local variable) during.Probe probeprobe &&
(!(local variable) during.Probe probeprobe.bool during.Probe.isSupported(during.io_uring.Operation op) const pure nothrow @nogc @safeIs operation supported?
isSupported((enum) during.io_uring.OperationDescribes the operation to be performed
Operation.(enum value) during.io_uring.Operation.TIMEOUT = cast(ubyte)11uIORING_OP_TIMEOUT
TIMEOUT) || !(local variable) during.Probe probeprobe.bool during.Probe.isSupported(during.io_uring.Operation op) const pure nothrow @nogc @safeIs operation supported?
isSupported((enum) during.io_uring.OperationDescribes the operation to be performed
Operation.(enum value) during.io_uring.Operation.LINK_TIMEOUT = cast(ubyte)15uIORING_OP_LINK_TIMEOUT
LINK_TIMEOUT)))
{
void std.stdio.writefln!char(in char[] fmt) @safeEquivalent to writef(fmt, args, '\n').
writefln("SKIP: TIMEOUT/LINK_TIMEOUT op not supported on this kernel");
return 0;
}
// ---- Part A: a standalone relative TIMEOUT that should expire with -ETIME ----
(struct) during.io_uring.KernelTimespecTime specification as defined in kernel headers (used by TIMEOUT operations)
KernelTimespec (local variable) during.io_uring.KernelTimespec tsAtsA;
(local variable) during.io_uring.KernelTimespec tsAtsA.(field) long during.io_uring.KernelTimespec.tv_secseconds
tv_sec = 0;
(local variable) during.io_uring.KernelTimespec tsAtsA.(field) long during.io_uring.KernelTimespec.tv_nsecnanoseconds
tv_nsec = 30_000_000; // 30ms
(local variable) during.Uring ioio.putWith!((ref SubmissionEntry e, ref KernelTimespec t) {
// count = 0 => purely time-based: fire after the duration elapses, not
// after N completions. REL => the timespec is relative to "now".
e.prepTimeout(t, 0, TimeoutFlags.REL);
e.user_data = 1;
})(during.Uring during.Uring.putWith!(function (ref during.io_uring.SubmissionEntry e, ref during.io_uring.KernelTimespec t) nothrow @nogc @safe
{
prepTimeout(e, t, 0LU, TimeoutFlags.REL);
e.user_data = 1LU;
}
, during.io_uring.KernelTimespec)(ref during.io_uring.KernelTimespec __param_0) nothrow @nogc return ref @safeAdds new entry to the SubmissionQueue.
Note that this just adds entry to the queue and doesn't advance the tail
marker kernel sees. For that finishSq() is needed to be called next.
Also note that to actually enter new entries to kernel,
it's needed to call submit().
tsA);
const (local variable) const(int) submittedAsubmittedA = (local variable) during.Uring ioio.int during.Uring.submit(uint want) nothrow @nogc @safeSubmits qued SubmissionEntry to be processed by kernel.
submit(1);
if ((local variable) const(int) submittedAsubmittedA < 0)
{
// -EINVAL here would mean the op shape is unsupported on this kernel.
if (-(local variable) const(int) submittedAsubmittedA == 22 /* EINVAL */)
{
void std.stdio.writefln!char(in char[] fmt) @safeEquivalent to writef(fmt, args, '\n').
writefln("SKIP: TIMEOUT submit rejected (EINVAL) — unsupported on this kernel");
return 0;
}
stderr.std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @systemwritefln("Part A submit failed: errno %d", -(local variable) const(int) submittedAsubmittedA);
return 1;
}
(local variable) during.Uring ioio.int during.Uring.wait(uint want = 1u) nothrow @nogcSimmilar to submit but with this method we just wait for required number
of CompletionEntries.
wait(1);
const (local variable) const(int) resAresA = (local variable) during.Uring ioio.during.io_uring.CompletionEntry during.Uring.front() pure nothrow @nogc return ref @safeGet first CompletionEntry from cq ring
front.(field) int during.io_uring.CompletionEntry.resresult code for this event
res;
(local variable) during.Uring ioio.void during.Uring.popFront() pure nothrow @nogc @safeMove to next CompletionEntry
popFront();
if ((local variable) const(int) resAresA == -22 /* -EINVAL */)
{
void std.stdio.writefln!char(in char[] fmt) @safeEquivalent to writef(fmt, args, '\n').
writefln("SKIP: TIMEOUT returned -EINVAL — unsupported on this kernel");
return 0;
}
if ((local variable) const(int) resAresA != -(constant) int core.stdc.errno.ETIME = 62ETIME)
{
stderr.std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @systemwritefln("Part A: expected -ETIME (%d), got %d", -(constant) int core.stdc.errno.ETIME = 62ETIME, (local variable) const(int) resAresA);
return 1;
}
// ---- Part B: POLL_ADD --IO_LINK--> LINK_TIMEOUT; the timeout cancels the poll ----
int[2] (local variable) int[2] fdsfds;
if (() @trusted { return int core.sys.posix.unistd.pipe(ref int[2]) nothrow @nogc @trustedpipe((local variable) int[2] fdsfds); }() != 0)
{
stderr.std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @systemwritefln("pipe() failed");
return 1;
}
scope (exit) { int core.sys.posix.unistd.close(int) nothrow @nogc @trustedclose((local variable) int[2] fdsfds[0]); int core.sys.posix.unistd.close(int) nothrow @nogc @trustedclose((local variable) int[2] fdsfds[1]); }
// SQE 1: poll the pipe read end for readability. Nothing is ever written to
// the pipe, so on its own this poll would block forever. IO_LINK ties the
// *next* SQE to it.
(local variable) during.Uring ioio.putWith!((ref SubmissionEntry e, int rfd) {
e.prepPollAdd(rfd, PollEvents.IN);
e.user_data = 10;
e.flags |= SubmissionEntryFlags.IO_LINK;
})(during.Uring during.Uring.putWith!(function (ref during.io_uring.SubmissionEntry e, int rfd) nothrow @nogc @safe
{
prepPollAdd(e, rfd, PollEvents.IN, PollFlags.NONE);
e.user_data = 10LU;
cast(int)e.flags |= 4;
}
, int)(ref int __param_0) nothrow @nogc return ref @safeAdds new entry to the SubmissionQueue.
Note that this just adds entry to the queue and doesn't advance the tail
marker kernel sees. For that finishSq() is needed to be called next.
Also note that to actually enter new entries to kernel,
it's needed to call submit().
fds[0]);
// SQE 2: the link timeout. Because the previous SQE set IO_LINK, this fires
// ~30ms after the poll starts and cancels it.
(struct) during.io_uring.KernelTimespecTime specification as defined in kernel headers (used by TIMEOUT operations)
KernelTimespec (local variable) during.io_uring.KernelTimespec tsBtsB;
(local variable) during.io_uring.KernelTimespec tsBtsB.(field) long during.io_uring.KernelTimespec.tv_secseconds
tv_sec = 0;
(local variable) during.io_uring.KernelTimespec tsBtsB.(field) long during.io_uring.KernelTimespec.tv_nsecnanoseconds
tv_nsec = 30_000_000; // 30ms
(local variable) during.Uring ioio.putWith!((ref SubmissionEntry e, ref KernelTimespec t) {
e.prepLinkTimeout(t, TimeoutFlags.REL);
e.user_data = 11;
})(during.Uring during.Uring.putWith!(function (ref during.io_uring.SubmissionEntry e, ref during.io_uring.KernelTimespec t) nothrow @nogc @safe
{
prepLinkTimeout(e, t, TimeoutFlags.REL);
e.user_data = 11LU;
}
, during.io_uring.KernelTimespec)(ref during.io_uring.KernelTimespec __param_0) nothrow @nogc return ref @safeAdds new entry to the SubmissionQueue.
Note that this just adds entry to the queue and doesn't advance the tail
marker kernel sees. For that finishSq() is needed to be called next.
Also note that to actually enter new entries to kernel,
it's needed to call submit().
tsB);
const (local variable) const(int) submittedBsubmittedB = (local variable) during.Uring ioio.int during.Uring.submit(uint want) nothrow @nogc @safeSubmits qued SubmissionEntry to be processed by kernel.
submit(2);
if ((local variable) const(int) submittedBsubmittedB < 0)
{
stderr.std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @systemwritefln("Part B submit failed: errno %d", -(local variable) const(int) submittedBsubmittedB);
return 1;
}
// Both SQEs produce a CQE: the cancelled poll and the fired link-timeout.
(local variable) during.Uring ioio.int during.Uring.wait(uint want = 1u) nothrow @nogcSimmilar to submit but with this method we just wait for required number
of CompletionEntries.
wait(2);
int (local variable) int pollRespollRes = int.(constant) int int.max = 2147483647max;
int (local variable) int linkReslinkRes = int.(constant) int int.max = 2147483647max;
foreach ((local variable) int __; 0 .. 2)
{
const (local variable) const(during.io_uring.CompletionEntry) cc = (local variable) during.Uring ioio.during.io_uring.CompletionEntry during.Uring.front() pure nothrow @nogc return ref @safeGet first CompletionEntry from cq ring
front;
if ((local variable) const(during.io_uring.CompletionEntry) cc.(field) ulong during.io_uring.CompletionEntry.user_datasqe->data submission passed back
user_data == 10) (local variable) int pollRespollRes = (local variable) const(during.io_uring.CompletionEntry) cc.(field) int during.io_uring.CompletionEntry.resresult code for this event
res;
else if ((local variable) const(during.io_uring.CompletionEntry) cc.(field) ulong during.io_uring.CompletionEntry.user_datasqe->data submission passed back
user_data == 11) (local variable) int linkReslinkRes = (local variable) const(during.io_uring.CompletionEntry) cc.(field) int during.io_uring.CompletionEntry.resresult code for this event
res;
(local variable) during.Uring ioio.void during.Uring.popFront() pure nothrow @nogc @safeMove to next CompletionEntry
popFront();
}
// The poll must be cancelled by the firing link timeout.
if ((local variable) int pollRespollRes != -(constant) int core.stdc.errno.ECANCELED = 125ECANCELED)
{
stderr.std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @systemwritefln("Part B: expected poll res -ECANCELED (%d), got %d", -(constant) int core.stdc.errno.ECANCELED = 125ECANCELED, (local variable) int pollRespollRes);
return 1;
}
// The link timeout itself reports -ETIME (it fired) or 0 (kernel variation).
if ((local variable) int linkReslinkRes != -(constant) int core.stdc.errno.ETIME = 62ETIME && (local variable) int linkReslinkRes != 0)
{
stderr.std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @systemwritefln("Part B: expected link-timeout res -ETIME (%d) or 0, got %d", -(constant) int core.stdc.errno.ETIME = 62ETIME, (local variable) int linkReslinkRes);
return 1;
}
void std.stdio.writefln!(char, int, int)(in char[] fmt, int __param_1, int __param_2) @safeEquivalent to writef(fmt, args, '\n').
writefln("ok: TIMEOUT expired with -ETIME, and LINK_TIMEOUT cancelled a never-ready poll " ~
"(poll res=%d -ECANCELED, link res=%d)", (local variable) int pollRespollRes, (local variable) int linkReslinkRes);
return 0;
}