timeout-link-timeout.dhover×130all
#!/usr/bin/env dub
/+ dub.sdl:
    name "io_uring_timeout"
    dependency "during" version="~>0.5.0"
    platforms "linux"
    targetPath "build"
+/
/**
 * `io_uring` timeouts — standalone `IORING_OP_TIMEOUT` (Linux 5.4) and a chained
 * `IORING_OP_LINK_TIMEOUT` (Linux 5.5).
 *
 * Before timeouts, `io_uring` had no in-kernel notion of "give up after N
 * nanoseconds": you either blocked in `io_uring_enter` or polled. 5.4 added a
 * first-class TIMEOUT op (and the single-`mmap` setup); 5.5 added LINK_TIMEOUT,
 * a timeout *attached* to the preceding linked SQE that cancels it when it fires.
 *
 * Part A — standalone TIMEOUT: arm a ~30ms relative timeout (`count = 0`, so it
 * expires on time rather than after a number of completions) and confirm the CQE
 * reports `-ETIME`.
 *
 * Part B — LINK_TIMEOUT: arm a `POLL_ADD` on the read end of a pipe that never
 * becomes readable (there is no writer), flagged `IO_LINK` so the *next* SQE is
 * linked to it. That next SQE is a `LINK_TIMEOUT` of ~30ms. When the timeout
 * fires it cancels the still-pending poll: the poll CQE comes back `-ECANCELED`
 * and the link-timeout CQE reports `-ETIME` (or `0` on some kernels).
 *
 * Companion to the io_uring chronology:
 * see docs/research/async-io/io-uring/timeline.md § "5.4 — Timeouts and single mmap (November 2019)".
 *
 * Run with: `dub run --single timeout-link-timeout.d`
 *
 * Portability: if the running kernel has no `io_uring` (too old, or blocked by a
 * seccomp/container policy), or if the TIMEOUT/LINK_TIMEOUT ops are unsupported,
 * the program prints a `SKIP:` line and exits 0 so it stays green in CI.
 */
module 
(module) io_uring_timeout

io_uring timeouts — standalone IORING_OP_TIMEOUT (Linux 5.4) and a chained IORING_OP_LINK_TIMEOUT (Linux 5.5).

Before timeouts, io_uring had no in-kernel notion of "give up after N nanoseconds": you either blocked in io_uring_enter or polled. 5.4 added a first-class TIMEOUT op (and the single-mmap setup); 5.5 added LINK_TIMEOUT, a timeout attached to the preceding linked SQE that cancels it when it fires.

Part A — standalone TIMEOUT: arm a ~30ms relative timeout (count = 0, so it expires on time rather than after a number of completions) and confirm the CQE reports -ETIME.

Part B — LINK_TIMEOUT: arm a POLL_ADD on the read end of a pipe that never becomes readable (there is no writer), flagged IO_LINK so the next SQE is linked to it. That next SQE is a LINK_TIMEOUT of ~30ms. When the timeout fires it cancels the still-pending poll: the poll CQE comes back -ECANCELED and the link-timeout CQE reports -ETIME (or 0 on some kernels).

Companion to the io_uring chronology: see docs/research/async-io/io-uring/timeline.md § "5.4 — Timeouts and single mmap (November 2019)".

Run with: dub run --single timeout-link-timeout.d

Portability

if the running kernel has no io_uring (too old, or blocked by a seccomp/container policy), or if the TIMEOUT/LINK_TIMEOUT ops are unsupported, the program prints a SKIP: line and exits 0 so it stays green in CI.

io_uring_timeout
;
import
(module) during

Simple idiomatic dlang wrapper around linux io_uring (see: https://kernel.dk/io_uring.pdf) asynchronous API.

during
;
import
(package) core
core
.
(package) core.sys
sys
.
(package) core.sys.linux
linux
.
(module) core.sys.linux.errno

D header file for GNU/Linux

glibc stdlib/errno.h

errno
:
(alias constant) io_uring_timeout.ETIME = int core.stdc.errno.ETIME = 62
ETIME
,
(alias constant) io_uring_timeout.ECANCELED = int core.stdc.errno.ECANCELED = 125
ECANCELED
;
import
(package) core
core
.
(package) core.sys
sys
.
(package) core.sys.posix
posix
.
(module) core.sys.posix.unistd

D header file for POSIX.

@copyrightCopyright Sean Kelly 2005 - 2009.@licenseBoost License 1.0.@authorsSean Kelly@standardsThe Open Group Base Specifications Issue 8, IEEE Std 1003.1, 2024 Edition
unistd
:
(alias) io_uring_timeout.close = int core.sys.posix.unistd.close(int) nothrow @nogc @trusted
close
,
(alias) io_uring_timeout.pipe = int core.sys.posix.unistd.pipe(ref int[2]) nothrow @nogc @trusted
pipe
;
import
(package) std
std
.
(module) std.stdio
Category Symbols
File handles _popen File isFileHandle openNetwork stderr stdin stdout
Reading chunks lines readf readfln readln
Writing toFile write writef writefln writeln
Misc KeepTerminator LockType StdioException

Standard I/O functions that extend core.stdc.stdio. core.stdc.stdio is publically imported when importing std.stdio.

There are three layers of I/O:

  1. The lowest layer is the operating system layer. The two main schemes are Windows and Posix.

  2. C's stdio.h which unifies the two operating system schemes.

  3. std.stdio, this module, unifies the various stdio.h implementations into a high level package for D programs.

Source

std/stdio.d

@copyrightCopyright The D Language Foundation 2007-.@licenseBoost License 1.0.@authorsWalter Bright, Andrei Alexandrescu, Alex Rønne Petersen
stdio
:
(alias template) io_uring_timeout.writefln = std.stdio.writefln(alias fmt, A...)(A args) if (isSomeString!(typeof(fmt)))

Equivalent to writef(fmt, args, '\n').

writefln
, stderr;
int
int D main()
main
()
{
(struct) during.Uring

Main entry point to work with io_uring.

It hides SubmissionQueue and CompletionQueue behind standard range interface. We put in SubmissionEntry entries and take out CompletionEntry entries.

Use predefined prepXX methods to fill required fields of SubmissionEntry before put or during putWith.

Note

prepXX functions doesn't touch previous entry state, just fills in operation properties. This is because for less error prone interface it is cleared automatically when prepared using putWith. So when using on own SubmissionEntry (outside submission queue), that would be added to the submission queue using put, be sure its cleared if it's reused for multiple operations.

Uring
(local variable) during.Uring io
io
;
const
(local variable) const(int) setupRet
setupRet
=
(local variable) during.Uring io
io
.
int during.setup(ref during.Uring uring, uint entries = 128u, during.io_uring.SetupFlags flags = SetupFlags.NONE) nothrow @nogc @safe

Setup new instance of io_uring into provided Uring structure.

@paramuring Uring structure to be initialized (must not be already initialized)@paramentries Number of entries to initialize uring with@paramflags SetupFlags to use to initialize uring.@returnsOn succes it returns 0, -errno otherwise.
setup
(8);
if (
(local variable) const(int) setupRet
setupRet
< 0)
{
void std.stdio.writefln!(char, const(int))(in char[] fmt, const(int) __param_1) @safe

Equivalent to writef(fmt, args, '\n').

writefln
("SKIP: io_uring_setup failed (errno %d) — io_uring unavailable on this host", -
(local variable) const(int) setupRet
setupRet
);
return 0; } // TIMEOUT (5.4) and LINK_TIMEOUT (5.5) are old enough that almost every // io_uring-capable kernel has them, but probe defensively so CI on the // oldest hosts still degrades to a SKIP rather than a hard failure. auto
(local variable) during.Probe probe
probe
=
(local variable) during.Uring io
io
.
during.Probe during.Uring.probe() nothrow @nogc @safe

Probes supported operations

probe
();
if (cast(bool)
(local variable) during.Probe probe
probe
&&
(!
(local variable) during.Probe probe
probe
.
bool during.Probe.isSupported(during.io_uring.Operation op) const pure nothrow @nogc @safe

Is operation supported?

isSupported
(
(enum) during.io_uring.Operation

Describes the operation to be performed

@seeio_uring_enter(2)
Operation
.
(enum value) during.io_uring.Operation.TIMEOUT = cast(ubyte)11u

IORING_OP_TIMEOUT

TIMEOUT
) || !
(local variable) during.Probe probe
probe
.
bool during.Probe.isSupported(during.io_uring.Operation op) const pure nothrow @nogc @safe

Is operation supported?

isSupported
(
(enum) during.io_uring.Operation

Describes the operation to be performed

@seeio_uring_enter(2)
Operation
.
(enum value) during.io_uring.Operation.LINK_TIMEOUT = cast(ubyte)15u

IORING_OP_LINK_TIMEOUT

LINK_TIMEOUT
)))
{
void std.stdio.writefln!char(in char[] fmt) @safe

Equivalent to writef(fmt, args, '\n').

writefln
("SKIP: TIMEOUT/LINK_TIMEOUT op not supported on this kernel");
return 0; } // ---- Part A: a standalone relative TIMEOUT that should expire with -ETIME ----
(struct) during.io_uring.KernelTimespec

Time specification as defined in kernel headers (used by TIMEOUT operations)

KernelTimespec
(local variable) during.io_uring.KernelTimespec tsA
tsA
;
(local variable) during.io_uring.KernelTimespec tsA
tsA
.
(field) long during.io_uring.KernelTimespec.tv_sec

seconds

tv_sec
= 0;
(local variable) during.io_uring.KernelTimespec tsA
tsA
.
(field) long during.io_uring.KernelTimespec.tv_nsec

nanoseconds

tv_nsec
= 30_000_000; // 30ms
(local variable) during.Uring io
io
.putWith!((ref SubmissionEntry e, ref KernelTimespec t) {
// count = 0 => purely time-based: fire after the duration elapses, not // after N completions. REL => the timespec is relative to "now". e.prepTimeout(t, 0, TimeoutFlags.REL); e.user_data = 1; })(
during.Uring during.Uring.putWith!(function (ref during.io_uring.SubmissionEntry e, ref during.io_uring.KernelTimespec t) nothrow @nogc @safe { prepTimeout(e, t, 0LU, TimeoutFlags.REL); e.user_data = 1LU; } , during.io_uring.KernelTimespec)(ref during.io_uring.KernelTimespec __param_0) nothrow @nogc return ref @safe

Adds new entry to the SubmissionQueue.

Note that this just adds entry to the queue and doesn't advance the tail marker kernel sees. For that finishSq() is needed to be called next.

Also note that to actually enter new entries to kernel, it's needed to call submit().

@paramFN Function to fill next entry in queue by ref (should be faster). It is expected to be in a form of void function(ARGS)(ref SubmissionEntry, auto ref ARGS). Note that in this case queue entry is cleaned first before function is called.@paramentry Custom built SubmissionEntry to be posted as is. Note that in this case it is copied whole over one in the SubmissionQueue.@paramargs Optional arguments passed to the function@returnsreference to Uring structure so it's possible to chain multiple commands.
tsA
);
const
(local variable) const(int) submittedA
submittedA
=
(local variable) during.Uring io
io
.
int during.Uring.submit(uint want) nothrow @nogc @safe

Submits qued SubmissionEntry to be processed by kernel.

@paramwant number of CompletionEntries to wait for. If 0, this just submits queued entries and returns. If > 0, it blocks until at least wanted number of entries were completed.@paramsig See io_uring_enter(2) man page@returnsNumber of submitted entries on success, -errno on error
submit
(1);
if (
(local variable) const(int) submittedA
submittedA
< 0)
{ // -EINVAL here would mean the op shape is unsupported on this kernel. if (-
(local variable) const(int) submittedA
submittedA
== 22 /* EINVAL */)
{
void std.stdio.writefln!char(in char[] fmt) @safe

Equivalent to writef(fmt, args, '\n').

writefln
("SKIP: TIMEOUT submit rejected (EINVAL) — unsupported on this kernel");
return 0; } stderr.
std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @system
writefln
("Part A submit failed: errno %d", -
(local variable) const(int) submittedA
submittedA
);
return 1; }
(local variable) during.Uring io
io
.
int during.Uring.wait(uint want = 1u) nothrow @nogc

Simmilar to submit but with this method we just wait for required number of CompletionEntries.

@returns0 on success, -errno on error
wait
(1);
const
(local variable) const(int) resA
resA
=
(local variable) during.Uring io
io
.
during.io_uring.CompletionEntry during.Uring.front() pure nothrow @nogc return ref @safe

Get first CompletionEntry from cq ring

front
.
(field) int during.io_uring.CompletionEntry.res

result code for this event

res
;
(local variable) during.Uring io
io
.
void during.Uring.popFront() pure nothrow @nogc @safe

Move to next CompletionEntry

popFront
();
if (
(local variable) const(int) resA
resA
== -22 /* -EINVAL */)
{
void std.stdio.writefln!char(in char[] fmt) @safe

Equivalent to writef(fmt, args, '\n').

writefln
("SKIP: TIMEOUT returned -EINVAL — unsupported on this kernel");
return 0; } if (
(local variable) const(int) resA
resA
!= -
(constant) int core.stdc.errno.ETIME = 62
ETIME
)
{ stderr.
std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @system
writefln
("Part A: expected -ETIME (%d), got %d", -
(constant) int core.stdc.errno.ETIME = 62
ETIME
,
(local variable) const(int) resA
resA
);
return 1; } // ---- Part B: POLL_ADD --IO_LINK--> LINK_TIMEOUT; the timeout cancels the poll ---- int[2]
(local variable) int[2] fds
fds
;
if (() @trusted { return
int core.sys.posix.unistd.pipe(ref int[2]) nothrow @nogc @trusted
pipe
(
(local variable) int[2] fds
fds
); }() != 0)
{ stderr.
std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @system
writefln
("pipe() failed");
return 1; } scope (exit) {
int core.sys.posix.unistd.close(int) nothrow @nogc @trusted
close
(
(local variable) int[2] fds
fds
[0]);
int core.sys.posix.unistd.close(int) nothrow @nogc @trusted
close
(
(local variable) int[2] fds
fds
[1]); }
// SQE 1: poll the pipe read end for readability. Nothing is ever written to // the pipe, so on its own this poll would block forever. IO_LINK ties the // *next* SQE to it.
(local variable) during.Uring io
io
.putWith!((ref SubmissionEntry e, int rfd) {
e.prepPollAdd(rfd, PollEvents.IN); e.user_data = 10; e.flags |= SubmissionEntryFlags.IO_LINK; })(
during.Uring during.Uring.putWith!(function (ref during.io_uring.SubmissionEntry e, int rfd) nothrow @nogc @safe { prepPollAdd(e, rfd, PollEvents.IN, PollFlags.NONE); e.user_data = 10LU; cast(int)e.flags |= 4; } , int)(ref int __param_0) nothrow @nogc return ref @safe

Adds new entry to the SubmissionQueue.

Note that this just adds entry to the queue and doesn't advance the tail marker kernel sees. For that finishSq() is needed to be called next.

Also note that to actually enter new entries to kernel, it's needed to call submit().

@paramFN Function to fill next entry in queue by ref (should be faster). It is expected to be in a form of void function(ARGS)(ref SubmissionEntry, auto ref ARGS). Note that in this case queue entry is cleaned first before function is called.@paramentry Custom built SubmissionEntry to be posted as is. Note that in this case it is copied whole over one in the SubmissionQueue.@paramargs Optional arguments passed to the function@returnsreference to Uring structure so it's possible to chain multiple commands.
fds
[0]);
// SQE 2: the link timeout. Because the previous SQE set IO_LINK, this fires // ~30ms after the poll starts and cancels it.
(struct) during.io_uring.KernelTimespec

Time specification as defined in kernel headers (used by TIMEOUT operations)

KernelTimespec
(local variable) during.io_uring.KernelTimespec tsB
tsB
;
(local variable) during.io_uring.KernelTimespec tsB
tsB
.
(field) long during.io_uring.KernelTimespec.tv_sec

seconds

tv_sec
= 0;
(local variable) during.io_uring.KernelTimespec tsB
tsB
.
(field) long during.io_uring.KernelTimespec.tv_nsec

nanoseconds

tv_nsec
= 30_000_000; // 30ms
(local variable) during.Uring io
io
.putWith!((ref SubmissionEntry e, ref KernelTimespec t) {
e.prepLinkTimeout(t, TimeoutFlags.REL); e.user_data = 11; })(
during.Uring during.Uring.putWith!(function (ref during.io_uring.SubmissionEntry e, ref during.io_uring.KernelTimespec t) nothrow @nogc @safe { prepLinkTimeout(e, t, TimeoutFlags.REL); e.user_data = 11LU; } , during.io_uring.KernelTimespec)(ref during.io_uring.KernelTimespec __param_0) nothrow @nogc return ref @safe

Adds new entry to the SubmissionQueue.

Note that this just adds entry to the queue and doesn't advance the tail marker kernel sees. For that finishSq() is needed to be called next.

Also note that to actually enter new entries to kernel, it's needed to call submit().

@paramFN Function to fill next entry in queue by ref (should be faster). It is expected to be in a form of void function(ARGS)(ref SubmissionEntry, auto ref ARGS). Note that in this case queue entry is cleaned first before function is called.@paramentry Custom built SubmissionEntry to be posted as is. Note that in this case it is copied whole over one in the SubmissionQueue.@paramargs Optional arguments passed to the function@returnsreference to Uring structure so it's possible to chain multiple commands.
tsB
);
const
(local variable) const(int) submittedB
submittedB
=
(local variable) during.Uring io
io
.
int during.Uring.submit(uint want) nothrow @nogc @safe

Submits qued SubmissionEntry to be processed by kernel.

@paramwant number of CompletionEntries to wait for. If 0, this just submits queued entries and returns. If > 0, it blocks until at least wanted number of entries were completed.@paramsig See io_uring_enter(2) man page@returnsNumber of submitted entries on success, -errno on error
submit
(2);
if (
(local variable) const(int) submittedB
submittedB
< 0)
{ stderr.
std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @system
writefln
("Part B submit failed: errno %d", -
(local variable) const(int) submittedB
submittedB
);
return 1; } // Both SQEs produce a CQE: the cancelled poll and the fired link-timeout.
(local variable) during.Uring io
io
.
int during.Uring.wait(uint want = 1u) nothrow @nogc

Simmilar to submit but with this method we just wait for required number of CompletionEntries.

@returns0 on success, -errno on error
wait
(2);
int
(local variable) int pollRes
pollRes
= int.
(constant) int int.max = 2147483647
max
;
int
(local variable) int linkRes
linkRes
= int.
(constant) int int.max = 2147483647
max
;
foreach (
(local variable) int _
_
; 0 .. 2)
{ const
(local variable) const(during.io_uring.CompletionEntry) c
c
=
(local variable) during.Uring io
io
.
during.io_uring.CompletionEntry during.Uring.front() pure nothrow @nogc return ref @safe

Get first CompletionEntry from cq ring

front
;
if (
(local variable) const(during.io_uring.CompletionEntry) c
c
.
(field) ulong during.io_uring.CompletionEntry.user_data

sqe->data submission passed back

user_data
== 10)
(local variable) int pollRes
pollRes
=
(local variable) const(during.io_uring.CompletionEntry) c
c
.
(field) int during.io_uring.CompletionEntry.res

result code for this event

res
;
else if (
(local variable) const(during.io_uring.CompletionEntry) c
c
.
(field) ulong during.io_uring.CompletionEntry.user_data

sqe->data submission passed back

user_data
== 11)
(local variable) int linkRes
linkRes
=
(local variable) const(during.io_uring.CompletionEntry) c
c
.
(field) int during.io_uring.CompletionEntry.res

result code for this event

res
;
(local variable) during.Uring io
io
.
void during.Uring.popFront() pure nothrow @nogc @safe

Move to next CompletionEntry

popFront
();
} // The poll must be cancelled by the firing link timeout. if (
(local variable) int pollRes
pollRes
!= -
(constant) int core.stdc.errno.ECANCELED = 125
ECANCELED
)
{ stderr.
std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @system
writefln
("Part B: expected poll res -ECANCELED (%d), got %d", -
(constant) int core.stdc.errno.ECANCELED = 125
ECANCELED
,
(local variable) int pollRes
pollRes
);
return 1; } // The link timeout itself reports -ETIME (it fired) or 0 (kernel variation). if (
(local variable) int linkRes
linkRes
!= -
(constant) int core.stdc.errno.ETIME = 62
ETIME
&&
(local variable) int linkRes
linkRes
!= 0)
{ stderr.
std.stdio.File std.stdio.makeGlobal!"core.stdc.stdio.stderr"() nothrow @nogc @property ref @system
writefln
("Part B: expected link-timeout res -ETIME (%d) or 0, got %d", -
(constant) int core.stdc.errno.ETIME = 62
ETIME
,
(local variable) int linkRes
linkRes
);
return 1; }
void std.stdio.writefln!(char, int, int)(in char[] fmt, int __param_1, int __param_2) @safe

Equivalent to writef(fmt, args, '\n').

writefln
("ok: TIMEOUT expired with -ETIME, and LINK_TIMEOUT cancelled a never-ready poll " ~
"(poll res=%d -ECANCELED, link res=%d)",
(local variable) int pollRes
pollRes
,
(local variable) int linkRes
linkRes
);
return 0; }