doc/man/rseq.2

   1 .\" Copyright 2015-2020 Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
   2 .\"
   3 .\" %%%LICENSE_START(VERBATIM)
   4 .\" Permission is granted to make and distribute verbatim copies of this
   5 .\" manual provided the copyright notice and this permission notice are
   6 .\" preserved on all copies.
   7 .\"
   8 .\" Permission is granted to copy and distribute modified versions of this
   9 .\" manual under the conditions for verbatim copying, provided that the
  10 .\" entire resulting derived work is distributed under the terms of a
  11 .\" permission notice identical to this one.
  12 .\"
  13 .\" Since the Linux kernel and libraries are constantly changing, this
  14 .\" manual page may be incorrect or out-of-date.  The author(s) assume no
  15 .\" responsibility for errors or omissions, or for damages resulting from
  16 .\" the use of the information contained herein.  The author(s) may not
  17 .\" have taken the same level of care in the production of this manual,
  18 .\" which is licensed free of charge, as they might when working
  19 .\" professionally.
  20 .\"
  21 .\" Formatted or processed versions of this manual, if unaccompanied by
  22 .\" the source, must acknowledge the copyright and authors of this work.
  23 .\" %%%LICENSE_END
  24 .\"
  25 .TH RSEQ 2 2020-06-05 "Linux" "Linux Programmer's Manual"
  26 .SH NAME
  27 rseq \- Restartable sequences and cpu number cache
  28 .SH SYNOPSIS
  29 .nf
  30 .B #include <linux/rseq.h>
  31 .sp
  32 .BI "int rseq(struct rseq * " rseq ", uint32_t " rseq_len ", int " flags ", uint32_t " sig ");
  33 .sp
  34 .SH DESCRIPTION
  35 The
  36 .BR rseq ()
  37 ABI accelerates user-space operations on per-cpu data by defining a
  38 shared data structure ABI between each user-space thread and the kernel.
  39
  40 It allows user-space to perform update operations on per-cpu data
  41 without requiring heavy-weight atomic operations.
  42
  43 The term CPU used in this documentation refers to a hardware execution
  44 context.
  45
  46 Restartable sequences are atomic with respect to preemption (making it
  47 atomic with respect to other threads running on the same CPU), as well
  48 as signal delivery (user-space execution contexts nested over the same
  49 thread). They either complete atomically with respect to preemption on
  50 the current CPU and signal delivery, or they are aborted.
  51
  52 It is suited for update operations on per-cpu data.
  53
  54 It can be used on data structures shared between threads within a
  55 process, and on data structures shared between threads across different
  56 processes.
  57
  58 .PP
  59 Some examples of operations that can be accelerated or improved
  60 by this ABI:
  61 .IP \[bu] 2
  62 Memory allocator per-cpu free-lists,
  63 .IP \[bu] 2
  64 Querying the current CPU number,
  65 .IP \[bu] 2
  66 Incrementing per-CPU counters,
  67 .IP \[bu] 2
  68 Modifying data protected by per-CPU spinlocks,
  69 .IP \[bu] 2
  70 Inserting/removing elements in per-CPU linked-lists,
  71 .IP \[bu] 2
  72 Writing/reading per-CPU ring buffers content.
  73 .IP \[bu] 2
  74 Accurately reading performance monitoring unit counters
  75 with respect to thread migration.
  76
  77 .PP
  78 Restartable sequences must not perform system calls. Doing so may result
  79 in termination of the process by a segmentation fault.
  80
  81 .PP
  82 The
  83 .I rseq
  84 argument is a pointer to the thread-local rseq structure to be shared
  85 between kernel and user-space.
  86
  87 .PP
  88 The layout of
  89 .B struct rseq
  90 is as follows:
  91 .TP
  92 .B Structure alignment
  93 This structure is aligned on 32-byte boundary.
  94 .TP
  95 .B Structure size
  96 This structure is extensible. Its size is passed as parameter to the
  97 rseq system call.
  98 .TP
  99 .B Fields
 100
 101 .TP
 102 .in +4n
 103 .I cpu_id_start
 104 Optimistic cache of the CPU number on which the current thread is
 105 running. Its value is guaranteed to always be a possible CPU number,
 106 even when rseq is not initialized. The value it contains should always
 107 be confirmed by reading the cpu_id field.
 108
 109 This field is an optimistic cache in the sense that it is always
 110 guaranteed to hold a valid CPU number in the range [ 0 ..
 111 nr_possible_cpus - 1 ]. It can therefore be loaded by user-space and
 112 used as an offset in per-cpu data structures without having to
 113 check whether its value is within the valid bounds compared to the
 114 number of possible CPUs in the system.
 115
 116 For user-space applications executed on a kernel without rseq support,
 117 the cpu_id_start field stays initialized at 0, which is indeed a valid
 118 CPU number. It is therefore valid to use it as an offset in per-cpu data
 119 structures, and only validate whether it's actually the current CPU
 120 number by comparing it with the cpu_id field within the rseq critical
 121 section. If the kernel does not provide rseq support, that cpu_id field
 122 stays initialized at -1, so the comparison always fails, as intended.
 123
 124 It is then up to user-space to use a fall-back mechanism, considering
 125 that rseq is not available.
 126 .in
 127 .TP
 128 .in +4n
 129 .I cpu_id
 130 Cache of the CPU number on which the current thread is running.
 131 -1 if uninitialized.
 132 .in
 133 .TP
 134 .in +4n
 135 .I rseq_cs
 136 The rseq_cs field is a pointer to a struct rseq_cs. Is is NULL when no
 137 rseq assembly block critical section is active for the current thread.
 138 Setting it to point to a critical section descriptor (struct rseq_cs)
 139 marks the beginning of the critical section.
 140 .in
 141 .TP
 142 .in +4n
 143 .I flags
 144 Flags indicating the restart behavior for the current thread. This is
 145 mainly used for debugging purposes. Can be either:
 146 .IP \[bu]
 147 RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT
 148 .IP \[bu]
 149 RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL
 150 .IP \[bu]
 151 RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE
 152 .in
 153
 154 .PP
 155 The layout of
 156 .B struct rseq_cs
 157 version 0 is as follows:
 158 .TP
 159 .B Structure alignment
 160 This structure is aligned on 32-byte boundary.
 161 .TP
 162 .B Structure size
 163 This structure has a fixed size of 32 bytes.
 164 .TP
 165 .B Fields
 166
 167 .TP
 168 .in +4n
 169 .I version
 170 Version of this structure.
 171 .in
 172 .TP
 173 .in +4n
 174 .I flags
 175 Flags indicating the restart behavior of this structure. Can be
 176 a combination of:
 177 .IP \[bu]
 178 RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT
 179 .IP \[bu]
 180 RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL
 181 .IP \[bu]
 182 RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE
 183 .TP
 184 .in +4n
 185 .I start_ip
 186 Instruction pointer address of the first instruction of the sequence of
 187 consecutive assembly instructions.
 188 .in
 189 .TP
 190 .in +4n
 191 .I post_commit_offset
 192 Offset (from start_ip address) of the address after the last instruction
 193 of the sequence of consecutive assembly instructions.
 194 .in
 195 .TP
 196 .in +4n
 197 .I abort_ip
 198 Instruction pointer address where to move the execution flow in case of
 199 abort of the sequence of consecutive assembly instructions.
 200 .in
 201
 202 .PP
 203 The
 204 .I rseq_len
 205 argument is the size of the
 206 .I struct rseq
 207 to register.
 208
 209 .PP
 210 The
 211 .I flags
 212 argument is 0 for registration, and
 213 .IR RSEQ_FLAG_UNREGISTER
 214 for unregistration.
 215
 216 .PP
 217 The
 218 .I sig
 219 argument is the 32-bit signature to be expected before the abort
 220 handler code.
 221
 222 .PP
 223 A single library per process should keep the rseq structure in a
 224 thread-local storage variable.
 225 The
 226 .I cpu_id
 227 field should be initialized to -1, and the
 228 .I cpu_id_start
 229 field should be initialized to a possible CPU value (typically 0).
 230
 231 .PP
 232 Each thread is responsible for registering and unregistering its rseq
 233 structure. No more than one rseq structure address can be registered
 234 per thread at a given time.
 235
 236 .PP
 237 Memory of a registered rseq object must not be freed before the thread
 238 exits. Reclaim of rseq object's memory must only be done after either an
 239 explicit rseq unregistration is performed or after the thread exits. Keep
 240 in mind that the implementation of the Thread-Local Storage (C language
 241 __thread) lifetime does not guarantee existence of the TLS area up until
 242 the thread exits.
 243
 244 .PP
 245 In a typical usage scenario, the thread registering the rseq
 246 structure will be performing loads and stores from/to that structure. It
 247 is however also allowed to read that structure from other threads.
 248 The rseq field updates performed by the kernel provide relaxed atomicity
 249 semantics, which guarantee that other threads performing relaxed atomic
 250 reads of the cpu number cache will always observe a consistent value.
 251
 252 .SH RETURN VALUE
 253 A return value of 0 indicates success. On error, \-1 is returned, and
 254 .I errno
 255 is set appropriately.
 256
 257 .SH ERRORS
 258 .TP
 259 .B EINVAL
 260 Either
 261 .I flags
 262 contains an invalid value, or
 263 .I rseq
 264 contains an address which is not appropriately aligned, or
 265 .I rseq_len
 266 contains a size that does not match the size received on registration.
 267 .TP
 268 .B ENOSYS
 269 The
 270 .BR rseq ()
 271 system call is not implemented by this kernel.
 272 .TP
 273 .B EFAULT
 274 .I rseq
 275 is an invalid address.
 276 .TP
 277 .B EBUSY
 278 Restartable sequence is already registered for this thread.
 279 .TP
 280 .B EPERM
 281 The
 282 .I sig
 283 argument on unregistration does not match the signature received
 284 on registration.
 285
 286 .SH VERSIONS
 287 The
 288 .BR rseq ()
 289 system call was added in Linux 4.18.
 290
 291 .SH CONFORMING TO
 292 .BR rseq ()
 293 is Linux-specific.
 294
 295 .in
 296 .SH SEE ALSO
 297 .BR sched_getcpu (3) ,
 298 .BR membarrier (2)