rseq percpu alloc: Implement numa support

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I732b632f476ffef362a1ab486bcf425e4ded6644

Implement per-cpu memory allocator

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I54f43ccec743b7c5311221fac22a0b690a863377

Introduce __RSEQ_ASM_DEFINE_CS_FIELDS

The new macro __RSEQ_ASM_DEFINE_CS_FIELDS can be used by
__RSEQ_ASM_DEFINE_TABLE as well as __RSEQ_ASM_DEFINE_ABORT on
arm and mips, thus removing redundant code.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ie03f5d51d20027a3eb3b9eba636b0f5cc890c333

Introduce common generic header file

Move:

-  RSEQ_ASM_DEFINE_TABLE,
- __RSEQ_ASM_DEFINE_TABLE,
- RSEQ_ASM_DEFINE_EXIT_POINT.

to a generic common header file now that those are exact copy across all
architectures.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I1f27e8889b13d263467a1bab92b272cc80aa0743

x86: Introduce RSEQ_ASM_U32

This will allow moving __RSEQ_ASM_DEFINE_TABLE and
RSEQ_ASM_DEFINE_EXIT_POINT to a generic common header file.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I966b67f00bf39bd0b26f8efd6c4f40cba4caa1cf

riscv: Introduce RSEQ_ASM_U64_PTR and RSEQ_ASM_U32

These will allow moving __RSEQ_ASM_DEFINE_TABLE and
RSEQ_ASM_DEFINE_EXIT_POINT to a generic common header file.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I9cac7076b9ef9bd7dd4a818b9ad7aae1df45ff5a

s390: Introduce RSEQ_ASM_U32

This will allow moving __RSEQ_ASM_DEFINE_TABLE and
RSEQ_ASM_DEFINE_EXIT_POINT to a generic common header file.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I0af09a3fa3970efbe523db04154f1969241369a1

ppc: Introduce RSEQ_ASM_U32

This will allow moving __RSEQ_ASM_DEFINE_TABLE and
RSEQ_ASM_DEFINE_EXIT_POINT to a generic common header file.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ic476d6dfca6f714cf892f84560884157abbc0360

mips: Introduce RSEQ_ASM_U32

This will allow moving __RSEQ_ASM_DEFINE_TABLE and
RSEQ_ASM_DEFINE_EXIT_POINT to a generic common header file.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I5a68ce2782bc36d5fc8c030ca7716e41e925cd54

arm: Introduce RSEQ_ASM_U32

This will allow moving __RSEQ_ASM_DEFINE_TABLE and
RSEQ_ASM_DEFINE_EXIT_POINT to a generic common header file.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I32a0eff75998afb1f8257c3312187ab2938d59a3

aarch64: Introduce RSEQ_ASM_U32

This will allow moving __RSEQ_ASM_DEFINE_TABLE and
RSEQ_ASM_DEFINE_EXIT_POINT to a generic common header file.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I4496c41d37094983b724fc8bb101688f75fefbbd

thread-pointer: suffix include guard with _H

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Id35f1cb103e1d68bde10785e44c5c16104f000d4

arm: Fix big endian, introduce RSEQ_ASM_U64_PTR

Fix support for Big endian ARM by introducing RSEQ_ASM_U64_PTR.

This will allow moving __RSEQ_ASM_DEFINE_TABLE and
RSEQ_ASM_DEFINE_EXIT_POINT to a generic common header file.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I6271f217b0e1463968bd7d520ddd0fb61678bb84

arm: Cleanup header codying style

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Iab1aee9103eef2179fcdc8c2171afda3d311379b

aarch64: Introduce RSEQ_ASM_U64_PTR

This will allow moving __RSEQ_ASM_DEFINE_TABLE and
RSEQ_ASM_DEFINE_EXIT_POINT to a generic common header file.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ic5470bb851be1b79bbba387a4db2ce0911bbe23d

aarch64: Cleanup header coding style

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I7986cf871be81fc8bb94d0d44d9f61ee41ba0fc1

s390: Introduce code deduplication macros

Introduce RSEQ_ASM_U64_PTR() which expresses the type and instance
of the defined variable, allowing deduplication of the following macros:

- __RSEQ_ASM_DEFINE_TABLE,
- RSEQ_ASM_DEFINE_EXIT_POINT.

Separate load/store/op helper macros, RSEQ_ASM_STORE_RSEQ_CS, and
RSEQ_ASM_U64_PTR into their respective ifdef regions, as the resulting
documentation is clearer (and not duplicated).

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I37c481c9eb726b49dfdf00539c42d68bb500b2c7

ppc: Introduce code deduplication macros

Introduce RSEQ_ASM_U64_PTR() which expresses the type and instance
of the defined variable, allowing deduplication of the following macros:

- __RSEQ_ASM_DEFINE_TABLE,
- RSEQ_ASM_DEFINE_EXIT_POINT.

Separate load/store/op helper macros, RSEQ_ASM_STORE_RSEQ_CS, and
RSEQ_ASM_U64_PTR into their respective ifdef regions, as the resulting
documentation is clearer (and not duplicated).

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I7accb4925e1e294f2d65267641fd4c8dccb4837f

x86: Introduce code deduplication macros

Introduce RSEQ_ASM_U64_PTR() which expresses the type and instance
of the defined variable, allowing deduplication of the following macros:

- __RSEQ_ASM_DEFINE_TABLE,
- RSEQ_ASM_DEFINE_EXIT_POINT.

Separate RSEQ_ASM_TP_SEGMENT, RSEQ_ASM_STORE_RSEQ_CS, and
RSEQ_ASM_U64_PTR into their respective ifdef regions, as the resulting
documentation is clearer (and not duplicated).

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I13e90120f9f50bb898f99f545c3eb205fe9d8437

mips: Reorganise code deduplication macros

Introduce RSEQ_ASM_U64_PTR() which expresses the type and instance
of the defined variable.

Separate the RSEQ_ASM_LONG* macros from the 64-bit pointer types, as
those two are unrelated. Having them in the same vincinity is
misleading.

Test on RSEQ_BITS_PER_LONG rather than _MIPS_SZLONG to make it
consistent across architectures.

Use both __BYTE_ORDER and __BIG_ENDIAN to detect byte order.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I1a65210e9ab0c00a5625c9feba7971081eb34c35

Re-organise public headers

The structure is inspired by the liburcu project.

- Remove the double namespacing 'rseq/rseq-'
- Add a structured architecture support layout
- Add missing guards on arch headers
- Split the main rseq.h header into utils.h and inject.h
- Add the always_inline attribute to all static inline functions

Change-Id: I6a244e5f364d28d5d8c3d426d6039c4d8aa15a36
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix supported autotools versions in README.md

Change-Id: I07257e51544546a8c39325dd05a0a0b99b47a5e2
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

RSEQ_ASM_DEFINE_TABLE: use parentheses around parameters on all archs

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I09ad49a2eb132768d9fca4547d602240bc3d84a4

Comment s390 macros implementation

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ifaeeed3e7721b9c0f758abef63c286a4d22e5268

s390: namespace public header macros under RSEQ_ASM_

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I777a7b3feb9361c9dbc5327f643fc2d09aaa238e

Comment riscv macros implementation

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I4299061a38443bff9ff0701b73752153d2c3df54

riscv: Add missing "inc" parameter to RSEQ_ASM_OP_R_DEREF_ADDV

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Iff9fb8f46acef741a22065a6c0b976794c4606dd

riscv: namespace public header macros under RSEQ_ASM_

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: If8750ceaee9ca4d30309c974e675336ea4290d19

mips: Move RSEQ_ASM_DEFINE_ABORT table_label parameter to 4th argument

Makes the first 3 arguments same as other architectures.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Icf41ded4ce5860cf8c12a23dff1fe434210844b0

arm: Move RSEQ_ASM_DEFINE_ABORT table_label parameter to 4th argument

Makes the first 3 arguments same as other architectures.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I855d645ec5e315ef19836adebfd2f9350a3763c5

Comment mips macros implementation

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I6a1646a8cc18cf249b7323afc9ceb0600f001649

Comment ppc macros implementation

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I0b51030c416d875de296ba7f4490bd59ba810f72

Comment arm64 macros implementation

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Id918fa702bfbee014766f949329d1068a0404340

Comment arm macros implementation

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ibb74479b2219cf2da92a17034785f94cb7dba932

Comment x86 asm macros implementation

Combine common code between x86-32/x86-64. Comment asm macro
implementation for both architectures.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: If6bb763edd4821173babfbc3ac1557f4ae576a86

mips: Namespace type helpers under RSEQ_ASM_*

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: If509780615849c0bf69b36a4363a6ff146b22c50

ppc: Rename RSEQ_ASM_OP_R_MEMCPY to RSEQ_ASM_OP_R_BYTEWISE_MEMCPY

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I5ef55c2d23a0cd5b62c0276698a9ccf8e839fc26

riscv: Rename RSEQ_ASM_OP_R_BAD_MEMCPY to RSEQ_ASM_OP_R_BYTEWISE_MEMCPY

This is more descriptive.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I3f397cc63f60c2d806b1665de9b6bae4106f1db1

arm64: Rename RSEQ_ASM_OP_R_BAD_MEMCPY to RSEQ_ASM_OP_R_BYTEWISE_MEMCPY

This is more descriptive.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ic5b33693ae39729859ea734721ffd323a7151c68

ppc: Rename RSEQ_ helpers to RSEQ_ASM_

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I0baec337248866688b2534164a29ab525c18265f

Rename RSEQ_TEMPLATE macros

For added clarity, rename:

RSEQ_TEMPLATE_CPU_ID -> RSEQ_TEMPLATE_INDEX_CPU_ID
RSEQ_TEMPLATE_MM_CID -> RSEQ_TEMPLATE_INDEX_MM_CID
RSEQ_TEMPLATE_CPU_ID_NONE -> RSEQ_TEMPLATE_INDEX_NONE

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ie3c80d15755b295543ae41b86e8879a61d7a8863

x86: Use RSEQ_ASM_ prefix for RSEQ_CPU_ID_OFFSET, RSEQ_CS_OFFSET, RSEQ_MM_CID_OFFSET

This makes this clearly internal and not part of the public API.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I695af5866ac8bb8f1b62c705df85f96edaa40fa0

x86-32: Remove support for old IDT WinChip and PPRO

The rseq system call first appeared in Linux v4.18, which did not
support the old IDT WinChip Intel clones and PentiumPro CPUs anymore.
Therefore, remove CPU barrier instructions from the smp_rmb() and
smp_wmb() macros given that librseq will realistically never be used on
such hardware.

Expect all x86-32 hardware to be TSO.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I07a03006e2eb8ec0928c63f16df94ad42d440b10

Fix: seccomp tests require shared libs

Disable the seccomp dependent tests when the project is built without
shared library support.

Change-Id: If17caf056c612a22b649281f25775f55546074f1
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

x86-64: change jnz/jz for jne/je after cmp/test

Those are the exact same Jcc instructions (they both depend on the ZF),
but it's clearer to use jne/je after cmp/test rather than jnz/jz.

Note that the jnz instruction is kept when it is used to check the
result of a dec instruction.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ibc2e5902b3047cb291d11d82b7e9dfa530a46891

Rename RSEQ_ASM_DEFINE_CMPFAIL to RSEQ_ASM_DEFINE_TEARDOWN

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ic386511c72e99b0818c6ff29fd2996d110296e59

Rename branch labels from cmpfail to ne/eq

This is done to match the pseudo-code.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I56e2bd0c838e16ef2de459da7d15493dd191df4a

Update rseq critical section identifiers to match pseudo-code

Apply the following mapping:

rseq_cmpeqv_storev -> rseq_load_cbne_store__ptr
rseq_addv -> rseq_load_add_store__ptr
rseq_cmpnev_storeoffp_load -> rseq_load_cbeq_store_add_load_store__ptr
rseq_offset_deref_addv -> rseq_load_add_load_add_store__ptr
rseq_cmpeqv_cmpeqv_storev -> rseq_load_cbne_load_cbne_store__ptr
rseq_cmpeqv_trystorev_storev -> rseq_load_cbne_store_store__ptr
rseq_cmpeqv_trymemcpy_storev -> rseq_load_cbne_memcpy_store__ptr

Summary of the identifier convention:
- Begin with an "rseq_" prefix,
- Followed by the pseudo-code,
- Followed by __ and the type (or eventually types) on which the API
  applies (similar to the approach taken for C++ mangling).

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I59f251bcf177779ee5f8eb18c629dcd8bf1d633a

Add rseq critical section pseudocode documentation

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ic7bd7df3566401562a41ffc8112fe95d740c0016

tests: add unregistered and no syscall tests

Add tests to validate the state of the public symbols when no
registration has occured and when the rseq syscall is unavailable.

The no syscall test uses seccomp to deny access to rseq, libseccomp is
added as an optionnal build dependency.

Change-Id: I0bf6249fa13bd39af80c21a9d2892cd132644e69
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

fix: always set the rseq offset and flags

Even when the rseq syscall is unavailable, set the __rseq_offset to the
correct value as application or library code might try to check the
registration status by reading the cpu_id.

Change-Id: I8f579af236e17a45dfb71c3310bc41a9c17652fd
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Add 'teardown' parameter to RSEQ_ASM_DEFINE_ABORT on all arch

For consistency, add the 'teardown' parameter to the
RSEQ_ASM_DEFINE_ABORT macro on architectures where it was missing.

Change-Id: I2c4bc46fc0f4a4e4ba0f704950753bf6485794d3
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Add comments to rseq_init()

Change-Id: Ie81eb51d0892e7f84db5b57476c8b14c3a6f5940
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

doc: fix typo in rseq(2) man page

Change-Id: I52573c813418fd6d4961e3780cfb7c054f5519f2
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Update autotools archive macros

Change-Id: I2e200e6c5e9b14e2cfeeda6319f00a4d1795f6f2
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Clarify logic of RSEQ_ASM*_CMP* macros

The RSEQ_ASM*_CMP* macros come from the x86 implementation which has
"RSEQ_ASM_CMP_CPU_ID()" and static inline functions such as
"rseq_cmpeqv_storev()". The meaning of the "cmp" here is that the
critical section does _not_ abort (does not branch) if the comparison
matches.

But I understand how the ASM helpers that were contributed for other
architectures such as "RSEQ_ASM_OP_CMPEQ()", with the same semantic
of "do not branch to abort if the comparison matches" can be misleading
for someone used to reading assembler on pretty much any architecture,
where the conditional branch is expected to be taken if the condition
matches. So what I have here in librseq is backwards.

Fortunately, librseq is still just a master branch (no releases yet),
and the copy in the Linux kernel selftests is internal to that selftest,
so there are no stable API expectations at this stage.

So I don't think the semantic of e.g. "rseq_cmpeqv_storev()" is
misleading: it proceeds to do the store if the comparison matches.

However, the ASM macros would benefit from a logic flip.

Introduce this in a way that will allow users of the API to catch the
change at compile-time.

This commit applies the following remapping of the macros for added
clarity:

RSEQ_ASM_OP_CMPNE becomes RSEQ_ASM_OP_CBEQ (compare and branch if equal)
RSEQ_ASM_OP_CMPEQ becomes RSEQ_ASM_OP_CBNE (compare and branch if not equal)
RSEQ_ASM_CMP_CPU_ID becomes RSEQ_ASM_CBNE_CPU_ID (compare and branch if cpu id is not equal)

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I13bff7e22569b11565a23aca87cf571431d57106

Document alignment of rseq_abi for allocated size

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ie78f475a2d6f8756b166db45d96367104871c2a8

Fix: do not skip !allowed_cpus for mm_cid

Indexing with mm_cid is incompatible with skipping disallowed cpumask.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I138882eef8a5a447e82dddeca9df73047987545f

Add arm64 headers to list of headers to be installed

Currently these aren't being installed leading to compile failures on
arm64 when trying to include rseq.h after a `make install`.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Id8bb911ad4157e710ee2731436c92756f73cc1be

riscv: Fix __always_inline causes duplicate inline

In file included from /usr/include/features.h:503,
from /usr/include/bits/libc-header-start.h:33,
from /usr/include/stdint.h:26,
from /usr/lib64/gcc/riscv64-suse-linux/13/include/stdint.h:9,
from ../../../tests/unit/arch-mo.c:5,
from ../../../tests/unit/arch-mo-cxx.cpp:5:
../../../include/rseq/rseq-riscv-bits.h:9:15: error: duplicate 'inline'
9 | static inline __always_inline
| ^~~~~~~~~~~~~~~

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Iabf16e221e6a84ac31163d58c7e6edf2e87c5d18

Add missing copyright headers

Change-Id: Icb6343e541608ec00b0c86c6f6c7231abffce25e
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Remove whiteline

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I745c5e1daef3575e1a5201e0232f451c0f832c64

Introduce rseq_scalar_type_to_expr to eliminate code duplication

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I6fd0aa556b4f486db5d5fb86fd78e3cb1ff9c35b

fix: compiler warning `-Wswitch-enum`

Fixes warning "enumeration value ‘XXXX’ not handled in switch
[-Wswitch-enum]" which was observed on gcc 11.3.0 on Ubuntu 22.04 LTS
Linux 6.3.0 x86-64. This warning was enabled by the flag `-Wswitch-enum`
(see:
https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wswitch-enum)

Signed-off-by: Gerrit Klein <therealthingy95@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I86cde0be01bf32cb5f6ea0ad313001bba717a9ff

fix: compiler warning `-Wswitch-default`

Fixes warning "switch missing default case [-Wswitch-default]" which was
observed on gcc 11.3.0 on Ubuntu 22.04 LTS Linux 6.3.0 x86-64. This
warning was enabled by the flag `-Wswitch-default` (see:
https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wswitch-default)

Signed-off-by: Gerrit Klein <therealthingy95@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Iad356fd94b0b964a49c382c32a7d193b4ad43d35

Re-introduce union for rseq_cs ptr

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I8a2a9ec606e9e578a8e9370ee13a340a768c7132

Fix: make dist missing header files

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: If888ede3080a971dd927d7e475ae3361691bfcae

Fix: percpu ops mm_cid tests

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I9f114f1af1febd7f646d4771e858d6784273402e

Skip mm_cid tests when unavailable

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I53e9d8f3c654b4fe8d5fc5275984f6340f8bc872

parametrized test: Report/abort on negative concurrency ID

Report and abort when a negative concurrency ID value is observed by the
spinlock test.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ied1e3dafcd882c1071bcc432c7a5102ca77c4316

Implement parametrized mm_cid test

Adapt to the rseq.h API changes introduced by commits
"selftests/rseq: <arch>: Template memory ordering and percpu access mode".

Build a new param_test_mm_cid, param_test_mm_cid_benchmark, and
param_test_mm_cid_compare_twice executables to test the new "mm_cid"
rseq field.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I36405e733876abd7d4f04f07e256e7029c2ea1d2

Implement basic percpu ops mm_cid test

Adapt to the rseq.h API changes introduced by commits
"selftests/rseq: <arch>: Template memory ordering and percpu access mode".

Build a new basic_percpu_ops_mm_cid_test to test the new "mm_cid" rseq
field.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I35faa59281b0c7c0d0a449352a45a2d2b0df480d

riscv: Template memory ordering and percpu access mode

Introduce a rseq-riscv-bits.h template header which is internally included
to generate the static inline functions covering:

- relaxed and release memory ordering,
- per-cpu-id and per-mm-cid per-cpu data access.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ia88fd435f0593f31f31eec5cd8cd8099b1cb8af8

s390: Template memory ordering and percpu access mode

Introduce a rseq-s390-bits.h template header which is internally included
to generate the static inline functions covering:

- relaxed and release memory ordering,
- per-cpu-id and per-mm-cid per-cpu data access.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I61b84bef1ebb280f7e04c99f66cd823635780560

ppc: Template memory ordering and percpu access mode

Introduce a rseq-ppc-bits.h template header which is internally included
to generate the static inline functions covering:

- relaxed and release memory ordering,
- per-cpu-id and per-mm-cid per-cpu data access.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ia2587647c56692f05672fa678779e37e7618d0a0

mips: Template memory ordering and percpu access mode

Introduce a rseq-mips-bits.h template header which is internally
included to generate the static inline functions covering:

- relaxed and release memory ordering,
- per-cpu-id and per-mm-cid per-cpu data access.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I5dc6a9bc28d6d9a3d85e5ccd871ad5de9d210f91

arm64: Template memory ordering and percpu access mode

Introduce a rseq-arm64-bits.h template header which is internally
included to generate the static inline functions covering:

- relaxed and release memory ordering,
- per-cpu-id and per-mm-cid per-cpu data access.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I4d0ae931549d6f61ea2b2f8183a4297db97c3144

arm: Template memory ordering and percpu access mode

Introduce a rseq-arm-bits.h template header which is internally included
to generate the static inline functions covering:

- relaxed and release memory ordering,
- per-cpu-id and per-mm-cid per-cpu data access.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I062d49375e1a14efaac8e16f6cc065df2cf9d26c

x86: Template memory ordering and percpu access mode

Introduce a rseq-x86-bits.h template header which is internally included
to generate the static inline functions covering:

- relaxed and release memory ordering,
- per-cpu-id and per-mm-cid per-cpu data access.

This introduces changes to the rseq.h selftests API which require to
update the rseq selftest programs. Similar API/templating changes need
to be done for other architectures.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: If6889ccebedda2f9bc3b1abe7faad1ba626473c3

Implement rseq numa node id field selftest

Test the NUMA node id extension rseq field. Compare it against the value
returned by the getcpu(2) system call while pinned on a specific core.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ia01752b2bdd3ab9bf5ef4f2aef12bd1704169e7a

rseq.h: implement mm_cid and node_id APIs

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ic82b240e654702416936d612a03d8675f9fb869c

Update rseq.c to query ELF auxvec

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I562b07aac2c7277eb8be3a31782d402e1d693be1

Cleanup: rseq.h: add extra space around __VA_ARGS__

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Iec19b59c40b513be33e5ab3207d947c05037adb0

Add mm_cid and numa node id to rseq ABI header

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I6def7df36e25d47027b3274cf325ce1beb841b75

Remove RSEQ_SKIP_FASTPATH code

This code is not currently build by the test Makefile, adds complexity,
and is not overall useful considering that the abort handling loops to
retry the fast-path.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I2bf5ead4c0314bb0f28de63d9e6b2a2943c59aa2

Fix: `void *` arithmetic compiler warning

Fixes warning "pointer of type ‘void *’ used in subtraction [-Wpointer-arith]"
which was observed on gcc 11.3.0 on Ubuntu 22.04 LTS Linux 6.3.0 RC7 x86-64.

This warning was enabled by the flag `-Wpedantic`.

Link: https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wpointer-arith
Signed-off-by: Gerrit Klein <therealthingy95@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Icfcf1d39bcde3730bf38c9ec6ed1521ed4503625

All arch: use rseq_unqual_scalar_typeof in load-acquire

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I08f8aedf09ac37fa9b06640a494f20cfbcfeb557

c++: compiler: use remove_cv and remove_reference in rseq_unqual_scalar_typeof

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I4c278af941c4e17fbef1b14bf97820100790b6f1

rseq_arm64: use rseq_unqual_scalar_typeof in load-acquire

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I1cdb0b776f5c81d3cca8bb9fbd03f9549bf40d77

Revert "compiler: C++: simplify rseq_unqual_scalar_typeof implementation"

This reverts commit 38f60c8d69e147b5997f850e4f163c36b2e1c91a.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Revert "compiler: c++: use decltype"

This reverts commit 51e83f8f967435d40541b7ae07b932e5cb1c0d47.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

compiler: c++: use decltype

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I23e4d035029efd9bab6ca2b1bba13146c5eb0375

compiler: C++: simplify rseq_unqual_scalar_typeof implementation

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I55b57c217e4338ebeb4947c3452de1422fe5f10c

rseq-arm64: Fix buggy load-acquire/store-release macros

The arm64 load-acquire/store-release macros from the Linux kernel rseq
selftests are buggy. Remplace them by a working implementation.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I84cf01cb0fabee89f76cc8d608d203c975397037

compiler: implement rseq_unqual_scalar_typeof

Allow defining variables and cast with a typeof which removes the
volatile and const qualitiers.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ibe7068e2c307ed47a2eb6db891e717fb7db47282

Use __asm__ __volatile__ in rseq_after_asm_goto for c11 compatibility

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I63cde85aeaf5efd03e613c255f31f1091c649790

architecture headers: add extra parentheses around macro parameters

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I34a68a8f420ce993b4d6ab21e3c1b86ad5ea4113

Tests: Introduce arch-mo unit tests

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ibef415f2c9d2c9356c00b11870057f74f89d75d5

Fix: gcc extension compiler warning

Use __VA_ARGS__ rather than GNU args... in macros for improved compiler
compatibility.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ie17e73019b1afae721a2270b0b43197c8dca8b88

Initialize librseq from `rseq_register_current_thread`

Invoke rseq_init() explicitly from rseq_register_current_thread to make
sure the library is already initialized if thread registration is done
before the library constructor is executed.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I6c808ff846cd280522c3b8d50c0c64ca587261ee