-barectf
-=======
+# barectf
**barectf** is a command-line utility which generates pure C99
-code that is able to write native
-[CTF](http://git.efficios.com/?p=ctf.git;a=blob_plain;f=common-trace-format-specification.txt;hb=master)
-(the Common Trace Format) out of a pre-written CTF metadata file.
+code that is able to write native [Common Trace Format](http://diamon.org/ctf)
+(CTF) binary streams.
You will find barectf interesting if:
- 1. You need to trace a program.
- 2. You need tracing to be as fast as possible, but also very flexible:
+ 1. You need to trace an application.
+ 2. You need tracing to be efficient, yet flexible:
record integers of custom sizes, custom floating point numbers,
- enumerations mapped to a specific integer type, structure fields,
- NULL-terminated strings, static and dynamic arrays, etc.
+ enumerations supported by a specific integer type, and
+ null-terminated UTF-8/ASCII strings (C strings).
3. You need to be able to convert the recorded binary events to
human-readable text, as well as analyze them with Python scripts
([Babeltrace](http://www.efficios.com/babeltrace) does all that,
given a CTF input).
4. You _cannot_ use [LTTng](http://lttng.org/), an efficient tracing
framework for the Linux kernel and Linux/BSD user applications, which
- outputs CTF.
+ also outputs CTF.
The target audience of barectf is developers who need to trace bare metal
systems (without an operating system). The code produced by barectf
-is pure C99 and is lightweight enough to fit on a tiny microcontroller.
-Each event described in the CTF metadata input becomes one C function with
-one parameter mapped to one event field. CTF data is recorded in a buffer of
-any size provided by the user. This buffer corresponds to one CTF packet.
-The generated tracing functions report when the buffer is full. The user
-is entirely responsible for the buffering scheme: leave the buffer in memory,
-save it to some permanent storage, swap it with another empty buffer and
-concatenate recorded packets, etc.
+is pure C99 and can be lightweight enough to fit on a tiny microcontroller.
-barectf is written in Python 3 and currently uses
-[pytsdl](https://github.com/eepp/pytsdl) to parse the CTF metadata file
-provided by the user.
+**Key features**:
+ * Single input: easy-to-write [YAML](https://en.wikipedia.org/wiki/YAML)
+ configuration file (documentation below)
+ * 1-to-1 mapping from tracing function parameters to event fields
+ * Custom and bundled _platforms_ hiding the details of opening/closing
+ packets and writing them to a back-end (continuous tracing), getting
+ the clock values, etc.:
+ * _linux-fs_: basic Linux application tracing writing stream files to
+ the file system for demonstration purposes
+ * _parallella_: Adapteva Epiphany/[Parallella](http://parallella.org/)
+ with host-side consumer
+ * CTF metadata generated by the command-line tool (automatic trace UUID,
+ stream IDs, and event IDs)
+ * All basic CTF types are supported: integers, floating point numbers,
+ enumerations, and null-terminated strings (C strings)
+ * Binary streams produced by the generated C code and metadata file
+ produced by barectf are CTF 1.8-compliant
+ * Human-readable error reporting
-installing
-----------
+**Current limitations**:
-Make sure you have `pip` for Python 3. On the latest Ubuntu releases,
-it is called `pip3`:
+As of this version:
+
+ * All the generated tracing C functions, for a given barectf
+ stream-specific context, need to be called from the same thread, and cannot
+ be called from an interrupt handler, unless a user-provided
+ synchronization mechanism is used.
+ * CTF compound types (array, sequence, structure, variant) are not supported
+ yet, except at some very specific locations in the metadata.
+
+barectf is written in Python 3.
+
+
+## Installing
+
+Make sure you have Python 3 and `pip` for Python 3 installed, then
+install barectf.
+
+Note that you may pass the `--user` argument to
+`pip install` to install the tool in your home directory (instead of
+installing globally).
+
+**Latest Ubuntu**:
sudo apt-get install python3-pip
+ sudo pip3 install barectf
-On Ubuntu 12.04, you need to install `setuptools` first, then use
-`easy_install3` to install `pip3`:
+**Ubuntu 12.04 and lower**:
sudo apt-get install python3-setuptools
sudo easy_install3 pip
+ sudo pip3 install barectf
-Install barectf:
+**Debian**:
+ sudo apt-get install python3-pip
sudo pip3 install barectf
+**Fedora 20 and up**:
+
+ sudo yum install python3-pip
+ sudo pip3 install barectf
-using
------
+**Arch Linux**:
-Using barectf involves:
+ sudo install python-pip
+ sudo pip install barectf
- 1. Writing the CTF metadata file describing the various headers,
- contexts and event fields.
- 2. Running the `barectf` command to generate C99 files out of
- the CTF metadata file.
- 3. Using the generated C code in your specific application.
+**OS X**
-The following subsections explain the three steps above.
+With [Homebrew](http://brew.sh/):
+ brew install python3
+ pip3 install barectf
-### writing the CTF metadata
-The **Common Trace Format** is a specialized file format for recording
-trace data. CTF is designed to be very fast to write and very flexible.
-All headers, contexts and event fields written in binary files are
-described using a custom C-like, declarative language, TSDL (Trace
-Stream Description Language). The file containing this description is
-called the **CTF metadata**. The latter may be automatically generated
-by a tracer, like it is the case of LTTng, or written by hand. This
-metadata file is then used by CTF trace readers to know the layout of
-CTF binary files containing actual event contexts and fields.
+## What is CTF?
-The CTF metadata file contains several blocks describing various CTF
-binary layouts. A CTF trace file is a concatenation of several CTF
-packets. Here's the anatomy of a CTF packet:
+See the [CTF in a nutshell](http://diamon.org/ctf/#ctf-in-a-nutshell)
+section of CTF's website to understand the basics of this
+trace format.
-
+The most important thing to understand about CTF, for barectf use
+cases, is the layout of a binary stream packet:
-A CTF packet belongs to a specific CTF stream. While the packet header
-is the same for all streams of a given CTF trace, everything else is
-specified per stream. Following this packet header is a packet context,
-and then actual recorded events. Each event starts with a mandatory
-header (same event header for all events of a given stream). The event
-header is followed by an optional event context with a layout shared
-by all events of a given stream. Then follows another optional event
-context, although this one has a layout specific to the event type.
-Finally, event fields are written.
+ * Packet header (defined at the trace level)
+ * Packet context (defined at the stream level)
+ * Sequence of events (defined at the stream level):
+ * Event header (defined at the stream level)
+ * Stream event context (defined at the stream level)
+ * Event context (defined at the event level)
+ * Event payload (defined at the event level)
-barectf asks you to write the CTF metadata by hand. Although its official
-[specification](http://git.efficios.com/?p=ctf.git;a=blob_plain;f=common-trace-format-specification.txt;hb=master)
-is thorough, you will almost always start from this template:
+The following diagram, stolen without remorse from CTF's website, shows
+said packet layout:
-```
-/* CTF 1.8 */
-
-// a few useful standard integer aliases
-typealias integer {size = 8; align = 8;} := uint8_t;
-typealias integer {size = 16; align = 16;} := uint16_t;
-typealias integer {size = 32; align = 32;} := uint32_t;
-typealias integer {size = 64; align = 64;} := uint64_t;
-typealias integer {size = 8; align = 8; signed = true;} := int8_t;
-typealias integer {size = 16; align = 16; signed = true;} := int16_t;
-typealias integer {size = 32; align = 32; signed = true;} := int32_t;
-typealias integer {size = 64; align = 64; signed = true;} := int64_t;
-
-// IEEE 754 standard-precision floating point alias
-typealias floating_point {
- exp_dig = 8;
- mant_dig = 24;
- align = 32;
-} := float;
-
-// IEEE 754 double-precision floating point alias
-typealias floating_point {
- exp_dig = 11;
- mant_dig = 53;
- align = 64;
-} := double;
-
-// trace block
-trace {
- // CTF version 1.8; leave this as is
- major = 1;
- minor = 8;
-
- /* Native byte order (`le` or `be`). This is used by barectf to generate
- * the appropriate code when writing data to the packet.
- */
- byte_order = le;
-
- /* Packet header. All packets (buffers) will have the same header.
- *
- * Special fields recognized by barectf (must appear in this order):
- *
- * magic: will be set to CTF's magic number (must be the first field)
- * (32-bit unsigned integer)
- * stream_id: will be set to the ID of the stream associated with
- * this packet (unsigned integer of your choice) (mandatory)
- */
- packet.header := struct {
- uint32_t magic;
- uint32_t stream_id;
- };
-};
-
-// environment variables; you may add custom entries
-env {
- domain = "bare";
- tracer_name = "barectf";
- tracer_major = 0;
- tracer_minor = 1;
- tracer_patchlevel = 0;
-};
-
-// clock descriptor
-clock {
- // clock name
- name = my_clock;
-
- // clock frequency (Hz)
- freq = 1000000000;
-
- // optional clock value offset (offfset from Epoch is: offset * (1 / freq))
- offset = 0;
-};
-
-// alias for integer used to hold clock cycles
-typealias integer {
- size = 32;
-
- // map to the appropriate clock using its name
- map = clock.my_clock.value;
-} := my_clock_int_t;
-
-/* A stream. You may have as many streams as you want. Events are unique
- * within their own stream. The main advantage of having multiple streams
- * is having different event headers, stream event contexts and stream
- * packet contexts for each one.
- */
-stream {
- /* Mandatory stream ID (must fit the integer type of
- * `trace.packet.header.stream_id`.
- */
- id = 0;
-
- /* Mandatory packet context. This structure follows the packet header
- * (see `trace.packet.header`) immediately in CTF binary streams.
- *
- * Special fields recognized by barectf:
- *
- * timestamp_begin: will be set to the current clock value when opening
- * the packet (same integer type as the clock's value)
- * timestamp_end: will be set to the current clock value when closing
- * the packet (same integer type as the clock's value)
- * content_size: will be set to the content size, in bits, of this
- * stream (unsigned 32-bit or 64-bit integer) (mandatory)
- * packet_size: will be set to the packet size, in bits, of this
- * stream (unsigned 32-bit or 64-bit integer) (mandatory)
- * cpu_id: if present, the barectf_open_packet() function of this
- * stream will accept an additional parameter to specify the
- * ID of the CPU associated with this stream (a given
- * stream should only be written to by a specific CPU)
- * (unsigned integer of your choice)
- *
- * `timestamp_end` must be present if `timestamp_begin` exists.
- */
- packet.context := struct {
- my_clock_int_t timestamp_begin;
- my_clock_int_t timestamp_end;
- uint64_t content_size;
- uint64_t packet_size;
- uint32_t cpu_id;
- };
+
- /* Mandatory event header. All events recorded in this stream will start
- * with this structure.
- *
- * Special fields recognized by barectf:
- *
- * id: will be filled by the event ID corresponding to a tracing
- * function (unsigned integer of your choice)
- * timestamp: will be filled by the current clock's value (same integer
- * type as the clock's value)
- */
- event.header := struct {
- uint32_t id;
- my_clock_int_t timestamp;
- };
+Any of those six dynamic scopes, if defined at all, has an associated
+CTF type. barectf requires them to be structure types.
- /* Optional stream event context (you may remove the whole block or leave
- * the structure empty if you don't want any). This structure follows the
- * event header (see `stream.event.header`) immediately in CTF binary streams.
- */
- event.context := struct {
- int32_t _some_stream_event_context_field;
- };
-};
-
-/* An event. Events have an ID, a name, an optional context and fields. An
- * event is associated to a specific stream using its stream ID.
- */
-event {
- /* Mandatory event name. This is used by barectf to generate the suffix
- * of this event's corresponding tracing function, so make sure it follows
- * the C identifier syntax even though it's a quoted string here.
- */
- name = "my_event";
-
- /* Mandatory event ID (must fit the integer type of
- * in `stream.event.header.id` of the associated stream).
- */
- id = 0;
-
- // ID of the stream in which this event will be recorded
- stream_id = 0;
-
- /* Optional event context (you may remove the whole block or leave the
- * structure empty if you don't want one). This structure follows the
- * stream event context (if it exists) immediately in CTF binary streams.
- */
- context := struct {
- int32_t _some_event_context_field;
- };
- /* Mandatory event fields (although the structure may be left empty if this
- * event has no fields). This structure follows the event context (if it
- * exists) immediately in CTF binary streams.
- */
- fields := struct {
- uint32_t _a;
- uint32_t _b;
- uint16_t _c;
- string _d;
- };
-};
-```
+## Using
-The top `/* CTF 1.8 */` is actually needed right there, and as is, since it
-acts as a CTF metadata magic number for CTF readers.
+Using barectf involves the following steps:
-Only one stream and one event (belonging to this single stream) are described
-in this template, but you may add as many as you need.
+ 1. Writing the YAML configuration file defining the various header,
+ context, and event field types.
+ 2. Running the `barectf` command-line tool with this configuration file
+ to generate the CTF metadata and C files.
+ 3. Using the generated C code (tracing functions), along with the C code
+ provided by the appropriate barectf platform, in the source code of
+ your own application.
+ 4. Running your application, along with anything the barectf platform
+ you chose requires, to generate the binary streams of a CTF trace.
-The following subsections describe the features of CTF metadata supported
-by barectf.
+Your application, when running, will generate CTF packets. Depending
+on the chosen barectf platform, those packets will be consumed and
+sequentially written at some place for later viewing/analysis.
+Here's a diagram summarizing the steps described above:
-#### types
+
-The supported structure field types are:
+The following subsections explain the four steps above.
- * **integers** of any size (64-bit and less), any alignment (power of two)
- * **floating point numbers** of any total size (64-bit and less), any
- alignment (power of two)
- * NULL-terminated **strings** of bytes
- * **enumerations** associated with a specific integer type
- * **static** and **dynamic arrays** of any type
- * **structures** containing only integers, floating point numbers,
- enumerations and _static_ arrays
+Also, have a look at the [`doc/examples`](doc/examples) directory, which
+contains complete examples.
-CTF also supports _variants_ (dynamic selection between different types),
-but barectf **does not**. Any detected variant will throw an error when
-running `barectf`.
+### Writing the YAML configuration file
-##### integers
+The barectf [YAML](https://en.wikipedia.org/wiki/YAML) configuration file
+is the only input the `barectf` command-line tool needs in order to generate
+the corresponding CTF metadata and C files.
-CTF integers are defined like this:
+To start with a concrete configuration, here's some minimal configuration:
-```
-integer {
- // mandatory size in bits (64-bit and less)
- size = 16;
-
- /* Optional alignment in bits (power of two). Default is 8 when the
- * size is a multiple of 8, and 1 otherwise.
- */
- align = 16;
-
- // optional signedness (`true` or `false`); default is unsigned
- signed = true;
-
- /* Optional byte order (`le`, `be`, `native` or `network`). `native`
- * will use the byte order specified by the `trace.byte_order`.
- * Default is `native`.
- */
- byte_order = le;
-
- /* Optional display base, used to display the integer value when
- * reading the trace. Valid values are 2 (or `binary`, `bin` and `b`),
- * 8 (or `o`, `oct` or `octal`), 10 (or `u`, `i`, `d`, `dec` or
- * `decimal`), and 16 (or `x`, `X`, `p`, `hex` or `hexadecimal`).
- * Default is 10.
- */
- base = hex;
-
- /* Encoding (if this integer represents a character). Valid values
- * are `none`, `UTF8` and `ASCII`. Default is `none`.
- */
- encoding = UTF8;
-}
+```yaml
+version: '2.0'
+metadata:
+ type-aliases:
+ uint16:
+ class: int
+ size: 16
+ trace:
+ byte-order: le
+ streams:
+ my_stream:
+ packet-context-type:
+ class: struct
+ fields:
+ packet_size: uint16
+ content_size: uint16
+ events:
+ my_event:
+ payload-type:
+ class: struct
+ fields:
+ my_field:
+ class: int
+ size: 8
```
-The size (the only mandatory property) does _not_ have to be a power of two:
+The `version` property must be set to the `2.0` _string_ (hence the single
+quotes). As features are added to barectf and to its configuration file schema,
+this version will be bumped accordingly.
+The `metadata` property is where the properties and layout of the
+eventual CTF trace are defined. The accepted properties of each object
+are documented later in this document. For the moment, note simply
+that the native byte order of the trace is set to `le` (little-endian),
+and that there's one defined stream named `my_stream`, having one
+defined event named `my_event`, having a structure as its payload
+type, with a single 8-bit unsigned integer type field named `my_field`. Also,
+the stream packet context type is a structure defining the mandatory
+`packet_size` and `content_size` special fields as 16-bit unsigned integer
+types.
+
+Running `barectf` with the configuration above (as a file named `config.yaml`):
+
+ barectf config.yaml
+
+will produce a C file (`barectf.c`), and its header file (`barectf.h`),
+the latter declaring the following function:
+
+```c
+void barectf_my_stream_trace_my_event(
+ struct barectf_my_stream_ctx *ctx, uint8_t ep_my_field);
```
-integer {size = 23;}
+
+`ctx` is the barectf context for the stream named `my_stream` (usually
+initialized and provided by the barectf platform), and `ep_my_field` is the
+value of the `my_event` event payload's `my_field` field.
+
+The following subsections define all the objects of the YAML configuration
+file.
+
+
+#### Configuration object
+
+The top-level object of the YAML configuration file.
+
+**Properties**:
+
+| Property | Type | Description | Required? | Default value |
+|---|---|---|---|---|
+| `version` | String | Must be set to `'2.0'` | Required | N/A |
+| `prefix` | String | Prefix to be used for function names, file names, etc. | Optional | `barectf_` |
+| `metadata` | [Metadata object](#metadata-object) | Trace metadata | Required | N/A |
+
+The `prefix` property must be set to a valid C identifier. It can be
+overridden by the `barectf` command-line tool's `--prefix` option.
+
+**Example**:
+
+```yaml
+version: '2.0'
+prefix: axx_
+metadata:
+ type-aliases:
+ uint16:
+ class: int
+ size: 16
+ trace:
+ byte-order: le
+ streams:
+ my_stream:
+ packet-context-type:
+ class: struct
+ fields:
+ packet_size: uint16
+ content_size: uint16
+ events:
+ my_event:
+ payload-type:
+ class: struct
+ fields:
+ a:
+ class: int
+ size: 8
```
-is perfectly valid.
-A CTF integer field will make barectf produce a corresponding C integer
-function parameter with an appropriate size. For example, the 23-bit integer
-above would produce an `uint32_t` parameter (of which only the first 23
-least significant bits will be written to the trace), while the first
-16-bit one will produce an `int16_t` parameter.
+#### Metadata object
+
+A metadata object defines the desired layout of the CTF trace to be
+produced by the generated C code. It is used by barectf to generate C code,
+as well as a corresponding CTF metadata file.
+
+**Properties**:
+
+| Property | Type | Description | Required? | Default value |
+|---|---|---|---|---|
+| `type-aliases` | Associative array of strings (alias names) to [type objects](#type-objects) or strings (previous alias names) | Type aliases to be used in trace, stream, and event objects | Optional | `{}` |
+| `log-levels` | Associative array of strings (log level names) to log level constant integers | Log levels to be used in event objects | Optional | `{}` |
+| `clocks` | Associative array of strings (clock names) to [clock objects](#clock-object) | Trace clocks | Optional | `{}` |
+| `env` | Associative array of strings (names) to strings or integers (values) | Trace environment variables | Optional | `{}` |
+| `trace` | [Trace object](#trace-object) | Metadata common to the whole trace | Required | N/A |
+| `streams` | Associative array of strings (stream names) to [stream objects](#stream-object) | Trace streams | Required | N/A |
+
+Each clock name of the `clocks` property must be a valid C identifier.
+
+The `streams` property must contain at least one entry. Each stream name must be
+a valid C identifier.
+
+Each environment variable name in the `env` property must be a valid
+C identifier. Those variables will be appended to some environment
+variables set by barectf itself.
+
+The order of the `type-aliases` entries is important: a type alias may only
+inherit from another type alias if the latter is defined before.
+
+**Example**:
+
+```yaml
+type-aliases:
+ uint8:
+ class: integer
+ size: 8
+ uint16:
+ class: integer
+ size: 16
+ uint32:
+ class: integer
+ size: 32
+ uint64:
+ class: integer
+ size: 64
+ clock-int:
+ inherit: uint64
+ property-mappings:
+ - type: clock
+ name: my_clock
+ property: value
+ byte: uint8
+ uuid:
+ class: array
+ length: 16
+ element-type: byte
+log-levels:
+ emerg: 0
+ alert: 1
+ critical: 2
+ error: 3
+ warning: 4
+ notice: 5
+ info: 6
+clocks:
+ my_clock:
+ freq: 1000000000
+ offset:
+ seconds: 1434072888
+ return-ctype: uint64_t
+env:
+ my_system_version: '0.3.2-2015.03'
+ bID: 15
+trace:
+ byte-order: le
+ uuid: auto
+ packet-header-type:
+ class: struct
+ min-align: 8
+ fields:
+ magic: uint32
+ uuid: uuid
+ stream_id: uint8
+streams:
+ my_stream:
+ packet-context-type:
+ class: struct
+ fields:
+ timestamp_begin: clock-int
+ timestamp_end: clock-int
+ packet_size: uint32
+ something: float
+ content_size: uint32
+ events_discarded: uint32
+ event-header-type:
+ class: struct
+ fields:
+ timestamp: clock-int
+ id: uint16
+ events:
+ simple_uint32:
+ log-level: error
+ payload-type:
+ class: struct
+ fields:
+ value: uint32
+ simple_int16:
+ payload-type:
+ class: struct
+ fields:
+ value:
+ inherit: uint16
+ signed: true
+```
-The `integer` block also accepts a `map` property which is only used
-when defining the integer used to carry the value of a specified
-clock. You may always follow the example above.
+#### Clock object
-##### floating point numbers
+A CTF clock.
-CTF floating point numbers are defined like this:
+**Properties**:
-```
-floating_point {
- // exponent size in bits
- exp_dig = 8;
-
- // mantissa size in bits
- mant_dig = 24;
-
- /* Optional alignment (power of two). Default is 8 when the total
- * size (exponent + mantissa) is a multiple of 8, and 1 otherwise.
- */
- align = 32;
-
- /* Optional byte order (`le`, `be`, `native` or `network`). `native`
- * will use the byte order specified by the `trace.byte_order`.
- * Default is `native`.
- */
- byte_order = le;
-}
+| Property | Type | Description | Required? | Default value |
+|---|---|---|---|---|
+| `freq` | Integer (positive) | Frequency (Hz) | Optional | 1000000000 |
+| `description` | String | Description | Optional | No description |
+| `uuid` | String (UUID canonical format) | UUID (unique identifier of this clock) | Optional | No UUID |
+| `error-cycles` | Integer (zero or positive) | Error (uncertainty) of clock in clock cycles | Optional | 0 |
+| `offset` | [Clock offset object](#clock-offset-object) | Offset | Optional | Default clock offset object |
+| `absolute` | Boolean | Absolute clock | Optional | `false` |
+| `return-ctype` | String | Return C type of the associated clock callback | Optional | `uint32_t` |
+
+The `return-ctype` property must be set to a valid C integer type
+(or valid type definition). This is not currently validated by barectf
+itself, but the C compiler will fail to compile the generated C code
+if the clock's return type is not a valid C integer type.
+
+**Example**:
+
+```yaml
+freq: 2450000000
+description: CCLK/A2 (System clock, A2 clock domain)
+uuid: 184883f6-6b6e-4bfd-bcf7-1e45c055c56a
+error-cycles: 23
+offset:
+ seconds: 1434072888
+ cycles: 2003912
+absolute: false
+return-ctype: unsigned long long
```
-If a CTF floating point number is defined with an 8-bit exponent, a 24-bit
-mantissa and a 32-bit alignment, its barectf C function parameter type will
-be `float`. It will be `double` for an 11-bit exponent, 53-bit mantissa
-and 64-bit aligned CTF floating point number. Any other configuration
-will produce a `uint64_t` function parameter (you will need to cast your
-custom floating point number to this when calling the tracing function).
+##### Clock offset object
-##### strings
+An offset in seconds and clock cycles from the Unix epoch.
-CTF strings are pretty simple to define:
+**Properties**:
-```
-string
+| Property | Type | Description | Required? | Default value |
+|---|---|---|---|---|
+| `seconds` | Integer (zero or positive) | Seconds since the Unix epoch | Optional | 0 |
+| `cycles` | Integer (zero or positive) | Clock cycles since the Unix epoch plus the value of the `seconds` property | Optional | 0 |
+
+**Example**:
+
+```yaml
+seconds: 1435617321
+cycles: 194570
```
-They may also have an encoding property:
+#### Trace object
+
+Metadata common to the whole trace.
+
+**Properties**:
+
+| Property | Type | Description | Required? | Default value |
+|---|---|---|---|---|
+| `byte-order` | String | Native byte order (`le` for little-endian or `be` for big-endian) | Required | N/A |
+| `uuid` | String (UUID canonical format or `auto`) | UUID (unique identifier of this trace); automatically generated if value is `auto` | Optional | No UUID |
+| `packet-header-type` | [Type object](#type-objects) or string (alias name) | Type of packet header (must be a [structure type object](#structure-type-object)) | Optional | No packet header |
+
+Each field of the packet header structure type (`packet-header-type` property)
+corresponds to one parameter
+of the generated packet opening function (prefixed with `tph_`), except for the
+following special fields, which are automatically written if present:
+
+ * `magic` (32-bit unsigned [integer type object](#integer-type-object)):
+ packet magic number
+ * `uuid` ([array type object](#array-type-object) of 8-bit unsigned
+ [integer type objects](#integer-type-object), of length 16):
+ trace UUID (`uuid` property of trace object must be set)
+ * `stream_id` (unsigned [integer type object](#integer-type-object)):
+ stream ID
+
+As per CTF 1.8, the `stream_id` field is mandatory if there's more
+than one defined stream.
+
+**Example**:
+
+```yaml
+byte-order: le
+uuid: auto
+packet-header-type:
+ class: struct
+ fields:
+ magic: uint32
+ uuid:
+ class: array
+ length: 16
+ element-type: uint8
+ stream_id: uint16
```
-string {
- // encoding: `none`, `UTF8` or `ASCII`; default is `none`
- encoding = UTF8;
-}
+
+
+#### Stream object
+
+A CTF stream.
+
+**Properties**:
+
+| Property | Type | Description | Required? | Default value |
+|---|---|---|---|---|
+| `packet-context-type` | [Type object](#type-objects) or string (alias name) | Type of packet context (must be a [structure type object](#structure-type-object)) | Required | N/A |
+| `event-header-type` | [Type object]((#type-objects)) or string (alias name) | Type of event header (must be a [structure type object](#structure-type-object)) | Optional | No event header |
+| `event-context-type` | [Type object]((#type-objects)) or string (alias name) | Type of stream event context (must be a [structure type object](#structure-type-object)) | Optional | No stream event context |
+| `events` | Associative array of event names (string) to [event objects](#event-object) | Stream events | Required | N/A |
+
+Each field of the packet context structure type (`packet-context-type` property)
+corresponds to one parameter
+of the generated packet opening function (prefixed with `spc_`), except for the
+following special fields, which are automatically written if present:
+
+ * `timestamp_begin` and `timestamp_end` (unsigned
+ [integer type objects](#integer-type-object), with
+ a clock value property mapping): resp. open and close timestamps
+ * `packet_size` (unsigned [integer type object](#integer-type-object),
+ mandatory): packet size
+ * `content_size` (unsigned [integer type object](#integer-type-object),
+ mandatory): content size
+ * `events_discarded` (unsigned [integer type object](#integer-type-object)):
+ number of discarded events so far
+
+The `timestamp_end` field must exist if the `timestamp_begin` field exists,
+and vice versa.
+
+Each field of the event header structure type (`event-header-type` property)
+corresponds to one parameter of the generated tracing function
+(prefixed with `eh_`) (for a given event), except for the following special
+fields, which are automatically written if present:
+
+ * `id` (unsigned [integer type object](#integer-type-object)): event ID
+ * `timestamp` (unsigned [integer type object](#integer-type-object), with
+ a clock value property mapping): event timestamp
+
+The `id` field must exist if there's more than one defined event in the
+stream.
+
+Each field of the stream event context structure type (`event-context-type`
+property) corresponds to one parameter of the generated tracing function
+(prefixed with `seh_`) (for a given event).
+
+Each field name of the `packet-context-type`, `event-header-type`,
+and `event-context-type` properties must be a valid C identifier.
+
+The `events` property must contain at least one entry.
+
+**Example**:
+
+```yaml
+packet-context-type:
+ class: struct
+ fields:
+ timestamp_begin: clock-int
+ timestamp_end: clock-int
+ packet_size: uint32
+ content_size: uint32
+ events_discarded: uint16
+ my_custom_field: int12
+event-header-type:
+ class: struct
+ fields:
+ id: uint16
+ timestamp: clock-int
+event-context-type:
+ class: struct
+ fields:
+ obj_id: uint8
+events:
+ msg_in:
+ payload-type: msg-type
```
-CTF strings are always byte-aligned.
-A CTF string field will make barectf produce a corresponding C function
-parameter of type `const char*`. Bytes will be copied from this pointer
-until a byte of value 0 is found (which will also be written to the
-buffer to mark the end of the recorded string).
+#### Event object
+A CTF event.
-##### enumerations
+**Properties**:
-CTF enumerations associate labels to ranges of integer values. They
-are a great way to trace named states using an integer. Here's an
-example:
+| Property | Type | Description | Required? | Default value |
+|---|---|---|---|---|
+| `log-level` | String (predefined log level name) or integer (zero or positive) | Log level of this event | Optional | No log level |
+| `context-type` | [Type object](#type-objects) or string (alias name) | Type of event context (must be a [structure type object](#structure-type-object)) | Optional | No event context |
+| `payload-type` | [Type object](#type-objects) or string (alias name) | Type of event payload (must be a [structure type object](#structure-type-object)) | Required | N/A |
-```
-enum : uint32_t {
- ZERO,
- ONE,
- TWO,
- TEN = 10,
- ELEVEN,
- "label with spaces",
- RANGE = 23 ... 193
-}
+Available log level names, for a given event, are defined by the
+`log-levels` property of the [metadata object](#metadata-object)
+containing it.
+
+Each field of the event context structure type (`context-type` property)
+corresponds to one parameter
+of the generated tracing function (prefixed with `ec_`).
+
+Each field of the event payload structure type (`payload-type` property)
+corresponds to one parameter
+of the generated tracing function (prefixed with `ep_`). The event
+payload structure type must contain at least one field.
+
+Each field name of the `context-type` and `payload-type` properties must be a
+valid C identifier.
+
+**Example**:
+
+```yaml
+log-level: error
+context-type:
+ class: struct
+ fields:
+ msg_id: uint16
+payload-type:
+ class: struct
+ fields:
+ src:
+ type: string
+ dst:
+ type: string
+ payload_sz: uint32
```
-Unless the first entry specifies a value, CTF enumerations are
-always started at 0. They work pretty much like their C counterpart,
-although they support ranges and literal strings as labels.
-CTF enumerations are associated with a CTF integer type (`uint32_t`
-above). This identifier must be an existing integer type alias.
+#### Type objects
-A CTF enumeration field will make barectf produce a corresponding C
-integer function parameter compatible with the associated CTF integer type.
+Type objects represent CTF types.
+**Common properties**:
-##### static arrays
+| Property | Type | Description | Required? | Default value |
+|---|---|---|---|---|
+| `class` | String | Type class | Required if `inherit` property is absent | N/A |
+| `inherit` | String | Name of type alias from which to inherit properties | Required if `class` property is absent | N/A |
-Structure field names may be followed by a subscripted constant to
-define a static array of the field type:
+The accepted values for the `class` property are:
-```
-struct {
- integer {size = 16;} _field[10];
-}
-```
+| `class` property value | CTF type |
+|---|---|
+| `int`<br>`integer` | Integer type |
+| `flt`<br>`float`<br>`floating-point` | Floating point number type |
+| `enum`<br>`enumeration` | Enumeration type |
+| `str`<br>`string` | String type |
+| `struct`<br>`structure` | Structure type |
+| `array` | Array/sequence types |
+| `var`<br>`variant` | Variant type |
-In the above structure, `_field` is a static array of ten 16-bit integers.
+The `inherit` property accepts the name of any previously defined
+type alias. Any propery in a type object that inherits from another
+type object overrides the parent properties as follows:
-A CTF static array field will make barectf produce a `const void*` C function
-parameter. Bytes will be copied from this pointer to match the total static
-array size. In the example above, the integer size is 16-bit, thus its
-default alignment is 8-bit, so 20 bytes would be copied.
+ * Booleans, numbers, and strings: value of parent property with
+ the same name is replaced
+ * Arrays: new elements are appended to parent array
+ * Associative arrays: properties sharing the name of parent
+ properties completely replace them; new properties are
+ added to the parent associative array
-The inner element of a CTF static array _must be at least byte-aligned_
-(8-bit), either by forcing its alignment, or by ensuring it manually
-when placing fields one after the other. This means the following static
-array is valid for barectf:
-```
-struct {
- // ...
- integer {size = 5;} _field[10];
-}
-```
+##### Integer type object
-as long as the very first 5-bit, 1-bit aligned integer element starts
-on an 8-bit boundary.
+A CTF integer type.
+**Properties**:
-##### dynamic arrays
+| Property | Type | Description | Required? | Default value |
+|---|---|---|---|---|
+| `size` | Integer (positive) | Size (bits) (1 to 64) | Required | N/A |
+| `align` | Integer (positive) | Alignment (bits) (power of two) | Optional | 8 if `size` property is a multiple of 8, else 1 |
+| `signed` | Boolean | Signedness | Optional | `false` (unsigned) |
+| `base` | Integer | Display radix (2, 8, 10, or 16) | Optional | 10 |
+| `byte-order` | String | Byte order (`le` for little-endian, `be` for big-endian, or `native` to use the byte order defined at the trace level) | Optional | `native` |
+| `property-mappings` | Array of [property mapping objects](#property-mapping-object) | Property mappings of this integer type | Optional | N/A |
-Just like static arrays, dynamic arrays are defined using a subscripted
-length, albeit in this case, this length refers to another field using
-the dot notation. Dynamic arrays are called _sequences_ in the CTF
-specification.
+The `property-mappings` array property currently accepts only one element.
-Here's an example:
+**Example**:
+```yaml
+class: int
+size: 12
+signed: false
+base: 8
+byte-order: le
+property-mappings:
+ - type: clock
+ name: my_clock
+ property: value
```
-struct {
- uint32_t _length;
- integer {size = 16;} _field[_length];
-}
-```
-In the above structure, `_field` is a dynamic array of `_length`
-16-bit integers.
+**Equivalent C type**:
+
+ * Unsigned: `uint8_t`, `uint16_t`, `uint32_t`, or `uint64_t`, depending on the
+ `size` property
+ * Signed: `int8_t`, `int16_t`, `int32_t`, or `int64_t`, depending on the
+ `size` property
+
+
+###### Property mapping object
+
+A property mapping object associates an integer type with a stateful
+object's property. When the integer type is decoded from a CTF binary
+stream, the associated object's property is updated.
+
+Currently, the only available stateful object's property is the
+current value of a given clock.
-There are various scopes to which a dynamic array may refer:
+**Properties**:
- * no prefix: previous field in the same structure, or in parent
- structures until found
- * `event.fields.` prefix: field of the event fields
- * `event.context.` prefix: field of the event context if it exists
- * `stream.event.context.` prefix: field of the stream event context
- if it exists
- * `stream.event.header.` prefix: field of the event header
- * `stream.packet.context.` prefix: field of the packet context
- * `trace.packet.header.` prefix: field of the packet header
- * `env.` prefix: static property of the environment block
+| Property | Type | Description | Required? | Default value |
+|---|---|---|---|---|
+| `type` | String | Object type (always `clock`) | Required | N/A |
+| `name` | String | Clock name | Required | N/A |
+| `property` | String | Clock property name (always `value`) | Required | N/A |
-Here's another, more complex example:
+**Example**:
+```yaml
+type: clock
+name: my_clock
+property: value
```
-struct {
- uint32_t _length;
- string _other_field[stream.event.context.length];
- float _static_array_of_dynamic_arrays[10][_length];
-}
+
+
+##### Floating point number type object
+
+A CTF floating point number type.
+
+**Properties**:
+
+| Property | Type | Description | Required? | Default value |
+|---|---|---|---|---|
+| `size` | [Floating point number type size object](#floating-point-number-type-size-object) | Size parameters | Required | N/A |
+| `align` | Integer (positive) | Alignment (bits) (power of two) | Optional | 8 |
+| `byte-order` | String | Byte order (`le` for little-endian, `be` for big-endian, or `native` to use the byte order defined at the trace level) | Optional | `native` |
+
+**Example**:
+
+```yaml
+class: float
+size:
+ exp: 11
+ mant: 53
+align: 64
+byte-order: be
```
-The above examples also demonstrates that dynamic arrays and static
-arrays may contain eachother. `_other_field` is a dynamic array of
-`stream.event.context.length` strings. `_static_array_of_dynamic_arrays`
-is a static array of 10 dynamic arrays of `_length` floating point
-numbers. This syntax follows the C language.
+**Equivalent C type**:
-A CTF dynamic array field will make barectf produce a `const void*` C function
-parameter. Bytes will be copied from this pointer to match the
-total dynamic array size. The previously recorded length will be
-found automatically (always an offset from the beginning of the
-stream packet, or from the beginning of the current event).
+ * 8-bit exponent, 24-bit mantissa, 32-bit alignment: `float`
+ * 11-bit exponent, 53-bit mantissa, 64-bit alignment: `double`
+ * Every other combination: `uint64_t`
-barectf has a few limitations concerning dynamic arrays:
- * The inner element of a CTF dynamic array _must be at least byte-aligned_
- (8-bit), either by forcing its alignment, or by ensuring it manually
- when placing fields one after the other.
- * The length type must be a 32-bit, byte-aligned unsigned integer
- with a native byte order.
+###### Floating point number type size object
+The CTF floating point number type is encoded, in a binary stream,
+following [IEEE 754-2008](https://en.wikipedia.org/wiki/IEEE_floating_point)'s
+interchange format. The required parameters are the exponent and
+significand sizes, in bits. In CTF, the _mantissa_ size includes the
+sign bit, whereas IEEE 754-2008's significand size does not include it.
-##### structures
+**Properties**:
-Structures contain fields associating a name to a type. The fields
-are recorded in the specified order within the CTF binary stream.
+| Property | Type | Description | Required? | Default value |
+|---|---|---|---|---|
+| `exp` | Integer (positive) | Exponent size (bits) | Required | N/A |
+| `mant` | Integer (positive) | Mantissa size (significand size + 1) (bits) | Required | N/A |
-Here's an example:
+As per IEEE 754-2008, the sum of the `exp` and `mant` properties must be a
+multiple of 32.
-```
-struct {
- uint32_t _a;
- int16_t _b;
- string {encoding = ASCII;} _c;
-}
-```
+The sum of the `exp` and `mant` properties must be lesser than or equal to 64.
-The default alignment of a structure is the largest alignment amongst
-its fields. For example, the following structure has a 32-bit alignment:
+**Example**:
-```
-struct {
- uint16_t _a; // alignment: 16
- struct { // alignment: 32
- uint32_t _a; // alignment: 32
- string; _b; // alignment: 8
- } _b;
- integer {size = 64;} _c; // alignment: 8
-}
+```yaml
+exp: 8
+mant: 24
```
-This default alignment may be overridden using a special `align()`
-option after the structure is closed:
-```
-struct {
- uint16_t _a; // alignment: 16
- struct { // alignment: 32
- uint32_t _a; // alignment: 32
- string; _b; // alignment: 8
- } _b;
- integer {size = 64;} _c; // alignment: 8
-} align(16)
-```
+##### Enumeration type object
-You may use structures as field types, although they must have a
-_known size_ when running barectf. This means they cannot contain
-sequences or strings.
+A CTF enumeration type.
-A CTF structure field will make barectf produce a `const void*` C function
-parameter. The structure (of known size) will be copied as is to the
-current buffer, respecting its alignment.
+Each label of an enumeration type is mapped to a single value, or to a
+range of values.
-Note that barectf requires inner structures to be at least byte-aligned.
+**Properties**:
-Be careful when using CTF structures for recording binary structures
-declared in C. You need to make sure your C compiler aligns structure
-fields and adds padding exactly in the way you define your equivalent
-CTF structure. For example, using GCC on the x86 architecture, 3 bytes
-are added after field `a` in the following C structure since `b` is
-32-bit aligned:
+| Property | Type | Description | Required? | Default value |
+|---|---|---|---|---|
+| `value-type` | [Integer type object](#integer-type-object) or string (alias name) | Supporting integer type | Required | N/A |
+| `members` | Array of [enumeration type member objects](#enumeration-type-member-object) | Enumeration members | Required | N/A |
-```c
-struct my_struct {
- char a;
- unsigned int b;
-};
-```
+The `members` property must contain at least one element. If the member
+is a string, its associated value is computed as follows:
-It would be wrong to use the following CTF structure:
+ * If the member is the first one of the `members` array, its value
+ is 0.
+ * If the previous member is a string, its value is the previous
+ member's computed value + 1.
+ * If the previous member is a single value member, its value is
+ the previous member's value + 1.
+ * If the previous member is a range member, its value is the previous
+ member's upper bound + 1.
-```
-struct {
- integer {size = 8; signed = true;} a;
- integer {size = 32;} b;
-}
-```
+The member values must not overlap each other.
-since field `b` is byte-aligned by default. This one would work fine:
+**Example**:
-```
-struct {
- integer {size = 8; signed = true;} a;
- integer {size = 32; align = 32;} b;
-}
+```yaml
+class: enum
+value-type: uint8
+members:
+ - ZERO
+ - ONE
+ - TWO
+ - label: SIX
+ value: 6
+ - SE7EN
+ - label: TWENTY TO FOURTY
+ value: [10, 40]
+ - FORTY-ONE
```
-CTF structures can prove very useful for recording protocols with named
-fields when reading the trace. For example, here's the CTF structure
-describing the IPv4 header (excluding options):
+**Equivalent C type**: equivalent C type of supporting integer type
+(see [integer type object documentation](#integer-type-object) above).
-```
-struct ipv4_header {
- integer {size = 4;} version;
- integer {size = 4;} ihl;
- integer {size = 6;} dscp;
- integer {size = 2;} ecn;
- integer {size = 16; byte_order = network;} total_length;
- integer {size = 16; byte_order = network;} identification;
- integer {size = 1;} flag_more_fragment;
- integer {size = 1;} flag_dont_fragment;
- integer {size = 1;} flag_reserved;
- integer {size = 13; byte_order = network;} fragment_offset;
- integer {size = 8;} ttl;
- integer {size = 8;} protocol;
- integer {size = 16; byte_order = network;} header_checksum;
- integer {size = 8;} src_ip_addr[4];
- integer {size = 8;} dst_ip_addr[4];
-}
-```
-Although this complex structure has more than ten independent fields,
-the generated C function would only call a 20-byte `memcpy()`, making
-it fast to record. Bits will be unpacked properly and values displayed
-in a human-readable form by the CTF reader thanks to the CTF metadata.
+###### Enumeration type member object
+The member of a CTF enumeration type.
-#### type aliases
+If it's a string, the string is the member's label, and the members's
+value depends on the last member's value (see explanation in
+[enumeration type object documentation](#enumeration-type-object) above).
-Type aliases associate a name with a type definition. Any type may have
-any name. They are similar to C `typedef`s.
+Otherwise, it's a complete member object, with the following properties:
-Examples:
+| Property | Type | Description | Required? | Default value |
+|---|---|---|---|---|
+| `label` | String | Member's label | Required | N/A |
+| `value` | Integer (single value) or array of two integers (range value) | Member's value | Required | N/A |
-```
-typealias integer {
- size = 16;
- align = 4;
- signed = true;
- byte_order = network;
- base = hex;
- encoding = UTF8;
-} := my_int;
-```
+If the `value` property is an array of two integers, the member's label is
+associated to this range, both lower and upper bounds included. The array's
+first element must be lesser than or equal to the second element.
-```
-typealias floating_point {
- exp_dig = 8;
- mant_dig = 8;
- align = 16;
- byte_order = be;
-} := my_float;
-```
+**Example**:
+```yaml
+label: my enum label
+value: [-25, 78]
```
-typealias string {
- encoding = ASCII;
-} := my_string;
+
+
+##### String type object
+
+A CTF null-terminated string type.
+
+This object has no properties.
+
+**Example**:
+
+```yaml
+class: string
```
+**Equivalent C type**: `const char *`.
+
+
+##### Array type object
+
+A CTF array or sequence (variable-length array) type.
+
+**Properties**:
+
+| Property | Type | Description | Required? | Default value |
+|---|---|---|---|---|
+| `element-type` | [Type object](#type-objects) or string (alias name) | Type of array's elements | Required | N/A |
+| `length` | Positive integer (static array) or string (variable-length array) | Array type's length | Required | N/A |
+
+If the `length` property is a string, the array type has a
+variable length (CTF sequence). In this case, the property's value
+refers to a previous structure field. The `length` property's value
+may be prefixed with one of the following strings to indicate an
+absolute lookup within a previous (or current) dynamic scope:
+
+ * `trace.packet.header.`: trace packet header
+ * `stream.packet.context.`: stream packet context
+ * `stream.event.header.`: stream event header
+ * `stream.event.context.`: stream event context
+ * `event.context.`: event context
+ * `event.payload.`: event payload
+
+The pointed field must have an unsigned integer type.
+
+**Example** (16 bytes):
+
+```yaml
+class: array
+length: 16
+element-type:
+ class: int
+ size: 8
```
-typealias enum : uint32_t {
- ZERO,
- ONE,
- TWO,
- TEN = 10,
- ELEVEN,
- "label with spaces",
- RANGE = 23 ... 193
-} := my_enum;
+
+**Example** (variable-length array of null-terminated strings):
+
+```yaml
+class: array
+length: previous_field
+element-type:
+ class: string
```
+
+##### Structure type object
+
+A CTF structure type, i.e. a list of fields, each field
+having a name and a CTF type.
+
+**Properties**:
+
+| Property | Type | Description | Required? | Default value |
+|---|---|---|---|---|
+| `min-align` | Integer (positive) | Minimum alignment (bits) (power of two) | Optional | 1 |
+| `fields` | Associative array of field names (string) to [type objects](#type-objects) or strings (alias names) | Structure type's fields | Optional | `{}` |
+
+The order of the entries in the `fields` property is important; it is in
+this order that the fields are serialized in binary streams.
+
+**Example**:
+
+```yaml
+class: struct
+min-align: 32
+fields:
+ msg_id: uint8
+ src:
+ class: string
+ dst:
+ class: string
```
-typealias struct {
- uint32_t _length;
- string _other_field;
- float _hello[10][_length];
-} align(8) := my_struct;
+
+
+##### Variant type object
+
+A CTF variant type, i.e. a tagged union of CTF types.
+
+**Properties**:
+
+| Property | Type | Description | Required? | Default value |
+|---|---|---|---|---|
+| `tag` | String | Variant type's tag | Required | N/A |
+| `types` | Associative array of strings to [type objects](#type-objects) or strings (alias names) | Possible types | Required | N/A |
+
+The `tag` property's value refers to a previous structure field.
+The value may be prefixed with one of the following strings to indicate
+an absolute lookup within a previous (or current) dynamic scope:
+
+ * `trace.packet.header.`: trace packet header
+ * `stream.packet.context.`: stream packet context
+ * `stream.event.header.`: stream event header
+ * `stream.event.context.`: stream event context
+ * `event.context.`: event context
+ * `event.payload.`: event payload
+
+The pointed field must have an enumeration type. Each type name in the
+`types` property must have its equivalent member's label in this
+enumeration type. This is how a variant's type is selected using the
+value of its tag.
+
+**Example**:
+
+```yaml
+class: variant
+tag: my_choice
+types:
+ a:
+ class: string
+ b: int32
+ c:
+ class: float
+ size:
+ align: 32
+ exp: 8
+ mant: 24
```
-### running the `barectf` command
+### Running the `barectf` command
Using the `barectf` command-line utility is easy. In its simplest form,
-it outputs a few C99 files out of a CTF metadata file:
+it outputs a CTF metadata file and a few C files out of a
+YAML configuration file:
- barectf metadata
+ barectf config.yaml
-will output in the current working directory:
+will output, in the current working directory:
- * `barectf_bitfield.h`: macros used by tracing functions to pack bits
+ * `metadata`: CTF metadata file
+ * `barectf-bitfield.h`: macros used by tracing functions to pack bits
* `barectf.h`: other macros and prototypes of context/tracing functions
* `barectf.c`: context/tracing functions
-You may also want to produce `static inline` functions if your target
-system has enough memory to hold the extra code:
-
- barectf --static-inline metadata
+`barectf_` is the default name of the files and the default prefix of
+barectf C functions and structures. The prefix is read from the
+configuration file (see the
+[configuration object documentation](#configuration-object)), but
+you may override it on the command line:
-`barectf` is the default name of the files and the default prefix of
-barectf C functions and structures. You may use a custom prefix:
-
- barectf --prefix trace metadata
+ barectf --prefix my_app_ config.yaml
You may also output the files elsewhere:
- barectf --output /custom/path metadata
+ barectf --code-dir src --headers-dir include --metadata-dir ctf config.yaml
+
-### using the generated C99 code
+### Using the generated C code
This section assumes you ran `barectf` with no options:
- barectf metadata
+ barectf config.yaml
-The command generates C99 structures and functions to initialize
-and finalize bare CTF contexts. It also generates as many tracing functions
-as there are events described in the CTF metadata file.
+The command generates C structures and functions to initialize
+barectf contexts, open packets, and close packets. It also generates as many
+tracing functions as there are events defined in the YAML configuration
+file.
-Before starting the record events, you must initialize a barectf
-context. This is done using `barectf_init()`.
+An application should never have to initialize barectf contexts,
+open packets, or close packets; this is the purpose of a specific barectf
+platform, which wraps those calls in its own initialization and
+finalization functions.
-The clock callback parameter (`clock_cb`) is used to get the clock whenever
-a tracing function is called. Each platform has its own way of obtaining
-the a clock value, so this is left to user implementation. The actual
-return type of the clock callback depends on the clock value CTF integer
-type defined in the CTF metadata.
+The barectf project provides a few platforms in the [`platforms`](platforms)
+directory. Each one contains a `README.md` file explaining how to use
+the platform. If you're planning to write your own platform,
+read the next subsection. Otherwise, skip it.
-The `barectf_init()` function name will contain the decimal stream
-ID if you have more than one stream. You must allocate the context
-structure yourself.
-Example:
+#### Writing a barectf platform
-```c
-struct barectf_ctx* barectf_ctx = platform_alloc(sizeof(*barectf_ctx));
+A **_barectf platform_** is responsible for:
-barectf_init(barectf_ctx, buf, 8192, platform_get_clock, NULL);
-```
+ 1. Providing some initialization and finalization functions
+ for the tracing infrastructure of the target. The initialization
+ function is responsible for initializing a barectf context,
+ providing the platform callback functions, and for opening the very
+ first stream packet(s). The finalization function is responsible
+ for closing, usually when not empty, the very last stream
+ packet(s).
+ 2. Implementing the platform callback functions to accomodate the target
+ system. The main purposes of those callback functions are:
+ * Getting the current value of clock(s).
+ * Doing something with a packet once it's full. This is how
+ a ring buffer of packets may be implemented. The platform
+ may also be naive and write the full packets to the file system
+ directly.
-This initializes a barectf context with a buffer of 8192 bytes.
+Thus, the traced application itself should never have to call
+the barectf initialization, packet opening, and packet closing
+funcions. The application only deals with initializing/finalizing
+the platform, and calling the tracing functions.
-After the barectf context is initialized, open a packet using
-`barectf_open_packet()`. If you have any non-special fields in
-your stream packet context, `barectf_open_packet()` accepts a
-parameter for each of them since the packet context is written
-at this moment:
+The following diagram shows how each part connects with
+each other:
-```
-barectf_open_packet(barectf_ctx);
-```
+
+
+The following subsections explain what should exist in each
+platform function.
-Once the packet is opened, you may call any of the tracing functions to record
-CTF events into the context's buffer.
-As an example, let's take the following CTF event definition:
+##### Platform initialization function
+A barectf platform initialization function is responsible for
+initializing barectf context(s) (calling `barectf_init()`,
+where `barectf_` is the configured prefix), and opening the very
+first packet (calling `barectf_stream_open_packet()` with
+target-specific parameters, for each stream, where `stream` is
+the stream name).
+
+barectf generates one context C structure for each defined stream.
+They all contain the same first member, a structure with common
+properties.
+
+barectf generates a single context initialization function:
+
+```c
+void barectf_init(
+ void *ctx,
+ uint8_t *buf,
+ uint32_t buf_size,
+ struct barectf_platform_callbacks cbs,
+ void *data
+);
```
-event {
- name = "my_event";
- id = 0;
- stream_id = 0;
- fields := struct {
- integer {size = 32;} _a;
- integer {size = 14; signed = true;} _b;
- floating_point {exp_dig = 8; mant_dig = 24; align = 32;} _c;
- struct {
- uint32_t _a;
- uint32_t _b;
- } _d;
- string _e;
+
+This function must be called with each stream-specific context
+structure to be used afterwards. The parameters are:
+
+ * `ctx`: stream-specific barectf context (allocated by caller)
+ * `buf`: buffer to use for this stream's packet (allocated by caller)
+ * `buf_size`: size of `buf` in bytes
+ * `cbs`: platform callback functions to be used with this
+ stream-specific context
+ * `data`: user data passed to platform callback functions (`cbs`)
+
+**Example**:
+
+```c
+#define BUF_SZ 4096
+
+void platform_init(/* ... */)
+{
+ struct barectf_my_stream_ctx *ctx;
+ uint8_t *buf;
+ struct my_data *my_data;
+ struct barectf_platform_callbacks cbs = {
+ /* ... */
};
-};
+
+ ctx = platform_alloc(sizeof(*ctx));
+ buf = platform_alloc(BUF_SZ);
+ my_data = platform_alloc(sizeof(*my_data));
+ my_data->ctx = ctx;
+ barectf_init(ctx, buf, BUF_SZ, cbs, my_data);
+
+ /* ... */
+}
```
-In this example, we assume the stream event context and the event context
-are not defined for this event. `barectf` generates the following tracing
-function prototype:
+barectf generates one packet opening and one packet closing
+function per defined stream, since each stream may have custom
+parameters at the packet opening time, and custom offsets of
+fields to write at packet closing time.
+
+The platform initialization should open the very first packet
+of each stream to use because the tracing functions expect the
+current packet to be opened.
+
+Here's an example of a packet opening function prototype:
```c
-int barectf_trace_my_event(
- struct barectf_ctx* ctx,
- uint32_t param_ef__a,
- int16_t param_ef__b,
- float param_ef__c,
- const void* param_ef__d,
- const char* param_ef__e
+void barectf_my_stream_open_packet(
+ struct barectf_my_stream_ctx *ctx,
+ float spc_something
);
```
-When called, this function first calls the clock callback to get a clock
-value as soon as possible. It then proceeds to record each field with
-proper alignment and updates the barectf context. On success, 0 is returned.
-Otherwise, one of the following negative errors is returned:
+The function needs the stream-specific barectf context, as well as any
+custom trace packet header or stream packet context field; in this
+last example, `something` is a floating point number stream packet context
+field.
+
+
+##### barectf packet information API
+
+There's a small API to query stuff about the current packet of a
+given barectf context:
+
+```c
+uint32_t barectf_packet_size(void *ctx);
+int barectf_packet_is_full(void *ctx);
+int barectf_packet_is_empty(void *ctx);
+uint32_t barectf_packet_events_discarded(void *ctx);
+uint8_t *barectf_packet_buf(void *ctx);
+void barectf_packet_set_buf(void *ctx, uint8_t *buf, uint32_t buf_size);
+uint32_t barectf_packet_buf_size(void *ctx);
+int barectf_packet_is_open(void *ctx);
+```
+
+`barectf_packet_is_full()` returns 1 if the context's current packet
+is full (no space left for any event), 0 otherwise.
+
+`barectf_packet_is_empty()` returns 1 if the context's current packet
+is empty (no recorded events), 0 otherwise.
+
+`barectf_packet_events_discarded()` returns the number of lost (discarded)
+events _so far_ for a given stream.
+
+The buffer size (`buf_size` parameter of `barectf_packet_set_buf()` and
+return value of `barectf_packet_buf_size()`) is always a number of bytes.
+
+`barectf_packet_is_open()` returns 1 if the context's current packet
+is open (the packet opening function was called with this context).
+
+
+##### Platform callback functions
+
+The callback functions to implement for a given platform are
+in the generated `barectf_platform_callbacks` C structure. This
+structure will contain:
+
+ * One callback function per defined clock, using the clock's
+ return C type. Those functions must return the current clock
+ values.
+ * `is_backend_full()`: is the back-end full? If a new packet
+ is opened now, does it have its reserved space in the back-end?
+ Return 0 if it does, 1 otherwise.
+ * `open_packet()`: this callback function **must** call the relevant
+ packet opening function.
+ * `close_packet()`: this callback function **must** call the
+ relevant packet closing function _and_ copy/move the current packet
+ to the back-end.
+
+What exactly is a _back-end_ is left to the platform implementor. It
+could be a ring buffer of packets, or it could be dumber: `close_packet()`
+always appends the current packet to some medium, and `is_backend_full()`
+always returns 0 (back-end is never full).
+
+Typically, if `is_backend_full()` returns 0, then the next
+call to `close_packet()` should be able to write the current packet.
+If `is_backend_full()` returns 1, there will be lost (discarded)
+events. If a stream packet context has an `events_discarded` field,
+it will be written to accordingly when a packet is closed.
+
+If a platform needs double buffering, `open_packet()` is the callback
+function where packet buffers would be swapped (before calling
+the barectf packet opening function).
+
+
+##### Platform finalization function
+
+The platform finalization function should be called by the application
+when tracing is no more required. It is responsible for closing the
+very last packet of each stream.
+
+Typically, assuming there's only one stream (named `my_stream` in this
+example), the finalization function will look like this:
+
+```c
+void platform_tracing_finalize(struct platform_data *platform_data)
+{
+ if (barectf_packet_is_open(platform_data->ctx) &&
+ !barectf_packet_is_empty(platform_data->ctx)) {
+ barectf_my_stream_close_packet(platform_data->ctx);
+
+ /*
+ * Do whatever is necessary here to write the packet
+ * to the platform's back-end.
+ */
+ }
+}
+```
+
+That is: if the packet is still open (thus not closed and written yet)
+_and_ it contains at least one event (not empty), close and write the last
+packet.
+
+Note, however, that you might be interested in closing an open empty
+packet, since its packet context could update the discarded events count
+(if there were lost events between the last packet closing time and
+now, which is quite possible if the back-end became full after closing
+and writing the previous packet).
+
+
+#### Calling the generated tracing functions
+
+Calling the generated tracing functions is what the traced application
+actually does.
+
+For a given prefix named `barectf`, a given stream named `stream`, and
+a given event named `event`, the generated tracing function name is
+`barectf_stream_trace_event()`.
+
+The first parameter of a tracing function is always the stream-specific
+barectf context. Then, in this order:
+
+ * One parameter for each custom event header field
+ (prefixed with `seh_`)
+ * One parameter for each custom stream event context field
+ (prefixed with `sec_`)
+ * One parameter for each custom event context field
+ (prefixed with `ec_`)
+ * One parameter for each custom event payload field
+ (prefixed with `ep_`)
+
+A tracing function returns nothing: it either succeeds (the event
+is serialized in the current packet) or fails when there's no
+space left (the context's discarded events count is incremented).
+
+**Example**:
+
+Given the following [event object](#event-object), named `my_event`,
+placed in a stream named `default` with no custom event header/stream event
+context fields:
+
+```yaml
+context-type:
+ class: struct
+ fields:
+ msg_id:
+ class: int
+ size: 16
+payload-type:
+ class: struct
+ fields:
+ src:
+ class: string
+ dst:
+ class: string
+ a_id:
+ class: int
+ size: 3
+ b_id:
+ class: int
+ size: 7
+ signed: true
+ c_id:
+ class: int
+ size: 15
+ amt:
+ class: float
+ align: 32
+ size:
+ exp: 8
+ mant: 24
+```
- * `-EBARECTF_NOSPC`: no space left in the context's buffer; the event
- was **not** recorded. You should call `barectf_close_packet()` to finalize the
- CTF packet.
+barectf will generate the following tracing function prototype:
-`barectf_close_packet()` may be called at any time.
-When `barectf_close_packet()` returns, the packet is complete and ready
-to be read by a CTF reader. CTF packets may be concatenated in a single
-CTF stream file. You may reuse the same context and buffer to record another
-CTF packet, as long as you call `barectf_open_packet()` before calling any
-tracing function.
+```c
+/* trace (stream "default", event "my_event") */
+void barectf_default_trace_my_event(
+ struct barectf_default_ctx *ctx,
+ uint16_t ec_msg_id,
+ const char *ep_src,
+ const char *ep_dst,
+ uint8_t ep_a_id,
+ int8_t ep_b_id,
+ uint16_t ep_c_id,
+ float amt
+);
+```
-### reading CTF traces
+### Reading CTF traces
-To form a complete CTF trace, put your CTF metadata file (it should be
-named `metadata`) and your binary stream files (concatenations of CTF
-packets written by C code generated by barectf) in the same directory.
+To form a complete CTF trace, the `metadata` file generated by the
+`barectf` command-line tool and the binary stream files generated
+by the application (or by an external consumer, depending on the
+platform) should be placed in the same directory.
To read a CTF trace, use [Babeltrace](http://www.efficios.com/babeltrace).
-Babeltrace is packaged by most major distributions (`babeltrace`).
-Babeltrace ships with a command-line utility that can convert a CTF trace
-to human-readable text output. Also, it includes a Python binding so
-that you may analyze a CTF trace using a custom script.
+Babeltrace is packaged by most major distributions as the `babeltrace`
+package. Babeltrace ships with a command-line utility that can convert a
+CTF trace to human-readable text output. Also, it includes Python bindings
+so that you may analyze a CTF trace using a custom script.
In its simplest form, the `babeltrace` command-line converter is quite
easy to use:
babeltrace /path/to/directory/containing/ctf/files
-See `babeltrace --help` for more options.
-
-You may also use the Python 3 binding of Babeltrace to create custom
-analysis scripts.
+See `babeltrace --help` and `man babeltrace` for more options.
projects / barectf.git / blobdiff