Commit | Line | Data |
---|---|---|
fffd4c7d PP |
1 | barectf |
2 | ======= | |
3 | ||
4 | **barectf** is a command-line utility which generates pure C99 | |
5 | code that is able to write native | |
6 | [CTF](http://git.efficios.com/?p=ctf.git;a=blob_plain;f=common-trace-format-specification.txt;hb=master) | |
7 | (the Common Trace Format) out of a pre-written CTF metadata file. | |
8 | ||
9 | You will find barectf interesting if: | |
10 | ||
11 | 1. You need to trace a program. | |
12 | 2. You need tracing to be as fast as possible, but also very flexible: | |
13 | record integers of custom sizes, custom floating point numbers, | |
14 | enumerations mapped to a specific integer type, structure fields, | |
15 | NULL-terminated strings, static and dynamic arrays, etc. | |
16 | 3. You need to be able to convert the recorded binary events to | |
17 | human-readable text, as well as analyze them with Python scripts | |
18 | ([Babeltrace](http://www.efficios.com/babeltrace) does all that, | |
19 | given a CTF input). | |
20 | 4. You _cannot_ use [LTTng](http://lttng.org/), an efficient tracing | |
21 | framework for the Linux kernel and Linux/BSD user applications, which | |
22 | outputs CTF. | |
23 | ||
24 | The target audience of barectf is developers who need to trace bare metal | |
25 | systems (without an operating system). The code produced by barectf | |
26 | is pure C99 and is lightweight enough to fit on a tiny microcontroller. | |
27 | Each event described in the CTF metadata input becomes one C function with | |
28 | one parameter mapped to one event field. CTF data is recorded in a buffer of | |
29 | any size provided by the user. This buffer corresponds to one CTF packet. | |
30 | The generated tracing functions report when the buffer is full. The user | |
31 | is entirely responsible for the buffering scheme: leave the buffer in memory, | |
32 | save it to some permanent storage, swap it with another empty buffer and | |
33 | concatenate recorded packets, etc. | |
34 | ||
35 | barectf is written in Python 3 and currently uses | |
cc8ceb9a | 36 | [pytsdl](https://github.com/efficios/pytsdl) to parse the CTF metadata file |
fffd4c7d PP |
37 | provided by the user. |
38 | ||
39 | ||
40 | installing | |
41 | ---------- | |
42 | ||
43 | Make sure you have `pip` for Python 3. On the latest Ubuntu releases, | |
44 | it is called `pip3`: | |
45 | ||
46 | sudo apt-get install python3-pip | |
47 | ||
48 | On Ubuntu 12.04, you need to install `setuptools` first, then use | |
49 | `easy_install3` to install `pip3`: | |
50 | ||
51 | sudo apt-get install python3-setuptools | |
52 | sudo easy_install3 pip | |
53 | ||
54 | Install barectf: | |
55 | ||
56 | sudo pip3 install barectf | |
57 | ||
58 | ||
59 | using | |
60 | ----- | |
61 | ||
62 | Using barectf involves: | |
63 | ||
64 | 1. Writing the CTF metadata file describing the various headers, | |
65 | contexts and event fields. | |
66 | 2. Running the `barectf` command to generate C99 files out of | |
67 | the CTF metadata file. | |
68 | 3. Using the generated C code in your specific application. | |
69 | ||
70 | The following subsections explain the three steps above. | |
71 | ||
72 | ||
73 | ### writing the CTF metadata | |
74 | ||
75 | The **Common Trace Format** is a specialized file format for recording | |
76 | trace data. CTF is designed to be very fast to write and very flexible. | |
77 | All headers, contexts and event fields written in binary files are | |
78 | described using a custom C-like, declarative language, TSDL (Trace | |
79 | Stream Description Language). The file containing this description is | |
80 | called the **CTF metadata**. The latter may be automatically generated | |
81 | by a tracer, like it is the case of LTTng, or written by hand. This | |
82 | metadata file is then used by CTF trace readers to know the layout of | |
83 | CTF binary files containing actual event contexts and fields. | |
84 | ||
85 | The CTF metadata file contains several blocks describing various CTF | |
86 | binary layouts. A CTF trace file is a concatenation of several CTF | |
87 | packets. Here's the anatomy of a CTF packet: | |
88 | ||
89 | ![CTF packet anatomy](doc/ctf-packet.png) | |
90 | ||
91 | A CTF packet belongs to a specific CTF stream. While the packet header | |
92 | is the same for all streams of a given CTF trace, everything else is | |
93 | specified per stream. Following this packet header is a packet context, | |
94 | and then actual recorded events. Each event starts with a mandatory | |
95 | header (same event header for all events of a given stream). The event | |
96 | header is followed by an optional event context with a layout shared | |
97 | by all events of a given stream. Then follows another optional event | |
98 | context, although this one has a layout specific to the event type. | |
99 | Finally, event fields are written. | |
100 | ||
101 | barectf asks you to write the CTF metadata by hand. Although its official | |
102 | [specification](http://git.efficios.com/?p=ctf.git;a=blob_plain;f=common-trace-format-specification.txt;hb=master) | |
103 | is thorough, you will almost always start from this template: | |
104 | ||
105 | ``` | |
106 | /* CTF 1.8 */ | |
107 | ||
224b997f | 108 | /* a few useful standard integer aliases */ |
fffd4c7d PP |
109 | typealias integer {size = 8; align = 8;} := uint8_t; |
110 | typealias integer {size = 16; align = 16;} := uint16_t; | |
111 | typealias integer {size = 32; align = 32;} := uint32_t; | |
112 | typealias integer {size = 64; align = 64;} := uint64_t; | |
113 | typealias integer {size = 8; align = 8; signed = true;} := int8_t; | |
114 | typealias integer {size = 16; align = 16; signed = true;} := int16_t; | |
115 | typealias integer {size = 32; align = 32; signed = true;} := int32_t; | |
116 | typealias integer {size = 64; align = 64; signed = true;} := int64_t; | |
117 | ||
224b997f | 118 | /* IEEE 754 standard-precision floating point alias */ |
fffd4c7d PP |
119 | typealias floating_point { |
120 | exp_dig = 8; | |
121 | mant_dig = 24; | |
122 | align = 32; | |
123 | } := float; | |
124 | ||
224b997f | 125 | /* IEEE 754 double-precision floating point alias */ |
fffd4c7d PP |
126 | typealias floating_point { |
127 | exp_dig = 11; | |
128 | mant_dig = 53; | |
129 | align = 64; | |
130 | } := double; | |
131 | ||
224b997f | 132 | /* trace block */ |
fffd4c7d | 133 | trace { |
224b997f | 134 | /* CTF version 1.8; leave this as is */ |
fffd4c7d PP |
135 | major = 1; |
136 | minor = 8; | |
137 | ||
224b997f PP |
138 | /* |
139 | * Native byte order (`le` or `be`). This is used by barectf to generate | |
fffd4c7d PP |
140 | * the appropriate code when writing data to the packet. |
141 | */ | |
142 | byte_order = le; | |
143 | ||
224b997f PP |
144 | /* |
145 | * Packet header. All packets (buffers) will have the same header. | |
fffd4c7d PP |
146 | * |
147 | * Special fields recognized by barectf (must appear in this order): | |
148 | * | |
149 | * magic: will be set to CTF's magic number (must be the first field) | |
224b997f | 150 | * (32-bit unsigned integer) (mandatory) |
fffd4c7d PP |
151 | * stream_id: will be set to the ID of the stream associated with |
152 | * this packet (unsigned integer of your choice) (mandatory) | |
153 | */ | |
154 | packet.header := struct { | |
155 | uint32_t magic; | |
156 | uint32_t stream_id; | |
157 | }; | |
158 | }; | |
159 | ||
224b997f | 160 | /* environment variables; you may add custom entries */ |
fffd4c7d PP |
161 | env { |
162 | domain = "bare"; | |
163 | tracer_name = "barectf"; | |
164 | tracer_major = 0; | |
165 | tracer_minor = 1; | |
166 | tracer_patchlevel = 0; | |
167 | }; | |
168 | ||
224b997f | 169 | /* clock descriptor */ |
fffd4c7d | 170 | clock { |
224b997f | 171 | /* clock name */ |
fffd4c7d PP |
172 | name = my_clock; |
173 | ||
224b997f | 174 | /* clock frequency (Hz) */ |
fffd4c7d PP |
175 | freq = 1000000000; |
176 | ||
224b997f | 177 | /* optional clock value offset; offset from Epoch is: offset * (1 / freq) */ |
fffd4c7d PP |
178 | offset = 0; |
179 | }; | |
180 | ||
224b997f | 181 | /* alias for integer used to hold clock cycles */ |
fffd4c7d PP |
182 | typealias integer { |
183 | size = 32; | |
184 | ||
224b997f | 185 | /* map to the appropriate clock using its name */ |
fffd4c7d PP |
186 | map = clock.my_clock.value; |
187 | } := my_clock_int_t; | |
188 | ||
224b997f PP |
189 | /* |
190 | * A stream. You may have as many streams as you want. Events are unique | |
fffd4c7d PP |
191 | * within their own stream. The main advantage of having multiple streams |
192 | * is having different event headers, stream event contexts and stream | |
193 | * packet contexts for each one. | |
194 | */ | |
195 | stream { | |
224b997f PP |
196 | /* |
197 | * Mandatory stream ID (must fit the integer type of | |
198 | * `trace.packet.header.stream_id`). | |
fffd4c7d PP |
199 | */ |
200 | id = 0; | |
201 | ||
224b997f PP |
202 | /* |
203 | * Mandatory packet context. This structure follows the packet header | |
fffd4c7d PP |
204 | * (see `trace.packet.header`) immediately in CTF binary streams. |
205 | * | |
206 | * Special fields recognized by barectf: | |
207 | * | |
208 | * timestamp_begin: will be set to the current clock value when opening | |
209 | * the packet (same integer type as the clock's value) | |
210 | * timestamp_end: will be set to the current clock value when closing | |
211 | * the packet (same integer type as the clock's value) | |
212 | * content_size: will be set to the content size, in bits, of this | |
213 | * stream (unsigned 32-bit or 64-bit integer) (mandatory) | |
214 | * packet_size: will be set to the packet size, in bits, of this | |
215 | * stream (unsigned 32-bit or 64-bit integer) (mandatory) | |
216 | * cpu_id: if present, the barectf_open_packet() function of this | |
217 | * stream will accept an additional parameter to specify the | |
218 | * ID of the CPU associated with this stream (a given | |
219 | * stream should only be written to by a specific CPU) | |
220 | * (unsigned integer of your choice) | |
221 | * | |
222 | * `timestamp_end` must be present if `timestamp_begin` exists. | |
223 | */ | |
224 | packet.context := struct { | |
225 | my_clock_int_t timestamp_begin; | |
226 | my_clock_int_t timestamp_end; | |
227 | uint64_t content_size; | |
228 | uint64_t packet_size; | |
229 | uint32_t cpu_id; | |
230 | }; | |
231 | ||
224b997f PP |
232 | /* |
233 | * Mandatory event header. All events recorded in this stream will start | |
fffd4c7d PP |
234 | * with this structure. |
235 | * | |
236 | * Special fields recognized by barectf: | |
237 | * | |
238 | * id: will be filled by the event ID corresponding to a tracing | |
239 | * function (unsigned integer of your choice) | |
240 | * timestamp: will be filled by the current clock's value (same integer | |
241 | * type as the clock's value) | |
242 | */ | |
243 | event.header := struct { | |
244 | uint32_t id; | |
245 | my_clock_int_t timestamp; | |
246 | }; | |
247 | ||
224b997f PP |
248 | /* |
249 | * Optional stream event context (you may remove the whole block or leave | |
fffd4c7d | 250 | * the structure empty if you don't want any). This structure follows the |
224b997f PP |
251 | * event header (see `stream.event.header`) immediately in CTF binary |
252 | * streams. | |
fffd4c7d PP |
253 | */ |
254 | event.context := struct { | |
255 | int32_t _some_stream_event_context_field; | |
256 | }; | |
257 | }; | |
258 | ||
224b997f PP |
259 | /* |
260 | * An event. Events have an ID, a name, an optional context and fields. An | |
fffd4c7d PP |
261 | * event is associated to a specific stream using its stream ID. |
262 | */ | |
263 | event { | |
224b997f PP |
264 | /* |
265 | * Mandatory event name. This is used by barectf to generate the suffix | |
fffd4c7d PP |
266 | * of this event's corresponding tracing function, so make sure it follows |
267 | * the C identifier syntax even though it's a quoted string here. | |
268 | */ | |
269 | name = "my_event"; | |
270 | ||
224b997f PP |
271 | /* |
272 | * Mandatory event ID (must fit the integer type of in | |
273 | * `stream.event.header.id` of the associated stream). | |
fffd4c7d PP |
274 | */ |
275 | id = 0; | |
276 | ||
224b997f | 277 | /* ID of the stream in which this event will be recorded */ |
fffd4c7d PP |
278 | stream_id = 0; |
279 | ||
224b997f PP |
280 | /* |
281 | * Optional event context (you may remove the whole block or leave the | |
fffd4c7d PP |
282 | * structure empty if you don't want one). This structure follows the |
283 | * stream event context (if it exists) immediately in CTF binary streams. | |
284 | */ | |
285 | context := struct { | |
286 | int32_t _some_event_context_field; | |
287 | }; | |
288 | ||
224b997f PP |
289 | /* |
290 | * Mandatory event fields (although the structure may be left empty if this | |
fffd4c7d PP |
291 | * event has no fields). This structure follows the event context (if it |
292 | * exists) immediately in CTF binary streams. | |
293 | */ | |
294 | fields := struct { | |
295 | uint32_t _a; | |
296 | uint32_t _b; | |
297 | uint16_t _c; | |
298 | string _d; | |
299 | }; | |
300 | }; | |
301 | ``` | |
302 | ||
303 | The top `/* CTF 1.8 */` is actually needed right there, and as is, since it | |
304 | acts as a CTF metadata magic number for CTF readers. | |
305 | ||
306 | Only one stream and one event (belonging to this single stream) are described | |
307 | in this template, but you may add as many as you need. | |
308 | ||
309 | The following subsections describe the features of CTF metadata supported | |
310 | by barectf. | |
311 | ||
312 | ||
313 | #### types | |
314 | ||
315 | The supported structure field types are: | |
316 | ||
317 | * **integers** of any size (64-bit and less), any alignment (power of two) | |
318 | * **floating point numbers** of any total size (64-bit and less), any | |
319 | alignment (power of two) | |
320 | * NULL-terminated **strings** of bytes | |
321 | * **enumerations** associated with a specific integer type | |
322 | * **static** and **dynamic arrays** of any type | |
323 | * **structures** containing only integers, floating point numbers, | |
324 | enumerations and _static_ arrays | |
325 | ||
326 | CTF also supports _variants_ (dynamic selection between different types), | |
327 | but barectf **does not**. Any detected variant will throw an error when | |
328 | running `barectf`. | |
329 | ||
330 | ||
331 | ##### integers | |
332 | ||
333 | CTF integers are defined like this: | |
334 | ||
335 | ``` | |
336 | integer { | |
224b997f | 337 | /* mandatory size in bits (64-bit and less) */ |
fffd4c7d PP |
338 | size = 16; |
339 | ||
224b997f PP |
340 | /* |
341 | * Optional alignment in bits (power of two). Default is 8 when the | |
fffd4c7d PP |
342 | * size is a multiple of 8, and 1 otherwise. |
343 | */ | |
344 | align = 16; | |
345 | ||
224b997f | 346 | /* optional signedness (`true` or `false`); default is unsigned */ |
fffd4c7d PP |
347 | signed = true; |
348 | ||
224b997f PP |
349 | /* |
350 | * Optional byte order (`le`, `be`, `native` or `network`). `native` | |
fffd4c7d PP |
351 | * will use the byte order specified by the `trace.byte_order`. |
352 | * Default is `native`. | |
353 | */ | |
354 | byte_order = le; | |
355 | ||
224b997f PP |
356 | /* |
357 | * Optional display base, used to display the integer value when | |
fffd4c7d PP |
358 | * reading the trace. Valid values are 2 (or `binary`, `bin` and `b`), |
359 | * 8 (or `o`, `oct` or `octal`), 10 (or `u`, `i`, `d`, `dec` or | |
360 | * `decimal`), and 16 (or `x`, `X`, `p`, `hex` or `hexadecimal`). | |
361 | * Default is 10. | |
362 | */ | |
363 | base = hex; | |
364 | ||
224b997f PP |
365 | /* |
366 | * Encoding (if this integer represents a character). Valid values | |
fffd4c7d PP |
367 | * are `none`, `UTF8` and `ASCII`. Default is `none`. |
368 | */ | |
369 | encoding = UTF8; | |
370 | } | |
371 | ``` | |
372 | ||
373 | The size (the only mandatory property) does _not_ have to be a power of two: | |
374 | ||
375 | ``` | |
376 | integer {size = 23;} | |
377 | ``` | |
378 | ||
379 | is perfectly valid. | |
380 | ||
381 | A CTF integer field will make barectf produce a corresponding C integer | |
382 | function parameter with an appropriate size. For example, the 23-bit integer | |
383 | above would produce an `uint32_t` parameter (of which only the first 23 | |
384 | least significant bits will be written to the trace), while the first | |
385 | 16-bit one will produce an `int16_t` parameter. | |
386 | ||
387 | The `integer` block also accepts a `map` property which is only used | |
388 | when defining the integer used to carry the value of a specified | |
389 | clock. You may always follow the example above. | |
390 | ||
391 | ||
392 | ##### floating point numbers | |
393 | ||
394 | CTF floating point numbers are defined like this: | |
395 | ||
396 | ``` | |
397 | floating_point { | |
224b997f | 398 | /* exponent size in bits */ |
fffd4c7d PP |
399 | exp_dig = 8; |
400 | ||
224b997f | 401 | /* mantissa size in bits */ |
fffd4c7d PP |
402 | mant_dig = 24; |
403 | ||
224b997f PP |
404 | /* |
405 | * Optional alignment (power of two). Default is 8 when the total | |
fffd4c7d PP |
406 | * size (exponent + mantissa) is a multiple of 8, and 1 otherwise. |
407 | */ | |
408 | align = 32; | |
409 | ||
224b997f PP |
410 | /* |
411 | * Optional byte order (`le`, `be`, `native` or `network`). `native` | |
fffd4c7d PP |
412 | * will use the byte order specified by the `trace.byte_order`. |
413 | * Default is `native`. | |
414 | */ | |
415 | byte_order = le; | |
416 | } | |
417 | ``` | |
418 | ||
419 | If a CTF floating point number is defined with an 8-bit exponent, a 24-bit | |
420 | mantissa and a 32-bit alignment, its barectf C function parameter type will | |
421 | be `float`. It will be `double` for an 11-bit exponent, 53-bit mantissa | |
422 | and 64-bit aligned CTF floating point number. Any other configuration | |
423 | will produce a `uint64_t` function parameter (you will need to cast your | |
424 | custom floating point number to this when calling the tracing function). | |
425 | ||
426 | ||
427 | ##### strings | |
428 | ||
429 | CTF strings are pretty simple to define: | |
430 | ||
431 | ``` | |
432 | string | |
433 | ``` | |
434 | ||
435 | They may also have an encoding property: | |
436 | ||
437 | ``` | |
438 | string { | |
224b997f | 439 | /* encoding: `none`, `UTF8` or `ASCII`; default is `none` */ |
fffd4c7d PP |
440 | encoding = UTF8; |
441 | } | |
442 | ``` | |
443 | ||
444 | CTF strings are always byte-aligned. | |
445 | ||
446 | A CTF string field will make barectf produce a corresponding C function | |
447 | parameter of type `const char*`. Bytes will be copied from this pointer | |
448 | until a byte of value 0 is found (which will also be written to the | |
449 | buffer to mark the end of the recorded string). | |
450 | ||
451 | ||
452 | ##### enumerations | |
453 | ||
454 | CTF enumerations associate labels to ranges of integer values. They | |
455 | are a great way to trace named states using an integer. Here's an | |
456 | example: | |
457 | ||
458 | ``` | |
459 | enum : uint32_t { | |
460 | ZERO, | |
461 | ONE, | |
462 | TWO, | |
463 | TEN = 10, | |
464 | ELEVEN, | |
465 | "label with spaces", | |
466 | RANGE = 23 ... 193 | |
467 | } | |
468 | ``` | |
469 | ||
470 | Unless the first entry specifies a value, CTF enumerations are | |
471 | always started at 0. They work pretty much like their C counterpart, | |
472 | although they support ranges and literal strings as labels. | |
473 | ||
474 | CTF enumerations are associated with a CTF integer type (`uint32_t` | |
475 | above). This identifier must be an existing integer type alias. | |
476 | ||
477 | A CTF enumeration field will make barectf produce a corresponding C | |
478 | integer function parameter compatible with the associated CTF integer type. | |
479 | ||
480 | ||
481 | ##### static arrays | |
482 | ||
483 | Structure field names may be followed by a subscripted constant to | |
484 | define a static array of the field type: | |
485 | ||
486 | ``` | |
487 | struct { | |
488 | integer {size = 16;} _field[10]; | |
489 | } | |
490 | ``` | |
491 | ||
492 | In the above structure, `_field` is a static array of ten 16-bit integers. | |
493 | ||
494 | A CTF static array field will make barectf produce a `const void*` C function | |
495 | parameter. Bytes will be copied from this pointer to match the total static | |
496 | array size. In the example above, the integer size is 16-bit, thus its | |
497 | default alignment is 8-bit, so 20 bytes would be copied. | |
498 | ||
499 | The inner element of a CTF static array _must be at least byte-aligned_ | |
500 | (8-bit), either by forcing its alignment, or by ensuring it manually | |
501 | when placing fields one after the other. This means the following static | |
502 | array is valid for barectf: | |
503 | ||
504 | ``` | |
505 | struct { | |
224b997f | 506 | /* ... */ |
fffd4c7d PP |
507 | integer {size = 5;} _field[10]; |
508 | } | |
509 | ``` | |
510 | ||
511 | as long as the very first 5-bit, 1-bit aligned integer element starts | |
512 | on an 8-bit boundary. | |
513 | ||
514 | ||
515 | ##### dynamic arrays | |
516 | ||
517 | Just like static arrays, dynamic arrays are defined using a subscripted | |
518 | length, albeit in this case, this length refers to another field using | |
519 | the dot notation. Dynamic arrays are called _sequences_ in the CTF | |
520 | specification. | |
521 | ||
522 | Here's an example: | |
523 | ||
524 | ``` | |
525 | struct { | |
526 | uint32_t _length; | |
527 | integer {size = 16;} _field[_length]; | |
528 | } | |
529 | ``` | |
530 | ||
531 | In the above structure, `_field` is a dynamic array of `_length` | |
532 | 16-bit integers. | |
533 | ||
534 | There are various scopes to which a dynamic array may refer: | |
535 | ||
536 | * no prefix: previous field in the same structure, or in parent | |
537 | structures until found | |
538 | * `event.fields.` prefix: field of the event fields | |
539 | * `event.context.` prefix: field of the event context if it exists | |
540 | * `stream.event.context.` prefix: field of the stream event context | |
541 | if it exists | |
542 | * `stream.event.header.` prefix: field of the event header | |
543 | * `stream.packet.context.` prefix: field of the packet context | |
544 | * `trace.packet.header.` prefix: field of the packet header | |
545 | * `env.` prefix: static property of the environment block | |
546 | ||
547 | Here's another, more complex example: | |
548 | ||
549 | ``` | |
550 | struct { | |
551 | uint32_t _length; | |
552 | string _other_field[stream.event.context.length]; | |
553 | float _static_array_of_dynamic_arrays[10][_length]; | |
554 | } | |
555 | ``` | |
556 | ||
557 | The above examples also demonstrates that dynamic arrays and static | |
558 | arrays may contain eachother. `_other_field` is a dynamic array of | |
559 | `stream.event.context.length` strings. `_static_array_of_dynamic_arrays` | |
560 | is a static array of 10 dynamic arrays of `_length` floating point | |
561 | numbers. This syntax follows the C language. | |
562 | ||
563 | A CTF dynamic array field will make barectf produce a `const void*` C function | |
564 | parameter. Bytes will be copied from this pointer to match the | |
565 | total dynamic array size. The previously recorded length will be | |
566 | found automatically (always an offset from the beginning of the | |
567 | stream packet, or from the beginning of the current event). | |
568 | ||
569 | barectf has a few limitations concerning dynamic arrays: | |
570 | ||
571 | * The inner element of a CTF dynamic array _must be at least byte-aligned_ | |
572 | (8-bit), either by forcing its alignment, or by ensuring it manually | |
573 | when placing fields one after the other. | |
574 | * The length type must be a 32-bit, byte-aligned unsigned integer | |
575 | with a native byte order. | |
576 | ||
577 | ||
578 | ##### structures | |
579 | ||
580 | Structures contain fields associating a name to a type. The fields | |
581 | are recorded in the specified order within the CTF binary stream. | |
582 | ||
583 | Here's an example: | |
584 | ||
585 | ``` | |
586 | struct { | |
587 | uint32_t _a; | |
588 | int16_t _b; | |
589 | string {encoding = ASCII;} _c; | |
590 | } | |
591 | ``` | |
592 | ||
593 | The default alignment of a structure is the largest alignment amongst | |
594 | its fields. For example, the following structure has a 32-bit alignment: | |
595 | ||
596 | ``` | |
597 | struct { | |
224b997f PP |
598 | uint16_t _a; /* alignment: 16 */ |
599 | struct { /* alignment: 32 */ | |
600 | uint32_t _a; /* alignment: 32 */ | |
601 | string; _b; /* alignment: 8 */ | |
fffd4c7d | 602 | } _b; |
224b997f | 603 | integer {size = 64;} _c; /* alignment: 8 */ |
fffd4c7d PP |
604 | } |
605 | ``` | |
606 | ||
607 | This default alignment may be overridden using a special `align()` | |
608 | option after the structure is closed: | |
609 | ||
610 | ``` | |
611 | struct { | |
224b997f PP |
612 | uint16_t _a; |
613 | struct { | |
614 | uint32_t _a; | |
615 | string; _b; | |
fffd4c7d | 616 | } _b; |
224b997f | 617 | integer {size = 64;} _c; |
fffd4c7d PP |
618 | } align(16) |
619 | ``` | |
620 | ||
621 | You may use structures as field types, although they must have a | |
622 | _known size_ when running barectf. This means they cannot contain | |
623 | sequences or strings. | |
624 | ||
625 | A CTF structure field will make barectf produce a `const void*` C function | |
626 | parameter. The structure (of known size) will be copied as is to the | |
627 | current buffer, respecting its alignment. | |
628 | ||
629 | Note that barectf requires inner structures to be at least byte-aligned. | |
630 | ||
631 | Be careful when using CTF structures for recording binary structures | |
632 | declared in C. You need to make sure your C compiler aligns structure | |
633 | fields and adds padding exactly in the way you define your equivalent | |
634 | CTF structure. For example, using GCC on the x86 architecture, 3 bytes | |
635 | are added after field `a` in the following C structure since `b` is | |
636 | 32-bit aligned: | |
637 | ||
638 | ```c | |
639 | struct my_struct { | |
640 | char a; | |
641 | unsigned int b; | |
642 | }; | |
643 | ``` | |
644 | ||
645 | It would be wrong to use the following CTF structure: | |
646 | ||
647 | ``` | |
648 | struct { | |
649 | integer {size = 8; signed = true;} a; | |
650 | integer {size = 32;} b; | |
651 | } | |
652 | ``` | |
653 | ||
654 | since field `b` is byte-aligned by default. This one would work fine: | |
655 | ||
656 | ``` | |
657 | struct { | |
658 | integer {size = 8; signed = true;} a; | |
659 | integer {size = 32; align = 32;} b; | |
660 | } | |
661 | ``` | |
662 | ||
663 | CTF structures can prove very useful for recording protocols with named | |
664 | fields when reading the trace. For example, here's the CTF structure | |
665 | describing the IPv4 header (excluding options): | |
666 | ||
667 | ``` | |
668 | struct ipv4_header { | |
669 | integer {size = 4;} version; | |
670 | integer {size = 4;} ihl; | |
671 | integer {size = 6;} dscp; | |
672 | integer {size = 2;} ecn; | |
673 | integer {size = 16; byte_order = network;} total_length; | |
674 | integer {size = 16; byte_order = network;} identification; | |
675 | integer {size = 1;} flag_more_fragment; | |
676 | integer {size = 1;} flag_dont_fragment; | |
677 | integer {size = 1;} flag_reserved; | |
678 | integer {size = 13; byte_order = network;} fragment_offset; | |
679 | integer {size = 8;} ttl; | |
680 | integer {size = 8;} protocol; | |
681 | integer {size = 16; byte_order = network;} header_checksum; | |
682 | integer {size = 8;} src_ip_addr[4]; | |
683 | integer {size = 8;} dst_ip_addr[4]; | |
684 | } | |
685 | ``` | |
686 | ||
d38ca19b | 687 | Although this complex structure has more than ten independent fields, |
503819da PP |
688 | the generated C function would only call a 20-byte `memcpy()`, making |
689 | it fast to record. Bits will be unpacked properly and values displayed | |
690 | in a human-readable form by the CTF reader thanks to the CTF metadata. | |
691 | ||
fffd4c7d PP |
692 | |
693 | #### type aliases | |
694 | ||
695 | Type aliases associate a name with a type definition. Any type may have | |
696 | any name. They are similar to C `typedef`s. | |
697 | ||
698 | Examples: | |
699 | ||
700 | ``` | |
701 | typealias integer { | |
702 | size = 16; | |
703 | align = 4; | |
704 | signed = true; | |
705 | byte_order = network; | |
706 | base = hex; | |
707 | encoding = UTF8; | |
708 | } := my_int; | |
709 | ``` | |
710 | ||
711 | ``` | |
712 | typealias floating_point { | |
713 | exp_dig = 8; | |
714 | mant_dig = 8; | |
715 | align = 16; | |
716 | byte_order = be; | |
717 | } := my_float; | |
718 | ``` | |
719 | ||
720 | ``` | |
721 | typealias string { | |
722 | encoding = ASCII; | |
723 | } := my_string; | |
724 | ``` | |
725 | ||
726 | ``` | |
727 | typealias enum : uint32_t { | |
728 | ZERO, | |
729 | ONE, | |
730 | TWO, | |
731 | TEN = 10, | |
732 | ELEVEN, | |
733 | "label with spaces", | |
734 | RANGE = 23 ... 193 | |
735 | } := my_enum; | |
736 | ``` | |
737 | ||
738 | ``` | |
739 | typealias struct { | |
740 | uint32_t _length; | |
741 | string _other_field; | |
742 | float _hello[10][_length]; | |
743 | } align(8) := my_struct; | |
744 | ``` | |
745 | ||
746 | ||
747 | ### running the `barectf` command | |
748 | ||
749 | Using the `barectf` command-line utility is easy. In its simplest form, | |
750 | it outputs a few C99 files out of a CTF metadata file: | |
751 | ||
752 | barectf metadata | |
753 | ||
754 | will output in the current working directory: | |
755 | ||
756 | * `barectf_bitfield.h`: macros used by tracing functions to pack bits | |
757 | * `barectf.h`: other macros and prototypes of context/tracing functions | |
758 | * `barectf.c`: context/tracing functions | |
759 | ||
760 | You may also want to produce `static inline` functions if your target | |
761 | system has enough memory to hold the extra code: | |
762 | ||
763 | barectf --static-inline metadata | |
764 | ||
765 | `barectf` is the default name of the files and the default prefix of | |
766 | barectf C functions and structures. You may use a custom prefix: | |
767 | ||
768 | barectf --prefix trace metadata | |
769 | ||
770 | You may also output the files elsewhere: | |
771 | ||
772 | barectf --output /custom/path metadata | |
773 | ||
774 | ### using the generated C99 code | |
775 | ||
776 | This section assumes you ran `barectf` with no options: | |
777 | ||
778 | barectf metadata | |
779 | ||
e7dc6cb6 | 780 | The command generates C99 structures and functions to initialize |
fffd4c7d PP |
781 | and finalize bare CTF contexts. It also generates as many tracing functions |
782 | as there are events described in the CTF metadata file. | |
783 | ||
784 | Before starting the record events, you must initialize a barectf | |
785 | context. This is done using `barectf_init()`. | |
786 | ||
787 | The clock callback parameter (`clock_cb`) is used to get the clock whenever | |
788 | a tracing function is called. Each platform has its own way of obtaining | |
789 | the a clock value, so this is left to user implementation. The actual | |
790 | return type of the clock callback depends on the clock value CTF integer | |
791 | type defined in the CTF metadata. | |
792 | ||
793 | The `barectf_init()` function name will contain the decimal stream | |
794 | ID if you have more than one stream. You must allocate the context | |
795 | structure yourself. | |
796 | ||
797 | Example: | |
798 | ||
799 | ```c | |
800 | struct barectf_ctx* barectf_ctx = platform_alloc(sizeof(*barectf_ctx)); | |
801 | ||
802 | barectf_init(barectf_ctx, buf, 8192, platform_get_clock, NULL); | |
803 | ``` | |
804 | ||
805 | This initializes a barectf context with a buffer of 8192 bytes. | |
806 | ||
807 | After the barectf context is initialized, open a packet using | |
808 | `barectf_open_packet()`. If you have any non-special fields in | |
809 | your stream packet context, `barectf_open_packet()` accepts a | |
810 | parameter for each of them since the packet context is written | |
811 | at this moment: | |
812 | ||
813 | ``` | |
814 | barectf_open_packet(barectf_ctx); | |
815 | ``` | |
816 | ||
817 | Once the packet is opened, you may call any of the tracing functions to record | |
818 | CTF events into the context's buffer. | |
819 | ||
820 | As an example, let's take the following CTF event definition: | |
821 | ||
822 | ``` | |
823 | event { | |
824 | name = "my_event"; | |
825 | id = 0; | |
826 | stream_id = 0; | |
827 | fields := struct { | |
828 | integer {size = 32;} _a; | |
829 | integer {size = 14; signed = true;} _b; | |
830 | floating_point {exp_dig = 8; mant_dig = 24; align = 32;} _c; | |
831 | struct { | |
832 | uint32_t _a; | |
833 | uint32_t _b; | |
834 | } _d; | |
835 | string _e; | |
836 | }; | |
837 | }; | |
838 | ``` | |
839 | ||
840 | In this example, we assume the stream event context and the event context | |
841 | are not defined for this event. `barectf` generates the following tracing | |
842 | function prototype: | |
843 | ||
844 | ```c | |
845 | int barectf_trace_my_event( | |
3d994ca1 PP |
846 | struct barectf_ctx* ctx, |
847 | uint32_t param_ef__a, | |
848 | int16_t param_ef__b, | |
849 | float param_ef__c, | |
850 | const void* param_ef__d, | |
851 | const char* param_ef__e | |
fffd4c7d PP |
852 | ); |
853 | ``` | |
854 | ||
855 | When called, this function first calls the clock callback to get a clock | |
856 | value as soon as possible. It then proceeds to record each field with | |
857 | proper alignment and updates the barectf context. On success, 0 is returned. | |
858 | Otherwise, one of the following negative errors is returned: | |
859 | ||
860 | * `-EBARECTF_NOSPC`: no space left in the context's buffer; the event | |
861 | was **not** recorded. You should call `barectf_close_packet()` to finalize the | |
862 | CTF packet. | |
863 | ||
864 | `barectf_close_packet()` may be called at any time. | |
865 | When `barectf_close_packet()` returns, the packet is complete and ready | |
866 | to be read by a CTF reader. CTF packets may be concatenated in a single | |
867 | CTF stream file. You may reuse the same context and buffer to record another | |
868 | CTF packet, as long as you call `barectf_open_packet()` before calling any | |
869 | tracing function. | |
870 | ||
871 | ||
872 | ### reading CTF traces | |
873 | ||
874 | To form a complete CTF trace, put your CTF metadata file (it should be | |
875 | named `metadata`) and your binary stream files (concatenations of CTF | |
876 | packets written by C code generated by barectf) in the same directory. | |
877 | ||
878 | To read a CTF trace, use [Babeltrace](http://www.efficios.com/babeltrace). | |
879 | Babeltrace is packaged by most major distributions (`babeltrace`). | |
880 | Babeltrace ships with a command-line utility that can convert a CTF trace | |
881 | to human-readable text output. Also, it includes a Python binding so | |
882 | that you may analyze a CTF trace using a custom script. | |
883 | ||
884 | In its simplest form, the `babeltrace` command-line converter is quite | |
885 | easy to use: | |
886 | ||
887 | babeltrace /path/to/directory/containing/ctf/files | |
888 | ||
889 | See `babeltrace --help` for more options. | |
890 | ||
891 | You may also use the Python 3 binding of Babeltrace to create custom | |
892 | analysis scripts. |