Commit | Line | Data |
---|---|---|
2facbdc3 PP |
1 | = babeltrace2-intro(7) |
2 | :manpagetype: manual page | |
3 | :revdate: 14 September 2019 | |
838dd456 PP |
4 | |
5 | ||
2facbdc3 PP |
6 | == NAME |
7 | ||
8 | babeltrace2-intro - Introduction to Babeltrace 2 | |
9 | ||
838dd456 | 10 | |
2facbdc3 | 11 | == DESCRIPTION |
838dd456 | 12 | |
2facbdc3 | 13 | This manual page is an introduction to the Babeltrace~2 project. |
838dd456 | 14 | |
2facbdc3 PP |
15 | The <<what-is,``WHAT IS BABELTRACE~2?''>> section describes the |
16 | parts of the project and shows the major changes from Babeltrace~1 | |
17 | to Babeltrace~2 while the <<concepts,``BABELTRACE~2 | |
18 | CONCEPTS''>> section defines the core concepts of Babeltrace~2. | |
838dd456 | 19 | |
2facbdc3 PP |
20 | The <<graph-repr,``TRACE PROCESSING GRAPH REPRESENTATION''>> section |
21 | shows how some <<concepts,concepts>> are visually represented in other | |
22 | Babeltrace~2 manual pages. | |
838dd456 PP |
23 | |
24 | ||
25 | [[what-is]] | |
2facbdc3 PP |
26 | == WHAT IS BABELTRACE~2? |
27 | ||
28 | Babeltrace~2 is an open-source software project of which the | |
29 | purpose is to process or convert | |
838dd456 PP |
30 | https://en.wikipedia.org/wiki/Tracing_(software)[traces]. |
31 | ||
2facbdc3 | 32 | The Babeltrace~2 project includes the following parts: |
838dd456 | 33 | |
2facbdc3 | 34 | [[libbabeltrace2]]Babeltrace~2 library (libbabeltrace2):: |
838dd456 PP |
35 | A shared library with a C API. |
36 | + | |
c1f82f3b | 37 | With libbabeltrace2, you can programmatically create <<plugin,plugins>> |
2facbdc3 PP |
38 | and <<comp-cls,component classes>>, build and run <<graph,trace |
39 | processing graphs>>, and more (see the <<concepts,``BABELTRACE~2 | |
40 | CONCEPTS''>> section for more details about those concepts). | |
41 | + | |
42 | All the other Babeltrace~2 parts rely on this library. | |
838dd456 | 43 | |
2facbdc3 | 44 | [[babeltrace2]]`babeltrace2` command-line program:: |
c1f82f3b | 45 | A command-line interface which uses libbabeltrace2 to load plugins, |
2facbdc3 PP |
46 | create a trace processing graph, create <<comp,components>>, connect |
47 | their <<port,ports>> correctly, and run the graph. | |
838dd456 | 48 | + |
2facbdc3 PP |
49 | You can also use `babeltrace2` to list the available plugins or to |
50 | <<query,query>> an object from a component class. | |
838dd456 | 51 | + |
c1f82f3b | 52 | See man:babeltrace2(1). |
838dd456 | 53 | |
2facbdc3 PP |
54 | [[python-bindings]]Babeltrace~2 Python bindings:: |
55 | A Python~3 package (`bt2`) which offers a Pythonic interface of | |
c1f82f3b | 56 | libbabeltrace2. |
838dd456 | 57 | + |
2facbdc3 PP |
58 | You can perform the same operations which are available in |
59 | libbabeltrace2 with the Python bindings, but more conveniently and with | |
60 | less code. However, the Python bindings are less performant than | |
61 | libbabeltrace2. | |
838dd456 | 62 | |
2facbdc3 PP |
63 | Babeltrace~2 project's plugins:: |
64 | The Babeltrace~2 <<plugin,plugins>> shipped with the project. | |
838dd456 | 65 | + |
2facbdc3 PP |
66 | Those plugins are not special in that they only rely on libbabeltrace2 |
67 | and you don't need them to use libbabeltrace2, man:babeltrace2(1), or | |
68 | the Python bindings. However, the project's plugins provide many widely | |
69 | used trace format encoders/decoders as well as common <<graph,trace | |
70 | processing graph>> utilities. | |
838dd456 | 71 | + |
2facbdc3 | 72 | The Babeltrace~2 project's plugins are: |
838dd456 PP |
73 | + |
74 | -- | |
75 | `ctf`:: | |
2facbdc3 PP |
76 | https://diamon.org/ctf/[Common Trace Format] (CTF) input/output, |
77 | including the LTTng live source. | |
838dd456 | 78 | + |
c1f82f3b | 79 | See man:babeltrace2-plugin-ctf(7). |
838dd456 PP |
80 | |
81 | `lttng-utils`:: | |
2facbdc3 | 82 | Graph utilities specific to https://lttng.org/[LTTng] traces. |
838dd456 | 83 | + |
c1f82f3b | 84 | See man:babeltrace2-plugin-lttng-utils(7). |
838dd456 PP |
85 | |
86 | `text`:: | |
2facbdc3 | 87 | Plain text input/output. |
838dd456 | 88 | + |
c1f82f3b | 89 | See man:babeltrace2-plugin-text(7). |
838dd456 PP |
90 | |
91 | `utils`:: | |
2facbdc3 | 92 | Common graph utilities (muxer, trimmer, counter, dummy sink). |
838dd456 | 93 | + |
c1f82f3b | 94 | See man:babeltrace2-plugin-utils(7). |
838dd456 PP |
95 | -- |
96 | ||
838dd456 | 97 | |
2facbdc3 | 98 | === Changes since Babeltrace~1 |
838dd456 | 99 | |
2facbdc3 PP |
100 | This manual page is an introduction to Babeltrace~2, a rewrite of |
101 | Babeltrace~1 with a focus on extensibility, flexibility, and | |
102 | interoperability. | |
838dd456 | 103 | |
2facbdc3 PP |
104 | Babeltrace~1 exists since 2010. The major improvements brought by |
105 | Babeltrace~2 are: | |
838dd456 | 106 | |
2facbdc3 PP |
107 | * Full plugin support: any user can distribute a Babeltrace~2 |
108 | plugin and, as long as <<libbabeltrace2,libbabeltrace2>> finds it, any | |
109 | application linked to libbabeltrace2 can load it and use it. | |
838dd456 | 110 | + |
2facbdc3 PP |
111 | Plugins are not just trace format encoders and decoders: they provide |
112 | source, filter, and sink <<comp-cls,component classes>> so that you can | |
113 | connect specialized, reusable components together in a trace processing | |
114 | graph to create a customized trace conversion or analysis device. | |
838dd456 | 115 | |
2facbdc3 PP |
116 | * In order to support user components, many of the objects of |
117 | libbabeltrace2 have a reference count. The possible reference cycles | |
118 | are handled internally so that the library's API is clean and | |
119 | predictable. | |
120 | + | |
121 | Objects which are often used on the "fast path" (for example, events, | |
122 | fields, and clock snapshots) are unique: they have no reference count. | |
838dd456 | 123 | |
2facbdc3 PP |
124 | * All the parts of the Babeltrace~2 project run on the major |
125 | operating systems, including Windows and macOS. | |
838dd456 PP |
126 | |
127 | ||
128 | [[concepts]] | |
2facbdc3 PP |
129 | == BABELTRACE~2 CONCEPTS |
130 | ||
131 | This section defines the main concepts of the Babeltrace~2 project. | |
132 | ||
133 | These concepts translate into types and functions in | |
c1f82f3b | 134 | <<libbabeltrace2,libbabeltrace2>> and its <<python-bindings,Python |
838dd456 | 135 | bindings>>, but also as command-line actions and options in the |
2facbdc3 PP |
136 | <<babeltrace2,`babeltrace2` program>>. The other Babeltrace~2 |
137 | manual pages assume that you are familiar with the following | |
138 | definitions. | |
838dd456 | 139 | |
2facbdc3 PP |
140 | Some Babeltrace~2 concepts are interdependent: it is normal to jump |
141 | from one definition to another to understand the big picture. | |
838dd456 PP |
142 | |
143 | [[comp-cls]]Component class:: | |
2facbdc3 PP |
144 | A reusable class which you can instantiate as one or more |
145 | <<comp,components>> within a <<graph,trace processing graph>>. | |
838dd456 | 146 | + |
2facbdc3 PP |
147 | There are three types of component classes used to create the three |
148 | types of components: source, filter, and sink. | |
838dd456 | 149 | + |
2facbdc3 | 150 | A component class implements methods, one of which is an initialization |
838dd456 PP |
151 | method, or constructor, to create a component. You pass _initialization |
152 | parameters_ to this method to customize the created component. For | |
2facbdc3 PP |
153 | example, the initialization method of the compcls:source.ctf.fs |
154 | component class accepts a mandatory manparam:source.ctf.fs:inputs | |
155 | parameter which is an array of file system path(s) to the CTF trace(s). | |
156 | It also accepts an optional manparam:source.ctf.fs:clock-class-offset-ns | |
157 | parameter which is an offset, in nanoseconds, to add to all the clock | |
158 | classes (descriptors of stream clocks) found in the traces's metadata. | |
159 | + | |
160 | A component class can have a description and a help text. | |
838dd456 PP |
161 | |
162 | [[comp]]Component:: | |
163 | A node within a <<graph,trace processing graph>>. | |
164 | + | |
165 | There are three types of components: | |
166 | + | |
167 | -- | |
168 | Source component:: | |
2facbdc3 | 169 | An input component which produces <<msg,messages>>. |
838dd456 | 170 | + |
2facbdc3 | 171 | Examples: CTF files input, log file input, LTTng live input, random |
838dd456 PP |
172 | event generator. |
173 | ||
174 | Filter component:: | |
2facbdc3 PP |
175 | An intermediate component which can transform the messages it |
176 | consumes, augment them, sort them, discard them, or create new ones. | |
838dd456 | 177 | + |
2facbdc3 PP |
178 | Examples: filter which removes messages based on an expression, |
179 | filter which adds debugging information to selected events, message | |
180 | muxer, trace trimmer. | |
838dd456 PP |
181 | |
182 | Sink component:: | |
2facbdc3 PP |
183 | An output component which consumes messages and usually writes them |
184 | to one or more formatted files. | |
838dd456 | 185 | + |
2facbdc3 PP |
186 | Examples: log file output, CTF files output, pretty-printed plain text |
187 | output. | |
838dd456 PP |
188 | -- |
189 | + | |
190 | Components are connected together within a <<graph,trace processing | |
191 | graph>> through their <<port,ports>>. Source components have output | |
192 | ports, sink components have input ports, and filter components have | |
193 | both. | |
194 | + | |
195 | A component is the instance of a <<comp-cls,component class>>. The terms | |
2facbdc3 | 196 | _component_ and _component class instance_ are equivalent. |
838dd456 PP |
197 | + |
198 | Within a trace processing graph, each component has a unique name. This | |
199 | is not the name of its component class, but an instance name. If `human` | |
2facbdc3 PP |
200 | is a component class name, than `Nancy` and `John` could be component |
201 | names. | |
202 | + | |
203 | Once a <<graph,graph>> is configured (the first time it runs), you | |
204 | cannot add components to it for the remaining graph's lifetime. | |
838dd456 PP |
205 | |
206 | [[port]]Port:: | |
207 | A connection point, on a <<comp,component>>, from which are sent or | |
2facbdc3 PP |
208 | where are received <<msg,messages>> when the <<graph,trace |
209 | processing graph>> runs. | |
838dd456 | 210 | + |
2facbdc3 PP |
211 | An output port is from where messages are sent. An input port is where |
212 | messages are received. Source components have output ports, sink | |
838dd456 PP |
213 | components have input ports, and filter components have both. |
214 | + | |
2facbdc3 | 215 | You can only connect an output port to a single input port. |
838dd456 | 216 | + |
2facbdc3 PP |
217 | All ports do not need to be connected. |
218 | + | |
219 | A filter or sink component receiving messages from its input ports | |
220 | is said to _consume_ messages. | |
838dd456 PP |
221 | + |
222 | The link between an output port and input port is a <<conn,connection>>. | |
223 | + | |
2facbdc3 PP |
224 | Once a <<graph,graph>> is configured (the first time it runs), you |
225 | cannot connect ports for the remaining graph's lifetime. | |
838dd456 PP |
226 | |
227 | [[conn]]Connection:: | |
228 | The link between an output <<port,port>> and an input port through | |
2facbdc3 PP |
229 | which <<msg,messages>> flow when a <<graph,trace processing |
230 | graph>> runs. | |
231 | ||
232 | [[msg-iter]]Message iterator:: | |
233 | An iterator on an input <<port,port>> of which the returned elements | |
234 | are <<msg,messages>>. | |
235 | + | |
236 | A <<comp,component>> or another message iterator can create many message | |
237 | iterators on a single input port, before or while the <<graph,trace | |
238 | processing graph>> runs. | |
838dd456 | 239 | |
2facbdc3 PP |
240 | [[msg]]Message:: |
241 | The element of a <<msg-iter,message iterator>>. | |
242 | + | |
243 | Messages flow from output <<port,ports>> to input ports. | |
838dd456 | 244 | + |
2facbdc3 PP |
245 | A source <<comp,component>> <<msg-iter,message iterator>> produces |
246 | messages, while a sink component consumes them. A filter component | |
247 | message iterator can both consume and produce messages. | |
838dd456 | 248 | + |
2facbdc3 | 249 | The main types of messages are: |
838dd456 PP |
250 | + |
251 | -- | |
252 | Event:: | |
2facbdc3 | 253 | A trace event record within a packet or within a stream. |
838dd456 PP |
254 | |
255 | Packet beginning:: | |
256 | The beginning of a packet within a stream. | |
257 | + | |
2facbdc3 | 258 | A packet is a conceptual container of events. |
838dd456 PP |
259 | |
260 | Packet end:: | |
261 | The end of a packet within a stream. | |
262 | ||
263 | Stream beginning:: | |
264 | The beginning of a stream. | |
265 | + | |
2facbdc3 | 266 | A stream is a conceptual container of packets and/or events. |
838dd456 | 267 | + |
2facbdc3 PP |
268 | Usually, a given source component's output port sends packet and event |
269 | messages which belong to a single stream, but it's not required. | |
838dd456 PP |
270 | |
271 | Stream end:: | |
272 | The end of a stream. | |
273 | ||
274 | Discarded events:: | |
275 | A count of discarded events within a given time interval for a given | |
276 | stream. | |
277 | ||
278 | Discarded packets:: | |
279 | A count of discarded packets within a given time interval for a | |
280 | given stream. | |
281 | -- | |
282 | ||
283 | [[graph]]Trace processing graph:: | |
284 | A https://en.wikipedia.org/wiki/Filter_graph[filter graph] where | |
2facbdc3 | 285 | nodes are <<comp,components>> and <<msg,messages>> flow from |
838dd456 PP |
286 | output <<port,ports>> to input ports. |
287 | + | |
288 | You can build a trace processing graph with | |
2facbdc3 PP |
289 | <<libbabeltrace2,libbabeltrace2>>, with the |
290 | <<python-bindings,Babeltrace~2 Python bindings>>, or with the | |
291 | man:babeltrace2-run(1) and man:babeltrace2-convert(1) CLI commands. | |
838dd456 | 292 | + |
2facbdc3 PP |
293 | When a trace processing graph _runs_, the sink components consume |
294 | messages from their input ports, making all the graph's | |
295 | <<msg-iter,message iterators>> work one message at a time to perform the | |
296 | trace conversion or analysis duty. | |
838dd456 PP |
297 | |
298 | [[plugin]]Plugin:: | |
2facbdc3 PP |
299 | A container, or package, of <<comp-cls,component classes>> as a |
300 | shared library or Python module. | |
838dd456 PP |
301 | + |
302 | Each component class within a plugin has a type (source, filter, or | |
303 | sink) and a name. The type and name pair is unique within a given | |
304 | plugin. | |
305 | + | |
2facbdc3 PP |
306 | <<libbabeltrace2,libbabeltrace2>> can load a plugin (`.so`, `.dll`, or |
307 | `.py` file) at run time: the result is a plugin object in which you can | |
308 | find a specific component class and instantiate it within a | |
309 | <<graph,trace processing graph>> as a <<comp,component>>. | |
838dd456 | 310 | + |
2facbdc3 PP |
311 | The <<babeltrace2,`babeltrace2` program>> uses the |
312 | 'COMP-CLS-TYPE.PLUGIN-NAME.COMP-CLS-NAME' format to identify a specific | |
313 | component class within a specific plugin. 'COMP-CLS-TYPE' is either | |
314 | `source` (or `src`), `filter` (or `flt`), or `sink`. | |
838dd456 | 315 | + |
2facbdc3 | 316 | You can list the available Babeltrace~2 plugins with the |
c1f82f3b | 317 | man:babeltrace2-list-plugins(1) command. |
838dd456 PP |
318 | |
319 | [[query]]Query:: | |
320 | An operation with which you can get a named object from a | |
2facbdc3 | 321 | <<comp-cls,component class>>, possibly with custom query parameters. |
838dd456 PP |
322 | + |
323 | The plain text metadata stream of a CTF trace and the available LTTng | |
2facbdc3 PP |
324 | live sessions of a given LTTng relay daemon are examples of query |
325 | objects. | |
838dd456 | 326 | + |
2facbdc3 PP |
327 | You can use <<libbabeltrace2,libbabeltrace2>>, the |
328 | <<python-bindings,Babeltrace~2 Python bindings>>, or the | |
329 | man:babeltrace2-query(1) CLI command to query a component class's | |
330 | object. | |
838dd456 PP |
331 | |
332 | ||
333 | [[graph-repr]] | |
2facbdc3 PP |
334 | == TRACE PROCESSING GRAPH REPRESENTATION |
335 | ||
336 | In the Babeltrace~2 manual pages, a component is represented with a | |
337 | box. The box has the <<comp-cls,component class>> type, | |
338 | <<plugin,plugin>> name, and component class name at the top. Just below, | |
339 | between square brackets, is its component name within the <<graph,trace | |
838dd456 | 340 | processing graph>>. Each <<port,port>> is represented with an `@` symbol |
2facbdc3 PP |
341 | on the border(s) of the component box with its name inside the box. |
342 | Output ports are on the box's right border while input ports are on the | |
343 | box's left border. | |
838dd456 PP |
344 | |
345 | For example, here's a source component box: | |
346 | ||
347 | ---- | |
348 | +------------+ | |
349 | | src.ctf.fs | | |
350 | | [my-src] | | |
351 | | | | |
352 | | stream0 @ | |
353 | | stream1 @ | |
354 | | stream2 @ | |
355 | +------------+ | |
356 | ---- | |
357 | ||
2facbdc3 PP |
358 | This one is an instance of the compcls:source.ctf.fs component class |
359 | named `my-src`. It has three output ports named `stream0`, `stream1`, | |
360 | and `stream2`. | |
838dd456 PP |
361 | |
362 | A trace processing graph is represented with multiple component boxes | |
363 | connected together. The <<conn,connections>> are arrows from output | |
364 | ports to input ports. | |
365 | ||
366 | For example, here's a simple conversion graph: | |
367 | ||
368 | ---- | |
369 | +------------+ +-----------------+ +------------------+ | |
370 | | src.ctf.fs | | flt.utils.muxer | | sink.text.pretty | | |
371 | | [ctf] | | [muxer] | | [text] | | |
372 | | | | | | | | |
373 | | stream0 @--->@ in0 out @--->@ in | | |
374 | | stream1 @--->@ in1 | +------------------+ | |
375 | | stream2 @--->@ in2 | | |
376 | +------------+ @ in3 | | |
377 | +-----------------+ | |
378 | ---- | |
379 | ||
2facbdc3 PP |
380 | Note that input port `in3` of component `muxer` is not connected in this |
381 | example. | |
838dd456 PP |
382 | |
383 | Sometimes, we symbolically represent other resources which are consumed | |
384 | from or produced by components. In this case, arrows are used, but they | |
2facbdc3 PP |
385 | do not go to or from port symbols (`@`), except for messages. For |
386 | example, in the graph above, the `ctf` source component consumes a CTF | |
387 | trace and the `text` sink component prints plain text to the terminal, | |
388 | so here's a more complete diagram: | |
838dd456 PP |
389 | |
390 | ---- | |
2facbdc3 PP |
391 | CTF trace |
392 | | | |
393 | | +------------+ +-----------------+ +------------------+ | |
394 | | | src.ctf.fs | | flt.utils.muxer | | sink.text.pretty | | |
395 | '-->| [ctf] | | [muxer] | | [text] | | |
396 | | | | | | | | |
397 | | stream0 @--->@ in0 out @--->@ in | | |
398 | | stream1 @--->@ in1 | +-----+------------+ | |
399 | | stream2 @--->@ in2 | | | |
400 | +------------+ @ in3 | '--> Terminal | |
401 | +-----------------+ | |
838dd456 PP |
402 | ---- |
403 | ||
404 | Here's another example of a more complex graph which splits a specific | |
405 | stream using some criteria: | |
406 | ||
407 | ---- | |
408 | +------------+ +-----------------+ +------------------+ | |
409 | | src.ctf.fs | | flt.utils.muxer | | sink.text.pretty | | |
410 | | [ctf-in] | | [muxer] | | [text] | | |
411 | | | | | | | | |
412 | | stream0 @--->@ in0 out @--->@ in | | |
413 | | stream1 @--->@ in1 | +------------------+ | |
414 | | stream2 @-. @ in2 | | |
415 | +------------+ | +-----------------+ +-------------+ | |
416 | | | sink.ctf.fs | | |
417 | | | [ctf-out0] | | |
418 | | +-------------------+ | | | |
419 | | | flt.some.splitter | .->@ in | | |
420 | | | [splitter] | | +-------------+ | |
421 | | | | | | |
422 | '->@ in A @-' +-------------+ | |
423 | | B @-. | sink.ctf.fs | | |
424 | +-------------------+ | | [ctf-out1] | | |
425 | | | | | |
426 | '->@ in | | |
427 | +-------------+ | |
428 | ---- | |
429 | ||
430 | ||
431 | include::common-footer.txt[] | |
432 | ||
433 | ||
2facbdc3 PP |
434 | == SEE ALSO |
435 | ||
c1f82f3b | 436 | man:babeltrace2(1) |