Philippe Proulx [Sat, 20 Jul 2019 21:16:24 +0000 (17:16 -0400)]
lib: create input port msg iterator from self {msg iterator, sink comp.}
This patch makes an input port message iterator have a link to the
upstream message iterators it needs and vice versa.
The motivation behind this is to eventually be able to transmit some
state or property automatically to all the upstream message iterators,
recursively, of a given message iterator without any special user code.
For example, consider this graph of message iterators (arrow means "is
an upstream message iterator of"):
A <──┐
├──┬── C <──┬── F
B <──┘ │ │
│ E <──┘
D <─────┘
G <───── H
Here, setting the state/property on F would also set it on A, B, D, C,
and E. Setting it on C would set it on A, B, and D. Setting it on H
would only set it on G.
The message iterator graph, which can be considered like another,
"dynamic" graph on top of the static component graph used only to limit
a message iterator graph's topology, is always directed and acyclic,
where each node has a single parent. In other words, it is not permitted
that two message iterators use the same downstream message iterator, for
example:
A <──┐
├───── C <───── F
B <──┘ │ │
│ │
D <─────────┴────────┘
(D has both C and F as parents).
This is so that each message iterator and its upstream message iterators
constitute a single, private state.
This patch replaces
bt_self_component_port_input_message_iterator_create() with
bt_self_component_port_input_message_iterator_create_from_message_iterator()
and
bt_self_component_port_input_message_iterator_create_from_sink_component().
bt_self_component_port_input_message_iterator_create_from_sink_component()
does nothing with the self sink component parameter (except checking
that it's not `NULL`), but it's needed to make the functions type safe.
Internally, an input port message iterator contains both a weak pointer
to its downstream message iterator and an array of weak pointers to its
upstream message iterators.
When you create a message iterator with
bt_self_component_port_input_message_iterator_create_from_message_iterator(),
the function sets the new message iterator's downstream message iterator
to the provided self message iterator. It also adds the new message
iterator to the self message iterator's array of upstream message
iterators.
On message iterator finalization:
* The message iterator removes itself as the downstream message iterator
of all its upstream message iterators. It also clears the upstream
message iterator array.
* The message iterator removes itself as one of the upstream message
iterators of its downstream message iterator.
I didn't find any race condition regarding the new message iterator
links, but I could be wrong. The future and more tests will tell us.
In the `bt2` Python package:
* _UserComponentInputPort.create_message_iterator() is removed.
* _UserMessageIterator._create_input_port_message_iterator() is added
(accepts a user component input port).
* _UserSinkComponent._create_input_port_message_iterator() is added
(accepts a user component input port).
Tests are updated accordingly.
I added a test for
_UserMessageIterator._create_input_port_message_iterator() specifically
(needs a filter message iterator) because we use
_UserSinkComponent._create_input_port_message_iterator() in the current
tests.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I46cefca445353c453fbd9404b97687b81fa5f443
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1732
Simon Marchi [Tue, 23 Jul 2019 19:03:26 +0000 (15:03 -0400)]
bt2: fix some whitespace issues
Change-Id: Ic49586dd2e19ad175ad44190d519ce2e857605c1
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1747
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Wed, 24 Jul 2019 03:08:17 +0000 (23:08 -0400)]
lib: make default query implementation return INVALID_OBJECT, remove UNSUPPORTED status
Queries against component classes that don't implement the query method
currently return a special UNSUPPORTED status. There is however no need
for a different status than the one a query method returns when it
doesn't know about the object, INVALID_OBJECT.
Make the default implemention of "query" return INVALID_OBJECT, and
remove any trace of UNSUPPORTED, including in the Python bindings.
Change-Id: I43f59ef749f3803a832a1ad14e640d03424b3420
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1755
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Wed, 24 Jul 2019 02:49:46 +0000 (22:49 -0400)]
lib: remove LOADING_ERROR status
Remove the LOADING_ERROR status, use ERROR instead. It's not so useful
to return a separate kind of error, now that we have a way to
communicate detailed error causes.
Change-Id: I45682b12748fdf8a8466211323dd8563f01d423b
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1754
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Wed, 24 Jul 2019 02:38:20 +0000 (22:38 -0400)]
bt2: remove NonexistentClockSnapshot exception type
The NonexistentClockSnapshot exception type is used when trying to get a
clock snapshot from an object that doesn't have clock snapshots (because
of the way it's configured). It does not have much value over using a
standard exception type. Replace it with ValueError.
In the test add some validation about the error message, because it
would be possible for the test to be a false positive if some ValueError
is raised for another reason (a programming error on our part, for
example).
Change-Id: Iebf86bc53e578407bb60f2145246dbd110cafaac
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1753
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Wed, 24 Jul 2019 01:56:58 +0000 (21:56 -0400)]
bt2: prepend underscore to exceptions not meant to be raised by user
The exceptions Error, MemoryError and LoadingError are only meant to be
raised by the Python bindings, in response to corresponding status codes
from the Babeltrace API. As per our convention, names of classes not
meant to be instantiated directly by the user are prepended with an
underscore, so change these accordingly.
Note that they are still accessible to the user if they need to catch
an exception:
try:
...
except bt2._Error:
...
Change-Id: If094d817dac3c507b6bf3e1e794373f1c7fc33e4
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1752
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Tue, 23 Jul 2019 06:08:59 +0000 (02:08 -0400)]
bt2: rename CreationError to MemoryError, handle it in and out of Python bindings
Babeltrace API calls can explicitly return the MEMORY_ERROR status,
which is currently converted to a bt2.Error in Python. If this
bt2.Error goes back to the native side of the bindings, is is converted
back to the ERROR status. This is not ideal, as the status value should
stay MEMORY_ERROR when unwinding the stack.
Also, when creation functions fail (return NULL/None), we currently
raise a CreationError. Those creation functions only return NULL/None
if there were memory issues, there is not other way they can
legitimately fail (otherwise they would return a status). So it would
make sense for this to be reported as a MEMORY_ERROR.
To address these two issues, this patch:
- renames bt2.CreationError to bt2.MemoryError
- makes utils._handle_func_status raise a bt2.MemoryError when receiving
the MEMORY_ERROR status
- makes the native bindings return MEMORY_ERROR when catching a
bt2.MemoryError
Since there's no easy way to generate an actual MemoryError, I tested
this manually by hacking bt_graph_create to make it fail.
Change-Id: I4969e8ccd9618d8361fe45d0973df980bc9b7abd
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1751
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Wed, 17 Jul 2019 22:44:23 +0000 (18:44 -0400)]
bt2: make bt2.Error wrap current thread's error
This patch makes the Python bindings integrate better with the
Babeltrace error framework.
When a Babeltrace API call fails, it returns an ERROR or MEMORY_ERROR
status and appends a cause to the current thread's error object. When
that API call is made from the Python, we convert that to raising a
bt2.Error execption. With this patch, we now steal the current thread's
error object and give it to the bt2.Error exception object.
The Python code can catch this exception and consult the error causes
in the error object:
def _consume(self):
try:
something_that_raises_a_bt2_error()
except bt2.Error as exc:
for cause in exc:
print(cause)
If the Python code catches the exception and does nothing else with
it (as in the example above), the exception object is destroyed,
destroying the wrapped bt_error object with it. It can be seen as if
the Python code catches the error and recovers from it.
If the Python code lets the exception propagate back to the C code, like
this:
def _consume(self):
something_that_raises_a_bt2_error()
... the bt2.Error is converted back to an ERROR status. But we now also
take back the bt_error object from the bt2.Error exception object and
restore it as the current thread's error object, so that the error is
propagated. We also append a new cause to it, with the traceback of
where the bt2.Error exception was raised.
A more complex case is if the user raises their own exception from the
bt2.Error, using Python exception chaining:
def _consume(self):
try:
something_that_raises_a_bt2_error()
except bt2.Error as exc:
raise MyOwnExceptionType from exc
In this case, we start by restoring the bt_error as the thread's error
object, to put things back as they were before we called Python. We
then append one cause per exception in the chain, starting with the end
of the chain. That is, one cause for the bt2.Error (with the traceback
to where it was raised) and one cause for the MyOwnExceptionType
exception (with the traceback to where it was raised).
When it handles a bt2.Error, the Python code can obtain information
about the error, mainly to access the list of causes. It does so by
accessing the bt2.Error as a sequence of causes. Each cause kind (from
unknown actor, from component class actor, from component actor and from
message iterator actor) is represented by a class, that gives access to
the appropriate properties of the cause (e.g. component class name, port
name, etc).
Various design choices:
- Some causes have a `component_class_type` property, which returns the
type of component class in which the cause happened. This property
returns a value of the `bt_component_class_type` enum. The user can
compare the returned value against the properties of the newly
introduced ComponentClassType class.
- Because it's now non-trivial, the Error type was moved to the new
error.py file.
- Add BT_CURRENT_THREAD_MOVE_ERROR_AND_RESET macro, which calls
bt_current_thread_move_error and then clears the passed lvalue. It's not
essential, but may help avoid mistakes, since once you moved back the
error into place, it's no longer yours.
- Removed the raise bt2.Error in message._create_from_ptr. If we get an
unexpected message type, there's not a lot we can do, it's just a
programming error / bug that needs to be fixed. If it happens, we will
get a KeyError with the key that was not found, so it should be pretty
obvious.
Change-Id: Ibb63d57838a248fadda9cb741d920538fe588680
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1738
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Tue, 23 Jul 2019 04:32:05 +0000 (00:32 -0400)]
bt2: make bt2 add graph listener wrappers append error causes
If bt_bt2_graph_add_port_added_listener or
bt_bt2_graph_add_ports_connected_listener fail, it can be either because
one of the calls to the Babeltrace API fails or a Python object creation
fails. On the Python side, we raise a bt2.Error on failure. This seems
fine, although in a subsequent patch, it will become a pre-condition for
raising bt2.Error that we have an error set for the current thread.
Add some calls to BT_CURRENT_THREAD_ERROR_APPEND_CAUSE_FROM_UNKNOWN in
the errors paths of these wrappers that don't come from the library,
i.e. on which there would not already be an error cause appended.
On the Python side, use bt2.Error directly instead of
utils._raise_bt2_error. Since these were the last call sites of
utils._raise_bt2_error, remove it.
Change-Id: I1556b626a941e9cc127fb036d6e3b9b6346fde88
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1745
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Tue, 23 Jul 2019 04:11:44 +0000 (00:11 -0400)]
bt2: make bt_bt2_trace_{,class}_add_destruction_listener return a status
If the calls to bt_trace_class_add_destruction_listener and
bt_trace_add_destruction_listener in the Python bindings fail (due to an
hypothetical memory error that can't happen today), abort is called.
Strangely, back on the Python side, there is a "listener_id is None"
check that is a bit useless, since we would have aborted had there been
a failure.
Regularize the situation by making the
bt_bt2_trace_class_add_destruction_listener and
bt_bt2_trace_add_destruction_listener wrappers return a status, like any
other API function. We can then use utils._handle_func_status like
everywhere else.
Change-Id: I17901a0278f1615b45e6adc3b7223f1b3b3ff35e
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1744
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Tue, 23 Jul 2019 03:37:03 +0000 (23:37 -0400)]
bt2: remove utils._handle_ptr
A future patch will want to get rid of spots that raise bt2.Error, which
are not in direct response to a failed Babeltrace API call. The
utils._handle_ptr is one such spot. It turns out that the usages of
utils._handle_ptr are not useful anymore, because they check
situations that can either never happen, or are check by pre-condition
checks.
In connection.py, it's not possible anymore for
bt_connection_borrow_downstream_port_const or
bt_connection_borrow_upstream_port_const to return NULL/None, as
connections always have both ports set.
In field.py, the call in _create_field_from_ptr after
bt_field_borrow_class_const can be removed, as it's impossible for this
function to legitimately return NULL/None.
The call in _VariantField.selected_option is unnecessary, as it is after
a pre-condition check that the variant's selected option is non-NULL.
If we want to make Python bindings user-friendly (avoid hitting
pre-condition checks), we check before the call if the variant has a
currently selection option, and raise ValueError otherwise. However,
there does not seem to be an API for that at the moment.
Change-Id: I585e57030d58256cbf51161e75394503c281addf
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1743
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Sun, 21 Jul 2019 16:49:05 +0000 (12:49 -0400)]
py-common: make bt_py_common_format_exception accept an arbitrary exception
A following patch will need to format each exception of the exception
chain independently. It will therefore need a function like
bt_py_common_format_exception, but that:
- takes the exception to format as a parameter, rather than relying on
the Python error indicator
- has an option for formatting just that exception, not following the
causes chain
This patch changes bt_py_common_format_exception to meet these
requirements. The behavior of the old bt_py_common_format_exception is
now offered by bt_py_common_format_current_exception, so the existing
callers are updated to use that one.
Change-Id: I7bbb442bc1dc363e5441b3b11ed6f0c6e213ba79
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1737
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Sun, 21 Jul 2019 17:05:21 +0000 (13:05 -0400)]
Fix: bt2: Make bindings target depend on convenience libraries
Three convenience libraries are used to build the bindings' native
library. This dependency is not expressed in the Makefile, so if you
change one of them and re-run make, the bindings won't be rebuilt, and
the changes in that convenience lib won't be taken into account. For
example, if you touch src/py-common/.libs/libbabeltrace2-py-common.a and
re-run make, the bindings should (but currently don't) get re-built.
This patch adds the explicit dependency in the Makefile, so that any
change to one of these libs will trigger a rebuild of the bindings.
Change-Id: Iea47833b206bcd0f1cc0d3f72975b14fd6eded93
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1736
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Tue, 23 Jul 2019 16:15:08 +0000 (12:15 -0400)]
tests: fix Windows support in test_convert_args
Additionnal variables were added to the common case, add them for
Windows too.
Change-Id: I33ec919f305a3d69020ee906e29ce4c2636ae54f
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1750
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Tue, 23 Jul 2019 15:43:13 +0000 (11:43 -0400)]
tests: readlink 'canonicalize' is GNU specific
On platforms not using the GNU coreutils, namely MacOs, readlink doesn't
have a canonicalize option.
Use it only when availabe, otherwise use the relative paths.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I798e6320dda78c5d9b33a2568b663c57d35d42b3
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1749
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Mon, 22 Jul 2019 21:36:02 +0000 (17:36 -0400)]
tests: fix Python plugin provider tests on Windows
On Windows, the embedded Python interpreter in the Python plugin
provider can't automatically locate the path to it's own modules. Set
the PYTHONHOME variable to the proper value in 'run_python_bt2()'.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: Ie30ac3212a208d77654add78604c534c05efe518
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1748
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Tue, 23 Jul 2019 16:23:35 +0000 (12:23 -0400)]
Standardize `!ptr` i/o `ptr == NULL`, `ptr` i/o `ptr != NULL`
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I3e47fe9ca121cf3fc4f71e6292b9fa9efc6df341
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1746
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Mon, 22 Jul 2019 17:45:22 +0000 (13:45 -0400)]
bt2: do not require sink component's _graph_is_configured() method
The "graph is configured" method is not required when you implement a
component class in C, as it's not strictly needed to make the graph
runnable, so it should not be required in Python either.
A default _UserSinkComponent._graph_is_configured() method is added to
make it optional.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I3ef4e3d159af61e2d06a0a47dddc1234aed93ae1
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1740
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Mon, 22 Jul 2019 17:42:26 +0000 (13:42 -0400)]
bt2: move _bt_graph_is_configured_from_native() to `_UserSinkComponent`
This method is only called on a sink user component, it does not need to
be in the base `_UserComponent`.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ie537c6871d145d2cb3c96eaa0e95573b02731cc9
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1739
Tested-by: jenkins <jenkins@lttng.org>
Michael Jeanson [Mon, 22 Jul 2019 21:39:31 +0000 (17:39 -0400)]
Update python bindings gitignore for native_bt.c
In
e7d63bf32268fcaf9e1f5724cf4def49f3f9c081, 'native_bt_wrap.c' was
renamed to 'native_bt.c' but the gitignore was not updated.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: Ibd4186ea0ce12bb9078ceee18cff781649cdd2fe
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1742
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Fri, 5 Jul 2019 19:50:58 +0000 (15:50 -0400)]
cli: automatically detect sources for leftover arguments
This patch adds a source auto-discovery feature. The goal is for the
user to be able to just pass some inputs (which can be strings of a form
understood by a particular component class, or paths to
files/directories) and for babeltrace to figure out which component
classes are best suited to handle those inputs.
Currently, any leftover argument passed by the user is passed to a
src.ctf.fs instance. To use a different source component, the user must
use the --component argument. This system will therefore help usability
for users of non-src.ctf.fs sources. It will also allow splitting the
src.ctf.fs source into a "generic CTF" one and an "LTTng CTF" one, which
includes the fixups specific to CTF files produced by various LTTng
versions.
The big picture is that for each leftover argument (called `input`), we
ask all component classes if they can handle it (see `support-info
query` below), the component classes reply with a weight, and the input
is attributed to the component class that gave the largest weight.
More precisely, this is what we do for each leftover argument:
1. Ask all known source component classes if they recognize the
argument as an arbitrary string, which would be in a format that
makes sense to them (but doesn't point to a file or directory on
disk, unless it's a coincidence). The obvious example is
src.ctf.lttng-live, which would be apt to handle paths of the form
'net://...'. If some component classes claim to understand the
argument, the one with the largest weight is chosen, and a component
of that class will be instantiated.
2. If no component class has claimed the argument as an arbitrary
string and the argument points to a file or directory on disk, ask
them all if they recognize it as a file or directory they can handle.
If some component classes do, choose the one with the biggest weight.
3. If no component class has recognized it so far, and the input points
to a directory, we start to dig: for each child of that directory,
apply the sophisticated algorithm described in step 2.
If a leftover argument (including all its children, if it's a directory)
is not handled by any source component class, we show a warning. If
nothing is discovered and no explicit source is instantiated either, the
excecution fails on the "No source component" check that is already
there.
Component classes have the ability to "group" their inputs as they wish.
This means that multiple inputs attributed to a given component class
can be passed to a single instance of that class, all to separate
instances, or any combination in between. To achieve this, component
classes are able to also reply with a group key (a string of their
choice). Inputs with the same group key will be passed to the same
instance.
Implementation details and choices
----------------------------------
* Since leftovers are now passed to the source auto-discovery mechanism
as opposed to an implicit src.ctf.fs component previously, this
src.ctf.fs implicit component is no longer needed. This changes how we
handle some of the legacy (compatibility with babeltrace 1) flags.
If the user passes --clock-offset or --clock-offset-ns, we will search
in the auto-discovered sources and apply it to any src.ctf.fs instance
created. If no src.ctf.fs component would be instantiated, we issue
an error.
If the user passes --input-format ctf, we don't want other formats
possibly being read. We still use the auto-discovery mechanism, but
we restrict it to the src.ctf.fs component class (other component
classes won't be queried and therefore won't be instantiated).
* This also means that to keep the basic use case of "babeltrace2
<dir-with-ctf-traces>" working, we need to implement the support-info
query for the src.ctf.fs component class in the current patch. Not
doing so immediatly would break many tests. The simplest possible
implementation was added. It looks for inputs of type "directory",
which have a "metadata" file in it. It always reply with the same
group value, such that a single instance of the component class is
used (keeping the current behavior).
* Since the strings we pass to support-info queries (and eventually to
components we instantiate) are not necessarily paths to directories or
files on disk (they can be URLs, for example), we now use the term
"inputs" rather than "paths" for all of them.
* A support-info query returning ERROR aborts the auto-discovery
process, making it return an ERROR as well.
* If we can't open a directory because we don't have permission to read
it (EACCES), we log a warning and continue. Other errors abort the
auto-discovery process.
support-info query
------------------
The support-info query is a contract between the CLI and source
component classes. Source component classes that don't support it will
simply not be able to automatically discover inputs, and will have to be
explicitly instantiated using --component.
The parameters of a support-info query are a map containing these keys:
- `type` (string): possible values are "string", "directory" or "file",
indicating the nature of `input`, described below. The value "string"
means that the component class may try to interpret `input` as a
string with a format it can recognize. "directory" and "file"
respectively mean to interpret `input` as a directory and file. When
type is "directory" or "file", the component class can assume that
the corresponding directory or file exists on disk.
- input (string): input descriptor, to be interpreted according to
`type`.
A support-info response can be
- A real or integer value between 0 and 1, representing the weight
- A map value containing these keys:
- `weight` (real or integer), mandatory: between 0 and 1
- `group` (string or null), optional: a key by which to group inputs,
when instantiating components.
A component class that does not support a given must reply with a weight
of 0.
All inputs attributed to the same component class, sharing the same
group key, will be passed to the same component instance. inputs whose
group key is missing or null are not grouped with other inputs.
Components created by the auto-discovery mechanism are passed the
`inputs` parameter, an array of strings containing all inputs attributed
to this instance.
testing
-------
I have a brief catch-all test for this, it just covers a few important
cases. The test strategy is the following:
- Run babeltrace2 with an arbitrary string and a directory as
leftovers.
- One source recognizes the arbitrary string.
- Various sources recognizes files and directories inside the passed
directory.
- Each instantiated source outputs one line including its name and the
sorted list of its inputs.
- We sort the output of babeltrace and compare it with an expected
string. Since everything is sorted, the output should be stable.
Change-Id: I7f884551d7cb576974ea53420ead9c4a8005e99d
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1644
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Mon, 22 Jul 2019 18:35:14 +0000 (14:35 -0400)]
src.ctf.lttng-live: LOGI instead of LOGW when getting queried for an unknown object
Receiving a query about an unknown object happens in the normal course
of operations, and is not warning-worthy. In particular, a following
patch will query all source component classes using the `support-info`
object, to see if the source component class supports a given input.
Having a LOGW here would generate a lot of unimportant warnings.
Change-Id: I067424cb72f78aeda6ce6791b12fa00d7d90b5c0
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1741
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Mathieu Desnoyers [Mon, 17 Jun 2019 21:56:02 +0000 (17:56 -0400)]
sink.text.pretty: don't use printf for binary values
printf is slow compared to simply appending '1' or '0' characters to the
GString.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I2605c1e6d2dc463be237e8eb00f70a84e4c5fd1a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1507
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Mathieu Desnoyers [Mon, 17 Jun 2019 21:05:58 +0000 (17:05 -0400)]
sink.text.pretty: use bt_common_g_string_append and bt_common_g_string_append_c
Use our own inline implementation of g_string_append and g_string_append_c
to improve text pretty-printing speed.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I39254ae912cb9bd227c68519552b4b610c15755f
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1506
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Mathieu Desnoyers [Mon, 17 Jun 2019 20:12:49 +0000 (16:12 -0400)]
sink.text.pretty: remove field filtering
Field filtering uses GQuarks internally. Those generate g hash table
lookups, which appear at the top of perf reports.
This field filtering mechanism is not needed anymore, so remove it.
It's not needed anymore because the packet context fields and the event
fields are not special or associated to CTF anymore as the library's
trace IR is CTF-agnostic. So those special fields do not exist anyway in
the messages this sink consumes.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I7bf0238e6692fb277dafc32f864b0e20e89bf3d2
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1505
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Mathieu Desnoyers [Mon, 17 Jun 2019 19:33:51 +0000 (15:33 -0400)]
sink.text.pretty: do not printf field name strings
Use g_string_append rather than the printf counterpart to output field
name strings. This significantly improves pretty printing speed
(34s -> 29s on reference trace).
This is a performance improvement mainly because only strings are passed
to printf in this case.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I4df52308c61346a81e0b79b079d53be52084707f
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1504
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Mathieu Desnoyers [Mon, 17 Jun 2019 16:53:22 +0000 (12:53 -0400)]
sink.text.pretty: use bt_common_g_string_append_printf
g_string_append_printf() internally allocates a temporary buffer
through use of vasnprintf for each call. This clearly appears at
the top of perf report.
Use babeltrace's own bt_common_g_string_append_printf which operates
directly on the GString buffer, increasing its size to nearby next power
of two as needed.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I60b4dccfa19c4321eb5997233826cfabaf924ff2
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1503
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Mathieu Desnoyers [Wed, 19 Jun 2019 18:23:09 +0000 (14:23 -0400)]
common: implement bt_common_g_string_append and bt_common_g_string_append_c
Implement inline versions of g_string_append and
g_string_append_c. Since those calls are performed very often within
sink.text.pretty, it is relevant for pretty-printing speed.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ib1069c7d21c6e707f75d839f333f034a50e2a43e
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1515
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Mathieu Desnoyers [Wed, 19 Jun 2019 18:21:57 +0000 (14:21 -0400)]
common: implement bt_common_g_string_append_printf
g_string_append_printf() internally allocates a temporary buffer
through use of vasnprintf for each call. This clearly appears at
the top of perf report.
Implement our own private bt_g_string_append_printf which operates
directly on the GString buffer.
See proof of memory allocation/free near the top of this perf report
when converting a 400M LTTng trace to text (piped to /dev/null):
14.76% babeltrace2 libc-2.24.so [.] vfprintf
7.92% babeltrace2 [kernel.kallsyms] [k] vmacache_find
6.39% babeltrace2 libglib-2.0.so.0.5000.3 [.] g_string_insert_len
4.77% babeltrace2 babeltrace-plugin-ctf.so [.] bt_bfcr_start
4.23% babeltrace2 libc-2.24.so [.] _int_malloc
4.06% babeltrace2 libc-2.24.so [.] _IO_default_xsputn
3.60% babeltrace2 libc-2.24.so [.] strlen
3.60% babeltrace2 libc-2.24.so [.] __strchrnul
3.60% babeltrace2 libbabeltrace2.so.0.0.0 [.] bt_attributes_borrow_field_by_name
3.51% babeltrace2 libc-2.24.so [.] _int_free
2.88% babeltrace2 libc-2.24.so [.] __vasprintf_chk
2.88% babeltrace2 libc-2.24.so [.] _IO_str_init_static_internal
2.88% babeltrace2 libc-2.24.so [.] __strcmp_sse2_unaligned
2.16% babeltrace2 libc-2.24.so [.] malloc
2.16% babeltrace2 libc-2.24.so [.] __tzstring_len
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I956305ac8891d244dbdab6a5b0dde82ecc960e25
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1514
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Sun, 21 Jul 2019 14:50:39 +0000 (10:50 -0400)]
tests/Makefile.am: remove needless comment
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I5e70b8b0fc3e5faf9663cfabf657d447fc5ed710
Simon Marchi [Sun, 21 Jul 2019 03:10:41 +0000 (23:10 -0400)]
Fix: bt2: incref Py_None in get_msg_range_common on error
When get_msg_range_common processes a result that is not OK, it
returns a tuple whose second element is None:
(status, None)
However, when setting the second element of the tuple to None, we are
missing an incref of Py_None. This incref is necessary, because
PyTuple_SET_ITEM steals the reference of the object we pass.
This patch fixes that.
Change-Id: I8a5af2853172a8399dd1779d90e0fcb8fc265032
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1731
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Sat, 20 Jul 2019 22:18:40 +0000 (18:18 -0400)]
lib: logging: add `%!R` conv. specifier for integer range set and use it
This patch also renames "range set" to "integer range set" in logging
and precondition assertion messages.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I59aaf541d956d84fee5cad22393b230635558877
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1730
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Sat, 20 Jul 2019 16:38:21 +0000 (12:38 -0400)]
lib: rename functions to clearly indicate API inheritance
This patch renames some functions of the library's API to show how
an API inherits another one, for example (before):
bt_field_signed_integer_get_value()
bt_field_unsigned_integer_get_value()
vs. (after):
bt_field_integer_signed_get_value
bt_field_integer_unsigned_get_value
The second version clearly shows that both are part of the
`bt_field_integer` API.
This patch aligns the names of those functions with other parts of the
API which already have the correct order of prefixes, for example:
bt_component_class_source_create()
bt_component_class_filter_create()
bt_component_class_sink_create()
bt_component_class_get_name()
bt_message_event_create()
bt_message_get_type()
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I5f7b637d1d72d610f88f3f7d0122c8975f26481b
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1728
Tested-by: jenkins <jenkins@lttng.org>
Francis Deslauriers [Sat, 13 Jul 2019 14:39:21 +0000 (10:39 -0400)]
Tests: flt.lttng-utils.debug-info: update debug-info tests
* Convert the currently disabled Python bindings based debug-info
testcase to a `sink.text.details` based test.
* Update debug-info test trace using a recently changed `libhello_so`
file.
* Move all debug-info test files to
`tests/plugins/flt.lttng-utils.debug-info/`
* Move `tests/data/debug-info/README.md` content to `CONTRIBUTING.adoc`
* Update `tests/data/plugins/flt.lttng-utils.debug-info/README.md` to
explain how to generate the debug-info test trace.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I67b8fa729e462137fb3e584c0f1eaf472256e3c9
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1704
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Fri, 19 Jul 2019 19:14:45 +0000 (15:14 -0400)]
Fix: flt.utils.muxer: don't clear an empty `GPtrArray`
GLib < 2.48.0 asserts when clearing an empty `GPtrArray` with
g_array_remove_range():
GLib-CRITICAL **: g_ptr_array_remove_range: assertion
`index_ < array->len' failed
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: If9dad0869404ec72ee15724b72ad88780b671619
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1725
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Jonathan Rajotte [Mon, 15 Jul 2019 21:50:11 +0000 (17:50 -0400)]
`test_plugin.py`: use absolute paths for plugin path comparison
Under a MinGW 64 test environment, the `BABELTRACE_PLUGIN_PATH`
environment variable gets translated to NT path by MinGW since the
passed paths fit the translation criteria. The relative part of the
paths also get thrown away and the relative paths are replaced by their
absolute equivalent.
Performing absolute path conversion on both test paths ensure
consistency across platforms of this test.
The error was:
error: PASS: bindings/python/bt2/test_python_bt2 6513 - test_name (test_plugin.PluginTestCase)
c:/users/joraj/babeltrace/src/plugins/ctf/babeltrace-plugin-ctf.la
c:/users/joraj/babeltrace/tests/../src/plugins
not ok 6514 - test_path (test_plugin.PluginTestCase)
FAIL: bindings/python/bt2/test_python_bt2 6514 - test_path (test_plugin.PluginTestCase)
Traceback (most recent call last):
bindings/python/bt2/test_python_bt2: Traceback (most recent call last):
File "C:/msys64/mingw64/lib/python3.7\unittest\case.py", line 59, in testPartExecutor
bindings/python/bt2/test_python_bt2: File "C:/msys64/mingw64/lib/python3.7\unittest\case.py", line 59, in testPartExecutor
yield
bindings/python/bt2/test_python_bt2: yield
File "C:/msys64/mingw64/lib/python3.7\unittest\case.py", line 628, in run
bindings/python/bt2/test_python_bt2: File "C:/msys64/mingw64/lib/python3.7\unittest\case.py", line 628, in run
testMethod()
bindings/python/bt2/test_python_bt2: testMethod()
File "C:/Users/joraj/babeltrace/tests/bindings/python/bt2\test_plugin.py", line 96, in test_path
bindings/python/bt2/test_python_bt2: File "C:/Users/joraj/babeltrace/tests/bindings/python/bt2\test_plugin.py", line 96, in test_path
self.assertTrue(plugin_path.startswith(plugin_path_env))
bindings/python/bt2/test_python_bt2: self.assertTrue(plugin_path.startswith(plugin_path_env))
File "C:/msys64/mingw64/lib/python3.7\unittest\case.py", line 705, in assertTrue
bindings/python/bt2/test_python_bt2: File "C:/msys64/mingw64/lib/python3.7\unittest\case.py", line 705, in assertTrue
raise self.failureException(msg)
bindings/python/bt2/test_python_bt2: raise self.failureException(msg)
AssertionError: False is not true
bindings/python/bt2/test_python_bt2: AssertionError: False is not true
Change-Id: I4b3dc1c52a13dff5be88e417a9d1a877c81f66da
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1709
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Jonathan Rajotte [Mon, 15 Jul 2019 21:49:56 +0000 (17:49 -0400)]
`test_query_trace_info.py`: adapt regex to NT path style
In a MinGW 64 test environment, the stream path is expressed using NT
path (`C:\\...`). In `test_query_trace_info.py`, adapt the regex
accordingly using Python's `pathlib` API.
Change-Id: Ieee3b9cac0881aec7086cef69ecf681ad71c0722
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1708
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Francis Deslauriers [Fri, 19 Jul 2019 21:49:39 +0000 (17:49 -0400)]
tests/plugins/flt.utils.trimmer/Makefile.am: remove useless HAVE_PYTHON guard
Philippe Proulx:
The `test_trimming` test does not depend on Python: it uses the CLI,
a `sink.text.details` component, and expectation strings to validate
that an `flt.utils.trimmer` message iterator trims as expected.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: If08b014259676a75c57db7eda109de55306aa062
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1727
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Francis Deslauriers [Fri, 19 Jul 2019 13:24:04 +0000 (09:24 -0400)]
flt.utils.muxer: support stream messages with default clock snapshot
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I46fef189f846999dbfedc3504a7ed5d908464e99
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1724
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Thu, 18 Jul 2019 18:06:03 +0000 (14:06 -0400)]
bt2: fix: don't return in bt_bool out typemap
I noticed this leak in the Valgrind output:
==11115== 5 bytes in 1 blocks are definitely lost in loss record 9 of 5,534
==11115== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11115== by 0x8EA64A8: SWIG_AsCharPtrAndSize (native_bt.c:3539)
==11115== by 0x8EDC722: _wrap_value_map_has_entry (native_bt.c:22039)
...
This is due to the fact that we return in the bt_bool out typemap, which
we shouldn't. In the case of the bt_value_map_has_entry wrapper, a
string is dynamically allocated for the "const char *" parameter. It is
normally freed at the end of the wrapper, except that because we return
early, it is never freed. The same problem is likely to happen with
other functions returning bt_bool.
The leak can be triggered by running this Python script:
from bt2 import value
m = value.MapValue()
m['allo'] = 2
print('allo' in m)
The generated code before this patch looks like:
result = (bt_bool)bt_value_map_has_entry((bt_value const *)arg1,(char const *)arg2);
{
if (result > 0) {
resultobj = Py_True;
} else {
resultobj = Py_False;
}
Py_INCREF(resultobj);
return resultobj;
}
if (alloc2 == SWIG_NEWOBJ) free((char*)buf2);
return resultobj;
fail:
if (alloc2 == SWIG_NEWOBJ) free((char*)buf2);
return NULL;
}
The memory leak is quite obvious. After this patch, it looks better:
result = (bt_bool)bt_value_map_has_entry((bt_value const *)arg1,(char const *)arg2);
{
if (result > 0) {
resultobj = Py_True;
} else {
resultobj = Py_False;
}
Py_INCREF(resultobj);
}
if (alloc2 == SWIG_NEWOBJ) free((char*)buf2);
return resultobj;
fail:
if (alloc2 == SWIG_NEWOBJ) free((char*)buf2);
return NULL;
}
Change-Id: Icdfcdf750610465331619cd9edab1e89dc930f64
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1723
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Thu, 18 Jul 2019 15:33:35 +0000 (11:33 -0400)]
bt2: remove unrelated comment from native_bt_integer_range_set.i
This was copied from native_bt_port.i, but does not apply to
native_bt_integer_range_set.i.
Change-Id: I13f900d531e202afd09e807a85f7f817def698a6
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1722
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Francis Deslauriers [Thu, 18 Jul 2019 14:11:55 +0000 (10:11 -0400)]
Apply black code formatter on all Python code
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I2b66d32d12c93d353097b25d80050e4dcfe5b1ff
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1642
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Francis Deslauriers [Thu, 11 Jul 2019 17:28:03 +0000 (13:28 -0400)]
Explicitly mention `black` in CodingStyle guidelines
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I782042dc24acb09506205abbbc7b6b22c2c0cb59
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1687
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Francis Deslauriers [Thu, 11 Jul 2019 17:17:05 +0000 (13:17 -0400)]
Tests: add `tap` utils code to Black formatter exclusion list
Those files are not written by us but rather copied into the project.
Let's not reformat them automatically so we can compare them with future
versions.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I98ccd8475b5367f1b3ed9084ba06965f28bc6f93
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1686
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Thu, 18 Jul 2019 05:33:32 +0000 (01:33 -0400)]
lib: remove `BT_ASSERT_PRE_FUNC`
`BT_ASSERT_PRE_FUNC` is defined to nothing, so remove it.
This should have been done in
bdb288b3e94e412a33c8647d44f6cfac66ca0665.
`BT_ASSERT_PRE_DEV_FUNC` is defined in non-developer mode to mark the
function as unused.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I382f87693a6079eeb13f537e2669402995c58f41
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1721
Philippe Proulx [Wed, 17 Jul 2019 22:12:39 +0000 (18:12 -0400)]
lib: bt_field_class_dynamic_array_create(): accept length FC parameter
This patch makes bt_field_class_dynamic_array_create() similar to
bt_field_class_variant_with_selector_create() for consistency: it
accepts the length field class on creation instead of setting it
afterwards.
You can pass `NULL` to create a dynamic array field class without a
length field class.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ib94adcc32128154b5979120ef0623be912f28734
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1718
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Fri, 12 Jul 2019 19:03:40 +0000 (15:03 -0400)]
lib: decouple variant FC option names from selector FC mapping names
This patch makes the options of a variant field class (FC) which has a
selector have their own range set, decoupled from any mapping of the
selector FC.
Motivation and solution
=======================
In CTF 1.8, structure FC member and variant FC option names which start
with `_` in the metadata stream must be "unescaped" by removing the `_`
prefix. For example:
struct {
string _salut;
string meow;
};
`_salut` becomes `salut` in trace IR, while `meow` stays `meow`.
CTF 1.8.2 specifies:
> Fields starting with an underscore should have their leading
> underscore removed by the CTF trace readers.
Here, we interpret "should" as "must" because Babeltrace 1, Trace
Compass, and other CTF consumers honor this recommandation.
It is not specified, however, that this strategy applies to enumeration
FC mapping labels. For example:
enum {
_SALUT,
MEOW,
};
In Babeltrace 1 (`text` output format and API), `_SALUT` and `MEOW`
remain as is, so Babeltrace 2 should do the same.
There's an issue however when an enumeration FC is used as a tag, or
selector, for a variant FC:
enum {
_salut,
_meow,
} tag;
variant <tag> {
string _salut;
int _meow;
};
This is valid TSDL 1.8, but once in trace IR, the enumeration FC mapping
labels need to stay as is, while the variant FC option names need to be
unescaped. Once in trace IR, the equivalent would be:
enum {
_salut,
_meow,
} tag;
variant <tag> {
string salut;
int meow;
};
Before this patch, this is not valid because, when a variant FC has a
selector FC, the option and mapping names must match exactly: we don't
want to bring this CTF-specific escaping logic into the CTF-agnostic
API.
The current hack to avoid this, performed by `src.ctf.fs`, is to also
unescape the mapping names, so as to get:
enum {
salut,
meow,
} tag;
variant <tag> {
string salut;
int meow;
};
However, this makes Babeltrace 2 not behave like Babeltrace 1 because
the enumeration FC mapping names are different. For example, the
`sink.text.pretty` outputs differ for the same CTF trace.
The solution brought by this patch to fix this issue is to make the
options of a variant FC have their own range set. Using the example
above, the layout in trace IR becomes:
enum {
_salut,
_meow,
} tag;
variant <tag> {
string salut {0};
int meow {1};
};
where `{0}` and `{1}` are the range sets associated to the option.
Here's another example:
enum {
_salut,
_meow = 12,
coucou = 17 ... 45,
_meow = 1 ... 5,
} tag;
variant <tag> {
string salut {0};
int meow {1 ... 5, 12};
int coucou[5] {17 ... 45};
};
This change allows `src.ctf.fs` to keep the enumeration FC mapping names
as is while properly escaping the variant FC option names.
Library changes
===============
The simple variant field class type is replaced with three new variant
field class types:
Variant without a selector:
You can create a variant FC without a selector FC, just like before,
with bt_field_class_variant_create(), passing `NULL` as the
`selector_field_class` parameter.
The field class's type is
`BT_FIELD_CLASS_TYPE_VARIANT_WITHOUT_SELECTOR`.
You can use bt_field_class_variant_without_selector_append_option()
to append an option (name and FC) to a variant FC without a
selector.
You can use bt_field_class_variant_borrow_option_by_index_const()
and bt_field_class_variant_borrow_option_by_name_const() to borrow
a variant FC (base) option from a variant FC without a selector.
Variant with a selector:
You can create a variant FC with a specific selector FC with
bt_field_class_variant_create().
The selector FC must be an _integer_ FC. This is less strict than
before: because each option has its own range set, the selector FC
does not need to be an enumeration FC. It can be an enumeration FC,
but there's no association between its mapping names and the variant
FC option names.
The field class types are
`BT_FIELD_CLASS_TYPE_VARIANT_WITH_UNSIGNED_SELECTOR` and
`BT_FIELD_CLASS_TYPE_VARIANT_WITH_SIGNED_SELECTOR`.
There's an unsigned and a signed type to make the API typing more
strict: a variant FC with an unsigned selector has specific option
objects from which you can borrow unsigned integer range sets.
You can use
bt_field_class_variant_with_unsigned_selector_append_option() and
bt_field_class_variant_with_signed_selector_append_option() to
append an option (name, FC, and range set) to a variant FC with a
selector.
You can use
bt_field_class_variant_with_unsigned_selector_borrow_option_by_index_const(),
bt_field_class_variant_with_unsigned_selector_borrow_option_by_name_const(),
bt_field_class_variant_with_signed_selector_borrow_option_by_index_const(),
or
bt_field_class_variant_with_signed_selector_borrow_option_by_name_const()
to borrow a variant FC with a selector option from a variant FC
with a selector.
The option object's type is either
`bt_field_class_variant_with_unsigned_selector_option` or
`bt_field_class_variant_with_signed_selector_option`. You can
convert it to a base option (to get its name and borrow its field
class) with
bt_field_class_variant_with_unsigned_selector_option_as_option_const()
or
bt_field_class_variant_with_signed_selector_option_as_option_const().
You can borrow the ranges of a variant FC with a selector option
with
bt_field_class_variant_with_unsigned_selector_option_borrow_ranges_const()
or
bt_field_class_variant_with_signed_selector_option_borrow_ranges_const().
You can use
bt_field_class_variant_with_selector_borrow_selector_field_path_const()
to borrow the selector field path from a variant FC with a selector.
I also added the following functions for convenience:
* bt_field_variant_borrow_selected_class_option_const()
* bt_field_variant_with_unsigned_selector_borrow_selected_class_option_const()
* bt_field_variant_with_signed_selector_borrow_selected_class_option_const()
For consistency, bt_field_variant_select_option_field() is renamed to
bt_field_variant_select_option_field_by_index().
This patch also makes an enumeration FC mapping contain an integer range
set object. This was planned anyway, and not doing it here would have
meant to duplicate and adapt code for the variant FC option ranges, for
example in `sink.text.details` to sort the ranges of a range set.
This means that bt_field_class_unsigned_enumeration_map_range() and
bt_field_class_signed_enumeration_map_range() are replaced with
bt_field_class_unsigned_enumeration_add_mapping() and
bt_field_class_signed_enumeration_add_mapping() which accept resp.
unsigned and signed integer range sets.
This also means that
bt_field_class_enumeration_mapping_get_range_count(),
bt_field_class_unsigned_enumeration_mapping_get_range_by_index(), and
bt_field_class_signed_enumeration_mapping_get_range_by_index() are
replaced with
bt_field_class_unsigned_enumeration_mapping_borrow_ranges_const() and
bt_field_class_signed_enumeration_mapping_borrow_ranges_const().
Because I needed it when adapting the project's plugins, I also added
the following functions for convenience:
* bt_field_class_unsigned_enumeration_borrow_mapping_by_label_const()
* bt_field_class_signed_enumeration_borrow_mapping_by_label_const()
Noteworthy plugin changes
=========================
`src.ctf.fs`:
* Enumeration FC mapping names are not unescaped anymore: they are
kept as is.
* A CTF IR named FC contains the original name and the escaped name.
The original name is used to find the ranges of a variant FC
option in the selector FC as the mapping names are not escaped
anymore, so they potentially do not match the variant FC option
names.
* When translating a CTF IR variant FC to a trace IR variant FC, the
trace IR selector (enumeration) FC's integer range set references
are reused directly to append the corresponding variant FC
options.
`sink.ctf.fs`:
* Enumeration FC mapping names are not escaped anymore: they are
kept as is.
* If a variant FC has a selector, then for each option, the
component finds the corresponding mapping in the selector FC _by
range set_ to know whether or not to escape the option name.
This is because this must work (from `src.ctf.fs`):
enum {
salut,
_meow,
} tag;
variant <tag> {
string salut;
int _meow;
};
Once in trace IR, the `_meow` option becomes `meow`, but the
`_meow` mapping keeps its original name. However, we know that,
for `src.ctf.fs`, the range sets match exactly. For the `meow`
option, the corresponding mapping is `_meow` because they both
have the range set with the single range [1, 1]. In that case,
when going back to TSDL, `sink.ctf.fs` writes `_meow` for the
option name, while `salut` remains `salut`.
I added new `sink.ctf.fs` tests, with new succeeding CTF traces,
to verify that the component works as expected with those specific
cases.
If there's any issue when doing this, `sink.ctf.fs` falls back to
creating a dedicated selector FC for the variant FC. For example,
in trace IR, this is totally valid:
enum {
a = 2,
b = 5,
d = 8,
} tag;
variant <tag> {
string a {2};
int b {11};
int c[22] {15 ... 19};
};
because there's no association between mapping names and option
names. This too:
int tag;
variant <tag> {
string a {2};
int b {11};
int c[22] {15 ... 19};
};
Those specimens cannot be translated to TSDL 1.8 however.
* Because of changes in the way TSDL identifers are protected and
validated, clock class names are not systematically escaped if
it's not needed. Therefore the clock class name `default` remains
`default`; it does not become `_default` like before.
`sink.text.details`:
* Variant FC option ranges are written next to the option's name:
var: Variant (unsigned selector) (3 options, Selector field path [Event payload: 0]):
COSSETTE: [0]: String
_PELCHAT: [1]: String
VOISINE: [2] [5, 19]: String
* Enumeration FC mapping ranges are written next to the option's
name instead of having one per line. I find this is more compact
and easier to read as mappings typically do not contain a lot of
ranges:
tag: Unsigned enumeration (8-bit, Base 10, 3 mappings):
COSSETTE: [0]
VOISINE: [2] [5, 19]
__PELCHAT: [1]
Python bindings changes
=======================
Essentially, the `bt2` Python package is updated to match the library's
API changes:
* `_EnumerationFieldClassMapping` is not a set anymore: it has
a `ranges` property (which is a set).
* _EnumerationFieldClass.map_range() is replaced with
_EnumerationFieldClass.add_mapping() to which you pass an integer
range set.
* _EnumerationFieldClass.labels_for_value() is replaced with
_EnumerationFieldClass.mappings_for_value() to get mappings instead of
simple labels.
* _EnumerationFieldClass.__getitem__() now uses
bt_field_class_unsigned_enumeration_borrow_mapping_by_label_const()
or
bt_field_class_signed_enumeration_borrow_mapping_by_label_const().
* The new `_VariantFieldClassWithSelectorOption` object inherits
`_VariantFieldClassOption`, adding a `ranges` property.
* With a `_VariantFieldClass`, you can borrow base options
(`_VariantFieldClassOption`) to get their names and field classes.
* The new types `_VariantFieldClassWithoutSelector`,
`_VariantFieldClassWithUnsignedSelector`, and
`_VariantFieldClassWithSignedSelector` are analogous to the new
types in the library's API.
You can create them with
_TraceClass.create_variant_field_class_without_selector() and
_TraceClass.create_variant_field_class_with_selector(). The selector
FC object must be an instance of `_IntegerFieldClass`.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I084b03ea816ff8bee03ef5315c24fa24cfe74d80
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1717
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Fri, 12 Jul 2019 18:04:54 +0000 (14:04 -0400)]
bt2: add integer range set support
This patch adds `bt2` wrappers for the Babeltrace library integer range
and integer range set library objects.
The new `test_integer_range_set.py` file tests all the new objects.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I9e6852ca2ec6b04bcbc3b6622c03c3abcfa6e32f
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1716
Philippe Proulx [Fri, 12 Jul 2019 17:50:39 +0000 (13:50 -0400)]
lib: add integer range and integer range set API
This patch adds an integer range and range set API.
An integer range is a pair of lower and upper integer values. It can be
signed (`int64_t`) or unsigned (`uint64_t`).
An integer range set is a set of integer ranges, also either signed or
unsigned.
The goal of this is to use this API for enumeration field class mappings
and, with a subsequent patch, for the integer range set of the option of
a variant field class which has a selector.
You can create an integer range set with
bt_integer_range_set_unsigned_create() or
bt_integer_range_set_signed_create(). Integer range sets have a
reference count.
You can add an integer range to an integer range set with
bt_integer_range_set_unsigned_add_range() or
bt_integer_range_set_signed_add_range().
You can borrow an integer range object (unique, no reference count) from
an integer range set by index with
bt_integer_range_set_unsigned_borrow_range_by_index_const() or
bt_integer_range_set_signed_borrow_range_by_index_const().
As of this patch, you cannot remove an integer range from an integer
range set.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I0ba5c33a38a33c2ff2f92fbbd8f871ff6635eac8
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1715
Philippe Proulx [Thu, 18 Jul 2019 04:17:21 +0000 (00:17 -0400)]
Fix various "may be used uninitialized" warnings (GCC)
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I22634b22121dd32ddfdf1309e10e4f413263572d
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1720
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Tue, 16 Jul 2019 21:19:44 +0000 (17:19 -0400)]
Fix: CTF writer: bt_ctf_field_unsigned_integer_set_value() -> *get_value()
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I1c25f5fe31af31c78cc54fa5b18eb9e0a146b760
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1714
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Mon, 15 Jul 2019 21:10:17 +0000 (17:10 -0400)]
Fix: BT_ASSERT_PRE_DEV(): use `##__VA_ARGS`, not `#__VA_ARGS`
`#__VA_ARGS` stringifies the arguments: that's bad.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ifa4c3494d3d082fd28aad72af726319f1578df2e
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1713
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Fri, 12 Jul 2019 04:45:38 +0000 (00:45 -0400)]
bt2: report errors from Python component and component class callbacks
This patchs makes the Python bindings report errors when the user Python
code raises an exception which results in the status code ERROR. The
result looks like this:
ERROR: [Babeltrace CLI] (/home/smarchi/src/babeltrace/src/cli/babeltrace2.c:2534)
Cannot create components.
CAUSED BY [Babeltrace CLI] (/home/smarchi/src/babeltrace/src/cli/babeltrace2.c:2357)
Cannot create component: plugin-name="gpx", comp-cls-name="GpxSource", comp-cls-type=0, comp-name="source.gpx.GpxSource"
CAUSED BY [Babeltrace library] (/home/smarchi/src/babeltrace/src/lib/graph/graph.c:1337)
Component initialization method failed: status=ERROR, comp-addr=0x55d020aeb8b0, comp-name="source.gpx.GpxSource", comp-log-level=BT_LOGGING_LEVEL_WARNING, comp-class-type=BT_COMPONENT_CLASS_TYPE_SOURCE,
comp-class-name="GpxSource", comp-class-partial-descr="", comp-class-is-frozen=0, comp-input-port-count=0, comp-output-port-count=0
CAUSED BY [source.gpx.GpxSource: 'source.gpx.GpxSource'] (bt2/native_bt_wrap.c:3863)
Traceback (most recent call last):
File "/home/smarchi/build/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/component.py", line 474, in _bt_init_from_native
self.__init__(params)
File "/home/smarchi/src/babeltrace-fun-plugins/gpx/bt_plugin_gpx.py", line 83, in __init__
raise ValueError("GpxSource: missing `inputs` parameter")
ValueError: GpxSource: missing `inputs` parameter
Only the callbacks in the component and component classes area are done.
The other ones I see that could use this feature are the graph's port
added and connected callbacks. However, I can't really test this yet,
because it's not possible to access error causes from Python yet. They
will be done once we add support for that.
Change-Id: Ic7a2a97831cbfba34730a4e68ada24e06d9fa8f3
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1699
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Mon, 15 Jul 2019 03:19:24 +0000 (23:19 -0400)]
bt2: check for _graph_is_configured method in user sink classes
Since a _graph_is_configured method is an essential method to implement
in any useful sink, check for the presence of that method when
instantiating a user sink component class in Python (that is, when the
Python class is created). We already do that for the _consume method,
and I think it greatly helps the user writing a sink by telling them
what not to forget.
Change-Id: Ic8c3741b121eccc2857afac809521dd7213aa679
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1707
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Mon, 15 Jul 2019 02:45:56 +0000 (22:45 -0400)]
plugins: log failure to load plugin as a warning
In my experience writing a Python plugin, it is really helpful to see
when it fails to load, with a traceback of the failure. We currently
output this failure at the "info" level, which is not shown by default.
Even if I set the log level to info, that failure is lost in a sea of
messages.
So, even if fail_on_load_error is false, a plugin that fails to load
is a bit suspicious and is warn-worthy, in my opinion.
Change-Id: I82844aaa62d33dc0ad1059d515ae1581356f4fe9
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1706
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Sat, 29 Jun 2019 14:06:58 +0000 (10:06 -0400)]
tests: run_python_bt2: error out if BT_TESTS_BUILDDIR does not point to a build directory
This patch improves the user experience when building out of tree, using
run_python_bt2, but forgetting to set BT_TESTS_BUILDDIR.
Currently, the script doesn't complain when you do this, but sets
variables (e.g. PYTHONPATH) to point to files/directories that don't
exist:
$ /home/simark/src/babeltrace/tests/utils/run_python_bt2 python3
>>> import bt2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'bt2'
With this patch, the user gets notified that they did something wrong:
/home/simark/src/babeltrace/tests/utils/run_python_bt2: BT_TESTS_BUILDDIR does
not point to a valid directory
(/home/simark/src/babeltrace/tests/utils/../utils/../Makefile does not exist).
If building out-of-tree, set BT_TESTS_BUILDDIR to point to the tests directory
in the build tree.
Change-Id: Ibbc6e11b6cbfce560ad97081d709ed4794c406b1
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1583
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Wed, 17 Jul 2019 03:54:29 +0000 (23:54 -0400)]
bt2: don't copy swig interface files to build directory
A pet peeve of mine is that if you touch a bt2 py file (e.g. `touch
src/bindings/python/bt2/bt2/event.py`) and re-build, it will
re-run swig and rebuild the native module, even though it's not
necessary (all that is needed is to copy the .py files over to the build
directory). This takes precious seconds from my development cycle.
This is done because the .i files are copied over to the build directory
(as they are in STATIC_BINDINGS_DEPS), along with the .py files. And
since the modification time of the .i files has changed, it triggers the
bt2/native_bt.i -> bt2/native_bt.c conversion to run again.
Fix this by not copying the .i files to the build directory. This is
not necessary anyway, as swig can read them perfectly fine from the
source directory.
And while at it, the logging.h/logging.c files don't need to be copied
to the build directory either, so make it so they are not copied and
read from the source directory (I don't think this provides any
advantage other than it being "cleaner").
Change-Id: Ib1ee03cd7795e0f9f3eaedb8476c052221acb9ee
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1712
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Michael Jeanson <mjeanson@efficios.com>
Simon Marchi [Wed, 17 Jul 2019 03:04:06 +0000 (23:04 -0400)]
bt2: run swig by hand instead of through "setup.py build_ext"
When we build the bt2 Python module, we call "setup.py build_ext", which
runs swig to generate the .c file from .i files and builds the native
library. We then run "setup.py build" to copy the Python files over to
build_lib directory.
However, "setup.py build" insists on running "build_ext" itself, seeing
that we have an ext_modules entry in our setup.py. This results in the
.i -> .so process being run twice. It's a bit wasteful to do the
process twice, but in itself it's not a big problem.
A bigger problem is that the second time swig is ran (through "setup.py
build"), it is done without honoring the SWIG Makefile variable. Here, I
have configured my build to use SWIG=/tmp/swig/bin/swig, and the two
swig invocations are:
/tmp/swig/bin/swig -python -I/home/simark/src/babeltrace/include -o bt2/native_bt_wrap.c bt2/native_bt.i
swig -python -I/home/simark/src/babeltrace/include -o bt2/native_bt_wrap.c bt2/native_bt.i
This means that trying to use a custom swig executable through the SWIG
configuration variable will not work, as it will always be overwritten
by the output of the "swig" in the path.
I haven't found a way to get the right swig executable to be used the
second time:
- "setup.py build" doesn't have a --swig option
- We can't use swig= in setup.py
- We can't override using an environment variable
The solution I found was to run swig ourselves to generate the .c file,
and feed that to the native extension built with distutils. We now
don't need to call build_ext explicitly, since the call done through
"build" is sufficient. It therefore fixes both problems: swigging and
building the library is now done only once, and the right swig
executable is used.
Change-Id: I9ed9d22fae1f5675d42af08e77607515dfdf788a
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1711
Reviewed-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Wed, 17 Jul 2019 01:03:58 +0000 (21:03 -0400)]
configure: report SWIG and SWIG_LIB values
I am trying to use an alternative SWIG installation, and though it would
be useful to have this information in the configure report.
Change-Id: I4532062afb03c27caac5e811d33224d06a598b87
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1710
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Fri, 5 Jul 2019 18:58:06 +0000 (14:58 -0400)]
cli: move plugin loading code to its own file
The following patch will need to call load and access plugins from a
different file (within the CLI). Move everything that manages the
plugins in the CLI to its own file, to make it easier to call those
functions in separate files.
Change-Id: I7f62b554066ff7605bd6e1a8cdccd3d8d468ca2e
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1643
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Thu, 11 Jul 2019 19:38:18 +0000 (15:38 -0400)]
cli: Replace printf_err with BT_CLI_LOGE_APPEND_CAUSE
The printf_err macro is currently used to report an error in the CLI.
However, we now have a proper error reporting framework. This patches
makes the CLI use it.
All usages of printf_err are replaced with BT_CLI_LOGE_APPEND_CAUSE.
All usages of print_err_oom are replaced with a new macro that calls
BT_CLI_LOGE_APPEND_CAUSE("Out of memory.").
I haven't tested all the errors paths to make sure they worked well, but
here are some examples of the result:
$ babeltrace2 convert --run-args --legault
ERROR: [Babeltrace CLI] (/home/smarchi/src/babeltrace/src/cli/babeltrace2.c:2874)
Command-line error: retcode=1
CAUSED BY [Babeltrace CLI] (/home/smarchi/src/babeltrace/src/cli/babeltrace2-cfg-cli-args.c:3621)
While parsing command-line options, at option --legault: unknown option
$ babeltrace2 convert --run-args -i salut
ERROR: [Babeltrace CLI] (/home/smarchi/src/babeltrace/src/cli/babeltrace2.c:2874)
Command-line error: retcode=1
CAUSED BY [Babeltrace CLI] (/home/smarchi/src/babeltrace/src/cli/babeltrace2-cfg-cli-args.c:3834)
Unknown legacy input format:
salut
Change-Id: I891c0518c66f61a62fd2e73cf3c438b622aabd9a
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1692
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Sat, 13 Jul 2019 20:15:44 +0000 (16:15 -0400)]
CONTRIBUTING.adoc: add more content to the "Testing" section
This patch organizes `CONTRIBUTING.adoc`'s "Testing" section so that it
explains the basics of the testing environment first, and then
specificities of the Python bindings tests and guides.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I6f9afb0f847990eacd4a7cdf2bdf7401d6e02877
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1705
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Michael Jeanson [Thu, 11 Jul 2019 17:48:05 +0000 (13:48 -0400)]
tests: Move tap-driver.sh out of the autotools aux directory
We have made local modifications to this script, move it to the test
suite directory so it doesn't get overwritten by an updated version from
autotools.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: Ie7c848c1dbd3739ed5caeb22befe73e6540e3133
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1688
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Thu, 11 Jul 2019 21:26:22 +0000 (17:26 -0400)]
Fix: bt2: remove circular import (not supported before Python 3.5)
This fixes the bindings failing to load with a circular import error on
SLES12 with Python 3.4:
Traceback (most recent call last):
File "/usr/lib64/python3.4/unittest/loader.py", line 323, in _find_tests
module = self._get_module_from_name(name)
File "/usr/lib64/python3.4/unittest/loader.py", line 301, in _get_module_from_name
__import__(name)
File "/home/mjeanson/Git/babeltrace/tests/bindings/python/bt2/test_clock_class.py", line 22, in <module>
import bt2
File "/home/mjeanson/Git/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/__init__.py", line 26, in <module>
from bt2.clock_class import *
File "/home/mjeanson/Git/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/clock_class.py", line 23, in <module>
from bt2 import native_bt, object, utils
File "/home/mjeanson/Git/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/utils.py", line 24, in <module>
import bt2.logging
File "/home/mjeanson/Git/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/logging.py", line 23, in <module>
from bt2 import native_bt, object, utils
ImportError: cannot import name 'utils'
Jonathan Rajotte investigated why this happens with Python 3.4:
This is due to a circular import. utils import logging, logging import
utils.
The proposed solution is valid.
PDB callstack leading to error:
-> tests = loader.discover(start_dir, pattern)
/usr/lib64/python3.4/unittest/loader.py(275)discover()
-> tests = list(self._find_tests(start_dir, pattern))
/usr/lib64/python3.4/unittest/loader.py(323)_find_tests()
-> module = self._get_module_from_name(name)
/usr/lib64/python3.4/unittest/loader.py(301)_get_module_from_name()
-> __import__(name)
<frozen importlib._bootstrap>(2237)_find_and_load()
<frozen importlib._bootstrap>(2226)_find_and_load_unlocked()
<frozen importlib._bootstrap>(1200)_load_unlocked()
<frozen importlib._bootstrap>(1129)_exec()
<frozen importlib._bootstrap>(1471)exec_module()
<frozen importlib._bootstrap>(321)_call_with_frames_removed()
/home/jenkins/babeltrace/tests/plugins/src.ctf.fs/query/test_query_trace_info.py(21)<module>()
-> import bt2
<frozen importlib._bootstrap>(2237)_find_and_load()
<frozen importlib._bootstrap>(2226)_find_and_load_unlocked()
<frozen importlib._bootstrap>(1200)_load_unlocked()
<frozen importlib._bootstrap>(1129)_exec()
<frozen importlib._bootstrap>(1471)exec_module()
<frozen importlib._bootstrap>(321)_call_with_frames_removed()
/home/jenkins/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/__init__.py(26)<module>()
-> from bt2.clock_class import *
<frozen importlib._bootstrap>(2237)_find_and_load()
<frozen importlib._bootstrap>(2226)_find_and_load_unlocked()
<frozen importlib._bootstrap>(1200)_load_unlocked()
<frozen importlib._bootstrap>(1129)_exec()
<frozen importlib._bootstrap>(1471)exec_module()
<frozen importlib._bootstrap>(321)_call_with_frames_removed()
/home/jenkins/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/clock_class.py(23)<module>()
-> from bt2 import native_bt, object, utils
<frozen importlib._bootstrap>(2284)_handle_fromlist()
<frozen importlib._bootstrap>(321)_call_with_frames_removed()
<frozen importlib._bootstrap>(2237)_find_and_load()
<frozen importlib._bootstrap>(2226)_find_and_load_unlocked()
<frozen importlib._bootstrap>(1200)_load_unlocked()
<frozen importlib._bootstrap>(1129)_exec()
<frozen importlib._bootstrap>(1471)exec_module()
<frozen importlib._bootstrap>(321)_call_with_frames_removed()
/home/jenkins/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/utils.py(24)<module>()
-> import bt2.logging
<frozen importlib._bootstrap>(2237)_find_and_load()
<frozen importlib._bootstrap>(2226)_find_and_load_unlocked()
<frozen importlib._bootstrap>(1200)_load_unlocked()
<frozen importlib._bootstrap>(1129)_exec()
<frozen importlib._bootstrap>(1471)exec_module()
<frozen importlib._bootstrap>(321)_call_with_frames_removed()
> /home/jenkins/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/logging.py(23)<module>()
-> from bt2 import native_bt, object, utils
This can be reproduce easily on python 3.4.6 using pyenv and the following reproducer:
(my-virtual-env-3.4.6) joraj@/tmp/test[]$ tree
.
├── reprod
│ ├── __init__.py
│ ├── pkg1.py
│ ├── pkg2.py
│ └── start.py
└── test.py
test.py:
import reprod
print("got here")
__init__.py:
__version__= '1.0.0'
from reprod.start import *
pkg1.py:
from reprod import pkg2
pkg2.py:
from reprod import pkg1
start.py:
from reprod import pkg1
Using Python 3.4.6:
(my-virtual-env-3.4.6) joraj@/tmp/test[]$ python --version
Python 3.4.6
(my-virtual-env-3.4.6) joraj@/tmp/test[]$ python test.py
Traceback (most recent call last):
File "test.py", line 1, in <module>
import reprod
File "/tmp/test/reprod/__init__.py", line 3, in <module>
from reprod.start import *
File "/tmp/test/reprod/start.py", line 1, in <module>
from reprod import pkg1
File "/tmp/test/reprod/pkg1.py", line 1, in <module>
from reprod import pkg2
File "/tmp/test/reprod/pkg2.py", line 1, in <module>
from reprod import pkg1
ImportError: cannot import name 'pkg1'
(my-virtual-env-3.4.6) joraj@/tmp/test[]$
The same reproducer run with python 3.5.6 works fine (again using pyenv):
(my-virtual-env-3.5.6) joraj@/tmp/test[]$ python --version
Python 3.5.6
(my-virtual-env-3.5.6) joraj@/tmp/test[]$ python test.py
got here
(my-virtual-env-3.5.6) joraj@/tmp/test[]$
This is mostly due to the work done regarding circular dependency import
involving relative import[1][2].
[1] https://docs.python.org/3.5/whatsnew/3.5.html#other-language-changes
[2] https://bugs.python.org/issue17636
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: If43a6f9fad3d2b082fcd912f9ff6c193537d9f77
Francis Deslauriers [Sat, 13 Jul 2019 12:21:50 +0000 (08:21 -0400)]
tests: bt2: remove test_clock_class_priority_map.py
The clock class priority map concept does not exist anymore and all the
testcases were marked as `skip` since then.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Ib8e7d03662420f02446c5f901e86cf89fe09a09a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1702
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Francis Deslauriers [Sat, 13 Jul 2019 12:46:11 +0000 (08:46 -0400)]
Fix: flt.lttng-utils.debug-info: possible leak of `bt_message` on error
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I8587be191edcc644c0a8b9a9c9d6198fb308bb2d
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1703
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Francis Deslauriers [Fri, 12 Jul 2019 19:30:25 +0000 (15:30 -0400)]
tests: src.ctf.fs: add 2packets test trace
This test trace is used to test the regression fixed by following
commit:
commit
de38c26a14b60cb3b6d31cc124c187e2c1816bf5
Author: Francis Deslauriers <francis.deslauriers@efficios.com>
Date: Fri Jul 5 11:19:32 2019 -0400
Fix: src.ctf.fs: pointer arithmetics on non-adjacent memory
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I0c625905417d39339642a5c7fb31ceaa84ff5832
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1700
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Michael Jeanson <mjeanson@efficios.com>
Francis Deslauriers [Fri, 5 Jul 2019 16:02:47 +0000 (12:02 -0400)]
tests: bt_diff_cli: compare expected `stderr` too
Useful to confirm that a trace is read without warnings but could also
be used to confirm that an expected warning is indeed present.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I0c48c68a0aac832729da2d9c5b849fb58ec578e4
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1638
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Francis Deslauriers [Fri, 12 Jul 2019 21:27:29 +0000 (17:27 -0400)]
lib: plugin-so.c: ERROR and INFO level messages logged in callee
Failing to load a plugin from a file is not an error as we often need to
try to load shared libraries to tell if they actually contain a
Babeltrace plugin.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Ic5b72e0f0c050b7e11896d264787d852139eaf56
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1701
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Thu, 11 Jul 2019 19:35:11 +0000 (15:35 -0400)]
fix: use g_setenv for portability
setenv is not available on Windows and we already use g_setenv
everywhere else in the code base.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I2d39aa0777ad7bbf15c9d8c94f67cc18268c6835
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1691
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Francis Deslauriers [Fri, 12 Jul 2019 04:05:47 +0000 (00:05 -0400)]
Do not check `NULL` to call free(): free() accepts `NULL`
Semantic patch:
@@
expression e;
@@
- if (e) { free(e); }
+ free(e);
@@
expression e;
@@
- if (e != NULL) { free(e); }
+ free(e);
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I4d2d79a8beda4ca1ec7bacd5424c3c8e19d158f0
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1698
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Fri, 12 Jul 2019 04:11:27 +0000 (00:11 -0400)]
Fix: current-thread.h: append from message iterator, not from component
Reported-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I96c5f54aea280f5a3f16dc6285cb30a5ea05f2ad
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1697
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
Jonathan Rajotte [Thu, 11 Jul 2019 21:48:02 +0000 (17:48 -0400)]
tests: on Windows the plugin object extension is 'dll' not 'so'
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: I48eea5cb3ba8ea11d9994b56ec4b98e9cc577c6b
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1695
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Jonathan Rajotte [Thu, 11 Jul 2019 21:45:56 +0000 (17:45 -0400)]
Fix: cygwin does not honour LD_LIBRARY_PATH
Cygwin, just like Windows, relies on PATH to find libraries.
Error:
15:16:01 FAIL: python-plugin-provider/test_python_plugin_provider 1 - test_python_plugin_provider (unittest.loader._FailedTest)
15:16:01 # python-plugin-provider/test_python_plugin_provider: Traceback (most recent call last):
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "/usr/lib/python3.6/unittest/case.py", line 59, in testPartExecutor
15:16:01 # python-plugin-provider/test_python_plugin_provider: yield
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "/usr/lib/python3.6/unittest/case.py", line 605, in run
15:16:01 # python-plugin-provider/test_python_plugin_provider: testMethod()
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "/usr/lib/python3.6/unittest/loader.py", line 34, in testFailure
15:16:01 # python-plugin-provider/test_python_plugin_provider: raise self._exception
15:16:01 # python-plugin-provider/test_python_plugin_provider: ImportError: Failed to import test module: test_python_plugin_provider
15:16:01 # python-plugin-provider/test_python_plugin_provider: Traceback (most recent call last):
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "/cygdrive/c/Users/jenkins/workspace/babeltrace_master_winbuild/arch/cygwin/build/std/conf/std/src/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/native_bt.py", line 14, in swig_import_helper
15:16:01 # python-plugin-provider/test_python_plugin_provider: return importlib.import_module(mname)
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
15:16:01 # python-plugin-provider/test_python_plugin_provider: return _bootstrap._gcd_import(name[level:], package, level)
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "<frozen importlib._bootstrap>", line 994, in _gcd_import
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "<frozen importlib._bootstrap>", line 971, in _find_and_load
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "<frozen importlib._bootstrap>", line 658, in _load_unlocked
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "<frozen importlib._bootstrap>", line 571, in module_from_spec
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "<frozen importlib._bootstrap_external>", line 922, in create_module
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
15:16:01 # python-plugin-provider/test_python_plugin_provider: ImportError: No such file or directory
15:16:01 # python-plugin-provider/test_python_plugin_provider: During handling of the above exception, another exception occurred:
15:16:01 # python-plugin-provider/test_python_plugin_provider: Traceback (most recent call last):
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "/usr/lib/python3.6/unittest/loader.py", line 428, in _find_test_path
15:16:01 # python-plugin-provider/test_python_plugin_provider: module = self._get_module_from_name(name)
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "/usr/lib/python3.6/unittest/loader.py", line 369, in _get_module_from_name
15:16:01 # python-plugin-provider/test_python_plugin_provider: __import__(name)
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "/cygdrive/c/Users/jenkins/workspace/babeltrace_master_winbuild/arch/cygwin/build/std/conf/std/src/babeltrace/tests/python-plugin-provider/test_python_plugin_provider.py", line 20, in <module>
15:16:01 # python-plugin-provider/test_python_plugin_provider: import bt2
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "/cygdrive/c/Users/jenkins/workspace/babeltrace_master_winbuild/arch/cygwin/build/std/conf/std/src/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/__init__.py", line 26, in <module>
15:16:01 # python-plugin-provider/test_python_plugin_provider: from bt2.clock_class import *
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "/cygdrive/c/Users/jenkins/workspace/babeltrace_master_winbuild/arch/cygwin/build/std/conf/std/src/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/clock_class.py", line 23, in <module>
15:16:01 # python-plugin-provider/test_python_plugin_provider: from bt2 import native_bt, object, utils
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "/cygdrive/c/Users/jenkins/workspace/babeltrace_master_winbuild/arch/cygwin/build/std/conf/std/src/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/native_bt.py", line 17, in <module>
15:16:01 # python-plugin-provider/test_python_plugin_provider: _native_bt = swig_import_helper()
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "/cygdrive/c/Users/jenkins/workspace/babeltrace_master_winbuild/arch/cygwin/build/std/conf/std/src/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/native_bt.py", line 16, in swig_import_helper
15:16:01 # python-plugin-provider/test_python_plugin_provider: return importlib.import_module('_native_bt')
15:16:01 # python-plugin-provider/test_python_plugin_provider: File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
15:16:01 # python-plugin-provider/test_python_plugin_provider: return _bootstrap._gcd_import(name[level:], package, level)
15:16:01 # python-plugin-provider/test_python_plugin_provider: ModuleNotFoundError: No module named '_native_bt'
Change-Id: Ife8ce952a9dc34a78f3ee293e52a13c7f8a1310d
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1694
Reviewed-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Francis Deslauriers [Thu, 11 Jul 2019 18:57:07 +0000 (14:57 -0400)]
cleanup: configure.ac: replace spaces by tabs
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Iebe75fcb8cbf726b00d3dcc5752e4bbdbb8ae055
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1689
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Thu, 11 Jul 2019 18:59:47 +0000 (14:59 -0400)]
Fix warnings with bison >= 3.4 and autoconf < 2.70
Autoconf up to version 2.69 calls bison with '-y' to keep the output
filenames the same fixed values as the original YACC so the Makefiles
stay compatible with both tools. Starting with Bison 3.4 using '-y' will
generate warnings for non-POSIX YACC statements.
Our parser is not compatible with POSIX YACC, we already check for Bison
in the configure script. Bump the minimal version to 2.5 and add
'-Wno-yacc' to disable the warnings.
Autoconf 2.70 will drop the '-y' and use '-o y.tab.c' to get the same
behavior without enabling the warnings which will still work with this
workaround.
Change-Id: Iad3ba0dcfd9e6b4e2727236abd6ecb4a83de0c70
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1690
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Wed, 10 Jul 2019 19:39:10 +0000 (15:39 -0400)]
Make the in-tree CLI use the in-tree PPP
We build an in-tree version of the CLI binary that refers to in-tree
versions of the plugins and other artefacts which currently doesn't
support the Pyhton plugin provider.
Add the logic to use the in-tree version of the Python plugin provider
using the LIBBABELTRACE2_PLUGIN_PROVIDER_DIR environment variable.
If the variable is already set, do not overwrite it. If the python
plugin is enabled, set it to the appropriate in-tree path, otherwise set
it to an invalid path so we don't load an hypothetical system installed
provider.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I4151c36fef90c791cc8b515233d0d3e8fefcdbd5
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1683
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Wed, 10 Jul 2019 15:57:36 +0000 (11:57 -0400)]
Make the Python plugin provider a libtool module
The Python plugin provider is currently built as a library with an ABI
number and installed in the libdir. It is however not a library, it does
not provide a public API, no program should link to it and we load it
with g_module which is basically a wrapper over dlopen.
As such, build it as libtool "module" installed in the "plugin-providers"
subdirectory which is not part of the library search path.
Add an env variable 'LIBBABELTRACE2_PLUGIN_PROVIDER_DIR' that if is set
will override the builtin path to load the provider.
Change-Id: If475a89733c21f87d6a647e281326c13532a3d3d
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1682
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 10 Jul 2019 21:33:41 +0000 (17:33 -0400)]
Standardize `strcmp(a, b) == 0` instead of `!strcmp(a, b)`
`!strcmp(a, b)` always looks like a negation to me (not equal), while
the `== 0` version clearly shows that we're after equality.
Semantic patch:
@@
expression a;
expression b;
@@
- !strcmp(a, b)
+ strcmp(a, b) == 0
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I8e05b544c5cbd733e3d3e5b2c3554ee98669d1d5
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1685
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Wed, 10 Jul 2019 21:28:00 +0000 (17:28 -0400)]
Do not check `NULL` to call g_free(): g_free() accepts `NULL`
Semantic patch:
@@
expression e;
@@
- if (e) { g_free(e); }
+ g_free(e);
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ifbbca48f04cc198543aaa5d06d3b906b32fdf137
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1684
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Tue, 9 Jul 2019 21:11:33 +0000 (17:11 -0400)]
Fix: bt2: adjust reference counting of value objects
This patch fixes some reference counting bugs when handling bt_values in
Python.
The first one is when a component class implemented in Python returns
None to a query. The _UserComponentType._bt_query_from_native method
must return a bt_value address, and it must be a new reference that it
transfers to its caller. Currently, it returns a pointer to
bt_value_null, without ever acquiring a new reference to it.
The second bug, in the same area, is when the component class returns
some value. We currently call bt2.object._release, effectively
transferring the Python object's reference to the value to the caller.
However, the user could have kept a reference to that value, wishing to
keep using it. We should instead return a new reference to this value.
_UserComponentType._bt_query_from_native is therefore modified to
acquire a new reference to return to its caller, regardless of if the
value is bt_value_null or another value.
The third bug is in bt2.value._create_from_ptr. This function receives
a pointer to a bt_value, and steals it from its caller. It creates a
Python object that, when destroy, will put the reference. However, when
the passed pointer is bt_value_null, it just returns None. The
reference to bt_value_null that has been passed therefore never gets
put.
Bugs 1 and 3 would cancel each other: when doing a query that returned
bt_value_null from Python to a Python component class, we would miss
getting a ref to bt_value_null (bug 1) but we would also miss putting it
(bug 3).
The patch adds a test where Python code queries a Python component
class, which returns None. While it might not have caught all the bugs
fixed in this patch, it at least exercises the code paths touched in
this patch, which were not tested before, AFAICT.
Change-Id: I0f7bf1004d9d766db4cc0c83475ed37ce2e3f1ed
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1666
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Michael Jeanson [Tue, 9 Jul 2019 22:01:40 +0000 (18:01 -0400)]
Rename PLUGINSDIR to BABELTRACE_PLUGINS_DIR and enable it
The configure script uses the PLUGINSDIR variable to set the
installation directory of the plugins on the system. However this
variable is not used when setting this value in the library, the
default value happens to match.
Fix the build system to honor this variable and rename it to
BABELTRACE_PLUGINS_DIR.
Change-Id: I8575d455017f2964e67fdfab609f5c609434d37b
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1681
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Tue, 9 Jul 2019 16:27:49 +0000 (12:27 -0400)]
tests: update test_trimming to remove the wrapper
Run babeltrace with the python environment set in bt_diff_cli instead
of wrapping the test script in another script. Also fix some related
shellcheck errors in test_trimming.
Change-Id: Ie465dfd65c15a0e51f9474e7d3ebe693a2308488
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1657
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Jonathan Rajotte [Wed, 10 Jul 2019 14:32:38 +0000 (10:32 -0400)]
trimmer: proper formatting of timestamp conversion error
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: I7da192ae19fd1223bce9d10d2263d91aa698f925
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1677
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Jonathan Rajotte [Tue, 9 Jul 2019 16:53:44 +0000 (12:53 -0400)]
tests: specify the timezone offset
On mingw (Windows) EST alone resolve to UTC since windows does not
recognize EST.
We could also use "America/Toronto" but this would require more change
to the test. Since this test is only there to test the TZ usage the
actual value of TZ does not matter. UTC5 would have done the same job.
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: I05270504c69a90d4e9893435d1c1b583579041ae
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1662
Reviewed-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Jonathan Rajotte [Tue, 9 Jul 2019 19:57:52 +0000 (15:57 -0400)]
Fix: use return of bt_localtime_r as success criteria
Encountered error:
jenkins@ci-node-bionic-amd64-05-10:~/babeltrace/tests/cli$ TZ=EST4 ./../utils/../../src/cli/babeltrace2 ./../utils/../data/ctf-traces/succeed/wk-heartbeat-u/ --begin 12:48:17.
587029529
07-09 15:31:04.183
1224375 1224375 E PLUGIN/FLT.UTILS.TRIMMER Cannot convert timestamp to date and time: No such file or directory[trimmer] ts=
1351532897
07-09 15:31:04.183
1224375 1224375 W LIB/MSG-ITER @iterator.c:865 Component input port message iterator's "next" method failed: iter-addr=0x55705c7ee870, iter-type=BT_MESSAGE_ITERATOR_TYPE_SELF_COMPONENT_PORT_INPUT, iter-upstream-comp-name="trimmer", iter-upstream-comp-log-level=BT_LOGGING_LEVEL_WARN, iter-upstream-comp-class-type=BT_COMPONENT_CLASS_TYPE_FILTER, iter-upstream-comp-class-name="trimmer", iter-upstream-comp-class-partial-descr="Keep messages that occur within ", iter-upstream-port-type=BT_PORT_TYPE_OUTPUT, iter-upstream-port-name="out", status=ERROR
07-09 15:31:04.183
1224375 1224375 W LIB/GRAPH @graph.c:580 Component's "consume" method failed: status=ERROR, comp-addr=0x55705c7ec4a0, comp-name="pretty", comp-log-level=BT_LOGGING_LEVEL_WARN, comp-class-type=BT_COMPONENT_CLASS_TYPE_SINK, comp-class-name="pretty", comp-class-partial-descr="Pretty-print messages (`text` fo", comp-class-is-frozen=0, comp-class-so-handle-addr=0x55705c808840, comp-class-so-handle-path="/home/jenkins/babeltrace/src/plugins/text/babeltrace-plugin-text.la", comp-input-port-count=1, comp-output-port-count=0
07-09 15:31:04.183
1224375 1224375 E CLI Graph failed to complete successfully
But the same use of TZ=EST4 works fine on date.
bt_localtime_r behave the same as localtime_r. The success criteria of
localtime_r is that the returned pointer is not null. Checking the value
of errno as a success criteria is never a good idea.
man errno:
The value in errno is significant only when the return value of the
call indicated an error (i.e., -1 from most system calls; -1 or NULL
from most library functions); a function that succeeds is allowed to
change errno. The value of errno is never set to zero by any system
call or library function.
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: I1454ed1d5f8fe2aa095c8fcbb7b315bcb3c83871
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1664
Reviewed-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 10 Jul 2019 06:15:51 +0000 (02:15 -0400)]
lib: put __BT_UPCAST() and __BT_UPCAST_CONST() in `babeltrace.h`
This makes it possible to clear those macros at the end of
`babeltrace.h`. This is safe because each public header is guaranteed to
be included from `babeltrace.h`.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ib030f70869b48e76bd78a91be55e581aeb3f3435
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1676
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 10 Jul 2019 06:05:42 +0000 (02:05 -0400)]
Move `include/babeltrace2/ctf-writer` -> `include/babeltrace2-ctf-writer`
CTF writer is not part of the Babeltrace library.
With this patch, `include/babeltrace2` contains only the Babeltrace
library public headers, and `include/babeltrace2-ctf-writer` contains
only the Babeltrace CTF writer library public headers.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I605ef01e1afbf0188f1ba5b094aed48717d46aff
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1675
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 10 Jul 2019 06:01:05 +0000 (02:01 -0400)]
Remove legacy `include/babeltrace2/{ctf-ir,ctf}` directories
Because Babeltrace 2's CTF writer is not exactly a drop-in replacement
of Babeltrace 1's CTF writer since all the headers are part of the
`babeltrace2` directory (to be able to install both on the same system),
we can remove those legacy directories.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I28362031162aae851cfd42d2d8d07828e5f69515
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1674
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 10 Jul 2019 05:58:12 +0000 (01:58 -0400)]
lib: standardize public header copyright notice
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I7be25f0d622d6ec98fb83ec48d19b3578078ca4a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1673
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 10 Jul 2019 05:51:15 +0000 (01:51 -0400)]
lib: remove "For *" comments in public headers
Those comments are very hard to maintain and are pretty much useless.
There are tools to ensure that headers include exactly what they need.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Idb7545526426e0a18e6a2fa8af20b9f22ed88407
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1672
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 10 Jul 2019 05:33:02 +0000 (01:33 -0400)]
lib: force user to include `<babeltrace2/babeltrace.h>`
This patch makes all the library headers except
`<babeltrace2/babeltrace.h>` refuse to be included directly. This will
allow us to change how those headers are organized, and even remove some
if needed (as long as we move the existing content to another existing
header), because we know the user does not include specific headers.
To do this, the main `<babeltrace2/babeltrace.h>` defines
`__BT_IN_BABELTRACE_H`, and each specific header does:
#ifndef __BT_IN_BABELTRACE_H
# error "Please include <babeltrace2/babeltrace.h> instead."
#endif
at the beginning.
Because each specific header has the guarantee that it is included from
`<babeltrace2/babeltrace.h>`, they don't include
`<babeltrace2/func-status.h>`: `<babeltrace2/babeltrace.h>` includes it,
and clears the private status definitions itself at the end. Therefore
this patch removes `undef-func-status.h`.
Some parts of the project need to have access to the `__BT_FUNC_STATUS_*`
definitions. To do that, they do:
#define __BT_IN_BABELTRACE_H
#include <babeltrace2/func-status.h>
Because `<babeltrace2/babeltrace.h>` is now manually written and
maintained, this patch removes `extras/gen-babeltrace-h.py` which we
used to generate it from `include/Makefile.am`.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Icc3bf0bec18a465024b316f02f84df7a75429b87
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1671
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 10 Jul 2019 04:23:35 +0000 (00:23 -0400)]
lib: standardize public include guards
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ic65bbac790724a2cd05bde25049f5a02a702e383
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1670
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 10 Jul 2019 05:39:33 +0000 (01:39 -0400)]
src/{ctfser,fd-cache}/Makefile.am: remove unneeded `AM_CPPFLAGS`
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Id18bcca5047c89335de39a55c4e890683188c2f6
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1669
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 10 Jul 2019 05:31:15 +0000 (01:31 -0400)]
Fix: lib: conflicting types for `bt_plugin_set_get_plugin_count`
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I8324668d820bf8139a54df66bfa1a81312c5da3c
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1668
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 10 Jul 2019 05:28:13 +0000 (01:28 -0400)]
Fix: CTF writer: make library have its own public `types.h`
This was overlooked when segregating the CTF writer library from the
Babeltrace library: CTF writer needs its own `bt_ctf_bool` type with its
own `BT_CTF_TRUE` and `BT_CTF_FALSE` definitions.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I4e6d0b2d0956479a554cc8358d646fc1676873f6
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1667
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Tue, 9 Jul 2019 12:25:44 +0000 (08:25 -0400)]
lib: make BT_ASSERT_{PRE,POST}() always on; add BT_ASSERT_{PRE,POST}_DEV()
This patch makes the BT_ASSERT_PRE() and BT_ASSERT_POST() macros, and
all their variants, always executed. In other words, the checks are
executed in developer mode and in production mode.
What used to be BT_ASSERT_PRE() and BT_ASSERT_POST(), enabled only in
developer mode, is now BT_ASSERT_PRE_DEV() and BT_ASSERT_POST_DEV().
The goal of this patch is to add precondition and postcondition checks
on the slow path which are enabled in production to catch more
programming errors. As I meticulously chose which one to invoke at each
site, this patch does not change the performance of the library during a
graph run, which is most of the execution time.
Because all the freezing functions are only enabled in developer mode,
`assert-pre.h` only offers BT_ASSERT_PRE_DEV_HOT() currently (there's no
production mode BT_ASSERT_PRE_HOT()). This is because some freezing
functions can be called on the fast path (for example, the internal
bt_event_create() freezes the event class, packet, and stream; this is
called for each event message), while other are only called on the slow
path. This patch does not decouple the frequent ones from the infrequent
ones; this work should be done by a subsequent patch.
Where all the precondition/postcondition assertions used to be only
enabled in developer mode, the library now contains 105 production
checks.
When I test with a 1.4 GiB CTF trace and a dummy output in production
mode, the difference of the total execution times with and without this
patch falls within the range of uncertainty.
The guideline to use one or the other is:
* Use BT_ASSERT_PRE_DEV() and BT_ASSERT_POST_DEV() when the call site
could occur at least once per message.
This is why all the borrowing functions use the developer mode
versions as we don't know at which frequency what will be accessed.
* Use BT_ASSERT_PRE() and BT_ASSERT_POST() everywhere else.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I2ccdec4cee332a13d330474d287c564b7cb90352
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1661
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Tue, 9 Jul 2019 20:11:44 +0000 (16:11 -0400)]
lib: rename `bt_*_labels_by_value` -> `bt_*_labels_for_value`
It just feels more right this way.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I6cbe9542417ba4b9688d1920189591727597d3ba
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1665
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Tue, 9 Jul 2019 19:37:09 +0000 (15:37 -0400)]
Rename WARN log level to WARNING
It's weird that this log level is abbreviated, but not the other ones.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Id14da1ba6cd38f7d824079218280bff9dea1e276
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1663
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Tue, 9 Jul 2019 06:48:49 +0000 (02:48 -0400)]
src/lib/{current-thread,error}.c: put precond. assertion at right place
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Id43d7d881253cd00e9dec46770ba517fc22f0adf
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1660
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
This page took 0.058312 seconds and 4 git commands to generate.