Add support for plugins written in Python
Now that the bt2 package exists and allows a user to create its own
component classes, we're not so far from a Python plugin support, a
Babeltrace plugin being only a set of basic attributes and a list of
component classes.
The chosen approach here is to add a module to the bt2 package,
py_plugin, which contains the following:
* plugin_component_class(): Function to be used as a decorator to tag a
given user component class as being part of a Babeltrace plugin.
* register_plugin(): Function to be called anywhere from a Python module
to register this module as a Babeltrace plugin. This call receives
the name of the plugin and other optional attributes (description,
author, license, etc.).
* _try_load_plugin_module(): Function reserved for the Babeltrace
library to try to get a plugin information object from a given
path (Python file).
The same logic could be implemented with the Python C API, but since
a Babeltrace Python plugin needs to import the bt2 package anyway,
we're sure that the package exists and that we can use it. Arguably
this function could be located in another module, outside the bt2
package, for example in /usr/lib/babeltrace/plugin_utils.py.
Here's a very simple Python plugin which contains a single sink
component class named MySink:
import bt2
@bt2.plugin_component_class
class MySink(bt2.UserSinkComponent):
def __init__(self, params, name):
self._stuff = params['stuff']
def _consume(self):
got_one = False
for notif_iter in self._input_notification_iterators:
try:
notif = next(notif_iter)
except StopIteration:
continue
got_one = True
if isinstance(notif, bt2.TraceEventNotification):
event = notif.event
print('>>>', self._stuff, event.name)
if not got_one:
raise bt2.Stop
bt2.register_plugin(__name__, name='my_plugin',
author='Philippe Proulx', license='MIT',
version=(2, 35, 3, '-dev'))
Here's what happens when the C API user does this:
plugins = bt_plugin_create_all_from_file(
"python-plugins/bt_plugin_hello.py");
1. bt_plugin_create_all_from_file() calls
bt_plugin_so_create_all_from_file() without success, because the
extension does not match.
Then it tries bt_plugin_python_create_all_from_file().
2. bt_plugin_python_create_all_from_file() makes sure that the base name
of the file to load starts with `bt_plugin_` and ends with `.py`. The
prefix is up for debate, but my argument is that we don't want to
import all the Python files when loading all the plugins of a
directory: some Python files could be modules imported by plugins,
not actual plugins. Having the prefix at least reduces the
possibility of importing a non-plugin Python module.
3. init_python() is called. This function initializes the Python
interpreter if it's not already done. It also refuses to do so if
BABELTRACE_DISABLE_PYTHON_PLUGINS=1 (environment variable). It also
imports the bt2.py_plugin module and finds the
_try_load_plugin_module() function (global object, put in the
library's destructor).
If the needed global objects are still not set after this call,
bt_plugin_python_create_all_from_file() returns an error. If
init_python() fails, all the future calls to
bt_plugin_python_create_all_from_file() will also fail (we don't
attempt to initialize Python again if it failed once).
4. bt_plugin_python_create_all_from_file() calls
_try_load_plugin_module() (Python) with the path, let's say
`python-plugins/bt_plugin_hello.py`.
a) A module name is created for this plugin. Since the plugin system
can load plugins from files having the same base name, but in
different directories, we cannot use the base name here. For
example, we cannot name the module `bt_plugin_hello`: module names
are unique in Python (for the whole interpreter) and are registered
in the sys.modules dictionary.
What's done here is to hash the path and prefix this with
`bt_plugin_`. For example, this module would have the name
bt_plugin_9dc80c7b58e49667cb5898697a5197d22f3a09386d017e08e2....
b) The function tries to import the module using importlib. The module
is executed from top to bottom:
i) The @bt2.plugin_component_class decorator is called on the MySink
class (which, thanks to its metaclass, has an associated native
BT component class created): as a reminder, this is the
equivalent of:
MySink = bt2.plugin_component_class(MySink)
This function adds an attribute to MySink: it sets
MySink._bt_plugin_component_class to None. The mere existence
of this attribute, whatever its value, means that this user
component class is part of the plugin.
ii) The module calls bt2.register_plugin(), passing __name__ as the
first argument. This is used for bt2.register_plugin() to find
the module from which it is called. In this case, __name__
is bt_plugin_9dc80c7b58e49667cb5898697a5197d22f3a09386d017e...
bt2.register_plugin() gets the actual module object from
sys.modules and adds the _bt_plugin_info attribute to it after
checking that all the arguments are correct. This is an object
which contains, so far, the name and other optional properties
of the plugin.
c) If the import is successful, the function looks for the
_bt_plugin_info attribute in the module object. If it's not found,
an error is raised.
d) The function then uses the inspect module to find all the classes
in the module which contain the _bt_plugin_component_class
attribute. All the native BT component class addresses are appended
to a list which is set as the comp_class_addrs attribute of the
plugin info object.
_try_load_plugin_module() returns this plugin info object.
5. bt_plugin_python_create_all_from_file() calls
bt_plugin_from_python_plugin_info() to convert the returned Python
object to a bt_plugin object, which is returned to the user.
If anything goes wrong on the Python side during this call, and if
BABELTRACE_VERBOSE=1, the Python traceback is printed.
With the plugin above, we can run the converter like this to use its
sink component class:
babeltrace --plugin-path python-plugins \
-i ctf.fs -P /path/to/trace \
-o my_plugin.MySink -p 'stuff="hello there!"'
We get something like this:
>>> hello there! sched_switch
>>> hello there! sys_open
>>> hello there! sched_wakeup
...
It seems like Py_InitializeEx() and PyImport_ImportModule() can override
the current SIGINT handler, which is why we save the old handler before
calling those and then restore it afterwards. This seems to work.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
13 files changed:
This page took 0.029983 seconds and 4 git commands to generate.