Parse comments between tokens

Parse comments between tokens

This patch replaces many whitespace parsing between the individual
tokens of an item by whitespace and comment parsing.

This makes it possible to comment out parts of items:

    aa bb cc @#32#64#~ffh# 88

Result:

    aa bb cc 00 00 00 00 00 88

Moreover, insignificant symbols are only supported between and around
items, hexadecimal nibbles, and binary bits now. This is where they're
the most useful to improve readability, not between tokens of an item.
For example, this is not useful:

    s/:?latin1,"hello world"=*|5

The new `tests/pass-comment-all.nt` file tests comments in all positions
between eligible tokens.

Change-Id: I7167440723010c2549f614fc6ab41621df0fd8b4
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-on: https://review.lttng.org/c/normand/+/11020
Tested-by: jenkins <jenkins@lttng.org>