This package offers both a portable {py3} module and a command-line
tool.
-WARNING: This version of Normand is 0.8, meaning both the Normand
+WARNING: This version of Normand is 0.9, meaning both the Normand
language and the module/CLI interface aren't stable.
ifdef::env-github[]
The encoded integer is the evaluation of a valid {py3} expression which
may include label and variable names.
+Conditional::
++
+Input:
++
+----
+aa bb cc
+
+(
+ "foo"
+
+ !if {ICITTE > 10}
+ "bar"
+ !end
+) * 4
+----
++
+Output:
++
+----
+aa bb cc 66 6f 6f 66 6f 6f 66 6f 6f 62 61 72 66 ┆ •••foofoofoobarf
+6f 6f 62 61 72 ┆ oobar
+----
+
Repetition::
+
Input:
|
The current offset has an effect on the value of <<label,labels>> and of
the special `ICITTE` name in <<fixed-length-number,fixed-length
-number>>, <<leb-128-integer,LEB128 integer>>, and
-<<variable-assignment,variable assignment>> expression evaluation.
+number>>, <<leb-128-integer,LEB128 integer>>,
+<<variable-assignment,variable assignment>>,
+<<conditional-block,conditional block>>, <<repetition-block,repetition
+block>>, and <<post-item-repetition,post-item repetition>> expression
+evaluation.
Each generated byte increments the current offset.
|One or more `--label` options.
|<<variable-assignment,Variables>>
-|Mapping of variable names to integral values.
+|Mapping of variable names to integral or floating point number values.
|`init_variables` parameter of the `parse()` function.
|One or more `--var` options.
|===
* A <<group,group>>, that is, a scoped sequence of items.
+* A <<conditional-block,conditional block>>.
+
* A <<repetition-block,repetition block>>.
Moreover, you can repeat many items above a constant or variable number
number or expression.
* Between the ``!repeat``/``!r`` prefix and the following constant
integer, name, or expression of a repetition block.
+* Between the ``!if`` prefix and the following name or expression of a
+ conditional block.
A comment is anything between two ``pass:[#]`` characters on the same
line, or from ``pass:[#]`` until the end of the line. Whitespaces and
. An encoding length in bits amongst:
+
--
-The expression evaluates to an `int` value::
+The expression evaluates to an `int` or `bool` value::
`8`, `16`, `24`, `32`, `40`, `48`, `56`, and `64`.
++
+NOTE: Normand automatically converts a `bool` value to `int`.
The expression evaluates to a `float` value::
`32` and `64`.
. The ``pass:[{]`` prefix.
-. A valid {py3} expression.
+. A valid {py3} expression of which the evaluation result type
+ is `int` or `bool` (automatically converted to `int`).
+
For an LEB128 integer at some source location{nbsp}__**L**__, this
expression may contain:
. The `<` prefix.
-. A valid {py3} name which is not `ICITTE` (see
- <<fixed-length-number>>, <<leb128-integer>>, and
- <<variable-assignment>> to learn more).
+. A valid {py3} name which is not `ICITTE`.
. The `>` suffix.
. The ``pass:[{]`` prefix.
-. A valid {py3} name which is not `ICITTE` (see
- <<fixed-length-number>>, <<leb128-integer>>, and
- <<variable-assignment>> to learn more).
+. A valid {py3} name which is not `ICITTE`.
. The `=` character.
-. A valid {py3} expression.
+. A valid {py3} expression of which the evaluation result type
+ is `int`, `float`, or `bool` (automatically converted to `int`).
+
For a variable assignment at some source location{nbsp}__**L**__, this
expression may contain the name of any accessible <<label,label>> (not
----
====
+=== Conditional block
+
+A _conditional block_ represents either the bytes of one or more items
+if some expression is true, or no bytes at all if it's false.
+
+A conditional block is:
+
+. The `!if` prefix.
+
+. One of:
+
+** The ``pass:[{]`` prefix, a valid {py3} expression of which the
+ evaluation result type is `int` or `bool` (automatically converted to
+ `int`), and the ``pass:[}]`` suffix.
++
+For a repetition at some source location{nbsp}__**L**__, this expression
+may contain:
++
+--
+* The name of any <<label,label>> defined before{nbsp}__**L**__
+ which isn't within a nested group.
+* The name of any <<variable-assignment,variable>> known
+ at{nbsp}__**L**__ which doesn't, directly or indirectly, refer to a
+ label defined after{nbsp}__**L**__.
+--
++
+The value of the special name `ICITTE` (`int` type) in this expression
+is the <<cur-offset,current offset>> (before handling the contained
+items).
+
+** A valid {py3} name.
++
+For the name `__NAME__`, this is equivalent to the
+`pass:[{]__NAME__pass:[}]` form above.
+
+. Zero or more items.
+
+. The `!end` suffix.
+
+====
+Input:
+
+----
+{at = 1}
+{rep_count = 9}
+
+!repeat rep_count
+ "meow "
+
+ !if {ICITTE > 25}
+ "mix"
+
+ !if {at < rep_count} 20 !end
+ !end
+
+ {at = at + 1}
+!end
+----
+
+Output:
+
+----
+6d 65 6f 77 20 6d 65 6f 77 20 6d 65 6f 77 20 6d ┆ meow meow meow m
+65 6f 77 20 6d 65 6f 77 20 6d 65 6f 77 20 6d 69 ┆ eow meow meow mi
+78 20 6d 65 6f 77 20 6d 69 78 20 6d 65 6f 77 20 ┆ x meow mix meow
+6d 69 78 20 6d 65 6f 77 20 6d 69 78 ┆ mix meow mix
+----
+====
+
+====
+Input:
+
+----
+<str_beg>
+u16le"meow mix!"
+<str_end>
+
+!if {str_end - str_beg > 10}
+ " BIG"
+!end
+----
+
+Output:
+
+----
+6d 00 65 00 6f 00 77 00 20 00 6d 00 69 00 78 00 ┆ m•e•o•w• •m•i•x•
+21 00 20 42 49 47 ┆ !• BIG
+----
+====
+
=== Repetition block
A _repetition block_ represents the bytes of one or more items repeated
** A positive integer (hexadecimal starting with `0x` or `0X` accepted)
which is the number of times to repeat the previous item.
-** The ``pass:[{]`` prefix, a valid {py3} expression, and the
- ``pass:[}]`` suffix.
+** The ``pass:[{]`` prefix, a valid {py3} expression of which the
+ evaluation result type is `int` or `bool` (automatically converted to
+ `int`), and the ``pass:[}]`` suffix.
+
For a repetition at some source location{nbsp}__**L**__, this expression
may contain:
+
--
-* The name of any <<label,label>> defined before{nbsp}__**L**__.
+* The name of any <<label,label>> defined before{nbsp}__**L**__
+ which isn't within a nested group.
* The name of any <<variable-assignment,variable>> known
at{nbsp}__**L**__ which doesn't, directly or indirectly, refer to a
label defined after{nbsp}__**L**__.
----
====
-====
-This example shows how to use a repetition block as a conditional
-section depending on some predefined variable.
-
-Input:
-
-----
-aa bb cc dd
-
-!repeat cond
- ee ff "meow mix" 00
-!end
-
-{be} {-1993:16}
-----
-
-Output (`cond` is 0):
-
-----
-aa bb cc dd f8 37
-----
-
-Output (`cond` is 1):
-
-----
-aa bb cc dd ee ff 6d 65 6f 77 20 6d 69 78 00 f8 ┆ ••••••meow mix••
-37 ┆ 7
-----
-====
-
=== Post-item repetition
A _post-item repetition_ represents the bytes of an item repeated a
A post-item repetition is:
-. Any item except:
+. One of those items:
-** A <<current-byte-order-setting,current byte order setting>>.
-** A <<current-offset-setting,current offset setting>>.
-** A <<label,label>>.
-** A <<offset-alignment,offset alignment>>.
-** A <<variable-assignment,variable assignment>>.
-** A <<repetition-block,repetition block>>.
+** A <<byte-constant,byte constant>>.
+** A <<literal-string,literal string>>.
+** A <<fixed-length-number,fixed-length number>>.
+** An <<leb128-integer,LEB128 integer>>.
+** A <<group,group>>.
. The ``pass:[*]`` character.
** A positive integer (hexadecimal starting with `0x` or `0X` accepted)
which is the number of times to repeat the previous item.
-** The ``pass:[{]`` prefix, a valid {py3} expression, and the
- ``pass:[}]`` suffix.
+** The ``pass:[{]`` prefix, a valid {py3} expression of which the
+ evaluation result type is `int` or `bool` (automatically converted to
+ `int`), and the ``pass:[}]`` suffix.
+
For a repetition at some source location{nbsp}__**L**__, this expression
may contain:
+
--
-* The name of any <<label,label>> defined before{nbsp}__**L**__ and
- which isn't part of its repeated item.
+* The name of any <<label,label>> defined before{nbsp}__**L**__
+ which isn't within a nested group and
+ which isn't part of the repeated item.
* The name of any <<variable-assignment,variable>> known
at{nbsp}__**L**__, which isn't part of its repeated item, and which
doesn't, directly or indirectly, refer to a label defined
# Upstream repository: <https://github.com/efficios/normand>.
__author__ = "Philippe Proulx"
-__version__ = "0.8.0"
+__version__ = "0.9.0"
__all__ = [
"ByteOrder",
"parse",
)
+# Conditional item.
+class _Cond(_Item, _ExprMixin):
+ def __init__(
+ self, item: _Item, expr_str: str, expr: ast.Expression, text_loc: TextLocation
+ ):
+ super().__init__(text_loc)
+ _ExprMixin.__init__(self, expr_str, expr)
+ self._item = item
+
+ # Conditional item.
+ @property
+ def item(self):
+ return self._item
+
+ def __repr__(self):
+ return "_Cond({}, {}, {}, {})".format(
+ repr(self._item),
+ repr(self._expr_str),
+ repr(self._expr),
+ repr(self._text_loc),
+ )
+
+
# Expression item type.
-_ExprItemT = Union[_FlNum, _Leb128Int, _VarAssign, _Rep]
+_ExprItemT = Union[_FlNum, _Leb128Int, _VarAssign, _Rep, _Cond]
# A parsing error containing a message and a text location.
return _AlignOffset(val, pad_val, begin_text_loc)
# Patterns for _expect_rep_mul_expr()
- _rep_expr_prefix_pat = re.compile(r"\{")
- _rep_expr_pat = re.compile(r"[^}p]+")
- _rep_expr_suffix_pat = re.compile(r"\}")
-
- # Parses the multiplier expression of a repetition (block or
- # post-item) and returns the expression string and AST node.
- def _expect_rep_mul_expr(self):
+ _rep_cond_expr_prefix_pat = re.compile(r"\{")
+ _rep_cond_expr_pat = re.compile(r"[^}]+")
+ _rep_cond_expr_suffix_pat = re.compile(r"\}")
+
+ # Parses the expression of a conditional block or of a repetition
+ # (block or post-item) and returns the expression string and AST
+ # node.
+ def _expect_rep_cond_expr(self, accept_int: bool):
expr_text_loc = self._text_loc
# Constant integer?
- m = self._try_parse_pat(self._pos_const_int_pat)
+ m = None
+
+ if accept_int:
+ m = self._try_parse_pat(self._pos_const_int_pat)
if m is None:
# Name?
if m is None:
# Expression?
- if self._try_parse_pat(self._rep_expr_prefix_pat) is None:
+ if self._try_parse_pat(self._rep_cond_expr_prefix_pat) is None:
+ if accept_int:
+ mid_msg = "a positive constant integer, a name, or `{`"
+ else:
+ mid_msg = "a name or `{`"
+
# At this point it's invalid
- self._raise_error(
- "Expecting a positive integral multiplier, a name, or `{`"
- )
+ self._raise_error("Expecting {}".format(mid_msg))
# Expect an expression
expr_text_loc = self._text_loc
- m = self._expect_pat(self._rep_expr_pat, "Expecting an expression")
+ m = self._expect_pat(self._rep_cond_expr_pat, "Expecting an expression")
expr_str = m.group(0)
# Expect `}`
- self._expect_pat(self._rep_expr_suffix_pat, "Expecting `}`")
+ self._expect_pat(self._rep_cond_expr_suffix_pat, "Expecting `}`")
else:
expr_str = m.group(0)
else:
return self._ast_expr_from_str(expr_str, expr_text_loc)
+ # Parses the multiplier expression of a repetition (block or
+ # post-item) and returns the expression string and AST node.
+ def _expect_rep_mul_expr(self):
+ return self._expect_rep_cond_expr(True)
+
+ # Common block end pattern
+ _block_end_pat = re.compile(r"!end\b\s*")
+
# Pattern for _try_parse_rep_block()
_rep_block_prefix_pat = re.compile(r"!r(?:epeat)?\b\s*")
- _rep_block_end_pat = re.compile(r"!end\b\s*")
# Tries to parse a repetition block, returning a repetition item on
# success.
# Expect end of block
self._skip_ws_and_comments()
self._expect_pat(
- self._rep_block_end_pat, "Expecting an item or `!end` (end of repetition)"
+ self._block_end_pat, "Expecting an item or `!end` (end of repetition block)"
)
# Return item
return _Rep(_Group(items, items_text_loc), expr_str, expr, begin_text_loc)
+ # Pattern for _try_parse_cond_block()
+ _cond_block_prefix_pat = re.compile(r"!if\b\s*")
+
+ # Tries to parse a conditional block, returning a conditional item
+ # on success.
+ def _try_parse_cond_block(self):
+ begin_text_loc = self._text_loc
+
+ # Match prefix
+ if self._try_parse_pat(self._cond_block_prefix_pat) is None:
+ # No match
+ return
+
+ # Expect expression
+ self._skip_ws_and_comments()
+ expr_str, expr = self._expect_rep_cond_expr(False)
+
+ # Parse items
+ self._skip_ws_and_comments()
+ items_text_loc = self._text_loc
+ items = self._parse_items()
+
+ # Expect end of block
+ self._skip_ws_and_comments()
+ self._expect_pat(
+ self._block_end_pat,
+ "Expecting an item or `!end` (end of conditional block)",
+ )
+
+ # Return item
+ return _Cond(_Group(items, items_text_loc), expr_str, expr, begin_text_loc)
+
# Tries to parse a base item (anything except a repetition),
# returning it on success.
def _try_parse_base_item(self):
if item is not None:
return item
+ # Conditional block item?
+ item = self._try_parse_cond_block()
+
+ if item is not None:
+ return item
+
# Pattern for _try_parse_rep_post()
_rep_post_prefix_pat = re.compile(r"\*")
#
# The steps of generation are:
#
-# 1. Validate that each repetition and LEB128 integer expression uses
-# only reachable names.
+# 1. Validate that each repetition, conditional, and LEB128 integer
+# expression uses only reachable names.
#
-# 2. Compute and keep the effective repetition count and LEB128 integer
-# value for each repetition and LEB128 integer instance.
+# 2. Compute and keep the effective repetition count, conditional value,
+# and LEB128 integer value for each repetition and LEB128 integer
+# instance.
#
# 3. Generate bytes, updating the initial state as it goes which becomes
# the final state after the operation.
#
-# During the generation, when handling a `_Rep` or `_Leb128Int` item,
-# we already have the effective repetition count or value of the
-# instance.
+# During the generation, when handling a `_Rep`, `_Cond`, or
+# `_Leb128Int` item, we already have the effective repetition count,
+# conditional value, or value of the instance.
#
# When handling a `_Group` item, first update the current labels with
# all the immediate (not nested) labels, and then handle each
visitor.visit(expr)
return visitor.names
- # Validates that all the repetition and LEB128 integer expressions
- # within `group` don't refer, directly or indirectly, to subsequent
- # labels.
+ # Validates that all the repetition, conditional, and LEB128 integer
+ # expressions within `group` don't refer, directly or indirectly, to
+ # subsequent labels.
#
# The strategy here is to keep a set of allowed label names, per
# group, initialized to `allowed_label_names`, and a set of allowed
# it's in there): a variable which refers to an unreachable name
# is unreachable itself.
#
- # `_Rep` and `_Leb128`:
+ # `_Rep`, `_Cond`, and `_Leb128`:
# Make sure all the names within its expression are allowed.
#
# `_Group`:
_ExprValidator(item, allowed_label_names | allowed_variable_names).visit(
item.expr
)
- elif type(item) is _Rep:
+ elif type(item) is _Rep or type(item) is _Cond:
# Validate the expression first
_ExprValidator(item, allowed_label_names | allowed_variable_names).visit(
item.expr
item,
)
+ # Convert `bool` result type to `int` to normalize
+ if type(val) is bool:
+ val = int(val)
+
# Validate result type
expected_types = {int} # type: Set[type]
type_msg = "`int`"
align_bytes = item.val // 8
return (offset + align_bytes - 1) // align_bytes * align_bytes
- # Computes the effective value for each repetition and LEB128
- # integer instance, filling `instance_vals` (if not `None`) and
- # returning `instance_vals`.
+ # Computes the effective value for each repetition, conditional, and
+ # LEB128 integer instance, filling `instance_vals` (if not `None`)
+ # and returning `instance_vals`.
#
# At this point it must be known that, for a given variable-length
# item, its expression only contains reachable names.
#
- # When handling a `_Rep` item, this function appends its effective
- # multiplier to `instance_vals` _before_ handling its repeated item.
+ # When handling a `_Rep` or `_Cond` item, this function appends its
+ # effective multiplier/value to `instance_vals` _before_ handling
+ # its repeated/conditional item.
#
# When handling a `_VarAssign` item, this function only evaluates it
# if all its names are reachable.
item,
)
- # Add to repetition instance values
+ # Add to variable-length item instance values
instance_vals.append(val)
# Process the repeated item `val` times
for _ in range(val):
_Gen._compute_vl_instance_vals(item.item, state, instance_vals)
+ elif type(item) is _Cond:
+ # Evaluate the expression and keep the result
+ val = _Gen._eval_item_expr(item, state)
+
+ # Add to variable-length item instance values
+ instance_vals.append(val)
+
+ # Process the conditional item if needed
+ if val:
+ _Gen._compute_vl_instance_vals(item.item, state, instance_vals)
elif type(item) is _Group:
prev_labels = state.labels.copy()
return next_vl_instance
+ def _dry_handle_cond_item(
+ self, item: _Cond, state: _GenState, next_vl_instance: int
+ ):
+ # Get the value from `self._vl_instance_vals` _before_
+ # incrementing `next_vl_instance` to honor the order of
+ # _compute_vl_instance_vals().
+ val = self._vl_instance_vals[next_vl_instance]
+ next_vl_instance += 1
+
+ if val:
+ next_vl_instance = self._dry_handle_item(item.item, state, next_vl_instance)
+
+ return next_vl_instance
+
def _dry_handle_align_offset_item(
self, item: _AlignOffset, state: _GenState, next_vl_instance: int
):
return next_vl_instance
+ # Handles the conditional item `item`.
+ def _handle_cond_item(self, item: _Rep, state: _GenState, next_vl_instance: int):
+ # Get the precomputed conditional value
+ val = self._vl_instance_vals[next_vl_instance]
+
+ # Consumed this instance
+ next_vl_instance += 1
+
+ if val:
+ next_vl_instance = self._handle_item(item.item, state, next_vl_instance)
+
+ return next_vl_instance
+
# Handles the offset setting item `item`.
def _handle_set_offset_item(
self, item: _SetOffset, state: _GenState, next_vl_instance: int
self._item_handlers = {
_AlignOffset: self._handle_align_offset_item,
_Byte: self._handle_byte_item,
+ _Cond: self._handle_cond_item,
_FlNum: self._handle_fl_num_item,
_Group: self._handle_group_item,
_Label: self._handle_label_item,
self._dry_handle_item_funcs = {
_AlignOffset: self._dry_handle_align_offset_item,
_Byte: self._dry_handle_scalar_item,
+ _Cond: self._dry_handle_cond_item,
_FlNum: self._dry_handle_scalar_item,
_Group: self._dry_handle_group_item,
_Label: self._update_offset_noop,
projects / normand.git / commitdiff
