This package offers both a portable {py3} module and a command-line
tool.
-WARNING: This version of Normand is 0.5, meaning both the Normand
+WARNING: This version of Normand is 0.6, meaning both the Normand
language and the module/CLI interface aren't stable.
ifdef::env-github[]
The value of a variable assignment is the evaluation of a valid {py3}
expression which may include label and variable names.
-Fixed-length integer with a given length (8{nbsp}bits to 64{nbsp}bits) and byte order::
+Fixed-length number with a given length (8{nbsp}bits to 64{nbsp}bits) and byte order::
+
Input:
+
{strength = 4}
{be} 67 <lbl> 44 $178 {(end - lbl) * 8 + strength : 16} $99 <end>
{le} {-1993 : 32}
+{-3.141593 : 64}
----
+
Output:
+
----
-67 44 b2 00 2c 63 37 f8 ff ff
+67 44 b2 00 2c 63 37 f8 ff ff 7f bd c2 82 fb 21
+09 c0
----
+
-The encoded integer is the evaluation of a valid {py3} expression which
+The encoded number is the evaluation of a valid {py3} expression which
may include label and variable names.
https://en.wikipedia.org/wiki/LEB128[LEB128] integer::
|[[cur-offset]] Current offset
|
The current offset has an effect on the value of <<label,labels>> and of
-the special `ICITTE` name in <<fixed-length-integer,fixed-length
-integer>>, <<leb-128-integer,LEB128 integer>>, and
+the special `ICITTE` name in <<fixed-length-number,fixed-length
+number>>, <<leb-128-integer,LEB128 integer>>, and
<<variable-assignment,variable assignment>> expression evaluation.
Each generated byte increments the current offset.
|[[cur-bo]] Current byte order
|
The current byte order has an effect on the encoding of
-<<fixed-length-integer,fixed-length integers>>.
+<<fixed-length-number,fixed-length numbers>>.
A <<current-byte-order-setting,current byte order setting>> may change
the current byte order.
* A <<current-byte-order-setting,current byte order setting>> (big or
little endian).
-* A <<fixed-length-integer,fixed-length integer>> using the
- <<cur-bo,current byte order>> and of which the value is the result of
- a {py3} expression.
+* A <<fixed-length-number,fixed-length number>> (integer or
+ floating point) using the <<cur-bo,current byte order>> and of which
+ the value is the result of a {py3} expression.
* An <<leb128-integer,LEB128 integer>> of which the value is the result
of a {py3} expression.
``pass:[{be}]``:: Set the current byte order to big endian.
``pass:[{le}]``:: Set the current byte order to little endian.
-=== Fixed-length integer
+=== Fixed-length number
-A _fixed-length integer_ represents a fixed number of bytes encoding an
-unsigned or signed integer which is the result of evaluating a {py3}
-expression using the <<cur-bo,current byte order>>.
+A _fixed-length number_ represents a fixed number of bytes encoding
+either:
+
+* An unsigned or signed integer (two's complement).
++
+The available lengths are 8, 16, 24, 32, 40, 48, 56, and 64.
+
+* A floating point number
+ ([IEEE{nbsp}754-2008[https://standards.ieee.org/standard/754-2008.html]).
++
+The available length are 32 (_binary32_) and 64 (_binary64_).
-A fixed-length integer is:
+The value is the result of evaluating a {py3} expression using the
+<<cur-bo,current byte order>>.
+
+A fixed-length number is:
. The ``pass:[{]`` prefix.
. A valid {py3} expression.
+
-For a fixed-length integer at some source location{nbsp}__**L**__, this
+For a fixed-length number at some source location{nbsp}__**L**__, this
expression may contain the name of any accessible <<label,label>> (not
within a nested group), including the name of a label defined
after{nbsp}__**L**__, as well as the name of any
<<variable-assignment,variable>> known at{nbsp}__**L**__.
+
-The value of the special name `ICITTE` in this expression is the
-<<cur-offset,current offset>> (before encoding the integer).
+The value of the special name `ICITTE` (`int` type) in this expression
+is the <<cur-offset,current offset>> (before encoding the number).
. The `:` character.
-. An encoding length in bits amongst `8`, `16`, `24`, `32`, `40`,
- `48`, `56`, and `64`.
+. An encoding length in bits amongst:
++
+--
+The expression evaluates to an `int` value::
+ `8`, `16`, `24`, `32`, `40`, `48`, `56`, and `64`.
+
+The expression evaluates to a `float` value::
+ `32` and `64`.
+--
. The `}` suffix.
----
====
+====
+Input:
+
+----
+{le}
+{2 * 0.0529 : 32}
+----
+
+Output:
+
+----
+ac ad d8 3d
+----
+====
+
=== LEB128 integer
An _LEB128 integer_ represents a variable number of bytes encoding an
defined after{nbsp}__**L**__.
--
+
-The value of the special name `ICITTE` in this expression is the
-<<cur-offset,current offset>> (before encoding the integer).
+The value of the special name `ICITTE` (`int` type) in this expression
+is the <<cur-offset,current offset>> (before encoding the integer).
. The `:` character.
. The `<` prefix.
. A valid {py3} name which is not `ICITTE` (see
- <<fixed-length-integer>>, <<leb128-integer>>, and
+ <<fixed-length-number>>, <<leb128-integer>>, and
<<variable-assignment>> to learn more).
. The `>` suffix.
. The ``pass:[{]`` prefix.
. A valid {py3} name which is not `ICITTE` (see
- <<fixed-length-integer>>, <<leb128-integer>>, and
+ <<fixed-length-number>>, <<leb128-integer>>, and
<<variable-assignment>> to learn more).
. The `=` character.
after{nbsp}__**L**__, as well as the name of any
<<variable-assignment,variable>> known at{nbsp}__**L**__.
+
-The value of the special name `ICITTE` in this expression is the
-<<cur-offset,current offset>>.
+The value of the special name `ICITTE` (`int` type) in this expression
+is the <<cur-offset,current offset>>.
. The `}` suffix.
# Upstream repository: <https://github.com/efficios/normand>.
__author__ = "Philippe Proulx"
-__version__ = "0.5.0"
+__version__ = "0.6.0"
__all__ = [
"ByteOrder",
"parse",
)
-# Fixed-length integer, possibly needing more than one byte.
-class _FlInt(_ScalarItem, _RepableItem, _ExprMixin):
+# Fixed-length number, possibly needing more than one byte.
+class _FlNum(_ScalarItem, _RepableItem, _ExprMixin):
def __init__(
self, expr_str: str, expr: ast.Expression, len: int, text_loc: TextLoc
):
return self._len // 8
def __repr__(self):
- return "_FlInt({}, {}, {}, {})".format(
+ return "_FlNum({}, {}, {}, {})".format(
repr(self._expr_str), repr(self._expr), repr(self._len), self._text_loc
)
# Expression item type.
-_ExprItemT = Union[_FlInt, _Leb128Int, _VarAssign, _Rep]
+_ExprItemT = Union[_FlNum, _Leb128Int, _VarAssign, _Rep]
# A parsing error containing a message and a text location.
return expr_str, expr
- # Patterns for _try_parse_val_and_attr()
+ # Patterns for _try_parse_num_and_attr()
_val_expr_pat = re.compile(r"([^}:]+):\s*")
- _fl_int_len_attr_pat = re.compile(r"8|16|24|32|40|48|56|64")
+ _fl_num_len_attr_pat = re.compile(r"8|16|24|32|40|48|56|64")
_leb128_int_attr_pat = re.compile(r"(u|s)leb128")
# Tries to parse a value and attribute (fixed length in bits or
# `leb128`), returning a value item on success.
- def _try_parse_val_and_attr(self):
+ def _try_parse_num_and_attr(self):
begin_text_loc = self._text_loc
# Match
expr_str, expr = self._ast_expr_from_str(m_expr.group(1), begin_text_loc)
# Length?
- m_attr = self._try_parse_pat(self._fl_int_len_attr_pat)
+ m_attr = self._try_parse_pat(self._fl_num_len_attr_pat)
if m_attr is None:
# LEB128?
cls = _ULeb128Int if m_attr.group(1) == "u" else _SLeb128Int
return cls(expr_str, expr, begin_text_loc)
else:
- # Return fixed-length integer item
- return _FlInt(
+ # Return fixed-length number item
+ return _FlNum(
expr_str,
expr,
int(m_attr.group(0)),
begin_text_loc,
)
- # Patterns for _try_parse_val_and_attr()
+ # Patterns for _try_parse_num_and_attr()
_var_assign_pat = re.compile(
r"(?P<name>{})\s*=\s*(?P<expr>[^}}]+)".format(_py_name_pat.pattern)
)
item = self._try_parse_var_assign()
if item is None:
- # Fixed-length value item?
- item = self._try_parse_val_and_attr()
+ # Number item?
+ item = self._try_parse_num_and_attr()
if item is None:
# Byte order setting item?
if item is None:
# At this point it's invalid
self._raise_error(
- "Expecting a fixed-length integer, a variable assignment, or a byte order setting"
+ "Expecting a fixed-length number, a variable assignment, or a byte order setting"
)
# Expect suffix
#
# If `allow_icitte` is `True`, then the `ICITTE` name is available
# for the expression to evaluate.
+ #
+ # If `allow_float` is `True`, then the type of the result may be
+ # `float` too.
@staticmethod
- def _eval_item_expr(item: _ExprItemT, state: _GenState, allow_icitte: bool):
+ def _eval_item_expr(
+ item: _ExprItemT,
+ state: _GenState,
+ allow_icitte: bool,
+ allow_float: bool = False,
+ ):
syms = state.labels.copy()
# Set the `ICITTE` name to the current offset, if any
item,
)
- # Validate result
- if type(val) is not int:
+ # Validate result type
+ expected_types = {int} # type: Set[type]
+ type_msg = "`int`"
+
+ if allow_float:
+ expected_types.add(float)
+ type_msg += " or `float`"
+
+ if type(val) not in expected_types:
_raise_error_for_item(
- "Invalid expression `{}`: expecting result type `int`, not `{}`".format(
- item.expr_str, type(val).__name__
+ "Invalid expression `{}`: expecting result type {}, not `{}`".format(
+ item.expr_str, type_msg, type(val).__name__
),
item,
)
if do_eval:
# Evaluate the expression and keep the result
- state.variables[item.name] = _Gen._eval_item_expr(item, state, True)
+ state.variables[item.name] = _Gen._eval_item_expr(
+ item, state, True, True
+ )
elif type(item) is _SetOffset:
state.offset = item.val
elif isinstance(item, _Leb128Int):
return next_vl_instance
# Handles the fixed-length integer item `item`.
- def _handle_fl_int_item(
- self, item: _FlInt, state: _GenState, next_vl_instance: int
- ):
- # Compute value
- val = self._eval_item_expr(item, state, True)
-
+ def _handle_fl_int_item(self, val: int, item: _FlNum, state: _GenState):
# Validate range
if val < -(2 ** (item.len - 1)) or val > 2**item.len - 1:
_raise_error_for_item(
# Encode result on 64 bits (to extend the sign bit whatever the
# value of `item.len`).
- if state.bo is None and item.len > 8:
- _raise_error_for_item(
- "Current byte order isn't defined at first value (`{}`) to encode on more than 8 bits".format(
- item.expr_str
- ),
- item,
- )
-
data = struct.pack(
"{}{}".format(
">" if state.bo in (None, ByteOrder.BE) else "<",
# Append to current bytes and update offset
self._data += data
- state.offset += len(data)
+
+ # Handles the fixed-length integer item `item`.
+ def _handle_fl_float_item(self, val: float, item: _FlNum, state: _GenState):
+ # Validate length
+ if item.len not in (32, 64):
+ _raise_error_for_item(
+ "Invalid {}-bit length for a fixed-length floating point number (value {:,})".format(
+ item.len, val
+ ),
+ item,
+ )
+
+ # Encode result
+ self._data += struct.pack(
+ "{}{}".format(
+ ">" if state.bo in (None, ByteOrder.BE) else "<",
+ "f" if item.len == 32 else "d",
+ ),
+ val,
+ )
+
+ # Handles the fixed-length number item `item`.
+ def _handle_fl_num_item(
+ self, item: _FlNum, state: _GenState, next_vl_instance: int
+ ):
+ # Compute value
+ val = self._eval_item_expr(item, state, True, True)
+
+ # Validate current byte order
+ if state.bo is None and item.len > 8:
+ _raise_error_for_item(
+ "Current byte order isn't defined at first fixed-length number (`{}`) to encode on more than 8 bits".format(
+ item.expr_str
+ ),
+ item,
+ )
+
+ # Handle depending on type
+ if type(val) is int:
+ self._handle_fl_int_item(val, item, state)
+ else:
+ assert type(val) is float
+ self._handle_fl_float_item(val, item, state)
+
+ # Update offset
+ state.offset += item.size
+
return next_vl_instance
# Handles the LEB128 integer item `item`.
# Item handlers
self._item_handlers = {
_Byte: self._handle_byte_item,
- _FlInt: self._handle_fl_int_item,
+ _FlNum: self._handle_fl_num_item,
_Group: self._handle_group_item,
_Label: self._handle_label_item,
_Rep: self._handle_rep_item,
# Item size getters
self._item_size_funcs = {
_Byte: self._scalar_item_size,
- _FlInt: self._scalar_item_size,
+ _FlNum: self._scalar_item_size,
_Group: self._group_item_size,
_Label: self._zero_item_size,
_Rep: self._rep_item_size,
[tool.poetry]
name = 'normand'
-version = '0.5.0'
+version = '0.6.0'
description = 'Text-to-binary processor with its own language'
license = 'MIT'
authors = ['Philippe Proulx <eeppeliteloop@gmail.com>']
--- /dev/null
+{be}
+{ 18.49 : 16 }
+---
+2:3 - Invalid 16-bit length for a fixed-length floating point number (value 18.49)
+{be}
"yoyo"
{ 2**32 : 32 }
---
-2:3 - Value 4,294,967,296 is outside the 32-bit range when evaluating expression `2**32` at byte offset 4
+3:3 - Value 4,294,967,296 is outside the 32-bit range when evaluating expression `2**32` at byte offset 4
+++ /dev/null
-{ : 8 }
----
-1:3 - Expecting a fixed-length integer, a variable assignment, or a byte order setting
+++ /dev/null
-{ 'salut' : 8 }
----
-1:3 - Invalid expression `'salut'`: expecting result type `int`, not `str`
+++ /dev/null
-{ 23 + zoom(14) : 8 }
----
-1:3 - Failed to evaluate expression `23 + zoom(14)`: name 'zoom' is not defined
+++ /dev/null
-{ 23 : 17 }
----
-1:8 - Expecting a length (multiple of eight bits), `uleb128`, or `sleb128`
+++ /dev/null
-{ 19 + / 23 : 8 }
----
-1:3 - Invalid expression `19 + / 23`: invalid syntax
+++ /dev/null
-{ 34 : 16 }
----
-1:3 - Current byte order isn't defined at first value (`34`) to encode on more than 8 bits
+++ /dev/null
-{ 23 : }
----
-1:9 - Expecting a length (multiple of eight bits), `uleb128`, or `sleb128`
+++ /dev/null
-{ meow : 8 }
----
-1:3 - Illegal (unknown or unreachable) variable/label name `meow` in expression `meow`; the legal names are {`ICITTE`}
+++ /dev/null
-{ meow : 8 }
-{ meow = 34 }
----
-1:3 - Illegal (unknown or unreachable) variable/label name `meow` in expression `meow`; the legal names are {`ICITTE`}
--- /dev/null
+{ : 8 }
+---
+1:3 - Expecting a fixed-length number, a variable assignment, or a byte order setting
--- /dev/null
+{ 'salut' : 8 }
+---
+1:3 - Invalid expression `'salut'`: expecting result type `int` or `float`, not `str`
--- /dev/null
+{ 23 + zoom(14) : 8 }
+---
+1:3 - Failed to evaluate expression `23 + zoom(14)`: name 'zoom' is not defined
--- /dev/null
+{ 23 : 17 }
+---
+1:8 - Expecting a length (multiple of eight bits), `uleb128`, or `sleb128`
--- /dev/null
+{ 19 + / 23 : 8 }
+---
+1:3 - Invalid expression `19 + / 23`: invalid syntax
--- /dev/null
+{ 34 : 16 }
+---
+1:3 - Current byte order isn't defined at first fixed-length number (`34`) to encode on more than 8 bits
--- /dev/null
+{ 23 : }
+---
+1:9 - Expecting a length (multiple of eight bits), `uleb128`, or `sleb128`
--- /dev/null
+{ meow : 8 }
+---
+1:3 - Illegal (unknown or unreachable) variable/label name `meow` in expression `meow`; the legal names are {`ICITTE`}
--- /dev/null
+{ meow : 8 }
+{ meow = 34 }
+---
+1:3 - Illegal (unknown or unreachable) variable/label name `meow` in expression `meow`; the legal names are {`ICITTE`}
--- /dev/null
+77 * { [1, 2, 3] }
+---
+1:4 - Invalid expression `[1, 2, 3]`: expecting result type `int`, not `list`
--- /dev/null
+77 * { 3.4 }
+---
+1:4 - Invalid expression `3.4`: expecting result type `int`, not `float`
+++ /dev/null
-77 * { [1, 2, 3] }
----
-1:4 - Invalid expression `[1, 2, 3]`: expecting result type `int`, not `list`
{ : sleb128 }
---
-1:3 - Expecting a fixed-length integer, a variable assignment, or a byte order setting
+1:3 - Expecting a fixed-length number, a variable assignment, or a byte order setting
{ : uleb128 }
---
-1:3 - Expecting a fixed-length integer, a variable assignment, or a byte order setting
+1:3 - Expecting a fixed-length number, a variable assignment, or a byte order setting
--- /dev/null
+{ 'salut' : uleb128 }
+---
+1:3 - Invalid expression `'salut'`: expecting result type `int`, not `str`
--- /dev/null
+{ 45.2 : uleb128 }
+---
+1:3 - Invalid expression `45.2`: expecting result type `int`, not `float`
+++ /dev/null
-{ 'salut' : uleb128 }
----
-1:3 - Invalid expression `'salut'`: expecting result type `int`, not `str`
{ meow = 'salut' }
---
-1:3 - Invalid expression `'salut'`: expecting result type `int`, not `str`
+1:3 - Invalid expression `'salut'`: expecting result type `int` or `float`, not `str`
{ hello }
---
-1:3 - Expecting a fixed-length integer, a variable assignment, or a byte order setting
+1:3 - Expecting a fixed-length number, a variable assignment, or a byte order setting
--- /dev/null
+{be}
+{2.99792458e8:32}
+{-1.6021766208:32}
+---
+4d 8e f3 c2
+bf cd 14 20
--- /dev/null
+{le}
+{2.99792458e8:32}
+{-1.6021766208:32}
+---
+c2 f3 8e 4d
+20 14 cd bf
--- /dev/null
+{be}
+{2.99792458e8:64}
+{-1.6021766208:64}
+---
+41 b1 de 78 4a 00 00 00
+bf f9 a2 83 f3 cc 07 58
--- /dev/null
+{le}
+{2.99792458e8:64}
+{-1.6021766208:64}
+---
+00 00 00 4a 78 de b1 41
+58 07 cc f3 83 a2 f9 bf
--- /dev/null
+{strength = 4}
+{be} 67 <lbl> 44 $178 {(end - lbl) * 8 + strength : 16} $99 <end>
+{le} {-1993 : 32}
+{-3.141593 : 64}
+---
+67 44 b2 00 2c 63 37 f8 ff ff 7f bd c2 82 fb 21
+09 c0
+++ /dev/null
-{le} {345:16}
-{be} {-0xabcd:32}
----
-59 01 ff ff 54 33
+++ /dev/null
-{be}
-
-# String length in bits
-{8 * (str_end - str_beg) : 16}
-
-# String
-<str_beg>
- "hello world!"
-<str_end>
----
-00 60 68 65 6c 6c 6f 20 77 6f 72 6c 64 21
+++ /dev/null
-{20 - ICITTE : 8} * 10
----
-14 13 12 11 10 0f 0e 0d 0c 0b
--- /dev/null
+{le} {345:16}
+{be} {-0xabcd:32}
+---
+59 01 ff ff 54 33
--- /dev/null
+{be}
+
+# String length in bits
+{8 * (str_end - str_beg) : 16}
+
+# String
+<str_beg>
+ "hello world!"
+<str_end>
+---
+00 60 68 65 6c 6c 6f 20 77 6f 72 6c 64 21
--- /dev/null
+{20 - ICITTE : 8} * 10
+---
+14 13 12 11 10 0f 0e 0d 0c 0b
--- /dev/null
+{le}
+{2 * 0.0529 : 32}
+---
+ac ad d8 3d
projects / normand.git / commitdiff
