This package offers both a portable {py3} module and a command-line
tool.
-WARNING: This version of Normand is 0.11, meaning both the Normand
+WARNING: This version of Normand is 0.12, meaning both the Normand
language and the module/CLI interface aren't stable.
ifdef::env-github[]
ff 85 ff ff 00 00 15 d0
----
+Filling::
++
+Input:
++
+----
+{le}
+{0xdeadbeef:32}
+{-1993:16}
+{9:16}
++0x40
+{ICITTE:8}
+"meow mix"
++200~0xff
+{ICITTE:8}
+----
++
+Output:
++
+----
+ef be ad de 37 f8 09 00 00 00 00 00 00 00 00 00 ┆ ••••7•••••••••••
+00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ┆ ••••••••••••••••
+00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ┆ ••••••••••••••••
+00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ┆ ••••••••••••••••
+40 6d 65 6f 77 20 6d 69 78 ff ff ff ff ff ff ff ┆ @meow mix•••••••
+ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ┆ ••••••••••••••••
+ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ┆ ••••••••••••••••
+ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ┆ ••••••••••••••••
+ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ┆ ••••••••••••••••
+ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ┆ ••••••••••••••••
+ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ┆ ••••••••••••••••
+ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ┆ ••••••••••••••••
+ff ff ff ff ff ff ff ff c8 ┆ •••••••••
+----
+
Multilevel grouping::
+
Input:
* A <<current-offset-alignment,current offset alignment>>.
+* A <<filling,filling>>.
+
* A <<label,label>>, that is, a named constant holding the current
offset.
+
comment may exist:
----
-/ \ ? & : ; . , + [ ] _ = | -
+/ \ ? & : ; . , [ ] _ = | -
----
The latter serve to improve readability so that you may write, for
----
====
+=== Filling
+
+A _filling_ represents zero or more padding bytes to make the
+<<cur-offset,current offset>> reach a given value.
+
+A filling is:
+
+. The ``pass:[+]`` prefix.
+
+. One of:
+
+** A positive integer (hexadecimal starting with `0x` or `0X` accepted)
+ which is the current offset target.
+
+** The ``pass:[{]`` prefix, a valid {py3} expression of which the
+ evaluation result type is `int` or `bool` (automatically converted to
+ `int`), and the ``pass:[}]`` suffix.
++
+For a filling at some source location{nbsp}__**L**__, this expression
+may contain:
++
+--
+* The name of any <<label,label>> defined before{nbsp}__**L**__
+ which isn't within a nested group.
+* The name of any <<variable-assignment,variable>> known
+ at{nbsp}__**L**__.
+--
++
+The value of the special name `ICITTE` (`int` type) in this expression
+is the <<cur-offset,current offset>> (before handling the items to
+repeat).
+
+** A valid {py3} name.
++
+For the name `__NAME__`, this is equivalent to the
+`pass:[{]__NAME__pass:[}]` form above.
+
++
+This value must be greater than or equal to the current offset where
+it's used.
+
+. **Optional**:
++
+--
+. The ``pass:[~]`` prefix.
+. A positive integer (hexadecimal starting with `0x` or `0X` accepted)
+ which is the value of the byte to use as padding to reach the
+ current offset target.
+--
++
+Without this section, the padding byte value is zero.
+
+====
+Input:
+
+----
+aa bb cc dd
++0x40
+"hello world"
+----
+
+Output:
+
+----
+aa bb cc dd 00 00 00 00 00 00 00 00 00 00 00 00 ┆ ••••••••••••••••
+00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ┆ ••••••••••••••••
+00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ┆ ••••••••••••••••
+00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ┆ ••••••••••••••••
+68 65 6c 6c 6f 20 77 6f 72 6c 64 ┆ hello world
+----
+====
+
+====
+Input:
+
+----
+!macro part(iter, fill)
+ <0> "particular security " {ord('0') + iter : 8} +fill~0x80
+!end
+
+{iter = 1}
+
+!repeat 5
+ m:part(iter, {32 + 4 * iter})
+ {iter = iter + 1}
+!end
+----
+
+Output:
+
+----
+70 61 72 74 69 63 75 6c 61 72 20 73 65 63 75 72 ┆ particular secur
+69 74 79 20 31 80 80 80 80 80 80 80 80 80 80 80 ┆ ity 1•••••••••••
+80 80 80 80 70 61 72 74 69 63 75 6c 61 72 20 73 ┆ ••••particular s
+65 63 75 72 69 74 79 20 32 80 80 80 80 80 80 80 ┆ ecurity 2•••••••
+80 80 80 80 80 80 80 80 80 80 80 80 70 61 72 74 ┆ ••••••••••••part
+69 63 75 6c 61 72 20 73 65 63 75 72 69 74 79 20 ┆ icular security
+33 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 ┆ 3•••••••••••••••
+80 80 80 80 80 80 80 80 70 61 72 74 69 63 75 6c ┆ ••••••••particul
+61 72 20 73 65 63 75 72 69 74 79 20 34 80 80 80 ┆ ar security 4•••
+80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 ┆ ••••••••••••••••
+80 80 80 80 80 80 80 80 70 61 72 74 69 63 75 6c ┆ ••••••••particul
+61 72 20 73 65 63 75 72 69 74 79 20 35 80 80 80 ┆ ar security 5•••
+80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 ┆ ••••••••••••••••
+80 80 80 80 80 80 80 80 80 80 80 80 ┆ ••••••••••••
+----
+====
+
=== Label
A _label_ associates a name to the <<cur-offset,current offset>>.
# Upstream repository: <https://github.com/efficios/normand>.
__author__ = "Philippe Proulx"
-__version__ = "0.11.0"
+__version__ = "0.12.0"
__all__ = [
"__author__",
"__version__",
return self._expr
+# Fill until some offset.
+class _FillUntil(_Item, _ExprMixin):
+ def __init__(
+ self, expr_str: str, expr: ast.Expression, pad_val: int, text_loc: TextLocation
+ ):
+ super().__init__(text_loc)
+ _ExprMixin.__init__(self, expr_str, expr)
+ self._pad_val = pad_val
+
+ # Padding byte value.
+ @property
+ def pad_val(self):
+ return self._pad_val
+
+ def __repr__(self):
+ return "_FillUntil({}, {}, {}, {})".format(
+ repr(self._expr_str),
+ repr(self._expr),
+ repr(self._pad_val),
+ repr(self._text_loc),
+ )
+
+
# Variable assignment.
class _VarAssign(_Item, _ExprMixin):
def __init__(
# Pattern for _skip_ws_and_comments()
_ws_or_syms_or_comments_pat = re.compile(
- r"(?:[\s/\\?&:;.,+[\]_=|-]|#[^#]*?(?:\n|#))*"
+ r"(?:[\s/\\?&:;.,[\]_=|-]|#[^#]*?(?:\n|#))*"
)
# Skips as many whitespaces, insignificant symbol characters, and
self._expect_pat(self._label_set_offset_suffix_pat, "Expecting `>`")
return item
+ # Pattern for _parse_pad_val()
+ _pad_val_prefix_pat = re.compile(r"~")
+
+ # Tries to parse a padding value, returning the padding value, or 0
+ # if none.
+ def _parse_pad_val(self):
+ # Padding value?
+ self._skip_ws()
+ pad_val = 0
+
+ if self._try_parse_pat(self._pad_val_prefix_pat) is not None:
+ self._skip_ws()
+ pad_val_text_loc = self._text_loc
+ m = self._expect_pat(
+ self._pos_const_int_pat,
+ "Expecting a positive constant integer (byte value)",
+ )
+
+ # Validate
+ pad_val = int(m.group(0), 0)
+
+ if pad_val > 255:
+ _raise_error(
+ "Invalid padding byte value {}".format(pad_val),
+ pad_val_text_loc,
+ )
+
+ return pad_val
+
# Patterns for _try_parse_align_offset()
_align_offset_prefix_pat = re.compile(r"@")
_align_offset_val_pat = re.compile(r"\d+")
- _align_offset_pad_val_prefix_pat = re.compile(r"~")
# Tries to parse an offset alignment, returning an offset alignment
# item on success.
# No match
return
- self._skip_ws()
-
# Expect an alignment
+ self._skip_ws()
align_text_loc = self._text_loc
m = self._expect_pat(
self._align_offset_val_pat,
align_text_loc,
)
- # Padding value?
- self._skip_ws()
- pad_val = 0
+ # Padding value
+ pad_val = self._parse_pad_val()
- if self._try_parse_pat(self._align_offset_pad_val_prefix_pat) is not None:
- self._skip_ws()
- pad_val_text_loc = self._text_loc
- m = self._expect_pat(self._pos_const_int_pat, "Expecting a byte value")
+ # Return item
+ return _AlignOffset(val, pad_val, begin_text_loc)
- # Validate
- pad_val = int(m.group(0), 0)
+ # Patterns for _try_parse_fill_until()
+ _fill_until_prefix_pat = re.compile(r"\+")
+ _fill_until_pad_val_prefix_pat = re.compile(r"~")
- if pad_val > 255:
- _raise_error(
- "Invalid padding byte value {}".format(pad_val),
- pad_val_text_loc,
- )
+ # Tries to parse a filling, returning a filling item on success.
+ def _try_parse_fill_until(self):
+ begin_text_loc = self._text_loc
+
+ # Match prefix
+ if self._try_parse_pat(self._fill_until_prefix_pat) is None:
+ # No match
+ return
+
+ # Expect expression
+ self._skip_ws()
+ expr_str, expr = self._expect_const_int_name_expr(True)
+
+ # Padding value
+ pad_val = self._parse_pad_val()
# Return item
- return _AlignOffset(val, pad_val, begin_text_loc)
+ return _FillUntil(expr_str, expr, pad_val, begin_text_loc)
# Patterns for _expect_rep_mul_expr()
_inner_expr_prefix_pat = re.compile(r"\{")
# Offset alignment item?
item = self._try_parse_align_offset()
+ if item is not None:
+ return item
+
+ # Filling item?
+ item = self._try_parse_fill_until()
+
if item is not None:
return item
# `float` too.
@staticmethod
def _eval_item_expr(
- item: Union[_FlNum, _Leb128Int, _VarAssign, _Rep, _Cond],
+ item: Union[_FlNum, _Leb128Int, _FillUntil, _VarAssign, _Rep, _Cond],
state: _GenState,
allow_float: bool = False,
):
def _handle_set_offset_item(self, item: _SetOffset, state: _GenState):
state.offset = item.val
- # Handles offset alignment item `item` (adds padding).
+ # Handles the offset alignment item `item` (adds padding).
def _handle_align_offset_item(self, item: _AlignOffset, state: _GenState):
init_offset = state.offset
align_bytes = item.val // 8
state.offset = (state.offset + align_bytes - 1) // align_bytes * align_bytes
self._data += bytes([item.pad_val] * (state.offset - init_offset))
+ # Handles the filling item `item` (adds padding).
+ def _handle_fill_until_item(self, item: _FillUntil, state: _GenState):
+ # Compute the new offset
+ new_offset = _Gen._eval_item_expr(item, state)
+
+ # Validate the new offset
+ if new_offset < state.offset:
+ _raise_error_for_item(
+ "Invalid expression `{}`: new offset {:,} is less than current offset {:,}".format(
+ item.expr_str, new_offset, state.offset
+ ),
+ item,
+ )
+
+ # Fill
+ self._data += bytes([item.pad_val] * (new_offset - state.offset))
+
+ # Update offset
+ state.offset = new_offset
+
# Handles the label item `item`.
def _handle_label_item(self, item: _Label, state: _GenState):
state.labels[item.name] = state.offset
_AlignOffset: self._handle_align_offset_item,
_Byte: self._handle_byte_item,
_Cond: self._handle_cond_item,
+ _FillUntil: self._handle_fill_until_item,
_FlNum: self._handle_fl_num_item,
_Group: self._handle_group_item,
_Label: self._handle_label_item,
[tool.poetry]
name = 'normand'
-version = '0.11.0'
+version = '0.12.0'
description = 'Text-to-binary processor with its own language'
license = 'MIT'
authors = ['Philippe Proulx <eeppeliteloop@gmail.com>']
88 44 @16~-45 "meow"
@ 56 ~ 127 $132
---
-1:11 - Expecting a byte value
+1:11 - Expecting a positive constant integer (byte value)
88 44 @16~yo "meow"
@ 56 ~ 127 $132
---
-1:11 - Expecting a byte value
+1:11 - Expecting a positive constant integer (byte value)
--- /dev/null
+aa bb cc +2 "zoom"
+---
+1:10 - Invalid expression `2`: new offset 2 is less than current offset 3
--- /dev/null
+88 44 +16~-45 "meow"
++ 100 ~ 127 $132
+---
+1:11 - Expecting a positive constant integer (byte value)
--- /dev/null
+88 44 +16~yo "meow"
++ 56 ~ 127 $132
+---
+1:11 - Expecting a positive constant integer (byte value)
--- /dev/null
+88 44 +-16 "meow"
++ 56 ~ 127 $132
+---
+1:8 - Expecting a positive constant integer
--- /dev/null
+88 44 +"yo" "meow"
++ 56 ~ 127 $132
+---
+1:8 - Expecting a positive constant integer, a name, or `{`
--- /dev/null
+88 44 +16~256 "meow"
++ 56 ~ 127 $132
+---
+1:11 - Invalid padding byte value 256
--- /dev/null
+{zoom = 32}
+aa bb cc +yo "zoom"
+---
+2:10 - Illegal (unknown or unreachable) variable/label name `yo` in expression `yo`; the legal names are {`ICITTE`, `zoom`}
--- /dev/null
+{zoom = 32}
+aa bb cc +yo "zoom"
+{yo = 0x99}
+---
+2:10 - Illegal (unknown or unreachable) variable/label name `yo` in expression `yo`; the legal names are {`ICITTE`, `zoom`}
-%1/01\0?1&0:1;0 %.0,1+0[1]0_1=0|1 %1111-0000
+%1/01\0?1&0:1;0 %.0,10[1]0_1=0|1 %1111-0000
% 1 0 / 1 \ 0 ? 1 & 0 : 1 ; 0
-% . 0 , 1 + 0 [ 1 ] 0 _ 1 = 0 | 1
+% . 0 , 1 0 [ 1 ] 0 _ 1 = 0 | 1
% 1111 - 0000
---
aa 55 f0
-(012/3\4?5&6:7;8.9,a+b[c]d_e=f|0-1(/\?&:;.,+[]_=|-))
+(012/3\4?5&6:7;8.9,ab[c]d_e=f|0-1(/\?&:;.,[]_=|-))
---
01 23 45 67 89 ab cd ef 01
-/ \ ? & : ; . , + [ ] _ = | -
-012/3\4?5&6:7;8.9,a+b[c]d_e=f|0-1
-0 1 2 / 3 \ 4 ? 5 & 6 : 7 ; 8 . 9 , a + b [ c ] d_e = f | 0 - 1
+/ \ ? & : ; . , [ ] _ = | -
+012/3\4?5&6:7;8.9,ab[c]d_e=f|0-1
+0 1 2 / 3 \ 4 ? 5 & 6 : 7 ; 8 . 9 , a b [ c ] d_e = f | 0 - 1
---
01 23 45 67 89 ab cd ef 01
01 23 45 67 89 ab cd ef 01
# repetition
-ff/\?&:;.,+[]_=|-*/\?&:;.,+[]_=|-5
+ff/\?&:;.,[]_=|-*/\?&:;.,[]_=|-5
---
ff ff ff ff ff
--- /dev/null
+aa bb cc +{ICITTE} {ICITTE:8}
+---
+aa bb cc 03
--- /dev/null
+aa {zoom = 10} bb cc +zoom {ICITTE:8}
+---
+aa bb cc 00 00
+00 00 00 00 00
+0a
--- /dev/null
+aa {zoom = 10} bb cc +zoom~0xcc {ICITTE:8}
++ 15 ~ 255 {ICITTE:8}
+---
+aa bb cc cc cc
+cc cc cc cc cc
+0a ff ff ff ff
+0f
--- /dev/null
+aa bb cc +10 {ICITTE:8}
+---
+aa bb cc 00 00
+00 00 00 00 00
+0a
--- /dev/null
+{le}
+{0xdeadbeef:32}
+{-1993:16}
+{9:16}
++0x40
+{ICITTE:8}
+"meow mix"
++200~0xff
+{ICITTE:8}
+---
+ef be ad de 37 f8 09 00 00 00 00 00 00 00 00 00
+00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
+00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
+00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
+40 6d 65 6f 77 20 6d 69 78 ff ff ff ff ff ff ff
+ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
+ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
+ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
+ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
+ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
+ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
+ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
+ff ff ff ff ff ff ff ff c8
--- /dev/null
+aa bb cc dd
++0x40
+"hello world"
+---
+aa bb cc dd 00 00 00 00 00 00 00 00 00 00 00 00
+00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
+00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
+00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
+68 65 6c 6c 6f 20 77 6f 72 6c 64
--- /dev/null
+!macro part(iter, fill)
+ <0> "particular security " {ord('0') + iter : 8} +fill~0x80
+!end
+
+{iter = 1}
+
+!repeat 5
+ m:part(iter, {32 + 4 * iter})
+ {iter = iter + 1}
+!end
+---
+70 61 72 74 69 63 75 6c 61 72 20 73 65 63 75 72
+69 74 79 20 31 80 80 80 80 80 80 80 80 80 80 80
+80 80 80 80 70 61 72 74 69 63 75 6c 61 72 20 73
+65 63 75 72 69 74 79 20 32 80 80 80 80 80 80 80
+80 80 80 80 80 80 80 80 80 80 80 80 70 61 72 74
+69 63 75 6c 61 72 20 73 65 63 75 72 69 74 79 20
+33 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80
+80 80 80 80 80 80 80 80 70 61 72 74 69 63 75 6c
+61 72 20 73 65 63 75 72 69 74 79 20 34 80 80 80
+80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80
+80 80 80 80 80 80 80 80 70 61 72 74 69 63 75 6c
+61 72 20 73 65 63 75 72 69 74 79 20 35 80 80 80
+80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80
+80 80 80 80 80 80 80 80 80 80 80 80