1 // SPDX-FileCopyrightText: 2023 Philippe Proulx <eeppeliteloop@gmail.com>
2 // SPDX-License-Identifier: CC-BY-SA-4.0
4 // Show ToC at a specific location for a GitHub rendering
13 // This is to mimic what GitHub does so that anchors work in an offline
24 image::normand-logo.png[]
27 image:https://img.shields.io/pypi/v/normand.svg?label=Latest%20version[link="https://pypi.python.org/pypi/normand"]
30 _**Normand**_ is a text-to-binary processor with its own language.
32 This package offers both a portable {py3} module and a command-line
35 WARNING: This version of Normand is 0.23, meaning both the Normand
36 language and the module/CLI interface aren't stable.
39 // ToC location for a GitHub rendering
45 The purpose of Normand is to consume human-readable text representing
46 bytes and to produce the corresponding binary data.
50 Consider the following Normand input:
53 4f 55 32 bb $167 fe %10100111 a9 $-32
56 The generated nine bytes are:
59 4f 55 32 bb a7 fe a7 a9 e0
63 As you can see in the last example, the fundamental unit of the Normand
64 language is the _byte_. The order in which you list bytes will be the
65 order of the generated data.
67 The Normand language is more than simple lists of bytes, though. Its
70 Comments, including a bunch of insignificant symbols which may improve readability::
75 ff bb %1101:0010 # This is a comment
76 78 29 af $192 # This too # 99 $-80
77 fe80::6257:18ff:fea3:4229
79 10839636-5d65-4a68-8e6a-21608ddf7258
85 ff bb d2 78 29 af c0 99 b0 fe 80 62 57 18 ff fe
86 a3 42 29 60 57 18 a3 42 29 10 83 96 36 5d 65 4a
87 68 8e 6a 21 60 8d df 72 58
90 Hexadecimal, decimal, and binary byte constants::
95 aa bb $247 $-89 %0011_0010 %11.01= 10/10
110 u16le"stress\nverdict 🤣"
111 s:latin3{hex(ICITTE)}
117 68 65 6c 6c 6f 20 77 6f 72 6c 64 21 00 73 00 74 ┆ hello world!•s•t
118 00 72 00 65 00 73 00 73 00 0a 00 76 00 65 00 72 ┆ •r•e•s•s•••v•e•r
119 00 64 00 69 00 63 00 74 00 20 00 3e d8 23 dd 30 ┆ •d•i•c•t• •>•#•0
123 Labels: special variables holding the offset where they're defined::
126 <beg> b2 52 e3 bc 91 05
127 $100 $50 <chair> 33 9f fe
134 5e 65 {tower = 47} c6 7f f2 c4
135 44 {hurl = tower - 14} b5 {tower = hurl} 26 2d
138 The value of a variable assignment is the evaluation of a valid {py3}
139 expression which may include label and variable names.
141 Fixed-length number with a given length (8{nbsp}bits to 64{nbsp}bits) and byte order::
147 !be 67 <lbl> 44 $178 [(end - lbl) * 8 + strength : 16] $99 <end>
155 67 44 b2 00 2c 63 37 f8 ff ff c0 09 21 fb 82 c2
159 The encoded number is the evaluation of a valid {py3} expression which
160 may include label and variable names.
162 https://en.wikipedia.org/wiki/LEB128[LEB128] integer::
167 aa bb cc [-1993 : sleb128] <meow> dd ee ff
168 [meow * 199 : uleb128]
174 aa bb cc b7 70 dd ee ff e3 07
177 The encoded integer is the evaluation of a valid {py3} expression which
178 may include label and variable names.
201 aa bb cc 66 6f 6f 66 69 67 68 74 66 6f 6f 66 69 ┆ •••foofightfoofi
202 67 68 74 66 6f 6f 62 61 72 66 6f 6f 62 61 72 ┆ ghtfoobarfoobar
210 aa bb * 5 cc <zoom> "yeah\0" * {zoom * 3}
220 aa bb bb bb bb bb cc 79 65 61 68 00 79 65 61 68 ┆ •••••••yeah•yeah
221 00 79 65 61 68 00 79 65 61 68 00 79 65 61 68 00 ┆ •yeah•yeah•yeah•
222 79 65 61 68 00 79 65 61 68 00 79 65 61 68 00 79 ┆ yeah•yeah•yeah•y
223 65 61 68 00 79 65 61 68 00 79 65 61 68 00 79 65 ┆ eah•yeah•yeah•ye
224 61 68 00 79 65 61 68 00 79 65 61 68 00 79 65 61 ┆ ah•yeah•yeah•yea
225 68 00 79 65 61 68 00 79 65 61 68 00 79 65 61 68 ┆ h•yeah•yeah•yeah
226 00 79 65 61 68 00 79 65 61 68 00 79 65 61 68 00 ┆ •yeah•yeah•yeah•
227 ff ee 6a 75 69 63 65 ff ee 6a 75 69 63 65 ff ee ┆ ••juice••juice••
228 6a 75 69 63 65 ┆ juice
247 00 00 00 c7 00 00 00 00 00 00 00 00 00 00 00 2b
248 ff 85 ff ff 00 00 15 d0
270 ef be ad de 37 f8 09 00 00 00 00 00 00 00 00 00 ┆ ••••7•••••••••••
271 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ┆ ••••••••••••••••
272 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ┆ ••••••••••••••••
273 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ┆ ••••••••••••••••
274 40 6d 65 6f 77 20 6d 69 78 ff ff ff ff ff ff ff ┆ @meow mix•••••••
275 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ┆ ••••••••••••••••
276 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ┆ ••••••••••••••••
277 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ┆ ••••••••••••••••
278 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ┆ ••••••••••••••••
279 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ┆ ••••••••••••••••
280 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ┆ ••••••••••••••••
281 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ┆ ••••••••••••••••
282 ff ff ff ff ff ff ff ff c8 ┆ •••••••••
290 "end of file @ " [end:8]
293 "this part will be gzipped"
302 65 6e 64 20 6f 66 20 66 69 6c 65 20 40 20 3c 1f ┆ end of file @ <•
303 8b 08 00 7b 7b 26 65 02 ff 2b c9 c8 2c 56 28 48 ┆ •••{{&e••+••,V(H
304 2c 2a 51 28 cf cc c9 51 48 4a 55 48 af ca 2c 28 ┆ ,*Q(•••QHJUH••,(
305 48 4d 01 00 d4 cc 5b 8a 19 00 00 00 ┆ HM••••[•••••
308 Multilevel grouping::
313 ff ((aa bb "zoom" cc) * 5) * 3 $-34 * 4
319 ff aa bb 7a 6f 6f 6d cc aa bb 7a 6f 6f 6d cc aa ┆ •••zoom•••zoom••
320 bb 7a 6f 6f 6d cc aa bb 7a 6f 6f 6d cc aa bb 7a ┆ •zoom•••zoom•••z
321 6f 6f 6d cc aa bb 7a 6f 6f 6d cc aa bb 7a 6f 6f ┆ oom•••zoom•••zoo
322 6d cc aa bb 7a 6f 6f 6d cc aa bb 7a 6f 6f 6d cc ┆ m•••zoom•••zoom•
323 aa bb 7a 6f 6f 6d cc aa bb 7a 6f 6f 6d cc aa bb ┆ ••zoom•••zoom•••
324 7a 6f 6f 6d cc aa bb 7a 6f 6f 6d cc aa bb 7a 6f ┆ zoom•••zoom•••zo
325 6f 6d cc aa bb 7a 6f 6f 6d cc de de de de ┆ om•••zoom•••••
335 !if world " world" !end
340 m:hello({ICITTE > 15 and ICITTE < 60})
347 ff ff ff ff 68 65 6c 6c 6f ff ff ff ff 68 65 6c ┆ ••••hello••••hel
348 6c 6f ff ff ff ff 68 65 6c 6c 6f 20 77 6f 72 6c ┆ lo••••hello worl
349 64 ff ff ff ff 68 65 6c 6c 6f 20 77 6f 72 6c 64 ┆ d••••hello world
350 ff ff ff ff 68 65 6c 6c 6f 20 77 6f 72 6c 64 ff ┆ ••••hello world•
351 ff ff ff 68 65 6c 6c 6f ff ff ff ff 68 65 6c 6c ┆ •••hello••••hell
352 6f ff ff ff ff 68 65 6c 6c 6f ff ff ff ff 68 65 ┆ o••••hello••••he
353 6c 6c 6f ff ff ff ff 68 65 6c 6c 6f ff ff ff ff ┆ llo••••hello••••
354 68 65 6c 6c 6f ff ff ff ff 68 65 6c 6c 6f ff ff ┆ hello••••hello••
355 ff ff 68 65 6c 6c 6f ff ff ff ff 68 65 6c 6c 6f ┆ ••hello••••hello
356 ff ff ff ff 68 65 6c 6c 6f ff ff ff ff 68 65 6c ┆ ••••hello••••hel
357 6c 6f ff ff ff ff 68 65 6c 6c 6f ┆ lo••••hello
360 Precise error reporting::
363 /tmp/meow.normand:10:24 - Expecting a bit (`0` or `1`).
367 /tmp/meow.normand:32:6 - Unexpected character `k`.
371 /tmp/meow.normand:24:19 - Illegal (unknown or unreachable) variable/label name `meow` in expression `(meow - 45) // 8`; the legal names are {`ICITTE`, `mix`, `zoom`}.
375 /tmp/meow.normand:32:19 - While expanding the macro `meow`:
376 /tmp/meow.normand:35:5 - While expanding the macro `zzz`:
377 /tmp/meow.normand:18:9 - Value 315 is outside the 8-bit range when evaluating expression `end - ICITTE`.
380 You can use Normand to track data source files in your favorite VCS
381 instead of raw binary files. The binary files that Normand generates can
382 be used to test file format decoding, including malformatted data, for
383 example, as well as for education.
385 See <<learn-normand>> to explore all the Normand features.
389 Normand requires Python ≥ 3.4.
394 $ python3 -m pip install --user normand
398 https://packaging.python.org/en/latest/tutorials/installing-packages/#installing-to-the-user-site[Installing to the User Site]
399 to learn more about a user site installation.
403 Normand has a single module file, `normand.py`, which you can copy as is
404 to your project to use it (both the <<python3-api,`normand.parse()`>>
405 function and the <<command-line-tool,command-line tool>>).
407 `normand.py` has _no external dependencies_, but if you're using
408 Python{nbsp}3.4 or Python{nbsp}3.5, you'll need a local copy of the
409 standard `typing` module.
414 The design goals of Normand are:
417 We're making sure `normand.py` works with Python{nbsp}≥{nbsp}3.4 and
418 doesn't have any external dependencies so that you may just copy the
419 module as is to your own project.
422 The most basic Normand input is a sequence of hexadecimal constants
423 (for example, `4e6f726d616e64`) which produce exactly what you'd
426 Most Normand features map to programming language concepts you already
427 know and understand: constant integers, literal strings, variables,
428 conditionals, repetitions/loops, and the rest.
430 Concise and readable input::
431 We could have chosen XML or YAML as the input format, but having a
432 DSL here makes a Normand input compact and easy to read, two
433 important traits when using Normand to write tests, for example.
435 Compare the following Normand input and some hypothetical XML
436 equivalent, for example:
438 .Actual Normand input.
440 ff dd 01 ab $192 $-128 %1101:0011
447 # five times because xyz
449 "hello world " [iter:8]
457 .Hypothetical Normand XML input.
460 <?xml version="1.0" encoding="utf-8" ?>
462 <byte base="x" val="ff" />
463 <byte base="x" val="dd" />
464 <byte base="x" val="1" />
465 <byte base="x" val="ab" />
466 <byte base="d" val="192" />
467 <byte base="d" val="-128" />
468 <byte base="b" val="11010011" />
469 <fixed-len-num expr="end" len="8" />
470 <var-assign name="iter" expr="1" />
471 <cond expr="not something">
472 <!-- five times because xyz -->
474 <str>hello world </str>
475 <fixed-len-num expr="iter" len="8" />
476 <var-assign name="iter" expr="iter + 1" />
485 A Normand text input is a sequence of items which represent a sequence
488 [[state]] During the processing of items to data, Normand relies on a
493 |State variable |Description |Initial value: <<python3-api,{py3} API>> |Initial value: <<command-line-tool,CLI>>
495 |[[cur-offset]] Current offset
497 The current offset has an effect on the value of <<label,labels>> and of
498 the special `ICITTE` name in <<fixed-length-number,fixed-length
499 number>>, <<leb128-integer,LEB128 integer>>, <<string,string>>,
500 <<filling,filling>>, <<variable-assignment,variable assignment>>,
501 <<conditional-block,conditional block>>, <<repetition-block,repetition
502 block>>, <<macro-expansion,macro expansion>>, and
503 <<post-item-repetition,post-item repetition>> expression evaluation.
505 Each generated byte increments the current offset.
507 A <<current-offset-setting,current offset setting>> may change the
508 current offset without generating data.
510 An <<current-offset-alignment,current offset alignment>> generates
511 padding bytes to make the current offset satisfy a given alignment.
512 |`init_offset` parameter of the `parse()` function.
515 |[[cur-bo]] Current byte order
517 The current byte order can have an effect on the encoding of
518 <<fixed-length-number,fixed-length numbers>>.
520 A <<current-byte-order-setting,current byte order setting>> may change
521 the current byte order.
522 |`init_byte_order` parameter of the `parse()` function.
523 |`--byte-order` option.
526 |Mapping of label names to integral values.
527 |`init_labels` parameter of the `parse()` function.
528 |One or more `--label` options.
530 |<<variable-assignment,Variables>>
531 |Mapping of variable names to integral or floating point number values.
532 |`init_variables` parameter of the `parse()` function.
533 |One or more `--var` or `--var-str` options.
536 The available items are:
538 * A <<byte-constant,constant integer>> representing one or more
541 * A <<literal-string,literal string>> representing a constant sequence
542 of bytes encoding UTF-8, UTF-16, UTF-32, or Latin-1 to Latin-10 data.
544 * A <<current-byte-order-setting,current byte order setting>> (big or
547 * A <<fixed-length-number,fixed-length number>> (integer or
548 floating point), possibly using the <<cur-bo,current byte order>>, and
549 of which the value is the result of a {py3} expression.
551 * An <<leb128-integer,LEB128 integer>> of which the value is the result
552 of a {py3} expression.
554 * A <<string,string>> representing a sequence of bytes encoding UTF-8,
555 UTF-16, UTF-32, or Latin-1 to Latin-10 data, and of which the value is
556 the result of a {py3} expression.
558 * A <<current-offset-setting,current offset setting>>.
560 * A <<current-offset-alignment,current offset alignment>>.
562 * A <<filling,filling>>.
564 * A <<label,label>>, that is, a named constant holding the current
567 This is similar to an assembly label.
569 * A <<variable-assignment,variable assignment>> associating a name to
570 the integral result of an evaluated {py3} expression.
572 * A <<group,group>>, that is, a scoped sequence of items.
574 * A <<conditional-block,conditional block>>.
576 * A <<repetition-block,repetition block>>.
578 * A <<transformation-block,transformation block>>.
580 * A <<macro-definition-block,macro definition block>>.
582 * A <<macro-expansion,macro expansion>>.
584 Moreover, you can repeat many items above a constant or variable number
585 of times with the ``pass:[*]`` operator _after_ the item to repeat. This
586 is called a <<post-item-repetition,post-item repetition>>.
588 A Normand comment may exist pretty much anywhere between tokens.
590 A comment is anything between two ``pass:[#]`` characters on the same
591 line, or from ``pass:[#]`` until the end of the line. Whitespaces are
592 also considered comments. The following symbols are also considered
593 comments around and between items, as well as between hexadecimal
594 nibbles and binary bits of <<byte-constant,byte constants>>:
597 & , - . / : ; = ? \ _ |
600 The latter serve to improve readability so that you may write, for
601 example, a MAC address or a UUID as is.
603 [[const-int]] Many items require a _constant integer_, possibly
604 negative, in which case it may start with `-` for a negative integer. A
605 positive constant integer is any of:
608 One or mode digits (`0` to `9`).
613 * The `0x` or `0X` prefix followed with one or more hexadecimal digits
614 (`0` to `9`, `a` to `f`, or `A` to `F`).
615 * One or more hexadecimal digits followed with the `h` or `H` suffix.
620 * The `0o` or `0O` prefix followed with one or more octal digits
622 * One or more octal digits followed with the `o`, `O`, `q`, or `Q`
628 * The `0b` or `0B` prefix followed with one or more bits (`0` or `1`).
629 * One or more bits followed with the `b` or `B` suffix.
631 In general, anything between `pass:[{]` and `}` is a {py3} expression.
633 You can test the examples of this section with the `normand`
634 <<command-line-tool,command-line tool>> as such:
637 $ normand file | hexdump -C
640 where `file` is the name of a file containing the Normand input.
644 A _byte constant_ represents one or more constant bytes.
649 Two consecutive hexadecimal digits representing a single byte.
652 One or more digits after the `$` prefix representing a single byte.
654 Binary form:: {empty}
657 . __**N**__ `%` prefixes (at least one).
659 The number of `%` characters is the number of subsequent expected bytes.
661 . __**N**__{nbsp}×{nbsp}8 bits (`0` or `1`).
682 $192 %1100/0011 $ -77
696 58f64689-6316-4d55-8a1a-04cada366172
697 fe80::6257:18ff:fea3:4229
703 58 f6 46 89 63 16 4d 55 8a 1a 04 ca da 36 61 72 ┆ X•F•c•MU•••••6ar
704 fe 80 62 57 18 ff fe a3 42 29 ┆ ••bW••••B)
712 %01110011 %01100001 %01101100 %01110101 %01110100
713 %%%1101:0010 11111111 #A#11 #B#00 #C#011 #D#1
719 73 61 6c 75 74 d2 ff c7 ┆ salut•••
725 A _literal string_ represents the encoded bytes of a literal string
726 using the UTF-8, UTF-16, UTF-32, or Latin-1 to Latin-10 encoding.
728 The string to encode isn't implicitly null-terminated: use `\0` at the
729 end of the string to add a null character.
733 . **Optional**: one of the following encodings instead of the default
789 . The ``pass:["]`` prefix.
791 . A sequence of zero or more characters, possibly containing escape
794 An escape sequence is the ``\`` character followed by one of:
800 `b`:: Backspace (U+0008)
801 `e`:: Escape (U+001B)
802 `f`:: Form feed (U+000C)
803 `n`:: End of line (U+000A)
804 `r`:: Carriage return (U+000D)
805 `t`:: Character tabulation (U+0009)
806 `v`:: Line tabulation (U+000B)
807 ``\``:: Reverse solidus (U+005C)
808 ``pass:["]``:: Quotation mark (U+0022)
811 . The ``pass:["]`` suffix.
817 "coucou tout le monde!"
823 63 6f 75 63 6f 75 20 74 6f 75 74 20 6c 65 20 6d ┆ coucou tout le m
824 6f 6e 64 65 21 ┆ onde!
832 u16le"I am not young enough to know everything."
838 49 00 20 00 61 00 6d 00 20 00 6e 00 6f 00 74 00 ┆ I• •a•m• •n•o•t•
839 20 00 79 00 6f 00 75 00 6e 00 67 00 20 00 65 00 ┆ •y•o•u•n•g• •e•
840 6e 00 6f 00 75 00 67 00 68 00 20 00 74 00 6f 00 ┆ n•o•u•g•h• •t•o•
841 20 00 6b 00 6e 00 6f 00 77 00 20 00 65 00 76 00 ┆ •k•n•o•w• •e•v•
842 65 00 72 00 79 00 74 00 68 00 69 00 6e 00 67 00 ┆ e•r•y•t•h•i•n•g•
851 s:u32be "\"illusion is the first\nof all pleasures\" 🦉"
857 00 00 00 22 00 00 00 69 00 00 00 6c 00 00 00 6c ┆ •••"•••i•••l•••l
858 00 00 00 75 00 00 00 73 00 00 00 69 00 00 00 6f ┆ •••u•••s•••i•••o
859 00 00 00 6e 00 00 00 20 00 00 00 69 00 00 00 73 ┆ •••n••• •••i•••s
860 00 00 00 20 00 00 00 74 00 00 00 68 00 00 00 65 ┆ ••• •••t•••h•••e
861 00 00 00 20 00 00 00 66 00 00 00 69 00 00 00 72 ┆ ••• •••f•••i•••r
862 00 00 00 73 00 00 00 74 00 00 00 0a 00 00 00 6f ┆ •••s•••t•••••••o
863 00 00 00 66 00 00 00 20 00 00 00 61 00 00 00 6c ┆ •••f••• •••a•••l
864 00 00 00 6c 00 00 00 20 00 00 00 70 00 00 00 6c ┆ •••l••• •••p•••l
865 00 00 00 65 00 00 00 61 00 00 00 73 00 00 00 75 ┆ •••e•••a•••s•••u
866 00 00 00 72 00 00 00 65 00 00 00 73 00 00 00 22 ┆ •••r•••e•••s•••"
867 00 00 00 20 00 01 f9 89 ┆ ••• ••••
875 s:latin1 "Paul Piché"
881 50 61 75 6c 20 50 69 63 68 e9 ┆ Paul Pich•
885 === Current byte order setting
887 This special item sets the <<cur-bo,_current byte order_>>.
889 The two accepted forms are:
892 `!be`:: Set the current byte order to big endian.
893 `!le`:: Set the current byte order to little endian.
895 === Fixed-length number
897 A _fixed-length number_ represents a fixed number of bytes encoding
900 * An unsigned or signed integer (two's complement).
902 The available lengths are 8, 16, 24, 32, 40, 48, 56, and 64.
904 * A floating point number
905 (https://standards.ieee.org/standard/754-2008.html[IEEE{nbsp}754-2008]).
907 The available lengths are 32 (_binary32_) and 64 (_binary64_).
909 The value is the result of evaluating a {py3} expression.
911 The byte order to use to encode the value is either directly specified
912 or is the <<cur-bo,current byte order>>.
914 A fixed-length number is:
918 . A valid {py3} expression.
920 For a fixed-length number at some source location{nbsp}__**L**__, this
921 expression may contain the name of any accessible <<label,label>> (not
922 within a nested group), including the name of a label defined
923 after{nbsp}__**L**__ (except within a
924 <<transformation-block,transformation block>>), as well as the name of
925 any <<variable-assignment,variable>> known at{nbsp}__**L**__.
927 The value of the special name `ICITTE` (`int` type) in this expression
928 is the <<cur-offset,current offset>> (before encoding the number).
932 . An encoding length in bits amongst:
935 The expression evaluates to an `int` or `bool` value::
936 `8`, `16`, `24`, `32`, `40`, `48`, `56`, and `64`.
938 NOTE: Normand automatically converts a `bool` value to `int`.
940 The expression evaluates to a `float` value::
944 . **Optional**: a suffix of the previous encoding length, without
945 any whitespace, amongst:
949 `be`:: Encode in big endian.
950 `le`:: Encode in little endian.
953 Without this suffix, the encoding byte order is the <<cur-bo,current
954 byte order>> which must be defined if the encoding length is greater
980 # String length in bits
981 [8 * (str_end - str_beg) : 16]
992 00 60 68 65 6c 6c 6f 20 77 6f 72 6c 64 21 ┆ •`hello world!
1000 [20 - ICITTE : 8] * 10
1006 14 13 12 11 10 0f 0e 0d 0c 0b
1026 An _LEB128 integer_ represents a variable number of bytes encoding an
1027 unsigned or signed integer which is the result of evaluating a {py3}
1028 expression following the https://en.wikipedia.org/wiki/LEB128[LEB128]
1031 An LEB128 integer is:
1035 . A valid {py3} expression of which the evaluation result type
1036 is `int` or `bool` (automatically converted to `int`).
1038 For an LEB128 integer at some source location{nbsp}__**L**__, this
1039 expression may contain:
1042 * The name of any <<label,label>> defined before{nbsp}__**L**__
1043 which isn't within a nested group.
1044 * The name of any <<variable-assignment,variable>> known
1048 The value of the special name `ICITTE` (`int` type) in this expression
1049 is the <<cur-offset,current offset>> (before encoding the integer).
1051 . The `:` character.
1057 `uleb128`:: Use the unsigned LEB128 format.
1058 `sleb128`:: Use the signed LEB128 format.
1084 [-981238311 + (meow * -23) : sleb128]
1091 aa bb cc dd ee ff fd fa 8d ac 7c 68 65 6c 6c 6f ┆ ••••••••••|hello
1097 A _string_ represents a variable number of bytes encoding a string which
1098 is the result of evaluating a {py3} expression using the UTF-8, UTF-16,
1099 UTF-32, or Latin-1 to Latin-10 encoding.
1101 A string has two possible forms:
1103 Encoding prefix form:: {empty}
1105 . An encoding amongst:
1160 . The ``pass:[{]`` prefix.
1162 . A valid {py3} expression of which the evaluation result type
1163 is `bool`, `int`, `float`, or `str` (the first three automatically
1164 converted to `str`).
1166 For a string at some source location{nbsp}__**L**__, this expression may
1170 * The name of any <<label,label>> defined before{nbsp}__**L**__
1171 which isn't within a nested group.
1172 * The name of any <<variable-assignment,variable>> known
1176 The value of the special name `ICITTE` (`int` type) in this expression
1177 is the <<cur-offset,current offset>> (before encoding the string).
1181 Encoding suffix form:: {empty}
1185 . A valid {py3} expression of which the evaluation result type
1186 is `bool`, `int`, `float`, or `str` (the first three automatically
1187 converted to `str`).
1189 For a string at some source location{nbsp}__**L**__, this expression may
1193 * The name of any <<label,label>> defined before{nbsp}__**L**__
1194 which isn't within a nested group.
1195 * The name of any <<variable-assignment,variable>> known
1199 The value of the special name `ICITTE` (`int` type) in this expression
1200 is the <<cur-offset,current offset>> (before encoding the string).
1202 . The `:` character.
1204 . A string encoding amongst:
1271 31 20 32 20 33 20 34 20 35 20 36 20 37 20 38 20 ┆ 1 2 3 4 5 6 7 8
1272 39 20 31 30 20 ┆ 9 10
1280 {meow = 'salut jérémie'}
1281 [meow.upper() : s:latin1]
1287 53 41 4c 55 54 20 4a c9 52 c9 4d 49 45 ┆ SALUT J•R•MIE
1291 === Current offset setting
1293 This special item sets the <<cur-offset,_current offset_>>.
1295 A current offset setting is:
1299 . A <<const-int,positive constant integer>> which is the new current
1309 <0x61> [ICITTE : 8] * 8
1315 00 01 02 03 04 05 06 07 61 62 63 64 65 66 67 68 ┆ ••••••••abcdefgh
1323 aa bb cc dd <meow> ee ff
1324 <12> 11 22 33 <mix> 44 55
1325 [meow : 8] [mix : 8]
1331 aa bb cc dd ee ff 11 22 33 44 55 04 0f ┆ •••••••"3DU••
1335 === Current offset alignment
1337 A _current offset alignment_ represents zero or more padding bytes to
1338 make the <<cur-offset,current offset>> meet a given
1339 https://en.wikipedia.org/wiki/Data_structure_alignment[alignment] value.
1341 More specifically, for an alignment value of{nbsp}__**N**__{nbsp}bits,
1342 a current offset alignment represents the required padding bytes until
1343 the current offset is a multiple of __**N**__{nbsp}/{nbsp}8.
1345 A current offset alignment is:
1349 . A <<const-int,positive constant integer>> which is the alignment value
1352 This value must be greater than zero and a multiple of{nbsp}8.
1357 . The ``pass:[~]`` prefix.
1358 . A <<const-int,positive constant integer>> which is the value of the
1359 byte to use as padding to align the <<cur-offset,current offset>>.
1362 Without this section, the padding byte value is zero.
1368 11 22 (@32 aa bb cc) * 3
1374 11 22 00 00 aa bb cc 00 aa bb cc 00 aa bb cc
1384 @32~0xcc [-893.5:32]
1391 77 88 cc cc 00 60 5f c4 55 55 55 55 55 55 55 55 ┆ w••••`_•UUUUUUUU
1392 6d 65 6f 77 ┆ meow
1400 aa bb cc <29> @64~255 "zoom"
1406 aa bb cc ff ff ff 7a 6f 6f 6d ┆ ••••••zoom
1412 A _filling_ represents zero or more padding bytes to make the
1413 <<cur-offset,current offset>> reach a given value.
1417 . The ``pass:[+]`` prefix.
1421 ** A <<const-int,positive constant integer>> which is the current offset
1424 ** The ``pass:[{]`` prefix, a valid {py3} expression of which the
1425 evaluation result type is `int` or `bool` (automatically converted to
1426 `int`), and the `}` suffix.
1428 For a filling at some source location{nbsp}__**L**__, this expression
1432 * The name of any <<label,label>> defined before{nbsp}__**L**__
1433 which isn't within a nested group.
1434 * The name of any <<variable-assignment,variable>> known
1438 The value of the special name `ICITTE` (`int` type) in this expression
1439 is the <<cur-offset,current offset>> (before handling the items to
1442 ** A valid {py3} name.
1444 For the name `__NAME__`, this is equivalent to the
1445 `pass:[{]__NAME__}` form above.
1448 This value must be greater than or equal to the current offset where
1454 . The ``pass:[~]`` prefix.
1455 . A <<const-int,positive constant integer>> which is the value of the
1456 byte to use as padding to reach the current offset target.
1459 Without this section, the padding byte value is zero.
1473 aa bb cc dd 00 00 00 00 00 00 00 00 00 00 00 00 ┆ ••••••••••••••••
1474 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ┆ ••••••••••••••••
1475 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ┆ ••••••••••••••••
1476 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ┆ ••••••••••••••••
1477 68 65 6c 6c 6f 20 77 6f 72 6c 64 ┆ hello world
1485 !macro part(iter, fill)
1486 <0> "particular security " [ord('0') + iter : 8] +fill~0x80
1492 m:part(iter, {32 + 4 * iter})
1500 70 61 72 74 69 63 75 6c 61 72 20 73 65 63 75 72 ┆ particular secur
1501 69 74 79 20 31 80 80 80 80 80 80 80 80 80 80 80 ┆ ity 1•••••••••••
1502 80 80 80 80 70 61 72 74 69 63 75 6c 61 72 20 73 ┆ ••••particular s
1503 65 63 75 72 69 74 79 20 32 80 80 80 80 80 80 80 ┆ ecurity 2•••••••
1504 80 80 80 80 80 80 80 80 80 80 80 80 70 61 72 74 ┆ ••••••••••••part
1505 69 63 75 6c 61 72 20 73 65 63 75 72 69 74 79 20 ┆ icular security
1506 33 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 ┆ 3•••••••••••••••
1507 80 80 80 80 80 80 80 80 70 61 72 74 69 63 75 6c ┆ ••••••••particul
1508 61 72 20 73 65 63 75 72 69 74 79 20 34 80 80 80 ┆ ar security 4•••
1509 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 ┆ ••••••••••••••••
1510 80 80 80 80 80 80 80 80 70 61 72 74 69 63 75 6c ┆ ••••••••particul
1511 61 72 20 73 65 63 75 72 69 74 79 20 35 80 80 80 ┆ ar security 5•••
1512 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 ┆ ••••••••••••••••
1513 80 80 80 80 80 80 80 80 80 80 80 80 ┆ ••••••••••••
1519 A _label_ associates a name to the <<cur-offset,current offset>>.
1521 All the labels of a whole Normand input must have unique names.
1523 A label must not share the name of a <<variable-assignment,variable>>
1530 . A valid {py3} name which is not `ICITTE`.
1534 === Variable assignment
1536 A _variable assignment_ associates a name to the integral result of an
1537 evaluated {py3} expression.
1539 A variable assignment is:
1541 . The ``pass:[{]`` prefix.
1543 . A valid {py3} name which is not `ICITTE`.
1545 . The `=` character.
1547 . A valid {py3} expression of which the evaluation result type is `int`,
1548 `float`, or `bool` (automatically converted to `int`), or `str`.
1550 For a variable assignment at some source location{nbsp}__**L**__, this
1551 expression may contain:
1554 * The name of any <<label,label>> defined before{nbsp}__**L**__
1555 which isn't within a nested group.
1556 * The name of any <<variable-assignment,variable>> known
1560 The value of the special name `ICITTE` (`int` type) in this expression
1561 is the <<cur-offset,current offset>>.
1570 {meow = 42} 11 22 [meow:8] 33 {meow = ICITTE + 17}
1571 "yooo" [meow + mix : 16]
1577 11 22 2a 33 79 6f 6f 6f 7a 00 ┆ •"*3yoooz•
1583 A _group_ is a scoped sequence of items.
1585 The <<label,labels>> within a group aren't visible outside of it.
1587 The main purpose of a group is to <<post-item-repetition,repeat>> more
1588 than a single item and to isolate labels.
1592 . The `(`, `!group`, or `!g` opening.
1594 . Zero or more items except, recursively, a macro definition block.
1596 . Depending on the group opening:
1611 ((aa bb cc) dd () ee) "leclerc"
1617 aa bb cc dd ee 6c 65 63 6c 65 72 63 ┆ •••••leclerc
1626 (aa bb cc) * 3 dd ee
1633 aa bb cc aa bb cc aa bb cc dd ee aa bb cc aa bb
1634 cc aa bb cc dd ee aa bb cc aa bb cc aa bb cc dd
1635 ee aa bb cc aa bb cc aa bb cc dd ee aa bb cc aa
1636 bb cc aa bb cc dd ee
1646 <str_beg> u16le"sébastien diaz" <str_end>
1647 [ICITTE - str_beg : 8]
1648 [(end - str_beg) * 5 : 24]
1656 73 00 e9 00 62 00 61 00 73 00 74 00 69 00 65 00 ┆ s•••b•a•s•t•i•e•
1657 6e 00 20 00 64 00 69 00 61 00 7a 00 1c 00 01 e0 ┆ n• •d•i•a•z•••••
1658 73 00 e9 00 62 00 61 00 73 00 74 00 69 00 65 00 ┆ s•••b•a•s•t•i•e•
1659 6e 00 20 00 64 00 69 00 61 00 7a 00 1c 00 01 40 ┆ n• •d•i•a•z••••@
1660 73 00 e9 00 62 00 61 00 73 00 74 00 69 00 65 00 ┆ s•••b•a•s•t•i•e•
1661 6e 00 20 00 64 00 69 00 61 00 7a 00 1c 00 00 a0 ┆ n• •d•i•a•z•••••
1665 === Conditional block
1667 A _conditional block_ represents either the bytes of zero or more items
1668 if some expression is true, or the bytes of zero or more other items if
1671 A conditional block is:
1673 . The `!if` opening.
1677 ** The ``pass:[{]`` prefix, a valid {py3} expression of which the
1678 evaluation result type is `int` or `bool` (automatically converted to
1679 `int`), and the `}` suffix.
1681 For a conditional block at some source location{nbsp}__**L**__, this
1682 expression may contain:
1685 * The name of any <<label,label>> defined before{nbsp}__**L**__
1686 which isn't within a nested group.
1687 * The name of any <<variable-assignment,variable>> known
1691 The value of the special name `ICITTE` (`int` type) in this expression
1692 is the <<cur-offset,current offset>> (before handling the contained
1695 ** A valid {py3} name.
1697 For the name `__NAME__`, this is equivalent to the
1698 `pass:[{]__NAME__}` form above.
1700 . Zero or more items to be handled when the condition is true
1701 except, recursively, a macro definition block.
1705 .. The `!else` opening.
1706 .. Zero or more items to be handled when the condition is false
1707 except, recursively, a macro definition block
1709 . The `!end` closing.
1727 !if {at < rep_count} 20 !end
1736 6d 65 6f 77 20 7a 6f 6f 6d 20 6d 65 6f 77 20 7a ┆ meow zoom meow z
1737 6f 6f 6d 20 6d 65 6f 77 20 7a 6f 6f 6d 20 6d 65 ┆ oom meow zoom me
1738 6f 77 20 6d 69 78 20 6d 65 6f 77 20 6d 69 78 20 ┆ ow mix meow mix
1739 6d 65 6f 77 20 6d 69 78 20 6d 65 6f 77 20 6d 69 ┆ meow mix meow mi
1740 78 20 6d 65 6f 77 20 6d 69 78 20 6d 65 6f 77 20 ┆ x meow mix meow
1753 !if {str_end - str_beg > 10}
1761 6d 00 65 00 6f 00 77 00 20 00 6d 00 69 00 78 00 ┆ m•e•o•w• •m•i•x•
1762 21 00 20 42 49 47 ┆ !• BIG
1766 === Repetition block
1768 A _repetition block_ represents the bytes of one or more items repeated
1769 a given number of times.
1771 A repetition block is:
1773 . The `!repeat` or `!r` opening.
1777 ** A <<const-int,positive constant integer>> which is the number of
1778 times to repeat the previous item.
1780 ** The ``pass:[{]`` prefix, a valid {py3} expression of which the
1781 evaluation result type is `int` or `bool` (automatically converted to
1782 `int`), and the `}` suffix.
1784 For a repetition block at some source location{nbsp}__**L**__, this
1785 expression may contain:
1788 * The name of any <<label,label>> defined before{nbsp}__**L**__
1789 which isn't within a nested group.
1790 * The name of any <<variable-assignment,variable>> known
1794 The value of the special name `ICITTE` (`int` type) in this expression
1795 is the <<cur-offset,current offset>> (before handling the items to
1798 ** A valid {py3} name.
1800 For the name `__NAME__`, this is equivalent to the
1801 `pass:[{]__NAME__}` form above.
1803 . Zero or more items except, recursively, a macro definition block.
1805 . The `!end` closing.
1807 You may also use a <<post-item-repetition,post-item repetition>> after
1808 some items. The form ``!repeat{nbsp}__X__{nbsp}__ITEMS__{nbsp}!end``
1809 is equivalent to ``(__ITEMS__){nbsp}pass:[*]{nbsp}__X__``.
1816 [end - ICITTE - 1 : 8]
1825 ff fe fd fc fb fa f9 f8 f7 f6 f5 f4 f3 f2 f1 f0 ┆ ••••••••••••••••
1826 ef ee ed ec eb ea e9 e8 e7 e6 e5 e4 e3 e2 e1 e0 ┆ ••••••••••••••••
1827 df de dd dc db da d9 d8 d7 d6 d5 d4 d3 d2 d1 d0 ┆ ••••••••••••••••
1828 cf ce cd cc cb ca c9 c8 c7 c6 c5 c4 c3 c2 c1 c0 ┆ ••••••••••••••••
1829 bf be bd bc bb ba b9 b8 b7 b6 b5 b4 b3 b2 b1 b0 ┆ ••••••••••••••••
1830 af ae ad ac ab aa a9 a8 a7 a6 a5 a4 a3 a2 a1 a0 ┆ ••••••••••••••••
1831 9f 9e 9d 9c 9b 9a 99 98 97 96 95 94 93 92 91 90 ┆ ••••••••••••••••
1832 8f 8e 8d 8c 8b 8a 89 88 87 86 85 84 83 82 81 80 ┆ ••••••••••••••••
1833 7f 7e 7d 7c 7b 7a 79 78 77 76 75 74 73 72 71 70 ┆ •~}|{zyxwvutsrqp
1834 6f 6e 6d 6c 6b 6a 69 68 67 66 65 64 63 62 61 60 ┆ onmlkjihgfedcba`
1835 5f 5e 5d 5c 5b 5a 59 58 57 56 55 54 53 52 51 50 ┆ _^]\[ZYXWVUTSRQP
1836 4f 4e 4d 4c 4b 4a 49 48 47 46 45 44 43 42 41 40 ┆ ONMLKJIHGFEDCBA@
1837 3f 3e 3d 3c 3b 3a 39 38 37 36 35 34 33 32 31 30 ┆ ?>=<;:9876543210
1838 2f 2e 2d 2c 2b 2a 29 28 27 26 25 24 23 22 21 20 ┆ /.-,+*)('&%$#"!
1839 1f 1e 1d 1c 1b 1a 19 18 17 16 15 14 13 12 11 10 ┆ ••••••••••••••••
1840 0f 0e 0d 0c 0b 0a 09 08 07 06 05 04 03 02 01 00 ┆ ••••••••••••••••
1859 11 22 !repeat times 33 !end
1870 aa bb cc dd ee ff ee ff ee ff ee ff ee ff 11 22 ┆ •••••••••••••••"
1871 33 ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ 3•••••••••••••••
1872 ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
1873 ff ee ff ee ff 11 22 33 33 ee ff ee ff ee ff ee ┆ ••••••"33•••••••
1874 ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
1875 ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
1876 ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
1877 ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
1878 ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
1879 ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
1880 ff ee ff ee ff ee ff ee ff ee ff ee ff 11 22 33 ┆ ••••••••••••••"3
1881 33 33 63 6f 75 63 6f 75 21 ┆ 33coucou!
1885 === Transformation block
1887 A _transformation block_ represents the bytes of one or more items
1888 transformed into other bytes by a function.
1890 As of this version, Normand only offers a predetermined set of
1891 transformation functions.
1893 An encoded block is:
1895 . The `!transform` or `!t` opening.
1897 . A transformation function name amongst:
1903 Standard https://datatracker.ietf.org/doc/html/rfc4648.html#section-4[Base64].
1907 URL-safe Base64, using `-` instead of `pass:[+]` and `_` instead of
1912 Standard https://datatracker.ietf.org/doc/html/rfc4648.html#section-6[Base32].
1916 Standard https://datatracker.ietf.org/doc/html/rfc4648.html#section-8[Base16].
1920 https://en.wikipedia.org/wiki/Ascii85[Ascii85] without padding.
1924 Ascii85 with padding.
1928 https://en.wikipedia.org/wiki/Ascii85[Base85] (like Git-style binary
1929 diffs) without padding.
1933 Base85 with padding.
1938 https://datatracker.ietf.org/doc/html/rfc2045#section-6.7[quoted-printable]
1939 without quoted whitespaces.
1943 MIME quoted-printable with quoted whitespaces.
1947 https://en.wikipedia.org/wiki/Gzip[gzip].
1951 https://en.wikipedia.org/wiki/Bzip2[bzip2].
1954 . Zero or more items except, recursively, a macro definition block.
1956 Any {py3} expression within any of those items may not refer to a future
1959 The value of the special name `ICITTE` in any {py3} expression within
1960 any of those items is the <<cur-offset,current offset>> _before_ Normand
1961 applies the transformation function. Therefore, labels defined within
1962 those items also have the current offset value _before_ Normand applies
1963 the transformation function.
1965 . The `!end` closing.
1967 The <<cur-offset,current offset>> after having handled the last item of
1968 a transformation block is the value of the current offset before
1969 handling the first item plus the size of the generated (transformed)
1970 bytes. In other words, <<current-offset-setting,current offset
1971 settings>> within the items of the block have no impact outside said
1980 "size of compressed section: " [end - start : 8]
1985 "this will be compressed!"
1997 aa bb cc dd 73 69 7a 65 20 6f 66 20 63 6f 6d 70 ┆ ••••size of comp
1998 72 65 73 73 65 64 20 73 65 63 74 69 6f 6e 3a 20 ┆ ressed section:
1999 52 42 5a 68 39 31 41 59 26 53 59 68 e1 8c fc 00 ┆ RBZh91AY&SYh••••
2000 00 33 d1 e0 c0 00 60 00 5e 66 dc 80 00 20 00 80 ┆ •3••••`•^f••• ••
2001 00 08 20 00 31 40 d3 43 23 26 20 ca 87 a9 a1 e8 ┆ •• •1@•C#& •••••
2002 18 29 44 80 9c 80 49 bf cc b3 e8 45 ed e2 76 ad ┆ •)D•••I••••E••v•
2003 0f 12 8b 8a d6 cd 40 04 7e 2e e4 8a 70 a1 20 d1 ┆ ••••••@•~.••p• •
2004 c3 19 f8 79 65 73 21 ┆ •••yes!
2015 "I am determined to be cheerful and happy in whatever situation "
2016 "I may find myself. For I have learned that the greater part of "
2017 "our misery or unhappiness is determined not by our circumstance "
2018 "but by our disposition."
2023 !t qp <beg> [ICITTE - beg : 8] * 50 !end
2029 88 88 88 88 88 88 88 88 88 88 88 88 88 88 88 88 ┆ ••••••••••••••••
2030 38 4b 5f 47 59 2b 43 6f 26 2a 41 54 44 58 25 44 ┆ 8K_GY+Co&*ATDX%D
2031 49 6d 3f 24 46 44 69 3a 32 41 4b 59 4a 72 41 53 ┆ Im?$FDi:2AKYJrAS
2032 23 6d 6f 46 5f 69 31 2f 44 49 61 6c 27 40 3b 70 ┆ #moF_i1/DIal'@;p
2033 31 32 2b 44 47 5e 39 47 41 28 45 2c 41 54 68 58 ┆ 12+DG^9GA(E,AThX
2034 2a 2b 45 4d 37 3d 46 5e 5d 42 2b 44 66 2d 5b 68 ┆ *+EM7=F^]B+Df-[h
2035 2b 44 6b 50 34 2b 44 2c 3e 2a 41 30 3e 60 37 46 ┆ +DkP4+D,>*A0>`7F
2036 28 4b 30 22 2f 67 2a 57 25 45 5a 64 70 72 42 4f ┆ (K0"/g*W%EZdprBO
2037 51 27 71 2b 44 62 55 74 45 63 2c 48 21 2b 45 56 ┆ Q'q+DbUtEc,H!+EV
2038 3a 2a 46 3c 47 5b 3d 41 4b 59 57 2b 41 52 54 5b ┆ :*F<G[=AKYW+ART[
2039 6c 45 5a 66 3d 30 45 63 60 46 42 41 66 75 23 37 ┆ lEZf=0Ec`FBAfu#7
2040 45 5a 66 34 35 46 28 4b 42 3b 2b 45 29 39 43 46 ┆ EZf45F(KB;+E)9CF
2041 60 28 6c 24 45 2c 5d 4e 2f 41 54 4d 6f 38 42 6c ┆ `(l$E,]N/ATMo8Bl
2042 62 44 2d 41 54 56 4c 28 44 2f 21 6d 21 41 30 3e ┆ bD-ATVL(D/!m!A0>
2043 63 2e 46 3c 47 25 3c 2b 45 29 43 43 2b 43 66 2c ┆ c.F<G%<+E)CC+Cf,
2044 2b 40 73 29 58 30 46 43 42 26 73 41 4b 59 48 29 ┆ +@s)X0FCB&sAKYH)
2045 46 3c 47 25 3c 2b 45 29 43 43 2b 43 6f 32 2d 45 ┆ F<G%<+E)CC+Co2-E
2046 2c 54 66 33 46 44 35 5a 32 2f 63 99 99 99 99 99 ┆ ,Tf3FD5Z2/c•••••
2047 3d 30 30 3d 30 31 3d 30 32 3d 30 33 3d 30 34 3d ┆ =00=01=02=03=04=
2048 30 35 3d 30 36 3d 30 37 3d 30 38 3d 30 39 0a 3d ┆ 05=06=07=08=09•=
2049 30 42 3d 30 43 0d 3d 30 45 3d 30 46 3d 31 30 3d ┆ 0B=0C•=0E=0F=10=
2050 31 31 3d 31 32 3d 31 33 3d 31 34 3d 31 35 3d 31 ┆ 11=12=13=14=15=1
2051 36 3d 31 37 3d 31 38 3d 31 39 3d 31 41 3d 31 42 ┆ 6=17=18=19=1A=1B
2052 3d 31 43 3d 31 44 3d 31 45 3d 31 46 20 21 22 23 ┆ =1C=1D=1E=1F !"#
2053 24 25 26 27 28 29 2a 2b 2c 2d 3d 0a 2e 2f 30 31 ┆ $%&'()*+,-=•./01
2057 === Macro definition block
2059 A _macro definition block_ associates a name and parameter names to
2062 A macro definition block doesn't lead to generated bytes itself: a
2063 <<macro-expansion,macro expansion>> does so.
2065 A macro definition may only exist at the root level, that is, not within
2066 a <<group,group>>, a <<repetition-block,repetition block>>, a
2067 <<conditional-block,conditional block>>, or another
2068 <<macro-definition-block,macro definition block>>.
2070 All macro definitions must have unique names.
2072 A macro definition is:
2074 . The `!macro` or `!m` opening.
2076 . A valid {py3} name (the macro name).
2078 . The `(` parameter name list prefix.
2080 . A comma-separated list of zero or more unique parameter names,
2081 each one being a valid {py3} name.
2083 . The `)` parameter name list suffix.
2085 . Zero or more items except, recursively, a macro definition block.
2087 . The `!end` closing.
2092 !le [ICITTE * 8 : 16]
2093 u16le"predict explode"
2100 !macro nail(rep, with_extra, val)
2104 [val + iter : uleb128]
2118 A _macro expansion_ expands the items of a defined
2119 <<macro-definition-block,macro>>.
2121 The macro to expand must be defined _before_ the expansion.
2123 The <<state,state>> before handling the first item of the chosen macro
2126 <<cur-offset,Current offset>>::
2129 <<cur-bo,Current byte order>>::
2133 The only available variables initially are the macro parameters.
2138 The state after having handled the last item of the chosen macro is:
2141 The one before handling the first item of the macro plus the size
2142 of the generated data of the macro expansion.
2144 IMPORTANT: This means <<current-offset-setting,current offset setting>>
2145 items within the expanded macro don't impact the final current offset.
2147 Current byte order::
2148 The one before handling the first item of the macro.
2151 The ones before handling the first item of the macro.
2154 The ones before handling the first item of the macro.
2156 A macro expansion is:
2160 . A valid {py3} name (the name of the macro to expand).
2162 . The `(` parameter value list prefix.
2164 . A comma-separated list of zero or more unique parameter values.
2166 The number of parameter values must match the number of parameter
2167 names of the definition of the chosen macro.
2169 A parameter value is one of:
2172 * A <<const-int,constant integer>>, possibly negative.
2174 * A constant floating point number.
2176 * The ``pass:[{]`` prefix, a valid {py3} expression of which the
2177 evaluation result type is `int` or `bool` (automatically converted to
2178 `int`), and the `}` suffix.
2180 For a macro expansion at some source location{nbsp}__**L**__, this
2181 expression may contain:
2183 ** The name of any <<label,label>> defined before{nbsp}__**L**__
2184 which isn't within a nested group.
2185 ** The name of any <<variable-assignment,variable>> known
2189 The value of the special name `ICITTE` (`int` type) in this expression
2190 is the <<cur-offset,current offset>> (before handling the items of the
2193 * A valid {py3} name.
2195 For the name `__NAME__`, this is equivalent to the
2196 `pass:[{]__NAME__pass:[}]` form above.
2199 . The `)` parameter value list suffix.
2206 !le [ICITTE * 8 : 16]
2207 u16le"predict explode"
2210 "hello [" m:bake() "] world"
2218 68 65 6c 6c 6f 20 5b 38 00 70 00 72 00 65 00 64 ┆ hello [8•p•r•e•d
2219 00 69 00 63 00 74 00 20 00 65 00 78 00 70 00 6c ┆ •i•c•t• •e•x•p•l
2220 00 6f 00 64 00 65 00 5d 20 77 6f 72 6c 64 70 01 ┆ •o•d•e•] worldp•
2221 70 00 72 00 65 00 64 00 69 00 63 00 74 00 20 00 ┆ p•r•e•d•i•c•t• •
2222 65 00 78 00 70 00 6c 00 6f 00 64 00 65 00 70 02 ┆ e•x•p•l•o•d•e•p•
2223 70 00 72 00 65 00 64 00 69 00 63 00 74 00 20 00 ┆ p•r•e•d•i•c•t• •
2224 65 00 78 00 70 00 6c 00 6f 00 64 00 65 00 70 03 ┆ e•x•p•l•o•d•e•p•
2225 70 00 72 00 65 00 64 00 69 00 63 00 74 00 20 00 ┆ p•r•e•d•i•c•t• •
2226 65 00 78 00 70 00 6c 00 6f 00 64 00 65 00 70 04 ┆ e•x•p•l•o•d•e•p•
2227 70 00 72 00 65 00 64 00 69 00 63 00 74 00 20 00 ┆ p•r•e•d•i•c•t• •
2228 65 00 78 00 70 00 6c 00 6f 00 64 00 65 00 70 05 ┆ e•x•p•l•o•d•e•p•
2229 70 00 72 00 65 00 64 00 69 00 63 00 74 00 20 00 ┆ p•r•e•d•i•c•t• •
2230 65 00 78 00 70 00 6c 00 6f 00 64 00 65 00 ┆ e•x•p•l•o•d•e•
2238 !macro A(val, is_be)
2248 !macro B(rep, is_be)
2252 m:A({iter * 3}, is_be)
2264 00 03 00 06 00 09 00 0c 00 0f 03 00 06 00 09 00
2272 !macro flt32be(val) !be [val : 32] !end
2282 43 48 45 45 54 4f 53 c2 28 ae 14 3b b8 41 25 ┆ CHEETOS•(••;•A%
2286 === Post-item repetition
2288 A _post-item repetition_ represents the bytes of an item repeated a
2289 given number of times.
2291 A post-item repetition is:
2293 . One of those items:
2295 ** A <<byte-constant,byte constant>>.
2296 ** A <<literal-string,literal string>>.
2297 ** A <<fixed-length-number,fixed-length number>>.
2298 ** An <<leb128-integer,LEB128 integer>>.
2299 ** A <<string,string>>.
2300 ** A <<macro-expansion,macro-expansion>>.
2301 ** A <<transformation-block,transformation block>>.
2302 ** A <<group,group>>.
2304 . The ``pass:[*]`` character.
2308 ** A positive integer (hexadecimal starting with `0x` or `0X` accepted)
2309 which is the number of times to repeat the previous item.
2311 ** The ``pass:[{]`` prefix, a valid {py3} expression of which the
2312 evaluation result type is `int` or `bool` (automatically converted to
2313 `int`), and the `}` suffix.
2315 For a post-item repetition at some source location{nbsp}__**L**__, this
2316 expression may contain:
2319 * The name of any <<label,label>> defined before{nbsp}__**L**__
2320 which isn't within a nested group and
2321 which isn't part of the repeated item.
2322 * The name of any <<variable-assignment,variable>> known
2323 at{nbsp}__**L**__, which isn't part of its repeated item, and which
2327 The value of the special name `ICITTE` (`int` type) in this expression
2328 is the <<cur-offset,current offset>> (before handling the items to
2331 ** A valid {py3} name.
2333 For the name `__NAME__`, this is equivalent to the
2334 `pass:[{]__NAME__pass:[}]` form above.
2336 You may also use a <<repetition-block,repetition block>>. The form
2337 ``__ITEM__{nbsp}pass:[*]{nbsp}__X__`` is equivalent to
2338 ``!repeat{nbsp}__X__{nbsp}__ITEM__{nbsp}!end``.
2344 [end - ICITTE - 1 : 8] * 0x100 <end>
2350 ff fe fd fc fb fa f9 f8 f7 f6 f5 f4 f3 f2 f1 f0 ┆ ••••••••••••••••
2351 ef ee ed ec eb ea e9 e8 e7 e6 e5 e4 e3 e2 e1 e0 ┆ ••••••••••••••••
2352 df de dd dc db da d9 d8 d7 d6 d5 d4 d3 d2 d1 d0 ┆ ••••••••••••••••
2353 cf ce cd cc cb ca c9 c8 c7 c6 c5 c4 c3 c2 c1 c0 ┆ ••••••••••••••••
2354 bf be bd bc bb ba b9 b8 b7 b6 b5 b4 b3 b2 b1 b0 ┆ ••••••••••••••••
2355 af ae ad ac ab aa a9 a8 a7 a6 a5 a4 a3 a2 a1 a0 ┆ ••••••••••••••••
2356 9f 9e 9d 9c 9b 9a 99 98 97 96 95 94 93 92 91 90 ┆ ••••••••••••••••
2357 8f 8e 8d 8c 8b 8a 89 88 87 86 85 84 83 82 81 80 ┆ ••••••••••••••••
2358 7f 7e 7d 7c 7b 7a 79 78 77 76 75 74 73 72 71 70 ┆ •~}|{zyxwvutsrqp
2359 6f 6e 6d 6c 6b 6a 69 68 67 66 65 64 63 62 61 60 ┆ onmlkjihgfedcba`
2360 5f 5e 5d 5c 5b 5a 59 58 57 56 55 54 53 52 51 50 ┆ _^]\[ZYXWVUTSRQP
2361 4f 4e 4d 4c 4b 4a 49 48 47 46 45 44 43 42 41 40 ┆ ONMLKJIHGFEDCBA@
2362 3f 3e 3d 3c 3b 3a 39 38 37 36 35 34 33 32 31 30 ┆ ?>=<;:9876543210
2363 2f 2e 2d 2c 2b 2a 29 28 27 26 25 24 23 22 21 20 ┆ /.-,+*)('&%$#"!
2364 1f 1e 1d 1c 1b 1a 19 18 17 16 15 14 13 12 11 10 ┆ ••••••••••••••••
2365 0f 0e 0d 0c 0b 0a 09 08 07 06 05 04 03 02 01 00 ┆ ••••••••••••••••
2377 (ee ff) * {here + 1}
2387 aa bb cc dd ee ff ee ff ee ff ee ff ee ff 11 22 ┆ •••••••••••••••"
2388 33 ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ 3•••••••••••••••
2389 ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
2390 ff ee ff ee ff 11 22 33 33 ee ff ee ff ee ff ee ┆ ••••••"33•••••••
2391 ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
2392 ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
2393 ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
2394 ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
2395 ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
2396 ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
2397 ff ee ff ee ff ee ff ee ff ee ff ee ff 11 22 33 ┆ ••••••••••••••"3
2398 33 33 63 6f 75 63 6f 75 21 ┆ 33coucou!
2402 == Command-line tool
2404 If you <<install-normand,installed>> the `normand` package, then you
2405 can use the `normand` command-line tool:
2408 $ normand <<< '"ma gang de malades"' | hexdump -C
2412 00000000 6d 61 20 67 61 6e 67 20 64 65 20 6d 61 6c 61 64 |ma gang de malad|
2416 If you copy the `normand.py` module to your own project, then you can
2417 run the module itself:
2420 $ python3 -m normand <<< '"ma gang de malades"' | hexdump -C
2424 00000000 6d 61 20 67 61 6e 67 20 64 65 20 6d 61 6c 61 64 |ma gang de malad|
2428 Without a path argument, the `normand` tool reads from the standard
2431 The `normand` tool prints the generated binary data to the standard
2434 Various options control the initial <<state,state>> of the processor:
2435 use the `--help` option to learn more.
2439 The whole `normand` package/module public API is:
2444 class ByteOrder(enum.Enum):
2456 def line_no(self) -> int:
2461 def col_no(self) -> int:
2465 # Parsing error message.
2466 class ParseErrorMessage:
2472 # Source text location.
2474 def text_location(self):
2479 class ParseError(RuntimeError):
2480 # Parsing error messages.
2482 # The first message is the most _specific_ one.
2488 # Variables dictionary type (for type hints).
2489 VariablesT = typing.Dict[str, typing.Union[int, float]]
2492 # Labels dictionary type (for type hints).
2493 LabelsT = typing.Dict[str, int]
2500 def data(self) -> bytearray:
2503 # Updated variable values.
2505 def variables(self) -> SymbolsT:
2508 # Updated main group label values.
2510 def labels(self) -> SymbolsT:
2515 def offset(self) -> int:
2520 def byte_order(self) -> typing.Optional[ByteOrder]:
2524 # Parses the `normand` input using the initial state defined by
2525 # `init_variables`, `init_labels`, `init_offset`, and `init_byte_order`,
2526 # and returns the corresponding parsing result.
2527 def parse(normand: str,
2528 init_variables: typing.Optional[SymbolsT] = None,
2529 init_labels: typing.Optional[SymbolsT] = None,
2530 init_offset: int = 0,
2531 init_byte_order: typing.Optional[ByteOrder] = None) -> ParseResult:
2535 The `normand` parameter is the actual <<learn-normand,Normand input>>
2536 while the other parameters control the initial <<state,state>>.
2538 The `parse()` function raises a `ParseError` instance should it fail to
2539 parse the `normand` string for any reason.
2543 Normand is a https://python-poetry.org/[Poetry] project.
2545 To develop it, install it through Poetry and enter the virtual
2551 $ normand <<< '"lol" * 10 0a'
2554 `normand.py` is processed by:
2556 * https://microsoft.github.io/pyright/[Pyright]
2557 * https://github.com/psf/black[Black]
2558 * https://pycqa.github.io/isort/[isort]
2560 Licensing and copyright follows the
2561 https://reuse.software/tutorial/[REUSE] specification and is checked
2562 with the https://github.com/fsfe/reuse-tool[reuse tool].
2566 Use https://docs.pytest.org/[pytest] to test Normand once the package is
2567 part of your virtual environment, for example:
2571 $ poetry run pip3 install pytest
2575 The `pytest` project is currently not a development dependency in
2576 `pyproject.toml` due to backward compatibiliy issues with
2579 In the `tests` directory, each `*.nt` file is a test. The file name
2580 prefix indicates what it's meant to test:
2583 Everything above the `---` line is the valid Normand input
2586 Everything below the `---` line is the expected data
2587 (whitespace-separated hexadecimal bytes).
2590 Everything above the `---` line is the invalid Normand input
2593 Everything below the `---` line is the expected error message having
2602 Normand uses https://review.lttng.org/admin/repos/normand,general[Gerrit]
2605 To report a bug, https://github.com/efficios/normand/issues/new[create a