tft: script tokens, global variables, timers, control flow

build-tft now reproduces 35 Editor-made samples byte for byte: assignments,
arithmetic, 27 commands, system variables and colours, attribute references,
Program.s globals, all page/hotspot/timer events, if/else/while/for.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This commit is contained in:
mangavalkandClaude Fable 5.2 committed 2026-11-16 18:23:14 +01:00
1 parent f66c902443
commit 99a5240af0
7 files changed
+511 -50

No files matched your search

+1 -1
View File
@@ -32,7 +32,7 @@ Every percentage is a count with a stated basis, so it can be checked (details i
| Readable attribute names in JSON | **100 %** (31/31 types) | every value list the Editor documents, incl. the Intelligent-only types and load effects | | Readable attribute names in JSON | **100 %** (31/31 types) | every value list the Editor documents, incl. the Intelligent-only types and load effects |
| Rendering, Basic components | **100 %** (20/20 pixel-exact) | incl. gauges and QR codes (own encoder, no dependencies) | | Rendering, Basic components | **100 %** (20/20 pixel-exact) | incl. gauges and QR codes (own encoder, no dependencies) |
| Rendering, all components | **94 %** (29/31 pixel-exact) | Intelligent ComboBox and Switch are drawn, but their anti-aliased edges are approximate (~99 % / ~92 % of pixels exact) | | Rendering, all components | **94 %** (29/31 pixel-exact) | Intelligent ComboBox and Switch are drawn, but their anti-aliased edges are approximate (~99 % / ~92 % of pixels exact) |
| `.tft` compilation | **69 %** (5.5/8 milestones) | `nxhmi build-tft`: byte-identical to the Editor for NX4832T035 projects with pages, hotspots, int variables, pictures and fonts; other component types and most script not yet | | `.tft` compilation | **81 %** (6.5/8 milestones) | `nxhmi build-tft`: byte-identical to the Editor for NX4832T035 projects with pages, hotspots, int variables, timers, pictures, fonts and script (assignments, arithmetic, 27 commands, if/else/while/for, global variables); drawing component types not yet; never tried on a display |
## Compatibility ## Compatibility
+2 -2
View File
@@ -113,8 +113,8 @@ byte for byte, or a file written by nxhmi was opened in the Editor and showed th
| Trailer checksum | solved for any length (`tft.trailer`); one model measured | | Trailer checksum | solved for any length (`tft.trailer`); one model measured |
| 3 header checksums (0x44, 0xC4, 0x18C) | solved (`tft.header_checksums`, `tft.seal`) | | 3 header checksums (0x44, 0xC4, 0x18C) | solved (`tft.header_checksums`, `tft.seal`) |
| Pictures and fonts inside the TFT | solved (FORMAT.md 7.2) | | Pictures and fonts inside the TFT | solved (FORMAT.md 7.2) |
| Compiled page format | structure solved (code blob, name hashes, RAM image, attribute records, page table, object records -- FORMAT.md 7.3-7.5); per-type tables done for page, hotspot, int variable; the other types need their slot tables and generated draw scripts | | Compiled page format | structure solved (code blob, name hashes, RAM image, attribute records, page table, object records -- FORMAT.md 7.3-7.5); per-type tables done for page, hotspot, int variable, timer; the other types need their slot tables and generated draw scripts |
| Event-code tokens | encoding understood (command / system-variable / attribute-reference / number tokens); table being filled | | Event-code tokens | solved for what was sampled (FORMAT.md 7.6): 27 commands, 33 system variables/colours, numbers, global variables, attribute references, if/else/while/for. Not sampled: string functions and attributes, the remaining commands and system variables, `>=` op code |
| Builder (`nxhmi build-tft`) | byte-identical to the Editor for the supported subset (one model: NX4832T035); refuses anything else. **Never tested on a real display** | | Builder (`nxhmi build-tft`) | byte-identical to the Editor for the supported subset (one model: NX4832T035); refuses anything else. **Never tested on a real display** |
| Licensing | every `.tft` embeds Nextion's firmware files; the repo must never ship them -- a builder would read them from the user's own Editor installation | | Licensing | every `.tft` embeds Nextion's firmware files; the repo must never ship them -- a builder would read them from the user's own Editor installation |
+45 -10
View File
@@ -394,8 +394,8 @@ program
per page: code blocks of its components, name list, RAM image, init code per page: code blocks of its components, name list, RAM image, init code
``` ```
The first ten words describe three tables at 4 + (0x1C, 0x20, 0x24) with sizes 4, 4, 6*pages; The first eight words describe three tables at 4 + offset: table 0 (4 zero bytes, **OPEN**),
the first two are empty when the program declares no global variables (**OPEN**: their format). table 1 (global variables, see 7.6; 4 zero bytes without globals) and the page names.
**Name hash** = CRC of the name NUL-padded to 14 bytes. Names are not stored anywhere else. **Name hash** = CRC of the name NUL-padded to 14 bytes. Names are not stored anywhere else.
@@ -406,7 +406,7 @@ tokenised script line; an empty chunk ends a section.
`page 0` added by the Editor, then the empty chunk. `page 0` added by the Editor, then the empty chunk.
- Component block (the page itself is component 0): - Component block (the page itself is component 0):
- draw section -- lines, empty chunk. Page: `fill x,y,w,h,<bco>` (command `09 0D 04`, colour - draw section -- lines, empty chunk. Page: `fill x,y,w,h,<bco>` (command `09 0D 04`, colour
as a reference to the page's `bco` record), then the Preinitialize event lines. as a reference to the page's `bco` record).
Types that draw themselves (progress bar, gauge, QR code, text ...) have their own script Types that draw themselves (progress bar, gauge, QR code, text ...) have their own script
here, generated from the attributes (**OPEN** per type). here, generated from the attributes (**OPEN** per type).
- erase section (types with a rectangle only; empty for the page) -- `fill x,y,w,h,<page bco>`, - erase section (types with a rectangle only; empty for the page) -- `fill x,y,w,h,<page bco>`,
@@ -415,8 +415,8 @@ tokenised script line; an empty chunk ends a section.
lines, empty chunk. Page: down, up, unload. Hotspot: down, up. Variable: none. lines, empty chunk. Page: down, up, unload. Hotspot: down, up. Variable: none.
- After the last component of a page: one chunk with the name list `(u32 hash, u16 id)` sorted - After the last component of a page: one chunk with the name list `(u32 hash, u16 id)` sorted
by hash; one chunk with the RAM image followed by 8 zero bytes; type-specific init lines by hash; one chunk with the RAM image followed by 8 zero bytes; type-specific init lines
(timer, slider, touch cap -- **OPEN**); the chunk `09 21 08`; the page's Postinitialize lines; (timer: 7.6; slider, touch cap -- **OPEN**); the page's Preinitialize lines; the chunk
empty chunk. `09 21 08`; the page's Postinitialize lines; empty chunk.
RAM image: 8 x FF, then every RAM-stored attribute of every component in component order, RAM image: 8 x FF, then every RAM-stored attribute of every component in component order,
each aligned to its size, each component padded to 4. For a page: `sta` (u8), pad, `bco` (u16). each aligned to its size, each component padded to 4. For a page: `sta` (u8), pad, `bco` (u16).
@@ -430,11 +430,11 @@ in the object records.
| Offset | Field | | Offset | Field |
|---|---| |---|---|
| 0 | u32 8 for a page, 16 for other components | | 0 | u32 RAM address of the component's data (page: of its second word) |
| 4 | i32 value (constant attributes) or 4 + offset into the RAM image | | 4 | i32 value (constant attributes) or RAM address of the value |
| 8, 12 | i32 maximum, minimum | | 8, 12 | i32 maximum, minimum |
| 16 | u8 page number, u8 component id of the component to redraw on change; FF FF = none | | 16 | u8 page number, u8 component id of the component to redraw on change; FF FF = none |
| 18 | u16 change action (0 = none; timers use 1 for `tim`, 2 for `en`) | | 18 | u8 0, u8 change action (0 = none; timers use 1 for `tim`, 2 for `en`) |
| 20 | u8 kind: 0x01 constant; 0x0B/0x0C/0x0D/0x0E x/y/w/h; 0x08 endx/endy; 0x21, 0x22, 0x31, 0x32, 0x39, 0x3F RAM-stored | | 20 | u8 kind: 0x01 constant; 0x0B/0x0C/0x0D/0x0E x/y/w/h; 0x08 endx/endy; 0x21, 0x22, 0x31, 0x32, 0x39, 0x3F RAM-stored |
| 21 | u8 size in bytes | | 21 | u8 size in bytes |
| 22 | u16 0 | | 22 | u16 0 |
@@ -453,9 +453,44 @@ Object:
| 0 | u8 type, u8 id, u8 0, u8 flags (0x37 visible types, 0x27 timer/touch cap, 0x07 variable) | | 0 | u8 type, u8 id, u8 0, u8 flags (0x37 visible types, 0x27 timer/touch cap, 0x07 variable) |
| 4 | u32 offset of the draw section, FFFFFFFF if empty | | 4 | u32 offset of the draw section, FFFFFFFF if empty |
| 8 | u32 offset of the erase section, FFFFFFFF if none/empty | | 8 | u32 offset of the erase section, FFFFFFFF if none/empty |
| 0x0C-0x1B | more code offsets (0x14: timer event), FF when unused | | 0x0C | u32 x 4: offset of the first line of the down, up, timer, unload event; FFFFFFFF when the event is empty |
| 0x1C | u32 8 / 16 as in the records | | 0x1C | u32 RAM address as in the records |
| 0x20 | `00 00 7F 00`, u32 0 | | 0x20 | `00 00 7F 00`, u32 0 |
| 0x28 | u16 x, y, w, h, endx, endy (0,0,1,1,0,0 for invisible types) | | 0x28 | u16 x, y, w, h, endx, endy (0,0,1,1,0,0 for invisible types) |
| 0x34 | u32 offset of the component's code block | | 0x34 | u32 offset of the component's code block |
| 0x38 | 88 x u16: record number (page-relative) of well-known attributes, FFFF = absent. Positions: 1 vscope, 2 sta, 4 bco, 16 pco, 20 val, 42 x, 43 y, 44 endx, 45 endy, 46 w, 47 h, 49 id, 62 type (**OPEN**: the others) | | 0x38 | 88 x u16: record number (page-relative) of well-known attributes, FFFF = absent. Positions: 1 vscope, 2 sta, 4 bco, 16 pco, 20 val, 42 x, 43 y, 44 endx, 45 endy, 46 w, 47 h, 49 id, 62 type (**OPEN**: the others) |
### 7.6 Script tokens
A script line is stored as text with some parts replaced by binary tokens
(`nxhmi.tft_code`; tables there):
| Token | Meaning |
|---|---|
| `09 ii tt` | a command at the start of a line; `tt` = 04 for names of up to 4 letters, 08 for longer ones, `ii` = index in that table (`page` 0B/04, `print` 04/08 ...). The arguments follow, tokenised like an expression (`printh`: plain text) |
| `04 tt iiii 00` | system variable or colour constant, same two tables (`dim` 0003/04, `WHITE` 0000/08 ...) |
| `05` + u32 | global variable declared in Program.s: its offset in the global RAM (4 bytes each, in declaration order) |
| `01` + u32 | component attribute (`va0.val`, `page0.va0.val`): global attribute record number |
| `03` + i32 | number: hex literals and decimal literals of four or more digits (a leading minus is folded in); shorter decimals stay text |
Operators, commas, quotes, component and page names used as command arguments (`ref m0`,
`page page0`) and the array forms `b[1].x`, `p[0].b[2].val` stay text.
`int a=0,b=5` lines in Program.s produce no chunk: the values go into header table 1 of the code
blob (u32 each, then 4 zero bytes), and one 12-byte descriptor per variable `(CRC of the bare
name, offset, 9)`, sorted by hash, sits between the code blob and the attribute records (layout
word 2 = count, word 3 = offset of the attribute records). All RAM addresses in attribute
records start after this global RAM: address = 4 + 4*globals + offset in the page RAM image.
Control flow becomes jumps; `skip` counts bytes (chunks including their length words) from the
end of the jumping chunk and may be negative:
- `09 00 04 <lhs>,<rhs>,<op>,<03 skip>` -- continue when `lhs op rhs` holds, otherwise jump.
`op`: 1 `==`, 2 `<`, 3 `>`, 4 `<=`, 6 `!=`; 5 `>=` is inferred (**OPEN**).
- `T ` + `03 skip` -- jump always (end of an `if` body that has an `else`; end of a loop body).
- `a&&b`: one test per term, each skipping to the else part. `a||b`: every term but the last is
inverted and skips to the body.
- `for(init;cond;step)` = init line, tests, body, step line, jump back.
Timers: the event section is the label `timer`, a chunk `S ` + `03 1`, the event lines; the
page's init code gets `09 22 08 <en>,<id>,<tim>,0` and `L ` + `03 <offset of the first event line>`.
+16 -3
View File
@@ -152,6 +152,19 @@ Useful facts not obvious from the format description:
- Record references in scripts (`01` + u32) are global record numbers, so adding a component - Record references in scripts (`01` + u32) are global record numbers, so adding a component
renumbers every reference after it. renumbers every reference after it.
Not done: script tokens beyond `baud=115200` / `page N`; timers, sliders and every drawing Script (same day): tokens, global variables, timers and control flow are in (FORMAT.md 7.6,
component; global variables declared in Program.s (blob header tables 0 and 1); other models, `nxhmi.tft_code`); 35 Editor-made samples are reproduced byte for byte. Things to know:
portrait, other series; and nothing has been loaded onto a display.
- A token table entry exists only for names that were compiled once. To add names: put them in
a project, compile, read the token from the chunk (the analysis was a 30-line script printing
source line next to chunk).
- `>=` was never the plain operator in a sample (only as the inverse of `<` in an `||`); its
code 5 is the gap in 1..6. Verify before relying on it.
- Unverified and therefore refused: negative global initial values, `x-1000` (binary minus
before a long number), mixed `&&`/`||`, parentheses in conditions, braces-less `if`.
- The Editor appends one `page 0` chunk to the program even when Program.s already ends with it.
Not done: sliders, touch cap and every drawing component (text, number, button, progress bar,
gauge, waveform, picture, crop, checkbox, radio, QR code ...); string variables and string
functions; page backgrounds other than a solid colour; global (vscope) components; other
models, portrait, other series; and nothing has been loaded onto a display.
+85 -25
View File
@@ -14,6 +14,7 @@ import struct
from . import tft, tft_code from . import tft, tft_code
from .container import HMIFile from .container import HMIFile
from .page import Page from .page import Page
from .checksum import word_per_byte_crc
from .tft_code import Unsupported from .tft_code import Unsupported
FIRMWARE_FILES = ("input.bin", "asp0.dll", "cd0.dll", "qr0.bin", "syscom.bin", "h0-0.dll", "h1-0.dll") FIRMWARE_FILES = ("input.bin", "asp0.dll", "cd0.dll", "qr0.bin", "syscom.bin", "h0-0.dll", "h1-0.dll")
@@ -22,7 +23,7 @@ SUPPORTED_MODEL = 0xB23BC018 # NX4832T035_011, the only model measured
BLOCK = 0x10000 BLOCK = 0x10000
# attribute map positions in the object record (u16 table at +0x38) # attribute map positions in the object record (u16 table at +0x38)
MAP = {"vscope": 1, "sta": 2, "bco": 4, "val": 20, "x": 42, "y": 43, "endx": 44, "endy": 45, "w": 46, "h": 47, MAP = {"vscope": 1, "sta": 2, "bco": 4, "val": 20, "tim": 33, "en": 34, "x": 42, "y": 43, "endx": 44, "endy": 45, "w": 46, "h": 47,
"id": 49, "type": 62} "id": 49, "type": 62}
# attribute record templates: name -> (slot, kind, size, max, min) # attribute record templates: name -> (slot, kind, size, max, min)
_COMMON = {"type": (0, 0x01, 1, 255, 0), "id": (1, 0x01, 1, 255, 0), "vscope": (3, 0x01, 1, 1, 0)} _COMMON = {"type": (0, 0x01, 1, 255, 0), "id": (1, 0x01, 1, 255, 0), "vscope": (3, 0x01, 1, 1, 0)}
@@ -37,6 +38,8 @@ TYPES = {
52: dict(name="variable", slots=11, flag=0x07, group=16, 52: dict(name="variable", slots=11, flag=0x07, group=16,
const={**_COMMON, "sta": (7, 0x01, 1, 1, 0)}, const={**_COMMON, "sta": (7, 0x01, 1, 1, 0)},
ram={"val": (8, 0x39, 4, 2147483647, -2147483648)}, owner=False), ram={"val": (8, 0x39, 4, 2147483647, -2147483648)}, owner=False),
51: dict(name="timer", slots=11, flag=0x27, group=16, const=dict(_COMMON),
ram={"tim": (7, 0x32, 2, 65534, 50), "en": (8, 0x31, 1, 1, 0)}, owner=True, action={"tim": 1, "en": 2}),
} }
@@ -93,28 +96,64 @@ def resource_area(hmi: HMIFile, editor_dir: str) -> tuple[bytes, dict]:
def compile_pages(pages: list[Page], program: str): def compile_pages(pages: list[Page], program: str):
"""-> (code blob, attribute records, page table, object records, program offset).""" """-> (code blob, global variable descriptors, attribute records, page table, object records,
program offset)."""
names = sorted((tft.name_hash(p.components[0].get("objname")), i) for i, p in enumerate(pages)) names = sorted((tft.name_hash(p.components[0].get("objname")), i) for i, p in enumerate(pages))
n = len(pages) n = len(pages)
blob = bytearray(struct.pack("<10I", 0x24 + 6 * n, 3, 0x1C, 4, 0x20, 4, 0x24, 6 * n, 0, 0)) decl, program_lines = tft_code.declarations(tft_code.statements(program.splitlines()))
if any(v < 0 for v in decl.values()):
raise Unsupported("global variables with a negative initial value")
g = 4 * len(decl)
blob = bytearray(struct.pack("<8I", 0x24 + g + 6 * n, 3, 0x1C, 4, 0x20, 4 + g, 0x24 + g, 6 * n))
blob += bytes(4) + b"".join(struct.pack("<I", v) for v in decl.values()) + bytes(4)
for h, i in names: for h, i in names:
blob += struct.pack("<IH", h, i) blob += struct.pack("<IH", h, i)
program_off = len(blob) program_off = len(blob)
ctx = tft_code.Context() offsets = {name: 4 * i for i, name in enumerate(decl)}
descriptors = b"".join(struct.pack("<III", h, o, 9) for h, o in
sorted((word_per_byte_crc(name.encode("latin-1")), o) for name, o in offsets.items()))
# record numbers are needed before any script is compiled: number them first
where, base_rec = {}, 0 # (page name, component name) -> (first record, type)
for page in pages:
pname = page.components[0].get("objname")
for c in page.components:
tid = _u(c.attr("type"))
if tid not in TYPES:
raise Unsupported("component type %d (%s)" % (tid, c.get("objname")))
where[(pname, c.get("objname"))] = (base_rec, TYPES[tid])
base_rec += TYPES[tid]["slots"]
current = [None]
def attr(path):
if len(path) == 3:
key, name = (path[0], path[1]), path[2]
elif len(path) == 2:
key, name = (current[0], path[0]), path[1]
else:
return None
if key not in where:
return None
first, t = where[key]
slot = {**t["const"], **t["ram"]}.get(name)
return None if slot is None else first + slot[0]
ctx = tft_code.Context(offsets, {k[1] for k in where}, attr)
def section(lines, label=None): def section(lines, label=None):
if label: if label:
blob.extend(_chunk(label)) blob.extend(_chunk(label))
for ln in lines: for c in tft_code.compile_lines(lines, ctx):
blob.extend(_chunk(tft_code.compile_line(ln, ctx))) blob.extend(_chunk(c))
blob.extend(_chunk(b"")) blob.extend(_chunk(b""))
section(tft_code.statements(program.splitlines()) + ["page 0"]) section(program_lines + ["page 0"])
records, objects, table, rec_base = bytearray(), bytearray(), [], 0 records, objects, table, rec_base = bytearray(), bytearray(), [], 0
for pi, page in enumerate(pages): for pi, page in enumerate(pages):
ram = bytearray(b"\xff" * 8) ram = bytearray(b"\xff" * 8)
prec, first_obj, comp_names, init = bytearray(), len(objects) // tft.OBJ_SIZE, [], [] prec, first_obj, comp_names, init = bytearray(), len(objects) // tft.OBJ_SIZE, [], []
page_bco = 0 page_bco = 0
pre, post = [], []
current[0] = page.components[0].get("objname")
for ci, c in enumerate(page.components): for ci, c in enumerate(page.components):
a = {x.name: x for x in c.attrs()} a = {x.name: x for x in c.attrs()}
tid = _u(a["type"]) tid = _u(a["type"])
@@ -130,14 +169,16 @@ def compile_pages(pages: list[Page], program: str):
cid = _u(a["id"]) cid = _u(a["id"])
owner = struct.pack("<BB", pi, cid) if t["owner"] else b"\xff\xff" owner = struct.pack("<BB", pi, cid) if t["owner"] else b"\xff\xff"
slot0 = len(prec) // tft.REC_SIZE slot0 = len(prec) // tft.REC_SIZE
group = g + 4 + (4 if tid == 121 else len(ram)) # RAM address of the component's data
recs = [bytes(tft.REC_SIZE)] * t["slots"] recs = [bytes(tft.REC_SIZE)] * t["slots"]
amap = [0xFFFF] * 88 amap = [0xFFFF] * 88
for name, (slot, kind, size, hi, lo) in t["const"].items(): for name, (slot, kind, size, hi, lo) in t["const"].items():
recs[slot] = struct.pack("<Iiii2sHBBH", t["group"], _s(a[name], lo < 0), hi, lo, owner, 0, kind, size, 0) recs[slot] = struct.pack("<Iiii2sBBBBH", group, _s(a[name], lo < 0), hi, lo, owner, 0, 0, kind, size, 0)
amap[MAP[name]] = slot0 + slot amap[MAP[name]] = slot0 + slot
for name, (slot, kind, size, hi, lo) in t["ram"].items(): for name, (slot, kind, size, hi, lo) in t["ram"].items():
ram += bytes(-len(ram) % min(size, 4)) ram += bytes(-len(ram) % min(size, 4))
recs[slot] = struct.pack("<Iiii2sHBBH", t["group"], len(ram) + 4, hi, lo, owner, 0, kind, size, 0) recs[slot] = struct.pack("<Iiii2sBBBBH", group, g + 4 + len(ram), hi, lo, owner, 0,
t.get("action", {}).get(name, 0), kind, size, 0)
amap[MAP[name]] = slot0 + slot amap[MAP[name]] = slot0 + slot
if name == "bco" and tid == 121: if name == "bco" and tid == 121:
page_bco = rec_base + slot0 + slot page_bco = rec_base + slot0 + slot
@@ -153,7 +194,7 @@ def compile_pages(pages: list[Page], program: str):
if tid == 121: if tid == 121:
draw = len(blob) draw = len(blob)
blob += _chunk(fill) blob += _chunk(fill)
section(ev.get("codesload", [])) section([])
section([]) section([])
else: else:
section([]) section([])
@@ -161,29 +202,48 @@ def compile_pages(pages: list[Page], program: str):
erase = len(blob) erase = len(blob)
blob += _chunk(fill) blob += _chunk(fill)
section([]) section([])
if t["flag"] & 0x10: code_at = [0xFFFFFFFF] * 4 # down, up, timer, unload (object +0x0C..)
section(ev.get("codesdown", []), b"down") for i, (label, group_name, wanted) in enumerate((
section(ev.get("codesup", []), b"up") (b"down", "codesdown", t["flag"] & 0x10), (b"up", "codesup", t["flag"] & 0x10),
(None, None, False), (b"unload", "codesunload", tid == 121))):
if wanted:
if ev.get(group_name):
code_at[i] = len(blob) + 4 + len(label)
section(ev.get(group_name, []), label)
if tid == 121: if tid == 121:
section(ev.get("codesunload", []), b"unload") pre, post = ev.get("codesload", []), ev.get("codesloadend", [])
init = ev.get("codesloadend", []) if tid == 51:
# event: label, "S <1>", the event lines; registered by two init lines
blob += _chunk(b"timer")
code_at[2] = len(blob)
blob += _chunk(b"S \x03" + struct.pack("<I", 1))
first = rec_base + slot0
init.append(b"\x09\x22\x08\x01" + struct.pack("<I", first + 8) + (",%d," % cid).encode()
+ b"\x01" + struct.pack("<I", first + 7) + b",0")
init.append(b"L \x03" + struct.pack("<I", len(blob)))
section(ev.get("codestimer", []))
obj = bytearray(b"\xff" * tft.OBJ_SIZE) obj = bytearray(b"\xff" * tft.OBJ_SIZE)
struct.pack_into("<BBBBII", obj, 0, tid, cid, 0, t["flag"], draw, erase) struct.pack_into("<BBBBII", obj, 0, tid, cid, 0, t["flag"], draw, erase)
struct.pack_into("<IHBBI6HI", obj, 0x1C, t["group"], 0, 0x7F, 0, 0, *rect, start) struct.pack_into("<4I", obj, 0x0C, *code_at)
struct.pack_into("<IHBBI6HI", obj, 0x1C, group, 0, 0x7F, 0, 0, *rect, start)
struct.pack_into("<88H", obj, 0x38, *amap) struct.pack_into("<88H", obj, 0x38, *amap)
objects += obj objects += obj
comp_names.append((tft.name_hash(c.get("objname")), cid)) comp_names.append((tft.name_hash(c.get("objname")), cid))
names_off = len(blob) names_off = len(blob)
blob += _chunk(b"".join(struct.pack("<IH", h, i) for h, i in sorted(comp_names))) blob += _chunk(b"".join(struct.pack("<IH", h, i) for h, i in sorted(comp_names)))
blob += _chunk(bytes(ram) + bytes(8)) blob += _chunk(bytes(ram) + bytes(8))
for raw in init:
blob += _chunk(raw)
for c in tft_code.compile_lines(pre, ctx):
blob += _chunk(c)
blob += _chunk(b"\x09\x21\x08") blob += _chunk(b"\x09\x21\x08")
section(init) section(post)
table.append((first_obj, len(page.components), names_off, rec_base, len(ram))) table.append((first_obj, len(page.components), names_off, rec_base, len(ram)))
records += prec records += prec
rec_base += len(prec) // tft.REC_SIZE rec_base += len(prec) // tft.REC_SIZE
code_size = len(blob) rec_off = len(blob) + len(descriptors)
ptable = b"".join(struct.pack("<HHIII", f, cnt, no, code_size + rb * tft.REC_SIZE, rs) for f, cnt, no, rb, rs in table) ptable = b"".join(struct.pack("<HHIII", f, cnt, no, rec_off + rb * tft.REC_SIZE, rs) for f, cnt, no, rb, rs in table)
return bytes(blob), bytes(records), ptable, bytes(objects), program_off return bytes(blob), descriptors, bytes(records), ptable, bytes(objects), program_off
def build_tft(hmi: HMIFile, editor_dir: str = DEFAULT_EDITOR_DIR) -> bytes: def build_tft(hmi: HMIFile, editor_dir: str = DEFAULT_EDITOR_DIR) -> bytes:
@@ -197,10 +257,10 @@ def build_tft(hmi: HMIFile, editor_dir: str = DEFAULT_EDITOR_DIR) -> bytes:
raise Unsupported("display direction %d" % pm["direction"]) raise Unsupported("display direction %d" % pm["direction"])
w, h = project_size(hmi) w, h = project_size(hmi)
pages = [Page.parse(hmi.read(name)) for rtype, name in pm["resources"] if rtype == "pa"] pages = [Page.parse(hmi.read(name)) for rtype, name in pm["resources"] if rtype == "pa"]
blob, records, ptable, objects, program_off = compile_pages(pages, hmi.read("Program.s").decode("latin-1")) blob, descriptors, records, ptable, objects, program_off = compile_pages(pages, hmi.read("Program.s").decode("latin-1"))
area, info = resource_area(hmi, editor_dir) area, info = resource_area(hmi, editor_dir)
base = tft.RES_START + len(area) base = tft.RES_START + len(area)
body = blob + records + ptable + objects body = blob + descriptors + records + ptable + objects
body += b"\xff" * (-len(body) % 4) body += b"\xff" * (-len(body) % 4)
head = bytearray(b"\xff" * tft.RES_START) head = bytearray(b"\xff" * tft.RES_START)
head[:0x70] = bytes(0x70) head[:0x70] = bytes(0x70)
@@ -213,9 +273,9 @@ def build_tft(hmi: HMIFile, editor_dir: str = DEFAULT_EDITOR_DIR) -> bytes:
head[0x32:0x3C] = bytes.fromhex("2100000001000c000000") head[0x32:0x3C] = bytes.fromhex("2100000001000c000000")
struct.pack_into("<II", head, 0x3C, base + len(body) + 4, len(area)) struct.pack_into("<II", head, 0x3C, base + len(body) + 4, len(area))
head[0x4C:0x54] = b"\xff" * 8 head[0x4C:0x54] = b"\xff" * 8
struct.pack_into("<I", head, 0x58, len(blob) + len(records)) struct.pack_into("<I", head, 0x58, len(blob) + len(descriptors) + len(records))
pt = base + len(blob) + len(records) pt = base + len(blob) + len(descriptors) + len(records)
layout = [len(blob), len(blob), 0, len(blob), tft.RES_START, base, pt, pt + len(ptable), layout = [len(blob), len(blob), len(descriptors) // 12, len(blob) + len(descriptors), tft.RES_START, base, pt, pt + len(ptable),
tft.RES_START + info["pictures"], 0, 0, 0, tft.RES_START + info["fonts"], program_off, tft.RES_START + info["pictures"], 0, 0, 0, tft.RES_START + info["fonts"], program_off,
len(pages) | (len(objects) // tft.OBJ_SIZE) << 16, info["npics"], 0, info["nfonts"], 3] len(pages) | (len(objects) // tft.OBJ_SIZE) << 16, info["npics"], 0, info["nfonts"], 3]
for i, v in enumerate(layout): for i, v in enumerate(layout):
+249 -9
View File
@@ -1,4 +1,9 @@
"""Nextion script -> the tokenised form stored in a .tft (research stage; see docs/FORMAT.md).""" """Nextion script -> the tokenised lines ("chunks") stored in a .tft. See docs/FORMAT.md 7.6.
Only what was measured in Editor output is translated; everything else raises
:class:`Unsupported` (control flow -- if/while/for -- is compiled by the Editor into
conditional jumps with byte offsets and is not implemented).
"""
from __future__ import annotations from __future__ import annotations
import re import re
@@ -9,8 +14,36 @@ class Unsupported(Exception):
"""The project uses something whose compiled form has not been worked out yet.""" """The project uses something whose compiled form has not been worked out yet."""
# command -> (table, index): token 09 <index> <table>; table 4 = names up to 4 letters, 8 = longer
COMMANDS = {
"page": (4, 0x0B), "ref": (4, 0x03), "vis": (4, 0x05), "tsw": (4, 0x09), "get": (4, 0x07), "cls": (4, 0x06),
"rest": (4, 0x19), "fill": (4, 0x0D), "line": (4, 0x0C), "draw": (4, 0x1B), "cir": (4, 0x04), "cirs": (4, 0x14),
"click": (8, 0x00), "print": (8, 0x04), "printh": (8, 0x0B), "prints": (8, 0x0F), "touch_j": (8, 0x14),
"ref_stop": (8, 0x1D), "ref_star": (8, 0x1F), "code_c": (8, 0x06), "com_stop": (8, 0x1E), "com_star": (8, 0x20),
"randset": (8, 0x16), "doevents": (8, 0x21), "sendme": (8, 0x09),
}
# system variables and colour constants -> (table, index): token 04 <table> <u16 index> 00
SYSTEM = {
"dp": (4, 0x00), "RED": (4, 0x01), "thc": (4, 0x02), "dim": (4, 0x03), "GRAY": (4, 0x0C), "baud": (4, 0x0E),
"thsp": (4, 0x0F), "ussp": (4, 0x10), "dims": (4, 0x14), "spax": (4, 0x16), "spay": (4, 0x17),
"wup": (4, 0x04), "BLUE": (4, 0x0B), "rand": (4, 0x0D), "thup": (4, 0x11), "usup": (4, 0x12), "addr": (4, 0x13),
"WHITE": (8, 0x00), "BLACK": (8, 0x01), "GREEN": (8, 0x02), "BROWN": (8, 0x03), "thdra": (8, 0x04),
"bkcmd": (8, 0x06), "sleep": (8, 0x08), "bauds": (8, 0x0A), "delay": (8, 0x0B), "YELLOW": (8, 0x0C),
"recmod": (8, 0x0D), "sendxy": (8, 0x10),
}
RAW_ARGS = {"printh"} # arguments copied as text
_KEYWORDS = {"if", "else", "while", "for"}
_TOKEN = re.compile(r'"(?:[^"\\]|\\.)*"|(?:p\[\d+\]\.)?b\[\d+\]\.\w+|0x[0-9A-Fa-f]+|\d+|[A-Za-z_][A-Za-z0-9_]*(?:\.[A-Za-z_][A-Za-z0-9_]*)*|.', re.S)
class Context: class Context:
pass """Name resolution for one script: ``globals_`` (name -> offset), ``names`` (page and
component names that may appear as plain text), ``attr(path)`` -> global record number."""
def __init__(self, globals_=None, names=(), attr=None):
self.globals = dict(globals_ or {})
self.names = set(names)
self.attr = attr or (lambda path: None)
def group_lines(group) -> list[str]: def group_lines(group) -> list[str]:
@@ -18,18 +51,225 @@ def group_lines(group) -> list[str]:
def statements(lines) -> list[str]: def statements(lines) -> list[str]:
"""Script lines without comments, blank lines and surrounding spaces."""
out = [] out = []
for ln in lines: for ln in lines:
ln = ln.split("//")[0].strip() cut, quoted = len(ln), False
for i, ch in enumerate(ln):
if ch == '"':
quoted = not quoted
elif not quoted and ln.startswith("//", i):
cut = i
break
ln = ln[:cut].strip()
if ln: if ln:
out.append(ln) out.append(ln)
return out return out
def declarations(lines) -> tuple[dict, list[str]]:
"""Split Program.s statements into global ``int`` declarations (name -> initial value, in
order) and the remaining lines."""
decl, rest = {}, []
for ln in lines:
if ln.startswith("int "):
for part in ln[4:].split(","):
name, _, value = part.partition("=")
decl[name.strip()] = int(value.strip() or "0", 0)
else:
rest.append(ln)
return decl, rest
def _number(text: str, negative: bool) -> bytes | None:
"""Binary form (03 + i32) of a literal, or None when the Editor keeps it as text: decimal
numbers of up to three digits stay text."""
if text.lower().startswith("0x"):
v = int(text, 16)
elif len(text) >= 4:
v = int(text)
else:
return None
return b"\x03" + struct.pack("<I", (-v if negative else v) & 0xFFFFFFFF)
def _expr(text: str, ctx: Context) -> bytes:
out = bytearray()
prev = "" # previous token, for unary minus
for m in _TOKEN.finditer(text):
t = m.group(0)
if t[0] == '"':
out += t.encode("latin-1")
elif t[0].isdigit():
neg = out.endswith(b"-") and prev == "-"
b = _number(t, neg)
if b is None:
out += t.encode()
else:
if neg:
before = text[:m.start() - 1].rstrip()[-1:]
if before not in ("", "=", ",", "("):
raise Unsupported("number after a binary minus in %r" % text)
del out[-1]
out += b
elif "[" in t and len(t) > 1: # b[1].val, p[0].b[2].val stay text
out += t.encode()
elif t[0].isalpha() or t[0] == "_":
if t in _KEYWORDS:
raise Unsupported("control flow (%s)" % t)
if t in ctx.globals:
out += b"\x05" + struct.pack("<I", ctx.globals[t])
elif t in SYSTEM:
table, idx = SYSTEM[t]
out += struct.pack("<BBHB", 4, table, idx, 0)
elif "." in t:
rec = ctx.attr(t.split("."))
if rec is None:
raise Unsupported("attribute %r" % t)
out += b"\x01" + struct.pack("<I", rec)
elif t in ctx.names:
out += t.encode()
else:
raise Unsupported("name %r in %r" % (t, text))
elif t in "{}[]":
raise Unsupported("%r in %r" % (t, text))
else:
out += t.encode("latin-1")
prev = t
return bytes(out)
def compile_line(line: str, ctx: Context) -> bytes: def compile_line(line: str, ctx: Context) -> bytes:
if line == "baud=115200": word = re.match(r"[A-Za-z_][A-Za-z0-9_]*", line)
return b"\x04\x04\x0e\x00\x00=\x03" + struct.pack("<I", 115200) if word and word.group(0) in COMMANDS and line[word.end():word.end() + 1] in ("", " "):
m = re.fullmatch(r"page (\S+)", line) table, idx = COMMANDS[word.group(0)]
if m: args = line[word.end() + 1:]
return b"\x09\x0b\x04" + m.group(1).encode() body = args.encode("latin-1") if word.group(0) in RAW_ARGS else _expr(args, ctx)
raise Unsupported("script line %r" % line) return bytes([9, idx, table]) + body
return _expr(line, ctx)
# --- control flow -----------------------------------------------------------------------------
# The Editor compiles if/while/for into two kinds of chunk:
# 09 00 04 <lhs>,<rhs>,<op>,<03 i32 skip> continue when "lhs op rhs" holds, else jump
# "T " <03 i32 skip> jump always
# skip counts bytes (chunks with their 4-byte length) from the end of the jumping chunk.
COMPARE = {"==": 1, "<": 2, ">": 3, "<=": 4, ">=": 5, "!=": 6} # ">=" (5) is inferred, see NOTES
_INVERSE = {"==": "!=", "!=": "==", "<": ">=", ">=": "<", ">": "<=", "<=": ">"}
_HEAD = re.compile(r"(if|while|for|else if|else)\s*(?:\((.*)\))?$")
def _size(chunks) -> int:
return sum(4 + len(c) for c in chunks)
def _jump(skip: int) -> bytes:
return b"T \x03" + struct.pack("<i", skip)
def _test(term: str, op: str, skip: int, ctx: Context) -> bytes:
lhs, rhs = term
return (b"\x09\x00\x04" + _expr(lhs, ctx) + b"," + _expr(rhs, ctx) + (",%d," % COMPARE[op]).encode()
+ b"\x03" + struct.pack("<i", skip))
def _condition(text: str, body_size: int, ctx: Context) -> list[bytes]:
"""Chunks that fall through into the body when ``text`` holds and skip ``body_size`` bytes
otherwise. Terms may be joined by && or by || (not mixed, no parentheses)."""
if "&&" in text and "||" in text or "(" in text:
raise Unsupported("condition %r (mixed &&/|| or parentheses)" % text)
any_of = "||" in text
terms = []
for part in text.split("||" if any_of else "&&"):
m = re.fullmatch(r"(.+?)(==|!=|<=|>=|<|>)(.+)", part.strip())
if not m:
raise Unsupported("condition term %r" % part)
terms.append(((m.group(1).strip(), m.group(3).strip()), m.group(2)))
out = []
for i in range(len(terms) - 1, -1, -1): # back to front: later chunk sizes are known
term, op = terms[i]
if any_of and i < len(terms) - 1:
out.insert(0, _test(term, _INVERSE[op], _size(out), ctx)) # true -> jump to the body
else:
out.insert(0, _test(term, op, _size(out) + body_size, ctx)) # false -> jump past the body
return out
def _pieces(lines) -> list[str]:
out = []
for ln in lines:
while ln.startswith("}"):
out.append("}")
ln = ln[1:].strip()
tail = []
while ln.endswith("{"):
tail.append("{")
ln = ln[:-1].strip()
if ln:
out.append(ln)
out += tail
return out
def _block(p: list[str], i: int, ctx: Context) -> tuple[list[bytes], int]:
"""Compile statements from p[i] up to the matching '}' (or the end); returns (chunks, next index)."""
out = []
while i < len(p) and p[i] != "}":
m = _HEAD.match(p[i])
if not m or (m.group(1) != "else" and m.group(2) is None):
out.append(compile_line(p[i], ctx))
i += 1
continue
kind, arg = m.group(1), m.group(2)
if kind in ("else", "else if"):
raise Unsupported("else without if")
def body(j):
if j >= len(p) or p[j] != "{":
raise Unsupported("%s without braces" % kind)
chunks, j = _block(p, j + 1, ctx)
if j >= len(p):
raise Unsupported("missing }")
return chunks, j + 1
if kind == "if":
branches, tail = [], None
while True:
b, i = body(i + 1)
branches.append((arg, b))
nxt = _HEAD.match(p[i]) if i < len(p) else None
if nxt and nxt.group(1) == "else if":
arg = nxt.group(2)
continue
if nxt and nxt.group(1) == "else" and nxt.group(2) is None:
tail, i = body(i + 1)
break
rest = list(tail or []) # built back to front
last = tail is None
for cond, b in reversed(branches):
b = list(b) if last else b + [_jump(_size(rest))]
rest = _condition(cond, _size(b), ctx) + b + rest
last = False
out += rest
elif kind == "while":
b, i = body(i + 1)
cond = _condition(arg, _size(b) + 11, ctx)
out += cond + b + [_jump(-(_size(cond) + _size(b) + 11))]
else: # for(init;cond;step)
parts = [x.strip() for x in arg.split(";")]
if len(parts) != 3:
raise Unsupported("for(%s)" % arg)
b, i = body(i + 1)
b = b + [compile_line(parts[2], ctx)]
cond = _condition(parts[1], _size(b) + 11, ctx)
out += [compile_line(parts[0], ctx)] + cond + b + [_jump(-(_size(cond) + _size(b) + 11))]
return out, i
def compile_lines(lines, ctx: Context) -> list[bytes]:
"""Compile the statements of one event (or Program.s) into chunks."""
p = _pieces(statements(lines))
out, i = _block(p, 0, ctx)
if i != len(p):
raise Unsupported("unbalanced }")
return out
+113
View File
@@ -0,0 +1,113 @@
"""Script tokens of the .tft. Every expected value was read from Nextion Editor 1.68.2 output."""
import struct
import unittest
import helpers # noqa: F401 (adds src/ to the path)
from nxhmi import tft_code as T
def ctx():
records = {("va0", "val"): 0x44, ("page0", "bco"): 0x18, ("m0", "x"): 0x2A}
return T.Context({"sys0": 0, "sys1": 4, "sys2": 8}, {"m0", "page0", "va0"},
lambda path: records.get(tuple(path[-2:])))
def h(text):
return bytes.fromhex(text.replace(" ", ""))
SYS0, SYS1, SYS2 = h("05 00000000"), h("05 04000000"), h("05 08000000")
class Lines(unittest.TestCase):
def one(self, line):
return T.compile_line(line, ctx())
def test_system_variables(self):
self.assertEqual(self.one("dim=50"), h("04 04 0300 00") + b"=50")
self.assertEqual(self.one("bauds=9600"), h("04 08 0a00 00") + b"=" + h("03 80250000"))
self.assertEqual(self.one("thdra=1"), h("04 08 0400 00") + b"=1")
def test_numbers(self):
# up to three decimal digits stay text; longer and hex become 03 + i32
self.assertEqual(self.one("sys0=999"), SYS0 + b"=999")
self.assertEqual(self.one("sys0=1000"), SYS0 + b"=" + h("03 e8030000"))
self.assertEqual(self.one("sys0=-5"), SYS0 + b"=-5")
self.assertEqual(self.one("sys0=-1000"), SYS0 + b"=" + h("03 18fcffff"))
self.assertEqual(self.one("sys0=0x1F"), SYS0 + b"=" + h("03 1f000000"))
self.assertEqual(self.one("sys0=sys1+1000"), SYS0 + b"=" + SYS1 + b"+" + h("03 e8030000"))
def test_operators_stay_text(self):
self.assertEqual(self.one("sys0=sys1-sys2*3"), SYS0 + b"=" + SYS1 + b"-" + SYS2 + b"*3")
self.assertEqual(self.one("sys0=sys1<<2"), SYS0 + b"=" + SYS1 + b"<<2")
self.assertEqual(self.one("sys0+=2"), SYS0 + b"+=2")
def test_attribute_references(self):
self.assertEqual(self.one("va0.val++"), h("01 44000000") + b"++")
self.assertEqual(self.one("page0.va0.val=1"), h("01 44000000") + b"=1")
self.assertEqual(self.one("page0.bco=RED"), h("01 18000000") + b"=" + h("04 04 0100 00"))
self.assertEqual(self.one("b[1].x=5"), b"b[1].x=5")
self.assertEqual(self.one("p[0].b[2].val=7"), b"p[0].b[2].val=7")
def test_commands(self):
self.assertEqual(self.one("page 0"), h("09 0b 04") + b"0")
self.assertEqual(self.one("page page0"), h("09 0b 04") + b"page0")
self.assertEqual(self.one("click m0,1"), h("09 00 08") + b"m0,1")
self.assertEqual(self.one("get dim"), h("09 07 04") + h("04 04 0300 00"))
self.assertEqual(self.one("printh 0a 0d"), h("09 0b 08") + b"0a 0d")
self.assertEqual(self.one('print "abc"'), h("09 04 08") + b'"abc"')
self.assertEqual(self.one("print va0.val"), h("09 04 08") + h("01 44000000"))
self.assertEqual(self.one("cls 1024"), h("09 06 04") + h("03 00040000"))
self.assertEqual(self.one("cir 50,50,10,YELLOW"), h("09 04 04") + b"50,50,10," + h("04 08 0c00 00"))
self.assertEqual(self.one("doevents"), h("09 21 08"))
def test_unknown_things_are_refused(self):
for line in ("frobnicate 1", "sys0=nosuch", "t9.txt=\"x\"", "sys0=sys1-1000"):
with self.assertRaises(T.Unsupported):
self.one(line)
def test_statements_and_declarations(self):
st = T.statements(["// comment", "int sys0=0,sys1=0", " baud=115200 //x", 'print "a//b"', ""])
self.assertEqual(st, ["int sys0=0,sys1=0", "baud=115200", 'print "a//b"'])
self.assertEqual(T.declarations(st), ({"sys0": 0, "sys1": 0}, ["baud=115200", 'print "a//b"']))
class Flow(unittest.TestCase):
def cond(self, lhs, rhs, op, skip):
return h("09 00 04") + lhs + b"," + rhs + (",%d," % op).encode() + b"\x03" + struct.pack("<i", skip)
def jump(self, skip):
return b"T \x03" + struct.pack("<i", skip)
def test_if(self):
out = T.compile_lines(["if(sys0==1)", "{", " sys1=2", "}"], ctx())
self.assertEqual(out, [self.cond(SYS0, b"1", 1, 11), SYS1 + b"=2"])
def test_if_else(self):
out = T.compile_lines(["if(sys0!=1)", "{", " sys1=3", "}else", "{", " sys1=4", "}"], ctx())
self.assertEqual(out, [self.cond(SYS0, b"1", 6, 22), SYS1 + b"=3", self.jump(11), SYS1 + b"=4"])
def test_while_and_for(self):
out = T.compile_lines(["while(sys0<10)", "{", " sys0++", "}"], ctx())
self.assertEqual(out, [self.cond(SYS0, b"10", 2, 22), SYS0 + b"++", self.jump(-45)])
out = T.compile_lines(["for(sys0=0;sys0<10;sys0++)", "{", " sys1--", "}"], ctx())
self.assertEqual(out, [SYS0 + b"=0", self.cond(SYS0, b"10", 2, 33), SYS1 + b"--", SYS0 + b"++",
self.jump(-56)])
def test_and_or_else_if(self):
src = ["if(sys0>1&&sys1<2)", "{", " sys2=1", "}else if(sys0>=3||sys1<=4)", "{", " sys2=2", "}else", "{",
" sys2=3", "}"]
self.assertEqual(T.compile_lines(src, ctx()), [
self.cond(SYS0, b"1", 3, 44), self.cond(SYS1, b"2", 2, 22), SYS2 + b"=1", self.jump(77),
self.cond(SYS0, b"3", 2, 22), # first || term is inverted: true jumps to the body
self.cond(SYS1, b"4", 4, 22), SYS2 + b"=2", self.jump(11), SYS2 + b"=3"])
def test_refused(self):
for src in (["if(sys0==1)", "sys1=2"], ["}"], ["if(sys0==1&&sys1==2||sys2==3)", "{", "}"], ["else", "{", "}"]):
with self.assertRaises(T.Unsupported):
T.compile_lines(src, ctx())
if __name__ == "__main__":
unittest.main()