tft: script tokens, global variables, timers, control flow

build-tft now reproduces 35 Editor-made samples byte for byte: assignments,
arithmetic, 27 commands, system variables and colours, attribute references,
Program.s globals, all page/hotspot/timer events, if/else/while/for.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This commit is contained in:
mangavalkandClaude Fable 5.2 committed 2026-11-16 18:23:14 +01:00
1 parent f66c902443
commit 99a5240af0
7 files changed
+511 -50

No files matched your search

+1 -1
View File
@@ -32,7 +32,7 @@ Every percentage is a count with a stated basis, so it can be checked (details i
| Readable attribute names in JSON | **100 %** (31/31 types) | every value list the Editor documents, incl. the Intelligent-only types and load effects |
| Rendering, Basic components | **100 %** (20/20 pixel-exact) | incl. gauges and QR codes (own encoder, no dependencies) |
| Rendering, all components | **94 %** (29/31 pixel-exact) | Intelligent ComboBox and Switch are drawn, but their anti-aliased edges are approximate (~99 % / ~92 % of pixels exact) |
| `.tft` compilation | **69 %** (5.5/8 milestones) | `nxhmi build-tft`: byte-identical to the Editor for NX4832T035 projects with pages, hotspots, int variables, pictures and fonts; other component types and most script not yet |
| `.tft` compilation | **81 %** (6.5/8 milestones) | `nxhmi build-tft`: byte-identical to the Editor for NX4832T035 projects with pages, hotspots, int variables, timers, pictures, fonts and script (assignments, arithmetic, 27 commands, if/else/while/for, global variables); drawing component types not yet; never tried on a display |
## Compatibility
+2 -2
View File
@@ -113,8 +113,8 @@ byte for byte, or a file written by nxhmi was opened in the Editor and showed th
| Trailer checksum | solved for any length (`tft.trailer`); one model measured |
| 3 header checksums (0x44, 0xC4, 0x18C) | solved (`tft.header_checksums`, `tft.seal`) |
| Pictures and fonts inside the TFT | solved (FORMAT.md 7.2) |
| Compiled page format | structure solved (code blob, name hashes, RAM image, attribute records, page table, object records -- FORMAT.md 7.3-7.5); per-type tables done for page, hotspot, int variable; the other types need their slot tables and generated draw scripts |
| Event-code tokens | encoding understood (command / system-variable / attribute-reference / number tokens); table being filled |
| Compiled page format | structure solved (code blob, name hashes, RAM image, attribute records, page table, object records -- FORMAT.md 7.3-7.5); per-type tables done for page, hotspot, int variable, timer; the other types need their slot tables and generated draw scripts |
| Event-code tokens | solved for what was sampled (FORMAT.md 7.6): 27 commands, 33 system variables/colours, numbers, global variables, attribute references, if/else/while/for. Not sampled: string functions and attributes, the remaining commands and system variables, `>=` op code |
| Builder (`nxhmi build-tft`) | byte-identical to the Editor for the supported subset (one model: NX4832T035); refuses anything else. **Never tested on a real display** |
| Licensing | every `.tft` embeds Nextion's firmware files; the repo must never ship them -- a builder would read them from the user's own Editor installation |
+45 -10
View File
@@ -394,8 +394,8 @@ program
per page: code blocks of its components, name list, RAM image, init code
```
The first ten words describe three tables at 4 + (0x1C, 0x20, 0x24) with sizes 4, 4, 6*pages;
the first two are empty when the program declares no global variables (**OPEN**: their format).
The first eight words describe three tables at 4 + offset: table 0 (4 zero bytes, **OPEN**),
table 1 (global variables, see 7.6; 4 zero bytes without globals) and the page names.
**Name hash** = CRC of the name NUL-padded to 14 bytes. Names are not stored anywhere else.
@@ -406,7 +406,7 @@ tokenised script line; an empty chunk ends a section.
`page 0` added by the Editor, then the empty chunk.
- Component block (the page itself is component 0):
- draw section -- lines, empty chunk. Page: `fill x,y,w,h,<bco>` (command `09 0D 04`, colour
as a reference to the page's `bco` record), then the Preinitialize event lines.
as a reference to the page's `bco` record).
Types that draw themselves (progress bar, gauge, QR code, text ...) have their own script
here, generated from the attributes (**OPEN** per type).
- erase section (types with a rectangle only; empty for the page) -- `fill x,y,w,h,<page bco>`,
@@ -415,8 +415,8 @@ tokenised script line; an empty chunk ends a section.
lines, empty chunk. Page: down, up, unload. Hotspot: down, up. Variable: none.
- After the last component of a page: one chunk with the name list `(u32 hash, u16 id)` sorted
by hash; one chunk with the RAM image followed by 8 zero bytes; type-specific init lines
(timer, slider, touch cap -- **OPEN**); the chunk `09 21 08`; the page's Postinitialize lines;
empty chunk.
(timer: 7.6; slider, touch cap -- **OPEN**); the page's Preinitialize lines; the chunk
`09 21 08`; the page's Postinitialize lines; empty chunk.
RAM image: 8 x FF, then every RAM-stored attribute of every component in component order,
each aligned to its size, each component padded to 4. For a page: `sta` (u8), pad, `bco` (u16).
@@ -430,11 +430,11 @@ in the object records.
| Offset | Field |
|---|---|
| 0 | u32 8 for a page, 16 for other components |
| 4 | i32 value (constant attributes) or 4 + offset into the RAM image |
| 0 | u32 RAM address of the component's data (page: of its second word) |
| 4 | i32 value (constant attributes) or RAM address of the value |
| 8, 12 | i32 maximum, minimum |
| 16 | u8 page number, u8 component id of the component to redraw on change; FF FF = none |
| 18 | u16 change action (0 = none; timers use 1 for `tim`, 2 for `en`) |
| 18 | u8 0, u8 change action (0 = none; timers use 1 for `tim`, 2 for `en`) |
| 20 | u8 kind: 0x01 constant; 0x0B/0x0C/0x0D/0x0E x/y/w/h; 0x08 endx/endy; 0x21, 0x22, 0x31, 0x32, 0x39, 0x3F RAM-stored |
| 21 | u8 size in bytes |
| 22 | u16 0 |
@@ -453,9 +453,44 @@ Object:
| 0 | u8 type, u8 id, u8 0, u8 flags (0x37 visible types, 0x27 timer/touch cap, 0x07 variable) |
| 4 | u32 offset of the draw section, FFFFFFFF if empty |
| 8 | u32 offset of the erase section, FFFFFFFF if none/empty |
| 0x0C-0x1B | more code offsets (0x14: timer event), FF when unused |
| 0x1C | u32 8 / 16 as in the records |
| 0x0C | u32 x 4: offset of the first line of the down, up, timer, unload event; FFFFFFFF when the event is empty |
| 0x1C | u32 RAM address as in the records |
| 0x20 | `00 00 7F 00`, u32 0 |
| 0x28 | u16 x, y, w, h, endx, endy (0,0,1,1,0,0 for invisible types) |
| 0x34 | u32 offset of the component's code block |
| 0x38 | 88 x u16: record number (page-relative) of well-known attributes, FFFF = absent. Positions: 1 vscope, 2 sta, 4 bco, 16 pco, 20 val, 42 x, 43 y, 44 endx, 45 endy, 46 w, 47 h, 49 id, 62 type (**OPEN**: the others) |
### 7.6 Script tokens
A script line is stored as text with some parts replaced by binary tokens
(`nxhmi.tft_code`; tables there):
| Token | Meaning |
|---|---|
| `09 ii tt` | a command at the start of a line; `tt` = 04 for names of up to 4 letters, 08 for longer ones, `ii` = index in that table (`page` 0B/04, `print` 04/08 ...). The arguments follow, tokenised like an expression (`printh`: plain text) |
| `04 tt iiii 00` | system variable or colour constant, same two tables (`dim` 0003/04, `WHITE` 0000/08 ...) |
| `05` + u32 | global variable declared in Program.s: its offset in the global RAM (4 bytes each, in declaration order) |
| `01` + u32 | component attribute (`va0.val`, `page0.va0.val`): global attribute record number |
| `03` + i32 | number: hex literals and decimal literals of four or more digits (a leading minus is folded in); shorter decimals stay text |
Operators, commas, quotes, component and page names used as command arguments (`ref m0`,
`page page0`) and the array forms `b[1].x`, `p[0].b[2].val` stay text.
`int a=0,b=5` lines in Program.s produce no chunk: the values go into header table 1 of the code
blob (u32 each, then 4 zero bytes), and one 12-byte descriptor per variable `(CRC of the bare
name, offset, 9)`, sorted by hash, sits between the code blob and the attribute records (layout
word 2 = count, word 3 = offset of the attribute records). All RAM addresses in attribute
records start after this global RAM: address = 4 + 4*globals + offset in the page RAM image.
Control flow becomes jumps; `skip` counts bytes (chunks including their length words) from the
end of the jumping chunk and may be negative:
- `09 00 04 <lhs>,<rhs>,<op>,<03 skip>` -- continue when `lhs op rhs` holds, otherwise jump.
`op`: 1 `==`, 2 `<`, 3 `>`, 4 `<=`, 6 `!=`; 5 `>=` is inferred (**OPEN**).
- `T ` + `03 skip` -- jump always (end of an `if` body that has an `else`; end of a loop body).
- `a&&b`: one test per term, each skipping to the else part. `a||b`: every term but the last is
inverted and skips to the body.
- `for(init;cond;step)` = init line, tests, body, step line, jump back.
Timers: the event section is the label `timer`, a chunk `S ` + `03 1`, the event lines; the
page's init code gets `09 22 08 <en>,<id>,<tim>,0` and `L ` + `03 <offset of the first event line>`.
+16 -3
View File
@@ -152,6 +152,19 @@ Useful facts not obvious from the format description:
- Record references in scripts (`01` + u32) are global record numbers, so adding a component
renumbers every reference after it.
Not done: script tokens beyond `baud=115200` / `page N`; timers, sliders and every drawing
component; global variables declared in Program.s (blob header tables 0 and 1); other models,
portrait, other series; and nothing has been loaded onto a display.
Script (same day): tokens, global variables, timers and control flow are in (FORMAT.md 7.6,
`nxhmi.tft_code`); 35 Editor-made samples are reproduced byte for byte. Things to know:
- A token table entry exists only for names that were compiled once. To add names: put them in
a project, compile, read the token from the chunk (the analysis was a 30-line script printing
source line next to chunk).
- `>=` was never the plain operator in a sample (only as the inverse of `<` in an `||`); its
code 5 is the gap in 1..6. Verify before relying on it.
- Unverified and therefore refused: negative global initial values, `x-1000` (binary minus
before a long number), mixed `&&`/`||`, parentheses in conditions, braces-less `if`.
- The Editor appends one `page 0` chunk to the program even when Program.s already ends with it.
Not done: sliders, touch cap and every drawing component (text, number, button, progress bar,
gauge, waveform, picture, crop, checkbox, radio, QR code ...); string variables and string
functions; page backgrounds other than a solid colour; global (vscope) components; other
models, portrait, other series; and nothing has been loaded onto a display.
+85 -25
View File
@@ -14,6 +14,7 @@ import struct
from . import tft, tft_code
from .container import HMIFile
from .page import Page
from .checksum import word_per_byte_crc
from .tft_code import Unsupported
FIRMWARE_FILES = ("input.bin", "asp0.dll", "cd0.dll", "qr0.bin", "syscom.bin", "h0-0.dll", "h1-0.dll")
@@ -22,7 +23,7 @@ SUPPORTED_MODEL = 0xB23BC018 # NX4832T035_011, the only model measured
BLOCK = 0x10000
# attribute map positions in the object record (u16 table at +0x38)
MAP = {"vscope": 1, "sta": 2, "bco": 4, "val": 20, "x": 42, "y": 43, "endx": 44, "endy": 45, "w": 46, "h": 47,
MAP = {"vscope": 1, "sta": 2, "bco": 4, "val": 20, "tim": 33, "en": 34, "x": 42, "y": 43, "endx": 44, "endy": 45, "w": 46, "h": 47,
"id": 49, "type": 62}
# attribute record templates: name -> (slot, kind, size, max, min)
_COMMON = {"type": (0, 0x01, 1, 255, 0), "id": (1, 0x01, 1, 255, 0), "vscope": (3, 0x01, 1, 1, 0)}
@@ -37,6 +38,8 @@ TYPES = {
52: dict(name="variable", slots=11, flag=0x07, group=16,
const={**_COMMON, "sta": (7, 0x01, 1, 1, 0)},
ram={"val": (8, 0x39, 4, 2147483647, -2147483648)}, owner=False),
51: dict(name="timer", slots=11, flag=0x27, group=16, const=dict(_COMMON),
ram={"tim": (7, 0x32, 2, 65534, 50), "en": (8, 0x31, 1, 1, 0)}, owner=True, action={"tim": 1, "en": 2}),
}
@@ -93,28 +96,64 @@ def resource_area(hmi: HMIFile, editor_dir: str) -> tuple[bytes, dict]:
def compile_pages(pages: list[Page], program: str):
"""-> (code blob, attribute records, page table, object records, program offset)."""
"""-> (code blob, global variable descriptors, attribute records, page table, object records,
program offset)."""
names = sorted((tft.name_hash(p.components[0].get("objname")), i) for i, p in enumerate(pages))
n = len(pages)
blob = bytearray(struct.pack("<10I", 0x24 + 6 * n, 3, 0x1C, 4, 0x20, 4, 0x24, 6 * n, 0, 0))
decl, program_lines = tft_code.declarations(tft_code.statements(program.splitlines()))
if any(v < 0 for v in decl.values()):
raise Unsupported("global variables with a negative initial value")
g = 4 * len(decl)
blob = bytearray(struct.pack("<8I", 0x24 + g + 6 * n, 3, 0x1C, 4, 0x20, 4 + g, 0x24 + g, 6 * n))
blob += bytes(4) + b"".join(struct.pack("<I", v) for v in decl.values()) + bytes(4)
for h, i in names:
blob += struct.pack("<IH", h, i)
program_off = len(blob)
ctx = tft_code.Context()
offsets = {name: 4 * i for i, name in enumerate(decl)}
descriptors = b"".join(struct.pack("<III", h, o, 9) for h, o in
sorted((word_per_byte_crc(name.encode("latin-1")), o) for name, o in offsets.items()))
# record numbers are needed before any script is compiled: number them first
where, base_rec = {}, 0 # (page name, component name) -> (first record, type)
for page in pages:
pname = page.components[0].get("objname")
for c in page.components:
tid = _u(c.attr("type"))
if tid not in TYPES:
raise Unsupported("component type %d (%s)" % (tid, c.get("objname")))
where[(pname, c.get("objname"))] = (base_rec, TYPES[tid])
base_rec += TYPES[tid]["slots"]
current = [None]
def attr(path):
if len(path) == 3:
key, name = (path[0], path[1]), path[2]
elif len(path) == 2:
key, name = (current[0], path[0]), path[1]
else:
return None
if key not in where:
return None
first, t = where[key]
slot = {**t["const"], **t["ram"]}.get(name)
return None if slot is None else first + slot[0]
ctx = tft_code.Context(offsets, {k[1] for k in where}, attr)
def section(lines, label=None):
if label:
blob.extend(_chunk(label))
for ln in lines:
blob.extend(_chunk(tft_code.compile_line(ln, ctx)))
for c in tft_code.compile_lines(lines, ctx):
blob.extend(_chunk(c))
blob.extend(_chunk(b""))
section(tft_code.statements(program.splitlines()) + ["page 0"])
section(program_lines + ["page 0"])
records, objects, table, rec_base = bytearray(), bytearray(), [], 0
for pi, page in enumerate(pages):
ram = bytearray(b"\xff" * 8)
prec, first_obj, comp_names, init = bytearray(), len(objects) // tft.OBJ_SIZE, [], []
page_bco = 0
pre, post = [], []
current[0] = page.components[0].get("objname")
for ci, c in enumerate(page.components):
a = {x.name: x for x in c.attrs()}
tid = _u(a["type"])
@@ -130,14 +169,16 @@ def compile_pages(pages: list[Page], program: str):
cid = _u(a["id"])
owner = struct.pack("<BB", pi, cid) if t["owner"] else b"\xff\xff"
slot0 = len(prec) // tft.REC_SIZE
group = g + 4 + (4 if tid == 121 else len(ram)) # RAM address of the component's data
recs = [bytes(tft.REC_SIZE)] * t["slots"]
amap = [0xFFFF] * 88
for name, (slot, kind, size, hi, lo) in t["const"].items():
recs[slot] = struct.pack("<Iiii2sHBBH", t["group"], _s(a[name], lo < 0), hi, lo, owner, 0, kind, size, 0)
recs[slot] = struct.pack("<Iiii2sBBBBH", group, _s(a[name], lo < 0), hi, lo, owner, 0, 0, kind, size, 0)
amap[MAP[name]] = slot0 + slot
for name, (slot, kind, size, hi, lo) in t["ram"].items():
ram += bytes(-len(ram) % min(size, 4))
recs[slot] = struct.pack("<Iiii2sHBBH", t["group"], len(ram) + 4, hi, lo, owner, 0, kind, size, 0)
recs[slot] = struct.pack("<Iiii2sBBBBH", group, g + 4 + len(ram), hi, lo, owner, 0,
t.get("action", {}).get(name, 0), kind, size, 0)
amap[MAP[name]] = slot0 + slot
if name == "bco" and tid == 121:
page_bco = rec_base + slot0 + slot
@@ -153,7 +194,7 @@ def compile_pages(pages: list[Page], program: str):
if tid == 121:
draw = len(blob)
blob += _chunk(fill)
section(ev.get("codesload", []))
section([])
section([])
else:
section([])
@@ -161,29 +202,48 @@ def compile_pages(pages: list[Page], program: str):
erase = len(blob)
blob += _chunk(fill)
section([])
if t["flag"] & 0x10:
section(ev.get("codesdown", []), b"down")
section(ev.get("codesup", []), b"up")
code_at = [0xFFFFFFFF] * 4 # down, up, timer, unload (object +0x0C..)
for i, (label, group_name, wanted) in enumerate((
(b"down", "codesdown", t["flag"] & 0x10), (b"up", "codesup", t["flag"] & 0x10),
(None, None, False), (b"unload", "codesunload", tid == 121))):
if wanted:
if ev.get(group_name):
code_at[i] = len(blob) + 4 + len(label)
section(ev.get(group_name, []), label)
if tid == 121:
section(ev.get("codesunload", []), b"unload")
init = ev.get("codesloadend", [])
pre, post = ev.get("codesload", []), ev.get("codesloadend", [])
if tid == 51:
# event: label, "S <1>", the event lines; registered by two init lines
blob += _chunk(b"timer")
code_at[2] = len(blob)
blob += _chunk(b"S \x03" + struct.pack("<I", 1))
first = rec_base + slot0
init.append(b"\x09\x22\x08\x01" + struct.pack("<I", first + 8) + (",%d," % cid).encode()
+ b"\x01" + struct.pack("<I", first + 7) + b",0")
init.append(b"L \x03" + struct.pack("<I", len(blob)))
section(ev.get("codestimer", []))
obj = bytearray(b"\xff" * tft.OBJ_SIZE)
struct.pack_into("<BBBBII", obj, 0, tid, cid, 0, t["flag"], draw, erase)
struct.pack_into("<IHBBI6HI", obj, 0x1C, t["group"], 0, 0x7F, 0, 0, *rect, start)
struct.pack_into("<4I", obj, 0x0C, *code_at)
struct.pack_into("<IHBBI6HI", obj, 0x1C, group, 0, 0x7F, 0, 0, *rect, start)
struct.pack_into("<88H", obj, 0x38, *amap)
objects += obj
comp_names.append((tft.name_hash(c.get("objname")), cid))
names_off = len(blob)
blob += _chunk(b"".join(struct.pack("<IH", h, i) for h, i in sorted(comp_names)))
blob += _chunk(bytes(ram) + bytes(8))
for raw in init:
blob += _chunk(raw)
for c in tft_code.compile_lines(pre, ctx):
blob += _chunk(c)
blob += _chunk(b"\x09\x21\x08")
section(init)
section(post)
table.append((first_obj, len(page.components), names_off, rec_base, len(ram)))
records += prec
rec_base += len(prec) // tft.REC_SIZE
code_size = len(blob)
ptable = b"".join(struct.pack("<HHIII", f, cnt, no, code_size + rb * tft.REC_SIZE, rs) for f, cnt, no, rb, rs in table)
return bytes(blob), bytes(records), ptable, bytes(objects), program_off
rec_off = len(blob) + len(descriptors)
ptable = b"".join(struct.pack("<HHIII", f, cnt, no, rec_off + rb * tft.REC_SIZE, rs) for f, cnt, no, rb, rs in table)
return bytes(blob), descriptors, bytes(records), ptable, bytes(objects), program_off
def build_tft(hmi: HMIFile, editor_dir: str = DEFAULT_EDITOR_DIR) -> bytes:
@@ -197,10 +257,10 @@ def build_tft(hmi: HMIFile, editor_dir: str = DEFAULT_EDITOR_DIR) -> bytes:
raise Unsupported("display direction %d" % pm["direction"])
w, h = project_size(hmi)
pages = [Page.parse(hmi.read(name)) for rtype, name in pm["resources"] if rtype == "pa"]
blob, records, ptable, objects, program_off = compile_pages(pages, hmi.read("Program.s").decode("latin-1"))
blob, descriptors, records, ptable, objects, program_off = compile_pages(pages, hmi.read("Program.s").decode("latin-1"))
area, info = resource_area(hmi, editor_dir)
base = tft.RES_START + len(area)
body = blob + records + ptable + objects
body = blob + descriptors + records + ptable + objects
body += b"\xff" * (-len(body) % 4)
head = bytearray(b"\xff" * tft.RES_START)
head[:0x70] = bytes(0x70)
@@ -213,9 +273,9 @@ def build_tft(hmi: HMIFile, editor_dir: str = DEFAULT_EDITOR_DIR) -> bytes:
head[0x32:0x3C] = bytes.fromhex("2100000001000c000000")
struct.pack_into("<II", head, 0x3C, base + len(body) + 4, len(area))
head[0x4C:0x54] = b"\xff" * 8
struct.pack_into("<I", head, 0x58, len(blob) + len(records))
pt = base + len(blob) + len(records)
layout = [len(blob), len(blob), 0, len(blob), tft.RES_START, base, pt, pt + len(ptable),
struct.pack_into("<I", head, 0x58, len(blob) + len(descriptors) + len(records))
pt = base + len(blob) + len(descriptors) + len(records)
layout = [len(blob), len(blob), len(descriptors) // 12, len(blob) + len(descriptors), tft.RES_START, base, pt, pt + len(ptable),
tft.RES_START + info["pictures"], 0, 0, 0, tft.RES_START + info["fonts"], program_off,
len(pages) | (len(objects) // tft.OBJ_SIZE) << 16, info["npics"], 0, info["nfonts"], 3]
for i, v in enumerate(layout):
+249 -9
View File
@@ -1,4 +1,9 @@
"""Nextion script -> the tokenised form stored in a .tft (research stage; see docs/FORMAT.md)."""
"""Nextion script -> the tokenised lines ("chunks") stored in a .tft. See docs/FORMAT.md 7.6.
Only what was measured in Editor output is translated; everything else raises
:class:`Unsupported` (control flow -- if/while/for -- is compiled by the Editor into
conditional jumps with byte offsets and is not implemented).
"""
from __future__ import annotations
import re
@@ -9,8 +14,36 @@ class Unsupported(Exception):
"""The project uses something whose compiled form has not been worked out yet."""
# command -> (table, index): token 09 <index> <table>; table 4 = names up to 4 letters, 8 = longer
COMMANDS = {
"page": (4, 0x0B), "ref": (4, 0x03), "vis": (4, 0x05), "tsw": (4, 0x09), "get": (4, 0x07), "cls": (4, 0x06),
"rest": (4, 0x19), "fill": (4, 0x0D), "line": (4, 0x0C), "draw": (4, 0x1B), "cir": (4, 0x04), "cirs": (4, 0x14),
"click": (8, 0x00), "print": (8, 0x04), "printh": (8, 0x0B), "prints": (8, 0x0F), "touch_j": (8, 0x14),
"ref_stop": (8, 0x1D), "ref_star": (8, 0x1F), "code_c": (8, 0x06), "com_stop": (8, 0x1E), "com_star": (8, 0x20),
"randset": (8, 0x16), "doevents": (8, 0x21), "sendme": (8, 0x09),
}
# system variables and colour constants -> (table, index): token 04 <table> <u16 index> 00
SYSTEM = {
"dp": (4, 0x00), "RED": (4, 0x01), "thc": (4, 0x02), "dim": (4, 0x03), "GRAY": (4, 0x0C), "baud": (4, 0x0E),
"thsp": (4, 0x0F), "ussp": (4, 0x10), "dims": (4, 0x14), "spax": (4, 0x16), "spay": (4, 0x17),
"wup": (4, 0x04), "BLUE": (4, 0x0B), "rand": (4, 0x0D), "thup": (4, 0x11), "usup": (4, 0x12), "addr": (4, 0x13),
"WHITE": (8, 0x00), "BLACK": (8, 0x01), "GREEN": (8, 0x02), "BROWN": (8, 0x03), "thdra": (8, 0x04),
"bkcmd": (8, 0x06), "sleep": (8, 0x08), "bauds": (8, 0x0A), "delay": (8, 0x0B), "YELLOW": (8, 0x0C),
"recmod": (8, 0x0D), "sendxy": (8, 0x10),
}
RAW_ARGS = {"printh"} # arguments copied as text
_KEYWORDS = {"if", "else", "while", "for"}
_TOKEN = re.compile(r'"(?:[^"\\]|\\.)*"|(?:p\[\d+\]\.)?b\[\d+\]\.\w+|0x[0-9A-Fa-f]+|\d+|[A-Za-z_][A-Za-z0-9_]*(?:\.[A-Za-z_][A-Za-z0-9_]*)*|.', re.S)
class Context:
pass
"""Name resolution for one script: ``globals_`` (name -> offset), ``names`` (page and
component names that may appear as plain text), ``attr(path)`` -> global record number."""
def __init__(self, globals_=None, names=(), attr=None):
self.globals = dict(globals_ or {})
self.names = set(names)
self.attr = attr or (lambda path: None)
def group_lines(group) -> list[str]:
@@ -18,18 +51,225 @@ def group_lines(group) -> list[str]:
def statements(lines) -> list[str]:
"""Script lines without comments, blank lines and surrounding spaces."""
out = []
for ln in lines:
ln = ln.split("//")[0].strip()
cut, quoted = len(ln), False
for i, ch in enumerate(ln):
if ch == '"':
quoted = not quoted
elif not quoted and ln.startswith("//", i):
cut = i
break
ln = ln[:cut].strip()
if ln:
out.append(ln)
return out
def declarations(lines) -> tuple[dict, list[str]]:
"""Split Program.s statements into global ``int`` declarations (name -> initial value, in
order) and the remaining lines."""
decl, rest = {}, []
for ln in lines:
if ln.startswith("int "):
for part in ln[4:].split(","):
name, _, value = part.partition("=")
decl[name.strip()] = int(value.strip() or "0", 0)
else:
rest.append(ln)
return decl, rest
def _number(text: str, negative: bool) -> bytes | None:
"""Binary form (03 + i32) of a literal, or None when the Editor keeps it as text: decimal
numbers of up to three digits stay text."""
if text.lower().startswith("0x"):
v = int(text, 16)
elif len(text) >= 4:
v = int(text)
else:
return None
return b"\x03" + struct.pack("<I", (-v if negative else v) & 0xFFFFFFFF)
def _expr(text: str, ctx: Context) -> bytes:
out = bytearray()
prev = "" # previous token, for unary minus
for m in _TOKEN.finditer(text):
t = m.group(0)
if t[0] == '"':
out += t.encode("latin-1")
elif t[0].isdigit():
neg = out.endswith(b"-") and prev == "-"
b = _number(t, neg)
if b is None:
out += t.encode()
else:
if neg:
before = text[:m.start() - 1].rstrip()[-1:]
if before not in ("", "=", ",", "("):
raise Unsupported("number after a binary minus in %r" % text)
del out[-1]
out += b
elif "[" in t and len(t) > 1: # b[1].val, p[0].b[2].val stay text
out += t.encode()
elif t[0].isalpha() or t[0] == "_":
if t in _KEYWORDS:
raise Unsupported("control flow (%s)" % t)
if t in ctx.globals:
out += b"\x05" + struct.pack("<I", ctx.globals[t])
elif t in SYSTEM:
table, idx = SYSTEM[t]
out += struct.pack("<BBHB", 4, table, idx, 0)
elif "." in t:
rec = ctx.attr(t.split("."))
if rec is None:
raise Unsupported("attribute %r" % t)
out += b"\x01" + struct.pack("<I", rec)
elif t in ctx.names:
out += t.encode()
else:
raise Unsupported("name %r in %r" % (t, text))
elif t in "{}[]":
raise Unsupported("%r in %r" % (t, text))
else:
out += t.encode("latin-1")
prev = t
return bytes(out)
def compile_line(line: str, ctx: Context) -> bytes:
if line == "baud=115200":
return b"\x04\x04\x0e\x00\x00=\x03" + struct.pack("<I", 115200)
m = re.fullmatch(r"page (\S+)", line)
if m:
return b"\x09\x0b\x04" + m.group(1).encode()
raise Unsupported("script line %r" % line)
word = re.match(r"[A-Za-z_][A-Za-z0-9_]*", line)
if word and word.group(0) in COMMANDS and line[word.end():word.end() + 1] in ("", " "):
table, idx = COMMANDS[word.group(0)]
args = line[word.end() + 1:]
body = args.encode("latin-1") if word.group(0) in RAW_ARGS else _expr(args, ctx)
return bytes([9, idx, table]) + body
return _expr(line, ctx)
# --- control flow -----------------------------------------------------------------------------
# The Editor compiles if/while/for into two kinds of chunk:
# 09 00 04 <lhs>,<rhs>,<op>,<03 i32 skip> continue when "lhs op rhs" holds, else jump
# "T " <03 i32 skip> jump always
# skip counts bytes (chunks with their 4-byte length) from the end of the jumping chunk.
COMPARE = {"==": 1, "<": 2, ">": 3, "<=": 4, ">=": 5, "!=": 6} # ">=" (5) is inferred, see NOTES
_INVERSE = {"==": "!=", "!=": "==", "<": ">=", ">=": "<", ">": "<=", "<=": ">"}
_HEAD = re.compile(r"(if|while|for|else if|else)\s*(?:\((.*)\))?$")
def _size(chunks) -> int:
return sum(4 + len(c) for c in chunks)
def _jump(skip: int) -> bytes:
return b"T \x03" + struct.pack("<i", skip)
def _test(term: str, op: str, skip: int, ctx: Context) -> bytes:
lhs, rhs = term
return (b"\x09\x00\x04" + _expr(lhs, ctx) + b"," + _expr(rhs, ctx) + (",%d," % COMPARE[op]).encode()
+ b"\x03" + struct.pack("<i", skip))
def _condition(text: str, body_size: int, ctx: Context) -> list[bytes]:
"""Chunks that fall through into the body when ``text`` holds and skip ``body_size`` bytes
otherwise. Terms may be joined by && or by || (not mixed, no parentheses)."""
if "&&" in text and "||" in text or "(" in text:
raise Unsupported("condition %r (mixed &&/|| or parentheses)" % text)
any_of = "||" in text
terms = []
for part in text.split("||" if any_of else "&&"):
m = re.fullmatch(r"(.+?)(==|!=|<=|>=|<|>)(.+)", part.strip())
if not m:
raise Unsupported("condition term %r" % part)
terms.append(((m.group(1).strip(), m.group(3).strip()), m.group(2)))
out = []
for i in range(len(terms) - 1, -1, -1): # back to front: later chunk sizes are known
term, op = terms[i]
if any_of and i < len(terms) - 1:
out.insert(0, _test(term, _INVERSE[op], _size(out), ctx)) # true -> jump to the body
else:
out.insert(0, _test(term, op, _size(out) + body_size, ctx)) # false -> jump past the body
return out
def _pieces(lines) -> list[str]:
out = []
for ln in lines:
while ln.startswith("}"):
out.append("}")
ln = ln[1:].strip()
tail = []
while ln.endswith("{"):
tail.append("{")
ln = ln[:-1].strip()
if ln:
out.append(ln)
out += tail
return out
def _block(p: list[str], i: int, ctx: Context) -> tuple[list[bytes], int]:
"""Compile statements from p[i] up to the matching '}' (or the end); returns (chunks, next index)."""
out = []
while i < len(p) and p[i] != "}":
m = _HEAD.match(p[i])
if not m or (m.group(1) != "else" and m.group(2) is None):
out.append(compile_line(p[i], ctx))
i += 1
continue
kind, arg = m.group(1), m.group(2)
if kind in ("else", "else if"):
raise Unsupported("else without if")
def body(j):
if j >= len(p) or p[j] != "{":
raise Unsupported("%s without braces" % kind)
chunks, j = _block(p, j + 1, ctx)
if j >= len(p):
raise Unsupported("missing }")
return chunks, j + 1
if kind == "if":
branches, tail = [], None
while True:
b, i = body(i + 1)
branches.append((arg, b))
nxt = _HEAD.match(p[i]) if i < len(p) else None
if nxt and nxt.group(1) == "else if":
arg = nxt.group(2)
continue
if nxt and nxt.group(1) == "else" and nxt.group(2) is None:
tail, i = body(i + 1)
break
rest = list(tail or []) # built back to front
last = tail is None
for cond, b in reversed(branches):
b = list(b) if last else b + [_jump(_size(rest))]
rest = _condition(cond, _size(b), ctx) + b + rest
last = False
out += rest
elif kind == "while":
b, i = body(i + 1)
cond = _condition(arg, _size(b) + 11, ctx)
out += cond + b + [_jump(-(_size(cond) + _size(b) + 11))]
else: # for(init;cond;step)
parts = [x.strip() for x in arg.split(";")]
if len(parts) != 3:
raise Unsupported("for(%s)" % arg)
b, i = body(i + 1)
b = b + [compile_line(parts[2], ctx)]
cond = _condition(parts[1], _size(b) + 11, ctx)
out += [compile_line(parts[0], ctx)] + cond + b + [_jump(-(_size(cond) + _size(b) + 11))]
return out, i
def compile_lines(lines, ctx: Context) -> list[bytes]:
"""Compile the statements of one event (or Program.s) into chunks."""
p = _pieces(statements(lines))
out, i = _block(p, 0, ctx)
if i != len(p):
raise Unsupported("unbalanced }")
return out
+113
View File
@@ -0,0 +1,113 @@
"""Script tokens of the .tft. Every expected value was read from Nextion Editor 1.68.2 output."""
import struct
import unittest
import helpers # noqa: F401 (adds src/ to the path)
from nxhmi import tft_code as T
def ctx():
records = {("va0", "val"): 0x44, ("page0", "bco"): 0x18, ("m0", "x"): 0x2A}
return T.Context({"sys0": 0, "sys1": 4, "sys2": 8}, {"m0", "page0", "va0"},
lambda path: records.get(tuple(path[-2:])))
def h(text):
return bytes.fromhex(text.replace(" ", ""))
SYS0, SYS1, SYS2 = h("05 00000000"), h("05 04000000"), h("05 08000000")
class Lines(unittest.TestCase):
def one(self, line):
return T.compile_line(line, ctx())
def test_system_variables(self):
self.assertEqual(self.one("dim=50"), h("04 04 0300 00") + b"=50")
self.assertEqual(self.one("bauds=9600"), h("04 08 0a00 00") + b"=" + h("03 80250000"))
self.assertEqual(self.one("thdra=1"), h("04 08 0400 00") + b"=1")
def test_numbers(self):
# up to three decimal digits stay text; longer and hex become 03 + i32
self.assertEqual(self.one("sys0=999"), SYS0 + b"=999")
self.assertEqual(self.one("sys0=1000"), SYS0 + b"=" + h("03 e8030000"))
self.assertEqual(self.one("sys0=-5"), SYS0 + b"=-5")
self.assertEqual(self.one("sys0=-1000"), SYS0 + b"=" + h("03 18fcffff"))
self.assertEqual(self.one("sys0=0x1F"), SYS0 + b"=" + h("03 1f000000"))
self.assertEqual(self.one("sys0=sys1+1000"), SYS0 + b"=" + SYS1 + b"+" + h("03 e8030000"))
def test_operators_stay_text(self):
self.assertEqual(self.one("sys0=sys1-sys2*3"), SYS0 + b"=" + SYS1 + b"-" + SYS2 + b"*3")
self.assertEqual(self.one("sys0=sys1<<2"), SYS0 + b"=" + SYS1 + b"<<2")
self.assertEqual(self.one("sys0+=2"), SYS0 + b"+=2")
def test_attribute_references(self):
self.assertEqual(self.one("va0.val++"), h("01 44000000") + b"++")
self.assertEqual(self.one("page0.va0.val=1"), h("01 44000000") + b"=1")
self.assertEqual(self.one("page0.bco=RED"), h("01 18000000") + b"=" + h("04 04 0100 00"))
self.assertEqual(self.one("b[1].x=5"), b"b[1].x=5")
self.assertEqual(self.one("p[0].b[2].val=7"), b"p[0].b[2].val=7")
def test_commands(self):
self.assertEqual(self.one("page 0"), h("09 0b 04") + b"0")
self.assertEqual(self.one("page page0"), h("09 0b 04") + b"page0")
self.assertEqual(self.one("click m0,1"), h("09 00 08") + b"m0,1")
self.assertEqual(self.one("get dim"), h("09 07 04") + h("04 04 0300 00"))
self.assertEqual(self.one("printh 0a 0d"), h("09 0b 08") + b"0a 0d")
self.assertEqual(self.one('print "abc"'), h("09 04 08") + b'"abc"')
self.assertEqual(self.one("print va0.val"), h("09 04 08") + h("01 44000000"))
self.assertEqual(self.one("cls 1024"), h("09 06 04") + h("03 00040000"))
self.assertEqual(self.one("cir 50,50,10,YELLOW"), h("09 04 04") + b"50,50,10," + h("04 08 0c00 00"))
self.assertEqual(self.one("doevents"), h("09 21 08"))
def test_unknown_things_are_refused(self):
for line in ("frobnicate 1", "sys0=nosuch", "t9.txt=\"x\"", "sys0=sys1-1000"):
with self.assertRaises(T.Unsupported):
self.one(line)
def test_statements_and_declarations(self):
st = T.statements(["// comment", "int sys0=0,sys1=0", " baud=115200 //x", 'print "a//b"', ""])
self.assertEqual(st, ["int sys0=0,sys1=0", "baud=115200", 'print "a//b"'])
self.assertEqual(T.declarations(st), ({"sys0": 0, "sys1": 0}, ["baud=115200", 'print "a//b"']))
class Flow(unittest.TestCase):
def cond(self, lhs, rhs, op, skip):
return h("09 00 04") + lhs + b"," + rhs + (",%d," % op).encode() + b"\x03" + struct.pack("<i", skip)
def jump(self, skip):
return b"T \x03" + struct.pack("<i", skip)
def test_if(self):
out = T.compile_lines(["if(sys0==1)", "{", " sys1=2", "}"], ctx())
self.assertEqual(out, [self.cond(SYS0, b"1", 1, 11), SYS1 + b"=2"])
def test_if_else(self):
out = T.compile_lines(["if(sys0!=1)", "{", " sys1=3", "}else", "{", " sys1=4", "}"], ctx())
self.assertEqual(out, [self.cond(SYS0, b"1", 6, 22), SYS1 + b"=3", self.jump(11), SYS1 + b"=4"])
def test_while_and_for(self):
out = T.compile_lines(["while(sys0<10)", "{", " sys0++", "}"], ctx())
self.assertEqual(out, [self.cond(SYS0, b"10", 2, 22), SYS0 + b"++", self.jump(-45)])
out = T.compile_lines(["for(sys0=0;sys0<10;sys0++)", "{", " sys1--", "}"], ctx())
self.assertEqual(out, [SYS0 + b"=0", self.cond(SYS0, b"10", 2, 33), SYS1 + b"--", SYS0 + b"++",
self.jump(-56)])
def test_and_or_else_if(self):
src = ["if(sys0>1&&sys1<2)", "{", " sys2=1", "}else if(sys0>=3||sys1<=4)", "{", " sys2=2", "}else", "{",
" sys2=3", "}"]
self.assertEqual(T.compile_lines(src, ctx()), [
self.cond(SYS0, b"1", 3, 44), self.cond(SYS1, b"2", 2, 22), SYS2 + b"=1", self.jump(77),
self.cond(SYS0, b"3", 2, 22), # first || term is inverted: true jumps to the body
self.cond(SYS1, b"4", 4, 22), SYS2 + b"=2", self.jump(11), SYS2 + b"=3"])
def test_refused(self):
for src in (["if(sys0==1)", "sys1=2"], ["}"], ["if(sys0==1&&sys1==2||sys2==3)", "{", "}"], ["else", "{", "}"]):
with self.assertRaises(T.Unsupported):
T.compile_lines(src, ctx())
if __name__ == "__main__":
unittest.main()