Multi-byte fonts: GB2312, Big5, Shift-JIS, KS C 5601, UTF-8

Font Creator output decoded (lead/trail ranges and gaps in the header, map codes = bytes as
little-endian u16 or Unicode code points) and round-tripped byte for byte; from_ttf(chars=)
builds them; the renderer splits text like the simulator (any byte >= 0x81 starts a pair,
missing characters advance one space) -- pixel-exact for all five encodings.
Intelligent glyph data is 8-byte aligned (was 4). CLI add-font --chars/--chars-file, MCP chars.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
mangavalkandClaude Opus 5.5 committed 2026-11-16 13:42:50 +01:00
1 parent c102157ac2
commit 0138a0ba56
11 files changed
+137 -26

No files matched your search

+1 -2
View File
@@ -14,7 +14,7 @@ byte for byte, or a file written by nxhmi was opened in the Editor and showed th
| Components | **31 / 31 types** (20 Basic + 11 Intelligent-only) read/write/create |
| Readable JSON (enum names, colours) | done for every value list the Editor documents |
| Pictures | **complete**: format 0 (Basic) and format 4 (Intelligent, with alpha; run-length or JPEG for quality < 100), all display directions |
| Fonts | **complete** for single-byte encodings: black/white and the anti-aliasing flag, overhanging glyphs, all display directions |
| Fonts | **complete**: single- and multi-byte encodings (GB2312, Big5, Shift-JIS, KS C 5601, UTF-8), black/white and the anti-aliasing flag, overhanging glyphs, all display directions; generation from TrueType for all of them |
| Rendering pages like the Editor | **pixel-identical** for every Basic component and 9 of 11 Intelligent-only ones; ComboBox and Switch approximate at anti-aliased edges |
| New projects from scratch | `project.new_project(model)` for 12 Basic/Intelligent models; the generated demo is byte-identical to the Editor's re-save |
| Compiling `.tft` firmware | **research only** -- not possible yet |
@@ -93,7 +93,6 @@ byte for byte, or a file written by nxhmi was opened in the Editor and showed th
### `.HMI` side
| Item | Notes |
|---|---|
| Multi-byte encodings (GB2312, Big5, ...) | only single-byte fonts studied |
| Other display series | Discovery / Enhanced / Edge not sampled; only Basic + Intelligent templates |
| Display direction | byte 12 decoded and written (`new_project(portrait=...)`); picture order verified for all four (format 4); font glyph order for 90° only |
| Some header fields | main.HMI bytes 8, 13, 36, 0x41; page header 21/23; entry bytes 25-27 |
+26 -5
View File
@@ -266,12 +266,12 @@ Character map: 10 bytes per character: u16 code, u8 glyph advance width, u8 extr
left, u8 extra columns on the right (glyphs that overhang, e.g. `j`, `f`, `W`; the stored glyph is
advance + left + right wide and drawn starting `left` px before the pen position),
u24 glyph offset (relative to the map start), u16 glyph length. In Intelligent projects the glyph
data starts 4-byte aligned (zero padding after the map); Basic fonts have no padding.
data starts at a multiple of 8 from the map start (zero padding after the map); Basic fonts have no padding.
Glyph: mode byte (0x01 = black/white), then run-length codes over the pixels in the same order
as pictures for the project's display direction (runs continue across rows/columns): at 90° column
by column, bottom to top; at 0° row by row. When an Intelligent project is compiled, the Editor
converts a 90° font to the project direction; our encoder reproduces that conversion byte for byte.
as pictures for the project's display direction (runs continue across rows/columns): at 90° column
by column, bottom to top; at 0° row by row. When an Intelligent project is compiled, the Editor
converts a 90° font to the project direction; our encoder reproduces that conversion byte for byte.
| Code | Meaning |
|---|---|
@@ -286,7 +286,28 @@ converts a 90
The Editor encodes greedily (transparent run + 1..4 ink if possible, else the 3+3-bit form, else a
plain run); our encoder reproduces a 224-glyph Editor font byte for byte. The v3/v5 write-ups in
github.com/hagronnestad/nextion-font-editor were a useful starting point; v6 differs (scan order,
B/W codes `10`/`11`). **OPEN**: multi-byte encodings, the remaining unknown header bytes.
B/W codes `10`/`11`). **OPEN**: the remaining unknown header bytes (0x1F, 0x20).
Multi-byte fonts (Font Creator "Specified character" range; verified for GB2312, Big5, Shift-JIS,
KS C 5601 and UTF-8, round-trip byte-identical, rendered pixel-exact in the simulator):
| Header | Meaning | GB2312 | Big5 | Shift-JIS | KS C 5601 | UTF-8 |
|---|---|---|---|---|---|---|
| 1, 2 | end of the first trail-byte range, trail codes skipped after it | FF 00 | 7E 22 | 7E 01 | FF 00 | FF 00 |
| 4 | encoding (main.HMI numbering) | 2 | 16 | 23 | 15 | 24 |
| 5 | 2 = multi-byte (0 for single-byte fonts) | 2 | 2 | 2 | 2 | 2 |
| 6 | width (0 for multi-byte) | 0 | 0 | 0 | 0 | 0 |
| 8, 9 | lead-byte range | A1 F7 | A0 F9 | 81 EF | A1 C8 | FF FF |
| 10, 11 | trail-byte range | A1 FE | 40 FE | 40 FC | A1 FE | 00 FF |
| 0x1C, 0x1D | end of the first lead range, lead codes skipped | FF 00 | FF 00 | 9F 40 | FF 00 | FF 00 |
| 0x20 | ? | 4 | 5 | 5 | 4 | 5 |
Map codes: double-byte encodings store the character's two bytes as a little-endian u16 (GB2312
中 = bytes D6 D0 -> 0xD0D6); UTF-8 fonts store the Unicode code point. The map is sorted by
code. The display splits text into characters by taking every byte >= 0x81 together with the
next byte (not limited to header 8-9: KS C hanja leads above C8 still pair; Shift-JIS half-width
katakana pair up as well, so they cannot be shown); a character without a glyph advances by the
width of the space glyph.
## 6. Not yet covered
+2
View File
@@ -74,6 +74,8 @@ TextSelects, several of each other type):
blended as `bg + round((src - bg) * alpha / 255)` per channel. Exact against the Intelligent
simulator (342 samples). The Editor *canvas* blends slightly differently (up to 3 levels off at
high alpha); the renderer follows the simulator, i.e. the display.
- Text in multi-byte encodings: every byte >= 0x81 starts a two-byte character; characters the
font lacks advance by the space glyph's width (measured in the Intelligent simulator).
- Text never starts above/left of its box: centring clamps at 0 (a 32-px font in a 30-px box
starts at the top).
- Text is laid out inside the border: 1 px for 3D styles, `borderw` for style "border".