Flipdot-display/src/esp32/README.md

# ESP32: controller

PlatformIO environments: `esp32` (OTA) and `esp32_usb` (USB). Tests: `test_esp32`.

The ESP32 decides what is shown on the display. It never touches the panel itself. The
BROSE9323 driver is built with `-D ESPSERIAL`, so every GFX call is sent to the Nano over
`Serial2` (UART2, GPIO 17 TX / GPIO 16 RX, 115200 baud). `Serial` (USB) is only used for
debug output. Wiring is described in the [main README](../../README.md#wiring-between-esp32-and-nano).

## Files

| File | Contents |
|------|----------|
| `main.cpp` | `setup()` and `loop()`: the order of things at boot and in the loop |
| `app.*` | The objects (display, Nano link, controller, ...) and the actions the web UI uses: settings, page, live mode |
| `config.h` | Fixed settings: pins, timings, timezone, Home Assistant device id |
| `credentials.h` | Includes `secrets.h` (WiFi and MQTT credentials, **gitignored**); template: `secrets.example.h` |
| **`logic/`** | **Pure logic without Arduino code, tested on the PC** |
| `logic/display_controller.*` | Decides what is shown and when the display is powered: page, live mode, schedule, Home Assistant switches (`DisplayController`) |
| `logic/settings.*` | The settings of the web UI with their defaults and checks |
| `logic/schedule.*` | When the display is on (night, clock, standby), from the settings |
| `logic/bitmap_format.*` | 84 x 19 bitmap format, hex encoding, names of bitmaps, pages and fonts |
| `logic/page.*` | Dynamic pages: the struct, checks, time slots, texts of the elements (clock, date, Home Assistant value, text) |
| `logic/page_render.*` | Draws a dynamic page into a bitmap (display and web previews) |
| `logic/font.*` | Dot font format (built-in and uploaded fonts), text width, drawing |
| `logic/builtin_fonts.*` | Small, Normal, Large, Full height; the `.cpp` is generated by `tools/make_builtin_fonts.py` |
| `logic/builtin_pages.*` | The built-in Clock page (the clock of the old firmware) |
| `logic/power_timer.*` | Countdown to switching the display off |
| `logic/status_led.h` | Blink pattern of the built-in LED |
| `logic/boot_policy.*` | How to start after a reset: normal, OTA-only window, safe mode |
| **`system/`** | **Keeping OTA alive** (`system_store` and `boot_target` are shared with the recovery app) |
| `system/firmware_guard.*` | Rollback of bad OTA firmware, watchdog, crash counter, power-cycle trigger, status for Home Assistant |
| `system/boot_target.*` | Switching between the clock firmware and the recovery app |
| `system/system_store.*` | Settings in NVS: WiFi, recovery reason, firmware slot and version, power-on count |
| `system/settings_store.*` | The web UI settings in NVS |
| **`display/`** | **What is drawn** |
| `display/frames.*` | The screens: dynamic page, "No WiFi", page bitmap, live image, off image (`FlipdotFrames`) |
| `display/bitmap_store.*` | Built-in bitmaps and the uploaded ones on LittleFS |
| `display/page_store.*` | Built-in and stored dynamic pages (`/pages/<name>.pg`) |
| `display/font_store.*` | Built-in and uploaded fonts (`/fonts/<name>.fnt`) |
| `display/page_resources.*` | Gives the page renderer its bitmaps, fonts and Home Assistant values |
| `display/status_screen.*` | Two-line status messages at boot, after a crash and in the recovery app (`StatusScreen`) |
| `display/text.*` | Centered text lines on the 84 x 19 display |
| `display/bitmaps.h` | 1-bit bitmaps (Nintendo logo, Pikachu with the thought bubble, ...) |
| **`nano_link/`** | **Connection to the Nano** |
| `nano_link/nano_link.*` | Relay and serial link: power on (waits for `READY`), power off, bootloader start (`NanoLink`) |
| `nano_link/nano_flash_bridge.*` | WiFi to serial bridge for flashing the Nano over WiFi (TCP port 2323) |
| **`net/`** | **Network** |
| `net/wifi_ota.*` | WiFi (with a restart when it stays down), ESP32 OTA updates, OTA-only mode |
| `net/home_assistant.*` | MQTT and the Home Assistant entities (power switch, mode and off image dropdowns, IP and firmware sensors, recovery button) |
| `net/ha_api.*` | Reads Home Assistant entity values over the REST API for the dynamic pages |
| **`web/`** | **Web UI** (see the [main README](../../README.md#web-ui)) |
| `web/web_ui.*` | Web server and JSON API (the endpoints are listed in `web_ui.h`) |
| `web/page_json.*` | Dynamic pages as JSON |
| `web/web_page.h` | The page: HTML and JavaScript; the browser converts images and draws the previews |

`DisplayController` only talks to two small interfaces, `DisplayPower` (implemented by
`NanoLink`) and `Frames` (implemented by `FlipdotFrames`). The tests replace both with
fakes.

## Keeping OTA alive

The ESP32 must stay reachable over OTA whatever the firmware does, so it never needs USB
again. Several layers take care of that:

| Problem | What happens |
|---------|--------------|
| A new OTA firmware crashes, hangs or can't connect to WiFi | **Rollback.** A new firmware is only marked as good after it has run for 60 s with WiFi. If it restarts before that, the bootloader starts the previous firmware again. |
| The firmware hangs in `loop()` | **Watchdog.** If `loop()` doesn't run for 30 s, the ESP32 restarts. That counts as a crash. |
| The firmware crashes once or twice | **OTA window.** After a crash the ESP32 runs only WiFi and OTA for 15 s (LED blinks fast), then starts the main code again. |
| The firmware keeps crashing (3 times in a row) | **Recovery app** for 10 minutes (see the [main README](../../README.md#recovery-app)). Then the main code gets one more try; one more crash brings the recovery app back. Running 5 minutes without a crash clears the counter. Without the recovery app installed: only WiFi and OTA run, for 10 minutes. |
| WiFi doesn't connect at boot (30 s), or stays down for 10 minutes | **Recovery app**, which keeps trying and opens the **FlipDot-Setup** WiFi with a setup page to choose another WiFi. Without the recovery app: restart. |
| The firmware runs, but its OTA is broken | **Recovery app** through the Home Assistant button, or by switching the power on and off 3 times quickly. |

So in every case you can flash a fix with `pio run -e esp32 -t upload`, at the latest
after waiting for a safe-mode cycle.

The **FlipDot Display Firmware** sensor in Home Assistant shows the state, e.g.
`2.1.0, ok, 0 crashes`:

| State | Meaning |
|-------|---------|
| `testing` | New OTA firmware, not yet marked as good (first 60 s) |
| `ok` | Firmware marked as good |
| `ok, no rollback in bootloader` | The bootloader on the ESP32 can't roll back. The other layers still work. The one-time USB install of the recovery app also installs a bootloader that can. |
| `usb` | Flashed over USB, so there is no previous firmware to roll back to |

Rules for changing the code:

- Keep `otaLoop()` and `firmwareGuardLoop()` at the top of `loop()`.
- Never block `loop()` for 30 s or longer.
- Change `FIRMWARE_VERSION` in `config.h` with each release, so the sensor shows which
  version is running.

## Features

- **Status on the display:** `Connecting to WiFi...` and `Getting the time...` at boot,
  `Crashed!` messages in the OTA-only modes, and `No WiFi` with the time when WiFi is down
  for more than a minute. See the table in the
  [main README](../../README.md#recovery-app).

- **Frames only when something changes:** a new frame is sent when the minute changes, the
  screen changes (clock, page, off image) or the Nano was just powered on. The serial line is
  quiet in between, which also gives the Nano time for its periodic full redraw.
- **OTA updates** through ArduinoOTA. Hostname: `FlipDotDisplay`.
- **Robust OTA:** rollback, watchdog, OTA window after a crash and safe mode, see
  [Keeping OTA alive](#keeping-ota-alive).
- **NTP time** with timezone `CET-1CEST,M3.5.0,M10.5.0/3` (Europe/Amsterdam). Until the
  time is synced, the display shows "Connecting...".
- **Home Assistant** through MQTT (`home-assistant-integration` library). The switch and
  the dropdown report their new state back, so Home Assistant always shows the real state.
  - `FlipDot Display Power` switch: off = clear the display and power it off after 1 s
  - `FlipDot Display Mode` dropdown: the page shown during the day, `Clock` or a bitmap.
    The options follow the bitmaps saved in the web UI (`net/ha_page_select.*`, because
    ArduinoHA's `HASelect` can't change its options).
  - `FlipDot Display Off image` dropdown: `Blank` or a bitmap, drawn instead of a blank
    screen at night and when the Power switch turns the display off
  - `FlipDot Display IP` sensor: the IP address

  The old `FlipDot Display Clock` switch is removed from Home Assistant on connect (empty
  retained discovery config). Plus the **FlipDot Display Firmware** sensor (see
  [Keeping OTA alive](#keeping-ota-alive)) and the **FlipDot Display Recovery** button,
  which restarts into the recovery app for 10 minutes.
- **Relay on GPIO 13** switches the power for both the panels and the Nano. `LOW` turns it
  on.
  - **On:** the ESP waits for the Nano's `READY` message, at most 3 s
    (`NANO_BOOT_TIMEOUT`), before it sends frames.
  - **Off:** the ESP releases its TX pin (`Serial2.end()` and the pin set to input). An
    idle UART line is high and would otherwise power the Nano through its RX pin.
- **Nano flash bridge:** `pio run -e nano_ota -t upload` flashes the Nano through the ESP32.
  While a flash session runs, the clock is paused. See the
  [main README](../../README.md#flashing-the-nano-over-wifi).
- **Status LED** (built-in): off while the display is off, slow blink with WiFi, fast blink
  without WiFi.

## Schedule

| Time | Display |
|------|---------|
| 08:00 – 22:00 | The page; the built-in Clock page shows Pikachu and the clock (yawning from 17:00), updated every minute |
| 22:00 – 23:30 | Clock shown only during the minutes :00, :15, :30 and :45. In between the power is off, and the last time stays visible (flipdots keep their image without power). |
| 23:31 – 08:00 | Screen cleared, power off after 30 s |

The times are in `logic/schedule.cpp`.

## Tests

`pio test -e test_esp32` runs [test/test_esp32](../../test/test_esp32/test_main.cpp) on
the PC: the schedule boundaries, the power timer (including `millis()` overflow), the LED
pattern, the boot policy (crash counting, safe mode, rollback timing) and the display
controller (power on/off, redraws, both Home Assistant switches).

## Adding bitmaps

The easy way is the web UI (Draw / Live tab): load an image, adjust it, save it.

To build a bitmap into the firmware instead: convert it with
[image2cpp](https://javl.github.io/image2cpp/) to "Arduino code", horizontal, 1 bit per
pixel, 84 x 19. Paste the array into `display/bitmaps.h` and add it to `BUILTIN` in
`display/bitmap_store.cpp`.

## Live view of the display

The BROSE9323 driver keeps a shadow copy of what was sent to the Nano (`shownPixel()`),
updated on every `display()`. `GET /api/screen` returns it, and the web UI polls it. It
also keeps the last image while the display is switched off, like the dots do.