Keep ESP32 OTA reachable whatever the firmware does

- Rollback: override verifyRollbackLater() so a new OTA image stays
  pending until it ran 60 s with WiFi; any earlier restart rolls back
  (the Arduino core marked every image valid at boot before)
- Task watchdog (30 s) on loop(), paused during OTA uploads
- Crash counter in RTC memory: OTA-only window after a crash, 10 min
  safe mode (WiFi + OTA only) after 3 crashes in a row
- Restart when WiFi stays down for 10 min
- Home Assistant sensor 'FlipDot Display Firmware' with version/state
- logic/boot_policy with tests (60 tests total)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
mangavalkandClaude Opus 5.5 committed 2026-10-13 01:05:06 +02:00
1 parent 683a47c055
commit ddbdf7e65f
14 files changed
+517 -42

No files matched your search

+41 -5
View File
@@ -19,6 +19,9 @@ debug output. Wiring is described in the [main README](../../README.md#wiring-be
| `logic/schedule.*` | When the clock is shown (night, clock, standby) |
| `logic/power_timer.*` | Countdown to switching the display off |
| `logic/status_led.h` | Blink pattern of the built-in LED |
| `logic/boot_policy.*` | How to start after a reset: normal, OTA-only window, safe mode |
| **`system/`** | **Keeping OTA alive** |
| `system/firmware_guard.*` | Rollback of bad OTA firmware, watchdog, crash counter, status for Home Assistant |
| **`display/`** | **What is drawn** |
| `display/frames.*` | The screens: clock, logo, blank, text (`FlipdotFrames`) |
| `display/bitmaps.h` | 1-bit bitmaps (Pikachu, thought bubble, Nintendo logo, ...) |
@@ -26,22 +29,54 @@ debug output. Wiring is described in the [main README](../../README.md#wiring-be
| `nano_link/nano_link.*` | Relay and serial link: power on (waits for `READY`), power off, bootloader start (`NanoLink`) |
| `nano_link/nano_flash_bridge.*` | WiFi to serial bridge for flashing the Nano over WiFi (TCP port 2323) |
| **`net/`** | **Network** |
| `net/wifi_ota.*` | WiFi, ESP32 OTA updates and the OTA panic window |
| `net/wifi_ota.*` | WiFi (with a restart when it stays down), ESP32 OTA updates, OTA-only mode |
| `net/home_assistant.*` | MQTT and the two Home Assistant switches |
`DisplayController` only talks to two small interfaces, `DisplayPower` (implemented by
`NanoLink`) and `Frames` (implemented by `FlipdotFrames`). The tests replace both with
fakes.
## Keeping OTA alive
The ESP32 must stay reachable over OTA whatever the firmware does, so it never needs USB
again. Several layers take care of that:
| Problem | What happens |
|---------|--------------|
| A new OTA firmware crashes, hangs or can't connect to WiFi | **Rollback.** A new firmware is only marked as good after it has run for 60 s with WiFi. If it restarts before that, the bootloader starts the previous firmware again. |
| The firmware hangs in `loop()` | **Watchdog.** If `loop()` doesn't run for 30 s, the ESP32 restarts. That counts as a crash. |
| The firmware crashes once or twice | **OTA window.** After a crash the ESP32 runs only WiFi and OTA for 15 s (LED blinks fast), then starts the main code again. |
| The firmware keeps crashing (3 times in a row) | **Safe mode.** Only WiFi and OTA run, for 10 minutes. Then the main code gets one more try; one more crash brings safe mode back. Running 5 minutes without a crash clears the counter. |
| WiFi stays down | After 10 minutes without WiFi, the ESP32 restarts, in case the WiFi stack is stuck. |
So in every case you can flash a fix with `pio run -e esp32 -t upload`, at the latest
after waiting for a safe-mode cycle.
The **FlipDot Display Firmware** sensor in Home Assistant shows the state, e.g.
`2.1.0, ok, 0 crashes`:
| State | Meaning |
|-------|---------|
| `testing` | New OTA firmware, not yet marked as good (first 60 s) |
| `ok` | Firmware marked as good |
| `ok, no rollback in bootloader` | The bootloader on the ESP32 can't roll back. The other layers still work. |
| `usb` | Flashed over USB, so there is no previous firmware to roll back to |
Rules for changing the code:
- Keep `otaLoop()` and `firmwareGuardLoop()` at the top of `loop()`.
- Never block `loop()` for 30 s or longer.
- Change `FIRMWARE_VERSION` in `config.h` with each release, so the sensor shows which
version is running.
## Features
- **Frames only when something changes:** a new frame is sent when the minute changes, the
screen changes (clock, logo, blank) or the Nano was just powered on. The serial line is
quiet in between, which also gives the Nano time for its periodic full redraw.
- **OTA updates** through ArduinoOTA. Hostname: `FlipDotDisplay`.
- **OTA panic window:** after a crash or watchdog reset, the ESP32 runs only the OTA
handler for 15 seconds before it starts the main code, with the LED blinking fast. A
firmware that crashes on boot can still be replaced over the air.
- **Robust OTA:** rollback, watchdog, OTA window after a crash and safe mode, see
[Keeping OTA alive](#keeping-ota-alive).
- **NTP time** with timezone `CET-1CEST,M3.5.0,M10.5.0/3` (Europe/Amsterdam). Until the
time is synced, the display shows "Connecting...".
- **Home Assistant** through MQTT (`home-assistant-integration` library) with two switches.
@@ -76,7 +111,8 @@ The times are in `logic/schedule.cpp`.
`pio test -e test_esp32` runs [test/test_esp32](../../test/test_esp32/test_main.cpp) on
the PC: the schedule boundaries, the power timer (including `millis()` overflow), the LED
pattern and the display controller (power on/off, redraws, both Home Assistant switches).
pattern, the boot policy (crash counting, safe mode, rollback timing) and the display
controller (power on/off, redraws, both Home Assistant switches).
## Adding bitmaps