Fix shifted pictures: non-blocking Nano drawing, frame check, refresh settings

The Nano's periodic full redraw blocked for ~0.8 s. A frame that arrived
meanwhile overflowed its 64-byte serial buffer; lost bytes in a bitmap
shift the rest of the image (picture moved up), and diff frames never
repaired it.

- Nano: all drawing (frames, effects, the dot refresh) runs from update()
  in ~1 ms slices; the refresh pauses while a frame is received or shown
- New commands: C (reply with the image checksum) and I (refresh minutes,
  EEPROM)
- ESP32: NanoMonitor asks for the checksum after frames and sends the whole
  image again on a mismatch, missing answer or Nano restart; counts ERR
  replies. Periodic whole-image resync.
- Settings: "Dot refresh" and "Image resync" intervals (settings v5,
  v4 records are migrated)
- System tab: loop timing, free memory, frame check counts
- test_driver: the real driver against a simulated panel

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

CLAUDE.md

@@ -6,7 +6,8 @@ See README.md for the overview.
 ## Layout and build
 
 - `platformio.ini` has the environments `nano`, `nano_ota`, `esp32` (OTA to 10.1.1.198),
-  `esp32_usb` (COM11), and the PC test environments `test_nano` and `test_esp32`. The
+  `esp32_usb` (COM11), and the PC test environments `test_nano`, `test_esp32` and
+  `test_driver`. The
   sources are split with `build_src_filter` (`src/nano/`, `src/esp32/`).
 - `lib/BROSE9323/` is the shared driver. Its behaviour depends on build flags set in
   `platformio.ini`, not in the header:
@@ -27,7 +28,7 @@ See README.md for the overview.
   The uploader only uses the Python standard library. Test it against a simulated
   bootloader before changing it.
 - After any change, build both targets (`pio run`) and run the tests
-  (`pio test -e test_nano -e test_esp32`).
+  (`pio test -e test_nano -e test_esp32 -e test_driver`).
 
 ## Constraints
 
@@ -36,7 +37,7 @@ See README.md for the overview.
   data in `PROGMEM`.
 - The ESP32 talks to the Nano on `Serial2` (owned by `NanoLink`, GPIO 17 TX / GPIO 16 RX).
   `Serial` (UART0/USB) is free for debug output. Never write debug text to `Serial2`: the
-  Nano runs lines that start with s/S/F/D/P/B/R/T/G/N/E as commands.
+  Nano runs lines that start with s/S/F/D/P/B/R/T/G/N/E/I/C as commands.
 - Open hardware TODOs are listed in the README `TODO` section. Update it when one is done.
 - The README is the overview; the documentation is in `docs/` (web UI, pages, Home
   Assistant, how it works, wiring, Nano over WiFi, recovery app). Screenshots:
@@ -62,22 +63,28 @@ See README.md for the overview.
 - Line based and non-blocking: bytes are collected into a 64-character buffer and a
   command runs when `\n` arrives. Bitmap (`B`) pixel data is the exception. It is drawn
   while it arrives, so it can be longer than the buffer. Keep `loop()` free of busy-waits.
-- A full redraw (`display(true)`) blocks for about 0.5 s, which overflows the 64-byte
+- The driver never blocks while flipping: `display()` only starts a run, `update()` flips
-  serial receive buffer. Only run it when no data is expected: right after a `D`, or after
+  dots in slices of about 1 ms (frames, effects, and the periodic dot refresh from
-  2 s with no serial data. The ESP only sends a frame (ending with `D`) when the screen
+  `startRefresh()`, which waits while a frame is received or shown). Anything that blocks
-  changes, mostly once a minute.
+  longer than about 5 ms overflows the 64-byte serial receive buffer; lost bytes shift the
+  rest of a bitmap. That was the "picture moved up" bug. `test_driver` checks the time per
+  `update()` call with a simulated panel.
 - Flip time is stored in EEPROM as a magic word at address 0 and a `uint16_t` at
   address 2. Values outside 50–2000 µs are rejected.
-- The Nano only replies `READY` (at boot) and `ERR: ...`. There is no per-frame
+- The Nano replies `READY` (at boot), `ERR: ...` and `C xxxx` (checksum of its image, for
-  acknowledgement. The Nano's TX is wired to the ESP's RX (flashing over WiFi works), but
+  `C`). There is no per-frame acknowledgement: the ESP paces frames by time.
-  every wait for a reply still needs a timeout.
+  `nano_link/nano_monitor` reads the replies and sends `C` after frames;
+  `logic/frame_check` decides (wrong/missing answer or `READY` while on: send the whole
+  image again via `controller.pageChanged()`). Every wait for a reply needs a timeout.
+- Driver tests: `pio test -e test_driver` runs the real `BROSE9323.cpp` (Nano build)
+  against `test/driver_mocks/` (Arduino, Adafruit GFX, a panel that decodes the pins).
 - Unit tests: `pio test -e test_nano` runs `test/test_nano/` on the PC with the fakes in
   `test/mocks/` (Arduino, EEPROM, and a `BROSE9323` that records calls). It needs gcc on
   PATH (Strawberry Perl provides it at `C:\Strawberry\c\bin`). Add a test for every
   protocol change. The mocks only implement what the Nano code uses, so extend them when
   new Arduino or driver functions are called.
 - The board environments have `test_ignore = *`, so test firmware is never uploaded. Plain
-  `pio test` runs nothing; always pass `-e test_nano -e test_esp32`.
+  `pio test` runs nothing; always pass `-e test_nano -e test_esp32 -e test_driver`.
 
 ## ESP32 structure (`src/esp32/`)
 

README.md

@@ -58,7 +58,9 @@ src/recovery/           ESP32 recovery app: WiFi setup page + OTA
 partitions.csv          ESP32 flash layout (recovery app + two clock firmware slots)
 test/test_nano/         unit tests for the Nano firmware (run on the PC)
 test/test_esp32/        unit tests for the ESP32 logic (run on the PC)
+test/test_driver/       unit tests for the display driver with a simulated panel
 test/mocks/             fake Arduino, EEPROM and display driver for the Nano tests
+test/driver_mocks/      fake Arduino and Adafruit GFX for the driver tests
 tools/                  uploader for flashing the Nano over WiFi, font and picture scripts
 docs/                   documentation (see below)
 Pictures/               photos, pictures of the pages and screenshots of the web UI
@@ -92,7 +94,7 @@ pio run -e nano_ota -t upload    # flash the Nano over WiFi, through the ESP32
 pio run -e esp32 -t upload       # flash the ESP32 over the air
 pio run -e esp32_usb -t upload   # flash the ESP32 over USB
 pio device monitor -e nano       # serial monitor
-pio test -e test_nano -e test_esp32   # run the unit tests on the PC
+pio test -e test_nano -e test_esp32 -e test_driver   # run the unit tests on the PC
 ```
 
 The unit tests need `gcc`/`g++` on your PATH. On Windows you can get it from
@@ -135,3 +137,6 @@ In VS Code you can also pick the environment in the PlatformIO status bar.
 - [ ] **Check the boot timing.** Build the Nano with `MEASURE_REFRESH` to see how long a
   full redraw takes. If the reply wire isn't fitted, check that 3 s
   (`NANO_BOOT_TIMEOUT`) is enough for the Nano to boot.
+- [ ] **Watch the frame check.** After a few days, look at System, "Frame check": any
+  "wrong" or "no answer" counts mean bytes still get lost somewhere; "Nano restarted"
+  means power dips (e.g. while many dots flip).

docs/how-it-works.md

@@ -20,7 +20,13 @@
 3. **Home Assistant**: switches, dropdowns and sensors over MQTT, see
    [Home Assistant](home-assistant.md).
 4. **Nano.** The Nano reads commands from the serial line and updates its frame buffer.
-   On `D` it flips only the dots that changed. Every 2 minutes it redraws the whole screen
+   On `D` it flips only the dots that changed, in small steps so it never stops reading
-   to fix dots that did not flip.
+   the serial port. Every 2 minutes (Settings, "Dot refresh") it flips every dot once
+   more to fix dots that did not flip.
+5. **Frame check.** After a frame the ESP asks the Nano for a checksum of its image and
+   compares it with what it sent. When they differ (lost bytes can shift the picture),
+   the Nano doesn't answer, or it restarted, the ESP sends the whole image again. It also
+   does that every 10 minutes (Settings, "Image resync"); only dots that differ flip, so
+   normally nothing changes. The counts are on the System tab.
 
 The serial protocol is described in [src/nano/README.md](../src/nano/README.md).

docs/web-ui.md

@@ -88,7 +88,9 @@ The display schedule (on/off, when the display is on, the late-evening quarter-h
 when it switches off); the "No WiFi" screen; the flip time, the pause after each pulse,
 pulses per dot and the refresh effect (instant, sparkle, wipe, rain, from the middle out,
 diagonal, spiral, flip all then reveal, push, random; optionally only for small changes
-like the clock's minutes); test patterns; the status LED.
+like the clock's minutes); how often every dot is flipped again ("Dot refresh", default
+2 min) and how often the whole image is sent again ("Image resync", default 10 min; 0
+switches either off); test patterns; the status LED.
 
 ![Settings](../Pictures/web-ui/settings.png)
 
@@ -104,6 +106,12 @@ pages and the rules (REST API, plain http; empty URL: `http://<broker>:8123`). S
 ## System
 
 Firmware status, uptime, WiFi, storage; restart; start the [recovery app](recovery.md).
+**Loop:** how busy the ESP32 is: `loop()` passes per second and the average and longest
+pass over the last 10 s (and the longest since boot; a Home Assistant request can take a
+few seconds). `loop()` never sleeps, so a CPU percentage would always say 100 %.
+**Frame check:** how many checks of the Nano's image matched, differed or got no answer,
+how often the image was repaired, Nano restarts and `ERR` replies (with the last one).
+**Memory:** free heap now and the lowest since boot.
 
 **Backup:** export everything (settings, the schedule and rules, your bitmaps, pages and
 fonts; the MQTT password and HA token only when ticked) to one JSON file, and import a

lib/BROSE9323/BROSE9323.cpp

@@ -42,6 +42,8 @@ size_t BROSE9323::showBitmap(const uint8_t *bitmap, bool changes_only, bool inst
   size_t sent = frameCommands(bitmap, changes_only ? shown : nullptr, width(), height(), instant,
                               [this](const char *text) { stream->print(text); });
   free(shown);
+  if (sent)
+    _frames++;
 
   for (uint8_t y = 0; y < height(); y++)
     for (uint8_t x = 0; x < width(); x++)
@@ -102,6 +104,7 @@ void BROSE9323::display(bool force, bool instant)
 #ifdef ESPSERIAL
   stream->print(instant ? "D1\n" : "D\n");
   stream->flush();
+  _frames++;
   // Shadow copy of what the dots show now (for the web UI)
   memcpy(_old_buffer, _new_buffer, _buffer_size);
 #else
@@ -110,14 +113,11 @@ void BROSE9323::display(bool force, bool instant)
   bool all = force || _force_next;
   _force_next = false;
   _drawing = false;
-  // An explicit full redraw is always instant (the periodic refresh)
+  // A redraw of every dot (after boot) that this frame interrupts: the refresh finishes it
-  if (_effect == EFFECT_INSTANT || force || instant)
+  if (_animating && _animate_all)
-  {
+    startRefresh();
-    _animating = false;
+  // An explicit full redraw is always instant
-    _drawAll(all);
+  _startEffect(all, _effect == EFFECT_INSTANT || force || instant);
-  }
-  else
-    _startEffect(all);
 #endif
 }
 
@@ -224,6 +224,15 @@ void BROSE9323::setTiming(uint16_t t)
 #endif
 }
 
+#ifdef ESPSERIAL
+void BROSE9323::setRefreshInterval(uint16_t minutes)
+{
+  stream->write('I');
+  stream->print(minutes);
+  stream->write('\n');
+}
+#endif
+
 void BROSE9323::setGap(uint16_t t)
 {
 #ifdef ESPSERIAL
@@ -389,22 +398,12 @@ void BROSE9323::_flipDot(uint8_t x, uint8_t y, uint8_t pulses)
   _flipTo(x, y, _wanted(x, y), pulses);
 }
 
-// Column by column, as fast as possible. all: also the dots that look unchanged (one
+// Starts showing the new buffer from update(). instant: column by column as fast as
-// pulse each, they are only repeated in case they didn't flip before).
+// possible, otherwise with the effect. all: also the dots that look unchanged (one pulse
-void BROSE9323::_drawAll(bool all)
+// each, they are only repeated in case they didn't flip before).
+void BROSE9323::_startEffect(bool all, bool instant)
 {
-  for (uint8_t x = 0; x < width(); x++)
+  uint16_t total = width() * height();
-    for (uint8_t y = 0; y < height(); y++)
-    {
-      bool changed = _changed(x, y);
-      if (changed || all)
-        _flipDot(x, y, changed ? _pulses : 1);
-    }
-}
-
-void BROSE9323::_startEffect(bool all)
-{
-  uint16_t total = (uint16_t)width() * height();
   uint16_t changed = 0;
   for (uint8_t y = 0; y < height(); y++)
     for (uint8_t x = 0; x < width(); x++)
@@ -414,15 +413,11 @@ void BROSE9323::_startEffect(bool all)
     _animating = false;
     return;
   }
-  // E.g. only the clock's minutes get the effect, a new page appears at once
-  if ((_effect_flags & EFFECT_FLAG_SMALL_ONLY) && (all || changed > total / 10))
-  {
-    _animating = false;
-    _drawAll(all);
-    return;
-  }
 
-  uint8_t effect = _effect;
+  uint8_t effect = instant ? EFFECT_INSTANT : _effect;
+  // E.g. only the clock's minutes get the effect, a new page appears at once
+  if (effect != EFFECT_INSTANT && (_effect_flags & EFFECT_FLAG_SMALL_ONLY) && (all || changed > total / 10))
+    effect = EFFECT_INSTANT;
   if (effect == EFFECT_RANDOM)
     effect = 1 + micros() % (EFFECT_RANDOM - 1);
   // Pushing needs to know what is shown; after boot that is unknown
@@ -432,6 +427,14 @@ void BROSE9323::_startEffect(bool all)
   _animate_all = all;
   _phase = 0;
 
+  if (effect == EFFECT_INSTANT)
+  {
+    // Column order, no waiting between the dots
+    _sequence.begin(EFFECT_INSTANT, width(), height(), 0);
+    _step_time = 0;
+  }
+  else
+  {
     uint32_t steps;
     if (effect == EFFECT_PUSH)
     {
@@ -449,10 +452,34 @@ void BROSE9323::_startEffect(bool all)
         steps *= 2;
     }
     _step_time = (uint32_t)_effect_time * 1000 / steps;
+  }
   _next_step = micros();
   _animating = true;
 }
 
+void BROSE9323::startRefresh(void)
+{
+  _refreshing = true;
+  _refresh_pos = 0;
+}
+
+// The next dots of the full redraw, until about 1 ms after start
+void BROSE9323::_refreshStep(uint32_t start)
+{
+  uint8_t h = height();
+  uint16_t total = width() * h;
+  do
+  {
+    uint8_t x = _refresh_pos / h, y = _refresh_pos % h;
+    _flipDot(x, y, _changed(x, y) ? _pulses : 1);
+    if (++_refresh_pos >= total)
+    {
+      _refreshing = false;
+      return;
+    }
+  } while (micros() - start < 1000);
+}
+
 // One dot of the current push step. Shift s (1..width) shows old columns s.. on the left
 // and new columns 0..s-1 on the right: each column takes over its right neighbour (not
 // changed yet in this step, as the columns go left to right), the last one a new column.
@@ -478,12 +505,19 @@ bool BROSE9323::busy(void) const
 
 bool BROSE9323::update(void)
 {
-  // Wait for the D that ends the frame being drawn, it starts the effect again
+  // Wait for the D that ends the frame being drawn, it starts the effect again (and the
-  if (!_animating || _drawing)
+  // refresh must not show half a frame)
+  if (_drawing)
     return busy();
 
   // At most about 1 ms per call, so loop() keeps reading the serial port
   uint32_t start = micros();
+  if (!_animating)
+  {
+    if (_refreshing)
+      _refreshStep(start);
+    return busy();
+  }
   while ((int32_t)(micros() - _next_step) >= 0 && micros() - start < 1000)
   {
     if (_running_effect == EFFECT_PUSH)

lib/BROSE9323/BROSE9323.h

@@ -3,6 +3,7 @@
 
 #include <Adafruit_GFX.h>
 
+#include "buffer_crc.h"
 #include "dot_sequence.h"
 #ifdef ESPSERIAL
 #include "frame_commands.h"
@@ -29,6 +30,7 @@ private:
 
 #ifdef ESPSERIAL
     Stream *stream;
+    uint32_t _frames = 0; // frames sent (display() and showBitmap() calls that sent something)
 #else
     uint8_t _active_panel = 255;
     uint8_t _active_col = 255;
@@ -49,6 +51,11 @@ private:
     uint32_t _step_time = 0;        // us per step
     uint32_t _next_step = 0;        // micros() of the next step
 
+    // The full redraw (startRefresh()): one pulse for every dot, column by column, while
+    // no frame is being received or drawn
+    bool _refreshing = false;
+    uint16_t _refresh_pos = 0;
+
 #ifndef FLIPDOT_PLCC_ADAPTER
     const uint8_t ENABLE = 7,
                   ADDR_0 = 3,
@@ -99,8 +106,8 @@ private:
     void _flipTo(uint8_t x, uint8_t y, bool on, uint8_t pulses);
     void _flipDot(uint8_t x, uint8_t y, uint8_t pulses);
     bool _pushDot(void);
-    void _drawAll(bool all);
+    void _startEffect(bool all, bool instant);
-    void _startEffect(bool all);
+    void _refreshStep(uint32_t start);
 #endif
 public:
     BROSE9323(uint8_t, uint8_t, uint8_t, uint16_t ft = 280);
@@ -111,10 +118,16 @@ public:
     // (frame_commands.h); otherwise the whole screen. Returns the characters sent (0:
     // nothing changed, nothing sent).
     size_t showBitmap(const uint8_t *bitmap, bool changes_only, bool instant = false);
+    // Frames sent so far (display() and showBitmap() calls that sent something)
+    uint32_t framesSent(void) const { return _frames; }
+    // How often the Nano flips every dot again in minutes, 0 = never (I command)
+    void setRefreshInterval(uint16_t minutes);
 #endif
     void begin(void);
     // Shows the buffer: the changed dots, all with force. instant: without the refresh
     // effect (frames of moving content, e.g. scrolling text). ESP32: sends D or D1.
+    // Nano: never blocks, the dots are flipped from update() (also for instant frames,
+    // as fast as possible), so the serial port is read while they flip.
     void display(bool force = false, bool instant = false);
     void drawPixel(int16_t x, int16_t y, uint16_t color);
     void fillScreen(uint16_t);
@@ -130,10 +143,15 @@ public:
     // Effects run in steps from update(), so call that often (every loop).
     void setEffect(uint8_t effect, uint16_t time_ms, uint8_t flags = 0);
 #ifndef ESPSERIAL
-    // Runs the next steps of an effect. Returns busy().
+    // Flips the next dots of a frame or of the full redraw, at most about 1 ms per call
+    // (plus one dot), so loop() keeps reading the serial port. Returns busy().
     bool update(void);
-    // An effect is running (paused while a new frame is being drawn into the buffer)
+    // A frame is being shown (paused while a new frame is being drawn into the buffer)
     bool busy(void) const;
+    // Flip every dot once more (fixes dots that didn't flip), from update() whenever no
+    // frame is being received or shown. Starts again when one is running.
+    void startRefresh(void);
+    bool refreshing(void) const { return _refreshing; }
 #endif
 #ifndef ESPSERIAL
     void printBuffer(void);
@@ -141,5 +159,8 @@ public:
     size_t print(const char *ifsh);
     // State of a dot as of the last display() call (ESP32: shadow copy for the web UI)
     bool shownPixel(int16_t x, int16_t y) const;
+    // bufferCrc() of the drawn image (Nano: what it received, the C command; ESP32: what
+    // it sent)
+    uint16_t crc(void) const { return bufferCrc(_new_buffer, width(), height()); }
 };
 #endif //BROSE9323_H

lib/BROSE9323/buffer_crc.h

@@ -0,0 +1,25 @@
+#pragma once
+
+#include <stdint.h>
+
+// CRC-16/CCITT-FALSE of a display buffer as BROSE9323 keeps it: rows of (width + 7) / 8
+// bytes, bit (x & 7) is dot x. The unused bits at the end of each row are left out, so
+// only the dots count (fillScreen sets them, drawPixel never does). The Nano answers the
+// C command with it; the ESP32 compares it with its own copy (src/esp32/nano_link).
+inline uint16_t bufferCrc(const uint8_t *buffer, uint8_t width, uint8_t height)
+{
+	const uint8_t row_bytes = (width + 7) / 8;
+	const uint8_t last_mask = (width & 7) ? (uint8_t)((1 << (width & 7)) - 1) : 0xFF;
+	uint16_t crc = 0xFFFF;
+	for (uint8_t y = 0; y < height; y++)
+		for (uint8_t i = 0; i < row_bytes; i++)
+		{
+			uint8_t b = buffer[y * row_bytes + i];
+			if (i == row_bytes - 1)
+				b &= last_mask;
+			crc ^= (uint16_t)b << 8;
+			for (uint8_t k = 0; k < 8; k++)
+				crc = (crc & 0x8000) ? (uint16_t)((crc << 1) ^ 0x1021) : (uint16_t)(crc << 1);
+		}
+	return crc;
+}

platformio.ini

@@ -12,7 +12,7 @@
 ; Build one:        pio run -e nano
 ; Upload one:       pio run -e nano -t upload
 ; Serial monitor:   pio device monitor -e nano
-; Run tests:        pio test -e test_nano -e test_esp32
+; Run tests:        pio test -e test_nano -e test_esp32 -e test_driver
 ;
 ; https://docs.platformio.org/page/projectconf.html
 
@@ -104,7 +104,7 @@ upload_protocol = esptool
 upload_port = COM18
 
 ; ---------------------------------------------------------------------------
-; Unit tests on the PC (pio test -e test_nano -e test_esp32)
+; Unit tests on the PC (pio test -e test_nano -e test_esp32 -e test_driver)
 ; ---------------------------------------------------------------------------
 [native]
 platform = native
@@ -123,6 +123,14 @@ build_src_filter = +<nano/>
 ; test/mocks first: its BROSE9323.h replaces the driver, dot_sequence.h is the real one
 build_flags = -std=gnu++17 -I test/mocks -I lib/BROSE9323
 
+; The real display driver (test/test_driver, Nano build) with a simulated panel:
+; test/driver_mocks replaces Arduino and Adafruit GFX, the test includes BROSE9323.cpp
+[env:test_driver]
+extends = native
+test_filter = test_driver
+build_src_filter = -<*>
+build_flags = -std=gnu++17 -I test/driver_mocks -I lib/BROSE9323
+
 ; ESP32 logic (test/test_esp32): only the modules without Arduino dependencies
 [env:test_esp32]
 extends = native

src/esp32/app.cpp

@@ -25,6 +25,8 @@ BROSE9323 display(DISPLAY_WIDTH, DISPLAY_HEIGHT, PANEL_WIDTH);
 FlipdotFrames frames(display, pageResources);
 StatusScreen status(display, nano);
 DisplayController controller(nano, frames);
+LoopStats loopStats;
+NanoMonitor nanoMonitor(nano, display);
 
 static Settings settings;
 static bool restart_pending = false;
@@ -87,7 +89,7 @@ static void apply()
 {
 	controller.setSettings(settings.schedule, settings.clock);
 	controller.setEffectTime(effectTime(settings.drive));
-	status.setDrive(settings.flip_time, settings.drive);
+	status.setDrive(settings.flip_time, settings.drive, settings.refresh.dots);
 	frames.setDrive(settings.flip_time, settings.drive);
 	haApiSetSettings(settings.home_assistant);
 	applyOffPage();
@@ -137,7 +139,8 @@ const char *appSaveSettings(const Settings &new_settings, bool &restart_needed)
 	const DriveSettings &a = settings.drive, &b = new_settings.drive;
 	bool drive_changed = settings.flip_time != new_settings.flip_time || a.flip_gap != b.flip_gap ||
 						 a.pulses != b.pulses || a.effect != b.effect || a.effect_time != b.effect_time ||
-						 a.effect_small_only != b.effect_small_only;
+						 a.effect_small_only != b.effect_small_only ||
+						 settings.refresh.dots != new_settings.refresh.dots;
 	settings = new_settings;
 	saveSettings(settings);
 	selector.setChosen(settings.page);

src/esp32/app.h

@@ -8,16 +8,20 @@
 #include "display/status_screen.h"
 #include "logic/display_controller.h"
 #include "logic/local_time.h"
+#include "logic/loop_stats.h"
 #include "logic/page.h"
 #include "logic/page_schedule.h"
 #include "logic/settings.h"
 #include "nano_link/nano_link.h"
+#include "nano_link/nano_monitor.h"
 
 extern NanoLink nano;
 extern BROSE9323 display;
 extern FlipdotFrames frames;
 extern StatusScreen status;
 extern DisplayController controller;
+extern LoopStats loopStats;
+extern NanoMonitor nanoMonitor;
 
 // Settings: loaded at boot, changed in the web UI
 const Settings &appSettings();

src/esp32/config.h

@@ -55,6 +55,13 @@ const uint32_t HA_ERROR_INTERVAL = 60000;
 // this one (characters per ms at NANO_BAUD) and flipped its changed dots, plus a margin
 const uint32_t NANO_CHARS_PER_MS = NANO_BAUD / 10000;
 const uint32_t FRAME_MARGIN = 20;
+// Frame check (logic/frame_check.h): after a frame the Nano is asked for the checksum of
+// its image (C), at most this often (moving pages send many frames)
+const uint32_t FRAME_CHECK_INTERVAL = 3000;
+// No answer this long: counted as lost (the C or the frame before it got garbled)
+const uint32_t FRAME_CHECK_TIMEOUT = 2000;
+// A wrong image is repaired by sending the whole image again, at most this often
+const uint32_t FRAME_REPAIR_MIN_GAP = 10000;
 
 // ---------------------------------------------------------------------------
 // Flashing the Nano over WiFi (see README.md, pio run -e nano_ota -t upload)
@@ -105,7 +112,7 @@ const uint32_t SETUP_AP_DELAY = 20000;        // open the AP when WiFi fails thi
 const uint32_t WIFI_CONNECT_TIMEOUT = 30000;
 const uint32_t WIFI_RESTART_TIME = 10UL * 60 * 1000;
 
-#define FIRMWARE_VERSION "2.7.2"
+#define FIRMWARE_VERSION "2.8.0"
 
 // Home Assistant device id (do not change: Home Assistant's entities are linked to it)
 const uint8_t HA_DEVICE_ID[] = {0x34, 0x86, 0x5d, 0x3a, 0x08, 0x28};

src/esp32/display/status_screen.cpp

@@ -9,10 +9,11 @@ StatusScreen::StatusScreen(BROSE9323 &display, NanoLink &nano) : _display(displa
 {
 }
 
-void StatusScreen::setDrive(uint16_t flip_time, const DriveSettings &drive)
+void StatusScreen::setDrive(uint16_t flip_time, const DriveSettings &drive, uint16_t dot_refresh)
 {
 	_flip_time = flip_time;
 	_drive = drive;
+	_dot_refresh = dot_refresh;
 }
 
 void StatusScreen::sendDrive()
@@ -30,6 +31,7 @@ void StatusScreen::loop()
 	_display.setGap(_drive.flip_gap);
 	_display.setPulses(_drive.pulses);
 	_display.setEffect(_drive.effect, _drive.effect_time, _drive.effect_small_only ? EFFECT_FLAG_SMALL_ONLY : 0);
+	_display.setRefreshInterval(_dot_refresh);
 }
 
 void StatusScreen::show(const char *line1, const char *line2, bool keep_on)

src/esp32/display/status_screen.h

@@ -17,7 +17,7 @@ public:
 
 	// How the Nano drives the dots, sent with the first message (default: FLIP_TIME and
 	// the DriveSettings defaults, so the recovery app always shows its messages instantly)
-	void setDrive(uint16_t flip_time, const DriveSettings &drive);
+	void setDrive(uint16_t flip_time, const DriveSettings &drive, uint16_t dot_refresh = RefreshSettings().dots);
 	// Send them again (they changed), from loop()
 	void sendDrive();
 	// Sends changed drive settings once the display is on. Not while the Nano is flashed.
@@ -30,4 +30,5 @@ private:
 	bool _drive_pending = false;
 	uint16_t _flip_time;
 	DriveSettings _drive;
+	uint16_t _dot_refresh = RefreshSettings().dots;
 };

src/esp32/logic/frame_check.cpp

@@ -0,0 +1,120 @@
+#include "logic/frame_check.h"
+
+#include <string.h>
+
+#include "config.h"
+
+static int hexDigit(char c)
+{
+	if (c >= '0' && c <= '9')
+		return c - '0';
+	if (c >= 'A' && c <= 'F')
+		return c - 'A' + 10;
+	if (c >= 'a' && c <= 'f')
+		return c - 'a' + 10;
+	return -1;
+}
+
+void FrameCheck::reset(uint32_t now)
+{
+	_pending = false;
+	_checked_any = false;
+	_replied = false;
+	_problem = false;
+	// Switching on sends the whole image anyway
+	_last_resync = now;
+}
+
+bool FrameCheck::due(uint32_t frames_sent, uint32_t now) const
+{
+	// A repair is coming anyway: no need to count the same problem again
+	if (_pending || _problem)
+		return false;
+	if (!_checked_any)
+		return true;
+	return frames_sent != _checked_frames && now - _last_check >= FRAME_CHECK_INTERVAL;
+}
+
+void FrameCheck::sent(uint16_t expected, uint32_t frames_sent, uint32_t now)
+{
+	_pending = true;
+	_expected = expected;
+	_sent_at = now;
+	_checked_any = true;
+	_checked_frames = frames_sent;
+	_last_check = now;
+}
+
+void FrameCheck::line(const char *text, uint32_t now)
+{
+	(void)now;
+	size_t len = strlen(text);
+	while (len && (text[len - 1] == '\r' || text[len - 1] == ' '))
+		len--;
+
+	if (len == 6 && text[0] == 'C' && text[1] == ' ')
+	{
+		int value = 0;
+		for (int i = 2; i < 6; i++)
+		{
+			int d = hexDigit(text[i]);
+			if (d < 0)
+				return;
+			value = value << 4 | d;
+		}
+		_replied = true;
+		if (!_pending)
+			return; // late answer to a check that already timed out
+		_pending = false;
+		if (value == _expected)
+			_stats.checks++;
+		else
+		{
+			_stats.mismatches++;
+			_problem = true;
+		}
+		return;
+	}
+	if (strncmp(text, "ERR", 3) == 0)
+	{
+		_replied = true;
+		_stats.errors++;
+		size_t n = len < sizeof(_stats.last_error) - 1 ? len : sizeof(_stats.last_error) - 1;
+		memcpy(_stats.last_error, text, n);
+		_stats.last_error[n] = '\0';
+		return;
+	}
+	if (len >= 5 && strncmp(text, "READY", 5) == 0)
+	{
+		// The Nano restarted while it was on: its image is gone
+		_replied = true;
+		_stats.restarts++;
+		_pending = false;
+		_problem = true;
+	}
+}
+
+bool FrameCheck::resyncNow(uint32_t now, uint32_t resync_ms)
+{
+	if (_pending && now - _sent_at >= FRAME_CHECK_TIMEOUT)
+	{
+		_pending = false;
+		_stats.no_reply++;
+		// Only when answers can arrive at all (old Nano firmware, TX not wired: never)
+		if (_replied)
+			_problem = true;
+	}
+	if (_problem && now - _last_resync >= FRAME_REPAIR_MIN_GAP)
+	{
+		_problem = false;
+		_stats.repairs++;
+		_last_resync = now;
+		return true;
+	}
+	if (resync_ms && now - _last_resync >= resync_ms)
+	{
+		_last_resync = now;
+		return true;
+	}
+	return false;
+}

src/esp32/logic/frame_check.h

@@ -0,0 +1,58 @@
+#pragma once
+
+#include <stdint.h>
+
+// Checks that the Nano shows what was sent. The Nano has no acknowledgement and drops
+// bytes when its serial buffer overflows; a lost byte in a bitmap shifts the rest of the
+// image (the dots after it land too early). After a frame the ESP32 asks for the checksum
+// of the Nano's image (C command, bufferCrc() in lib/BROSE9323) and compares it with its
+// own copy. A wrong checksum, a missing answer or a restarted Nano (READY) makes it send
+// the whole image again (repair). Besides that the whole image is sent again every
+// resync interval, which also fixes what the check can't see.
+//
+// Pure logic (tested in test/test_esp32); nano_link/nano_monitor does the serial part.
+
+struct FrameCheckStats
+{
+	uint32_t checks = 0;     // answers that matched
+	uint32_t mismatches = 0; // answers that didn't
+	uint32_t no_reply = 0;   // checks without an answer in FRAME_CHECK_TIMEOUT
+	uint32_t errors = 0;     // ERR: lines from the Nano
+	uint32_t restarts = 0;   // READY while it was on: the Nano restarted (power dip?)
+	uint32_t repairs = 0;    // whole image sent again because of one of the above
+	char last_error[48] = "";
+};
+
+class FrameCheck
+{
+public:
+	// The Nano was switched on or off: forget the check in progress
+	void reset(uint32_t now);
+
+	// A check (C) should be sent now: a frame was sent since the last one (frames_sent
+	// changed), none is waiting for its answer and the last one is FRAME_CHECK_INTERVAL ago
+	bool due(uint32_t frames_sent, uint32_t now) const;
+	// The C was sent; expected: the checksum of the ESP32's copy
+	void sent(uint16_t expected, uint32_t frames_sent, uint32_t now);
+
+	// One line from the Nano (without the '\n')
+	void line(const char *text, uint32_t now);
+
+	// Call each loop(). True: send the whole image now (repair, or every resync_ms; 0 = no
+	// periodic resync)
+	bool resyncNow(uint32_t now, uint32_t resync_ms);
+
+	const FrameCheckStats &stats() const { return _stats; }
+
+private:
+	FrameCheckStats _stats;
+	bool _pending = false;
+	uint16_t _expected = 0;
+	uint32_t _sent_at = 0;
+	bool _checked_any = false;  // since reset()
+	uint32_t _checked_frames = 0;
+	uint32_t _last_check = 0;
+	bool _replied = false;      // the Nano answered since reset(): its TX is wired
+	bool _problem = false;      // repair at the next chance
+	uint32_t _last_resync = 0;
+};

src/esp32/logic/loop_stats.h

@@ -0,0 +1,54 @@
+#pragma once
+
+#include <stdint.h>
+
+// How busy loop() is. It never sleeps, so a CPU percentage would always be 100 %; what
+// matters is how long one pass takes (a slow pass delays OTA, the web UI and the frames).
+// tick() is called at the start of every pass with micros(); the numbers cover the last
+// complete window of LOOP_STATS_WINDOW_US.
+class LoopStats
+{
+public:
+	static const uint32_t WINDOW_US = 10000000UL; // 10 s
+
+	void tick(uint32_t now_us)
+	{
+		if (!_started)
+		{
+			_started = true;
+			_last = now_us;
+			_window_start = now_us;
+			return;
+		}
+		uint32_t pass = now_us - _last;
+		_last = now_us;
+		_count++;
+		if (pass > _max)
+			_max = pass;
+		if (pass > _max_ever)
+			_max_ever = pass;
+
+		uint32_t window = now_us - _window_start;
+		if (window >= WINDOW_US)
+		{
+			_loops_per_s = (uint32_t)((uint64_t)_count * 1000000UL / window);
+			_avg_us = window / _count;
+			_window_max_us = _max;
+			_count = 0;
+			_max = 0;
+			_window_start = now_us;
+		}
+	}
+
+	// Passes per second, average and longest pass in us (last window), longest since boot
+	uint32_t loopsPerSecond() const { return _loops_per_s; }
+	uint32_t averageUs() const { return _avg_us; }
+	uint32_t maxUs() const { return _window_max_us; }
+	uint32_t maxEverUs() const { return _max_ever; }
+
+private:
+	bool _started = false;
+	uint32_t _last = 0, _window_start = 0;
+	uint32_t _count = 0, _max = 0, _max_ever = 0;
+	uint32_t _loops_per_s = 0, _avg_us = 0, _window_max_us = 0;
+};

src/esp32/logic/settings.cpp

@@ -35,6 +35,8 @@ const char *validateSettings(const Settings &s)
 		return "Unknown refresh effect";
 	if (d.effect_time < 100 || d.effect_time > 10000)
 		return "The effect time must be between 100 and 10000 ms";
+	if (s.refresh.dots > REFRESH_MINUTES_MAX || s.refresh.image > REFRESH_MINUTES_MAX)
+		return "Refresh intervals must be between 0 and 1440 minutes";
 
 	const HomeAssistantSettings &ha = s.home_assistant;
 	if (!terminated(ha.host, sizeof(ha.host)) || !terminated(ha.user, sizeof(ha.user)) ||

src/esp32/logic/settings.h

@@ -54,6 +54,18 @@ struct DriveSettings
 	bool effect_small_only = false; // big changes (a new page) appear instantly
 };
 
+// Full redraws that fix the picture, in minutes (0 = never)
+const uint16_t REFRESH_MINUTES_MAX = 1440;
+struct RefreshSettings
+{
+	// The Nano flips every dot once more (fixes dots that didn't flip). Sent to the Nano,
+	// which keeps it in its EEPROM (I command, default there: 2).
+	uint16_t dots = 2;
+	// The ESP32 sends the whole image again (fixes a wrong image on the Nano, e.g. after
+	// lost bytes). Only the dots that differ flip, so a correct image doesn't change.
+	uint16_t image = 10;
+};
+
 // What is shown during the day: the name of a dynamic page (logic/page.h) or a bitmap.
 // Both share one namespace. When it is gone, the built-in Clock page is shown.
 #define DEFAULT_PAGE BUILTIN_PAGE_CLOCK
@@ -70,6 +82,7 @@ struct Settings
 	bool status_led = true;
 	char off_page[PAGE_NAME_SIZE] = OFF_PAGE_BLANK;
 	DriveSettings drive;
+	RefreshSettings refresh;
 	// New fields go here, at the end (see system/settings_store.cpp)
 };
 

src/esp32/main.cpp

@@ -73,6 +73,7 @@ void loop()
 	// OTA and the firmware guard come first, so they keep working whatever happens below
 	otaLoop();
 	firmwareGuardLoop(wifiConnected());
+	loopStats.tick(micros());
 	wifiWatchdog();
 
 	homeAssistantSetFirmwareStatus(firmwareGuardStatus());
@@ -92,6 +93,10 @@ void loop()
 	bool time_valid = appLocalTime(now);
 	appPageLoop(time_valid, now);
 	controller.update(millis(), time_valid, now, wifiConnected());
+	// Checks the frames just sent; a wrong image on the Nano (and every resync interval):
+	// the whole image again with the next update
+	if (nanoMonitor.loop(millis(), appSettings().refresh.image * 60000UL))
+		controller.pageChanged();
 
 	bool led = appSettings().status_led && statusLedLevel(millis(), wifiConnected(), nano.isOn());
 	digitalWrite(LED_PIN, led);

src/esp32/nano_link/nano_monitor.cpp

@@ -0,0 +1,43 @@
+#include "nano_link/nano_monitor.h"
+
+NanoMonitor::NanoMonitor(NanoLink &nano, BROSE9323 &display) : _nano(nano), _display(display)
+{
+}
+
+bool NanoMonitor::loop(uint32_t now, uint32_t resync_ms)
+{
+	bool on = _nano.isOn();
+	if (on != _was_on)
+	{
+		// Switched on (the Nano booted, NanoLink::on() took its READY) or off
+		_was_on = on;
+		_check.reset(now);
+		_len = 0;
+	}
+	if (!on)
+		return false;
+
+	// First the answers that arrived (also while loop() was blocked), then the timeouts
+	Stream &serial = _nano.stream();
+	while (serial.available())
+	{
+		char c = serial.read();
+		if (c == '\n')
+		{
+			_line[_len] = '\0';
+			_check.line(_line, now);
+			_len = 0;
+		}
+		else if (_len < sizeof(_line) - 1)
+			_line[_len++] = c;
+	}
+	bool resync = _check.resyncNow(now, resync_ms);
+
+	uint32_t frames = _display.framesSent();
+	if (_check.due(frames, now))
+	{
+		serial.print("C\n");
+		_check.sent(_display.crc(), frames, now);
+	}
+	return resync;
+}

src/esp32/nano_link/nano_monitor.h

@@ -0,0 +1,28 @@
+#pragma once
+
+#include "BROSE9323.h"
+#include "logic/frame_check.h"
+#include "nano_link/nano_link.h"
+
+// Reads what the Nano sends back while it is on (C answers, ERR:, READY) and asks for
+// the checksum after frames (logic/frame_check.h). Not while the Nano is being flashed:
+// then the flash bridge owns the serial port.
+class NanoMonitor
+{
+public:
+	NanoMonitor(NanoLink &nano, BROSE9323 &display);
+
+	// From loop(), after the frames of this pass were sent. True: send the whole image
+	// again (repair, or the periodic resync every resync_ms; 0 = never).
+	bool loop(uint32_t now, uint32_t resync_ms);
+
+	const FrameCheckStats &stats() const { return _check.stats(); }
+
+private:
+	NanoLink &_nano;
+	BROSE9323 &_display;
+	FrameCheck _check;
+	bool _was_on = false;
+	char _line[64];
+	uint8_t _len = 0;
+};

src/esp32/system/settings_store.cpp

@@ -9,7 +9,7 @@ static const char *KEY = "settings";
 // Increase when the Settings struct changes; old records are then ignored (defaults),
 // unless the code below says how to read them. When fields are only added at the end,
 // an old record is the start of the new layout: add an OldVersion for it like v3 does.
-static const uint16_t VERSION = 4;
+static const uint16_t VERSION = 5;
 
 struct OldVersion
 {
@@ -78,6 +78,9 @@ static void convert(const Settings &old, ::Settings &s)
 
 static_assert(sizeof(v3::Settings) <= sizeof(Settings), "the old record must fit the buffer");
 
+// Version 4: the current layout without refresh (added at the end, so it is a prefix)
+static const OldVersion V4 = {4, offsetof(Settings, refresh)};
+
 struct StoredSettings
 {
 	uint16_t version;
@@ -108,6 +111,11 @@ void loadSettings(Settings &settings)
 	Settings loaded;
 	if (read == sizeof(stored) && stored.version == VERSION && stored.size == sizeof(Settings))
 		loaded = stored.settings;
+	else if (stored.version == V4.version)
+	{
+		if (!readOld(stored, read, V4.fields, loaded))
+			return;
+	}
 	else
 	{
 		bool ok = false;

src/esp32/web/web_page.h

@@ -145,8 +145,10 @@ details{margin-top:10px}summary{cursor:pointer;color:var(--dim);font-size:13px;m
 <option value="8">Push (slides in from the right)</option><option value="9">Random</option></select>
 <label>Effect for</label><label><input type="checkbox" id="h_small"> small changes only (the clock's minutes); a new page appears at once</label>
 <label>Effect time</label><label><input type="number" id="h_efft" min="100" max="10000" style="width:6em"> ms (100-10000, not for Instant; Push needs more when many dots change)</label>
+<label>Dot refresh</label><label>every <input type="number" id="h_rdots" min="0" max="1440" style="width:5em"> min (0 = off): every dot flipped again</label>
+<label>Image resync</label><label>every <input type="number" id="h_rimg" min="0" max="1440" style="width:5em"> min (0 = off): the whole image sent again</label>
 <label>Status LED</label><label><input type="checkbox" id="h_led"> blink while the display is on</label>
-</div><p class="hint">Dots that don't flip: a longer gap gives the power supply time to recover between pulses, a second pulse catches dots that missed the first one. Full redraws (after power on, every 2 minutes) use one pulse per dot.</p></div>
+</div><p class="hint">Dots that don't flip: a longer gap gives the power supply time to recover between pulses, a second pulse catches dots that missed the first one. The dot refresh (and the first frame after power on) uses one pulse per dot; it runs between frames. The image resync repairs a picture the Nano got wrong (e.g. shifted); only dots that differ flip, so normally nothing changes. A wrong picture is also found and repaired by the frame check (see System).</p></div>
 <div class="card"><h2>Test patterns</h2>
 <div class="row"><button class="s" onclick="pattern('yellow')">All yellow</button><button class="s" onclick="pattern('black')">All black</button>
 <button class="s" onclick="pattern('checker')">Checkerboard</button><button class="s" onclick="pattern('checker2')">Checkerboard inverted</button>
@@ -212,6 +214,9 @@ async function pollStatus(){try{status=await api('/api/status');
 function sysinfo(){if(!status)return;const r=[['Firmware',status.firmware],['Last restart',status.reset_reason],['Uptime',Math.floor(status.uptime/3600)+' h '+Math.floor(status.uptime%3600/60)+' min'],
  ['WiFi',status.wifi.ssid+' ('+status.wifi.ip+', '+status.wifi.rssi+' dBm)'],['Home Assistant',status.ha.enabled?(status.ha.connected?'connected':'not connected'):'disabled'],
  ['HA power switch',status.power_enabled?'on':'off'],
+ ['Loop',status.load?(status.load.loops_per_s?status.load.loops_per_s+'/s, avg '+(status.load.avg_us/1000).toFixed(1)+' ms, max '+status.load.max_ms+' ms (since boot '+status.load.max_ever_ms+' ms)':'measuring...'):''],
+ ['Frame check',status.nano_link?(()=>{const n=status.nano_link;return n.checks+' ok, '+n.mismatches+' wrong, '+n.no_reply+' no answer, '+n.repairs+' repaired'+(n.restarts?', Nano restarted '+n.restarts+'x':'')+(n.errors?', '+n.errors+' errors (last: '+n.last_error+')':'')})():''],
+ ['Memory',status.load?Math.round(status.load.heap_free/1024)+' KB free (lowest '+Math.round(status.load.heap_min/1024)+' KB)':''],
  ['Storage',status.storage.available?Math.round(status.storage.used/1024)+' of '+Math.round(status.storage.total/1024)+' KB used':'not available']];
  $('sysinfo').innerHTML='';r.forEach(([k,v])=>{const a=document.createElement('label');a.textContent=k;const b=document.createElement('span');b.textContent=v;$('sysinfo').append(a,b)})}
 // pages, bitmaps and fonts
@@ -536,7 +541,7 @@ async function stopTest(){clearInterval(testTimer);testTimer=null;await stopLive
 function fill(){const s=settings;$('s_en').checked=s.schedule.enabled;$('s_day').value=s.schedule.day_start;$('s_late').value=s.schedule.late_start;
  $('s_q').checked=s.schedule.quarter_hours;$('s_night').value=s.schedule.night_start;
  $('c_nw').checked=s.clock.no_wifi_screen;$('h_flip').value=s.flip_time;$('h_led').checked=s.status_led;
- $('h_gap').value=s.drive.flip_gap;$('h_pulses').value=s.drive.pulses;$('h_effect').value=s.drive.effect;$('h_efft').value=s.drive.effect_time;$('h_small').checked=s.drive.effect_small_only;
+ $('h_gap').value=s.drive.flip_gap;$('h_pulses').value=s.drive.pulses;$('h_effect').value=s.drive.effect;$('h_efft').value=s.drive.effect_time;$('h_small').checked=s.drive.effect_small_only;$('h_rdots').value=s.refresh.dots;$('h_rimg').value=s.refresh.image;
  $('ha_en').checked=s.home_assistant.enabled;$('ha_host').value=s.home_assistant.host;$('ha_port').value=s.home_assistant.port;$('ha_user').value=s.home_assistant.user;
  $('ha_pass').placeholder=s.home_assistant.password_set?'unchanged':'none';$('ha_url').value=s.home_assistant.api_url;
  $('ha_token').value='';$('ha_token').placeholder=s.home_assistant.token_set?'unchanged':'none'}
@@ -544,7 +549,7 @@ function collect(){const s=JSON.parse(JSON.stringify(settings));
  Object.assign(s.schedule,{enabled:$('s_en').checked,day_start:$('s_day').value,late_start:$('s_late').value,quarter_hours:$('s_q').checked,night_start:$('s_night').value.trim()});
  s.clock={no_wifi_screen:$('c_nw').checked};
  s.flip_time=+$('h_flip').value;s.status_led=$('h_led').checked;
- s.drive={flip_gap:+$('h_gap').value,pulses:+$('h_pulses').value,effect:+$('h_effect').value,effect_time:+$('h_efft').value,effect_small_only:$('h_small').checked};
+ s.drive={flip_gap:+$('h_gap').value,pulses:+$('h_pulses').value,effect:+$('h_effect').value,effect_time:+$('h_efft').value,effect_small_only:$('h_small').checked};s.refresh={dots:+$('h_rdots').value,image:+$('h_rimg').value};
  Object.assign(s.home_assistant,{enabled:$('ha_en').checked,host:$('ha_host').value.trim(),port:+$('ha_port').value,user:$('ha_user').value.trim(),api_url:$('ha_url').value.trim()});
  if($('ha_pass').value)s.home_assistant.password=$('ha_pass').value;if($('ha_token').value)s.home_assistant.token=$('ha_token').value.trim();return s}
 async function saveSettings(){const r=await post('/api/settings',collect());settings=r.settings;fill();msg(r.restart?'Saved, restarting...':'Settings saved')}

src/esp32/web/web_ui.cpp

@@ -170,6 +170,24 @@ static void handleStatus()
 	sw["running"] = stopwatch.running();
 	sw["ms"] = stopwatch.elapsed(millis());
 
+	JsonObject load = doc["load"].to<JsonObject>();
+	load["loops_per_s"] = loopStats.loopsPerSecond();
+	load["avg_us"] = loopStats.averageUs();
+	load["max_ms"] = loopStats.maxUs() / 1000;
+	load["max_ever_ms"] = loopStats.maxEverUs() / 1000;
+	load["heap_free"] = ESP.getFreeHeap();
+	load["heap_min"] = ESP.getMinFreeHeap();
+
+	const FrameCheckStats &fc = nanoMonitor.stats();
+	JsonObject link = doc["nano_link"].to<JsonObject>();
+	link["checks"] = fc.checks;
+	link["mismatches"] = fc.mismatches;
+	link["no_reply"] = fc.no_reply;
+	link["errors"] = fc.errors;
+	link["last_error"] = fc.last_error;
+	link["restarts"] = fc.restarts;
+	link["repairs"] = fc.repairs;
+
 	JsonObject ha = doc["ha"].to<JsonObject>();
 	ha["enabled"] = appSettings().home_assistant.enabled;
 	ha["connected"] = homeAssistantConnected();
@@ -570,6 +588,9 @@ static void settingsToJson(const Settings &s, JsonObject o)
 	drive["effect"] = s.drive.effect;
 	drive["effect_time"] = s.drive.effect_time;
 	drive["effect_small_only"] = s.drive.effect_small_only;
+	JsonObject refresh = o["refresh"].to<JsonObject>();
+	refresh["dots"] = s.refresh.dots;
+	refresh["image"] = s.refresh.image;
 
 	JsonObject schedule = o["schedule"].to<JsonObject>();
 	schedule["enabled"] = s.schedule.enabled;
@@ -633,6 +654,11 @@ static void handleSettingsPost()
 	if (drive["effect_time"].is<int>())
 		s.drive.effect_time = constrain(drive["effect_time"].as<int>(), 0, 65535);
 	copyBool(drive["effect_small_only"], s.drive.effect_small_only);
+	JsonVariantConst refresh = body["refresh"];
+	if (refresh["dots"].is<int>())
+		s.refresh.dots = constrain(refresh["dots"].as<int>(), 0, 65535);
+	if (refresh["image"].is<int>())
+		s.refresh.image = constrain(refresh["image"].as<int>(), 0, 65535);
 
 	JsonVariantConst schedule = body["schedule"];
 	copyBool(schedule["enabled"], s.schedule.enabled);

src/nano/README.md

@@ -46,11 +46,13 @@ Arduino core, EEPROM and the display driver are replaced by fakes in
   being read; while a new frame is being drawn into the buffer the effect pauses, and the
   next `D` starts it again. The orders come from `DotSequence` in
   [lib/BROSE9323/dot_sequence.h](../../lib/BROSE9323/dot_sequence.h).
-- **Every 2 minutes** (`REFRESH_TIME`) it does a full redraw to fix dots that got stuck.
+- **Nothing blocks.** Every frame (also instant ones, `D1`) is flipped from `update()` in
-  A full redraw blocks for about half a second, which is long enough to overflow the
+  slices of about 1 ms, so `loop()` keeps reading the serial port. A blocking redraw used
-  64-byte serial buffer. So it only runs right after a `D` (the end of a frame, when the
+  to overflow the 64-byte receive buffer; the lost bytes shifted the rest of a bitmap
-  ESP is quiet) or when no serial data has arrived for 2 seconds, and never while an
+  (the picture moved up and left) and nothing repaired it.
-  effect runs. It is always instant, with one pulse per dot.
+- **Every 2 minutes** (`I` command, stored in EEPROM, 0 = never) it flips every dot once
+  more, with one pulse, to fix dots that got stuck. This also runs in slices, only while
+  no frame is being received or shown, and continues where it was after a frame.
 - **Flip pulse length** (µs, 50–2000) is set with the `T` command and stored in EEPROM,
   so it is kept after a reboot. The default is 200 µs. `G`, `N` and `E` are stored the
   same way (addresses in `settings.cpp`; values that were never written fall back to the
@@ -78,19 +80,24 @@ skipped and the next line starts clean. Coordinates are 2-digit **hex**. Color i
 | `s` | `sXXYYc` | Set pixel in the buffer |
 | `S` | `SXXYYc` | Set pixel and flip it right away (direct mode) |
 | `F` | `Fc` | Fill the whole buffer |
-| `D` | `D` or `D1` | Push the buffer to the display (changed dots only). `D1`: a frame of moving content (scrolling text, seconds), shown without the refresh effect; the periodic full redraw then waits until the serial line is idle. |
+| `D` | `D` or `D1` | Push the buffer to the display (changed dots only). `D1`: a frame of moving content (scrolling text, seconds), shown without the refresh effect. |
 | `P` | `PXXYYcsTEXT` | Print text at (XX,YY), color `c`, size `s`. A literal `\n` (backslash, n) in TEXT starts a new line. The whole line can be at most 64 characters. |
 | `B` | `BXXYYWWHHcHEX...` | Draw a bitmap. WW must be a multiple of 4. Pixel data is one hex digit per 4 pixels, row by row. The data is drawn while it arrives, so it is not limited to 64 characters. |
 | `R` | `R` | Print the buffer as ASCII art on the serial port |
 | `T` | `Tnnn` | Set the flip pulse length in µs (decimal, 50–2000), stored in EEPROM |
 | `G` | `Gnnn` | Pause after each pulse in µs (decimal, 0–5000), stored in EEPROM |
 | `N` | `Nn` | Pulses for each changed dot (1–3), stored in EEPROM |
+| `I` | `Innn` | Minutes between full redraws (decimal, 0–1440, 0 = never), stored in EEPROM |
+| `C` | `C` | Reply `C xxxx`: checksum of the image received so far (CRC-16/CCITT of the buffer, only the dots, `lib/BROSE9323/buffer_crc.h`), 4 hex digits |
 | `E` | `Ee,nnnn[,f]` | Refresh effect `e` (0 instant, 1 sparkle, 2 wipe, 3 rain, 4 from the middle out, 5 diagonal, 6 spiral, 7 flip all then reveal, 8 push, 9 random), its time in ms (100–10000) and flags `f` (1 = only small changes, default 0), stored in EEPROM |
 
 Replies on the Nano's TX pin:
 
-- `READY` after booting. The ESP waits for it before sending the first frame.
+- `READY` after booting. The ESP waits for it before sending the first frame. Later
-- `ERR: ...` for invalid commands.
+  (while it is on) it means the Nano restarted, e.g. after a power dip.
+- `C xxxx` for `C`. The ESP sends `C` after frames (at most every 3 s) and compares the
+  answer with its own copy; a wrong or missing answer makes it send the whole image again.
+- `ERR: ...` for invalid commands. The ESP counts them (web UI, System).
 
 Example (what the ESP32 sends for one pixel and a refresh):
 

src/nano/command_parser.cpp

@@ -71,7 +71,6 @@ void CommandParser::feed(char c)
 		return;
 	}
 	_line[_line_len++] = c;
-	_frame_ended = false;
 
 	// Bitmap header complete: draw the pixel data as it arrives
 	if (_line_len == BITMAP_HEADER_LEN && isBitmapCommand(_line[0]))
@@ -82,11 +81,6 @@ void CommandParser::feed(char c)
 	}
 }
 
-bool CommandParser::frameEnded() const
-{
-	return _frame_ended;
-}
-
 // Run one complete command line (without the '\n')
 void CommandParser::_runLine()
 {
@@ -102,6 +96,10 @@ void CommandParser::_runLine()
 		// Complete bitmap headers are handled in feed()
 		_error(F("Incomplete bitmap header"));
 		break;
+	case 'c':
+	case 'C':
+		_checksum();
+		break;
 	case 'e':
 	case 'E':
 		_effect();
@@ -110,21 +108,18 @@ void CommandParser::_runLine()
 	case 'G':
 		_gap();
 		break;
+	case 'i':
+	case 'I':
+		_refreshInterval();
+		break;
 	case 'n':
 	case 'N':
 		_pulses();
 		break;
 	case 'd':
 	case 'D':
-		// D1: a frame of moving content, shown without the effect. More frames follow
+		// D1: a frame of moving content, shown without the effect
-		// soon, so the periodic full redraw waits until the serial line is idle.
+		_display.display(false, _line[1] == '1');
-		if (_line[1] == '1')
-			_display.display(false, true);
-		else
-		{
-			_display.display();
-			_frame_ended = true;
-		}
 		break;
 	case 'f':
 	case 'F':
@@ -320,3 +315,28 @@ void CommandParser::_effect()
 	_display.setEffect(effect, t, flags);
 	saveEffect(effect, t, flags);
 }
+
+// C: replies "C xxxx", the bufferCrc() of the image received so far as 4 hex digits. The
+// ESP32 sends it after a frame and compares it with what it sent.
+void CommandParser::_checksum()
+{
+	uint16_t crc = _display.crc();
+	char text[7] = "C ";
+	for (uint8_t i = 0; i < 4; i++)
+		text[2 + i] = "0123456789ABCDEF"[(crc >> (12 - 4 * i)) & 0xF];
+	text[6] = '\0';
+	_reply.println(text);
+}
+
+// Immm (mmm = decimal minutes between full redraws, 0 = never, up to 1440), stored in EEPROM
+void CommandParser::_refreshInterval()
+{
+	long m = atol(_line + 1);
+	if (_line_len < 2 || m < 0 || m > REFRESH_MINUTES_MAX)
+	{
+		_error(F("Refresh interval out of range (0-1440 min)"));
+		return;
+	}
+	_refresh_minutes = m;
+	saveRefreshMinutes(m);
+}

src/nano/command_parser.h

@@ -16,8 +16,9 @@ public:
 	// Handle one received byte
 	void feed(char c);
 
-	// True when the last command was D (end of a frame) and no new command has started
+	// Full redraw interval in minutes, 0 = never (I command; loaded from EEPROM at boot)
-	bool frameEnded() const;
+	uint16_t refreshMinutes() const { return _refresh_minutes; }
+	void setRefreshMinutes(uint16_t minutes) { _refresh_minutes = minutes; }
 
 private:
 	BROSE9323 &_display;
@@ -26,7 +27,7 @@ private:
 	char _line[LINE_MAX + 1];
 	uint8_t _line_len = 0;
 	bool _discard_line = false; // skip everything until the next '\n'
-	bool _frame_ended = false;
+	uint16_t _refresh_minutes = REFRESH_MINUTES_DEFAULT;
 
 	// Bitmap (B) pixel data is drawn while it arrives instead of being buffered
 	bool _bitmap_active = false;
@@ -47,4 +48,6 @@ private:
 	void _gap();
 	void _pulses();
 	void _effect();
+	void _checksum();
+	void _refreshInterval();
 };

src/nano/config.h

@@ -42,8 +42,10 @@ const uint16_t EFFECT_TIME_MIN = 100;
 const uint16_t EFFECT_TIME_MAX = 10000;
 const uint8_t EFFECT_FLAGS_MAX = 1; // EFFECT_FLAG_SMALL_ONLY
 
-// Full redraw interval, fixes dots that did not flip
+// Full redraw interval in minutes, fixes dots that did not flip (I command, EEPROM;
-const uint32_t REFRESH_TIME = 120000UL; // 2 min
+// 0 = never). It runs from update() between frames and never blocks the serial port.
+const uint16_t REFRESH_MINUTES_DEFAULT = 2;
+const uint16_t REFRESH_MINUTES_MAX = 1440;
 
 // ---------------------------------------------------------------------------
 // Serial link to the ESP32
@@ -51,8 +53,5 @@ const uint32_t REFRESH_TIME = 120000UL; // 2 min
 
 const uint32_t SERIAL_BAUD = 115200;
 
-// Serial line quiet for this long: safe to do a due full redraw without waiting for a D
-const uint32_t IDLE_TIME = 2000UL;
-
 // Longest command line (bitmap data is streamed and does not count)
 const uint8_t LINE_MAX = 64;

src/nano/main.cpp

@@ -12,20 +12,17 @@ BROSE9323 display(DISPLAY_WIDTH, DISPLAY_HEIGHT, PANEL_WIDTH, FLIP_TIME_DEFAULT)
 CommandParser parser(display, Serial);
 
 uint32_t last_refresh;
-uint32_t last_rx;
 
-// Force a full redraw every REFRESH_TIME. The redraw blocks long enough to overflow the
+// Flip every dot again each refresh interval. The driver does it from update(), between
-// 64 byte serial buffer, so it only runs when no data is expected: right after the ESP
+// frames and without blocking, so it can start at any time.
-// finished a frame, or when the serial line is idle.
 void refreshIfDue()
 {
-	// Not while an effect is drawing a frame (it flips every changed dot itself)
+	uint16_t minutes = parser.refreshMinutes();
-	if (display.busy())
+	if (minutes == 0 || display.refreshing())
 		return;
-	bool quiet = parser.frameEnded() || millis() - last_rx >= IDLE_TIME;
+	if (millis() - last_refresh >= minutes * 60000UL)
-	if (quiet && millis() - last_refresh >= REFRESH_TIME)
 	{
-		display.display(true);
+		display.startRefresh();
 		last_refresh = millis();
 	}
 }
@@ -37,9 +34,13 @@ void measureRefresh()
 
 	display.fillScreen(1);
 	display.display();
+	while (display.update())
+		;
 	display.fillScreen(0);
 	uint32_t t1 = millis();
 	display.display();
+	while (display.update())
+		;
 	uint32_t t2 = millis();
 	Serial.println(t2 - t1);
 }
@@ -56,6 +57,7 @@ void setup()
 	uint16_t effect_time;
 	loadEffect(effect, effect_time, effect_flags);
 	display.setEffect(effect, effect_time, effect_flags);
+	parser.setRefreshMinutes(loadRefreshMinutes());
 	display.begin();
 
 #ifdef MEASURE_REFRESH
@@ -63,7 +65,6 @@ void setup()
 #endif
 
 	last_refresh = millis();
-	last_rx = millis();
 
 	// Tells the ESP that frames can be sent (the relay powers the Nano, so this happens at every power on)
 	Serial.println(F("READY"));
@@ -72,10 +73,7 @@ void setup()
 void loop()
 {
 	while (Serial.available())
-	{
-		last_rx = millis();
 		parser.feed(Serial.read());
-	}
 
 	display.update();
 	refreshIfDue();

src/nano/settings.cpp

@@ -14,6 +14,7 @@ const int EEPROM_PULSES_ADDR = 6;
 const int EEPROM_EFFECT_ADDR = 7;
 const int EEPROM_EFFECT_TIME_ADDR = 8;
 const int EEPROM_EFFECT_FLAGS_ADDR = 10;
+const int EEPROM_REFRESH_ADDR = 11;
 const uint16_t EEPROM_MAGIC = 0xF11D;
 
 static bool valid()
@@ -93,3 +94,17 @@ void saveEffect(uint8_t effect, uint16_t time_ms, uint8_t flags)
 	save(EEPROM_EFFECT_TIME_ADDR, time_ms);
 	save(EEPROM_EFFECT_FLAGS_ADDR, flags);
 }
+
+uint16_t loadRefreshMinutes()
+{
+	uint16_t m;
+	EEPROM.get(EEPROM_REFRESH_ADDR, m);
+	if (!valid() || m > REFRESH_MINUTES_MAX)
+		return REFRESH_MINUTES_DEFAULT;
+	return m;
+}
+
+void saveRefreshMinutes(uint16_t m)
+{
+	save(EEPROM_REFRESH_ADDR, m);
+}

src/nano/settings.h

@@ -16,3 +16,7 @@ void savePulses(uint8_t n);
 
 void loadEffect(uint8_t &effect, uint16_t &time_ms, uint8_t &flags);
 void saveEffect(uint8_t effect, uint16_t time_ms, uint8_t flags);
+
+// Full redraw interval in minutes, 0 = never
+uint16_t loadRefreshMinutes();
+void saveRefreshMinutes(uint16_t m);

test/driver_mocks/Adafruit_GFX.h

@@ -0,0 +1,112 @@
+// Just enough Arduino and Adafruit GFX to run the real BROSE9323 driver (Nano build) on
+// the PC, with a simulated panel: test/test_driver. The panel decodes the pin levels at
+// each strobe (ENABLE low) into the dot that flips, like the Brose panel electronics.
+#pragma once
+
+#include <cstdint>
+#include <cstdio>
+#include <cstdlib>
+#include <cstring>
+#include <string>
+
+#define A0 14
+#define A1 15
+#define A2 16
+#define A3 17
+#define A4 18
+#define A5 19
+#define OUTPUT 1
+
+// Every micros() call takes a microsecond, so loops that wait for the time end
+inline uint32_t fake_micros = 0;
+inline uint32_t micros() { return ++fake_micros; }
+inline uint32_t millis() { return fake_micros / 1000; }
+inline void delayMicroseconds(unsigned int us) { fake_micros += us; }
+inline void pinMode(uint8_t, uint8_t) {}
+
+struct FakePanel
+{
+	static const int W = 84, H = 19, PANEL_W = 28;
+	bool dot[W][H] = {};
+	int flips[W][H] = {}; // strobes per dot
+	int strobes = 0;
+	bool pins[20] = {};
+
+	void clearCounts()
+	{
+		memset(flips, 0, sizeof(flips));
+		strobes = 0;
+	}
+	void write(uint8_t pin, bool level)
+	{
+		pins[pin] = level;
+		if (pin == 7 && !level) // ENABLE low: flip the addressed dot
+			strobe();
+	}
+	void strobe()
+	{
+		int panel = pins[3] | pins[5] << 1 | pins[6] << 2;
+		int col_code = pins[8] | pins[10] << 1 | pins[9] << 2 | pins[11] << 3 | pins[12] << 4;
+		int row_code = pins[A3] | pins[A4] << 1 | pins[A2] << 2 | pins[A0] << 3 | pins[A1] << 4;
+		int col = -1, row = -1;
+		for (int c = 0; c < PANEL_W; c++)
+			if (c + 1 + c / 7 == col_code)
+				col = c;
+		for (int r = 0; r < H; r++)
+			if (r + r / 7 == row_code)
+				row = r;
+		if (col < 0 || row < 0 || panel > 2)
+		{
+			fprintf(stderr, "bad address: panel %d col code %d row code %d\n", panel, col_code, row_code);
+			abort();
+		}
+		int x = panel * PANEL_W + col;
+		dot[x][row] = !pins[4]; // COL_DATA low sets the dot (yellow)
+		flips[x][row]++;
+		strobes++;
+	}
+};
+
+inline FakePanel panel;
+inline void digitalWrite(uint8_t pin, int level) { panel.write(pin, level != 0); }
+
+struct FakeSerialOut
+{
+	void write(char) {}
+	void println() {}
+};
+inline FakeSerialOut Serial;
+
+// Only what BROSE9323::print() uses
+class String
+{
+public:
+	String(const char *s) : _s(s) {}
+	void replace(const char *from, const char *to)
+	{
+		size_t pos = 0, n = strlen(from);
+		while ((pos = _s.find(from, pos)) != std::string::npos)
+		{
+			_s.replace(pos, n, to);
+			pos += strlen(to);
+		}
+	}
+	const char *c_str() const { return _s.c_str(); }
+
+private:
+	std::string _s;
+};
+
+class Adafruit_GFX
+{
+public:
+	Adafruit_GFX(int16_t w, int16_t h) : _width(w), _height(h) {}
+	virtual ~Adafruit_GFX() {}
+	virtual void drawPixel(int16_t x, int16_t y, uint16_t color) = 0;
+	int16_t width() const { return _width; }
+	int16_t height() const { return _height; }
+	size_t print(const char *s) { return strlen(s); }
+
+protected:
+	int16_t _width, _height;
+};

test/mocks/BROSE9323.h

@@ -19,6 +19,8 @@ public:
 	std::vector<DisplayCall> calls;
 	bool direct = false;
 	bool effect_running = false; // what busy() returns
+	bool refresh_running = false; // what refreshing() returns
+	uint16_t crc_value = 0;       // what crc() returns
 
 	BROSE9323(uint8_t, uint8_t, uint8_t, uint16_t) {}
 
@@ -33,6 +35,9 @@ public:
 	void setEffect(uint8_t effect, uint16_t time_ms, uint8_t flags = 0) { log("effect", effect, time_ms, flags); }
 	bool update() { return effect_running; }
 	bool busy() const { return effect_running; }
+	void startRefresh() { log("refresh"); }
+	bool refreshing() const { return refresh_running; }
+	uint16_t crc() const { return crc_value; }
 	void printBuffer() { log("printBuffer"); }
 	void setCursor(int16_t x, int16_t y) { log("cursor", x, y); }
 	void setTextColor(uint16_t color, uint16_t background) { log("textColor", color, background); }

test/test_driver/test_main.cpp

@@ -0,0 +1,241 @@
+// Tests for the real display driver (lib/BROSE9323, Nano build) on the PC, with the
+// simulated panel of test/driver_mocks: pio test -e test_driver
+//
+// What matters most: drawing never blocks loop() (the Nano must keep reading the serial
+// port, or bytes are lost and the picture ends up shifted), and the dots end up right.
+
+#include <unity.h>
+
+#include "../../lib/BROSE9323/BROSE9323.cpp"
+
+static const int W = 84, H = 19;
+
+// Like the user's settings: 3 pulses of 200 us with a 300 us gap
+static BROSE9323 *make()
+{
+	BROSE9323 *d = new BROSE9323(W, H, 28, 200);
+	d->setGap(300);
+	d->setPulses(3);
+	panel = FakePanel(); // before begin(): it sets the pins
+	d->begin();
+	return d;
+}
+
+// Runs update() until the frame and the refresh are done. Fails when one call takes
+// longer than max_us (fake time): then loop() would not read the serial port in time.
+static int finish(BROSE9323 &d, uint32_t max_us = 3000)
+{
+	int calls = 0;
+	while (d.busy() || d.refreshing())
+	{
+		uint32_t start = fake_micros;
+		d.update();
+		uint32_t took = fake_micros - start;
+		if (took > max_us)
+		{
+			char msg[64];
+			snprintf(msg, sizeof(msg), "update() took %u us", (unsigned)took);
+			TEST_FAIL_MESSAGE(msg);
+		}
+		TEST_ASSERT_LESS_THAN(1000000, ++calls);
+	}
+	return calls;
+}
+
+static void assertPanelShows(BROSE9323 &d)
+{
+	for (int x = 0; x < W; x++)
+		for (int y = 0; y < H; y++)
+			if (panel.dot[x][y] != d.shownPixel(x, y))
+			{
+				char msg[64];
+				snprintf(msg, sizeof(msg), "dot %d,%d differs", x, y);
+				TEST_FAIL_MESSAGE(msg);
+			}
+}
+
+static void drawPattern(BROSE9323 &d, int seed)
+{
+	for (int x = 0; x < W; x++)
+		for (int y = 0; y < H; y++)
+			d.drawPixel(x, y, ((x * 7 + y * 3 + seed) % 5) == 0);
+}
+
+static int dotsFlippedAtLeast(int n)
+{
+	int count = 0;
+	for (int x = 0; x < W; x++)
+		for (int y = 0; y < H; y++)
+			count += panel.flips[x][y] >= n;
+	return count;
+}
+
+void setUp() {}
+void tearDown() {}
+
+void test_first_frame_flips_every_dot_without_blocking()
+{
+	BROSE9323 *d = make();
+	drawPattern(*d, 0);
+	uint32_t start = fake_micros;
+	d->display();
+	TEST_ASSERT_LESS_THAN(3000, fake_micros - start); // display() itself returns at once
+	TEST_ASSERT_TRUE(d->busy());
+	int calls = finish(*d);
+	TEST_ASSERT_GREATER_THAN(100, calls); // spread over many loop() passes
+	TEST_ASSERT_EQUAL(W * H, dotsFlippedAtLeast(1));
+	assertPanelShows(*d);
+	delete d;
+}
+
+void test_instant_frame_flips_only_the_changed_dots_with_all_pulses()
+{
+	BROSE9323 *d = make();
+	drawPattern(*d, 0);
+	d->display();
+	finish(*d);
+	panel.clearCounts();
+
+	d->drawPixel(10, 5, !d->shownPixel(10, 5));
+	d->drawPixel(70, 18, !d->shownPixel(70, 18));
+	d->display(false, true);
+	finish(*d);
+	TEST_ASSERT_EQUAL(6, panel.strobes); // 2 dots, 3 pulses each
+	TEST_ASSERT_EQUAL(3, panel.flips[10][5]);
+	assertPanelShows(*d);
+	delete d;
+}
+
+void test_effects_end_with_the_right_image()
+{
+	for (uint8_t effect = EFFECT_SPARKLE; effect < EFFECT_COUNT; effect++)
+	{
+		BROSE9323 *d = make();
+		d->setEffect(effect, 500);
+		drawPattern(*d, 0);
+		d->display();
+		finish(*d);
+		assertPanelShows(*d);
+		drawPattern(*d, 2);
+		d->display();
+		finish(*d, 5000);
+		assertPanelShows(*d);
+		delete d;
+	}
+}
+
+void test_refresh_flips_every_dot_once_without_blocking()
+{
+	BROSE9323 *d = make();
+	drawPattern(*d, 0);
+	d->display();
+	finish(*d);
+	panel.clearCounts();
+
+	d->startRefresh();
+	TEST_ASSERT_TRUE(d->refreshing());
+	TEST_ASSERT_FALSE(d->busy()); // not a frame: loop() may take new frames meanwhile
+	int calls = finish(*d);
+	TEST_ASSERT_GREATER_THAN(100, calls);
+	TEST_ASSERT_EQUAL(W * H, panel.strobes); // one pulse each, nothing changed
+	TEST_ASSERT_EQUAL(W * H, dotsFlippedAtLeast(1));
+	assertPanelShows(*d);
+	delete d;
+}
+
+void test_refresh_pauses_while_a_frame_arrives_and_then_finishes()
+{
+	BROSE9323 *d = make();
+	drawPattern(*d, 0);
+	d->display();
+	finish(*d);
+	panel.clearCounts();
+
+	d->startRefresh();
+	d->update(); // a part of the dots
+	int before = panel.strobes;
+	TEST_ASSERT_GREATER_THAN(0, before);
+	TEST_ASSERT_LESS_THAN(W * H, before);
+
+	// A new frame is being received: the refresh must not show half of it
+	bool old = d->shownPixel(83, 18);
+	d->drawPixel(83, 18, !old);
+	for (int i = 0; i < 50; i++)
+		d->update();
+	TEST_ASSERT_EQUAL(before, panel.strobes);
+	TEST_ASSERT_EQUAL(old, panel.dot[83][18]);
+
+	d->display(false, true);
+	finish(*d);
+	assertPanelShows(*d);
+	TEST_ASSERT_EQUAL(W * H, dotsFlippedAtLeast(1)); // the refresh got to the end
+	TEST_ASSERT_FALSE(d->refreshing());
+	delete d;
+}
+
+void test_refresh_waits_for_a_running_effect()
+{
+	BROSE9323 *d = make();
+	d->setEffect(EFFECT_WIPE, 1000);
+	drawPattern(*d, 0);
+	d->display();
+	finish(*d);
+	drawPattern(*d, 1);
+	d->display();
+	d->startRefresh();
+	panel.clearCounts();
+	// The wipe goes left to right; the refresh would start at the left too and finish
+	// before the wipe reaches the right side if it ran at the same time
+	while (d->busy())
+		d->update();
+	TEST_ASSERT_TRUE(d->refreshing());
+	finish(*d);
+	assertPanelShows(*d);
+	delete d;
+}
+
+void test_interrupted_first_frame_still_reaches_every_dot()
+{
+	BROSE9323 *d = make();
+	drawPattern(*d, 0);
+	d->display(); // after boot: every dot
+	for (int i = 0; i < 20; i++)
+		d->update();
+	TEST_ASSERT_LESS_THAN(W * H, dotsFlippedAtLeast(1));
+
+	// The next frame (scrolling text) arrives before it is done
+	d->drawPixel(0, 0, !d->shownPixel(0, 0));
+	d->display(false, true);
+	finish(*d);
+	TEST_ASSERT_EQUAL(W * H, dotsFlippedAtLeast(1));
+	assertPanelShows(*d);
+	delete d;
+}
+
+void test_crc_of_equal_images_is_equal()
+{
+	BROSE9323 *a = make(), *b = make();
+	a->fillScreen(1); // also sets the unused bits at the end of each row
+	for (int x = 0; x < W; x++)
+		for (int y = 0; y < H; y++)
+			b->drawPixel(x, y, 1);
+	TEST_ASSERT_EQUAL_HEX16(a->crc(), b->crc());
+	b->drawPixel(40, 9, 0);
+	TEST_ASSERT_NOT_EQUAL(a->crc(), b->crc());
+	delete a;
+	delete b;
+}
+
+int main()
+{
+	UNITY_BEGIN();
+	RUN_TEST(test_first_frame_flips_every_dot_without_blocking);
+	RUN_TEST(test_instant_frame_flips_only_the_changed_dots_with_all_pulses);
+	RUN_TEST(test_effects_end_with_the_right_image);
+	RUN_TEST(test_refresh_flips_every_dot_once_without_blocking);
+	RUN_TEST(test_refresh_pauses_while_a_frame_arrives_and_then_finishes);
+	RUN_TEST(test_refresh_waits_for_a_running_effect);
+	RUN_TEST(test_interrupted_first_frame_still_reaches_every_dot);
+	RUN_TEST(test_crc_of_equal_images_is_equal);
+	return UNITY_END();
+}

test/test_esp32/test_main.cpp

@@ -12,6 +12,8 @@
 #include <vector>
 
 #include "config.h"
+#include "logic/frame_check.h"
+#include "logic/loop_stats.h"
 #include "logic/bitmap_format.h"
 #include "logic/builtin_fonts.h"
 #include "logic/builtin_pages.h"
@@ -1900,6 +1902,133 @@ void test_ha_text_map()
 	TEST_ASSERT_NOT_NULL(validatePage(page));
 }
 
+void test_loop_stats()
+{
+	LoopStats stats;
+	TEST_ASSERT_EQUAL_UINT32(0, stats.loopsPerSecond());
+	// 995 passes of 10 ms (the micros() counter wraps on the way), one of them 50 ms
+	uint32_t t = 0xFFFFFFFFUL - 5000000UL;
+	stats.tick(t);
+	for (int i = 0; i < 995; i++)
+	{
+		t += i == 500 ? 50000 : 10000;
+		stats.tick(t);
+	}
+	TEST_ASSERT_EQUAL_UINT32(0, stats.loopsPerSecond()); // window not complete yet
+	t += 10000;
+	stats.tick(t);
+	TEST_ASSERT_EQUAL_UINT32(99, stats.loopsPerSecond());
+	TEST_ASSERT_EQUAL_UINT32(10040, stats.averageUs());
+	TEST_ASSERT_EQUAL_UINT32(50000, stats.maxUs());
+	TEST_ASSERT_EQUAL_UINT32(50000, stats.maxEverUs());
+
+	// Next window without the slow pass: the window max drops, the boot max stays
+	for (int i = 0; i < 1000; i++)
+	{
+		t += 10000;
+		stats.tick(t);
+	}
+	TEST_ASSERT_EQUAL_UINT32(100, stats.loopsPerSecond());
+	TEST_ASSERT_EQUAL_UINT32(10000, stats.maxUs());
+	TEST_ASSERT_EQUAL_UINT32(50000, stats.maxEverUs());
+}
+
+void test_frame_check_ok()
+{
+	FrameCheck c;
+	c.reset(1000);
+	TEST_ASSERT_TRUE(c.due(5, 1000)); // first check right after switching on
+	c.sent(0x1234, 5, 1000);
+	TEST_ASSERT_FALSE(c.due(6, 5000)); // waiting for the answer
+	c.line("C 1234\r", 1010);
+	TEST_ASSERT_EQUAL_UINT32(1, c.stats().checks);
+	TEST_ASSERT_FALSE(c.resyncNow(1020, 0));
+
+	// The next one after a new frame, at most every FRAME_CHECK_INTERVAL
+	TEST_ASSERT_FALSE(c.due(5, 10000)); // no new frame
+	TEST_ASSERT_FALSE(c.due(6, 1000 + FRAME_CHECK_INTERVAL - 1));
+	TEST_ASSERT_TRUE(c.due(6, 1000 + FRAME_CHECK_INTERVAL));
+}
+
+void test_frame_check_mismatch_repairs()
+{
+	FrameCheck c;
+	c.reset(0);
+	c.sent(0x1234, 1, 20000);
+	c.line("C 1235", 20010);
+	TEST_ASSERT_EQUAL_UINT32(1, c.stats().mismatches);
+	TEST_ASSERT_FALSE(c.due(2, 30000)); // the repair comes first
+	TEST_ASSERT_TRUE(c.resyncNow(20010, 0));
+	TEST_ASSERT_EQUAL_UINT32(1, c.stats().repairs);
+	TEST_ASSERT_FALSE(c.resyncNow(20020, 0)); // once
+	TEST_ASSERT_TRUE(c.due(2, 30000));        // then the repaired frame is checked
+}
+
+void test_frame_check_repairs_at_most_every_gap()
+{
+	FrameCheck c;
+	c.reset(0);
+	c.sent(1, 1, 100);
+	c.line("C 0002", 110);
+	TEST_ASSERT_FALSE(c.resyncNow(FRAME_REPAIR_MIN_GAP - 1, 0)); // switched on just now
+	TEST_ASSERT_TRUE(c.resyncNow(FRAME_REPAIR_MIN_GAP, 0));
+}
+
+void test_frame_check_no_reply()
+{
+	FrameCheck c;
+	c.reset(0);
+	// Never answered (old Nano firmware, TX not wired): counted, but no repairs
+	c.sent(1, 1, 100);
+	TEST_ASSERT_FALSE(c.resyncNow(100 + FRAME_CHECK_TIMEOUT - 1, 0));
+	TEST_ASSERT_FALSE(c.resyncNow(100 + FRAME_CHECK_TIMEOUT + FRAME_REPAIR_MIN_GAP, 0));
+	TEST_ASSERT_EQUAL_UINT32(1, c.stats().no_reply);
+	TEST_ASSERT_EQUAL_UINT32(0, c.stats().repairs);
+
+	// It answered before: a lost answer means lost bytes, repair
+	uint32_t t = 100000;
+	c.sent(1, 2, t);
+	c.line("C 0001", t + 5);
+	t += FRAME_CHECK_INTERVAL;
+	c.sent(1, 3, t);
+	TEST_ASSERT_TRUE(c.resyncNow(t + FRAME_CHECK_TIMEOUT, 0));
+	TEST_ASSERT_EQUAL_UINT32(2, c.stats().no_reply);
+	TEST_ASSERT_EQUAL_UINT32(1, c.stats().repairs);
+	// A late answer to the timed out check is ignored
+	c.line("C 0009", t + FRAME_CHECK_TIMEOUT + 10);
+	TEST_ASSERT_EQUAL_UINT32(0, c.stats().mismatches);
+}
+
+void test_frame_check_errors_and_restarts()
+{
+	FrameCheck c;
+	c.reset(0);
+	c.line("ERR: Line too long", 10);
+	c.line("ERR: Invalid bitmap width (not dividable by 4) and a lot more text than fits", 20);
+	TEST_ASSERT_EQUAL_UINT32(2, c.stats().errors);
+	TEST_ASSERT_EQUAL_UINT32(sizeof(c.stats().last_error) - 1, strlen(c.stats().last_error));
+	TEST_ASSERT_EQUAL(0, strncmp(c.stats().last_error, "ERR: Invalid bitmap", 19));
+	TEST_ASSERT_FALSE(c.resyncNow(FRAME_REPAIR_MIN_GAP, 0)); // errors alone: the check decides
+
+	c.line("READY", 20000);
+	TEST_ASSERT_EQUAL_UINT32(1, c.stats().restarts);
+	TEST_ASSERT_TRUE(c.resyncNow(20000, 0));
+	c.line("garbage", 20010);
+	c.line("C 12", 20010);
+	TEST_ASSERT_EQUAL_UINT32(0, c.stats().checks + c.stats().mismatches);
+}
+
+void test_frame_check_periodic_resync()
+{
+	FrameCheck c;
+	c.reset(1000);
+	TEST_ASSERT_FALSE(c.resyncNow(1000 + 600000 - 1, 600000));
+	TEST_ASSERT_TRUE(c.resyncNow(1000 + 600000, 600000));
+	TEST_ASSERT_FALSE(c.resyncNow(1000 + 600001, 600000));
+	TEST_ASSERT_FALSE(c.resyncNow(0xFFFFFFF0u, 0)); // 0: never
+	TEST_ASSERT_EQUAL_UINT32(0, c.stats().repairs);   // not counted as a repair
+}
+
 int main()
 {
 	UNITY_BEGIN();
@@ -2022,6 +2151,13 @@ int main()
 	RUN_TEST(test_ha_duration);
 	RUN_TEST(test_ha_timestamps);
 	RUN_TEST(test_ha_text_map);
+	RUN_TEST(test_loop_stats);
+	RUN_TEST(test_frame_check_ok);
+	RUN_TEST(test_frame_check_mismatch_repairs);
+	RUN_TEST(test_frame_check_repairs_at_most_every_gap);
+	RUN_TEST(test_frame_check_no_reply);
+	RUN_TEST(test_frame_check_errors_and_restarts);
+	RUN_TEST(test_frame_check_periodic_resync);
 
 	return UNITY_END();
 }

test/test_nano/test_main.cpp

@@ -8,6 +8,7 @@
 #include "Arduino.h"
 #include "BROSE9323.h"
 #include "EEPROM.h"
+#include "buffer_crc.h"     // the real one, from lib/BROSE9323
 #include "frame_commands.h" // the real one, from lib/BROSE9323
 
 #include <cstdlib>
@@ -117,16 +118,15 @@ void test_display_instant_frame()
 	TEST_ASSERT_EQUAL(1, display.last("display")->b); // without the effect
 }
 
-void test_refresh_waits_for_idle_after_instant_frames()
+void test_refresh_starts_between_instant_frames()
 {
+	// The driver pauses the refresh while frames arrive, so it may start at any time
 	last_refresh = fake_millis;
 	fake_millis += 120000;
 	send("D1\n");
-	TEST_ASSERT_EQUAL(1, display.count("display")); // no full redraw: more frames follow
+	TEST_ASSERT_EQUAL(1, display.count("display"));
-	fake_millis += 3000;
+	TEST_ASSERT_EQUAL(0, display.last("display")->a); // the frame is not forced
-	loop();
+	TEST_ASSERT_EQUAL(1, display.count("refresh"));
-	TEST_ASSERT_EQUAL(2, display.count("display"));
-	TEST_ASSERT_EQUAL(1, display.calls.back().a);
 }
 
 void test_lowercase_and_uppercase_commands()
@@ -455,19 +455,6 @@ void test_invalid_effect_is_rejected()
 	TEST_ASSERT_TRUE(replied("ERR"));
 }
 
-void test_no_full_redraw_while_an_effect_runs()
-{
-	last_refresh = fake_millis;
-	fake_millis += 120000;
-	display.effect_running = true;
-	send("D\n");
-	TEST_ASSERT_EQUAL(1, display.count("display")); // only the frame itself
-	display.effect_running = false;
-	loop();
-	TEST_ASSERT_EQUAL(2, display.count("display"));
-	TEST_ASSERT_EQUAL(1, display.last("display")->a);
-}
-
 // Every effect order visits each dot exactly once
 static void checkSequence(uint8_t effect, uint8_t w, uint8_t h, uint16_t seed)
 {
@@ -560,15 +547,16 @@ void test_sparkle_order_is_scrambled()
 // Periodic full redraw
 // ---------------------------------------------------------------------------
 
-void test_refresh_waits_for_end_of_frame()
+void test_refresh_when_due()
 {
 	last_refresh = fake_millis;
-	fake_millis += 120000;
+	fake_millis += 119999;
-	send("s000001\n");
+	loop();
-	TEST_ASSERT_EQUAL(0, display.count("display"));
+	TEST_ASSERT_EQUAL(0, display.count("refresh"));
-	send("D\n");
+	fake_millis += 1;
-	TEST_ASSERT_EQUAL(2, display.count("display"));
+	loop();
-	TEST_ASSERT_EQUAL(1, display.calls.back().a); // forced full redraw after the frame
+	TEST_ASSERT_EQUAL(1, display.count("refresh"));
+	TEST_ASSERT_EQUAL(0, display.count("display")); // no frame, the driver does it
 }
 
 void test_refresh_only_once_per_interval()
@@ -576,18 +564,19 @@ void test_refresh_only_once_per_interval()
 	last_refresh = fake_millis;
 	fake_millis += 120000;
 	send("D\n");
-	display.calls.clear();
 	send("D\n");
-	TEST_ASSERT_EQUAL(1, display.count("display"));
+	loop();
+	TEST_ASSERT_EQUAL(1, display.count("refresh"));
 }
 
-void test_refresh_when_serial_is_idle()
+void test_running_refresh_is_not_restarted()
 {
 	last_refresh = fake_millis;
 	fake_millis += 120000;
+	display.refresh_running = true;
 	loop();
-	TEST_ASSERT_EQUAL(1, display.count("display"));
+	display.refresh_running = false;
-	TEST_ASSERT_EQUAL(1, display.last("display")->a);
+	TEST_ASSERT_EQUAL(0, display.count("refresh"));
 }
 
 void test_no_refresh_storm_after_millis_wraps()
@@ -597,12 +586,92 @@ void test_no_refresh_storm_after_millis_wraps()
 	fake_millis += 5000; // wraps past 0
 	for (int i = 0; i < 5; i++)
 		send("D\n");
-	TEST_ASSERT_EQUAL(5, display.count("display"));
+	TEST_ASSERT_EQUAL(0, display.count("refresh"));
 
 	fake_millis += 120000;
-	display.calls.clear();
 	send("D\n");
-	TEST_ASSERT_EQUAL(2, display.count("display"));
+	TEST_ASSERT_EQUAL(1, display.count("refresh"));
+}
+
+void test_refresh_interval_is_applied_and_stored()
+{
+	send("I5\n");
+	TEST_ASSERT_FALSE(replied("ERR"));
+	fake_millis += 120000;
+	loop();
+	TEST_ASSERT_EQUAL(0, display.count("refresh"));
+	fake_millis += 180000;
+	loop();
+	TEST_ASSERT_EQUAL(1, display.count("refresh"));
+
+	// Kept over a power cycle
+	setup();
+	last_refresh = fake_millis;
+	display.calls.clear();
+	fake_millis += 120000;
+	loop();
+	TEST_ASSERT_EQUAL(0, display.count("refresh"));
+	send("I2\n");
+}
+
+void test_refresh_interval_zero_never_refreshes()
+{
+	send("I0\n");
+	TEST_ASSERT_FALSE(replied("ERR"));
+	fake_millis += 24UL * 3600 * 1000;
+	loop();
+	TEST_ASSERT_EQUAL(0, display.count("refresh"));
+	send("I2\n");
+}
+
+void test_refresh_interval_out_of_range_is_rejected()
+{
+	send("I1441\n");
+	send("I-1\n");
+	send("I\n");
+	TEST_ASSERT_TRUE(replied("ERR"));
+	fake_millis += 120000;
+	loop();
+	TEST_ASSERT_EQUAL(1, display.count("refresh")); // still every 2 minutes
+}
+
+// ---------------------------------------------------------------------------
+// Checksum
+// ---------------------------------------------------------------------------
+
+void test_checksum_reply()
+{
+	display.crc_value = 0x0A1F;
+	send("C\n");
+	TEST_ASSERT_EQUAL_STRING("C 0A1F\n", Serial.out.c_str());
+	display.crc_value = 0xFFFF;
+	send("c\n");
+	TEST_ASSERT_EQUAL_STRING("C 0A1F\nC FFFF\n", Serial.out.c_str());
+}
+
+void test_checksum_follows_the_frame()
+{
+	// The reply comes after the frame's commands ran (they are handled in order)
+	display.crc_value = 0x1234;
+	send("B0000040118\nD\nC\n");
+	TEST_ASSERT_EQUAL(1, display.count("display"));
+	TEST_ASSERT_EQUAL(1, display.count("pixel"));
+	TEST_ASSERT_EQUAL_STRING("C 1234\n", Serial.out.c_str());
+}
+
+void test_crc_counts_only_the_dots()
+{
+	// 84 x 19: 11 bytes per row, the top 4 bits of each 11th byte are not dots
+	uint8_t a[11 * 19] = {}, b[11 * 19] = {};
+	TEST_ASSERT_EQUAL_HEX16(bufferCrc(a, 84, 19), bufferCrc(b, 84, 19));
+	for (int y = 0; y < 19; y++)
+		b[y * 11 + 10] = 0xF0; // what fillScreen(1) leaves there
+	TEST_ASSERT_EQUAL_HEX16(bufferCrc(a, 84, 19), bufferCrc(b, 84, 19));
+	b[5] = 0x01; // one dot
+	TEST_ASSERT_NOT_EQUAL(bufferCrc(a, 84, 19), bufferCrc(b, 84, 19));
+	// CRC-16/CCITT-FALSE check value
+	const uint8_t check[9] = {'1', '2', '3', '4', '5', '6', '7', '8', '9'};
+	TEST_ASSERT_EQUAL_HEX16(0x29B1, bufferCrc(check, 72, 1));
 }
 
 int main()
@@ -618,7 +687,7 @@ int main()
 	RUN_TEST(test_fill);
 	RUN_TEST(test_display);
 	RUN_TEST(test_display_instant_frame);
-	RUN_TEST(test_refresh_waits_for_idle_after_instant_frames);
+	RUN_TEST(test_refresh_starts_between_instant_frames);
 	RUN_TEST(test_lowercase_and_uppercase_commands);
 	RUN_TEST(test_command_split_over_several_reads_with_crlf);
 	RUN_TEST(test_print_text_with_newline_escape);
@@ -648,15 +717,21 @@ int main()
 	RUN_TEST(test_pulses_out_of_range_are_rejected);
 	RUN_TEST(test_effect_is_applied_and_stored);
 	RUN_TEST(test_invalid_effect_is_rejected);
-	RUN_TEST(test_no_full_redraw_while_an_effect_runs);
 	RUN_TEST(test_every_effect_visits_each_dot_once);
 	RUN_TEST(test_wipe_goes_left_to_right_and_center_starts_in_the_middle);
 	RUN_TEST(test_sparkle_order_is_scrambled);
 
-	RUN_TEST(test_refresh_waits_for_end_of_frame);
+	RUN_TEST(test_refresh_when_due);
 	RUN_TEST(test_refresh_only_once_per_interval);
-	RUN_TEST(test_refresh_when_serial_is_idle);
+	RUN_TEST(test_running_refresh_is_not_restarted);
 	RUN_TEST(test_no_refresh_storm_after_millis_wraps);
+	RUN_TEST(test_refresh_interval_is_applied_and_stored);
+	RUN_TEST(test_refresh_interval_zero_never_refreshes);
+	RUN_TEST(test_refresh_interval_out_of_range_is_rejected);
+
+	RUN_TEST(test_checksum_reply);
+	RUN_TEST(test_checksum_follows_the_frame);
+	RUN_TEST(test_crc_counts_only_the_dots);
 
 	return UNITY_END();
 }