laurence/openscribe
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
openscribe/state/NOTES.md
Laurence f84bbca80b
Some checks failed
ci / openapi (pull_request) Failing after 30s
Details
ci / firmware (pull_request) Failing after 35s
Details
ci / server (pull_request) Failing after 28s
Details
ci / emulator (pull_request) Failing after 36s
Details
test(dev): emulation + linting harness (Wokwi, native tests, CI, docs) 
Answers "how do we emulate/lint this?" with runnable tooling for every part, and
factors the pure firmware logic out so it is unit-testable on the host.

What changed:
- firmware/src/wav.h, firmware/src/httprange.h: pure, header-only WAV header builder
  and HTTP Range parser (no Arduino deps), so the tricky byte-layout and range logic
  can be tested on a PC. storage.cpp and api_http.cpp refactored to use them.
- firmware/test/test_pure/: Unity tests for the WAV header fields and Range parsing
  (start-end, open-ended, clamped, unsatisfiable, non-bytes). Run: pio test -e native.
- firmware/platformio.ini: add [env:native] (host tests) and [env:esp32s3_wokwi]
  (emulator build with default WiFi = Wokwi-GUEST so the API comes up in the sim).
- firmware/wokwi.toml + firmware/diagram.json: Wokwi emulator harness (ESP32-S3 +
  microSD + button + LED). Note: Wokwi has no I2S mic part, so audio isn't emulated;
  the harness targets boot + WiFi + REST API.
- firmware/src/main.cpp + config.cpp: bring up WiFi + API even if audio/SD init fails
  (device stays reachable, reports the fault via GET /device); compile-time default
  WiFi honoured when NVS is empty (used by the Wokwi build).
- .forgejo/workflows/ci.yml: add native tests + cppcheck to the firmware job; new
  openapi job (openapi-spec-validator) and emulator job (Wokwi build, plus a full run
  when WOKWI_CLI_TOKEN is set).
- docs/testing.md: the full emulate/lint guide for firmware, server, API, app, case.
- state/: NOTES points at the guide; TODO reflects this branch.

Why:
- The firmware can't be flashed yet (no parts) and doesn't build on this dev host, so
  we need host-runnable checks. Pure-logic unit tests + OpenAPI validation run anywhere;
  Wokwi emulates boot/WiFi/API; CI compiles the real firmware. Verified locally: the
  OpenAPI spec validates (12 paths, 10 schemas).

Notes:
- Native tests and cppcheck run in CI (no compiler on the dev host). The Wokwi full run
  is skipped unless a WOKWI_CLI_TOKEN secret is present; the build is still verified.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 12:16:25 +01:00

4 KiB
Raw Blame History

Notes

Working notes, gotchas, environment quirks, and dead ends to avoid. Free form. The point is to save a future session from rediscovering something the hard way.

How to run / build / test

See docs/testing.md for the full emulate/lint guide (firmware compile-check, pio check, native unit tests, Wokwi emulation, server/OpenAPI/app/case). Per component (see each component's README for detail):

  • Firmware (firmware/): PlatformIO.
    • pio run - build for the ESP32-S3 target.
    • pio run -t upload - flash over USB.
    • pio device monitor - serial console.
  • Server (server/): Python 3.11+.
    • python -m venv .venv && . .venv/Scripts/activate (Windows) or .venv/bin/activate.
    • pip install -r requirements.txt
    • uvicorn app.main:app --reload - dev server (OpenAPI at /docs).
  • App (app/): Flutter.
    • flutter pub get then flutter run. (Full Flutter project is created in M7.)
  • Case (case/): OpenSCAD. Open case/openscribe_case.scad; render/export STL.

Environment

  • Forge: https://git.discworld.casa/laurence/openscribe (Forgejo). Git auth via Windows Credential Manager (user laurence); the Forgejo API accepts it via basic auth. Never print or commit the token.
  • Dev host: Windows 11, PowerShell primary + Git Bash. Project root c:\temp\dev\openscribe.
  • Self-hosted services for the server (run on a NAS / mini-PC, not required for plain recording): MinIO (S3), Ollama (LLM), and faster-whisper (pip install, downloads models on first run).

Gotchas

  • ESP32-S3 board choice matters: pick a variant WITH PSRAM (e.g. N16R8, or XIAO ESP32-S3) for audio ring buffers. Classic ESP32 / Pico W are fallbacks with less headroom.
  • Windows + PlatformIO: install the CP210x/CH34x USB serial driver or the board will not enumerate a COM port.
  • WAV at 16 kHz mono 16-bit is ~115 MB/hour; transfer over WiFi, not BLE. Opus is a later size win but costs CPU on the S3.
  • iOS restricts background BLE: use BLE only for control/provisioning; do bulk transfer over WiFi.
  • Do not put object-store credentials or WiFi passwords on the microSD in clear; they live in ESP32 NVS.

Firmware specifics (M1)

  • I2S/PDM via the core-bundled ESP_I2S (I2SClass) - no extra lib_deps. Dev board uses I2S_MODE_STD (INMP441/ICS-43434); XIAO uses I2S_MODE_PDM_RX (onboard mic).
  • SD uses a dedicated HSPI bus (SPIClass(HSPI)) so it does not clash with other SPI use.
  • WAV writer streams a 44-byte header then patches RIFF/data sizes on close by seeking.
  • Recording ids are uptime-based until NTP lands (M2); metadata started_at is null until then. See the follow-ups in TODO.md.
  • Not yet compiled locally (no PlatformIO on the dev host) - CI builds both profiles.

Firmware specifics (M2)

  • REST API served by the core-bundled synchronous WebServer on port 80. Path params via server.on("/.../{}" , ...) + server.pathArg(0). collectHeaders is required to read Authorization and Range. Register /recordings/{}/audio before /recordings/{}.
  • Audio download implements HTTP Range (206 + Content-Range) by seeking the file and streaming with sendContent; full GET sends Accept-Ranges: bytes.
  • Only new dep is bblanchon/ArduinoJson@^7 (request parse + JSON responses). WiFi, WebServer, ESPmDNS, Preferences, SD are all core-bundled.
  • WiFi: station mode with stored creds; on no-creds/failure it opens a SoftAP OpenScribe-<macsuffix> (pass openscribe) so the API is reachable to provision WiFi. mDNS name is openscribe.local.
  • Config/secrets live in NVS (Preferences namespace openscribe), never on the SD card.

Security follow-ups (before any real deployment)

  • API auth is open when no token is set (convenient on a trusted LAN). Set an api_token via PUT /device/config to require Authorization: Bearer on mutating calls; consider requiring it for reads too. Tracked in TODO.
  • SoftAP uses a fixed default passphrase; make it user-set during provisioning (M4/BLE).

Dead ends

  • (none yet)