laurence/openscribe
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
openscribe/state/NOTES.md
Laurence 054b2f6981
Some checks failed
ci / server (pull_request) Failing after 34s
Details
ci / firmware (pull_request) Failing after 36s
Details
feat(firmware): M1 recording core - mic capture to WAV on SD 
Implements the first working feature: the device records audio to the microSD
card, toggled by the button, with LED status and per-recording JSON metadata.

What changed:
- firmware/src/audio.{h,cpp}: mic capture via the core-bundled ESP_I2S. Dev board
  uses I2S standard mode (INMP441/ICS-43434); XIAO uses PDM mode (onboard mic).
  Presents 16-bit PCM mono to callers regardless of board.
- firmware/src/storage.{h,cpp}: microSD on a dedicated HSPI bus + a streaming
  WavWriter that writes a 44-byte PCM header and patches RIFF/data sizes on close;
  plus sidecar JSON metadata writer.
- firmware/src/recorder.{h,cpp}: idle/recording state machine; creates
  /recordings/<id>.wav, pumps mic chunks in on update(), finalises + writes
  <id>.json (Recording schema from api/openapi.yaml) on stop.
- firmware/src/ux.{h,cpp}: debounced button (short press toggles) + status LED
  patterns (idle/recording/error), active-low aware.
- firmware/src/main.cpp: wires ux + recorder; loop toggles on button and drains
  the mic while recording.
- firmware/include/audio_config.h: 16 kHz mono 16-bit, chunk size, rec dir.
- firmware/include/pins.h: added XIAO PDM mic + onboard SD pins, LED active-low flag.
- state/: TODO, ARCHITECTURE, NOTES updated for M1 and the deferred follow-ups.

Why:
- Recording is the foundation every later milestone (transfer, upload, transcription)
  builds on. Kept dependency-free (only core-bundled ESP_I2S + SD) for simple CI builds.

Notes:
- Not compiled locally (no PlatformIO on the dev host) or hardware-verified; Forgejo
  Actions CI builds both board profiles. Follow-ups tracked in state/TODO.md:
  INMP441 16-bit level calibration, and real NTP timestamps (ids are uptime-based
  until M2 brings WiFi/NTP).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 11:35:34 +01:00

2.6 KiB
Raw Blame History

Notes

Working notes, gotchas, environment quirks, and dead ends to avoid. Free form. The point is to save a future session from rediscovering something the hard way.

How to run / build / test

Per component (see each component's README for detail):

  • Firmware (firmware/): PlatformIO.
    • pio run - build for the ESP32-S3 target.
    • pio run -t upload - flash over USB.
    • pio device monitor - serial console.
  • Server (server/): Python 3.11+.
    • python -m venv .venv && . .venv/Scripts/activate (Windows) or .venv/bin/activate.
    • pip install -r requirements.txt
    • uvicorn app.main:app --reload - dev server (OpenAPI at /docs).
  • App (app/): Flutter.
    • flutter pub get then flutter run. (Full Flutter project is created in M7.)
  • Case (case/): OpenSCAD. Open case/openscribe_case.scad; render/export STL.

Environment

  • Forge: https://git.discworld.casa/laurence/openscribe (Forgejo). Git auth via Windows Credential Manager (user laurence); the Forgejo API accepts it via basic auth. Never print or commit the token.
  • Dev host: Windows 11, PowerShell primary + Git Bash. Project root c:\temp\dev\openscribe.
  • Self-hosted services for the server (run on a NAS / mini-PC, not required for plain recording): MinIO (S3), Ollama (LLM), and faster-whisper (pip install, downloads models on first run).

Gotchas

  • ESP32-S3 board choice matters: pick a variant WITH PSRAM (e.g. N16R8, or XIAO ESP32-S3) for audio ring buffers. Classic ESP32 / Pico W are fallbacks with less headroom.
  • Windows + PlatformIO: install the CP210x/CH34x USB serial driver or the board will not enumerate a COM port.
  • WAV at 16 kHz mono 16-bit is ~115 MB/hour; transfer over WiFi, not BLE. Opus is a later size win but costs CPU on the S3.
  • iOS restricts background BLE: use BLE only for control/provisioning; do bulk transfer over WiFi.
  • Do not put object-store credentials or WiFi passwords on the microSD in clear; they live in ESP32 NVS.

Firmware specifics (M1)

  • I2S/PDM via the core-bundled ESP_I2S (I2SClass) - no extra lib_deps. Dev board uses I2S_MODE_STD (INMP441/ICS-43434); XIAO uses I2S_MODE_PDM_RX (onboard mic).
  • SD uses a dedicated HSPI bus (SPIClass(HSPI)) so it does not clash with other SPI use.
  • WAV writer streams a 44-byte header then patches RIFF/data sizes on close by seeking.
  • Recording ids are uptime-based until NTP lands (M2); metadata started_at is null until then. See the follow-ups in TODO.md.
  • Not yet compiled locally (no PlatformIO on the dev host) - CI builds both profiles.

Dead ends

  • (none yet)