scaffold: OpenScribe open-source self-hosted AI voice recorder
Bootstrap of the project (M0). Sets up the monorepo, design docs, hardware BOM, the open API contract, component skeletons, licensing and CI, following the Default Workflow SOP. What changed: - CLAUDE.md + docs/: copied the Default Workflow so sessions load the SOP. - state/: PROJECT, ARCHITECTURE, DECISIONS, TODO, NOTES filled in for OpenScribe. ARCHITECTURE captures the four-part design (firmware, server, app, case) and the three sync paths; DECISIONS records the hardware, AI-stack, storage, app and licensing choices; TODO lays out milestones M1-M9. - hardware/BOM.md: two build options (compact XIAO ESP32-S3 Sense; dev ESP32-S3 + I2S mic + SD), wiring/pinout, indicative cost. - api/openapi.yaml: the completely open API (device + server surfaces), including recording list/download/delete and exports (wav/ogg/txt/srt/vtt/md/json). - firmware/: PlatformIO ESP32-S3 project, two board profiles, pin map, boot scaffold with module seams for M1-M4. - server/: FastAPI skeleton mirroring the OpenAPI, config for self-hosted MinIO, faster-whisper and Ollama; stub routes browsable at /docs. - app/, case/: Flutter app plan; parametric OpenSCAD enclosure. - Licensing: GPL-3.0 (code), CERN-OHL-S-2.0 (hardware), CC-BY-SA-4.0 (case/docs), REUSE-style LICENSES/ with SPDX headers; LICENSING.md explains the split. - CI: Forgejo Actions workflow builds firmware (both profiles) and lints/imports server. Why: - Everything self-hosted and openly licensed per the user's requirements: an open API, three sync paths (BLE control, WiFi transfer, independent WiFi upload on charge to generic cloud storage), and a full self-hosted transcription+summary stack. Notes: - No custom PCB in v1; off-the-shelf modules. Physical verification waits on parts. - Component code is stubs at M0; features land milestone by milestone, each as its own branch/PR per the workflow. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
commit
031074c9a9
36 changed files with 2922 additions and 0 deletions
134
state/ARCHITECTURE.md
Normal file
134
state/ARCHITECTURE.md
Normal file
|
|
@ -0,0 +1,134 @@
|
|||
# Architecture
|
||||
|
||||
> How the system is built and why. Update this when the structure changes; a change is
|
||||
> not finished until this reflects it.
|
||||
|
||||
## Overview
|
||||
|
||||
Four parts, connected by an open REST API and a shared recording data model:
|
||||
|
||||
```
|
||||
[ Device: ESP32-S3 ] [ Self-hosted server ]
|
||||
mic -> I2S -> ring buffer (PSRAM) FastAPI
|
||||
-> encoder (WAV) -> microSD +-- ingest (from cloud store / upload)
|
||||
button/LED/haptic UX +-- faster-whisper (transcribe)
|
||||
power + charge detect +-- Ollama LLM (summarise)
|
||||
| BLE (control/provision) +-- object store (MinIO/local) + DB
|
||||
| WiFi REST API (LAN) `-- open REST API + exports
|
||||
| WiFi uploader (on charge) --> cloud store -----------^
|
||||
| |
|
||||
v v
|
||||
[ Flutter app: Android + iOS ] <---- open REST API (device + server)
|
||||
```
|
||||
|
||||
Three sync paths, exactly as specified:
|
||||
1. BLE: control, status, and WiFi provisioning (small data). Portable/battery mode.
|
||||
2. WiFi to app: bulk recording transfer via the device REST API (fast).
|
||||
3. Independent WiFi upload: when on charge / hard-powered the device auto-joins WiFi and
|
||||
pushes recordings to generic cloud storage with no phone present.
|
||||
|
||||
## Components
|
||||
|
||||
### firmware (device)
|
||||
- Responsibility: capture audio, store it, manage power/controls, expose control + data
|
||||
over BLE and WiFi, and upload autonomously when powered.
|
||||
- Location: `firmware/` (PlatformIO, Arduino-ESP32, target ESP32-S3).
|
||||
- Modules (planned):
|
||||
- `audio` - I2S mic driver, DMA capture, PSRAM ring buffer, WAV encoder.
|
||||
- `storage`- microSD (FAT) recording files + sidecar JSON metadata.
|
||||
- `recorder` - session state machine (idle/recording), file naming, metadata.
|
||||
- `ux` - button (start/stop, long-press pair), LED/haptic status.
|
||||
- `power` - battery read (ADC), charge/VBUS detect -> mode switch.
|
||||
- `config` - NVS-stored settings (WiFi creds, upload target + keys, codec).
|
||||
- `net_wifi` - WiFi manager (join, reconnect), mDNS.
|
||||
- `api_http` - on-device REST server (see `api/openapi.yaml`).
|
||||
- `uploader` - S3-compatible / WebDAV client; pushes audio + metadata when powered.
|
||||
- `ble` - GATT: device info, battery, record control, WiFi provisioning, status.
|
||||
- `ota` - firmware update over HTTP.
|
||||
- Depends on: microSD, I2S mic, LiPo + charge IC (see `hardware/BOM.md`).
|
||||
- Why this way: ESP32-S3 has WiFi + BLE 5 + PSRAM + USB in one cheap chip, so all three
|
||||
sync paths and audio buffering fit on one board with off-the-shelf modules.
|
||||
|
||||
### server (self-hosted AI)
|
||||
- Responsibility: ingest recordings, transcribe, summarise, store, and serve the open
|
||||
API with exports.
|
||||
- Location: `server/` (FastAPI).
|
||||
- Pipeline: ingest (from cloud store or direct upload) -> store raw audio (object store)
|
||||
-> transcribe (faster-whisper) -> summarise (Ollama LLM) -> index metadata (DB) ->
|
||||
expose REST API + exports (audio, TXT, SRT, VTT, Markdown, JSON).
|
||||
- Depends on: object storage (MinIO or local FS), a DB (SQLite to start, Postgres later),
|
||||
faster-whisper, Ollama. All self-hostable.
|
||||
- Why this way: keeps the device cheap and low-power (no on-device AI); all heavy compute
|
||||
runs on hardware the user owns; every step swappable and open.
|
||||
|
||||
### app (Flutter)
|
||||
- Responsibility: provision the device, browse the library, play audio, show transcripts
|
||||
and summaries, export/share, manage settings.
|
||||
- Location: `app/`.
|
||||
- Depends on: device BLE + REST API (provisioning/transfer) and server REST API (library,
|
||||
transcripts, summaries).
|
||||
- Why this way: one codebase for Android + iOS. iOS restricts background BLE, so BLE is
|
||||
used for control/provisioning and WiFi for bulk transfer, which matches the design.
|
||||
|
||||
### case (3D print)
|
||||
- Responsibility: enclosure for the chosen board + battery + mic + button + USB + LED.
|
||||
- Location: `case/` (OpenSCAD, parametric).
|
||||
- Why this way: code-defined parametric model re-tunes to exact module dimensions and
|
||||
stays fully open and diffable.
|
||||
|
||||
### hardware
|
||||
- Responsibility: BOM, wiring/pinout, build notes. No custom PCB in v1.
|
||||
- Location: `hardware/`.
|
||||
|
||||
### api
|
||||
- Responsibility: the single source of truth for the open API (device + server).
|
||||
- Location: `api/openapi.yaml`.
|
||||
|
||||
## Data and state
|
||||
|
||||
Recording metadata (sidecar JSON on device; row in server DB), canonical shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "rec_20260703T101500Z_ab12",
|
||||
"device_id": "openscribe-abc123",
|
||||
"started_at": "2026-07-03T10:15:00Z",
|
||||
"duration_s": 372.5,
|
||||
"sample_rate": 16000,
|
||||
"channels": 1,
|
||||
"codec": "wav_pcm_s16le",
|
||||
"size_bytes": 11920000,
|
||||
"sha256": "…",
|
||||
"source": "device",
|
||||
"sync_state": "local | uploaded | ingested | transcribed | summarised",
|
||||
"transcript_ref": null,
|
||||
"summary_ref": null
|
||||
}
|
||||
```
|
||||
|
||||
- On device: files on microSD (`/recordings/<id>.wav` + `<id>.json`); config + secrets in
|
||||
NVS (never on the SD card in clear).
|
||||
- In transit: audio + metadata JSON uploaded to the configured object store; server
|
||||
ingests from there (or accepts direct upload).
|
||||
- On server: audio + artefacts (transcript, summary, subtitle files) in the object store;
|
||||
metadata + refs in the DB.
|
||||
|
||||
## External dependencies
|
||||
|
||||
- ESP32-S3 (Espressif), Arduino-ESP32, PlatformIO - mature, free, WiFi + BLE + PSRAM.
|
||||
- faster-whisper (CTranslate2) - fast self-hosted STT, CPU or GPU.
|
||||
- Ollama - self-hosted local LLM runtime for summaries.
|
||||
- MinIO (or any S3-compatible / WebDAV target) - self-hosted object storage.
|
||||
- FastAPI, Flutter - open, well supported.
|
||||
All chosen to be self-hostable and open; no required proprietary SaaS.
|
||||
|
||||
## Constraints and trade-offs
|
||||
|
||||
- Audio default is WAV PCM 16 kHz mono for simplicity and quality; larger files, so WiFi
|
||||
is the real transfer channel and Opus/ADPCM is a later size optimisation.
|
||||
- No on-device transcription: keeps the device cheap/low-power; needs the server for AI.
|
||||
- BLE bulk transfer is slow and iOS-restricted, so BLE only does control/provisioning and
|
||||
hands transfers to WiFi.
|
||||
- v1 uses off-the-shelf modules (no PCB): easier to build, bigger case than a Plaud.
|
||||
- Security: device REST API and config writes must be authenticated (token in NVS);
|
||||
independent uploads use scoped object-store credentials. Hardening tracked in TODO.
|
||||
Loading…
Add table
Add a link
Reference in a new issue