Laurence
51321aa7c5
Lets the owner point transcription and summarisation at any AI: an open-standard
endpoint (OpenAI-compatible / local faster-whisper / Ollama) or a commercial API
(OpenAI, Anthropic, Gemini). Config-driven, self-hostable, no lock-in.
What changed:
- server/app/providers/: provider layer.
- base.py: Transcriber/Summariser protocols + shared summary prompt + tolerant JSON
parser (uniform Summary shape across providers).
- summary.py: OpenAICompatibleSummariser (any /chat/completions - OpenAI, Groq,
OpenRouter, LocalAI, LM Studio, vLLM, Ollama /v1) and AnthropicSummariser (Claude
via the official anthropic SDK; Messages API has no OpenAI-compatible endpoint).
- transcription.py: OpenAICompatibleTranscriber (/audio/transcriptions - OpenAI,
Groq, self-hosted whisper server) and LocalWhisperTranscriber (faster-whisper,
execution wired in M5).
- factory.py: builds the configured providers with per-provider defaults
(anthropic -> claude-opus-4-8, openai_compatible -> gpt-4o-mini, ollama -> llama3.1).
- config.py + .env.example: transcription_provider / llm_provider selectors + base_url,
key, model settings; local faster-whisper and Ollama kept as the self-hosted defaults.
- main.py: /health now reports the resolved provider names (no secrets).
- requirements.txt: httpx drives all HTTP providers; anthropic + faster-whisper are
optional, only for their respective providers.
- docs/ai-providers.md: config recipes for OpenAI, Groq, Anthropic, Gemini, LocalAI,
LM Studio, Ollama, self-hosted whisper.
- state/: DECISIONS, ARCHITECTURE, TODO updated.
Why:
- The user asked to connect the device to any open standard AI or commercial one; this
is also the core differentiator vs Plaud's locked cloud.
Notes:
- Anthropic provider uses the official SDK and defaults to claude-opus-4-8 (per the
claude-api guidance). AI deps are optional per chosen provider. Modules byte-compile
cleanly; end-to-end wiring into the ingest pipeline lands with M5.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
83 lines
4.5 KiB
Markdown
83 lines
4.5 KiB
Markdown
# TODO
|
|
|
|
> The current state of play. Keep it honest and current; this is what the next session
|
|
> reads to know what to do. Move items between sections as they progress.
|
|
|
|
## In progress
|
|
|
|
- [ ] (nothing active - M3 is next)
|
|
|
|
## Pending (roughly in build order)
|
|
|
|
- [ ] M3 Independent uploader: charge/VBUS detect -> powered mode, S3-compatible +
|
|
WebDAV upload of audio + metadata (config store already landed in M2).
|
|
(branch `feature/fw-uploader`)
|
|
- [ ] M4 BLE GATT: device info, battery, record control, WiFi provisioning, status
|
|
notifications; hand-off to WiFi for transfer. (branch `feature/fw-ble`)
|
|
- [ ] M5 Server ingest + transcription: FastAPI ingest from object store/upload, store,
|
|
faster-whisper transcription, DB, exports (audio/TXT/SRT/VTT/JSON). REST API.
|
|
(branch `feature/server-ingest-stt`)
|
|
- [ ] M6 Server summaries: Ollama LLM summary + action items, Markdown export.
|
|
(branch `feature/server-summary`)
|
|
- [ ] M7 Flutter app: BLE provisioning, device dashboard, library from server API,
|
|
playback, transcript/summary view, export/share, settings.
|
|
(branch `feature/app-mvp`)
|
|
- [ ] M8 3D case: parametric OpenSCAD enclosure fit to the BOM board + battery; print/fit.
|
|
(branch `feature/case-v1`)
|
|
- [ ] M9 Hardening + OTA: API auth, credential handling, OTA update, docs handoff.
|
|
(branch `feature/hardening-ota`)
|
|
|
|
## Pending - AI provider flexibility
|
|
|
|
- [x] Pluggable AI providers (server): transcription + LLM target any open-standard
|
|
endpoint (OpenAI-compatible / local / Ollama) OR commercial (OpenAI, Anthropic,
|
|
Gemini). Config-driven, no lock-in. Provider layer + HTTP + Anthropic providers
|
|
done (PR #5). Local faster-whisper execution + calling the providers from the
|
|
ingest pipeline complete in M5. See docs/ai-providers.md.
|
|
|
|
## Pending - Plaud-derived backlog (see docs/plaud-comparison.md)
|
|
|
|
- [ ] Speaker diarisation + labels (server); rename-speaker in the app.
|
|
- [ ] Summary templating: built-in templates (meeting/action-items/to-do/mind-map/
|
|
decisions) + user templates; role-based multidimensional summaries.
|
|
- [ ] Ask AI: chat grounded in a recording's transcript (RAG), via the chosen LLM provider.
|
|
- [ ] Custom vocabulary / prompt hints for transcription + summary.
|
|
- [ ] Multimodal attachments: text notes + images attached to a recording.
|
|
- [ ] Mark-a-moment: device long-press during recording -> highlight timestamp, surfaced
|
|
in the app and used to focus summaries.
|
|
- [ ] Auto title + auto formatting of transcripts and summaries.
|
|
- [ ] Library organisation: tags, folders, full-text search (server + app).
|
|
- [ ] Share links for summaries (extends the export endpoint).
|
|
- [ ] Optional hardware variant with a small OLED status display (InstantView equivalent).
|
|
- [ ] Calendar integration (nice-to-have).
|
|
|
|
## Done
|
|
|
|
- [x] Dev testing + emulation harness - native unit tests (wav/range), Wokwi emulator,
|
|
cppcheck + OpenAPI validation in CI, docs/testing.md (PR #3, 2026-07-03).
|
|
- [x] M2 On-device WiFi + REST API - WiFi/SoftAP + mDNS + config(NVS) + device REST API
|
|
(status, list, metadata, audio-with-Range, delete, record start/stop, config).
|
|
(PR #2 `feature/fw-rest-api`, 2026-07-03). Not yet hardware-verified.
|
|
- [x] M1 Firmware recording core - mic capture to WAV on SD + button/LED + metadata
|
|
(merged: PR #1 `feature/fw-recorder-core`, 2026-07-03). Not yet hardware-verified.
|
|
- [x] M0 Scaffold - repo, licences, state docs, BOM, OpenAPI, skeletons, CI
|
|
(merged to `main`, 2026-07-03)
|
|
- [x] Create Forgejo repo `laurence/openscribe` (public, main) - 2026-07-03
|
|
- [x] Decide hardware, sync model, AI stack, app platform, storage, licensing (see
|
|
DECISIONS.md) - 2026-07-03
|
|
|
|
## Follow-ups (deferred, do when convenient)
|
|
|
|
- [ ] Audio calibration: INMP441 emits 24-bit samples in a 32-bit slot. M1 reads in
|
|
16-bit mode (usable for speech, may be quiet). If levels are low on the dev build,
|
|
capture 32-bit and right-shift into int16 in `audio.cpp`.
|
|
- [ ] Real timestamps: ids/`started_at` are uptime-based (no RTC/NTP yet). Add NTP now
|
|
that WiFi exists (M2), and switch ids and `started_at` to real UTC.
|
|
- [ ] API auth is open when no token set; SoftAP has a fixed default passphrase. Harden
|
|
both before any deployment (see NOTES "Security follow-ups").
|
|
|
|
## Blocked / waiting
|
|
|
|
- [ ] Physical build + on-device verification - waiting on parts from `hardware/BOM.md`.
|
|
Until parts arrive, firmware is validated by CI build (Forgejo Actions) of both
|
|
board profiles, not on real hardware.
|