Laurence
51321aa7c5
Lets the owner point transcription and summarisation at any AI: an open-standard
endpoint (OpenAI-compatible / local faster-whisper / Ollama) or a commercial API
(OpenAI, Anthropic, Gemini). Config-driven, self-hostable, no lock-in.
What changed:
- server/app/providers/: provider layer.
- base.py: Transcriber/Summariser protocols + shared summary prompt + tolerant JSON
parser (uniform Summary shape across providers).
- summary.py: OpenAICompatibleSummariser (any /chat/completions - OpenAI, Groq,
OpenRouter, LocalAI, LM Studio, vLLM, Ollama /v1) and AnthropicSummariser (Claude
via the official anthropic SDK; Messages API has no OpenAI-compatible endpoint).
- transcription.py: OpenAICompatibleTranscriber (/audio/transcriptions - OpenAI,
Groq, self-hosted whisper server) and LocalWhisperTranscriber (faster-whisper,
execution wired in M5).
- factory.py: builds the configured providers with per-provider defaults
(anthropic -> claude-opus-4-8, openai_compatible -> gpt-4o-mini, ollama -> llama3.1).
- config.py + .env.example: transcription_provider / llm_provider selectors + base_url,
key, model settings; local faster-whisper and Ollama kept as the self-hosted defaults.
- main.py: /health now reports the resolved provider names (no secrets).
- requirements.txt: httpx drives all HTTP providers; anthropic + faster-whisper are
optional, only for their respective providers.
- docs/ai-providers.md: config recipes for OpenAI, Groq, Anthropic, Gemini, LocalAI,
LM Studio, Ollama, self-hosted whisper.
- state/: DECISIONS, ARCHITECTURE, TODO updated.
Why:
- The user asked to connect the device to any open standard AI or commercial one; this
is also the core differentiator vs Plaud's locked cloud.
Notes:
- Anthropic provider uses the official SDK and defaults to claude-opus-4-8 (per the
claude-api guidance). AI deps are optional per chosen provider. Modules byte-compile
cleanly; end-to-end wiring into the ingest pipeline lands with M5.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
101 lines
6 KiB
Markdown
101 lines
6 KiB
Markdown
# Decisions
|
|
|
|
> A dated, append only log of decisions and their rationale. Newest at the top. Never
|
|
> rewrite past entries; if a decision is reversed, add a new entry that says so.
|
|
|
|
## 2026-07-03 - Pluggable AI providers (bring your own AI)
|
|
|
|
- **Decision:** Transcription and summarisation each select a provider via config. LLM:
|
|
`ollama` (local, default) | `openai_compatible` (any base_url + key: OpenAI, Groq,
|
|
OpenRouter, LocalAI, LM Studio, vLLM, Gemini's OpenAI endpoint) | `anthropic` (Claude via
|
|
the official `anthropic` SDK). Transcription: `local_whisper` (faster-whisper) |
|
|
`openai_compatible` (OpenAI Whisper, Groq, self-hosted whisper server). See
|
|
`docs/ai-providers.md` and `server/app/providers/`.
|
|
- **Context:** User: "allow for connecting the device to any open standard AI, or even
|
|
commercial ones." This is also OpenScribe's key differentiator vs Plaud's locked cloud.
|
|
- **Rationale:** One OpenAI-compatible client covers most of the ecosystem; Anthropic needs
|
|
its own provider (different API shape, no OpenAI-compatible endpoint). AI dependencies are
|
|
optional per chosen provider, keeping the default install lean and fully self-hostable.
|
|
- **Consequences:** No lock-in; the owner picks cost/quality/privacy trade-offs. Anthropic
|
|
provider defaults to `claude-opus-4-8`. Local faster-whisper execution is wired in M5.
|
|
|
|
## 2026-07-03 - Licensing: copyleft, multi-part (REUSE-style)
|
|
|
|
- **Decision:** Code (firmware, server, app) under GPL-3.0-only; hardware design under
|
|
CERN-OHL-S-2.0; case models and documentation under CC-BY-SA-4.0. Licence texts live in
|
|
`LICENSES/`; top-level `LICENSE` is GPL-3.0 for forge detection; `LICENSING.md` explains
|
|
the split. Apache-2.0 text kept for a possible future permissive client SDK.
|
|
- **Context:** User asked for "as open source as possible".
|
|
- **Rationale:** Strong copyleft keeps derivatives open (the point of the project); CERN
|
|
and CC-BY-SA are the standard reciprocal licences for hardware and creative/docs.
|
|
- **Consequences:** Derivatives must stay open. If we later want wide third-party adoption
|
|
of a client library, that specific component can be relicensed permissive (Apache-2.0).
|
|
|
|
## 2026-07-03 - Self-hosted tools throughout
|
|
|
|
- **Decision:** Forge = Forgejo (git.discworld.casa); CI = Forgejo Actions; STT =
|
|
faster-whisper; summaries = Ollama (local LLM); object storage = MinIO (S3-compatible)
|
|
or local FS / WebDAV. No required proprietary SaaS anywhere.
|
|
- **Context:** User: "using selfhosted tools where possible".
|
|
- **Rationale:** Matches the own-your-data goal and keeps running costs at zero beyond the
|
|
user's own hardware.
|
|
- **Consequences:** User must run a server (NAS / mini-PC) for AI features; the device and
|
|
app work without it for plain recording + transfer.
|
|
|
|
## 2026-07-03 - Self-hosted AI stack in scope for v1
|
|
|
|
- **Decision:** Build the full pipeline: record -> transcribe (faster-whisper) ->
|
|
summarise (Ollama) -> export. AI runs on the server, not the device.
|
|
- **Context:** User chose "Full self-hosted AI stack" at the scope checkpoint.
|
|
- **Rationale:** Transcription + summary is Plaud's headline feature; server-side keeps the
|
|
device cheap and low-power while staying fully self-hosted.
|
|
- **Consequences:** Larger build; server is required for AI features. Device stays simple.
|
|
|
|
## 2026-07-03 - Independent upload target: generic cloud storage
|
|
|
|
- **Decision:** When on charge / hard-powered, the device uploads to configurable generic
|
|
storage: S3-compatible (default: self-hosted MinIO), with WebDAV/NAS as alternatives.
|
|
- **Context:** User chose "Generic cloud storage" for the independent WiFi path.
|
|
- **Rationale:** Decouples device from any bespoke always-on server; standard protocol;
|
|
self-hostable.
|
|
- **Consequences:** Server ingests from the store (watch/notify/poll). Object-store
|
|
credentials live in device NVS and must be scoped/rotatable.
|
|
|
|
## 2026-07-03 - Mobile app: Flutter (Android + iOS)
|
|
|
|
- **Decision:** One Flutter codebase targeting both platforms.
|
|
- **Context:** User chose Flutter (Android + iOS).
|
|
- **Rationale:** Single codebase, both stores.
|
|
- **Consequences:** iOS background BLE is restricted, so BLE = control/provisioning only;
|
|
WiFi handles bulk transfer (already the design).
|
|
|
|
## 2026-07-03 - Hardware: ESP32-S3 + I2S MEMS mic + microSD (off-the-shelf, no PCB)
|
|
|
|
- **Decision:** Target ESP32-S3 (PSRAM, WiFi + BLE 5, USB). Mic: I2S MEMS (INMP441
|
|
default, ICS-43434 upgrade). Storage: microSD. Power: LiPo + charge IC with charge/VBUS
|
|
detect. v1 uses modules on a carrier/protoboard; no custom PCB. Full list in
|
|
`hardware/BOM.md`.
|
|
- **Context:** User asked me to spec a BOM to buy; has ESP32 / Pico W / other boards.
|
|
- **Rationale:** ESP32-S3 does all three radios + audio buffering on one cheap chip; I2S
|
|
MEMS gives clean digital audio; microSD removes length limits. Pico W and classic ESP32
|
|
are viable fallbacks but weaker for audio/PSRAM.
|
|
- **Consequences:** Bigger enclosure than a commercial Plaud; a custom PCB is a later step.
|
|
|
|
## 2026-07-03 - Firmware toolchain: PlatformIO + Arduino-ESP32
|
|
|
|
- **Decision:** Build the firmware with PlatformIO using the Arduino-ESP32 framework.
|
|
- **Context:** Need approachable, reproducible builds and CI.
|
|
- **Rationale:** Lower barrier for contributors than raw ESP-IDF; good library support for
|
|
I2S, SD, BLE, HTTP; PlatformIO gives pinned, CI-friendly builds.
|
|
- **Consequences:** If we hit Arduino limits (fine-grained power, advanced BLE), we can
|
|
drop to ESP-IDF per-module or migrate; noted as a possible future change.
|
|
|
|
## 2026-07-03 - Project name and default audio format
|
|
|
|
- **Decision:** Name = "OpenScribe" (record + transcribe, open). Default recording format
|
|
= WAV PCM 16 kHz mono 16-bit; compressed codecs (ADPCM/Opus) are a later optimisation.
|
|
- **Context:** Project bootstrap.
|
|
- **Rationale:** Clear, descriptive, unencumbered name; WAV is simple and high quality for
|
|
speech and trivial to decode everywhere.
|
|
- **Consequences:** Larger files (~115 MB/hour) make WiFi the primary transfer path;
|
|
revisit with Opus for battery-mode transfer and storage.
|