feat(server): pluggable AI providers - any open-standard or commercial AI
Lets the owner point transcription and summarisation at any AI: an open-standard
endpoint (OpenAI-compatible / local faster-whisper / Ollama) or a commercial API
(OpenAI, Anthropic, Gemini). Config-driven, self-hostable, no lock-in.
What changed:
- server/app/providers/: provider layer.
- base.py: Transcriber/Summariser protocols + shared summary prompt + tolerant JSON
parser (uniform Summary shape across providers).
- summary.py: OpenAICompatibleSummariser (any /chat/completions - OpenAI, Groq,
OpenRouter, LocalAI, LM Studio, vLLM, Ollama /v1) and AnthropicSummariser (Claude
via the official anthropic SDK; Messages API has no OpenAI-compatible endpoint).
- transcription.py: OpenAICompatibleTranscriber (/audio/transcriptions - OpenAI,
Groq, self-hosted whisper server) and LocalWhisperTranscriber (faster-whisper,
execution wired in M5).
- factory.py: builds the configured providers with per-provider defaults
(anthropic -> claude-opus-4-8, openai_compatible -> gpt-4o-mini, ollama -> llama3.1).
- config.py + .env.example: transcription_provider / llm_provider selectors + base_url,
key, model settings; local faster-whisper and Ollama kept as the self-hosted defaults.
- main.py: /health now reports the resolved provider names (no secrets).
- requirements.txt: httpx drives all HTTP providers; anthropic + faster-whisper are
optional, only for their respective providers.
- docs/ai-providers.md: config recipes for OpenAI, Groq, Anthropic, Gemini, LocalAI,
LM Studio, Ollama, self-hosted whisper.
- state/: DECISIONS, ARCHITECTURE, TODO updated.
Why:
- The user asked to connect the device to any open standard AI or commercial one; this
is also the core differentiator vs Plaud's locked cloud.
Notes:
- Anthropic provider uses the official SDK and defaults to claude-opus-4-8 (per the
claude-api guidance). AI deps are optional per chosen provider. Modules byte-compile
cleanly; end-to-end wiring into the ingest pipeline lands with M5.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
34eb17abbc
commit
51321aa7c5
12 changed files with 380 additions and 13 deletions
|
|
@ -56,8 +56,11 @@ Three sync paths, exactly as specified:
|
|||
- Pipeline: ingest (from cloud store or direct upload) -> store raw audio (object store)
|
||||
-> transcribe (faster-whisper) -> summarise (Ollama LLM) -> index metadata (DB) ->
|
||||
expose REST API + exports (audio, TXT, SRT, VTT, Markdown, JSON).
|
||||
- AI is provider-pluggable (`server/app/providers/`, see docs/ai-providers.md): transcription
|
||||
and the LLM each target a configured provider - open-standard (OpenAI-compatible / local
|
||||
faster-whisper / Ollama) or commercial (OpenAI, Anthropic, Gemini). No lock-in.
|
||||
- Depends on: object storage (MinIO or local FS), a DB (SQLite to start, Postgres later),
|
||||
faster-whisper, Ollama. All self-hostable.
|
||||
and whichever AI providers are configured (default: self-hosted faster-whisper + Ollama).
|
||||
- Why this way: keeps the device cheap and low-power (no on-device AI); all heavy compute
|
||||
runs on hardware the user owns; every step swappable and open.
|
||||
|
||||
|
|
|
|||
|
|
@ -3,6 +3,22 @@
|
|||
> A dated, append only log of decisions and their rationale. Newest at the top. Never
|
||||
> rewrite past entries; if a decision is reversed, add a new entry that says so.
|
||||
|
||||
## 2026-07-03 - Pluggable AI providers (bring your own AI)
|
||||
|
||||
- **Decision:** Transcription and summarisation each select a provider via config. LLM:
|
||||
`ollama` (local, default) | `openai_compatible` (any base_url + key: OpenAI, Groq,
|
||||
OpenRouter, LocalAI, LM Studio, vLLM, Gemini's OpenAI endpoint) | `anthropic` (Claude via
|
||||
the official `anthropic` SDK). Transcription: `local_whisper` (faster-whisper) |
|
||||
`openai_compatible` (OpenAI Whisper, Groq, self-hosted whisper server). See
|
||||
`docs/ai-providers.md` and `server/app/providers/`.
|
||||
- **Context:** User: "allow for connecting the device to any open standard AI, or even
|
||||
commercial ones." This is also OpenScribe's key differentiator vs Plaud's locked cloud.
|
||||
- **Rationale:** One OpenAI-compatible client covers most of the ecosystem; Anthropic needs
|
||||
its own provider (different API shape, no OpenAI-compatible endpoint). AI dependencies are
|
||||
optional per chosen provider, keeping the default install lean and fully self-hostable.
|
||||
- **Consequences:** No lock-in; the owner picks cost/quality/privacy trade-offs. Anthropic
|
||||
provider defaults to `claude-opus-4-8`. Local faster-whisper execution is wired in M5.
|
||||
|
||||
## 2026-07-03 - Licensing: copyleft, multi-part (REUSE-style)
|
||||
|
||||
- **Decision:** Code (firmware, server, app) under GPL-3.0-only; hardware design under
|
||||
|
|
|
|||
|
|
@ -29,11 +29,11 @@
|
|||
|
||||
## Pending - AI provider flexibility
|
||||
|
||||
- [~] Pluggable AI providers (server): transcription + LLM can target any open-standard
|
||||
endpoint (OpenAI-compatible / local faster-whisper / Ollama) OR a commercial API
|
||||
(OpenAI, Anthropic, Gemini). Config-driven, no lock-in. (branch
|
||||
`feature/server-ai-providers`) - provider layer + HTTP providers landing now;
|
||||
local faster-whisper wiring completes in M5.
|
||||
- [x] Pluggable AI providers (server): transcription + LLM target any open-standard
|
||||
endpoint (OpenAI-compatible / local / Ollama) OR commercial (OpenAI, Anthropic,
|
||||
Gemini). Config-driven, no lock-in. Provider layer + HTTP + Anthropic providers
|
||||
done (PR #5). Local faster-whisper execution + calling the providers from the
|
||||
ingest pipeline complete in M5. See docs/ai-providers.md.
|
||||
|
||||
## Pending - Plaud-derived backlog (see docs/plaud-comparison.md)
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue