Laurence
51321aa7c5
Lets the owner point transcription and summarisation at any AI: an open-standard
endpoint (OpenAI-compatible / local faster-whisper / Ollama) or a commercial API
(OpenAI, Anthropic, Gemini). Config-driven, self-hostable, no lock-in.
What changed:
- server/app/providers/: provider layer.
- base.py: Transcriber/Summariser protocols + shared summary prompt + tolerant JSON
parser (uniform Summary shape across providers).
- summary.py: OpenAICompatibleSummariser (any /chat/completions - OpenAI, Groq,
OpenRouter, LocalAI, LM Studio, vLLM, Ollama /v1) and AnthropicSummariser (Claude
via the official anthropic SDK; Messages API has no OpenAI-compatible endpoint).
- transcription.py: OpenAICompatibleTranscriber (/audio/transcriptions - OpenAI,
Groq, self-hosted whisper server) and LocalWhisperTranscriber (faster-whisper,
execution wired in M5).
- factory.py: builds the configured providers with per-provider defaults
(anthropic -> claude-opus-4-8, openai_compatible -> gpt-4o-mini, ollama -> llama3.1).
- config.py + .env.example: transcription_provider / llm_provider selectors + base_url,
key, model settings; local faster-whisper and Ollama kept as the self-hosted defaults.
- main.py: /health now reports the resolved provider names (no secrets).
- requirements.txt: httpx drives all HTTP providers; anthropic + faster-whisper are
optional, only for their respective providers.
- docs/ai-providers.md: config recipes for OpenAI, Groq, Anthropic, Gemini, LocalAI,
LM Studio, Ollama, self-hosted whisper.
- state/: DECISIONS, ARCHITECTURE, TODO updated.
Why:
- The user asked to connect the device to any open standard AI or commercial one; this
is also the core differentiator vs Plaud's locked cloud.
Notes:
- Anthropic provider uses the official SDK and defaults to claude-opus-4-8 (per the
claude-api guidance). AI deps are optional per chosen provider. Modules byte-compile
cleanly; end-to-end wiring into the ingest pipeline lands with M5.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
86 lines
3.4 KiB
Markdown
86 lines
3.4 KiB
Markdown
<!-- SPDX-License-Identifier: CC-BY-SA-4.0 -->
|
|
# Connecting OpenScribe to any AI
|
|
|
|
OpenScribe does not hard-wire a single AI vendor. Transcription and summarisation each
|
|
target a **provider you choose in config**: an open-standard endpoint (OpenAI-compatible or
|
|
local), or a commercial API. Self-hosted or not, it is your choice and your data.
|
|
|
|
Set these in `server/.env` (see `server/.env.example`). Provider names never expose secrets
|
|
and are echoed at `GET /health` so you can confirm what is wired.
|
|
|
|
## Transcription
|
|
|
|
`OPENSCRIBE_TRANSCRIPTION_PROVIDER`:
|
|
|
|
| Value | What it uses | Config |
|
|
|-------|--------------|--------|
|
|
| `local_whisper` (default) | In-process faster-whisper, fully self-hosted | `OPENSCRIBE_WHISPER_MODEL`, `_DEVICE`, `_COMPUTE_TYPE` |
|
|
| `openai_compatible` | Any `/v1/audio/transcriptions` endpoint | `_TRANSCRIPTION_BASE_URL`, `_API_KEY`, `_MODEL` |
|
|
|
|
`openai_compatible` covers OpenAI Whisper, Groq (`whisper-large-v3`), and self-hosted
|
|
whisper.cpp / faster-whisper servers that expose that route. Examples:
|
|
|
|
```bash
|
|
# OpenAI
|
|
OPENSCRIBE_TRANSCRIPTION_PROVIDER=openai_compatible
|
|
OPENSCRIBE_TRANSCRIPTION_BASE_URL=https://api.openai.com/v1
|
|
OPENSCRIBE_TRANSCRIPTION_API_KEY=sk-...
|
|
OPENSCRIBE_TRANSCRIPTION_MODEL=whisper-1
|
|
|
|
# Groq (fast, cheap)
|
|
OPENSCRIBE_TRANSCRIPTION_BASE_URL=https://api.groq.com/openai/v1
|
|
OPENSCRIBE_TRANSCRIPTION_MODEL=whisper-large-v3
|
|
```
|
|
|
|
## Summaries and Ask-AI (LLM)
|
|
|
|
`OPENSCRIBE_LLM_PROVIDER`:
|
|
|
|
| Value | What it uses | Config |
|
|
|-------|--------------|--------|
|
|
| `ollama` (default) | Local Ollama via its `/v1` endpoint, fully self-hosted | `OPENSCRIBE_OLLAMA_URL`, `OPENSCRIBE_OLLAMA_MODEL` |
|
|
| `openai_compatible` | Any `/v1/chat/completions` endpoint | `_LLM_BASE_URL`, `_LLM_API_KEY`, `_LLM_MODEL` |
|
|
| `anthropic` | Claude via the official Anthropic SDK | `_LLM_API_KEY`, `_LLM_MODEL` |
|
|
|
|
Examples:
|
|
|
|
```bash
|
|
# Fully local (default) - Ollama
|
|
OPENSCRIBE_LLM_PROVIDER=ollama
|
|
OPENSCRIBE_OLLAMA_URL=http://localhost:11434
|
|
OPENSCRIBE_OLLAMA_MODEL=llama3.1
|
|
|
|
# OpenAI (or any OpenAI-compatible: OpenRouter, Together, LocalAI, LM Studio, vLLM)
|
|
OPENSCRIBE_LLM_PROVIDER=openai_compatible
|
|
OPENSCRIBE_LLM_BASE_URL=https://api.openai.com/v1
|
|
OPENSCRIBE_LLM_API_KEY=sk-...
|
|
OPENSCRIBE_LLM_MODEL=gpt-4o-mini
|
|
|
|
# LM Studio on your LAN (self-hosted, OpenAI-compatible)
|
|
OPENSCRIBE_LLM_PROVIDER=openai_compatible
|
|
OPENSCRIBE_LLM_BASE_URL=http://localhost:1234/v1
|
|
OPENSCRIBE_LLM_MODEL=local-model
|
|
|
|
# Anthropic Claude (commercial)
|
|
OPENSCRIBE_LLM_PROVIDER=anthropic
|
|
OPENSCRIBE_LLM_API_KEY=sk-ant-...
|
|
OPENSCRIBE_LLM_MODEL=claude-opus-4-8 # or claude-sonnet-4-6, claude-haiku-4-5
|
|
```
|
|
|
|
Google Gemini is reachable through the `openai_compatible` provider using Gemini's
|
|
OpenAI-compatible base URL and a Gemini model name.
|
|
|
|
## Why two shapes
|
|
|
|
Almost every provider speaks the OpenAI Chat Completions / Audio Transcriptions API, so a
|
|
single `openai_compatible` client covers most of the ecosystem, open and commercial.
|
|
Anthropic's Messages API has a different request/response shape and no OpenAI-compatible
|
|
endpoint, so Claude gets its own provider built on the official `anthropic` SDK. Dependencies
|
|
are optional: only install `anthropic` if you select the Anthropic provider, only install
|
|
`faster-whisper` if you select `local_whisper`.
|
|
|
|
## Adding a provider
|
|
|
|
Implement the `Transcriber` or `Summariser` protocol in `server/app/providers/` and wire it
|
|
into `factory.py`. The shared summary prompt and JSON parser in `providers/base.py` keep the
|
|
`Summary` output shape identical across providers.
|