laurence/openscribe
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
openscribe/docs/ai-providers.md
Laurence 51321aa7c5
Some checks failed
ci / firmware (pull_request) Failing after 1s
Details
ci / emulator (pull_request) Failing after 27s
Details
ci / openapi (pull_request) Failing after 33s
Details
ci / server (pull_request) Failing after 37s
Details
feat(server): pluggable AI providers - any open-standard or commercial AI 
Lets the owner point transcription and summarisation at any AI: an open-standard
endpoint (OpenAI-compatible / local faster-whisper / Ollama) or a commercial API
(OpenAI, Anthropic, Gemini). Config-driven, self-hostable, no lock-in.

What changed:
- server/app/providers/: provider layer.
  - base.py: Transcriber/Summariser protocols + shared summary prompt + tolerant JSON
    parser (uniform Summary shape across providers).
  - summary.py: OpenAICompatibleSummariser (any /chat/completions - OpenAI, Groq,
    OpenRouter, LocalAI, LM Studio, vLLM, Ollama /v1) and AnthropicSummariser (Claude
    via the official anthropic SDK; Messages API has no OpenAI-compatible endpoint).
  - transcription.py: OpenAICompatibleTranscriber (/audio/transcriptions - OpenAI,
    Groq, self-hosted whisper server) and LocalWhisperTranscriber (faster-whisper,
    execution wired in M5).
  - factory.py: builds the configured providers with per-provider defaults
    (anthropic -> claude-opus-4-8, openai_compatible -> gpt-4o-mini, ollama -> llama3.1).
- config.py + .env.example: transcription_provider / llm_provider selectors + base_url,
  key, model settings; local faster-whisper and Ollama kept as the self-hosted defaults.
- main.py: /health now reports the resolved provider names (no secrets).
- requirements.txt: httpx drives all HTTP providers; anthropic + faster-whisper are
  optional, only for their respective providers.
- docs/ai-providers.md: config recipes for OpenAI, Groq, Anthropic, Gemini, LocalAI,
  LM Studio, Ollama, self-hosted whisper.
- state/: DECISIONS, ARCHITECTURE, TODO updated.

Why:
- The user asked to connect the device to any open standard AI or commercial one; this
  is also the core differentiator vs Plaud's locked cloud.

Notes:
- Anthropic provider uses the official SDK and defaults to claude-opus-4-8 (per the
  claude-api guidance). AI deps are optional per chosen provider. Modules byte-compile
  cleanly; end-to-end wiring into the ingest pipeline lands with M5.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 18:56:58 +01:00

3.4 KiB
Raw Blame History

Connecting OpenScribe to any AI

OpenScribe does not hard-wire a single AI vendor. Transcription and summarisation each target a provider you choose in config: an open-standard endpoint (OpenAI-compatible or local), or a commercial API. Self-hosted or not, it is your choice and your data.

Set these in server/.env (see server/.env.example). Provider names never expose secrets and are echoed at GET /health so you can confirm what is wired.

Transcription

OPENSCRIBE_TRANSCRIPTION_PROVIDER:

Value What it uses Config
local_whisper (default) In-process faster-whisper, fully self-hosted OPENSCRIBE_WHISPER_MODEL, _DEVICE, _COMPUTE_TYPE
openai_compatible Any /v1/audio/transcriptions endpoint _TRANSCRIPTION_BASE_URL, _API_KEY, _MODEL

openai_compatible covers OpenAI Whisper, Groq (whisper-large-v3), and self-hosted whisper.cpp / faster-whisper servers that expose that route. Examples:

# OpenAI
OPENSCRIBE_TRANSCRIPTION_PROVIDER=openai_compatible
OPENSCRIBE_TRANSCRIPTION_BASE_URL=https://api.openai.com/v1
OPENSCRIBE_TRANSCRIPTION_API_KEY=sk-...
OPENSCRIBE_TRANSCRIPTION_MODEL=whisper-1

# Groq (fast, cheap)
OPENSCRIBE_TRANSCRIPTION_BASE_URL=https://api.groq.com/openai/v1
OPENSCRIBE_TRANSCRIPTION_MODEL=whisper-large-v3

Summaries and Ask-AI (LLM)

OPENSCRIBE_LLM_PROVIDER:

Value What it uses Config
ollama (default) Local Ollama via its /v1 endpoint, fully self-hosted OPENSCRIBE_OLLAMA_URL, OPENSCRIBE_OLLAMA_MODEL
openai_compatible Any /v1/chat/completions endpoint _LLM_BASE_URL, _LLM_API_KEY, _LLM_MODEL
anthropic Claude via the official Anthropic SDK _LLM_API_KEY, _LLM_MODEL

Examples:

# Fully local (default) - Ollama
OPENSCRIBE_LLM_PROVIDER=ollama
OPENSCRIBE_OLLAMA_URL=http://localhost:11434
OPENSCRIBE_OLLAMA_MODEL=llama3.1

# OpenAI (or any OpenAI-compatible: OpenRouter, Together, LocalAI, LM Studio, vLLM)
OPENSCRIBE_LLM_PROVIDER=openai_compatible
OPENSCRIBE_LLM_BASE_URL=https://api.openai.com/v1
OPENSCRIBE_LLM_API_KEY=sk-...
OPENSCRIBE_LLM_MODEL=gpt-4o-mini

# LM Studio on your LAN (self-hosted, OpenAI-compatible)
OPENSCRIBE_LLM_PROVIDER=openai_compatible
OPENSCRIBE_LLM_BASE_URL=http://localhost:1234/v1
OPENSCRIBE_LLM_MODEL=local-model

# Anthropic Claude (commercial)
OPENSCRIBE_LLM_PROVIDER=anthropic
OPENSCRIBE_LLM_API_KEY=sk-ant-...
OPENSCRIBE_LLM_MODEL=claude-opus-4-8   # or claude-sonnet-4-6, claude-haiku-4-5

Google Gemini is reachable through the openai_compatible provider using Gemini's OpenAI-compatible base URL and a Gemini model name.

Why two shapes

Almost every provider speaks the OpenAI Chat Completions / Audio Transcriptions API, so a single openai_compatible client covers most of the ecosystem, open and commercial. Anthropic's Messages API has a different request/response shape and no OpenAI-compatible endpoint, so Claude gets its own provider built on the official anthropic SDK. Dependencies are optional: only install anthropic if you select the Anthropic provider, only install faster-whisper if you select local_whisper.

Adding a provider

Implement the Transcriber or Summariser protocol in server/app/providers/ and wire it into factory.py. The shared summary prompt and JSON parser in providers/base.py keep the Summary output shape identical across providers.