docs/ai-providers.md

@@ -0,0 +1,86 @@
+<!-- SPDX-License-Identifier: CC-BY-SA-4.0 -->
+# Connecting OpenScribe to any AI
+
+OpenScribe does not hard-wire a single AI vendor. Transcription and summarisation each
+target a **provider you choose in config**: an open-standard endpoint (OpenAI-compatible or
+local), or a commercial API. Self-hosted or not, it is your choice and your data.
+
+Set these in `server/.env` (see `server/.env.example`). Provider names never expose secrets
+and are echoed at `GET /health` so you can confirm what is wired.
+
+## Transcription
+
+`OPENSCRIBE_TRANSCRIPTION_PROVIDER`:
+
+| Value | What it uses | Config |
+|-------|--------------|--------|
+| `local_whisper` (default) | In-process faster-whisper, fully self-hosted | `OPENSCRIBE_WHISPER_MODEL`, `_DEVICE`, `_COMPUTE_TYPE` |
+| `openai_compatible` | Any `/v1/audio/transcriptions` endpoint | `_TRANSCRIPTION_BASE_URL`, `_API_KEY`, `_MODEL` |
+
+`openai_compatible` covers OpenAI Whisper, Groq (`whisper-large-v3`), and self-hosted
+whisper.cpp / faster-whisper servers that expose that route. Examples:
+
+```bash
+# OpenAI
+OPENSCRIBE_TRANSCRIPTION_PROVIDER=openai_compatible
+OPENSCRIBE_TRANSCRIPTION_BASE_URL=https://api.openai.com/v1
+OPENSCRIBE_TRANSCRIPTION_API_KEY=sk-...
+OPENSCRIBE_TRANSCRIPTION_MODEL=whisper-1
+
+# Groq (fast, cheap)
+OPENSCRIBE_TRANSCRIPTION_BASE_URL=https://api.groq.com/openai/v1
+OPENSCRIBE_TRANSCRIPTION_MODEL=whisper-large-v3
+```
+
+## Summaries and Ask-AI (LLM)
+
+`OPENSCRIBE_LLM_PROVIDER`:
+
+| Value | What it uses | Config |
+|-------|--------------|--------|
+| `ollama` (default) | Local Ollama via its `/v1` endpoint, fully self-hosted | `OPENSCRIBE_OLLAMA_URL`, `OPENSCRIBE_OLLAMA_MODEL` |
+| `openai_compatible` | Any `/v1/chat/completions` endpoint | `_LLM_BASE_URL`, `_LLM_API_KEY`, `_LLM_MODEL` |
+| `anthropic` | Claude via the official Anthropic SDK | `_LLM_API_KEY`, `_LLM_MODEL` |
+
+Examples:
+
+```bash
+# Fully local (default) - Ollama
+OPENSCRIBE_LLM_PROVIDER=ollama
+OPENSCRIBE_OLLAMA_URL=http://localhost:11434
+OPENSCRIBE_OLLAMA_MODEL=llama3.1
+
+# OpenAI (or any OpenAI-compatible: OpenRouter, Together, LocalAI, LM Studio, vLLM)
+OPENSCRIBE_LLM_PROVIDER=openai_compatible
+OPENSCRIBE_LLM_BASE_URL=https://api.openai.com/v1
+OPENSCRIBE_LLM_API_KEY=sk-...
+OPENSCRIBE_LLM_MODEL=gpt-4o-mini
+
+# LM Studio on your LAN (self-hosted, OpenAI-compatible)
+OPENSCRIBE_LLM_PROVIDER=openai_compatible
+OPENSCRIBE_LLM_BASE_URL=http://localhost:1234/v1
+OPENSCRIBE_LLM_MODEL=local-model
+
+# Anthropic Claude (commercial)
+OPENSCRIBE_LLM_PROVIDER=anthropic
+OPENSCRIBE_LLM_API_KEY=sk-ant-...
+OPENSCRIBE_LLM_MODEL=claude-opus-4-8   # or claude-sonnet-4-6, claude-haiku-4-5
+```
+
+Google Gemini is reachable through the `openai_compatible` provider using Gemini's
+OpenAI-compatible base URL and a Gemini model name.
+
+## Why two shapes
+
+Almost every provider speaks the OpenAI Chat Completions / Audio Transcriptions API, so a
+single `openai_compatible` client covers most of the ecosystem, open and commercial.
+Anthropic's Messages API has a different request/response shape and no OpenAI-compatible
+endpoint, so Claude gets its own provider built on the official `anthropic` SDK. Dependencies
+are optional: only install `anthropic` if you select the Anthropic provider, only install
+`faster-whisper` if you select `local_whisper`.
+
+## Adding a provider
+
+Implement the `Transcriber` or `Summariser` protocol in `server/app/providers/` and wire it
+into `factory.py`. The shared summary prompt and JSON parser in `providers/base.py` keep the
+`Summary` output shape identical across providers.

server/app/config.py

@@ -28,12 +28,26 @@ class Settings(BaseSettings):
     # Metadata DB (SQLite to start; Postgres URL later)
     database_url: str = "sqlite:///./openscribe.db"
 
-    # Transcription (faster-whisper)
+    # --- AI providers (see docs/ai-providers.md) --------------------------------------
+    # Transcription provider: "local_whisper" (self-hosted) | "openai_compatible"
+    transcription_provider: str = "local_whisper"
+    transcription_base_url: str = ""      # e.g. https://api.openai.com/v1 or a Groq/local URL
+    transcription_api_key: str = ""
+    transcription_model: str = ""         # e.g. whisper-1, whisper-large-v3
+
+    # LLM provider for summaries + Ask-AI:
+    #   "ollama" (default, self-hosted) | "openai_compatible" | "anthropic"
+    llm_provider: str = "ollama"
+    llm_base_url: str = "https://api.openai.com/v1"   # used by openai_compatible
+    llm_api_key: str = ""
+    llm_model: str = ""                   # per-provider default applied if empty
+
+    # Local faster-whisper (used when transcription_provider == local_whisper)
     whisper_model: str = "base"           # tiny|base|small|medium|large-v3
     whisper_device: str = "cpu"           # cpu|cuda
     whisper_compute_type: str = "int8"
 
-    # Summarisation (Ollama, self-hosted)
+    # Local Ollama (used when llm_provider == ollama)
     ollama_url: str = "http://localhost:11434"
     ollama_model: str = "llama3.1"
 

server/app/main.py

@@ -24,11 +24,20 @@ _recordings: dict[str, Recording] = {}
 
 @app.get("/health")
 def health() -> dict:
+    # Report the resolved provider names (never secrets) so operators can confirm config.
+    from .providers import build_summariser, build_transcriber
+
+    try:
+        summariser = build_summariser().name
+    except Exception as exc:  # e.g. anthropic dep not installed
+        summariser = f"unavailable ({type(exc).__name__})"
     return {
         "status": "ok",
         "storage_backend": settings.storage_backend,
-        "whisper_model": settings.whisper_model,
-        "ollama_model": settings.ollama_model,
+        "transcription_provider": settings.transcription_provider,
+        "transcriber": build_transcriber().name,
+        "llm_provider": settings.llm_provider,
+        "summariser": summariser,
     }
 
 

server/app/providers/__init__.py

@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-3.0-only
+"""Pluggable AI providers for OpenScribe.
+
+Transcription and summarisation each target a provider chosen by config, so the owner can
+point OpenScribe at any open-standard endpoint (OpenAI-compatible, local faster-whisper,
+Ollama) or a commercial API (OpenAI, Anthropic, Gemini via its OpenAI-compatible endpoint),
+self-hosted or not, with no lock-in. See docs/ai-providers.md.
+"""
+
+from .factory import build_summariser, build_transcriber
+
+__all__ = ["build_summariser", "build_transcriber"]

server/app/providers/base.py

@@ -0,0 +1,54 @@
+# SPDX-License-Identifier: GPL-3.0-only
+"""Provider interfaces plus the shared summary prompt and a tolerant JSON parser.
+
+Keeping the prompt and parser here means every LLM provider produces the same Summary
+shape regardless of whether it is OpenAI-compatible, Anthropic, or a local model.
+"""
+from __future__ import annotations
+
+import json
+import re
+from typing import Protocol
+
+from ..models import Summary, Transcript
+
+SUMMARY_SYSTEM = (
+    "You are a meeting-notes assistant. Given a transcript, produce a concise overview, "
+    "the key points, and any action items. Respond with ONLY a JSON object of the form "
+    '{"overview": string, "key_points": [string], "action_items": [string]} and nothing '
+    "else."
+)
+
+
+def summary_user_prompt(transcript_text: str) -> str:
+    return f"Transcript:\n\n{transcript_text}\n\nProduce the JSON summary now."
+
+
+def parse_summary(recording_id: str, model: str, raw: str) -> Summary:
+    """Parse an LLM reply into a Summary, tolerating prose or code-fences around the JSON."""
+    data: dict = {}
+    match = re.search(r"\{.*\}", raw, re.DOTALL)
+    if match:
+        try:
+            data = json.loads(match.group(0))
+        except json.JSONDecodeError:
+            data = {}
+    return Summary(
+        recording_id=recording_id,
+        model=model,
+        overview=str(data.get("overview") or raw.strip()[:1000]),
+        key_points=[str(x) for x in (data.get("key_points") or [])],
+        action_items=[str(x) for x in (data.get("action_items") or [])],
+    )
+
+
+class Transcriber(Protocol):
+    name: str
+
+    def transcribe(self, audio_path: str, language: str | None = None) -> Transcript: ...
+
+
+class Summariser(Protocol):
+    name: str
+
+    def summarise(self, recording_id: str, transcript_text: str) -> Summary: ...

server/app/providers/factory.py

@@ -0,0 +1,40 @@
+# SPDX-License-Identifier: GPL-3.0-only
+"""Build the configured providers from settings, applying sensible per-provider defaults."""
+from __future__ import annotations
+
+from ..config import settings
+from .base import Summariser, Transcriber
+from .summary import AnthropicSummariser, OpenAICompatibleSummariser
+from .transcription import LocalWhisperTranscriber, OpenAICompatibleTranscriber
+
+
+def build_summariser() -> Summariser:
+    provider = settings.llm_provider
+    model = settings.llm_model
+
+    if provider == "anthropic":
+        return AnthropicSummariser(settings.llm_api_key, model or "claude-opus-4-8")
+
+    if provider == "openai_compatible":
+        return OpenAICompatibleSummariser(
+            settings.llm_base_url, settings.llm_api_key, model or "gpt-4o-mini"
+        )
+
+    # Default: Ollama via its OpenAI-compatible /v1 endpoint (fully self-hosted).
+    base = settings.ollama_url.rstrip("/") + "/v1"
+    return OpenAICompatibleSummariser(
+        base, settings.llm_api_key or "ollama", model or settings.ollama_model or "llama3.1"
+    )
+
+
+def build_transcriber() -> Transcriber:
+    if settings.transcription_provider == "openai_compatible":
+        return OpenAICompatibleTranscriber(
+            settings.transcription_base_url,
+            settings.transcription_api_key,
+            settings.transcription_model or "whisper-1",
+        )
+    # Default: local faster-whisper (self-hosted).
+    return LocalWhisperTranscriber(
+        settings.whisper_model, settings.whisper_device, settings.whisper_compute_type
+    )

server/app/providers/summary.py

@@ -0,0 +1,66 @@
+# SPDX-License-Identifier: GPL-3.0-only
+"""Summarisation providers.
+
+- OpenAICompatibleSummariser: any endpoint speaking the OpenAI /chat/completions API -
+  OpenAI, Groq, Together, OpenRouter, LocalAI, vLLM, LM Studio, and Ollama's /v1 endpoint.
+- AnthropicSummariser: Claude via the official Anthropic SDK (Messages API). Anthropic does
+  not expose an OpenAI-compatible endpoint, so it needs its own provider.
+"""
+from __future__ import annotations
+
+import httpx
+
+from ..models import Summary
+from .base import SUMMARY_SYSTEM, parse_summary, summary_user_prompt
+
+
+class OpenAICompatibleSummariser:
+    """Talks the OpenAI Chat Completions API. Works with any compatible base_url."""
+
+    def __init__(self, base_url: str, api_key: str, model: str):
+        self.base_url = base_url.rstrip("/")
+        self.api_key = api_key
+        self.model = model
+        self.name = f"openai_compatible:{model}"
+
+    def summarise(self, recording_id: str, transcript_text: str) -> Summary:
+        headers = {"Authorization": f"Bearer {self.api_key}"} if self.api_key else {}
+        payload = {
+            "model": self.model,
+            "messages": [
+                {"role": "system", "content": SUMMARY_SYSTEM},
+                {"role": "user", "content": summary_user_prompt(transcript_text)},
+            ],
+            "temperature": 0.2,
+            # Honoured by OpenAI/Groq/vLLM/etc.; ignored by servers that don't support it.
+            "response_format": {"type": "json_object"},
+        }
+        resp = httpx.post(
+            f"{self.base_url}/chat/completions", headers=headers, json=payload, timeout=120
+        )
+        resp.raise_for_status()
+        content = resp.json()["choices"][0]["message"]["content"]
+        return parse_summary(recording_id, self.name, content)
+
+
+class AnthropicSummariser:
+    """Claude via the official Anthropic SDK. Default model: claude-opus-4-8."""
+
+    def __init__(self, api_key: str, model: str):
+        import anthropic  # imported lazily so the dep is only needed for this provider
+
+        self._client = (
+            anthropic.Anthropic(api_key=api_key) if api_key else anthropic.Anthropic()
+        )
+        self.model = model
+        self.name = f"anthropic:{model}"
+
+    def summarise(self, recording_id: str, transcript_text: str) -> Summary:
+        resp = self._client.messages.create(
+            model=self.model,
+            max_tokens=2000,
+            system=SUMMARY_SYSTEM,
+            messages=[{"role": "user", "content": summary_user_prompt(transcript_text)}],
+        )
+        text = next((b.text for b in resp.content if b.type == "text"), "")
+        return parse_summary(recording_id, self.name, text)

server/app/providers/transcription.py

@@ -0,0 +1,65 @@
+# SPDX-License-Identifier: GPL-3.0-only
+"""Transcription providers.
+
+- OpenAICompatibleTranscriber: any endpoint speaking the OpenAI /audio/transcriptions API -
+  OpenAI Whisper, Groq (whisper-large-v3), or a self-hosted whisper.cpp/faster-whisper
+  server exposing that route.
+- LocalWhisperTranscriber: in-process faster-whisper. Wired up in M5 (needs the model
+  download); the interface and config exist now so it is selectable.
+"""
+from __future__ import annotations
+
+import os
+
+import httpx
+
+from ..models import Transcript, TranscriptSegment
+
+
+class OpenAICompatibleTranscriber:
+    def __init__(self, base_url: str, api_key: str, model: str):
+        self.base_url = base_url.rstrip("/")
+        self.api_key = api_key
+        self.model = model
+        self.name = f"openai_compatible:{model}"
+
+    def transcribe(self, audio_path: str, language: str | None = None) -> Transcript:
+        headers = {"Authorization": f"Bearer {self.api_key}"} if self.api_key else {}
+        data = {"model": self.model, "response_format": "verbose_json"}
+        if language:
+            data["language"] = language
+        with open(audio_path, "rb") as f:
+            files = {"file": (os.path.basename(audio_path), f, "audio/wav")}
+            resp = httpx.post(
+                f"{self.base_url}/audio/transcriptions",
+                headers=headers,
+                data=data,
+                files=files,
+                timeout=600,
+            )
+        resp.raise_for_status()
+        body = resp.json()
+        segments = [
+            TranscriptSegment(start=s.get("start", 0.0), end=s.get("end", 0.0),
+                              text=s.get("text", ""))
+            for s in body.get("segments", [])
+        ]
+        return Transcript(
+            recording_id="",
+            language=body.get("language") or language or "en",
+            model=self.name,
+            text=body.get("text", ""),
+            segments=segments,
+        )
+
+
+class LocalWhisperTranscriber:
+    def __init__(self, model: str, device: str, compute_type: str):
+        self.model = model
+        self.device = device
+        self.compute_type = compute_type
+        self.name = f"faster-whisper:{model}"
+
+    def transcribe(self, audio_path: str, language: str | None = None) -> Transcript:
+        # M5: load faster_whisper.WhisperModel(self.model, device, compute_type) and run it.
+        raise NotImplementedError("Local faster-whisper transcription lands in M5")

server/requirements.txt

@@ -10,9 +10,11 @@ python-multipart>=0.0.9
 boto3>=1.34            # S3-compatible (MinIO)
 webdavclient3>=3.14    # WebDAV / NAS
 
-# AI pipeline (self-hosted). Installed when M5/M6 land; listed here for reference.
-faster-whisper>=1.0    # transcription (CTranslate2)
-httpx>=0.27            # talk to Ollama for summaries
+# AI pipeline. httpx drives all OpenAI-compatible + local providers (transcription + LLM).
+httpx>=0.27
+# Optional, only needed for the specific provider you choose:
+anthropic>=0.40        # llm_provider=anthropic (commercial Claude)
+faster-whisper>=1.0    # transcription_provider=local_whisper (CTranslate2); wired in M5
 
 # Dev
 pytest>=8

state/ARCHITECTURE.md

@@ -56,8 +56,11 @@ Three sync paths, exactly as specified:
 - Pipeline: ingest (from cloud store or direct upload) -> store raw audio (object store)
   -> transcribe (faster-whisper) -> summarise (Ollama LLM) -> index metadata (DB) ->
   expose REST API + exports (audio, TXT, SRT, VTT, Markdown, JSON).
+- AI is provider-pluggable (`server/app/providers/`, see docs/ai-providers.md): transcription
+  and the LLM each target a configured provider - open-standard (OpenAI-compatible / local
+  faster-whisper / Ollama) or commercial (OpenAI, Anthropic, Gemini). No lock-in.
 - Depends on: object storage (MinIO or local FS), a DB (SQLite to start, Postgres later),
-  faster-whisper, Ollama. All self-hostable.
+  and whichever AI providers are configured (default: self-hosted faster-whisper + Ollama).
 - Why this way: keeps the device cheap and low-power (no on-device AI); all heavy compute
   runs on hardware the user owns; every step swappable and open.
 

state/DECISIONS.md

@@ -3,6 +3,22 @@
 > A dated, append only log of decisions and their rationale. Newest at the top. Never
 > rewrite past entries; if a decision is reversed, add a new entry that says so.
 
+## 2026-07-03 - Pluggable AI providers (bring your own AI)
+
+- **Decision:** Transcription and summarisation each select a provider via config. LLM:
+  `ollama` (local, default) | `openai_compatible` (any base_url + key: OpenAI, Groq,
+  OpenRouter, LocalAI, LM Studio, vLLM, Gemini's OpenAI endpoint) | `anthropic` (Claude via
+  the official `anthropic` SDK). Transcription: `local_whisper` (faster-whisper) |
+  `openai_compatible` (OpenAI Whisper, Groq, self-hosted whisper server). See
+  `docs/ai-providers.md` and `server/app/providers/`.
+- **Context:** User: "allow for connecting the device to any open standard AI, or even
+  commercial ones." This is also OpenScribe's key differentiator vs Plaud's locked cloud.
+- **Rationale:** One OpenAI-compatible client covers most of the ecosystem; Anthropic needs
+  its own provider (different API shape, no OpenAI-compatible endpoint). AI dependencies are
+  optional per chosen provider, keeping the default install lean and fully self-hostable.
+- **Consequences:** No lock-in; the owner picks cost/quality/privacy trade-offs. Anthropic
+  provider defaults to `claude-opus-4-8`. Local faster-whisper execution is wired in M5.
+
 ## 2026-07-03 - Licensing: copyleft, multi-part (REUSE-style)
 
 - **Decision:** Code (firmware, server, app) under GPL-3.0-only; hardware design under

state/TODO.md

@@ -29,11 +29,11 @@
 
 ## Pending - AI provider flexibility
 
-- [~] Pluggable AI providers (server): transcription + LLM can target any open-standard
-      endpoint (OpenAI-compatible / local faster-whisper / Ollama) OR a commercial API
-      (OpenAI, Anthropic, Gemini). Config-driven, no lock-in. (branch
-      `feature/server-ai-providers`) - provider layer + HTTP providers landing now;
-      local faster-whisper wiring completes in M5.
+- [x] Pluggable AI providers (server): transcription + LLM target any open-standard
+      endpoint (OpenAI-compatible / local / Ollama) OR commercial (OpenAI, Anthropic,
+      Gemini). Config-driven, no lock-in. Provider layer + HTTP + Anthropic providers
+      done (PR #5). Local faster-whisper execution + calling the providers from the
+      ingest pipeline complete in M5. See docs/ai-providers.md.
 
 ## Pending - Plaud-derived backlog (see docs/plaud-comparison.md)