diff --git a/docs/ai-providers.md b/docs/ai-providers.md new file mode 100644 index 0000000..ced2668 --- /dev/null +++ b/docs/ai-providers.md @@ -0,0 +1,86 @@ + +# Connecting OpenScribe to any AI + +OpenScribe does not hard-wire a single AI vendor. Transcription and summarisation each +target a **provider you choose in config**: an open-standard endpoint (OpenAI-compatible or +local), or a commercial API. Self-hosted or not, it is your choice and your data. + +Set these in `server/.env` (see `server/.env.example`). Provider names never expose secrets +and are echoed at `GET /health` so you can confirm what is wired. + +## Transcription + +`OPENSCRIBE_TRANSCRIPTION_PROVIDER`: + +| Value | What it uses | Config | +|-------|--------------|--------| +| `local_whisper` (default) | In-process faster-whisper, fully self-hosted | `OPENSCRIBE_WHISPER_MODEL`, `_DEVICE`, `_COMPUTE_TYPE` | +| `openai_compatible` | Any `/v1/audio/transcriptions` endpoint | `_TRANSCRIPTION_BASE_URL`, `_API_KEY`, `_MODEL` | + +`openai_compatible` covers OpenAI Whisper, Groq (`whisper-large-v3`), and self-hosted +whisper.cpp / faster-whisper servers that expose that route. Examples: + +```bash +# OpenAI +OPENSCRIBE_TRANSCRIPTION_PROVIDER=openai_compatible +OPENSCRIBE_TRANSCRIPTION_BASE_URL=https://api.openai.com/v1 +OPENSCRIBE_TRANSCRIPTION_API_KEY=sk-... +OPENSCRIBE_TRANSCRIPTION_MODEL=whisper-1 + +# Groq (fast, cheap) +OPENSCRIBE_TRANSCRIPTION_BASE_URL=https://api.groq.com/openai/v1 +OPENSCRIBE_TRANSCRIPTION_MODEL=whisper-large-v3 +``` + +## Summaries and Ask-AI (LLM) + +`OPENSCRIBE_LLM_PROVIDER`: + +| Value | What it uses | Config | +|-------|--------------|--------| +| `ollama` (default) | Local Ollama via its `/v1` endpoint, fully self-hosted | `OPENSCRIBE_OLLAMA_URL`, `OPENSCRIBE_OLLAMA_MODEL` | +| `openai_compatible` | Any `/v1/chat/completions` endpoint | `_LLM_BASE_URL`, `_LLM_API_KEY`, `_LLM_MODEL` | +| `anthropic` | Claude via the official Anthropic SDK | `_LLM_API_KEY`, `_LLM_MODEL` | + +Examples: + +```bash +# Fully local (default) - Ollama +OPENSCRIBE_LLM_PROVIDER=ollama +OPENSCRIBE_OLLAMA_URL=http://localhost:11434 +OPENSCRIBE_OLLAMA_MODEL=llama3.1 + +# OpenAI (or any OpenAI-compatible: OpenRouter, Together, LocalAI, LM Studio, vLLM) +OPENSCRIBE_LLM_PROVIDER=openai_compatible +OPENSCRIBE_LLM_BASE_URL=https://api.openai.com/v1 +OPENSCRIBE_LLM_API_KEY=sk-... +OPENSCRIBE_LLM_MODEL=gpt-4o-mini + +# LM Studio on your LAN (self-hosted, OpenAI-compatible) +OPENSCRIBE_LLM_PROVIDER=openai_compatible +OPENSCRIBE_LLM_BASE_URL=http://localhost:1234/v1 +OPENSCRIBE_LLM_MODEL=local-model + +# Anthropic Claude (commercial) +OPENSCRIBE_LLM_PROVIDER=anthropic +OPENSCRIBE_LLM_API_KEY=sk-ant-... +OPENSCRIBE_LLM_MODEL=claude-opus-4-8 # or claude-sonnet-4-6, claude-haiku-4-5 +``` + +Google Gemini is reachable through the `openai_compatible` provider using Gemini's +OpenAI-compatible base URL and a Gemini model name. + +## Why two shapes + +Almost every provider speaks the OpenAI Chat Completions / Audio Transcriptions API, so a +single `openai_compatible` client covers most of the ecosystem, open and commercial. +Anthropic's Messages API has a different request/response shape and no OpenAI-compatible +endpoint, so Claude gets its own provider built on the official `anthropic` SDK. Dependencies +are optional: only install `anthropic` if you select the Anthropic provider, only install +`faster-whisper` if you select `local_whisper`. + +## Adding a provider + +Implement the `Transcriber` or `Summariser` protocol in `server/app/providers/` and wire it +into `factory.py`. The shared summary prompt and JSON parser in `providers/base.py` keep the +`Summary` output shape identical across providers. diff --git a/server/app/config.py b/server/app/config.py index 697586a..6c3f84b 100644 --- a/server/app/config.py +++ b/server/app/config.py @@ -28,12 +28,26 @@ class Settings(BaseSettings): # Metadata DB (SQLite to start; Postgres URL later) database_url: str = "sqlite:///./openscribe.db" - # Transcription (faster-whisper) + # --- AI providers (see docs/ai-providers.md) -------------------------------------- + # Transcription provider: "local_whisper" (self-hosted) | "openai_compatible" + transcription_provider: str = "local_whisper" + transcription_base_url: str = "" # e.g. https://api.openai.com/v1 or a Groq/local URL + transcription_api_key: str = "" + transcription_model: str = "" # e.g. whisper-1, whisper-large-v3 + + # LLM provider for summaries + Ask-AI: + # "ollama" (default, self-hosted) | "openai_compatible" | "anthropic" + llm_provider: str = "ollama" + llm_base_url: str = "https://api.openai.com/v1" # used by openai_compatible + llm_api_key: str = "" + llm_model: str = "" # per-provider default applied if empty + + # Local faster-whisper (used when transcription_provider == local_whisper) whisper_model: str = "base" # tiny|base|small|medium|large-v3 whisper_device: str = "cpu" # cpu|cuda whisper_compute_type: str = "int8" - # Summarisation (Ollama, self-hosted) + # Local Ollama (used when llm_provider == ollama) ollama_url: str = "http://localhost:11434" ollama_model: str = "llama3.1" diff --git a/server/app/main.py b/server/app/main.py index c9c3cad..9e2a071 100644 --- a/server/app/main.py +++ b/server/app/main.py @@ -24,11 +24,20 @@ _recordings: dict[str, Recording] = {} @app.get("/health") def health() -> dict: + # Report the resolved provider names (never secrets) so operators can confirm config. + from .providers import build_summariser, build_transcriber + + try: + summariser = build_summariser().name + except Exception as exc: # e.g. anthropic dep not installed + summariser = f"unavailable ({type(exc).__name__})" return { "status": "ok", "storage_backend": settings.storage_backend, - "whisper_model": settings.whisper_model, - "ollama_model": settings.ollama_model, + "transcription_provider": settings.transcription_provider, + "transcriber": build_transcriber().name, + "llm_provider": settings.llm_provider, + "summariser": summariser, } diff --git a/server/app/providers/__init__.py b/server/app/providers/__init__.py new file mode 100644 index 0000000..a1234ab --- /dev/null +++ b/server/app/providers/__init__.py @@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-3.0-only +"""Pluggable AI providers for OpenScribe. + +Transcription and summarisation each target a provider chosen by config, so the owner can +point OpenScribe at any open-standard endpoint (OpenAI-compatible, local faster-whisper, +Ollama) or a commercial API (OpenAI, Anthropic, Gemini via its OpenAI-compatible endpoint), +self-hosted or not, with no lock-in. See docs/ai-providers.md. +""" + +from .factory import build_summariser, build_transcriber + +__all__ = ["build_summariser", "build_transcriber"] diff --git a/server/app/providers/base.py b/server/app/providers/base.py new file mode 100644 index 0000000..eeadc98 --- /dev/null +++ b/server/app/providers/base.py @@ -0,0 +1,54 @@ +# SPDX-License-Identifier: GPL-3.0-only +"""Provider interfaces plus the shared summary prompt and a tolerant JSON parser. + +Keeping the prompt and parser here means every LLM provider produces the same Summary +shape regardless of whether it is OpenAI-compatible, Anthropic, or a local model. +""" +from __future__ import annotations + +import json +import re +from typing import Protocol + +from ..models import Summary, Transcript + +SUMMARY_SYSTEM = ( + "You are a meeting-notes assistant. Given a transcript, produce a concise overview, " + "the key points, and any action items. Respond with ONLY a JSON object of the form " + '{"overview": string, "key_points": [string], "action_items": [string]} and nothing ' + "else." +) + + +def summary_user_prompt(transcript_text: str) -> str: + return f"Transcript:\n\n{transcript_text}\n\nProduce the JSON summary now." + + +def parse_summary(recording_id: str, model: str, raw: str) -> Summary: + """Parse an LLM reply into a Summary, tolerating prose or code-fences around the JSON.""" + data: dict = {} + match = re.search(r"\{.*\}", raw, re.DOTALL) + if match: + try: + data = json.loads(match.group(0)) + except json.JSONDecodeError: + data = {} + return Summary( + recording_id=recording_id, + model=model, + overview=str(data.get("overview") or raw.strip()[:1000]), + key_points=[str(x) for x in (data.get("key_points") or [])], + action_items=[str(x) for x in (data.get("action_items") or [])], + ) + + +class Transcriber(Protocol): + name: str + + def transcribe(self, audio_path: str, language: str | None = None) -> Transcript: ... + + +class Summariser(Protocol): + name: str + + def summarise(self, recording_id: str, transcript_text: str) -> Summary: ... diff --git a/server/app/providers/factory.py b/server/app/providers/factory.py new file mode 100644 index 0000000..e5c3097 --- /dev/null +++ b/server/app/providers/factory.py @@ -0,0 +1,40 @@ +# SPDX-License-Identifier: GPL-3.0-only +"""Build the configured providers from settings, applying sensible per-provider defaults.""" +from __future__ import annotations + +from ..config import settings +from .base import Summariser, Transcriber +from .summary import AnthropicSummariser, OpenAICompatibleSummariser +from .transcription import LocalWhisperTranscriber, OpenAICompatibleTranscriber + + +def build_summariser() -> Summariser: + provider = settings.llm_provider + model = settings.llm_model + + if provider == "anthropic": + return AnthropicSummariser(settings.llm_api_key, model or "claude-opus-4-8") + + if provider == "openai_compatible": + return OpenAICompatibleSummariser( + settings.llm_base_url, settings.llm_api_key, model or "gpt-4o-mini" + ) + + # Default: Ollama via its OpenAI-compatible /v1 endpoint (fully self-hosted). + base = settings.ollama_url.rstrip("/") + "/v1" + return OpenAICompatibleSummariser( + base, settings.llm_api_key or "ollama", model or settings.ollama_model or "llama3.1" + ) + + +def build_transcriber() -> Transcriber: + if settings.transcription_provider == "openai_compatible": + return OpenAICompatibleTranscriber( + settings.transcription_base_url, + settings.transcription_api_key, + settings.transcription_model or "whisper-1", + ) + # Default: local faster-whisper (self-hosted). + return LocalWhisperTranscriber( + settings.whisper_model, settings.whisper_device, settings.whisper_compute_type + ) diff --git a/server/app/providers/summary.py b/server/app/providers/summary.py new file mode 100644 index 0000000..2625e79 --- /dev/null +++ b/server/app/providers/summary.py @@ -0,0 +1,66 @@ +# SPDX-License-Identifier: GPL-3.0-only +"""Summarisation providers. + +- OpenAICompatibleSummariser: any endpoint speaking the OpenAI /chat/completions API - + OpenAI, Groq, Together, OpenRouter, LocalAI, vLLM, LM Studio, and Ollama's /v1 endpoint. +- AnthropicSummariser: Claude via the official Anthropic SDK (Messages API). Anthropic does + not expose an OpenAI-compatible endpoint, so it needs its own provider. +""" +from __future__ import annotations + +import httpx + +from ..models import Summary +from .base import SUMMARY_SYSTEM, parse_summary, summary_user_prompt + + +class OpenAICompatibleSummariser: + """Talks the OpenAI Chat Completions API. Works with any compatible base_url.""" + + def __init__(self, base_url: str, api_key: str, model: str): + self.base_url = base_url.rstrip("/") + self.api_key = api_key + self.model = model + self.name = f"openai_compatible:{model}" + + def summarise(self, recording_id: str, transcript_text: str) -> Summary: + headers = {"Authorization": f"Bearer {self.api_key}"} if self.api_key else {} + payload = { + "model": self.model, + "messages": [ + {"role": "system", "content": SUMMARY_SYSTEM}, + {"role": "user", "content": summary_user_prompt(transcript_text)}, + ], + "temperature": 0.2, + # Honoured by OpenAI/Groq/vLLM/etc.; ignored by servers that don't support it. + "response_format": {"type": "json_object"}, + } + resp = httpx.post( + f"{self.base_url}/chat/completions", headers=headers, json=payload, timeout=120 + ) + resp.raise_for_status() + content = resp.json()["choices"][0]["message"]["content"] + return parse_summary(recording_id, self.name, content) + + +class AnthropicSummariser: + """Claude via the official Anthropic SDK. Default model: claude-opus-4-8.""" + + def __init__(self, api_key: str, model: str): + import anthropic # imported lazily so the dep is only needed for this provider + + self._client = ( + anthropic.Anthropic(api_key=api_key) if api_key else anthropic.Anthropic() + ) + self.model = model + self.name = f"anthropic:{model}" + + def summarise(self, recording_id: str, transcript_text: str) -> Summary: + resp = self._client.messages.create( + model=self.model, + max_tokens=2000, + system=SUMMARY_SYSTEM, + messages=[{"role": "user", "content": summary_user_prompt(transcript_text)}], + ) + text = next((b.text for b in resp.content if b.type == "text"), "") + return parse_summary(recording_id, self.name, text) diff --git a/server/app/providers/transcription.py b/server/app/providers/transcription.py new file mode 100644 index 0000000..5b9df59 --- /dev/null +++ b/server/app/providers/transcription.py @@ -0,0 +1,65 @@ +# SPDX-License-Identifier: GPL-3.0-only +"""Transcription providers. + +- OpenAICompatibleTranscriber: any endpoint speaking the OpenAI /audio/transcriptions API - + OpenAI Whisper, Groq (whisper-large-v3), or a self-hosted whisper.cpp/faster-whisper + server exposing that route. +- LocalWhisperTranscriber: in-process faster-whisper. Wired up in M5 (needs the model + download); the interface and config exist now so it is selectable. +""" +from __future__ import annotations + +import os + +import httpx + +from ..models import Transcript, TranscriptSegment + + +class OpenAICompatibleTranscriber: + def __init__(self, base_url: str, api_key: str, model: str): + self.base_url = base_url.rstrip("/") + self.api_key = api_key + self.model = model + self.name = f"openai_compatible:{model}" + + def transcribe(self, audio_path: str, language: str | None = None) -> Transcript: + headers = {"Authorization": f"Bearer {self.api_key}"} if self.api_key else {} + data = {"model": self.model, "response_format": "verbose_json"} + if language: + data["language"] = language + with open(audio_path, "rb") as f: + files = {"file": (os.path.basename(audio_path), f, "audio/wav")} + resp = httpx.post( + f"{self.base_url}/audio/transcriptions", + headers=headers, + data=data, + files=files, + timeout=600, + ) + resp.raise_for_status() + body = resp.json() + segments = [ + TranscriptSegment(start=s.get("start", 0.0), end=s.get("end", 0.0), + text=s.get("text", "")) + for s in body.get("segments", []) + ] + return Transcript( + recording_id="", + language=body.get("language") or language or "en", + model=self.name, + text=body.get("text", ""), + segments=segments, + ) + + +class LocalWhisperTranscriber: + def __init__(self, model: str, device: str, compute_type: str): + self.model = model + self.device = device + self.compute_type = compute_type + self.name = f"faster-whisper:{model}" + + def transcribe(self, audio_path: str, language: str | None = None) -> Transcript: + # M5: load faster_whisper.WhisperModel(self.model, device, compute_type) and run it. + raise NotImplementedError("Local faster-whisper transcription lands in M5") diff --git a/server/requirements.txt b/server/requirements.txt index 6f71dc5..e5e18b0 100644 --- a/server/requirements.txt +++ b/server/requirements.txt @@ -10,9 +10,11 @@ python-multipart>=0.0.9 boto3>=1.34 # S3-compatible (MinIO) webdavclient3>=3.14 # WebDAV / NAS -# AI pipeline (self-hosted). Installed when M5/M6 land; listed here for reference. -faster-whisper>=1.0 # transcription (CTranslate2) -httpx>=0.27 # talk to Ollama for summaries +# AI pipeline. httpx drives all OpenAI-compatible + local providers (transcription + LLM). +httpx>=0.27 +# Optional, only needed for the specific provider you choose: +anthropic>=0.40 # llm_provider=anthropic (commercial Claude) +faster-whisper>=1.0 # transcription_provider=local_whisper (CTranslate2); wired in M5 # Dev pytest>=8 diff --git a/state/ARCHITECTURE.md b/state/ARCHITECTURE.md index ecd375e..a0333f0 100644 --- a/state/ARCHITECTURE.md +++ b/state/ARCHITECTURE.md @@ -56,8 +56,11 @@ Three sync paths, exactly as specified: - Pipeline: ingest (from cloud store or direct upload) -> store raw audio (object store) -> transcribe (faster-whisper) -> summarise (Ollama LLM) -> index metadata (DB) -> expose REST API + exports (audio, TXT, SRT, VTT, Markdown, JSON). +- AI is provider-pluggable (`server/app/providers/`, see docs/ai-providers.md): transcription + and the LLM each target a configured provider - open-standard (OpenAI-compatible / local + faster-whisper / Ollama) or commercial (OpenAI, Anthropic, Gemini). No lock-in. - Depends on: object storage (MinIO or local FS), a DB (SQLite to start, Postgres later), - faster-whisper, Ollama. All self-hostable. + and whichever AI providers are configured (default: self-hosted faster-whisper + Ollama). - Why this way: keeps the device cheap and low-power (no on-device AI); all heavy compute runs on hardware the user owns; every step swappable and open. diff --git a/state/DECISIONS.md b/state/DECISIONS.md index 95576d2..32688de 100644 --- a/state/DECISIONS.md +++ b/state/DECISIONS.md @@ -3,6 +3,22 @@ > A dated, append only log of decisions and their rationale. Newest at the top. Never > rewrite past entries; if a decision is reversed, add a new entry that says so. +## 2026-07-03 - Pluggable AI providers (bring your own AI) + +- **Decision:** Transcription and summarisation each select a provider via config. LLM: + `ollama` (local, default) | `openai_compatible` (any base_url + key: OpenAI, Groq, + OpenRouter, LocalAI, LM Studio, vLLM, Gemini's OpenAI endpoint) | `anthropic` (Claude via + the official `anthropic` SDK). Transcription: `local_whisper` (faster-whisper) | + `openai_compatible` (OpenAI Whisper, Groq, self-hosted whisper server). See + `docs/ai-providers.md` and `server/app/providers/`. +- **Context:** User: "allow for connecting the device to any open standard AI, or even + commercial ones." This is also OpenScribe's key differentiator vs Plaud's locked cloud. +- **Rationale:** One OpenAI-compatible client covers most of the ecosystem; Anthropic needs + its own provider (different API shape, no OpenAI-compatible endpoint). AI dependencies are + optional per chosen provider, keeping the default install lean and fully self-hostable. +- **Consequences:** No lock-in; the owner picks cost/quality/privacy trade-offs. Anthropic + provider defaults to `claude-opus-4-8`. Local faster-whisper execution is wired in M5. + ## 2026-07-03 - Licensing: copyleft, multi-part (REUSE-style) - **Decision:** Code (firmware, server, app) under GPL-3.0-only; hardware design under diff --git a/state/TODO.md b/state/TODO.md index 3647e5b..7ded7e9 100644 --- a/state/TODO.md +++ b/state/TODO.md @@ -29,11 +29,11 @@ ## Pending - AI provider flexibility -- [~] Pluggable AI providers (server): transcription + LLM can target any open-standard - endpoint (OpenAI-compatible / local faster-whisper / Ollama) OR a commercial API - (OpenAI, Anthropic, Gemini). Config-driven, no lock-in. (branch - `feature/server-ai-providers`) - provider layer + HTTP providers landing now; - local faster-whisper wiring completes in M5. +- [x] Pluggable AI providers (server): transcription + LLM target any open-standard + endpoint (OpenAI-compatible / local / Ollama) OR commercial (OpenAI, Anthropic, + Gemini). Config-driven, no lock-in. Provider layer + HTTP + Anthropic providers + done (PR #5). Local faster-whisper execution + calling the providers from the + ingest pipeline complete in M5. See docs/ai-providers.md. ## Pending - Plaud-derived backlog (see docs/plaud-comparison.md)