diff --git a/docs/ai-providers.md b/docs/ai-providers.md deleted file mode 100644 index ced2668..0000000 --- a/docs/ai-providers.md +++ /dev/null @@ -1,86 +0,0 @@ - -# Connecting OpenScribe to any AI - -OpenScribe does not hard-wire a single AI vendor. Transcription and summarisation each -target a **provider you choose in config**: an open-standard endpoint (OpenAI-compatible or -local), or a commercial API. Self-hosted or not, it is your choice and your data. - -Set these in `server/.env` (see `server/.env.example`). Provider names never expose secrets -and are echoed at `GET /health` so you can confirm what is wired. - -## Transcription - -`OPENSCRIBE_TRANSCRIPTION_PROVIDER`: - -| Value | What it uses | Config | -|-------|--------------|--------| -| `local_whisper` (default) | In-process faster-whisper, fully self-hosted | `OPENSCRIBE_WHISPER_MODEL`, `_DEVICE`, `_COMPUTE_TYPE` | -| `openai_compatible` | Any `/v1/audio/transcriptions` endpoint | `_TRANSCRIPTION_BASE_URL`, `_API_KEY`, `_MODEL` | - -`openai_compatible` covers OpenAI Whisper, Groq (`whisper-large-v3`), and self-hosted -whisper.cpp / faster-whisper servers that expose that route. Examples: - -```bash -# OpenAI -OPENSCRIBE_TRANSCRIPTION_PROVIDER=openai_compatible -OPENSCRIBE_TRANSCRIPTION_BASE_URL=https://api.openai.com/v1 -OPENSCRIBE_TRANSCRIPTION_API_KEY=sk-... -OPENSCRIBE_TRANSCRIPTION_MODEL=whisper-1 - -# Groq (fast, cheap) -OPENSCRIBE_TRANSCRIPTION_BASE_URL=https://api.groq.com/openai/v1 -OPENSCRIBE_TRANSCRIPTION_MODEL=whisper-large-v3 -``` - -## Summaries and Ask-AI (LLM) - -`OPENSCRIBE_LLM_PROVIDER`: - -| Value | What it uses | Config | -|-------|--------------|--------| -| `ollama` (default) | Local Ollama via its `/v1` endpoint, fully self-hosted | `OPENSCRIBE_OLLAMA_URL`, `OPENSCRIBE_OLLAMA_MODEL` | -| `openai_compatible` | Any `/v1/chat/completions` endpoint | `_LLM_BASE_URL`, `_LLM_API_KEY`, `_LLM_MODEL` | -| `anthropic` | Claude via the official Anthropic SDK | `_LLM_API_KEY`, `_LLM_MODEL` | - -Examples: - -```bash -# Fully local (default) - Ollama -OPENSCRIBE_LLM_PROVIDER=ollama -OPENSCRIBE_OLLAMA_URL=http://localhost:11434 -OPENSCRIBE_OLLAMA_MODEL=llama3.1 - -# OpenAI (or any OpenAI-compatible: OpenRouter, Together, LocalAI, LM Studio, vLLM) -OPENSCRIBE_LLM_PROVIDER=openai_compatible -OPENSCRIBE_LLM_BASE_URL=https://api.openai.com/v1 -OPENSCRIBE_LLM_API_KEY=sk-... -OPENSCRIBE_LLM_MODEL=gpt-4o-mini - -# LM Studio on your LAN (self-hosted, OpenAI-compatible) -OPENSCRIBE_LLM_PROVIDER=openai_compatible -OPENSCRIBE_LLM_BASE_URL=http://localhost:1234/v1 -OPENSCRIBE_LLM_MODEL=local-model - -# Anthropic Claude (commercial) -OPENSCRIBE_LLM_PROVIDER=anthropic -OPENSCRIBE_LLM_API_KEY=sk-ant-... -OPENSCRIBE_LLM_MODEL=claude-opus-4-8 # or claude-sonnet-4-6, claude-haiku-4-5 -``` - -Google Gemini is reachable through the `openai_compatible` provider using Gemini's -OpenAI-compatible base URL and a Gemini model name. - -## Why two shapes - -Almost every provider speaks the OpenAI Chat Completions / Audio Transcriptions API, so a -single `openai_compatible` client covers most of the ecosystem, open and commercial. -Anthropic's Messages API has a different request/response shape and no OpenAI-compatible -endpoint, so Claude gets its own provider built on the official `anthropic` SDK. Dependencies -are optional: only install `anthropic` if you select the Anthropic provider, only install -`faster-whisper` if you select `local_whisper`. - -## Adding a provider - -Implement the `Transcriber` or `Summariser` protocol in `server/app/providers/` and wire it -into `factory.py`. The shared summary prompt and JSON parser in `providers/base.py` keep the -`Summary` output shape identical across providers. diff --git a/server/app/config.py b/server/app/config.py index 6c3f84b..697586a 100644 --- a/server/app/config.py +++ b/server/app/config.py @@ -28,26 +28,12 @@ class Settings(BaseSettings): # Metadata DB (SQLite to start; Postgres URL later) database_url: str = "sqlite:///./openscribe.db" - # --- AI providers (see docs/ai-providers.md) -------------------------------------- - # Transcription provider: "local_whisper" (self-hosted) | "openai_compatible" - transcription_provider: str = "local_whisper" - transcription_base_url: str = "" # e.g. https://api.openai.com/v1 or a Groq/local URL - transcription_api_key: str = "" - transcription_model: str = "" # e.g. whisper-1, whisper-large-v3 - - # LLM provider for summaries + Ask-AI: - # "ollama" (default, self-hosted) | "openai_compatible" | "anthropic" - llm_provider: str = "ollama" - llm_base_url: str = "https://api.openai.com/v1" # used by openai_compatible - llm_api_key: str = "" - llm_model: str = "" # per-provider default applied if empty - - # Local faster-whisper (used when transcription_provider == local_whisper) + # Transcription (faster-whisper) whisper_model: str = "base" # tiny|base|small|medium|large-v3 whisper_device: str = "cpu" # cpu|cuda whisper_compute_type: str = "int8" - # Local Ollama (used when llm_provider == ollama) + # Summarisation (Ollama, self-hosted) ollama_url: str = "http://localhost:11434" ollama_model: str = "llama3.1" diff --git a/server/app/main.py b/server/app/main.py index 9e2a071..c9c3cad 100644 --- a/server/app/main.py +++ b/server/app/main.py @@ -24,20 +24,11 @@ _recordings: dict[str, Recording] = {} @app.get("/health") def health() -> dict: - # Report the resolved provider names (never secrets) so operators can confirm config. - from .providers import build_summariser, build_transcriber - - try: - summariser = build_summariser().name - except Exception as exc: # e.g. anthropic dep not installed - summariser = f"unavailable ({type(exc).__name__})" return { "status": "ok", "storage_backend": settings.storage_backend, - "transcription_provider": settings.transcription_provider, - "transcriber": build_transcriber().name, - "llm_provider": settings.llm_provider, - "summariser": summariser, + "whisper_model": settings.whisper_model, + "ollama_model": settings.ollama_model, } diff --git a/server/app/providers/__init__.py b/server/app/providers/__init__.py deleted file mode 100644 index a1234ab..0000000 --- a/server/app/providers/__init__.py +++ /dev/null @@ -1,12 +0,0 @@ -# SPDX-License-Identifier: GPL-3.0-only -"""Pluggable AI providers for OpenScribe. - -Transcription and summarisation each target a provider chosen by config, so the owner can -point OpenScribe at any open-standard endpoint (OpenAI-compatible, local faster-whisper, -Ollama) or a commercial API (OpenAI, Anthropic, Gemini via its OpenAI-compatible endpoint), -self-hosted or not, with no lock-in. See docs/ai-providers.md. -""" - -from .factory import build_summariser, build_transcriber - -__all__ = ["build_summariser", "build_transcriber"] diff --git a/server/app/providers/base.py b/server/app/providers/base.py deleted file mode 100644 index eeadc98..0000000 --- a/server/app/providers/base.py +++ /dev/null @@ -1,54 +0,0 @@ -# SPDX-License-Identifier: GPL-3.0-only -"""Provider interfaces plus the shared summary prompt and a tolerant JSON parser. - -Keeping the prompt and parser here means every LLM provider produces the same Summary -shape regardless of whether it is OpenAI-compatible, Anthropic, or a local model. -""" -from __future__ import annotations - -import json -import re -from typing import Protocol - -from ..models import Summary, Transcript - -SUMMARY_SYSTEM = ( - "You are a meeting-notes assistant. Given a transcript, produce a concise overview, " - "the key points, and any action items. Respond with ONLY a JSON object of the form " - '{"overview": string, "key_points": [string], "action_items": [string]} and nothing ' - "else." -) - - -def summary_user_prompt(transcript_text: str) -> str: - return f"Transcript:\n\n{transcript_text}\n\nProduce the JSON summary now." - - -def parse_summary(recording_id: str, model: str, raw: str) -> Summary: - """Parse an LLM reply into a Summary, tolerating prose or code-fences around the JSON.""" - data: dict = {} - match = re.search(r"\{.*\}", raw, re.DOTALL) - if match: - try: - data = json.loads(match.group(0)) - except json.JSONDecodeError: - data = {} - return Summary( - recording_id=recording_id, - model=model, - overview=str(data.get("overview") or raw.strip()[:1000]), - key_points=[str(x) for x in (data.get("key_points") or [])], - action_items=[str(x) for x in (data.get("action_items") or [])], - ) - - -class Transcriber(Protocol): - name: str - - def transcribe(self, audio_path: str, language: str | None = None) -> Transcript: ... - - -class Summariser(Protocol): - name: str - - def summarise(self, recording_id: str, transcript_text: str) -> Summary: ... diff --git a/server/app/providers/factory.py b/server/app/providers/factory.py deleted file mode 100644 index e5c3097..0000000 --- a/server/app/providers/factory.py +++ /dev/null @@ -1,40 +0,0 @@ -# SPDX-License-Identifier: GPL-3.0-only -"""Build the configured providers from settings, applying sensible per-provider defaults.""" -from __future__ import annotations - -from ..config import settings -from .base import Summariser, Transcriber -from .summary import AnthropicSummariser, OpenAICompatibleSummariser -from .transcription import LocalWhisperTranscriber, OpenAICompatibleTranscriber - - -def build_summariser() -> Summariser: - provider = settings.llm_provider - model = settings.llm_model - - if provider == "anthropic": - return AnthropicSummariser(settings.llm_api_key, model or "claude-opus-4-8") - - if provider == "openai_compatible": - return OpenAICompatibleSummariser( - settings.llm_base_url, settings.llm_api_key, model or "gpt-4o-mini" - ) - - # Default: Ollama via its OpenAI-compatible /v1 endpoint (fully self-hosted). - base = settings.ollama_url.rstrip("/") + "/v1" - return OpenAICompatibleSummariser( - base, settings.llm_api_key or "ollama", model or settings.ollama_model or "llama3.1" - ) - - -def build_transcriber() -> Transcriber: - if settings.transcription_provider == "openai_compatible": - return OpenAICompatibleTranscriber( - settings.transcription_base_url, - settings.transcription_api_key, - settings.transcription_model or "whisper-1", - ) - # Default: local faster-whisper (self-hosted). - return LocalWhisperTranscriber( - settings.whisper_model, settings.whisper_device, settings.whisper_compute_type - ) diff --git a/server/app/providers/summary.py b/server/app/providers/summary.py deleted file mode 100644 index 2625e79..0000000 --- a/server/app/providers/summary.py +++ /dev/null @@ -1,66 +0,0 @@ -# SPDX-License-Identifier: GPL-3.0-only -"""Summarisation providers. - -- OpenAICompatibleSummariser: any endpoint speaking the OpenAI /chat/completions API - - OpenAI, Groq, Together, OpenRouter, LocalAI, vLLM, LM Studio, and Ollama's /v1 endpoint. -- AnthropicSummariser: Claude via the official Anthropic SDK (Messages API). Anthropic does - not expose an OpenAI-compatible endpoint, so it needs its own provider. -""" -from __future__ import annotations - -import httpx - -from ..models import Summary -from .base import SUMMARY_SYSTEM, parse_summary, summary_user_prompt - - -class OpenAICompatibleSummariser: - """Talks the OpenAI Chat Completions API. Works with any compatible base_url.""" - - def __init__(self, base_url: str, api_key: str, model: str): - self.base_url = base_url.rstrip("/") - self.api_key = api_key - self.model = model - self.name = f"openai_compatible:{model}" - - def summarise(self, recording_id: str, transcript_text: str) -> Summary: - headers = {"Authorization": f"Bearer {self.api_key}"} if self.api_key else {} - payload = { - "model": self.model, - "messages": [ - {"role": "system", "content": SUMMARY_SYSTEM}, - {"role": "user", "content": summary_user_prompt(transcript_text)}, - ], - "temperature": 0.2, - # Honoured by OpenAI/Groq/vLLM/etc.; ignored by servers that don't support it. - "response_format": {"type": "json_object"}, - } - resp = httpx.post( - f"{self.base_url}/chat/completions", headers=headers, json=payload, timeout=120 - ) - resp.raise_for_status() - content = resp.json()["choices"][0]["message"]["content"] - return parse_summary(recording_id, self.name, content) - - -class AnthropicSummariser: - """Claude via the official Anthropic SDK. Default model: claude-opus-4-8.""" - - def __init__(self, api_key: str, model: str): - import anthropic # imported lazily so the dep is only needed for this provider - - self._client = ( - anthropic.Anthropic(api_key=api_key) if api_key else anthropic.Anthropic() - ) - self.model = model - self.name = f"anthropic:{model}" - - def summarise(self, recording_id: str, transcript_text: str) -> Summary: - resp = self._client.messages.create( - model=self.model, - max_tokens=2000, - system=SUMMARY_SYSTEM, - messages=[{"role": "user", "content": summary_user_prompt(transcript_text)}], - ) - text = next((b.text for b in resp.content if b.type == "text"), "") - return parse_summary(recording_id, self.name, text) diff --git a/server/app/providers/transcription.py b/server/app/providers/transcription.py deleted file mode 100644 index 5b9df59..0000000 --- a/server/app/providers/transcription.py +++ /dev/null @@ -1,65 +0,0 @@ -# SPDX-License-Identifier: GPL-3.0-only -"""Transcription providers. - -- OpenAICompatibleTranscriber: any endpoint speaking the OpenAI /audio/transcriptions API - - OpenAI Whisper, Groq (whisper-large-v3), or a self-hosted whisper.cpp/faster-whisper - server exposing that route. -- LocalWhisperTranscriber: in-process faster-whisper. Wired up in M5 (needs the model - download); the interface and config exist now so it is selectable. -""" -from __future__ import annotations - -import os - -import httpx - -from ..models import Transcript, TranscriptSegment - - -class OpenAICompatibleTranscriber: - def __init__(self, base_url: str, api_key: str, model: str): - self.base_url = base_url.rstrip("/") - self.api_key = api_key - self.model = model - self.name = f"openai_compatible:{model}" - - def transcribe(self, audio_path: str, language: str | None = None) -> Transcript: - headers = {"Authorization": f"Bearer {self.api_key}"} if self.api_key else {} - data = {"model": self.model, "response_format": "verbose_json"} - if language: - data["language"] = language - with open(audio_path, "rb") as f: - files = {"file": (os.path.basename(audio_path), f, "audio/wav")} - resp = httpx.post( - f"{self.base_url}/audio/transcriptions", - headers=headers, - data=data, - files=files, - timeout=600, - ) - resp.raise_for_status() - body = resp.json() - segments = [ - TranscriptSegment(start=s.get("start", 0.0), end=s.get("end", 0.0), - text=s.get("text", "")) - for s in body.get("segments", []) - ] - return Transcript( - recording_id="", - language=body.get("language") or language or "en", - model=self.name, - text=body.get("text", ""), - segments=segments, - ) - - -class LocalWhisperTranscriber: - def __init__(self, model: str, device: str, compute_type: str): - self.model = model - self.device = device - self.compute_type = compute_type - self.name = f"faster-whisper:{model}" - - def transcribe(self, audio_path: str, language: str | None = None) -> Transcript: - # M5: load faster_whisper.WhisperModel(self.model, device, compute_type) and run it. - raise NotImplementedError("Local faster-whisper transcription lands in M5") diff --git a/server/requirements.txt b/server/requirements.txt index e5e18b0..6f71dc5 100644 --- a/server/requirements.txt +++ b/server/requirements.txt @@ -10,11 +10,9 @@ python-multipart>=0.0.9 boto3>=1.34 # S3-compatible (MinIO) webdavclient3>=3.14 # WebDAV / NAS -# AI pipeline. httpx drives all OpenAI-compatible + local providers (transcription + LLM). -httpx>=0.27 -# Optional, only needed for the specific provider you choose: -anthropic>=0.40 # llm_provider=anthropic (commercial Claude) -faster-whisper>=1.0 # transcription_provider=local_whisper (CTranslate2); wired in M5 +# AI pipeline (self-hosted). Installed when M5/M6 land; listed here for reference. +faster-whisper>=1.0 # transcription (CTranslate2) +httpx>=0.27 # talk to Ollama for summaries # Dev pytest>=8 diff --git a/state/ARCHITECTURE.md b/state/ARCHITECTURE.md index a0333f0..ecd375e 100644 --- a/state/ARCHITECTURE.md +++ b/state/ARCHITECTURE.md @@ -56,11 +56,8 @@ Three sync paths, exactly as specified: - Pipeline: ingest (from cloud store or direct upload) -> store raw audio (object store) -> transcribe (faster-whisper) -> summarise (Ollama LLM) -> index metadata (DB) -> expose REST API + exports (audio, TXT, SRT, VTT, Markdown, JSON). -- AI is provider-pluggable (`server/app/providers/`, see docs/ai-providers.md): transcription - and the LLM each target a configured provider - open-standard (OpenAI-compatible / local - faster-whisper / Ollama) or commercial (OpenAI, Anthropic, Gemini). No lock-in. - Depends on: object storage (MinIO or local FS), a DB (SQLite to start, Postgres later), - and whichever AI providers are configured (default: self-hosted faster-whisper + Ollama). + faster-whisper, Ollama. All self-hostable. - Why this way: keeps the device cheap and low-power (no on-device AI); all heavy compute runs on hardware the user owns; every step swappable and open. diff --git a/state/DECISIONS.md b/state/DECISIONS.md index 32688de..95576d2 100644 --- a/state/DECISIONS.md +++ b/state/DECISIONS.md @@ -3,22 +3,6 @@ > A dated, append only log of decisions and their rationale. Newest at the top. Never > rewrite past entries; if a decision is reversed, add a new entry that says so. -## 2026-07-03 - Pluggable AI providers (bring your own AI) - -- **Decision:** Transcription and summarisation each select a provider via config. LLM: - `ollama` (local, default) | `openai_compatible` (any base_url + key: OpenAI, Groq, - OpenRouter, LocalAI, LM Studio, vLLM, Gemini's OpenAI endpoint) | `anthropic` (Claude via - the official `anthropic` SDK). Transcription: `local_whisper` (faster-whisper) | - `openai_compatible` (OpenAI Whisper, Groq, self-hosted whisper server). See - `docs/ai-providers.md` and `server/app/providers/`. -- **Context:** User: "allow for connecting the device to any open standard AI, or even - commercial ones." This is also OpenScribe's key differentiator vs Plaud's locked cloud. -- **Rationale:** One OpenAI-compatible client covers most of the ecosystem; Anthropic needs - its own provider (different API shape, no OpenAI-compatible endpoint). AI dependencies are - optional per chosen provider, keeping the default install lean and fully self-hostable. -- **Consequences:** No lock-in; the owner picks cost/quality/privacy trade-offs. Anthropic - provider defaults to `claude-opus-4-8`. Local faster-whisper execution is wired in M5. - ## 2026-07-03 - Licensing: copyleft, multi-part (REUSE-style) - **Decision:** Code (firmware, server, app) under GPL-3.0-only; hardware design under diff --git a/state/TODO.md b/state/TODO.md index 7ded7e9..3647e5b 100644 --- a/state/TODO.md +++ b/state/TODO.md @@ -29,11 +29,11 @@ ## Pending - AI provider flexibility -- [x] Pluggable AI providers (server): transcription + LLM target any open-standard - endpoint (OpenAI-compatible / local / Ollama) OR commercial (OpenAI, Anthropic, - Gemini). Config-driven, no lock-in. Provider layer + HTTP + Anthropic providers - done (PR #5). Local faster-whisper execution + calling the providers from the - ingest pipeline complete in M5. See docs/ai-providers.md. +- [~] Pluggable AI providers (server): transcription + LLM can target any open-standard + endpoint (OpenAI-compatible / local faster-whisper / Ollama) OR a commercial API + (OpenAI, Anthropic, Gemini). Config-driven, no lock-in. (branch + `feature/server-ai-providers`) - provider layer + HTTP providers landing now; + local faster-whisper wiring completes in M5. ## Pending - Plaud-derived backlog (see docs/plaud-comparison.md)