feat(server): pluggable AI providers - any open-standard or commercial AI
Lets the owner point transcription and summarisation at any AI: an open-standard
endpoint (OpenAI-compatible / local faster-whisper / Ollama) or a commercial API
(OpenAI, Anthropic, Gemini). Config-driven, self-hostable, no lock-in.
What changed:
- server/app/providers/: provider layer.
- base.py: Transcriber/Summariser protocols + shared summary prompt + tolerant JSON
parser (uniform Summary shape across providers).
- summary.py: OpenAICompatibleSummariser (any /chat/completions - OpenAI, Groq,
OpenRouter, LocalAI, LM Studio, vLLM, Ollama /v1) and AnthropicSummariser (Claude
via the official anthropic SDK; Messages API has no OpenAI-compatible endpoint).
- transcription.py: OpenAICompatibleTranscriber (/audio/transcriptions - OpenAI,
Groq, self-hosted whisper server) and LocalWhisperTranscriber (faster-whisper,
execution wired in M5).
- factory.py: builds the configured providers with per-provider defaults
(anthropic -> claude-opus-4-8, openai_compatible -> gpt-4o-mini, ollama -> llama3.1).
- config.py + .env.example: transcription_provider / llm_provider selectors + base_url,
key, model settings; local faster-whisper and Ollama kept as the self-hosted defaults.
- main.py: /health now reports the resolved provider names (no secrets).
- requirements.txt: httpx drives all HTTP providers; anthropic + faster-whisper are
optional, only for their respective providers.
- docs/ai-providers.md: config recipes for OpenAI, Groq, Anthropic, Gemini, LocalAI,
LM Studio, Ollama, self-hosted whisper.
- state/: DECISIONS, ARCHITECTURE, TODO updated.
Why:
- The user asked to connect the device to any open standard AI or commercial one; this
is also the core differentiator vs Plaud's locked cloud.
Notes:
- Anthropic provider uses the official SDK and defaults to claude-opus-4-8 (per the
claude-api guidance). AI deps are optional per chosen provider. Modules byte-compile
cleanly; end-to-end wiring into the ingest pipeline lands with M5.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
34eb17abbc
commit
51321aa7c5
12 changed files with 380 additions and 13 deletions
65
server/app/providers/transcription.py
Normal file
65
server/app/providers/transcription.py
Normal file
|
|
@ -0,0 +1,65 @@
|
|||
# SPDX-License-Identifier: GPL-3.0-only
|
||||
"""Transcription providers.
|
||||
|
||||
- OpenAICompatibleTranscriber: any endpoint speaking the OpenAI /audio/transcriptions API -
|
||||
OpenAI Whisper, Groq (whisper-large-v3), or a self-hosted whisper.cpp/faster-whisper
|
||||
server exposing that route.
|
||||
- LocalWhisperTranscriber: in-process faster-whisper. Wired up in M5 (needs the model
|
||||
download); the interface and config exist now so it is selectable.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
import httpx
|
||||
|
||||
from ..models import Transcript, TranscriptSegment
|
||||
|
||||
|
||||
class OpenAICompatibleTranscriber:
|
||||
def __init__(self, base_url: str, api_key: str, model: str):
|
||||
self.base_url = base_url.rstrip("/")
|
||||
self.api_key = api_key
|
||||
self.model = model
|
||||
self.name = f"openai_compatible:{model}"
|
||||
|
||||
def transcribe(self, audio_path: str, language: str | None = None) -> Transcript:
|
||||
headers = {"Authorization": f"Bearer {self.api_key}"} if self.api_key else {}
|
||||
data = {"model": self.model, "response_format": "verbose_json"}
|
||||
if language:
|
||||
data["language"] = language
|
||||
with open(audio_path, "rb") as f:
|
||||
files = {"file": (os.path.basename(audio_path), f, "audio/wav")}
|
||||
resp = httpx.post(
|
||||
f"{self.base_url}/audio/transcriptions",
|
||||
headers=headers,
|
||||
data=data,
|
||||
files=files,
|
||||
timeout=600,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
body = resp.json()
|
||||
segments = [
|
||||
TranscriptSegment(start=s.get("start", 0.0), end=s.get("end", 0.0),
|
||||
text=s.get("text", ""))
|
||||
for s in body.get("segments", [])
|
||||
]
|
||||
return Transcript(
|
||||
recording_id="",
|
||||
language=body.get("language") or language or "en",
|
||||
model=self.name,
|
||||
text=body.get("text", ""),
|
||||
segments=segments,
|
||||
)
|
||||
|
||||
|
||||
class LocalWhisperTranscriber:
|
||||
def __init__(self, model: str, device: str, compute_type: str):
|
||||
self.model = model
|
||||
self.device = device
|
||||
self.compute_type = compute_type
|
||||
self.name = f"faster-whisper:{model}"
|
||||
|
||||
def transcribe(self, audio_path: str, language: str | None = None) -> Transcript:
|
||||
# M5: load faster_whisper.WhisperModel(self.model, device, compute_type) and run it.
|
||||
raise NotImplementedError("Local faster-whisper transcription lands in M5")
|
||||
Loading…
Add table
Add a link
Reference in a new issue