Record or upload audio
+Nightjar transcribes it and writes a summary using the AI you configured. Your + recordings appear on the left.
+diff --git a/server/Dockerfile b/server/Dockerfile new file mode 100644 index 0000000..dd6c6b6 --- /dev/null +++ b/server/Dockerfile @@ -0,0 +1,11 @@ +# SPDX-License-Identifier: GPL-3.0-only +# Nightjar web app - runs the Plaud-style UI + API. AI is done by your configured +# providers (default: Groq), so this image stays slim (no local ML deps). +FROM python:3.11-slim +WORKDIR /app +RUN pip install --no-cache-dir \ + "fastapi>=0.111" "uvicorn[standard]>=0.30" "pydantic>=2.7" \ + "pydantic-settings>=2.3" "python-multipart>=0.0.9" "httpx>=0.27" "anthropic>=0.40" +COPY app ./app +EXPOSE 8000 +CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"] diff --git a/server/README.md b/server/README.md index 650de40..1c136ca 100644 --- a/server/README.md +++ b/server/README.md @@ -1,48 +1,60 @@ -# OpenScribe server +# Nightjar server + web app -Self-hosted FastAPI server: ingests recordings, transcribes them (faster-whisper), -summarises them (Ollama), and serves the open API with exports. Everything runs on -hardware you own. +A self-hosted FastAPI app with a **Plaud-style web interface**: upload or record audio in +the browser, watch it transcribe and summarise, browse a library, play back, and export. +The AI is done by whichever providers you configure, so you own the whole pipeline. -> Status: M0 scaffold. The API shape is live and browsable at `/docs` with in-memory -> stubs. Transcription lands in M5, summaries in M6, real storage/DB alongside. +## Try it in two minutes -## Run (dev) +You need one AI key. The default config uses **Groq** (free tier, fast Whisper + LLM) for +both transcription and summaries. ```bash cd server python -m venv .venv -. .venv/Scripts/activate # Windows; or: . .venv/bin/activate -pip install -r requirements.txt -cp .env.example .env # edit as needed +. .venv/bin/activate # Windows: .venv\Scripts\activate +pip install fastapi "uvicorn[standard]" pydantic pydantic-settings python-multipart httpx anthropic +cp .env.example .env # then paste your Groq key into .env (both API_KEY lines) uvicorn app.main:app --reload ``` -- API docs (Swagger UI): http://localhost:8000/docs -- Health: http://localhost:8000/health +Open **http://localhost:8000** and press Record (or Upload an audio file). It will show +`transcribing… → summarising… → done`, then the transcript, a summary with key points and +action items, an audio player, and export buttons (TXT / Markdown / SRT / VTT / JSON). -## Self-hosted dependencies +Confirm your providers loaded at **http://localhost:8000/health**. -For the AI features (M5/M6) you run, on your own kit: +### Or with Docker -- **MinIO** (or WebDAV / NAS) for object storage - `OPENSCRIBE_STORAGE_BACKEND=s3`. -- **Ollama** for summaries - `ollama serve` and `ollama pull llama3.1`. -- **faster-whisper** downloads its model on first use; CPU works, CUDA is faster. - -None of these are required for plain recording and transfer; they add transcription and -summaries. - -## Layout - -``` -app/main.py FastAPI app + routes (mirrors ../api/openapi.yaml) -app/config.py Settings from env / .env -app/models.py Pydantic models (kept in sync with the OpenAPI schemas) -requirements.txt -.env.example +```bash +cd server +docker build -t nightjar . +docker run -p 8000:8000 --env-file .env -v "$PWD/media:/app/media" nightjar ``` -## API +## Choosing a different AI -The contract is `../api/openapi.yaml`. The device implements the LAN "device" paths; this -server implements ingest, transcript, summary and export. \ No newline at end of file +Everything is set in `.env` (see `.env.example` and `docs/ai-providers.md`): OpenAI or any +OpenAI-compatible endpoint, Anthropic Claude, or fully local (Ollama + faster-whisper for a +"nothing leaves my machine" setup). Change the vars, restart, done. + +## What's inside + +``` +app/main.py FastAPI: upload/list/detail/audio/export + serves the web UI +app/pipeline.py audio -> transcribe -> summarise (background task) +app/store.py recordings in SQLite, audio on disk (demo storage) +app/providers/ the pluggable transcription + LLM providers +app/web/ the Plaud-style single-page UI (no build step) +``` + +This is the demo/single-node app. The hosted multi-tenant service (metering, billing, +object storage, Postgres, the Private-tier GPU node) is described in +`../docs/hosted-service.md` and `../docs/infrastructure.md`. + +## Notes + +- Recordings and the SQLite index live under `NIGHTJAR_LOCAL_MEDIA_DIR` (default `./media`). +- Browser recording produces WebM/Opus, which Groq and OpenAI accept directly - no ffmpeg + needed for the demo. +- The API contract for the device/server is `../api/openapi.yaml`. diff --git a/server/app/config.py b/server/app/config.py index 6c3f84b..bf63d14 100644 --- a/server/app/config.py +++ b/server/app/config.py @@ -10,7 +10,7 @@ from pydantic_settings import BaseSettings, SettingsConfigDict class Settings(BaseSettings): - model_config = SettingsConfigDict(env_prefix="OPENSCRIBE_", env_file=".env") + model_config = SettingsConfigDict(env_prefix="NIGHTJAR_", env_file=".env") # API api_token: str = "change-me" # bearer token for write endpoints diff --git a/server/app/main.py b/server/app/main.py index 9e2a071..6554dd5 100644 --- a/server/app/main.py +++ b/server/app/main.py @@ -1,77 +1,136 @@ # SPDX-License-Identifier: GPL-3.0-only -"""OpenScribe server - FastAPI app. +"""Nightjar web app - a Plaud-style interface plus the JSON API. -M0 scaffold: wires the routes from api/openapi.yaml with in-memory stubs so the API shape -is real and browsable at /docs. The AI pipeline (faster-whisper transcription in M5, -Ollama summaries in M6) and real storage/DB replace the stubs in later milestones. +Upload or record audio, watch it transcribe and summarise, browse a library, play back and +export. The heavy AI is done by whichever providers you configure (see .env.example); +point it at a free Groq key and it works end to end. """ from __future__ import annotations -from fastapi import FastAPI, HTTPException +import json +import os +from contextlib import asynccontextmanager +from fastapi import BackgroundTasks, FastAPI, HTTPException, UploadFile +from fastapi.responses import FileResponse, PlainTextResponse, Response +from fastapi.staticfiles import StaticFiles + +from . import store from .config import settings -from .models import Recording, RecordingPage, Summary, Transcript +from .pipeline import process -app = FastAPI( - title="OpenScribe API", - version="0.1.0", - description="Self-hosted AI voice recorder server. See api/openapi.yaml.", -) +WEB_DIR = os.path.join(os.path.dirname(__file__), "web") -# In-memory store stands in for the DB + object storage until M5. -_recordings: dict[str, Recording] = {} + +@asynccontextmanager +async def lifespan(app: FastAPI): + os.makedirs(settings.local_media_dir, exist_ok=True) + store.init() + yield + + +app = FastAPI(title="Nightjar", version="0.1.0", lifespan=lifespan) @app.get("/health") def health() -> dict: - # Report the resolved provider names (never secrets) so operators can confirm config. from .providers import build_summariser, build_transcriber try: summariser = build_summariser().name - except Exception as exc: # e.g. anthropic dep not installed + except Exception as exc: summariser = f"unavailable ({type(exc).__name__})" return { "status": "ok", - "storage_backend": settings.storage_backend, - "transcription_provider": settings.transcription_provider, "transcriber": build_transcriber().name, - "llm_provider": settings.llm_provider, "summariser": summariser, } -@app.get("/api/v1/recordings", response_model=RecordingPage, tags=["recordings"]) -def list_recordings(limit: int = 50) -> RecordingPage: - return RecordingPage(items=list(_recordings.values())[:limit], next_cursor=None) +@app.post("/api/recordings") +async def create_recording( + file: UploadFile, background: BackgroundTasks, title: str | None = None +) -> dict: + rec_id = store.create( + title=title or (file.filename or "Recording"), + filename=file.filename or "audio", + mime=file.content_type or "application/octet-stream", + ) + with open(store.audio_path(rec_id), "wb") as f: + while chunk := await file.read(1 << 20): + f.write(chunk) + background.add_task(process, rec_id) # transcribe + summarise off the request path + return store.get(rec_id) -@app.get("/api/v1/recordings/{rec_id}", response_model=Recording, tags=["recordings"]) -def get_recording(rec_id: str) -> Recording: - rec = _recordings.get(rec_id) - if rec is None: - raise HTTPException(status_code=404, detail="No such recording") +@app.get("/api/recordings") +def list_recordings() -> list[dict]: + return store.list_all() + + +@app.get("/api/recordings/{rec_id}") +def get_recording(rec_id: str) -> dict: + rec = store.get(rec_id) + if not rec: + raise HTTPException(404, "No such recording") return rec -@app.get("/api/v1/recordings/{rec_id}/transcript", response_model=Transcript, tags=["server"]) -def get_transcript(rec_id: str) -> Transcript: - # Implemented in M5 (faster-whisper). Until then, signal "not transcribed yet". - if rec_id not in _recordings: - raise HTTPException(status_code=404, detail="No such recording") - raise HTTPException(status_code=409, detail="Not transcribed yet (M5)") +@app.get("/api/recordings/{rec_id}/audio") +def get_audio(rec_id: str): + rec = store.get(rec_id) + if not rec: + raise HTTPException(404, "No such recording") + return FileResponse(store.audio_path(rec_id), media_type=rec["mime"] or "audio/webm") -@app.get("/api/v1/recordings/{rec_id}/summary", response_model=Summary, tags=["server"]) -def get_summary(rec_id: str) -> Summary: - # Implemented in M6 (Ollama). Until then, signal "not summarised yet". - if rec_id not in _recordings: - raise HTTPException(status_code=404, detail="No such recording") - raise HTTPException(status_code=409, detail="Not summarised yet (M6)") +@app.delete("/api/recordings/{rec_id}", status_code=204) +def delete_recording(rec_id: str): + if not store.delete(rec_id): + raise HTTPException(404, "No such recording") + return Response(status_code=204) -@app.post("/api/v1/ingest", response_model=Recording, status_code=202, tags=["server"]) -def ingest(recording: Recording) -> Recording: - # M5 will store audio to the object store and queue transcription + summary. - _recordings[recording.id] = recording - return recording \ No newline at end of file +def _ts(seconds: float, sep: str) -> str: + h, rem = divmod(int(seconds), 3600) + m, s = divmod(rem, 60) + ms = int((seconds - int(seconds)) * 1000) + return f"{h:02d}:{m:02d}:{s:02d}{sep}{ms:03d}" + + +@app.get("/api/recordings/{rec_id}/export") +def export_recording(rec_id: str, format: str = "txt"): + rec = store.get(rec_id) + if not rec: + raise HTTPException(404, "No such recording") + t = rec.get("transcript") or {} + s = rec.get("summary") or {} + segs = t.get("segments") or [] + + if format == "json": + return Response(json.dumps(rec, indent=2), media_type="application/json") + if format == "txt": + return PlainTextResponse(t.get("text", "")) + if format == "md": + body = [f"# {rec['title']}", "", "## Summary", s.get("overview", ""), ""] + if s.get("key_points"): + body += ["### Key points"] + [f"- {p}" for p in s["key_points"]] + [""] + if s.get("action_items"): + body += ["### Action items"] + [f"- [ ] {a}" for a in s["action_items"]] + [""] + body += ["## Transcript", "", t.get("text", "")] + return PlainTextResponse("\n".join(body), media_type="text/markdown") + if format in ("srt", "vtt"): + sep = "," if format == "srt" else "." + lines = ["WEBVTT", ""] if format == "vtt" else [] + for i, seg in enumerate(segs, 1): + if format == "srt": + lines.append(str(i)) + lines.append(f"{_ts(seg['start'], sep)} --> {_ts(seg['end'], sep)}") + lines.append(seg["text"].strip()) + lines.append("") + return PlainTextResponse("\n".join(lines)) + raise HTTPException(400, "Unknown format") + + +# Serve the web UI at the root (must be mounted after the API routes). +app.mount("/", StaticFiles(directory=WEB_DIR, html=True), name="web") diff --git a/server/app/pipeline.py b/server/app/pipeline.py new file mode 100644 index 0000000..984649f --- /dev/null +++ b/server/app/pipeline.py @@ -0,0 +1,36 @@ +# SPDX-License-Identifier: GPL-3.0-only +"""The processing pipeline: audio -> transcript -> summary, using the provider layer. + +Run as a background task after upload. Updates the recording's status as it goes so the +web UI can show progress (queued -> transcribing -> summarising -> done). +""" +from __future__ import annotations + +from . import store +from .providers import build_summariser, build_transcriber + + +def process(rec_id: str) -> None: + path = store.audio_path(rec_id) + try: + store.set_status(rec_id, "transcribing") + transcript = build_transcriber().transcribe(path) + + store.set_status(rec_id, "summarising") + summary = build_summariser().summarise(rec_id, transcript.text) + + store.set_result( + rec_id, + language=transcript.language, + transcript={ + "text": transcript.text, + "segments": [s.model_dump() for s in transcript.segments], + }, + summary={ + "overview": summary.overview, + "key_points": summary.key_points, + "action_items": summary.action_items, + }, + ) + except Exception as exc: # surface any provider/network error to the UI + store.set_status(rec_id, "error", f"{type(exc).__name__}: {exc}") diff --git a/server/app/store.py b/server/app/store.py new file mode 100644 index 0000000..690a3dd --- /dev/null +++ b/server/app/store.py @@ -0,0 +1,107 @@ +# SPDX-License-Identifier: GPL-3.0-only +"""Tiny persistence for the web app: recordings in SQLite, audio on disk. + +Deliberately dependency-free (sqlite3 is stdlib) so the demo runs anywhere. Not meant for +high concurrency; the hosted service uses Postgres + object storage (see docs/). +""" +from __future__ import annotations + +import json +import os +import sqlite3 +import time +import uuid +from typing import Any + +from .config import settings + +_DB = os.path.join(settings.local_media_dir, "nightjar.db") + + +def _conn() -> sqlite3.Connection: + os.makedirs(settings.local_media_dir, exist_ok=True) + c = sqlite3.connect(_DB) + c.row_factory = sqlite3.Row + return c + + +def init() -> None: + with _conn() as c: + c.execute( + """CREATE TABLE IF NOT EXISTS recordings ( + id TEXT PRIMARY KEY, + title TEXT, + filename TEXT, + mime TEXT, + created_at REAL, + status TEXT, -- queued|transcribing|summarising|done|error + error TEXT, + language TEXT, + transcript TEXT, -- JSON: {text, segments} + summary TEXT -- JSON: {overview, key_points, action_items} + )""" + ) + + +def audio_path(rec_id: str) -> str: + return os.path.join(settings.local_media_dir, f"{rec_id}.audio") + + +def create(title: str, filename: str, mime: str) -> str: + rec_id = "rec_" + uuid.uuid4().hex[:12] + with _conn() as c: + c.execute( + "INSERT INTO recordings (id,title,filename,mime,created_at,status) " + "VALUES (?,?,?,?,?,?)", + (rec_id, title, filename, mime, time.time(), "queued"), + ) + return rec_id + + +def set_status(rec_id: str, status: str, error: str | None = None) -> None: + with _conn() as c: + c.execute("UPDATE recordings SET status=?, error=? WHERE id=?", (status, error, rec_id)) + + +def set_result(rec_id: str, language: str, transcript: dict, summary: dict) -> None: + with _conn() as c: + c.execute( + "UPDATE recordings SET status='done', language=?, transcript=?, summary=? WHERE id=?", + (language, json.dumps(transcript), json.dumps(summary), rec_id), + ) + + +def _row(r: sqlite3.Row) -> dict[str, Any]: + return { + "id": r["id"], + "title": r["title"], + "created_at": r["created_at"], + "status": r["status"], + "error": r["error"], + "language": r["language"], + "mime": r["mime"], + "transcript": json.loads(r["transcript"]) if r["transcript"] else None, + "summary": json.loads(r["summary"]) if r["summary"] else None, + } + + +def get(rec_id: str) -> dict | None: + with _conn() as c: + r = c.execute("SELECT * FROM recordings WHERE id=?", (rec_id,)).fetchone() + return _row(r) if r else None + + +def list_all() -> list[dict]: + with _conn() as c: + rows = c.execute("SELECT * FROM recordings ORDER BY created_at DESC").fetchall() + return [_row(r) for r in rows] + + +def delete(rec_id: str) -> bool: + with _conn() as c: + cur = c.execute("DELETE FROM recordings WHERE id=?", (rec_id,)) + try: + os.remove(audio_path(rec_id)) + except OSError: + pass + return cur.rowcount > 0 diff --git a/server/app/web/app.js b/server/app/web/app.js new file mode 100644 index 0000000..692862e --- /dev/null +++ b/server/app/web/app.js @@ -0,0 +1,103 @@ +// SPDX-License-Identifier: GPL-3.0-only - Nightjar web UI logic (vanilla JS) +(function () { + const $ = (s) => document.querySelector(s); + const listEl = $("#list"), detailEl = $("#detail"); + const recBtn = $("#recBtn"), recTimer = $("#recTimer"), uploadEl = $("#upload"); + let selected = null, poller = null; + let mediaRecorder = null, chunks = [], recStart = 0, recInterval = null; + + const PROC = ["queued", "transcribing", "summarising"]; + const fmtTime = (s) => { s = Math.floor(s || 0); const m = Math.floor(s / 60); return `${m}:${String(s % 60).padStart(2, "0")}`; }; + const when = (t) => new Date((t || 0) * 1000).toLocaleString(); + const esc = (x) => (x || "").replace(/&/g, "&").replace(//g, ">"); + + async function j(url, opts) { const r = await fetch(url, opts); if (!r.ok && r.status !== 204) throw new Error(await r.text()); return r.status === 204 ? null : r.json(); } + + function badge(status) { + if (status === "done") return 'done'; + if (status === "error") return 'error'; + return `${status}`; + } + + async function refreshList() { + const items = await j("/api/recordings"); + listEl.innerHTML = items.map((r) => ` +
${esc(r.error)}
Check the server's AI provider config (see .env).
${esc(s.overview)}
+ ${s.key_points && s.key_points.length ? `Nightjar transcribes it and writes a summary using the AI you configured. Your + recordings appear on the left.
+