Laurence
8fd9b694f8
Persists the branding/commercial/infrastructure work (previously only in chat and on the live site) into version control. What changed: - docs/naming.md: due-diligence and the decision to brand the product Nightjar (OpenScribe domain/GitHub/trademark collisions; alternatives screened; why Nightjar). - docs/hosted-service.md: the commercial model - plan tiers (Self-host / Cloud Starter / Cloud Pro / Private), indicative pricing, backend mapping, and the build-to-sell roadmap (metering, billing, provisioning, DPA). - docs/infrastructure.md: backend infra plan. Primary option = self-host on the 3x Minisforum MS-02 cluster (one with an RTX 3050 6GB) fronted by a Cloudflare Tunnel, with node roles, the 3050 capacity reality, caveats, and a Hetzner cloud fallback. Provision-later. - site/: reproducible marketing-site source - block content for all 11 pages (rebranded to Nightjar), the navigation, the Contact Form 7 config (honeypot), the ApisCP SOAP helper (tools/apiscp.php, no secret), and a README on how the WordPress site is built and managed via the API. - state/: DECISIONS (Nightjar rebrand, hosted service, MS-02 backend), PROJECT (brand + commercial section), TODO (rename decision, trademark, mailbox, pricing, hosted-service build, infra provisioning). Why: - User asked to save everything to the repo. Captures the product rebrand, the commercialisation plan, and the infrastructure decision so a cold session has the full picture. Notes: - The repo is still named `openscribe`; the product/brand is Nightjar. A full codebase rename is deferred (tracked in TODO + docs/naming.md). - No secrets committed: the ApisCP API key is read from a local scratch file, never the repo. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
103 lines
5.2 KiB
Markdown
103 lines
5.2 KiB
Markdown
<!-- SPDX-License-Identifier: CC-BY-SA-4.0 -->
|
|
# Nightjar hosted backend - infrastructure plan
|
|
|
|
Plan for the infrastructure that runs the **hosted** Nightjar service (the commercial
|
|
plans). The open-source project itself does not require any of this; self-hosters run
|
|
their own. This is for 12 Hobbies Studio to operate the managed service.
|
|
|
|
Status: **plan only, not provisioned** (decided 2026-07). Stand it up at first beta /
|
|
paying users. See `docs/hosted-service.md` for the plan tiers this maps to.
|
|
|
|
## Two backends, matching the two plan families
|
|
|
|
| Plan family | AI runs on | Infra |
|
|
|-------------|-----------|-------|
|
|
| **Cloud Starter / Pro** | commercial API (Groq / OpenAI / Anthropic) | small orchestration node; heavy compute offloaded |
|
|
| **Private** | self-hosted open models (faster-whisper + local LLM) | a GPU node; audio never leaves our hardware |
|
|
|
|
Cloud tiers are cheap to run (compute is per-minute on a commercial provider, keeping the
|
|
GBP 6 / 18 pricing profitable). The Private tier needs real hardware but is the privacy /
|
|
sovereignty story.
|
|
|
|
## Primary option: self-host on the MS-02 cluster (best for small scale)
|
|
|
|
Hardware on hand: **3x Minisforum MS-02-class mini-workstations**, one fitted with an
|
|
**NVIDIA RTX 3050 6GB**. High-core CPUs, dual 10GbE, NVMe, PCIe GPU slot. This is a strong
|
|
small-scale backend and a better fit for the Private tier than rented cloud.
|
|
|
|
### Node roles
|
|
|
|
- **Node A (RTX 3050) - AI worker.** faster-whisper on the GPU + a local LLM (Ollama) for
|
|
summaries. This is the Private-tier engine; models run on our own hardware.
|
|
- **Node B - app tier.** FastAPI ingest + open API, Postgres (accounts, metadata,
|
|
billing), Redis queue, reverse proxy.
|
|
- **Node C - storage + overflow.** MinIO (S3-compatible object storage for audio), a CPU
|
|
transcription worker for extra throughput, backups, and HA spare.
|
|
|
|
Dual 10GbE makes shared storage and job distribution between the three effortless. Run
|
|
Docker Compose per node, or k3s / Docker Swarm across all three for failover.
|
|
|
|
### The RTX 3050 6GB in practice
|
|
|
|
- **Transcription: excellent.** faster-whisper (CTranslate2, int8) runs whisper
|
|
medium/large-v3 within 6GB and several times faster than real-time - hundreds of hours
|
|
of audio per day on one GPU. Far beyond small-scale need.
|
|
- **Summaries: fine with 7-8B models.** A quantised 8B model (Llama 3.1 8B, Qwen2.5 7B,
|
|
Mistral 7B at Q4) fits in 6GB and writes good summaries in seconds. 13B+ will not fit in
|
|
VRAM; for bigger, use CPU (slower) or add a larger GPU later. Not needed for summaries.
|
|
|
|
So the single 3050 covers both AI jobs the Private tier needs. Cloud tiers barely touch
|
|
this hardware (orchestration + storage only).
|
|
|
|
### Getting to it from the internet: Cloudflare Tunnel
|
|
|
|
Do not expose the home/office IP or open ports. We are already on Cloudflare - run a
|
|
**Cloudflare Tunnel**: the nodes make an outbound connection and `api.nightjar...` is served
|
|
through Cloudflare with TLS and DDoS protection, no inbound ports, no static IP. Free.
|
|
|
|
### Caveats of self-hosting a paid service
|
|
|
|
- **Uptime / power / bandwidth.** Fine for beta / early customers. Add a UPS, decent
|
|
upload bandwidth, tunnel resilience. For SLA-bound customers later, consider colo or a
|
|
hybrid.
|
|
- **Backups.** NVMe fails. Keep offsite backups of Postgres + audio (cheap cloud object
|
|
storage target).
|
|
- **Data residency.** On our own UK premises - a strong Private-tier pitch ("processed on
|
|
our own hardware in the UK, never sent to a third-party AI"). Mind UK GDPR obligations.
|
|
- **When we outgrow it.** Many concurrent Private customers, or wanting large LLMs - add
|
|
VRAM / a second GPU, or a cloud GPU node.
|
|
|
|
## Fallback / alternative option: cloud VMs (Hetzner)
|
|
|
|
If self-hosting uptime is a concern, or to launch before the cluster is ready:
|
|
|
|
- **Phase 1 (Cloud tiers):** one small Hetzner VM (CPX31, 4 vCPU / 8 GB, ~EUR 15/mo, EU
|
|
region for GDPR). Docker Compose: FastAPI + workers + Redis + Postgres + Caddy/LE.
|
|
Object storage via Hetzner Object Storage (~EUR 5/mo). AI offloaded to a commercial API.
|
|
All-in ~EUR 20-30/mo.
|
|
- **Phase 2 (Private tier):** a CPU-optimised (CCX23/33, ~EUR 30-60/mo) or GPU VM for
|
|
self-hosted whisper + LLM. Priced into the Private tier.
|
|
|
|
A hybrid also works: MS-02 cluster does the compute; a tiny cloud VM (or just the
|
|
Cloudflare Tunnel) is the public front door.
|
|
|
|
## Architecture (both options)
|
|
|
|
```
|
|
device --sync--> ingest API --> queue --> transcription worker --> summary (LLM)
|
|
| (GPU or commercial) (local or commercial)
|
|
v
|
|
Postgres (metadata/accounts) object storage (audio + artefacts)
|
|
|
|
|
open REST API + exports <-- app / customers
|
|
```
|
|
|
|
## Provisioning checklist (when ready)
|
|
|
|
- [ ] Decide compute home: MS-02 cluster (primary) vs cloud (fallback) vs hybrid.
|
|
- [ ] Cloudflare Tunnel for `api.nightjar...` (if self-hosting).
|
|
- [ ] Docker Compose (or k3s): FastAPI, workers, Redis, Postgres, MinIO/Object Storage.
|
|
- [ ] GPU node: faster-whisper (CUDA) + Ollama (7-8B) for the Private tier.
|
|
- [ ] Backups (Postgres + audio) offsite; UPS on the nodes.
|
|
- [ ] Metering + billing hookup for the plans (see `docs/hosted-service.md`).
|
|
- [ ] Data-processing agreement + retention policy (Private tier).
|