laurence/openscribe
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
openscribe/state/TODO.md
Laurence 19c3e156a0
Some checks failed
ci / firmware (pull_request) Failing after 2s
Details
ci / openapi (pull_request) Failing after 32s
Details
ci / emulator (pull_request) Failing after 35s
Details
ci / server (pull_request) Failing after 29s
Details
feat(web): Nightjar web app - Plaud-style record/transcribe/summarise UI 
A runnable, self-hosted web interface so the whole workflow can be tested end to
end: record or upload audio in the browser, watch it transcribe and summarise,
browse a library, play back, and export.

What changed:
- server/app/store.py: SQLite + on-disk audio storage for recordings (stdlib only).
- server/app/pipeline.py: background task audio -> transcribe -> summarise via the
  existing provider layer, updating status (queued/transcribing/summarising/done/error).
- server/app/main.py: web API - POST upload, list, detail, audio (Range), delete, and
  export (txt/md/srt/vtt/json) - and serves the SPA at /.
- server/app/web/: Plaud-style single-page UI (index.html, styles.css, app.js). Sidebar
  library, in-browser recording (MediaRecorder) + file upload, live status polling,
  audio player, summary (overview/key points/actions), timestamped transcript, exports.
- server/Dockerfile + README: two-minute run instructions (default provider: Groq free
  tier for both Whisper + LLM), and a Docker option.
- config: env prefix switched OPENSCRIBE_ -> NIGHTJAR_ to match the brand and the site
  tutorials; .env.example rewritten with a ready Groq quick-start.
- state/TODO: web app recorded as done.

Why:
- User asked for a Plaud-like web interface to test how it all works. Nothing testable
  existed before (marketing site is a brochure; pipeline was unwired). This delivers a
  real, runnable product demo and effectively lands M5/M6 for the HTTP providers.

Notes:
- Slim by design: AI is offloaded to the configured provider, so no local ML deps needed
  for the demo. Byte-compiles clean; JS passes node --check. Local faster-whisper still
  needs its model (M5) for the fully-offline path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 09:57:42 +01:00

5.7 KiB
Raw Blame History

TODO

The current state of play. Keep it honest and current; this is what the next session reads to know what to do. Move items between sections as they progress.

In progress

  • (nothing active - M3 is next)

Pending - brand + commercial (Nightjar)

  • Decide whether to rename the whole project/repo OpenScribe -> Nightjar (SPDX headers, firmware namespaces, docs, Forgejo repo). Website already rebranded. See docs/naming.md.
  • Formal trademark clearance for "Nightjar" (USPTO / UK IPO / EUIPO / attorney).
  • Create the hello@12hobbies.studio mailbox (site forms/CTAs point at it) or change it.
  • Confirm real hosted pricing + quotas (current numbers are indicative).
  • Hosted service build: metering, Stripe billing + entitlements, tenant provisioning, job routing (commercial vs Private node), DPA + retention policy. See docs/hosted-service.md.
  • Provision backend infra at first beta/paying users (MS-02 cluster + Cloudflare Tunnel, or Hetzner fallback). See docs/infrastructure.md. Optional: draft Docker Compose + tunnel config.

Pending (roughly in build order)

  • M3 Independent uploader: charge/VBUS detect -> powered mode, S3-compatible + WebDAV upload of audio + metadata (config store already landed in M2). (branch feature/fw-uploader)
  • M4 BLE GATT: device info, battery, record control, WiFi provisioning, status notifications; hand-off to WiFi for transfer. (branch feature/fw-ble)
  • M5 Server ingest + transcription: FastAPI ingest from object store/upload, store, faster-whisper transcription, DB, exports (audio/TXT/SRT/VTT/JSON). REST API. (branch feature/server-ingest-stt)
  • M6 Server summaries: Ollama LLM summary + action items, Markdown export. (branch feature/server-summary)
  • M7 Flutter app: BLE provisioning, device dashboard, library from server API, playback, transcript/summary view, export/share, settings. (branch feature/app-mvp)
  • M8 3D case: parametric OpenSCAD enclosure fit to the BOM board + battery; print/fit. (branch feature/case-v1)
  • M9 Hardening + OTA: API auth, credential handling, OTA update, docs handoff. (branch feature/hardening-ota)

Pending - AI provider flexibility

  • Pluggable AI providers (server): transcription + LLM target any open-standard endpoint (OpenAI-compatible / local / Ollama) OR commercial (OpenAI, Anthropic, Gemini). Config-driven, no lock-in. Provider layer + HTTP + Anthropic providers done (PR #5). Local faster-whisper execution + calling the providers from the ingest pipeline complete in M5. See docs/ai-providers.md.

Pending - Plaud-derived backlog (see docs/plaud-comparison.md)

  • Speaker diarisation + labels (server); rename-speaker in the app.
  • Summary templating: built-in templates (meeting/action-items/to-do/mind-map/ decisions) + user templates; role-based multidimensional summaries.
  • Ask AI: chat grounded in a recording's transcript (RAG), via the chosen LLM provider.
  • Custom vocabulary / prompt hints for transcription + summary.
  • Multimodal attachments: text notes + images attached to a recording.
  • Mark-a-moment: device long-press during recording -> highlight timestamp, surfaced in the app and used to focus summaries.
  • Auto title + auto formatting of transcripts and summaries.
  • Library organisation: tags, folders, full-text search (server + app).
  • Share links for summaries (extends the export endpoint).
  • Optional hardware variant with a small OLED status display (InstantView equivalent).
  • Calendar integration (nice-to-have).

Done

  • Nightjar web app (Plaud-style): browser record/upload -> transcribe -> summarise -> library, playback, transcript, summary, export. FastAPI + vanilla-JS SPA in server/, wired to the provider layer (Groq/OpenAI/Anthropic/local). Runnable demo; this is also the practical landing of M5/M6 for the HTTP providers. Config prefix switched to NIGHTJAR_. (PR: feature/web-app, 2026-07-06)
  • Dev testing + emulation harness - native unit tests (wav/range), Wokwi emulator, cppcheck + OpenAPI validation in CI, docs/testing.md (PR #3, 2026-07-03).
  • M2 On-device WiFi + REST API - WiFi/SoftAP + mDNS + config(NVS) + device REST API (status, list, metadata, audio-with-Range, delete, record start/stop, config). (PR #2 feature/fw-rest-api, 2026-07-03). Not yet hardware-verified.
  • M1 Firmware recording core - mic capture to WAV on SD + button/LED + metadata (merged: PR #1 feature/fw-recorder-core, 2026-07-03). Not yet hardware-verified.
  • M0 Scaffold - repo, licences, state docs, BOM, OpenAPI, skeletons, CI (merged to main, 2026-07-03)
  • Create Forgejo repo laurence/openscribe (public, main) - 2026-07-03
  • Decide hardware, sync model, AI stack, app platform, storage, licensing (see DECISIONS.md) - 2026-07-03

Follow-ups (deferred, do when convenient)

  • Audio calibration: INMP441 emits 24-bit samples in a 32-bit slot. M1 reads in 16-bit mode (usable for speech, may be quiet). If levels are low on the dev build, capture 32-bit and right-shift into int16 in audio.cpp.
  • Real timestamps: ids/started_at are uptime-based (no RTC/NTP yet). Add NTP now that WiFi exists (M2), and switch ids and started_at to real UTC.
  • API auth is open when no token set; SoftAP has a fixed default passphrase. Harden both before any deployment (see NOTES "Security follow-ups").

Blocked / waiting

  • Physical build + on-device verification - waiting on parts from hardware/BOM.md. Until parts arrive, firmware is validated by CI build (Forgejo Actions) of both board profiles, not on real hardware.