diff --git a/HANDOVER-2026-04-17.md b/HANDOVER-2026-04-17.md new file mode 100644 index 0000000..cf04569 --- /dev/null +++ b/HANDOVER-2026-04-17.md @@ -0,0 +1,228 @@ +# Kon Session Handover — 2026/04/17 + +## Session Summary + +Six-commit sprint executing the upgrade plan from +`/home/jake/Documents/CORBEL-Furnished-House/output/reports/kon-upgrade-plan-2026-04-17.md`. +Goal: get Kon from "core feature broken" to "ready to dogfood with friends." + +## Commits + +| Commit | Title | +|---|---| +| `96980c7` | Day 1 — fix mic capture: skip monitor sources, RMS validation, drop counting, list_devices, start_with_device | +| `41db162` | Day 1 follow-up — wire user's microphone choice through start_native_capture + live session | +| `19a6b83` | Day 2 — Codex follow-up hardening (channel disconnect, spawn_blocking, fallback silence guard, requeue counting, runtime error propagation) | +| `69d768e` | Day 3 — global toast system + first error-toast wiring on DictationPage | +| `1cce567` | Day 4 backend — FTS5 search + update_transcript + dictionary + paginated list + Tauri command surface | +| `0e22ec5` | Day 4 frontend — dual-write history to SQLite + persist History rename | +| `9f3be5c` | Day 5+6 — Settings → Vocabulary panel + Wayland self-relaunch | + +## What changed + +### Mic capture — now actually works + +The HANDOVER from 2026/04/04 flagged native live transcription as broken +(`Selected working microphone: null`, chunks repeatedly skipped as +near-silence). Root cause was PulseAudio/PipeWire monitor sources +(speaker loopback) winning the "first device that produces data within +350ms" race — silent monitor sources delivered zero-valued bytes that +satisfied that check. + +Fixed by: + +- **Skipping monitor sources** by name pattern (`.monitor` suffix, + `Monitor of ` prefix, `loopback` substring) +- **Validating by RMS energy** in a 350ms window, not just receipt + of bytes +- **Two-pass selection**: real inputs first, monitor sources only as + last resort with explicit warning log + dead-silence floor (1e-7) + guard so even fallback rejects all-zeros +- **Verbose tracing** at every step +- **Drop counter** (`Arc`) that tracks chunks lost to + backpressure, including in the validation requeue +- **Runtime error channel** so cpal stream errors after start succeeds + surface to the live session for toast display +- **`spawn_blocking`** wrapper so `start()`'s up-to-3.5s validation + window does not freeze the async runtime + +### Settings → Audio → Microphone picker + +User can now explicitly pick which input device to use. Auto mode +(empty) skips monitor sources and validates by RMS. Specific device +opens it by exact name. Setting persists in `settings.microphoneDevice` +(localStorage) and flows through to both `start_native_capture` and +`start_live_transcription_session`. + +### Toast system + +`src/lib/components/ToastViewport.svelte` mounted in root layout. +`toasts.error/warn/success/info(title, body)` from any component. +Brand-palette colours (moss/signal/ember). aria-live polite + role=alert +on errors. Honours `html.reduce-motion`. Sticky errors, auto-dismiss +others. + +First wired into DictationPage's "could not start recording" path. More +pages can adopt it incrementally — `invokeWithToast` helper makes +wrapping any Tauri call a one-liner. + +### SQLite as canonical store + +The transcripts table existed but no Tauri command read or wrote it +(Codex caught this in the joint review). Now exposed via 10 new +commands in `commands/transcripts.rs`: + +- `add_transcript`, `list_transcripts` (paginated), `count_transcripts`, + `get_transcript`, `update_transcript` (closes the long-standing + rename-never-persists TODO from `architecture-review.md §13`), + `delete_transcript`, `search_transcripts` (FTS5) +- `list_dictionary_command`, `add_dictionary_entry_command`, + `delete_dictionary_entry_command` + +Frontend `addToHistory`, `renameHistoryEntry`, `deleteFromHistory` now +dual-write to SQLite alongside localStorage. Best-effort: SQLite failure +keeps the in-memory copy and warns to console. HistoryPage rename now +calls `update_transcript`. + +Migration v2 added FTS5 virtual table with porter+unicode61 tokeniser, +diacritics-folded, plus INSERT/UPDATE/DELETE triggers to keep the FTS +index in sync. Dictionary table also added in v2. + +### Settings → Vocabulary + +New collapsible section. Add custom terms (medication names, jargon, +people's names) that the LLM cleanup prompt should preserve. Backed by +the `dictionary` SQLite table. The LLM client itself is currently a +stub; when wired, it imports `list_dictionary` from kon_storage and +injects terms into the prompt suffix. + +### Wayland self-relaunch + +`ensure_x11_on_wayland()` runs before `tauri::Builder` on Linux. If +`XDG_SESSION_TYPE=wayland`, sets `GDK_BACKEND=x11`, +`WINIT_UNIX_BACKEND=x11`, `WEBKIT_DISABLE_DMABUF_RENDERER=1` so the +HANDOVER env-var prefix is no longer needed. + +## How to dogfood + +### One-time setup on Menhir + +```bash +sudo dnf install cmake clang-devel +cd /home/jake/Documents/CORBEL-Projects/kon +npm install # if you have not already +``` + +### Launch (no env-var prefix needed any more) + +```bash +cd /home/jake/Documents/CORBEL-Projects/kon +npm run tauri dev +``` + +If anything goes wrong on Wayland, you can still fall back to: + +```bash +env GDK_BACKEND=x11 WINIT_UNIX_BACKEND=x11 \ + WEBKIT_DISABLE_DMABUF_RENDERER=1 npm run tauri dev +``` + +### What to test + +1. **Mic capture happy path.** Open Settings → Audio. Devices populate. + Pick your Blue Yeti (or whatever). Hit dictation. Speak. Text should + appear within 2 seconds. Stop, save, the recording should appear in + History. + +2. **Mic capture failure path.** Pull the USB mic mid-recording. A toast + should surface ("device disconnected" or similar). The session should + not silently produce empty transcripts. + +3. **Auto mode.** Clear the picker (set to "Auto"). Hit dictation. The + logs (terminal where you ran `npm run tauri dev`) should show: + - `[kon-audio] start: enumerated N input device(s)` + - `[kon-audio] trying '...'` for each candidate + - `[kon-audio] '...' validation: M samples, rms=...` + - `[kon-audio] selected microphone: '...'` + The selected mic should NOT be a `.monitor` source. + +4. **History rename.** Make a recording. In History, rename it to + something distinctive ("test rename 1"). Quit the app. Relaunch. + Open History. The rename should still be there (was previously + lost on relaunch — closes the old TODO). + +5. **Vocabulary panel.** Settings → Vocabulary. Add "Wren" with note + "CORBEL operating partner". Persists across restarts. (LLM cleanup + prompt is a stub so the term won't actually affect transcripts yet — + storage layer is ready for when LLM lands.) + +6. **Toasts on error.** Try to hit dictation with no microphone + connected at all. Should show a sticky error toast in the bottom-right + ("Could not start recording" + body) rather than failing silently. + +### What's deferred (does not block dogfood) + +- **Whisper pre-warm at startup.** Models still load on first dictation + (~2-5s cold start). Deferred because it needs careful threading work + to avoid blocking `setup()`. Easy to add later. +- **Auto-updater (`tauri-plugin-updater`).** Deferred because it needs + a release feed (GitHub releases or similar) which requires CI / signing + infrastructure decisions. +- **JACK monitor-name patterns.** Codex flagged that JACK setups may use + different naming conventions than PulseAudio. Test on a JACK host, + extend `is_monitor_name()` if needed. +- **HistoryPage search via FTS5.** The infrastructure is in place + (`search_transcripts` Tauri command) but HistoryPage still uses the + in-memory client-side filter, which is fine for small histories. +- **Read initial history from SQLite at boot.** Currently localStorage + is the cold-start source; SQLite catches up via dual-write. A backfill + / one-time sync command can land later. + +## Known limitations + +- **The full Tauri build needs `cmake` + `clang-devel`** for + whisper-rs-sys. Not a regression; pre-existing infra dep. +- **State is still split** between localStorage (cache) and SQLite + (canonical). Dual-write resolves the consistency problem in the + short term. The eventual destination is SQLite-only with localStorage + as a transparent cache. + +## Files changed this sprint + +``` +crates/audio/Cargo.toml +crates/audio/src/capture.rs (rewrite + Day 2 hardening) +crates/audio/src/lib.rs +crates/storage/src/database.rs (+ FTS5, update, search, dictionary) +crates/storage/src/lib.rs +crates/storage/src/migrations.rs (+ migration v2) +src-tauri/src/commands/audio.rs (+ device picker, spawn_blocking, M3 fix) +src-tauri/src/commands/live.rs (+ microphoneDevice config field) +src-tauri/src/commands/mod.rs +src-tauri/src/commands/transcripts.rs (NEW — 10 Tauri commands) +src-tauri/src/lib.rs (+ Wayland, command registrations) +src/lib/components/ToastViewport.svelte (NEW) +src/lib/pages/DictationPage.svelte (+ device wiring, error toast) +src/lib/pages/HistoryPage.svelte (+ rename via update_transcript) +src/lib/pages/SettingsPage.svelte (+ Audio + Vocabulary panels) +src/lib/stores/page.svelte.js (+ microphoneDevice, dual-write) +src/lib/stores/toasts.svelte.js (NEW) +src/routes/+layout.svelte (+ ToastViewport mount) +``` + +## Next steps after dogfood + +1. Real-user feedback from one to three friends. What confuses them? + What feels slow? What did they expect that did not happen? +2. Address the deferred items in priority of feedback signal. +3. Consider opening up the `kon-public-beta` channel — a single + GitHub release with the auto-updater plumbed. +4. The architecture review's other items (frontend test coverage, + monolithic component split, hardcoded hex colours, ARIA gaps) + become the "open beta polish" sprint. + +--- + +*Compiled 2026/04/17 by Wren. Kon goes from "live transcription does not +work" to "ready to put in front of one trusted friend." Six commits, no +horrors so far.*