docs: architecture map (initial 5-slice generation, 105 pages)
Five-slice navigable map of the entire codebase under
docs/architecture-map/. Each slice is a self-contained
breadcrumbed sub-tree:
01-frontend (16) Svelte/SvelteKit UI
02-tauri-runtime (26) src-tauri commands + lifecycle
03-audio-transcription (16) audio + transcription crates
04-llm-formatting-mcp (19) llm, ai-formatting, mcp, cloud
05-core-storage-hotkey-build core, storage, hotkey, workspace,
(26) CI, dev glue
Plus master README.md and data-flow-end-to-end.md tracing
audio bytes from microphone to FTS5 search to MCP read.
Generated by 5 parallel subagents on 2026/05/09 against
HEAD 3c47000. Each page has YAML frontmatter, file:line code
refs, sibling cross-links, plain-English summaries.
Aggregated debt surfaced (full lists in master README):
RB-08 macOS power assertion, schema head drift v14 vs v15,
VAD blocked on ort version conflict, streaming primitives
not wired into live.rs, no prompt versioning, MCP has no
auth, cloud-providers in-memory keystore, SettingsPage
2 484 LOC, commands/live.rs 1 737 LOC, dual theme system,
brand rename to Lumenote pending across the codebase.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
68
docs/architecture-map/02-tauri-runtime/commands/README.md
Normal file
68
docs/architecture-map/02-tauri-runtime/commands/README.md
Normal file
@@ -0,0 +1,68 @@
|
||||
---
|
||||
name: Commands index
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# Commands index
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → Commands
|
||||
|
||||
**Plain English summary.** This is the registry of every Tauri command file. Each `.rs` file under `src-tauri/src/commands/` either declares one or more `#[tauri::command]` functions invoked from the frontend, or is a utility (state, security, power) that the command files share. The full list of commands actually exposed to the frontend is registered via the `tauri::generate_handler!` macro in `src-tauri/src/lib.rs:321`.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Path: `src-tauri/src/commands/`.
|
||||
- 22 command modules (declare `#[tauri::command]`s) plus 3 utility modules (`mod.rs`, `power.rs`, `security.rs`).
|
||||
- 71 `#[tauri::command]` functions registered in `lib.rs`. A handful of additional `#[tauri::command]` functions exist in source but are NOT registered (notably the small `clipboard::copy_to_clipboard` IS registered; nothing meaningful is unregistered as of last verification).
|
||||
- Total LOC across `commands/`: 6,939 (Rust source, excluding tests folder).
|
||||
|
||||
## Module list
|
||||
|
||||
| Module | LOC | Registered commands | One-liner |
|
||||
|---|---|---|---|
|
||||
| [audio](audio.md) | 540 | 4 | Native cpal mic capture, list devices, save WAV |
|
||||
| [clipboard](small-commands.md#clipboardrs) | 11 | 1 | Copy text to system clipboard via arboard |
|
||||
| [diagnostics](diagnostics.md) | 534 | 6 | Panic hook, frontend error log, crash files, diagnostic-report bundler, OS info |
|
||||
| [feedback](feedback.md) | 110 | 2 | HITL feedback capture and retrieval (Phase 2) |
|
||||
| [fs](small-commands.md#fsrs) | 44 | 1 | Write UTF-8 text to a user-chosen path (Phase 9) |
|
||||
| [hardware](small-commands.md#hardwarers) | 69 | 2 | Probe RAM/CPU/GPU, rank models for current hardware |
|
||||
| [hotkey](hotkey.md) | 105 | 5 | evdev hotkey listener bridge (Linux Wayland-compatible) |
|
||||
| [intentions](intentions.md) | 308 | 5 | Phase 7 implementation-intention rules CRUD |
|
||||
| [live](live.md) | 1,737 | 2 | Live transcription session orchestration (the big one) |
|
||||
| [llm](llm.md) | 423 | 10 | LLM model lifecycle, transcript cleanup, content tags |
|
||||
| [meeting](small-commands.md#meetingrs) | 50 | 1 | Process-list poll for known meeting apps (Phase 8) |
|
||||
| [models](models.md) | 691 | 13 | Whisper / Parakeet model registry, download, load, runtime capabilities |
|
||||
| [mod](mod.md) | 114 | n/a | `build_initial_prompt` helper + module re-exports |
|
||||
| [nudges](small-commands.md#nudgesrs) | 63 | 1 | Phase 6 nudge delivery via tauri-plugin-notification |
|
||||
| [paste](paste.md) | 790 | 3 | Auto-insert at cursor across Linux/macOS/Windows |
|
||||
| [power](power-and-security.md#powerrs) | 208 | 0 | macOS App Nap power assertion (no commands) |
|
||||
| [profiles](profiles.md) | 185 | 9 | Profile + profile-term CRUD, auto-learn from edits |
|
||||
| [rituals](small-commands.md#ritualsrs) | 43 | 2 | Morning triage last-shown sentinel (Phase 5) |
|
||||
| [security](power-and-security.md#securityrs) | 30 | 0 | `ensure_main_window` defence-in-depth helper |
|
||||
| [tasks](tasks.md) | 402 | 11 | Task CRUD, decompose-and-store, extract-from-transcript |
|
||||
| [transcription](transcription.md) | 413 | 3 | Transcribe PCM (Whisper / Parakeet), transcribe file |
|
||||
| [transcripts](transcripts.md) | 253 | 8 | Transcripts CRUD + FTS5 search + meta patches |
|
||||
| [tts](tts.md) | 444 | 3 | Read aloud via spd-say / say / PowerShell SAPI (Phase 4) |
|
||||
| [update](small-commands.md#updaters) | 16 | 2 | Updater placeholder (returns "disabled" until signing wired) |
|
||||
| [windows](windows.md) | 197 | 4 | Open / close the secondary windows (preview, viewer, task float) |
|
||||
|
||||
## Per-command page assignments
|
||||
|
||||
- Big files have their own page: [audio](audio.md), [live](live.md), [paste](paste.md), [models](models.md), [diagnostics](diagnostics.md), [llm](llm.md), [tts](tts.md), [transcription](transcription.md), [tasks](tasks.md), [transcripts](transcripts.md), [intentions](intentions.md), [profiles](profiles.md), [windows](windows.md), [hotkey](hotkey.md), [feedback](feedback.md).
|
||||
- Utilities collapsed into one page: [Power assertions and security](power-and-security.md) covers `power.rs` and `security.rs`.
|
||||
- Small modules grouped: [Small commands](small-commands.md) covers `clipboard.rs`, `fs.rs`, `hardware.rs`, `meeting.rs`, `nudges.rs`, `rituals.rs`, `update.rs`.
|
||||
- Module entry: [mod.md](mod.md) covers `mod.rs` (re-exports + `build_initial_prompt` helper).
|
||||
|
||||
## How to navigate
|
||||
|
||||
- Looking for a frontend `invoke('foo', ...)` call? Search for `pub async fn foo` or `pub fn foo` under `commands/`. The page for that file lists all its commands with arg + return signatures.
|
||||
- Looking for an event the frontend listens for? See the slice README's "events emitted" section, or grep for the event name across `commands/`.
|
||||
- Looking for ACL gotchas? Pages flag the commands that include an `ensure_main_window` guard.
|
||||
|
||||
## See also
|
||||
|
||||
- [Slice README](../README.md) — back to the top of slice 02.
|
||||
- [App lifecycle](../app-lifecycle.md) — `lib.rs::run` is what registers everything listed here.
|
||||
- [Capabilities and ACL](../capabilities-and-acl.md) — the permission set that gates every invocation.
|
||||
103
docs/architecture-map/02-tauri-runtime/commands/audio.md
Normal file
103
docs/architecture-map/02-tauri-runtime/commands/audio.md
Normal file
@@ -0,0 +1,103 @@
|
||||
---
|
||||
name: Audio capture commands
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# `commands::audio`
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → Audio
|
||||
|
||||
**Plain English summary.** Owns the native cpal microphone capture path used outside live transcription. Exposes "list input devices", "start native capture", "stop native capture", and "save audio to WAV". Streams 16 kHz mono PCM chunks to the frontend via the `native-pcm` event. Also owns the deterministic recording-filename generator and the helper that resolves a destination WAV path so the live-transcription module can share it.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Path: `src-tauri/src/commands/audio.rs`.
|
||||
- LOC: 540.
|
||||
- Tauri commands exposed:
|
||||
- `list_audio_devices(window: WebviewWindow) -> Result<Vec<DeviceInfo>, String>` — main-window only. Wraps `MicrophoneCapture::list_devices` in `spawn_blocking`.
|
||||
- `start_native_capture(window, app, state, device_name: Option<String>) -> Result<(), String>` — main-window only. Starts a cpal stream, downsamples to 16 kHz mono, streams chunks via `native-pcm` events, also writes to a temp WAV.
|
||||
- `stop_native_capture(window, state) -> Result<Vec<f32>, String>` — main-window only. Awaits the worker join barrier (RB-06) and returns the accumulated samples.
|
||||
- `save_audio(window, app, samples, output_folder: Option<String>) -> Result<String, String>` — main-window only. Writes a fresh WAV to `app_local_data_dir/recordings/` (or a user-chosen folder) and returns its absolute path.
|
||||
- Events emitted: `native-pcm` (payload `{ samples: Vec<f32> }`, ~0.5 s of 16 kHz mono per emit) — `src-tauri/src/commands/audio.rs:249` and `:274`.
|
||||
- Depends on: `magnotia_audio::{DeviceInfo, MicrophoneCapture, WavWriter, write_wav}`, `magnotia_core::types::AudioSamples`, `magnotia_core::constants::WHISPER_SAMPLE_RATE`. Plus `tokio::{mpsc, Mutex, JoinHandle, spawn_blocking, sleep}` and `std::sync::{Arc, Mutex, atomic::AtomicBool}`.
|
||||
- Called from frontend at: dictation page (start / stop), Settings → Audio (device list), file-import flow (save_audio).
|
||||
|
||||
## What's in here
|
||||
|
||||
### Constants
|
||||
|
||||
- `MAX_NATIVE_CAPTURE_RETURN_SAMPLES = WHISPER_SAMPLE_RATE * 60 * 10` (`src-tauri/src/commands/audio.rs:14`). 10 minutes of 16 kHz mono. Caps the in-memory return buffer; the temp WAV on disk stays authoritative beyond this.
|
||||
|
||||
### `CaptureWorker` (`src-tauri/src/commands/audio.rs:34`)
|
||||
|
||||
Internal struct holding `stop_tx: tokio_mpsc::Sender<()>` and `join: JoinHandle<()>`. `stop_worker(worker).await` (`src-tauri/src/commands/audio.rs:44`) sends the stop signal and awaits the join. Consumes the worker because `stop_tx` and `join` are single-use. Errors from the join are logged and swallowed.
|
||||
|
||||
### `NativeCaptureState` (`src-tauri/src/commands/audio.rs:53`)
|
||||
|
||||
Tauri-managed state. Fields:
|
||||
|
||||
- `worker: AsyncMutex<Option<CaptureWorker>>` — `tokio::sync::Mutex` because `stop_worker` awaits while holding the lock.
|
||||
- `all_samples: Arc<Mutex<Vec<f32>>>` — capped compatibility buffer returned by `stop_native_capture`.
|
||||
- `wav_writer: Arc<Mutex<Option<WavWriter>>>` — temp WAV writer, finalised when the worker exits.
|
||||
- `temp_audio_path: Arc<Mutex<Option<PathBuf>>>` — destination for the temp WAV.
|
||||
- `capture_truncated: Arc<AtomicBool>` — set when `all_samples` was capped (the temp WAV still has the full capture).
|
||||
|
||||
### `append_recorded_chunk` (`src-tauri/src/commands/audio.rs:79`)
|
||||
|
||||
Helper that pushes a downsampled chunk into both the WAV writer and the in-memory buffer, respecting the buffer cap and flipping the `capture_truncated` flag when it bumps the ceiling.
|
||||
|
||||
### `start_native_capture` (`src-tauri/src/commands/audio.rs:113`)
|
||||
|
||||
Step-by-step:
|
||||
|
||||
1. `ensure_main_window(&window)` — secondary windows can't start a capture.
|
||||
2. Stop any in-flight worker and `await` its termination (RB-06 — without the join, `all_samples.clear()` below would race a draining worker and a rapid start→stop→start could leak old-session samples into the new session).
|
||||
3. Call `MicrophoneCapture::start` (or `start_with_device(name)`) inside `spawn_blocking` — the underlying cpal probe spends up to `DEVICE_VALIDATION_MS * N` per pass and would block the async runtime otherwise (Codex review 2026/04/17 D2).
|
||||
4. Open a temp WAV via `WavWriter::create(&temp_path, WHISPER_SAMPLE_RATE, 1)`.
|
||||
5. Spawn the worker. The worker loop:
|
||||
- Polls the stop channel non-blocking.
|
||||
- Drains `cpal::AudioChunk`s from the capture mpsc, distinguishing `Empty` (try again) from `Disconnected` (stream is dead, exit) — Codex review 2026/04/17 M3.
|
||||
- Downmixes to mono if the source is stereo (`chunk.samples.chunks(channels).map(|frame| sum / channels)`).
|
||||
- Decimates to 16 kHz (simple ratio-based decimation, matches the frontend pcm-processor.js pattern).
|
||||
- When the buffer ≥ 8000 samples (~0.5 s), drains, calls `append_recorded_chunk`, and emits `native-pcm`.
|
||||
- On exit, flushes any remaining samples, drops the cpal capture, finalises the WAV.
|
||||
6. Stash the worker in `state.worker`.
|
||||
|
||||
### `stop_native_capture` (`src-tauri/src/commands/audio.rs:306`)
|
||||
|
||||
Takes the worker out of state, calls `stop_worker(worker).await`, then `mem::take`s `all_samples` and returns it. Logs a warning if the buffer was truncated.
|
||||
|
||||
### `resolve_recording_path` (`src-tauri/src/commands/audio.rs:333`)
|
||||
|
||||
Public helper used by `commands::live::start_live_transcription_session`. Resolves the recordings dir (user-supplied folder if non-empty, else `app_local_data_dir/recordings/`), creates it, and returns `dir.join(recording_filename())`.
|
||||
|
||||
### `recording_filename` (`src-tauri/src/commands/audio.rs:365`)
|
||||
|
||||
Deterministic filename: `magnotia-<unix_secs>-<nanos:09>-<counter:04>.wav`. The counter is a process-lifetime `AtomicU64` (`RECORDING_COUNTER`) bumped on each call. The combination defeats wall-clock collisions both across launches (secs change) and within the same nanosecond (counter changes).
|
||||
|
||||
### `persist_audio_samples` and `save_audio` (`src-tauri/src/commands/audio.rs:512`, `:531`)
|
||||
|
||||
`save_audio` is the public command. Calls `persist_audio_samples`, which resolves the recording path, then calls `magnotia_audio::write_wav` inside `spawn_blocking`. Returns the absolute path as a string.
|
||||
|
||||
## Data flow
|
||||
|
||||
- **Start path:** `start_native_capture(deviceName)` → cpal opens stream → worker downmixes/downsamples → emits `native-pcm` events at 0.5 s cadence → frontend dictation page accumulates samples or feeds them to a transcription command.
|
||||
- **Stop path:** frontend invokes `stop_native_capture` → worker stop channel signalled → join awaited → samples returned (capped at 10 minutes) → frontend can request `save_audio` to persist.
|
||||
- **Live transcription path:** `commands::live` does not use `start_native_capture` directly; it owns its own `MicrophoneCapture::start` invocation but reuses `resolve_recording_path` and `recording_filename` from this file.
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- The in-memory buffer is capped at 10 minutes. Anything longer relies on the temp WAV; the frontend cannot just `stop_native_capture` and treat the returned vec as ground truth for long sessions. Today's UI assumes short captures, but the cap should be surfaced if you build a long-form recorder.
|
||||
- The downsampler is naive decimation. Acceptable for speech but not for music. If you ever extend Magnotia to general audio, swap in `magnotia_audio::StreamingResampler` (which is what `commands::live` uses).
|
||||
- `start_native_capture` does *not* engage `PowerAssertion` (App Nap pinning). `commands::live::run_live_session` does. If you build a long-form non-live recorder on top of this, add the assertion.
|
||||
- The worker emits `native-pcm` to the entire app handle (not a specific window). Every webview that listens will receive the chunks. If you ever route audio to a non-main window, double-check this is what you want.
|
||||
- `stop_worker_awaits_full_termination_no_writes_after_join` (`src-tauri/src/commands/audio.rs:454`) is the regression test for RB-06. Do not change `stop_worker` to a fire-and-forget pattern.
|
||||
|
||||
## See also
|
||||
|
||||
- [Live transcription](live.md) — uses `resolve_recording_path` and `recording_filename` from this file.
|
||||
- [Transcription](transcription.md) — receives `Vec<f32>` from `stop_native_capture` and runs Whisper / Parakeet on it.
|
||||
- [Models](models.md) — `ensure_model_loaded` is invoked by the transcription commands before they consume audio.
|
||||
- [Cargo and features](../cargo-and-features.md) — `arboard` and the audio crate dependency live there.
|
||||
111
docs/architecture-map/02-tauri-runtime/commands/diagnostics.md
Normal file
111
docs/architecture-map/02-tauri-runtime/commands/diagnostics.md
Normal file
@@ -0,0 +1,111 @@
|
||||
---
|
||||
name: Diagnostics and reports
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# `commands::diagnostics`
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → Diagnostics
|
||||
|
||||
**Plain English summary.** The local-only diagnostic plumbing. Installs the Rust panic hook (writes per-panic dumps to `crashes_dir`). Captures frontend errors via a global `window.onerror` handler. Lists recent error-log rows and crash files. Bundles a markdown diagnostic report (settings, recent errors, active power assertions, crash dumps, log tail) so the user can paste it into an email or GitHub issue. Privacy posture: nothing is transmitted; the user reviews exactly what would be shared before sharing it.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Path: `src-tauri/src/commands/diagnostics.rs`.
|
||||
- LOC: 534.
|
||||
- Tauri commands exposed (6 total):
|
||||
- `log_frontend_error(state, context, message, stack: Option<String>) -> Result<(), String>`. No window guard — fires from any window's global error handler.
|
||||
- `list_recent_errors_command(window, state, limit) -> Result<Vec<ErrorLogDto>, String>`. Main-window only.
|
||||
- `list_crash_files(window) -> Result<Vec<CrashFile>, String>`. Main-window only.
|
||||
- `generate_diagnostic_report(window, state, options: Option<ReportOptions>) -> Result<String, String>`. Main-window only.
|
||||
- `save_diagnostic_report(window, app, state, options) -> Result<String, String>`. Main-window only. Returns the absolute file path it wrote.
|
||||
- `get_os_info() -> OsInfo`.
|
||||
- Public Rust helper used by `lib.rs::run`: `install_panic_hook()`.
|
||||
- Events emitted: none.
|
||||
- Depends on: `magnotia_storage::{app_data_dir, crashes_dir, list_recent_errors, log_error, logs_dir, ErrorLogRow}`, `commands::power::active_assertions_snapshot`, `commands::security::ensure_main_window`. Plus `std::fs`, `std::panic`, `std::time`.
|
||||
- Called from frontend at: global `window.onerror` / `window.unhandledrejection` (`log_frontend_error`); Settings → About → Diagnostics (the bundle commands and `list_*` commands); top-of-app Cmd-vs-Ctrl labelling (`get_os_info`).
|
||||
|
||||
## What's in here
|
||||
|
||||
### Constants (`src-tauri/src/commands/diagnostics.rs:26`)
|
||||
|
||||
- `DEFAULT_RECENT_ERRORS = 50`.
|
||||
- `MAGNOTIA_VERSION = env!("CARGO_PKG_VERSION")`.
|
||||
|
||||
### `install_panic_hook` (`:34`)
|
||||
|
||||
Creates `crashes_dir()` if missing. Replaces the default panic hook with one that writes a minimal text dump to `crashes_dir/<unix_secs>-<short_hash>.crash` and then chains to the original hook so the panic still hits stderr. Dump includes version, timestamp, thread name, panic message, OS, arch, RUST_BACKTRACE env. Backtraces require `RUST_BACKTRACE=1` and `std::backtrace` plumbing — neither set up today, so production crashes will lack stack traces.
|
||||
|
||||
### `log_frontend_error` (`:86`)
|
||||
|
||||
Truncates the stack to 2,000 chars and calls `magnotia_storage::log_error` with context `"frontend"` (or the caller-supplied context), error code `"FRONTEND_ERROR"`, the message, and the stack as metadata.
|
||||
|
||||
### `ErrorLogDto` (`:114`) and `list_recent_errors_command` (`:138`)
|
||||
|
||||
`ErrorLogDto` is a camelCase shape over `ErrorLogRow`. The list command clamps `limit` to `[1, 1000]` and defaults to 50.
|
||||
|
||||
### `CrashFile` and `list_crash_files` (`:152`, `:162`)
|
||||
|
||||
`CrashFile` exposes filename, mtime_secs, size_bytes, plus a 600-char preview. `list_crash_files_inner` reads the crashes directory, builds the structs, sorts by mtime descending.
|
||||
|
||||
### Report options and assembler
|
||||
|
||||
- `ReportOptions` has four boolean flags: `include_settings`, `include_recent_errors`, `include_crashes`, `include_log_tail` (each defaulting true).
|
||||
- `redact_home(input)` replaces `home_dir().display()` substrings with `~`.
|
||||
- `looks_sensitive_key(key)` flags keys containing `apikey` / `token` / `secret` / `password` / `bearer`.
|
||||
- `redact_json_value(value, key_hint)` recursively walks a JSON value redacting strings under sensitive keys.
|
||||
- `sanitise_preferences_json(raw)` parses, redacts, re-emits.
|
||||
- `redact_line(input)` does line-level redaction for the error-message field.
|
||||
|
||||
### `generate_diagnostic_report_inner` (`:307`)
|
||||
|
||||
Builds a markdown document with sections:
|
||||
|
||||
1. Header (version, OS / arch, app data dir, generated timestamp, "local-only until you choose to share" notice).
|
||||
2. Settings (sanitised JSON via `sanitise_preferences_json`).
|
||||
3. Recent errors (50 rows, redacted).
|
||||
4. Power assertions (snapshots from `commands::power::active_assertions_snapshot`).
|
||||
5. Crash dumps (up to 5 most-recent, plus a count of older ones).
|
||||
6. Log tail (8 KB tail of `logs_dir/magnotia.log`, home-redacted).
|
||||
|
||||
### `save_diagnostic_report` (`:513`)
|
||||
|
||||
Generates the report, then writes it to `app_data_dir/diagnostic-reports/magnotia-diagnostic-<unix_secs>.md`. Returns the absolute path.
|
||||
|
||||
### `OsInfo` and `get_os_info` (`:449`, `:474`)
|
||||
|
||||
OS family label, arch, `uses_cmd` boolean (macOS only), `is_wayland` boolean (Linux only), `custom_hotkey_backend` (Linux only — true because evdev is the primary path on Linux), and `primary_modifier_label` (`"Cmd"` on macOS, `"Ctrl"` elsewhere). Frontend reads this once at boot to set the hotkey labels and "Open file location" terminology.
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
panic occurs -> install_panic_hook writes crashes_dir/<ts>.crash
|
||||
window.onerror fires in JS -> invoke('log_frontend_error', context, message, stack)
|
||||
-> magnotia_storage::log_error
|
||||
-> error_log table
|
||||
|
||||
Settings -> About -> Diagnostics
|
||||
list_crash_files -> [CrashFile, ...]
|
||||
list_recent_errors_command -> [ErrorLogDto, ...]
|
||||
generate_diagnostic_report(opts) -> markdown string
|
||||
user reviews, then optionally:
|
||||
save_diagnostic_report(opts) -> absolute path
|
||||
```
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **No backtraces in production crashes.** `RUST_BACKTRACE` is not set in the packaged binary. Either set it in a launcher script or call `std::backtrace::Backtrace::capture()` inside the panic hook.
|
||||
- **`log_frontend_error` is unguarded by ACL or window-label check.** Any window can log a frontend error. That is fine, but the storage layer should enforce a row-rate limiter before this surface ever ships to users with adversarial workloads.
|
||||
- **Crash file prefixes are unix-seconds-and-a-16-bit-hash.** Two panics in the same second with the same low-16-bits of seconds will collide. Vanishingly unlikely in practice. If you want bulletproof, use the same counter pattern as `commands::audio::recording_filename`.
|
||||
- **Diagnostic report is markdown by design** so it can be pasted anywhere. If you ever want a structured upload format, keep this command and add a sibling that returns JSON.
|
||||
- **Sensitive-key redaction is a heuristic.** It catches the obvious names (`api_key`, `apiKey`, `token`, `Bearer`) but won't catch a user-named key like `mySecretToken: "abc"` if the key string is normalised to lowercase first. Fine for "user reviews before sharing"; not fine if you ever auto-upload.
|
||||
- **`ensure_main_window` is missing on `log_frontend_error` and `get_os_info`.** Intentional — both should be callable from any window. But document this so future tightening attempts know to leave them alone.
|
||||
|
||||
## See also
|
||||
|
||||
- [App lifecycle](../app-lifecycle.md) — `install_panic_hook` is called from setup.
|
||||
- [Power assertions and security](power-and-security.md) — `active_assertions_snapshot` feeds the report.
|
||||
- [Capabilities and ACL](../capabilities-and-acl.md) — the report is local until the user chooses to share.
|
||||
- [Tests](../tests.md) — no specific test for the report itself, but the redactor helpers could use unit coverage.
|
||||
78
docs/architecture-map/02-tauri-runtime/commands/feedback.md
Normal file
78
docs/architecture-map/02-tauri-runtime/commands/feedback.md
Normal file
@@ -0,0 +1,78 @@
|
||||
---
|
||||
name: HITL feedback
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# `commands::feedback`
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → Feedback
|
||||
|
||||
**Plain English summary.** Phase 2: thumbs + correction capture on AI-generated output (microsteps from a task decomposition, task lines extracted from a transcript, or LLM cleanup). The captured rows feed a few-shot loop: subsequent prompts are conditioned on the user's preferred style by injecting the (input, preferred-output) pairs as exemplars.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Path: `src-tauri/src/commands/feedback.rs`.
|
||||
- LOC: 110.
|
||||
- Tauri commands exposed:
|
||||
- `record_feedback(state, input: RecordFeedbackInput) -> Result<i64, String>` — returns the new row id.
|
||||
- `list_feedback_examples_cmd(state, target_type, limit, min_rating, profile_id) -> Result<Vec<FeedbackDto>, String>`.
|
||||
- Events emitted: none.
|
||||
- Depends on: `magnotia_storage::{record_feedback, list_feedback_examples, FeedbackRow, FeedbackTargetType, RecordFeedbackParams}`.
|
||||
- Called from frontend at: dictation result panel (thumb up/down + correction-text on cleanup); Tasks page (thumb on extracted tasks and decomposed microsteps).
|
||||
|
||||
## What's in here
|
||||
|
||||
### `RecordFeedbackInput` (`src-tauri/src/commands/feedback.rs:15`)
|
||||
|
||||
Frontend-supplied shape:
|
||||
|
||||
- `targetType`: `"microstep" | "task_extraction" | "cleanup"`. Parsed via `FeedbackTargetType::parse`.
|
||||
- `targetId`: optional surface-specific id (subtask id, task id, transcript id).
|
||||
- `rating`: `-1` (thumbs down), `0` (correction, neutral), `+1` (thumbs up).
|
||||
- `originalText`: the AI-generated text the user is rating.
|
||||
- `correctedText`: the user's preferred text (when they corrected it).
|
||||
- `contextJson`: freeform JSON used by the prompt builder later to reconstruct the (input, preferred-output) pair.
|
||||
- `profileId`: scopes the row.
|
||||
|
||||
### `FeedbackDto` (`:38`)
|
||||
|
||||
camelCase mirror of `FeedbackRow`. Note `rating` widens to `i64` in the DTO (storage uses `i64`).
|
||||
|
||||
### `parse_target_type` (`:68`)
|
||||
|
||||
Wraps `FeedbackTargetType::parse(raw)`, returning `"unknown feedback target_type: <raw>"` on miss.
|
||||
|
||||
### `record_feedback` (`:73`)
|
||||
|
||||
`parse_target_type` then `db_record_feedback`. Returns the row id.
|
||||
|
||||
### `list_feedback_examples_cmd` (`:95`)
|
||||
|
||||
Clamps `limit` to `[1, 64]`, defaults 8. Clamps `min_rating` to `[-1, 1]`, default 0. Calls `db_list_feedback_examples`. Returns `FeedbackDto`s. Used by the `commands::tasks` few-shot exemplar pull and would be used by the equivalent in `commands::llm` if/when cleanup gets its own exemplar path.
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
dictation result thumbs-up -> invoke('record_feedback', { targetType: 'cleanup', rating: +1, originalText, correctedText, contextJson, profileId })
|
||||
-> magnotia_storage::record_feedback -> row id
|
||||
|
||||
decomposition thumbs-down + correction -> record_feedback({ targetType: 'microstep', rating: 0, originalText, correctedText: "user's preferred wording", contextJson: {parent_text}, profileId })
|
||||
|
||||
next decompose call -> list_feedback_examples_cmd('microstep', 5, 0, profile_id)
|
||||
-> [FeedbackDto, ...] -> few-shot exemplars
|
||||
```
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **No `ensure_main_window` guard.** Tasks float and History viewer secondary windows can also fire feedback. Intentional. If you ever want to lock down feedback writes, this is where to add the guard.
|
||||
- **`contextJson` is freeform.** Storage stores the raw string. `commands::tasks::to_llm_examples` parses it and skips rows that are malformed. Bad data therefore degrades gracefully but doesn't surface to the user. The `eprintln!` in `to_llm_examples` is the only visibility.
|
||||
- **`min_rating` clamp is `[-1, 1]`.** Pass `1` to get only thumbs-up examples, `0` for thumbs-up + corrections, `-1` for everything. The default of `0` is what `commands::tasks` picks.
|
||||
- **No deduplication.** A user thumbs-upping the same output twice creates two rows. The exemplar trim in `commands::tasks` does not dedupe by `originalText`. If two identical exemplars steal slots, that's just lost prompt budget.
|
||||
|
||||
## See also
|
||||
|
||||
- [Tasks](tasks.md) — the consumer of `list_feedback_examples_cmd`.
|
||||
- [LLM](llm.md) — the cleanup path that produces the text that thumbs-up/down feedback rates.
|
||||
- [Profiles](profiles.md) — `profileId` is the scoping key.
|
||||
87
docs/architecture-map/02-tauri-runtime/commands/hotkey.md
Normal file
87
docs/architecture-map/02-tauri-runtime/commands/hotkey.md
Normal file
@@ -0,0 +1,87 @@
|
||||
---
|
||||
name: Hotkey bridge
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# `commands::hotkey`
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → Hotkey bridge
|
||||
|
||||
**Plain English summary.** The Linux Wayland-compatible global hotkey backend. Tauri's `tauri-plugin-global-shortcut` works on macOS / Windows and on X11 Linux but fails silently on Wayland (the protocol forbids unprivileged keystroke grabs). Magnotia's bespoke evdev backend reads `/dev/input/event*` directly, parses key combos, and emits `magnotia:hotkey-pressed` / `magnotia:hotkey-released` events. Settings on Linux uses these commands instead of the global-shortcut plugin; everywhere else the plugin path is canonical.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Path: `src-tauri/src/commands/hotkey.rs`.
|
||||
- LOC: 105.
|
||||
- Tauri commands exposed (5 total):
|
||||
- `is_wayland_session() -> bool`. Pure env probe.
|
||||
- `check_hotkey_access() -> Result<(), String>`. Probes evdev access (user in `input` group, etc.).
|
||||
- `start_evdev_hotkey(app, state, hotkey: String) -> Result<(), String>`. Parses the Tauri-style combo string, stops any existing listener, starts a new one, spawns a forwarder task that converts evdev events to Tauri events.
|
||||
- `update_evdev_hotkey(state, hotkey: String) -> Result<(), String>`. Updates the combo on a running listener.
|
||||
- `stop_evdev_hotkey(state) -> Result<(), String>`. Stops cleanly.
|
||||
- Events emitted: `magnotia:hotkey-pressed` (no payload), `magnotia:hotkey-released` (no payload). Fired from the forwarder task in `start_evdev_hotkey` (`src-tauri/src/commands/hotkey.rs:67`, `:70`).
|
||||
- Depends on: `magnotia_hotkey::{EvdevHotkeyListener, HotkeyCombo, HotkeyEvent, check_evdev_access}`. Plus `tokio::sync::{mpsc, Mutex}`.
|
||||
- Called from frontend at: Settings → Hotkey on Linux. Other platforms call the `tauri-plugin-global-shortcut` plugin's JS API directly.
|
||||
|
||||
## What's in here
|
||||
|
||||
### `HotkeyState` (`src-tauri/src/commands/hotkey.rs:9`)
|
||||
|
||||
Tauri-managed: `listener: Arc<Mutex<Option<EvdevHotkeyListener>>>`. `tokio::sync::Mutex` because operations await across `.lock()`.
|
||||
|
||||
### `is_wayland_session` (`:23`)
|
||||
|
||||
Returns true if `WAYLAND_DISPLAY` is set or `XDG_SESSION_TYPE=wayland`. Used by Settings to decide whether to display the Linux-only hotkey UI.
|
||||
|
||||
### `check_hotkey_access` (`:32`)
|
||||
|
||||
Defers to `magnotia_hotkey::check_evdev_access` — that helper checks the user can read `/dev/input/event*` (typically requires being in the `input` group, or a udev rule). On failure, returns the actionable error string.
|
||||
|
||||
### `start_evdev_hotkey` (`:41`)
|
||||
|
||||
1. Parse the hotkey string (e.g. `"Shift+Cmd+Space"`) via `HotkeyCombo::from_tauri_str`. Errors become `"Cannot parse hotkey: ..."`.
|
||||
2. Lock the listener mutex; if a listener is already running, stop it and await termination.
|
||||
3. Build a 64-deep `tokio::sync::mpsc` channel for `HotkeyEvent`s.
|
||||
4. Call `EvdevHotkeyListener::start(combo, event_tx)`, stash the listener.
|
||||
5. Spawn a `tokio::spawn` forwarder that reads from the event_rx and emits `magnotia:hotkey-pressed` / `magnotia:hotkey-released` per event variant.
|
||||
|
||||
### `update_evdev_hotkey` (`:81`)
|
||||
|
||||
Re-parses the combo and calls `listener.set_hotkey(combo)` if a listener is running. Returns `"Hotkey listener not running"` otherwise. Avoids the cost of stopping and restarting the evdev thread.
|
||||
|
||||
### `stop_evdev_hotkey` (`:99`)
|
||||
|
||||
Take the listener out, call `listener.stop().await`. Idempotent — calling stop with no listener returns Ok silently.
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
Settings -> is_wayland_session() -> bool
|
||||
Settings -> check_hotkey_access() -> Ok or "join the input group / install udev rule"
|
||||
Settings -> start_evdev_hotkey("Shift+Cmd+Space")
|
||||
-> parse combo
|
||||
-> stop any prior listener
|
||||
-> EvdevHotkeyListener spawns its own thread reading /dev/input/event*
|
||||
-> forwarder spawn: HotkeyEvent -> Tauri event
|
||||
frontend listens on 'magnotia:hotkey-pressed' / '...-released' and starts/stops dictation
|
||||
Settings -> update_evdev_hotkey("Ctrl+Alt+M") (when user changes binding)
|
||||
Settings -> stop_evdev_hotkey() (when user disables)
|
||||
```
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **No `ensure_main_window` guard.** Settings is in the main window. If you ever expose hotkey re-binding in a secondary window, add the guard.
|
||||
- **Linux only.** macOS and Windows code paths in the frontend talk to `tauri-plugin-global-shortcut` directly — no Rust commands needed because the plugin handles register / unregister via JS. This module compiles and runs on every desktop OS but only *works* on Linux because evdev is Linux-specific (the workspace crate `magnotia_hotkey` has the platform shim).
|
||||
- **`/dev/input/event*` access requires either:** (a) the user is in the `input` group, or (b) a udev rule grants Magnotia explicit access. The Settings UI walks the user through option (a) on first run; option (b) is documented in `docs/dev-setup.md` for power users.
|
||||
- **Channel size of 64** is enough for a key smash burst. If a worker stalls and the channel fills, evdev events would be dropped silently. Consider logging a warning when the channel is at capacity.
|
||||
- **The forwarder spawn is detached.** No way to stop the spawn task explicitly; it exits when `event_rx.recv().await` returns None (which happens when the listener is dropped). Acceptable.
|
||||
- **No power assertion.** The evdev listener thread is pinned by the kernel via the file handle; idle CPU is near zero.
|
||||
|
||||
## See also
|
||||
|
||||
- [App lifecycle](../app-lifecycle.md) — `HotkeyState::new()` is stashed in setup.
|
||||
- [Diagnostics](diagnostics.md) — `OsInfo.custom_hotkey_backend` flags Linux as the evdev path.
|
||||
- [Paste](paste.md) — the consumer of "user pressed dictation hotkey": holds focus on the target window for the eventual paste.
|
||||
- [Live transcription](live.md) — the dictation flow that the hotkey starts.
|
||||
@@ -0,0 +1,90 @@
|
||||
---
|
||||
name: Implementation intentions
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# `commands::intentions`
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → Implementation intentions
|
||||
|
||||
**Plain English summary.** Phase 7. Tiny "if-then" automation rules: "at 09:00 each day, surface my Inbox", "when the morning triage finishes, start a 5-minute timer", "when task X is completed, speak 'Nice'". Frontend owns scheduling and execution because v1 triggers are app-local events plus a local clock check; Rust owns durable storage, validation, and a main-window-only firewall.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Path: `src-tauri/src/commands/intentions.rs`.
|
||||
- LOC: 308.
|
||||
- Tauri commands exposed (5 total), all main-window only:
|
||||
- `list_implementation_rules(window, state) -> Result<Vec<ImplementationRuleDto>, String>`.
|
||||
- `create_implementation_rule(window, state, request: CreateImplementationRuleRequest) -> Result<ImplementationRuleDto, String>`.
|
||||
- `set_implementation_rule_enabled(window, state, id, enabled) -> Result<ImplementationRuleDto, String>`.
|
||||
- `mark_implementation_rule_fired(window, state, id, fired_key) -> Result<ImplementationRuleDto, String>`.
|
||||
- `delete_implementation_rule(window, state, id) -> Result<(), String>`.
|
||||
- Events emitted: none.
|
||||
- Depends on: `magnotia_storage::{insert_implementation_rule, list_implementation_rules, set_implementation_rule_enabled, mark_implementation_rule_fired, delete_implementation_rule, get_task_by_id, ImplementationRuleRow}`, `uuid::Uuid`, `serde_json`. Plus `commands::security::ensure_main_window`.
|
||||
- Called from frontend at: Settings → Implementation intentions (CRUD); the rule-runner module (`mark_implementation_rule_fired` after firing).
|
||||
|
||||
## What's in here
|
||||
|
||||
### Trigger and action enums (`src-tauri/src/commands/intentions.rs:21`)
|
||||
|
||||
- `RuleTriggerKind`: `TimeOfDay`, `TaskCompleted`, `MorningTriageFinished`. Serialises as `"time_of_day"`, `"task_completed"`, `"morning_triage_finished"`.
|
||||
- `SurfaceTarget`: `Inbox`, `Today`, `Tasks`, `Task`. The `Task` variant requires a `taskId`.
|
||||
- `RuleAction` (tagged enum, `kind` discriminator): `Surface { target, taskId, label }`, `StartTimer { seconds, taskId, label }`, `SpeakLine { text }`.
|
||||
|
||||
### `CreateImplementationRuleRequest` (`:72`)
|
||||
|
||||
`triggerKind`, `triggerValue: String`, `actions: Vec<RuleAction>`, `enabled` (default true), `lastFiredKey: Option<String>`.
|
||||
|
||||
### `ImplementationRuleDto` (`:88`) and `TryFrom` (`:100`)
|
||||
|
||||
The DTO unpacks the storage row's `actions_json` back into typed `Vec<RuleAction>`. Returns Err on unknown `trigger_kind` strings or invalid JSON — surfaces drift instead of silently swallowing.
|
||||
|
||||
### Validators
|
||||
|
||||
- `validate_hhmm(value)` (`:125`) — strict HH:MM, two digits each, hours 0..=23, minutes 0..=59. Tested.
|
||||
- `validate_actions(state, actions)` (`:144`):
|
||||
- At least one action.
|
||||
- `Surface { target: Task }` requires a `taskId` and the task must exist (via `get_task_by_id`).
|
||||
- `StartTimer` rejects anything other than 300 seconds (v1 fixed at 5 minutes).
|
||||
- `SpeakLine` requires non-empty text and ≤ 240 chars.
|
||||
- `validate_request(state, request)` (`:192`) — checks the trigger value matches the trigger kind (`TimeOfDay` requires HH:MM, the others reject any non-empty trigger_value).
|
||||
|
||||
### Commands (`:207` onwards)
|
||||
|
||||
All five commands `ensure_main_window`. CRUD is otherwise straight wrapping. `create_implementation_rule` generates a UUIDv4, serialises actions to JSON, normalises `trigger_value` (empty for non-time triggers), inserts, and returns the freshly-fetched DTO. `mark_implementation_rule_fired` requires a non-empty `fired_key` (the frontend uses ISO date strings to dedupe daily fires).
|
||||
|
||||
### Tests (`:296`)
|
||||
|
||||
`hhmm_validation_accepts_and_rejects_expected_shapes` covers a handful of valid and invalid clock strings.
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
Settings -> create_implementation_rule(req)
|
||||
-> ensure_main_window
|
||||
-> validate_request (HH:MM if applicable, action shape, task existence for surface-task)
|
||||
-> uuid::v4 for id
|
||||
-> serde_json serialise actions
|
||||
-> magnotia_storage::insert_implementation_rule
|
||||
-> list/return DTO
|
||||
|
||||
frontend rule-runner cron -> evaluates triggers in JS
|
||||
-> when a rule fires: invoke('mark_implementation_rule_fired', id, "2026-05-09")
|
||||
-> stamp the dedupe key in DB
|
||||
```
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **Action timer is hard-capped to 300 seconds.** The frontend has to surface this constraint. Future versions will need to widen the validator to accept arbitrary durations.
|
||||
- **Rule execution lives entirely in the frontend.** If the user closes Magnotia at 08:55 with a 09:00 rule, the rule does not fire. There is no daemon. This is documented in the module header.
|
||||
- **Validation is sync against the DB.** `validate_actions` calls `get_task_by_id` for every surface-task action. Multiple actions in one rule = multiple round-trips. Acceptable today; consider batching if a rule grows huge.
|
||||
- **`mark_implementation_rule_fired` requires a non-empty `fired_key`.** The frontend should always pass an ISO date for daily rules and a per-event key for task-completed rules. Empty key = command rejection.
|
||||
- **No `PowerAssertion`.** None of these actions run inference. No need.
|
||||
|
||||
## See also
|
||||
|
||||
- [Tasks](tasks.md) — `Surface { target: Task }` and `StartTimer { taskId }` reference task ids.
|
||||
- [TTS](tts.md) — `SpeakLine` actions are dispatched by the frontend through `tts_speak`.
|
||||
- [Capabilities and ACL](../capabilities-and-acl.md) — main-window-only is the firewall here.
|
||||
185
docs/architecture-map/02-tauri-runtime/commands/live.md
Normal file
185
docs/architecture-map/02-tauri-runtime/commands/live.md
Normal file
@@ -0,0 +1,185 @@
|
||||
---
|
||||
name: Live transcription session
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# `commands::live`
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → Live transcription
|
||||
|
||||
**Plain English summary.** The 1,737-line beast that runs a live dictation session end-to-end. Captures audio via a dedicated `MicrophoneCapture`, streams chunks through a `StreamingResampler`, runs a speech gate to skip near-silent chunks, dispatches 2-second windows with 0.25-second overlap to whisper or parakeet, polls inference results on a background thread, dedupes overlapping segments against a recent-history buffer, post-processes with the formatting pipeline, and emits typed messages back to the frontend on two `tauri::ipc::Channel`s (one for results, one for status). Holds a macOS App Nap power assertion for the duration of the session. Writes audio progressively to a WAV file so a crash mid-session leaves a playable recording.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Path: `src-tauri/src/commands/live.rs`.
|
||||
- LOC: 1,737. Largest file in the slice.
|
||||
- Tauri commands exposed:
|
||||
- `start_live_transcription_session(window, app, state, live_state, config: StartLiveTranscriptionConfig, result_channel: Channel<LiveResultMessage>, status_channel: Channel<LiveStatusMessage>) -> Result<StartLiveTranscriptionResponse, String>` — main-window only.
|
||||
- `stop_live_transcription_session(window, app, live_state, session_id: u64) -> Result<StopLiveTranscriptionResponse, String>` — main-window only.
|
||||
- Events emitted: NONE in the conventional `app.emit(...)` sense. This module uses Tauri 2's typed `tauri::ipc::Channel<T>` API instead. The frontend creates the channel pair on the JS side via `new Channel<T>()`, passes it as a command argument, and Magnotia sends typed messages on it from the worker. Two channels:
|
||||
- `Channel<LiveResultMessage>` — per-chunk transcription results (segments, language, duration, raw_text, inference_ms, chunk_id, chunk_start_secs).
|
||||
- `Channel<LiveStatusMessage>` — tagged enum: `Warning { message }`, `Overload { dropped_audio_ms, message }`, `Error { message }`, `Finished { audio_path, dropped_audio_ms }`.
|
||||
- Depends on: `magnotia_audio::{AudioChunk, CaptureRuntimeError, MicrophoneCapture, StreamingResampler, WavWriter}`, `magnotia_core::constants::WHISPER_SAMPLE_RATE`, `magnotia_core::types::{AudioSamples, Segment, TranscriptionOptions}`, `magnotia_transcription::LocalEngine`, `magnotia_ai_formatting::{post_process_segments, FormatMode, PostProcessOptions}`, `magnotia_storage::{database::get_profile, database::list_profile_terms, DEFAULT_PROFILE_ID}`. Plus `commands::audio::resolve_recording_path`, `commands::build_initial_prompt`, `commands::models::{default_model_id_for_engine, ensure_model_loaded}`, `commands::power::PowerAssertion`, `commands::security::ensure_main_window`.
|
||||
- Called from frontend at: dictation page (when the user starts and stops a live session — most common entry).
|
||||
|
||||
## What's in here
|
||||
|
||||
### Constants (`src-tauri/src/commands/live.rs:30`)
|
||||
|
||||
Speech-gate, dedup, and chunking parameters. The headline numbers: `CHUNK_SAMPLES = 32_000` (2 s at 16 kHz), `OVERLAP_SAMPLES = 4_000` (0.25 s), `FINAL_CHUNK_MIN_SAMPLES = 4_000`, `MAX_PENDING_SAMPLES = CHUNK_SAMPLES`. Speech-gate thresholds (RMS / peak / consecutive-window counts) follow.
|
||||
|
||||
### State
|
||||
|
||||
- `LiveTranscriptionState` (`src-tauri/src/commands/live.rs:62`) — the Tauri-managed struct stashed by `lib.rs::run`. Fields:
|
||||
- `next_session_id: AtomicU64` — monotonic session-id generator.
|
||||
- `lifecycle: AsyncMutex<()>` — start/stop barrier.
|
||||
- `running: Mutex<Option<RunningLiveSession>>` — the currently-running session, if any.
|
||||
- `RunningLiveSession` (`src-tauri/src/commands/live.rs:68`) — id, stop_flag, JoinHandle for the blocking worker, the status channel.
|
||||
|
||||
### Public payload types
|
||||
|
||||
- `StartLiveTranscriptionConfig` (`src-tauri/src/commands/live.rs:77`) — engine, model_id, language, initial_prompt, save_audio, output_folder, post-processing flags, format_mode, microphone_device, profile_id.
|
||||
- `StartLiveTranscriptionResponse` — `{ session_id }`.
|
||||
- `StopLiveTranscriptionResponse` — `{ session_id, audio_path: Option<String>, dropped_audio_ms: u64 }`.
|
||||
- `LiveResultMessage` — per-chunk result.
|
||||
- `LiveStatusMessage` — tagged enum (4 variants).
|
||||
|
||||
### `ActiveCapture` (`src-tauri/src/commands/live.rs:166`)
|
||||
|
||||
Wraps `MicrophoneCapture` plus its `cpal` chunk receiver and the optional runtime-error receiver. `drain_runtime_errors` posts `LiveStatusMessage::Warning` for each cpal-side error.
|
||||
|
||||
### `LiveLoopState` (`src-tauri/src/commands/live.rs:208`)
|
||||
|
||||
Per-session mutable state: resampler, capture buffer, WAV writer, buffer start sample index, dropped-audio counter, chunk id, in-flight inference task, resampler-flushed flag, result-listener-lost flag, recent-segments dedup history.
|
||||
|
||||
### `LiveSessionRuntime` (`src-tauri/src/commands/live.rs:231`)
|
||||
|
||||
Owns everything for one session. Constructor opens the WAV writer. `run()` is the main loop:
|
||||
|
||||
```
|
||||
loop {
|
||||
poll_inference()?;
|
||||
capture.drain_runtime_errors();
|
||||
if let Some(chunk) = recv_audio()? { process_audio_chunk(chunk)?; }
|
||||
drop_pending_overflow(); // bounded buffer; emits Overload status
|
||||
flush_tail_if_stopping()?;
|
||||
if dispatch_inference_if_ready() { continue; }
|
||||
if should_exit_loop() { break; }
|
||||
}
|
||||
drain_inference()?;
|
||||
finish()
|
||||
```
|
||||
|
||||
Methods:
|
||||
|
||||
- `process_audio_chunk` — downmix, lazy-init `StreamingResampler`, push samples, append to capture buffer + WAV (`src-tauri/src/commands/live.rs:323`).
|
||||
- `drop_pending_overflow` — when the inflight inference is busy and the buffer exceeds `MAX_PENDING_SAMPLES`, drop the oldest samples and emit `LiveStatusMessage::Overload` with the cumulative dropped-audio counter (`:344`).
|
||||
- `flush_tail_if_stopping` — flush the resampler and the WAV header on shutdown (`:365`).
|
||||
- `dispatch_inference_if_ready` — wraps `maybe_dispatch_chunk` (the chunking + speech-gate + thread-spawn function) (`:396`).
|
||||
- `drain_inference` — busy-loops with 10 ms sleeps until the in-flight inference completes after stop (`:425`).
|
||||
- `finish` — finalise WAV, return `LiveSessionSummary` (`:433`).
|
||||
|
||||
### `start_live_transcription_session` (`src-tauri/src/commands/live.rs:484`)
|
||||
|
||||
1. `ensure_main_window`.
|
||||
2. `lifecycle.lock().await` — barrier against concurrent start/stop.
|
||||
3. Reject if a session is already running.
|
||||
4. Resolve profile_id, fetch profile + profile_terms from `magnotia_storage`.
|
||||
5. Collapse the effective `initial_prompt` via `build_initial_prompt` (so the worker doesn't have to know about profile fallback).
|
||||
6. Resolve model_id via `default_model_id_for_engine` if absent.
|
||||
7. `ensure_model_loaded(state, engine, model_id, None)` — `None` means don't enforce sequential-GPU mode (Settings owns that toggle).
|
||||
8. Resolve audio_path via `commands::audio::resolve_recording_path` if `save_audio` is true.
|
||||
9. `tokio::task::spawn_blocking(move || run_live_session(...))` — the real worker runs on a dedicated blocking thread, not the Tokio runtime, because Whisper inference itself spawns its own threads and the work is CPU-bound.
|
||||
10. Stash the new `RunningLiveSession`. Return the session_id.
|
||||
|
||||
### `stop_live_transcription_session` (`src-tauri/src/commands/live.rs:591`)
|
||||
|
||||
1. `ensure_main_window`, lifecycle lock.
|
||||
2. Take the running session out of state.
|
||||
3. Validate `session_id` matches; on mismatch, restore the session and return an error.
|
||||
4. Set the stop flag and await the worker `JoinHandle`.
|
||||
5. Read the summary, send `LiveStatusMessage::Finished` on the status channel, return the response.
|
||||
|
||||
### `run_live_session` (`src-tauri/src/commands/live.rs:646`)
|
||||
|
||||
The blocking entry. Holds a `PowerAssertion::begin("magnotia live dictation session")` for the entire scope. Constructs and runs `LiveSessionRuntime`. The drop on the power assertion ends the macOS App Nap pin.
|
||||
|
||||
### `maybe_dispatch_chunk` (`src-tauri/src/commands/live.rs:753`)
|
||||
|
||||
The brain of the chunking pipeline. Decides whether to dispatch a chunk now, based on capture buffer size and the stopping flag:
|
||||
|
||||
- Full chunk path: `target_len = CHUNK_SAMPLES`, with `OVERLAP_SAMPLES` of trim against the previous chunk to dedupe.
|
||||
- Stopping path: dispatch any partial chunk ≥ `FINAL_CHUNK_MIN_SAMPLES`.
|
||||
- Speech gate: `evaluate_speech_gate(speech_window)` (`:1305`) returns a decision based on per-frame RMS / peak amplitude / consecutive-speech-window counts. If skipped, drop those samples and emit a Warning.
|
||||
- On dispatch: spawn a `std::thread` that calls `engine.transcribe_sync` and posts the result back via a `std::sync::mpsc` channel. The 2025 version of this code used a Tokio task; switching to a plain thread keeps inference off the blocking pool entirely.
|
||||
|
||||
### `poll_inference` (`src-tauri/src/commands/live.rs:864`)
|
||||
|
||||
Polls the in-flight `InferenceTask`'s mpsc receiver. On result:
|
||||
|
||||
- Trim overlap segments against the previous chunk via `trim_overlap_segments`.
|
||||
- Run dedup vs the `recent_segments` history via `filter_duplicate_boundary_segments`.
|
||||
- Post-process with `post_process_segments` (using the dictionary terms and `PostProcessOptions`).
|
||||
- Build a `LiveResultMessage` and `emit_live_result(...)`.
|
||||
|
||||
### `emit_live_result` (`src-tauri/src/commands/live.rs:971`)
|
||||
|
||||
Sends on the result channel. If the listener is dead, sets `result_listener_lost = true` and tries to send a Warning on the status channel. If *that* also fails, self-asserts the stop flag so the worker drains and exits — otherwise the worker would burn CPU + GPU memory polling forever after the user closes the app window without a clean stop call.
|
||||
|
||||
### Dedup helpers (`src-tauri/src/commands/live.rs:1027` onwards)
|
||||
|
||||
- `filter_duplicate_boundary_segments` — drops segments at chunk boundaries that meaningfully overlap the recent-segments history.
|
||||
- `remember_recent_segments` — maintains the rolling window (~`DUPLICATE_HISTORY_RETENTION_SECS = 8.0`).
|
||||
- `build_nearby_transcript_candidates` — collects candidates in the leading-edge window (`DUPLICATE_CHECK_LEADING_SECS = 1.5`).
|
||||
- `transcripts_overlap` and `transcripts_loosely_overlap` — token-coverage / longest-common-subsequence checks against `LOW_SIGNAL_TOKENS` (a stop-word-equivalent list of ~60 high-frequency tokens).
|
||||
|
||||
### Speech gate (`src-tauri/src/commands/live.rs:1251` onwards)
|
||||
|
||||
- `record_speech_window`, `speech_gate_decision`, `evaluate_speech_gate`. Two thresholds: a strong-speech path (high RMS or high peak, or two consecutive speech windows) and a soft-speech path. `FLATLINE_PEAK_THRESHOLD` catches the silent-buffer case (e.g. mic disconnected). The gate keeps Whisper from hallucinating on near-silent audio, which Whisper is famous for doing ("you are watching the show").
|
||||
|
||||
### Other helpers
|
||||
|
||||
- `downmix_chunk` (`:1336`) — same pattern as `commands::audio`.
|
||||
- `pick_engine` (`:638`) — `state.whisper_engine` or `state.parakeet_engine`.
|
||||
- `open_wav_writer`, `finalize_wav_writer`, `append_resampled_audio` — progressive WAV plumbing (brief item #19).
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
frontend invoke('start_live_transcription_session', { config, result_channel, status_channel })
|
||||
-> Rust: validate, fetch profile, build prompt, ensure model loaded, spawn worker
|
||||
worker (blocking thread):
|
||||
loop:
|
||||
cpal -> ActiveCapture -> StreamingResampler -> capture_buffer + WAV
|
||||
when buffer >= 32k samples (or stopping with >= 4k):
|
||||
speech-gate -> if pass: thread::spawn(engine.transcribe_sync)
|
||||
poll inflight: filter overlap, dedup vs history, post_process_segments
|
||||
send LiveResultMessage on result_channel
|
||||
on overflow: drop oldest, send LiveStatusMessage::Overload
|
||||
on stop flag: flush resampler tail, drain inflight, finalise WAV
|
||||
return LiveSessionSummary
|
||||
frontend invoke('stop_live_transcription_session', { session_id })
|
||||
-> Rust: set stop flag, await worker, send LiveStatusMessage::Finished, return response
|
||||
```
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **Size.** 1,737 LOC in one file. The runtime + loop + speech gate + dedup + chunker really should be split. The pieces are already modular; pulling each out into its own file under `commands/live/` would make the surface much easier to read and audit.
|
||||
- **`thread::spawn` for inference.** Each chunk spawns a fresh OS thread (`live.rs:841`). Inside Whisper this is fine because whisper.cpp uses its own thread pool, and only one chunk is in flight at a time per session. Two simultaneous live sessions would multiply this; the lifecycle lock forbids that today.
|
||||
- **`poll_inference` busy-loops with 10 ms sleeps in `drain_inference`.** Acceptable because we only enter the drain on stop. Don't reuse this pattern for the main loop.
|
||||
- **Result-listener-lost path is critical.** Without it, closing the main window without a clean stop would leave the worker spinning forever, holding the GPU memory and the WAV file handle until process exit. The self-asserted stop flag is the safety net.
|
||||
- **Power assertion only does work on macOS.** On Linux the function is a no-op (see [Power assertions and security](power-and-security.md)). A long live-dictation session on Linux can still be idled by the compositor.
|
||||
- **The recent-segments history is bounded by time, not count.** A high chunk rate could grow it more than expected; the retention is `DUPLICATE_HISTORY_RETENTION_SECS = 8.0`.
|
||||
- **Channel back-pressure.** The result channel is the JS-side `Channel<T>` queue. If the frontend stops reading, the queue grows. Magnotia's overload-signalling currently uses the in-buffer `MAX_PENDING_SAMPLES` cap; it does NOT detect a JS-side stalled listener except via the `emit_live_result`-failure path.
|
||||
- **`ensure_model_loaded(state, engine, model_id, None)`** intentionally passes `None` for `concurrent`, so live sessions never trigger the sequential-GPU guard in `commands::models`. If you ever ship a tight-VRAM machine and the user has switched to sequential mode, this could OOM. Today's hardware survey indicates this is uncommon; flag this when revisiting Phase A.4.
|
||||
|
||||
## See also
|
||||
|
||||
- [Audio capture](audio.md) — `resolve_recording_path` and `recording_filename` are shared.
|
||||
- [Models](models.md) — `default_model_id_for_engine` and `ensure_model_loaded` are called from start.
|
||||
- [Transcription](transcription.md) — the non-live transcription path that shares the post-processing pipeline.
|
||||
- [Profiles](profiles.md) — the profile + profile-terms fetch happens before the worker spawns.
|
||||
- [Power assertions and security](power-and-security.md) — the App Nap pin and `ensure_main_window` guard.
|
||||
- [`commands::mod`](mod.md) — `build_initial_prompt` is the prompt assembler used here.
|
||||
118
docs/architecture-map/02-tauri-runtime/commands/llm.md
Normal file
118
docs/architecture-map/02-tauri-runtime/commands/llm.md
Normal file
@@ -0,0 +1,118 @@
|
||||
---
|
||||
name: Local LLM commands
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# `commands::llm`
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → LLM
|
||||
|
||||
**Plain English summary.** Owns the local LLM lifecycle (recommend tier, check, download with progress events, load with sequential-GPU coordination, unload, delete, status, test) plus two inference commands the frontend uses on demand: cleanup-transcript-text (the "fix this" pass that runs after a transcription) and extract-content-tags (Phase 9, surface topic / intent tags on the History page). Diagnostic test command classifies load failures into VRAM / corrupt / permission / other so Settings shows actionable hints rather than raw llama.cpp exceptions.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Path: `src-tauri/src/commands/llm.rs`.
|
||||
- LOC: 423.
|
||||
- Tauri commands exposed (10 total):
|
||||
- `recommend_llm_tier() -> Result<String, String>`. No window guard — pure hardware probe.
|
||||
- `check_llm_model(state, model_id) -> Result<LlmModelStatusDto, String>`.
|
||||
- `download_llm_model(window, app, model_id) -> Result<(), String>`. Main-window only. Emits `magnotia:llm-download-progress`.
|
||||
- `load_llm_model(window, state, model_id, use_gpu, concurrent) -> Result<(), String>`. Main-window only.
|
||||
- `unload_llm_model(window, state) -> Result<(), String>`. Main-window only.
|
||||
- `delete_llm_model(window, state, model_id) -> Result<(), String>`. Main-window only.
|
||||
- `get_llm_status(state) -> Result<bool, String>`.
|
||||
- `test_llm_model(window, state, model_id) -> Result<LlmTestResult, String>`. Main-window only.
|
||||
- `cleanup_transcript_text_cmd(window, state, transcript, profile_id, preset) -> Result<String, String>`. Main-window only.
|
||||
- `extract_content_tags_cmd(state, transcript) -> Result<ContentTags, String>`.
|
||||
- Events emitted: `magnotia:llm-download-progress` (`{ modelId, done, total, percent }`) — `src-tauri/src/commands/llm.rs:76`.
|
||||
- Depends on: `magnotia_llm::{LlmEngine, LlmModelId, ContentTags, model_manager::{download_model, model_path, model_info, recommend_tier, is_downloaded, delete_model}}`, `magnotia_ai_formatting::{llm_cleanup_text, LlmPromptPreset}`, `magnotia_core::hardware`, `magnotia_storage::database::list_profile_terms`. Plus `commands::power::PowerAssertion`, `commands::security::ensure_main_window`.
|
||||
- Called from frontend at: Settings → AI page (download / load / unload / delete / test), dictation result panel (`cleanup_transcript_text_cmd`), History page (`extract_content_tags_cmd`).
|
||||
|
||||
## What's in here
|
||||
|
||||
### `LlmModelStatusDto` (`src-tauri/src/commands/llm.rs:11`)
|
||||
|
||||
Frontend-facing status DTO: id, displayName, downloaded, loaded, sizeBytes, description, minimumRamBytes, recommendedVramBytes.
|
||||
|
||||
### `parse_model_id` (`:24`)
|
||||
|
||||
Wraps `model_id.parse()` (which goes through the `LlmModelId` parser).
|
||||
|
||||
### `recommend_llm_tier` (`:28`)
|
||||
|
||||
Probes RAM via `magnotia_core::hardware::probe_system`, multiplies the MB to bytes, then defers to `magnotia_llm::model_manager::recommend_tier(ram, vram)`. No window guard — Settings calls this at first paint to populate the default tier suggestion.
|
||||
|
||||
### `check_llm_model` (`:40`)
|
||||
|
||||
Combines `model_info(id)` (static metadata), `model_manager::is_downloaded(id)`, and `state.llm_engine.loaded_model_id()` into a `LlmModelStatusDto`. Used by Settings to render per-tier status chips.
|
||||
|
||||
### `download_llm_model` (`:61`)
|
||||
|
||||
`ensure_main_window`. Calls `model_manager::download_model(id, progress_cb)` with a closure that emits `magnotia:llm-download-progress` on each chunk. The percent calculation rounds against `total > 0` (avoids division by zero on a server that doesn't return a content-length).
|
||||
|
||||
### `load_llm_model` (`:90`)
|
||||
|
||||
`ensure_main_window`, `parse_model_id`, check the file exists. Implements the inverse Phase A.1 sequential-GPU guard: when `concurrent == Some(false)`, unloads BOTH `state.whisper_engine` and `state.parakeet_engine` before loading the LLM. Default `use_gpu = true`. Loads inside `spawn_blocking`.
|
||||
|
||||
### `unload_llm_model` (`:122`), `delete_llm_model` (`:131`), `get_llm_status` (`:145`)
|
||||
|
||||
Thin wrappers. `delete_llm_model` unloads first if the model is currently loaded.
|
||||
|
||||
### `LlmTestResult` (`:155`) and `test_llm_model` (`:186`)
|
||||
|
||||
Settings → AI's "Test LLM" button. Brief item B.1 #27. Decision tree:
|
||||
|
||||
1. File missing → `not-downloaded` with a "Click Download" hint.
|
||||
2. File present but ≤ 90% of expected size → `incomplete` (truncated download) with a re-download hint. 10% tolerance because `info.size_bytes` is rounded.
|
||||
3. Same model already loaded → `ready` without disturbing the engine.
|
||||
4. Otherwise attempt `engine.load_model(id, &path, use_gpu=true)` inside `spawn_blocking` and pass any error string to `classify_llm_load_error`.
|
||||
|
||||
### `classify_llm_load_error` (`:272`)
|
||||
|
||||
Pure string classifier. Tested independently. Buckets: `load-failed-vram` (looks for `out of memory`, `oom`, `allocation failed`, `vram`, `cudamalloc`), `load-failed-corrupt` (looks for `magic`, `invalid gguf`, `unsupported file format`, `tensor shape`), `load-failed-permission` (looks for `permission denied`, `access is denied`), and `load-failed-other` for everything else. Each bucket pairs with a user-facing hint string.
|
||||
|
||||
### `cleanup_transcript_text_cmd` (`:362`)
|
||||
|
||||
`ensure_main_window`. Resolves `profile_id`. Fetches profile_terms from storage. Resolves the `preset` (Brief item B.1 #15: `Default`, `Email`, `Notes`, `Code`). `spawn_blocking` runs `llm_cleanup_text(&engine, &transcript, &profile_terms, resolved_preset)` inside a `PowerAssertion::begin("magnotia LLM cleanup")` so macOS App Nap doesn't throttle mid-token.
|
||||
|
||||
### `extract_content_tags_cmd` (`:407`)
|
||||
|
||||
Phase 9 LLM tagging. NO window guard (the History page lives in main but the command would be safe even without it; if you ever expose the History page in a secondary window, add the guard). Errors out cleanly if no LLM is loaded. `spawn_blocking` runs `engine.extract_content_tags(&transcript)` under the same App Nap power-assertion pattern.
|
||||
|
||||
### Tests (`:306`)
|
||||
|
||||
Eight `classify_llm_load_error` cases cover the four classification buckets and edge cases (cudaMalloc, OOM, generic allocation failure, GGUF magic mismatch, unsupported file format, permission denied, Windows access denied, unknown error).
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
Settings -> recommend_llm_tier() -> hardware probe -> tier string
|
||||
Settings -> download_llm_model(model_id) -> model_manager::download_model
|
||||
-> per-chunk progress events
|
||||
-> file lands in ~/.magnotia/models/llm/
|
||||
Settings -> load_llm_model(model_id, use_gpu, concurrent)
|
||||
-> if concurrent=false: unload whisper + parakeet
|
||||
-> spawn_blocking(engine.load_model)
|
||||
Settings -> test_llm_model(model_id) -> tiered checks -> LlmTestResult
|
||||
History page -> extract_content_tags_cmd(transcript) -> ContentTags { topics, intents }
|
||||
Dictation result -> cleanup_transcript_text_cmd(transcript, profile_id, preset) -> cleaned text
|
||||
```
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **Sequential-GPU guard is the inverse here.** `load_llm_model` unloads transcription engines when `concurrent=false`. `commands::models::ensure_model_loaded` unloads the LLM. Settings owns the toggle; live transcription explicitly bypasses it (passes `None`). Confirm both halves stay in sync if you ever change the toggle's semantics.
|
||||
- **`extract_content_tags_cmd` lacks `ensure_main_window`.** Intentional today, but if you change the ACL to expose this to a non-main window, add the guard.
|
||||
- **`PowerAssertion` is a no-op outside macOS.** Long LLM cleanup on Linux can be idled by the compositor. See [Power assertions and security](power-and-security.md).
|
||||
- **Partial-download detection uses 10% tolerance.** A model that ships exactly 90% of expected size will be incorrectly flagged as incomplete. The expected size is rounded by `model_manager`, so empirically this is fine; if you tighten the rounding, also tighten the threshold.
|
||||
- **`download_llm_model` emits a percent rounded to `u8`.** A frontend that wants smoother progress bars needs to use the raw `done / total` fields.
|
||||
- **The cleanup command silently passes an unrecognised preset as `Default`** (`LlmPromptPreset::parse` returns `Default` for unknown strings). Consider erroring out instead so frontend bugs surface faster.
|
||||
|
||||
## See also
|
||||
|
||||
- [Models](models.md) — the inverse sequential-GPU guard.
|
||||
- [Tasks](tasks.md) — also calls `engine.decompose_task_with_feedback` / `extract_tasks_with_feedback` under the same App Nap pattern.
|
||||
- [Power assertions and security](power-and-security.md) — `PowerAssertion::begin` is shared.
|
||||
- [Diagnostics](diagnostics.md) — the test_llm_model failure messages cite "see Settings → About → Diagnostics bundle".
|
||||
- [Profiles](profiles.md) — `cleanup_transcript_text_cmd` reads profile_terms from storage.
|
||||
67
docs/architecture-map/02-tauri-runtime/commands/mod.md
Normal file
67
docs/architecture-map/02-tauri-runtime/commands/mod.md
Normal file
@@ -0,0 +1,67 @@
|
||||
---
|
||||
name: commands/mod.rs — module roots and prompt builder
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# `commands::mod`
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → mod.rs
|
||||
|
||||
**Plain English summary.** `mod.rs` declares all 25 child modules (22 command modules, 3 utility modules) and exports one shared helper: `build_initial_prompt`. The helper is the precedence rule that decides what the Whisper `initial_prompt` actually is when a command receives caller-supplied prompt text plus a profile prompt plus a list of profile vocabulary terms.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Path: `src-tauri/src/commands/mod.rs`.
|
||||
- LOC: 114 (60 of those are tests).
|
||||
- Tauri commands exposed: none.
|
||||
- Events emitted: none.
|
||||
- Depends on: nothing crate-external.
|
||||
- Called from: `commands::transcription` (whisper PCM path and file path), `commands::live::start_live_transcription_session`. Both call `build_initial_prompt(&request_prompt, &profile.initial_prompt, &profile_terms)` to assemble the final Whisper prompt before passing it into `TranscriptionOptions`.
|
||||
|
||||
## What's in here
|
||||
|
||||
### Module declarations (`src-tauri/src/commands/mod.rs:1`)
|
||||
|
||||
```
|
||||
audio, clipboard, diagnostics, feedback, fs, hardware,
|
||||
hotkey, intentions, live, llm, meeting, models,
|
||||
mod (this file), nudges, paste, power, profiles,
|
||||
rituals, security, tasks, transcription, transcripts,
|
||||
tts, update, windows
|
||||
```
|
||||
|
||||
All declared `pub mod`, so any sibling module can `use crate::commands::name`.
|
||||
|
||||
### `build_initial_prompt(request_prompt, profile_prompt, profile_terms) -> Option<String>` (`src-tauri/src/commands/mod.rs:39`)
|
||||
|
||||
Precedence:
|
||||
|
||||
1. Caller-supplied `request_prompt` wins outright when non-empty (the caller has already made the decision).
|
||||
2. Else: profile's stored prompt + profile terms (joined as `"<profile prompt> Vocabulary: term1, term2."`). The vocabulary line is the OpenWhispr pattern: feeding domain terms into Whisper's `initial_prompt` biases the decoder toward the correct spelling at decode time, before any LLM cleanup pass.
|
||||
3. Else: profile prompt alone, or `"Vocabulary: term1, term2."` alone if only terms are present.
|
||||
4. Else: `None`.
|
||||
|
||||
Whitespace-only terms are skipped. Whitespace-only prompts are treated as empty. The returned `Option<String>` is what every Whisper-side command stuffs into `TranscriptionOptions::initial_prompt`.
|
||||
|
||||
### Tests (`src-tauri/src/commands/mod.rs:65`)
|
||||
|
||||
Six test cases cover each branch of the precedence rule plus whitespace handling. These are pure-function tests, no DB.
|
||||
|
||||
## Data flow
|
||||
|
||||
The helper is called *after* the calling command has read the relevant `ProfileRow` and `ProfileTermRow`s from `magnotia_storage`. The DB I/O lives in the calling command (so the tests in this file stay pure).
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- The helper does not de-duplicate terms. If the same term appears twice in `profile_terms`, it lands twice in the vocabulary sentence. The storage layer's `add_profile_term` is what enforces uniqueness, but if the terms list ever comes from somewhere else, dedup at the call site.
|
||||
- Vocabulary length is not capped here. A profile with hundreds of terms will produce a very long `initial_prompt`, and Whisper has a context-window limit that depends on the model. If you ever ship a UI that lets users add unlimited terms, add a cap in the calling commands or here.
|
||||
- The whisper.cpp `initial_prompt` is best-effort only. It biases decoding but does not guarantee a particular word will be produced. Profile-edit-derived corrections (`commands::profiles::learn_profile_terms_from_edit_cmd`) feed back into this path on the next session.
|
||||
|
||||
## See also
|
||||
|
||||
- [Profiles](profiles.md) — `learn_profile_terms_from_edit_cmd` is what populates the `profile_terms` list this helper consumes.
|
||||
- [Transcription](transcription.md) — both whisper paths call this helper.
|
||||
- [Live transcription](live.md) — `start_live_transcription_session` calls this helper once at startup and stashes the result on the config struct.
|
||||
- [Commands index](README.md) — back to the index.
|
||||
113
docs/architecture-map/02-tauri-runtime/commands/models.md
Normal file
113
docs/architecture-map/02-tauri-runtime/commands/models.md
Normal file
@@ -0,0 +1,113 @@
|
||||
---
|
||||
name: Model registry and runtime
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# `commands::models`
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → Models
|
||||
|
||||
**Plain English summary.** Owns the Whisper and Parakeet model lifecycle: download with progress events, presence checks, list, load (with optional sequential-GPU coordination against the LLM), and the runtime-capabilities snapshot that Settings consumes (accelerators list, active compute device, CPU features, parallel-mode availability). Also owns the silent warm-up pass that pre-allocates the Whisper context window so the user's first transcription is fast. Emits runtime warnings on startup when the CPU lacks AVX2 / FMA or the Vulkan loader is absent.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Path: `src-tauri/src/commands/models.rs`.
|
||||
- LOC: 691.
|
||||
- Tauri commands exposed (13 total):
|
||||
- Whisper: `download_model(size)`, `check_model(size)`, `list_models()`, `load_model(size, concurrent)`, `prewarm_default_model_cmd()`, `check_engine()`, `get_runtime_capabilities()`.
|
||||
- Parakeet: `download_parakeet_model(name)`, `check_parakeet_model(name)`, `list_parakeet_models()`, `load_parakeet_model(name, concurrent)`, `check_parakeet_engine()`.
|
||||
- Public Rust helpers (used by other command modules): `default_model_id_for_engine`, `ensure_model_loaded`, `prewarm_default_model`, `load_model_from_disk`, `detect_active_compute_device`, `emit_runtime_warnings`.
|
||||
- Events emitted: `model-download-progress` (whisper), `parakeet-download-progress`, `runtime-warning` (tagged enum: `Avx2Missing`, `VulkanLoaderMissing`, future `CudaFallback`).
|
||||
- Depends on: `magnotia_transcription::{model_manager, load_whisper, load_parakeet, LocalEngine, Transcriber}`, `magnotia_core::{model_registry, hardware, constants, types}`.
|
||||
- Called from frontend at: Settings → Models page (download, load, status), the dictation page boot path (`prewarm_default_model_cmd`), Settings → About (runtime capabilities).
|
||||
|
||||
## What's in here
|
||||
|
||||
### Model id mapping
|
||||
|
||||
- `whisper_model_id(size)` (`src-tauri/src/commands/models.rs:18`) — maps legacy size strings (`"tiny"`, `"base"`, `"small"`, `"distil-small"`, `"medium"`, `"distil-large-v3"`) to canonical `ModelId`s used by `model_registry`.
|
||||
- `parakeet_model_id(name)` (`src-tauri/src/commands/models.rs:32`) — maps `"ctc-int8"` to `parakeet-ctc-0.6b-int8`.
|
||||
|
||||
### `load_model_from_disk` (`src-tauri/src/commands/models.rs:77`)
|
||||
|
||||
Looks up the registry entry, picks the engine path:
|
||||
|
||||
- `Engine::Whisper` (gated on `feature = "whisper"`) calls `load_whisper(&model_file)`. Without the feature, returns a clear error explaining the feature is off.
|
||||
- `Engine::Parakeet` calls `load_parakeet(&dir)`.
|
||||
- `Engine::Moonshine` returns "not yet supported".
|
||||
|
||||
Returns a `Box<dyn Transcriber + Send>`.
|
||||
|
||||
### `default_model_id_for_engine` (`src-tauri/src/commands/models.rs:111`)
|
||||
|
||||
Returns `parakeet-ctc-0.6b-int8` for `"parakeet"`, `whisper-base-en` for everything else. Used by `commands::transcription`, `commands::live`, and `prewarm_default_model`.
|
||||
|
||||
### `ensure_model_loaded` (`src-tauri/src/commands/models.rs:118`)
|
||||
|
||||
The shared model-load gateway used by both whisper and parakeet command paths. Validates the engine matches the model, checks the model is downloaded, and short-circuits if the engine already has the right model loaded. Implements the Phase A.1 sequential-GPU guard (brief item #28): when `concurrent == Some(false)` and the LLM is loaded, unloads the LLM before loading the transcription engine. The inverse guard lives in `commands::llm::load_llm_model`. Loads run inside `spawn_blocking`.
|
||||
|
||||
### `prewarm_default_model` (`src-tauri/src/commands/models.rs:180`)
|
||||
|
||||
Loads the default Whisper model (if downloaded and not already loaded) plus runs a 1-second silence pass through `transcribe_sync`. The silence run pre-allocates the Whisper context window and warms GPU shader caches so the first real user transcription completes in ≤ 1.5× steady-state latency rather than the 4–5 s cold-start path documented in `ufal/whisper_streaming` issues #96 and #135. All errors are logged, never panic.
|
||||
|
||||
### `prewarm_default_model_cmd` (`src-tauri/src/commands/models.rs:225`)
|
||||
|
||||
Tauri command wrapper. `ensure_main_window`. Triggers the warm-up. The frontend invokes this after the dictation page mounts, so warm-up runs in parallel with the first paint.
|
||||
|
||||
### Runtime-capabilities reporting
|
||||
|
||||
- `RuntimeCapabilities` (`src-tauri/src/commands/models.rs:237`) — frontend-facing struct. Contains `accelerators: Vec<String>`, `engines: Vec<EngineRuntimeCapabilities>`, `active_compute_device`, `cpu_features`, `parallel_mode_available`.
|
||||
- `ActiveComputeDevice` (`:259`) — `kind`, `label`, optional `reason` (set when we fell back from a richer backend).
|
||||
- `CpuFeaturesInfo` (`:270`) — avx2 / avx512f / fma / sse4_2 / neon plus a `has_ggml_baseline` shortcut.
|
||||
- `EngineRuntimeCapabilities` and `ModelRuntimeCapabilities` carry per-engine and per-model status (downloaded, loaded, language support, default model id).
|
||||
- `compose_accelerators` (`:337`) — pure helper combining whisper-feature flag + Vulkan-loader-availability + target family. Always starts with `"cpu"`, appends `"metal"` (macOS) or `"vulkan"` (other) only when whisper is compiled in AND the loader resolves. Replaces the old hard-coded `["cpu", "vulkan"]` (RB-07).
|
||||
- `detect_active_compute_device` (`:369`) — returns the `ActiveComputeDevice` snapshot. macOS gets `"metal"` (via MoltenVK) or a CPU-fallback message; everything else gets `"vulkan"` or a CPU-fallback message naming the missing loader package.
|
||||
- `emit_runtime_warnings` (`:428`) — called from `lib.rs::run` setup. Emits `runtime-warning` events for the `Avx2Missing` and `VulkanLoaderMissing` cases. The frontend renders a one-line banner.
|
||||
- `get_runtime_capabilities` (`:454`) — assembles and returns the `RuntimeCapabilities` snapshot. Called by Settings.
|
||||
|
||||
### Whisper commands (`src-tauri/src/commands/models.rs:515`)
|
||||
|
||||
- `download_model(size) -> Result<String, String>` — main-window only. Calls `model_manager::download` with a closure that emits `model-download-progress`.
|
||||
- `check_model(size) -> Result<bool, String>` — `model_manager::is_downloaded`.
|
||||
- `list_models() -> Result<Vec<String>, String>` — returns human-readable size names of currently-downloaded whisper models.
|
||||
- `load_model(size, concurrent) -> Result<String, String>` — main-window only. Calls `ensure_model_loaded`.
|
||||
- `check_engine(state) -> Result<bool, String>` — returns whether the whisper engine has any model loaded.
|
||||
|
||||
### Parakeet commands (`src-tauri/src/commands/models.rs:576`)
|
||||
|
||||
Same shape as whisper, prefixed `parakeet_` and emitting `parakeet-download-progress`.
|
||||
|
||||
### Tests (`src-tauri/src/commands/models.rs:631`)
|
||||
|
||||
Five `compose_accelerators` permutations cover the RB-07 regression. Confirm:
|
||||
- Whisper disabled + loader present → `["cpu"]`.
|
||||
- Whisper enabled + loader missing → `["cpu"]`.
|
||||
- Whisper enabled + loader present + macOS → `["cpu", "metal"]`.
|
||||
- Whisper enabled + loader present + Linux/Windows → `["cpu", "vulkan"]`.
|
||||
- `"cpu"` is always first (frontend index-0 contract).
|
||||
|
||||
## Data flow
|
||||
|
||||
- **Download:** frontend invokes `download_model(size)` → `model_manager::download` streams bytes → progress events fire on each chunk → frontend renders progress bar.
|
||||
- **Load:** frontend invokes `load_model(size, concurrent)` → `ensure_model_loaded` validates and unloads LLM if `concurrent=false` → `load_model_from_disk` runs in `spawn_blocking` → `LocalEngine.load(model, model_id)` stashes the model on the engine.
|
||||
- **Transcribe:** every transcription path calls `ensure_model_loaded` first. Idempotent when the right model is already loaded.
|
||||
- **Runtime warnings:** fired once at startup via `lib.rs::run` setup → frontend listens for `runtime-warning` and shows a banner in Settings.
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **The `concurrent` flag is a tri-state.** `None` and `Some(true)` keep the legacy parallel-residency behaviour. Only `Some(false)` triggers the unload-the-other-engine guard. Live transcription explicitly passes `None`. If you ever ship a 4 GB-VRAM Settings preset that flips `concurrent=false` by default, also test the live transcription path.
|
||||
- **`parallel_mode_available` is hard-wired to `false` until Phase A.4 lands a real GPU VRAM probe** (`src-tauri/src/commands/models.rs:509`). Today's frontend reads this and disables the toggle. Flag for follow-up.
|
||||
- **`prewarm_default_model` only runs whisper.** Parakeet has no warm-up. The Parakeet model is also smaller and loads faster, so the cold-start gap is less obvious. If you swap Magnotia's default to Parakeet, add an equivalent warm-up.
|
||||
- **Vulkan loader detection is per-platform** (`magnotia_core::hardware::vulkan_loader_available`). The CPU-fallback `reason` strings are user-visible; keep them actionable (the Linux one names `libvulkan1`, the macOS one names the Vulkan SDK runtime).
|
||||
- **Whisper feature gating.** `--no-default-features` builds compile but `load_model_from_disk` returns a runtime error when the user requests a Whisper model. `list_models` will still show whisper models that happened to be downloaded already; consider hiding them when the feature is off if a no-whisper build ever ships to users.
|
||||
- **Download progress events fire from a closure inside `model_manager::download`.** That closure is called from inside an async `spawn_blocking`-equivalent. The `let _ = app_clone.emit(...)` pattern silently drops emit errors; if a window has been closed mid-download the progress events stop landing but the download itself completes.
|
||||
|
||||
## See also
|
||||
|
||||
- [Transcription](transcription.md) — the consumer of `ensure_model_loaded`.
|
||||
- [Live transcription](live.md) — also calls `ensure_model_loaded` (with `concurrent=None`).
|
||||
- [LLM](llm.md) — the inverse sequential-GPU guard lives in `commands::llm::load_llm_model`.
|
||||
- [App lifecycle](../app-lifecycle.md) — `emit_runtime_warnings` is called from setup.
|
||||
- [Cargo and features](../cargo-and-features.md) — `feature = "whisper"` is what gates `load_model_from_disk`.
|
||||
113
docs/architecture-map/02-tauri-runtime/commands/paste.md
Normal file
113
docs/architecture-map/02-tauri-runtime/commands/paste.md
Normal file
@@ -0,0 +1,113 @@
|
||||
---
|
||||
name: Paste at cursor
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# `commands::paste`
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → Paste
|
||||
|
||||
**Plain English summary.** Auto-insert-at-cursor: copy the transcript onto the clipboard and synthesise the platform's paste keystroke (Ctrl+V / Cmd+V) so the text lands in whatever app the user was already focused on. Adds a replace-with-raw flow that fires undo first. Skips the keystroke when the focused window is a terminal emulator (terminals duplicate the keystroke through the PTY). Hides the always-on-top preview overlay before the keystroke so a Wayland compositor doesn't accidentally route the paste back into Magnotia. Restores the user's prior clipboard 300 ms after the paste to honour the "never silently clobber the user's clipboard" contract.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Path: `src-tauri/src/commands/paste.rs`.
|
||||
- LOC: 790.
|
||||
- Tauri commands exposed:
|
||||
- `paste_text(app, text: String) -> Result<PasteOutcome, String>`. Copy + paste at cursor. Returns a `PasteOutcome { backend, pasted, copied, message }`.
|
||||
- `paste_text_replacing(app, text: String) -> Result<PasteOutcome, String>`. Replace flow: undo → 60 ms gap → paste. Same return shape.
|
||||
- `detect_paste_backends() -> Vec<String>`. Pure probe used by Settings: returns the names of available backends on the current OS / session.
|
||||
- Events emitted: none.
|
||||
- Depends on: `arboard::Clipboard`, `tauri::Manager` (to find and hide the preview window), `std::process::Command` (every backend shells out).
|
||||
- Called from frontend at: dictation result toast / replace-with-raw button (the two main user-facing actions).
|
||||
|
||||
## What's in here
|
||||
|
||||
### Constants (`src-tauri/src/commands/paste.rs:29`)
|
||||
|
||||
- `CLIPBOARD_RESTORE_MS = 300` — window after paste before restoring the user's prior clipboard.
|
||||
- `PREVIEW_HIDE_SETTLE_MS = 80` — compositor settle time after hiding the preview overlay.
|
||||
- `UNDO_PASTE_GAP_MS = 60` — gap between undo and follow-up paste in the replace flow (brief item #17).
|
||||
|
||||
### `PasteOutcome` (`src-tauri/src/commands/paste.rs:43`)
|
||||
|
||||
Frontend-facing struct: `backend: Option<String>` (e.g. `"wtype"`, `"xdotool"`, `"ydotool"`, `"osascript"`, `"sendkeys"`), `pasted: bool`, `copied: bool`, `message: Option<String>`.
|
||||
|
||||
### `paste_text` (`src-tauri/src/commands/paste.rs:67`)
|
||||
|
||||
Step-by-step:
|
||||
|
||||
1. Snapshot the current clipboard text via `arboard::Clipboard::get_text` (or `None` for non-text content).
|
||||
2. Write the transcript onto the clipboard. If this fails, return immediately with `copied=false` and the error.
|
||||
3. Probe the focused window via `detect_focused_terminal()`. If a known terminal class hits, return with a "Terminal detected" message; the user can finish manually with Ctrl+Shift+V or right-click. Note: prior clipboard is intentionally NOT restored here, because the user's recovery path needs the transcript on the clipboard.
|
||||
4. `hide_preview_overlay_for_paste(&app).await` — find the `transcription-preview` window if it exists and is visible, hide it, sleep 80 ms.
|
||||
5. `trigger_paste_keystroke()` — per-OS dispatch.
|
||||
6. If the keystroke fired, schedule `restore_prior_clipboard` 300 ms later via a detached `tokio::spawn`.
|
||||
|
||||
### `paste_text_replacing` (`src-tauri/src/commands/paste.rs:185`)
|
||||
|
||||
Same shape as `paste_text` but with an extra step: after copying and hiding the preview, fire `trigger_undo_keystroke()`, sleep 60 ms, then paste. The undo removes whatever text Magnotia already inserted (the cleaned-up transcript), the paste inserts the raw text. Used by the "replace with raw" frontend button per brief item #17.
|
||||
|
||||
### `detect_paste_backends` (`src-tauri/src/commands/paste.rs:258`)
|
||||
|
||||
Pure probe: Linux returns the subset of `["wtype", "xdotool", "ydotool"]` that resolve via `command -v`; macOS always returns `["osascript"]`; Windows always returns `["sendkeys"]`. Settings uses this to tell the user "install wtype" when Linux is empty.
|
||||
|
||||
### Terminal classifier
|
||||
|
||||
- `KNOWN_TERMINAL_CLASSES` (`src-tauri/src/commands/paste.rs:292`) — substring list ordered longest-first so `windowsterminal` wins over `terminal`. Covers Windows (PowerShell, conhost, console, pwsh, cmd, WindowsTerminal), macOS (Terminal, iTerm, iTerm2), Linux (alacritty, konsole, gnome-terminal-server, kitty, wezterm, foot, st, urxvt, xterm, hyper, tilix).
|
||||
- `classify_terminal(raw)` (`src-tauri/src/commands/paste.rs:331`) — case-insensitive substring search over the list.
|
||||
- `detect_focused_window_class()` plus per-OS implementations (`:345`).
|
||||
|
||||
### Per-OS focused-window probes
|
||||
|
||||
- Linux X11: `xdotool getactivewindow getwindowclassname` (`:365`).
|
||||
- Linux Wayland: returns `None`. No reliable probe from an unprivileged client. Documented as "by design".
|
||||
- macOS: `osascript -e 'tell application "System Events" to get name of first application process whose frontmost is true'` (`:386`).
|
||||
- Windows: PowerShell P/Invoke wrapper around `GetForegroundWindow` + `GetWindowThreadProcessId` + `Get-Process` (`:407`).
|
||||
|
||||
### Per-OS paste / undo backends
|
||||
|
||||
- Linux: `linux_paste` / `linux_undo` walk `pick_linux_backend_order` (`:502`), which flips the preference order based on session type (`wtype, ydotool, xdotool` for Wayland; `xdotool, ydotool, wtype` for X11). Each tool is invoked via `Command::new`; `wtype` uses `-M ctrl v -m ctrl`, `xdotool` uses `key ctrl+v`, `ydotool` uses raw linux input keycodes (`29:1 47:1 47:0 29:0`). Undo replaces V (47) with Z (44).
|
||||
- macOS: `osascript -e 'tell ... to keystroke "v" using command down'` (`:594`). Undo swaps in `"z"`.
|
||||
- Windows: `powershell -Command "(New-Object -ComObject WScript.Shell).SendKeys('^v')"` (`:634`). Undo uses `'^z'`.
|
||||
|
||||
### Helpers
|
||||
|
||||
- `snapshot_clipboard_text` (`:125`) — wraps arboard.
|
||||
- `schedule_clipboard_restore` (`:137`) — fires a `tokio::spawn` that sleeps `CLIPBOARD_RESTORE_MS` and calls `restore_prior_clipboard`.
|
||||
- `restore_prior_clipboard` (`:152`) — reads the current clipboard, calls `should_restore` (`:171`), and writes back the prior content if and only if the clipboard still holds the transcript we wrote (i.e. the user has not copied something else in the interim).
|
||||
- `hide_preview_overlay_for_paste` (`:240`) — find the preview window, check visibility, hide, sleep 80 ms.
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
frontend invoke('paste_text', { text })
|
||||
-> snapshot clipboard
|
||||
-> write transcript to clipboard
|
||||
-> detect terminal: yes -> early return with "manual paste needed" message
|
||||
-> hide preview overlay (Wayland focus quirk)
|
||||
-> trigger paste keystroke per OS
|
||||
-> spawn 300 ms timer -> restore prior clipboard if clipboard still holds the transcript
|
||||
-> return PasteOutcome
|
||||
```
|
||||
|
||||
Replace flow inserts an `undo` keystroke and a 60 ms gap between hide and paste.
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **Focus must already be on the target window when the keystroke fires.** The global hotkey flow preserves this naturally; clicking Magnotia's own UI does not. The frontend Settings page surfaces the caveat next to the toggle.
|
||||
- **Wayland focused-window probe is intentionally absent.** No way to do this from an unprivileged Wayland client. Result: terminal detection on Wayland-Kitty users will miss; they fall back to the manual Ctrl+Shift+V they already use.
|
||||
- **Clipboard restore is best-effort.** If the user copies something else within the 300 ms window, `should_restore` will (correctly) decline to write back. If a slow Wayland compositor delays the keystroke past 300 ms, we restore too early and the synthesised Ctrl+V pastes the user's old clipboard. Tradeoff documented in Handy #921.
|
||||
- **Linux backend order is session-aware.** A user with `XDG_SESSION_TYPE=wayland` and only xdotool installed will fall through to xdotool with a warning; their X11-app focus on a Wayland session works fine but Wayland-native apps will not see the keystroke.
|
||||
- **No backend = clipboard-only.** If `trigger_paste_keystroke` fails on Linux (no tools installed), the user still has the transcript on the clipboard; Settings shows the install-wtype hint via `detect_paste_backends`.
|
||||
- **PowerShell process spawn cost on Windows.** Each paste spawns a fresh PowerShell. Acceptable for one-off paste invocations; if you ever build a streaming-paste mode, switch to native `SendInput` via the `windows` crate.
|
||||
- **Permissions.** The macOS path requires the user to have granted Accessibility permissions to Magnotia (System Settings → Privacy → Accessibility). Without that, `osascript` returns success but the keystroke is silently dropped. There is no probe today; flag this in onboarding.
|
||||
|
||||
## See also
|
||||
|
||||
- [Window management](windows.md) — `transcription-preview` is the window that gets hidden by `hide_preview_overlay_for_paste`.
|
||||
- [Small commands → clipboard](small-commands.md#clipboardrs) — the bare `copy_to_clipboard` command, used when the user explicitly wants clipboard-only.
|
||||
- [Live transcription](live.md) — the upstream of the dictation result that gets pasted.
|
||||
- [Hotkey bridge](hotkey.md) — the global hotkey path that preserves focus on the target window.
|
||||
@@ -0,0 +1,101 @@
|
||||
---
|
||||
name: Power assertions and security helpers
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# `commands::power` and `commands::security`
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → Power and security
|
||||
|
||||
**Plain English summary.** Two utility modules that the rest of the command files lean on. `power.rs` is the macOS App Nap power-assertion guard (no-op on Linux and Windows today). `security.rs` is `ensure_main_window`, the one-liner defence-in-depth check that several commands stamp at the top to reject calls from secondary windows even if the ACL ever drifted.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Paths: `src-tauri/src/commands/power.rs` (208 LOC), `src-tauri/src/commands/security.rs` (30 LOC).
|
||||
- Tauri commands exposed: none. Both are pure Rust utilities.
|
||||
- Events emitted: none.
|
||||
- Depends on:
|
||||
- `power.rs`: macOS only — `objc2::rc::Retained`, `objc2::runtime::ProtocolObject`, `objc2_foundation::{NSActivityOptions, NSObjectProtocol, NSProcessInfo, NSString}`.
|
||||
- `security.rs`: nothing.
|
||||
- Called from: every long-running command that wants to keep the OS awake (live transcription, LLM cleanup, content-tag extraction, task decomposition); every command that wants the extra firewall (intentions CRUD, nudges, updater stub, lots of Settings-only commands).
|
||||
|
||||
## `commands::power`
|
||||
|
||||
### `PowerAssertionSnapshot` (`src-tauri/src/commands/power.rs:28`)
|
||||
|
||||
Cloneable snapshot for the diagnostics report: `id, reason, backend, acquired`.
|
||||
|
||||
### `PowerAssertion` (`src-tauri/src/commands/power.rs:41`)
|
||||
|
||||
A `#[must_use]` guard. Holding it keeps the macOS App Nap pin alive; dropping it ends the assertion. Fields: `id`, `reason`, `backend`, `acquired`, plus a macOS-only `Option<ActivityHandle>`.
|
||||
|
||||
### `PowerAssertion::begin(reason)` (`src-tauri/src/commands/power.rs:73`)
|
||||
|
||||
Allocates a new id from a `NEXT_ID: AtomicUsize`, calls into `objc_bridge::begin_activity` on macOS, registers a snapshot in the global `assertion_registry()`. Returns the guard. Errors are logged; failure to acquire still returns a "no-op" guard (so the caller's code path is unchanged) with `acquired = false`.
|
||||
|
||||
On non-macOS, this is a no-op. The doc comment promises Linux logind inhibitors and Windows `SetThreadExecutionState`, but neither is wired up. See debt note in the slice README.
|
||||
|
||||
### `Drop for PowerAssertion` (`src-tauri/src/commands/power.rs:124`)
|
||||
|
||||
End the macOS activity if present, remove the registry entry. Logs the end on macOS so the diagnostics bundle has a breadcrumb.
|
||||
|
||||
### `assertion_registry()` (`src-tauri/src/commands/power.rs:53`)
|
||||
|
||||
Process-global `Mutex<HashMap<usize, PowerAssertionSnapshot>>` using `OnceLock`. `active_assertions_snapshot()` (`:58`) returns a sorted vector of clones for the diagnostic-report bundler.
|
||||
|
||||
### `objc_bridge` module (`src-tauri/src/commands/power.rs:140`)
|
||||
|
||||
macOS-only. Wraps `NSProcessInfo::processInfo().beginActivityWithOptions_reason(options, reason)` and `endActivity(handle)`. Options: `NSActivityOptions::UserInitiated | NSActivityOptions::LatencyCritical`. The activity object is `Retained<ProtocolObject<dyn NSObjectProtocol>>` and is `unsafe impl Send` (the activity is a process-wide pin and thread-safe).
|
||||
|
||||
### Tests (`:168`)
|
||||
|
||||
`power_assertion_is_a_no_op_drop` confirms the registry entry is added on `begin` and removed on `drop`. `multiple_assertions_get_unique_ids` confirms the id allocator works.
|
||||
|
||||
### Where `PowerAssertion::begin` is called
|
||||
|
||||
- `commands::live::run_live_session` (`live.rs:660`) — `"magnotia live dictation session"`.
|
||||
- `commands::llm::cleanup_transcript_text_cmd` (`llm.rs:394`) — `"magnotia LLM cleanup"`.
|
||||
- `commands::llm::extract_content_tags_cmd` (`llm.rs:417`) — `"magnotia LLM content-tag extraction"`.
|
||||
|
||||
NOT called from `commands::tasks::decompose_and_store` or `commands::tasks::extract_tasks_from_transcript_cmd`, even though both run multi-second LLM inference. Flag for follow-up.
|
||||
|
||||
## `commands::security`
|
||||
|
||||
### `ensure_main_window(window)` (`src-tauri/src/commands/security.rs:1`)
|
||||
|
||||
One-liner: `ensure_main_window_label(window.label())`.
|
||||
|
||||
### `ensure_main_window_label(label)` (`src-tauri/src/commands/security.rs:5`)
|
||||
|
||||
Returns `Ok(())` when label is `"main"`, `Err("This command is only available from the main window (got <label>).")` otherwise.
|
||||
|
||||
### Tests (`:15`)
|
||||
|
||||
`accepts_main_window` and `rejects_secondary_windows` (the latter checks `tasks-float`, `transcript-viewer`, and `transcription-preview` are all rejected).
|
||||
|
||||
### Where `ensure_main_window` is called
|
||||
|
||||
Approximately 30 commands. The Settings flows (model loads, LLM lifecycle, hotkey wiring, intentions CRUD), most of the diagnostics report flows, the transcription / live commands, the windows-builder commands, the audio-capture commands, the prefs save, the updater stubs.
|
||||
|
||||
NOT called from: profiles, transcripts, tasks (most), feedback, hardware, meeting, fs, clipboard, hardware, tts, hotkey-status probe, llm-status probe, llm content-tag extraction, get_os_info. Each omission is intentional today (secondary windows need to call them) but document the choice when adding new commands.
|
||||
|
||||
## Data flow
|
||||
|
||||
Both modules are passive utilities. `PowerAssertion` is RAII: hold it for the duration of the work, drop it (explicitly or by leaving scope) to end the pin. `ensure_main_window` is a synchronous gate at the top of a command body.
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **`PowerAssertion` is a no-op on Linux and Windows.** Long sessions on those platforms can be idled. Document this in the user-facing Settings → AI page if you ship without a Linux logind / Windows `SetThreadExecutionState` implementation. The macOS objc2 binding requires `objc2 = "0.6.4"` and `objc2-foundation = "0.3.2"` — bumping them is an ABI move.
|
||||
- **The registry is process-wide and locked behind a `Mutex`.** Acquiring or releasing an assertion takes the mutex. Acceptable for single-digit assertions per session; if you ever spam-create assertions, this will contend.
|
||||
- **`ensure_main_window` rejects with a static message containing the offending label.** Include the label so debugging via the frontend toast is obvious; do not log the full window or any secret data.
|
||||
- **The ACL plus `ensure_main_window` is two layers.** Removing the second layer is "defence-in-depth removal", which the in-repo code review (`docs/code-review-2026-04-22.md`) flags as a non-trivial security regression. Don't.
|
||||
|
||||
## See also
|
||||
|
||||
- [App lifecycle](../app-lifecycle.md) — the panic hook and runtime warnings live alongside power.
|
||||
- [Diagnostics](diagnostics.md) — the report bundler reads `active_assertions_snapshot()`.
|
||||
- [Capabilities and ACL](../capabilities-and-acl.md) — the ACL is the first layer; this is the second.
|
||||
- [Live](live.md), [LLM](llm.md) — the main consumers of `PowerAssertion`.
|
||||
- [Tests](../tests.md) — `accepts_main_window` and `rejects_secondary_windows` plus `power_assertion_is_a_no_op_drop`.
|
||||
91
docs/architecture-map/02-tauri-runtime/commands/profiles.md
Normal file
91
docs/architecture-map/02-tauri-runtime/commands/profiles.md
Normal file
@@ -0,0 +1,91 @@
|
||||
---
|
||||
name: Profiles
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# `commands::profiles`
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → Profiles
|
||||
|
||||
**Plain English summary.** Task 12 storage-backed profiles + per-profile vocabulary terms. Profiles are the unit that scopes a Whisper initial_prompt, a vocabulary list (biases the decoder toward correct spellings), HITL feedback exemplars, and transcripts. Nine commands: profile CRUD (5), profile-term CRUD (3), and the auto-learn pass that diffs an original transcript against an edited transcript and persists likely vocabulary corrections.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Path: `src-tauri/src/commands/profiles.rs`.
|
||||
- LOC: 185.
|
||||
- Tauri commands exposed (9 total):
|
||||
- `list_profiles_cmd(state) -> Result<Vec<ProfileDto>, String>`.
|
||||
- `get_profile_cmd(state, id) -> Result<Option<ProfileDto>, String>`.
|
||||
- `create_profile_cmd(state, name, initial_prompt) -> Result<ProfileDto, String>`.
|
||||
- `update_profile_cmd(state, id, name, initial_prompt) -> Result<(), String>`.
|
||||
- `delete_profile_cmd(state, id) -> Result<(), String>`. The Default profile is guarded at the storage layer (SQLite triggers + Rust pre-checks).
|
||||
- `list_profile_terms_cmd(state, profile_id) -> Result<Vec<ProfileTermDto>, String>`.
|
||||
- `add_profile_term_cmd(state, profile_id, term, note) -> Result<ProfileTermDto, String>`.
|
||||
- `learn_profile_terms_from_edit_cmd(state, profile_id, original_text, edited_text) -> Result<Vec<ProfileTermDto>, String>`.
|
||||
- `delete_profile_term_cmd(state, id) -> Result<(), String>`.
|
||||
- Events emitted: none.
|
||||
- Depends on: `magnotia_storage::{create_profile, update_profile, delete_profile, list_profiles, get_profile, add_profile_term, list_profile_terms, delete_profile_term, ProfileRow, ProfileTermRow}`, `magnotia_ai_formatting::extract_corrections`.
|
||||
- Called from frontend at: Settings → Profiles (full CRUD), profile picker, History viewer (the auto-learn flow runs after the user saves an edit).
|
||||
|
||||
## What's in here
|
||||
|
||||
### `ProfileDto` and `ProfileTermDto` (`src-tauri/src/commands/profiles.rs:31`, `:51`)
|
||||
|
||||
camelCase mirrors of the storage rows. `ProfileDto.initialPrompt` is the saved Whisper prompt; `ProfileTermDto.term` is the vocabulary entry, `note` is a freeform note (used to mark `"Auto-learned from transcript edit"` for the auto-learn flow).
|
||||
|
||||
### `AUTO_LEARNED_NOTE` (`:25`)
|
||||
|
||||
Constant so the auto-learn rows are uniformly tagged.
|
||||
|
||||
### CRUD wrappers
|
||||
|
||||
- `list_profiles_cmd` (`:72`), `get_profile_cmd` (`:82`), `create_profile_cmd` (`:93`), `update_profile_cmd` (`:105`), `delete_profile_cmd` (`:117`). Pure passthroughs to storage.
|
||||
- `list_profile_terms_cmd` (`:127`), `add_profile_term_cmd` (`:138`), `delete_profile_term_cmd` (`:177`). Same pattern.
|
||||
|
||||
### `learn_profile_terms_from_edit_cmd` (`:151`)
|
||||
|
||||
1. Pull the existing terms (so the diff doesn't propose duplicates).
|
||||
2. Call `magnotia_ai_formatting::extract_corrections(&original, &edited, &existing_terms)`.
|
||||
3. Persist each new term via `add_profile_term` with the `AUTO_LEARNED_NOTE`.
|
||||
4. Return the freshly-inserted DTOs.
|
||||
|
||||
The actual diff heuristic lives in the formatting crate; the command file is just wiring.
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
Settings Profiles tab -> list_profiles_cmd -> [ProfileDto, ...]
|
||||
profile picker -> get_profile_cmd(id)
|
||||
add profile -> create_profile_cmd(name, prompt) -> ProfileDto
|
||||
edit profile -> update_profile_cmd(id, name, prompt) -> ()
|
||||
delete profile -> delete_profile_cmd(id) -> () (Default is rejected at storage)
|
||||
|
||||
Profile terms tab -> list_profile_terms_cmd(profile_id)
|
||||
add term -> add_profile_term_cmd(profile_id, term, note) -> ProfileTermDto
|
||||
delete term -> delete_profile_term_cmd(id) -> ()
|
||||
|
||||
History viewer save edit:
|
||||
-> learn_profile_terms_from_edit_cmd(profile_id, original, edited)
|
||||
-> existing terms from DB
|
||||
-> extract_corrections(original, edited, existing) -> [String]
|
||||
-> add each as a profile term tagged "Auto-learned from transcript edit"
|
||||
-> [ProfileTermDto, ...]
|
||||
```
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **No `ensure_main_window` guard.** The History viewer is a secondary window (`transcript-viewer`) and needs to call `learn_profile_terms_from_edit_cmd` after a save. So the whole module is callable from anywhere with the secondary-windows capability. If you want to lock down profile *creation* / *deletion*, add `ensure_main_window` to the destructive ones.
|
||||
- **No `PowerAssertion`.** No inference here.
|
||||
- **`extract_corrections` runs synchronously (in the async fn).** Acceptable for the small-text shape of a single transcript edit.
|
||||
- **The Default profile is unkillable.** SQLite trigger rejects the delete. Frontend should grey out the delete button when `profile.id == DEFAULT_PROFILE_ID`.
|
||||
- **Auto-learned terms are recorded one at a time inside a loop** (`:166`). Each iteration is a separate insert. Acceptable; if a future heuristic produces dozens of terms per edit, batch.
|
||||
|
||||
## See also
|
||||
|
||||
- [`commands::mod`](mod.md) — `build_initial_prompt` consumes profile prompt + terms.
|
||||
- [Transcription](transcription.md) — fetches profile + terms before transcribing.
|
||||
- [Live transcription](live.md) — same upstream.
|
||||
- [Tasks](tasks.md) — feedback exemplars are profile-scoped.
|
||||
- [Transcripts](transcripts.md) — every transcript carries a `profileId`.
|
||||
@@ -0,0 +1,110 @@
|
||||
---
|
||||
name: Small command modules
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# Small command modules
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → Small commands
|
||||
|
||||
**Plain English summary.** Seven short modules that each declare one or two commands. Grouped here to keep the slice navigable. Covers clipboard, filesystem write, hardware probe, meeting auto-detect poll, nudges, rituals (morning triage), and the updater stub.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Modules covered:
|
||||
- `clipboard.rs` (11 LOC, 1 command).
|
||||
- `fs.rs` (44 LOC, 1 command).
|
||||
- `hardware.rs` (69 LOC, 2 commands).
|
||||
- `meeting.rs` (50 LOC, 1 command).
|
||||
- `nudges.rs` (63 LOC, 1 command).
|
||||
- `rituals.rs` (43 LOC, 2 commands).
|
||||
- `update.rs` (16 LOC, 2 commands).
|
||||
- Total: 11 commands across 7 small files.
|
||||
|
||||
## `clipboard.rs`
|
||||
|
||||
### `copy_to_clipboard(text: String) -> Result<(), String>` (`src-tauri/src/commands/clipboard.rs:5`)
|
||||
|
||||
Wraps `arboard::Clipboard::set_text`. No window guard. The frontend dictation path calls this when the user wants clipboard-only (no auto-paste), and the History viewer uses it for the copy button. Pairs with `commands::paste::paste_text` for the auto-paste flow.
|
||||
|
||||
Watch-out: arboard initialisation can fail on Linux Wayland environments without an X11 selection daemon; the error is propagated as `"Clipboard init failed: ..."`.
|
||||
|
||||
## `fs.rs`
|
||||
|
||||
### `write_text_file_cmd(path: String, contents: String) -> Result<(), String>` (`src-tauri/src/commands/fs.rs:13`)
|
||||
|
||||
Phase 9. Thin filesystem write for the save-dialog path. `tokio::fs::write(&path, contents)` with the path attached to the error message so the frontend toast is actionable. The caller is expected to obtain `path` via the OS save dialog (`tauri-plugin-dialog`); no traversal validation here because the dialog already constrains the user's choice.
|
||||
|
||||
Tests (`fs.rs:19`): `write_text_file_roundtrips_utf8` covers UTF-8 round-trip including non-ASCII; `write_text_file_errors_on_bad_parent` covers a non-existent parent directory.
|
||||
|
||||
## `hardware.rs`
|
||||
|
||||
### `SystemInfo` and `ModelRecommendation`
|
||||
|
||||
Frontend-facing structs. `SystemInfo` carries `ram_mb`, `cpu_brand`, `cpu_cores`, `os`, `gpu`. `ModelRecommendation` carries id / display_name / disk_size_mb / ram_required_mb / description / score / reason / is_downloaded.
|
||||
|
||||
### `probe_system() -> Result<SystemInfo, String>` (`src-tauri/src/commands/hardware.rs:29`)
|
||||
|
||||
Wraps `magnotia_core::hardware::probe_system`. Maps the OS enum to a string and the GPU vendor (if probed) to its `Debug` form.
|
||||
|
||||
### `rank_models() -> Result<Vec<ModelRecommendation>, String>` (`src-tauri/src/commands/hardware.rs:49`)
|
||||
|
||||
Calls `magnotia_core::recommendation::rank_recommendations` and decorates each entry with `magnotia_transcription::is_downloaded`. Used by Settings → Models for the "recommended for your hardware" list.
|
||||
|
||||
Both commands are unguarded (any window can call). Pure read-only probes.
|
||||
|
||||
## `meeting.rs`
|
||||
|
||||
### `MeetingState` (`src-tauri/src/commands/meeting.rs:16`)
|
||||
|
||||
Tauri-managed: `lister: Mutex<ProcessLister>`. Holds a long-lived `ProcessLister` so each poll refreshes the existing `sysinfo::System` in place rather than rebuilding the process table from scratch every 15 seconds (the previous implementation did).
|
||||
|
||||
### `detect_meeting_processes(state, patterns: Vec<String>) -> Result<Vec<String>, String>` (`src-tauri/src/commands/meeting.rs:34`)
|
||||
|
||||
Phase 8 meeting auto-capture (single-signal variant). Frontend polls this on an interval with the user's app patterns. On a positive hit, the frontend surfaces a non-modal toast that reminds the user to start recording with their hotkey. We do NOT start recording from this signal — the user decides.
|
||||
|
||||
If `patterns` is empty, returns an empty Vec without locking. Otherwise locks the `lister`, snapshots the process list, and runs `magnotia_core::process_watch::match_meeting_patterns` to filter. Returns the matched process names.
|
||||
|
||||
Watch-out: the `ProcessLister` lock is `std::sync::Mutex`. If the snapshot ever takes meaningful time, switch to async.
|
||||
|
||||
## `nudges.rs`
|
||||
|
||||
### `DeliverNudgeInput` (`src-tauri/src/commands/nudges.rs:28`)
|
||||
|
||||
`{ title: String, body: String }`.
|
||||
|
||||
### `deliver_nudge(app, window, input: DeliverNudgeInput) -> Result<(), String>` (`src-tauri/src/commands/nudges.rs:42`)
|
||||
|
||||
Phase 6. Main-window only via `ensure_main_window`. Trims title and body; if both are empty, return Ok silently (a blank nudge is worse than no nudge). Defaults the title to `"Magnotia"` if only the body is present. Calls `tauri_plugin_notification::NotificationExt::notification().builder().title(...).body(...).show()`.
|
||||
|
||||
The frontend nudge bus (`nudgeBus.svelte.ts`) owns cadence, suppression, and the hourly cap. This command is a blunt "push it now" primitive — no rate limiting at the Rust layer. Errors propagate verbatim so the bus can log + swallow.
|
||||
|
||||
## `rituals.rs`
|
||||
|
||||
### Morning-triage sentinel
|
||||
|
||||
Frontend owns rendering and logic; this module only persists the "last date the morning triage modal was shown" sentinel under SQLite settings key `magnotia_morning_triage_last_shown`.
|
||||
|
||||
- `get_last_morning_triage(state) -> Result<Option<String>, String>` (`src-tauri/src/commands/rituals.rs:23`).
|
||||
- `mark_morning_triage_shown(state, date: String) -> Result<(), String>` (`src-tauri/src/commands/rituals.rs:36`). Caller passes a YYYY-MM-DD string in the user's local timezone — Rust deliberately stays timezone-agnostic.
|
||||
|
||||
No `ensure_main_window` guard. Acceptable: the morning triage UI lives in the main window, but the data is harmless if a secondary window ever queries.
|
||||
|
||||
## `update.rs`
|
||||
|
||||
Updater stubs.
|
||||
|
||||
- `check_for_update(window) -> Result<Option<String>, String>` (`src-tauri/src/commands/update.rs:6`). Main-window only. Currently always returns `Ok(None)` ("up to date").
|
||||
- `install_update(window) -> Result<(), String>` (`src-tauri/src/commands/update.rs:13`). Main-window only. Currently returns `Err("Updates are disabled until release signing is configured.")`.
|
||||
|
||||
The matching `tauri.conf.json` `plugins.updater` entry is absent. The integration test `updater_is_signed_or_absent` (`src-tauri/tests/config_hardening.rs:35`) gates a future release: any updater config that ships must carry a non-empty `pubkey`.
|
||||
|
||||
## See also
|
||||
|
||||
- [Paste](paste.md) — the auto-paste sibling of `clipboard::copy_to_clipboard`.
|
||||
- [Diagnostics](diagnostics.md) — the report bundler that consumes the same OS info `hardware::probe_system` reports.
|
||||
- [Tauri config](../tauri-config.md) — the absent `plugins.updater` block that the integration test guards.
|
||||
- [Capabilities and ACL](../capabilities-and-acl.md) — the notification permission set that `nudges::deliver_nudge` relies on.
|
||||
- [Power assertions and security](power-and-security.md) — `ensure_main_window` guards live here.
|
||||
119
docs/architecture-map/02-tauri-runtime/commands/tasks.md
Normal file
119
docs/architecture-map/02-tauri-runtime/commands/tasks.md
Normal file
@@ -0,0 +1,119 @@
|
||||
---
|
||||
name: Tasks and decomposition
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# `commands::tasks`
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → Tasks
|
||||
|
||||
**Plain English summary.** Eleven commands wrapping the `magnotia_storage` task CRUD plus two LLM-driven actions. Standard CRUD: create / list / update / complete / uncomplete / delete a task, plus subtask CRUD (insert / list / complete) and a daily-completion-counts query for the Phase 8 Tasks-page momentum sparkline. The two LLM actions: `decompose_and_store` (break a parent task into subtasks via the local LLM with HITL feedback as few-shot exemplars) and `extract_tasks_from_transcript_cmd` (extract task lines from a recently-finished transcript, again with feedback exemplars).
|
||||
|
||||
## At a glance
|
||||
|
||||
- Path: `src-tauri/src/commands/tasks.rs`.
|
||||
- LOC: 402.
|
||||
- Tauri commands exposed (11 total):
|
||||
- `create_task_cmd(state, request: CreateTaskRequest) -> Result<TaskDto, String>`.
|
||||
- `list_tasks_cmd(state) -> Result<Vec<TaskDto>, String>`.
|
||||
- `update_task_cmd(state, id, patch: UpdateTaskRequest) -> Result<TaskDto, String>`. Patch shape (text / bucket / list_id / effort / notes).
|
||||
- `complete_task_cmd(state, id) -> Result<(), String>`. Stamps server-side `done_at`.
|
||||
- `delete_task_cmd(state, id) -> Result<(), String>`.
|
||||
- `uncomplete_task_cmd(state, id) -> Result<(), String>`. Clears `done_at`.
|
||||
- `set_task_energy_cmd(state, id, energy: Option<String>) -> Result<TaskDto, String>`. Always writes; passes `None` to clear.
|
||||
- `decompose_and_store(state, parent_task_id, profile_id) -> Result<Vec<TaskDto>, String>`.
|
||||
- `extract_tasks_from_transcript_cmd(state, transcript, profile_id) -> Result<Vec<String>, String>`.
|
||||
- `list_subtasks_cmd(state, parent_task_id) -> Result<Vec<TaskDto>, String>`.
|
||||
- `complete_subtask_cmd(state, subtask_id) -> Result<(), String>`.
|
||||
- `list_recent_completions_cmd(state, days) -> Result<Vec<DailyCompletionCount>, String>`.
|
||||
- Events emitted: none.
|
||||
- Depends on: `magnotia_storage::{insert_task, list_tasks, update_task, complete_task, uncomplete_task, delete_task, set_task_energy, get_task_by_id, insert_subtask, list_subtasks, complete_subtask_and_check_parent, list_recent_completions, list_feedback_examples, FeedbackTargetType, TaskRow, DailyCompletionCount, FeedbackRow}`, `magnotia_llm::prompts::FeedbackExample`, `uuid::Uuid`.
|
||||
- Called from frontend at: Tasks page (CRUD, subtasks, sparkline), dictation result panel ("Extract tasks" button), parent-task expand UI ("Decompose").
|
||||
|
||||
## What's in here
|
||||
|
||||
### `TaskDto` (`src-tauri/src/commands/tasks.rs:24`)
|
||||
|
||||
camelCase frontend shape with id, text, bucket, listId, effort, notes, done, doneAt, createdAt, sourceTranscriptId, parentTaskId, energy.
|
||||
|
||||
### Energy validation (`:60`)
|
||||
|
||||
`ENERGY_LEVELS = ["high", "medium", "brain_dead"]` — kept as a const so frontend and storage validate against the same set. SQLite migration v11 enforces the same set with a CHECK constraint.
|
||||
|
||||
### CRUD wrappers
|
||||
|
||||
- `create_task_cmd` (`:92`) inserts and re-fetches so the frontend gets the canonical row (server-side timestamps, defaulted columns).
|
||||
- `update_task_cmd` (`:140`) — patch shape, COALESCE-style updates in storage. `done` / `doneAt` are deliberately absent — those flow through `complete_task_cmd` / `uncomplete_task_cmd`.
|
||||
- `set_task_energy_cmd` (`:200`) — separate from update because update uses `COALESCE` semantics where `None` means "preserve". This command always writes, so passing `None` actually clears the column.
|
||||
|
||||
### `to_llm_examples` (`:222`)
|
||||
|
||||
Converts feedback rows from storage into the few-shot exemplar shape the LLM crate consumes. Reconstructs `input` from `context_json` (parent task text or transcript chunk). Rows with malformed `context_json` are logged and dropped instead of silently filtered, so data-integrity regressions surface.
|
||||
|
||||
### `example_char_cost` (`:263`) and `trim_to_budget` (`:276`)
|
||||
|
||||
Char budget for the few-shot exemplars: keeps the prompt under token budget regardless of how many feedback rows the user has accumulated. Cap at 5 examples AND a char budget.
|
||||
|
||||
### `decompose_and_store` (`:290`)
|
||||
|
||||
1. Fetch the parent task or 404.
|
||||
2. Pull up to 5 micro-step feedback exemplars filtered by profile, char-trimmed.
|
||||
3. `spawn_blocking` runs `engine.decompose_task_with_feedback(parent_text, &examples)`.
|
||||
4. Insert each generated step as a subtask (`uuid::v4()` for the id), re-fetch, accumulate.
|
||||
5. Return the list.
|
||||
|
||||
NO `PowerAssertion::begin` here — the App Nap pattern is consistent in `commands::llm` but not yet uniformly applied here. Flag for follow-up.
|
||||
|
||||
### `extract_tasks_from_transcript_cmd` (`:345`)
|
||||
|
||||
Same shape as `decompose_and_store` but with `FeedbackTargetType::TaskExtraction` and the engine's `extract_tasks_with_feedback`. Returns the raw task strings — the frontend decides whether to insert them.
|
||||
|
||||
### `list_subtasks_cmd` / `complete_subtask_cmd` (`:370`, `:381`)
|
||||
|
||||
`complete_subtask_cmd` calls `complete_subtask_and_check_parent` which is the storage helper that stamps the subtask done and, if all siblings are done, also stamps the parent (Phase 8 micro-completion bookkeeping).
|
||||
|
||||
### `list_recent_completions_cmd` (`:394`)
|
||||
|
||||
Fixed-length oldest-first daily counts. Empty days are explicit zeros. The Tasks-page sparkline reads this directly.
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
Tasks page mount -> list_tasks_cmd -> [TaskDto, ...]
|
||||
Tasks page create -> create_task_cmd(req) -> TaskDto
|
||||
Tasks page edit -> update_task_cmd(id, patch) -> TaskDto
|
||||
Tasks page check -> complete_task_cmd(id) -> ()
|
||||
Tasks page reopen -> uncomplete_task_cmd(id) -> ()
|
||||
Tasks page energy chip -> set_task_energy_cmd(id, energy) -> TaskDto
|
||||
Tasks page sparkline -> list_recent_completions_cmd(days) -> [{date, count}, ...]
|
||||
|
||||
Tasks page Decompose button -> decompose_and_store(parent_id, profile_id)
|
||||
-> get parent task
|
||||
-> list 5 micro-step feedback rows (profile-scoped)
|
||||
-> trim_to_budget
|
||||
-> spawn_blocking(engine.decompose_task_with_feedback)
|
||||
-> insert subtasks
|
||||
-> [TaskDto, ...]
|
||||
|
||||
Dictation panel Extract tasks -> extract_tasks_from_transcript_cmd(text, profile_id)
|
||||
-> list 5 task-extraction feedback rows
|
||||
-> spawn_blocking(engine.extract_tasks_with_feedback)
|
||||
-> [String, ...]
|
||||
```
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **No `ensure_main_window` guard.** Tasks UI lives in the main window AND in the always-on-top task float (which has the secondary-windows capability). The float can call list / complete / set-energy / list-recent-completions etc. — that's intentional — but it can also fire `decompose_and_store` and `extract_tasks_from_transcript_cmd`, which spend LLM tokens. If you want to lock this down, add the guard to the LLM-spending commands.
|
||||
- **No `PowerAssertion`.** `decompose_and_store` and `extract_tasks_from_transcript_cmd` both run synchronous LLM inference for several seconds. macOS can App Nap them. Add `PowerAssertion::begin("magnotia LLM task decomposition")` and similar.
|
||||
- **The patch shape passes `Option<String>` for every column.** Currently the storage layer's `update_task` uses COALESCE: `Some` overwrites, `None` preserves. This means there's no way to clear `notes` or `effort` to empty via `update_task_cmd` — you can only set them to a non-empty string. If a user wants to clear a field, they'd need a fresh task or a dedicated clear command.
|
||||
- **Decompose stores subtasks one-by-one in a loop** (`:329`). Each iteration is a separate DB transaction. Acceptable for typical 3–7-step decompositions; if a future LLM produces 50 steps, batch the inserts.
|
||||
- **`extract_tasks_from_transcript_cmd` returns just `Vec<String>` and does NOT insert.** Frontend chooses what to insert. Confusing because the sibling `decompose_and_store` *does* insert. Convention is dictated by UX (decomposition is one-click, extraction reviews-then-inserts).
|
||||
|
||||
## See also
|
||||
|
||||
- [LLM](llm.md) — the engine that runs the decomposition / extraction.
|
||||
- [Feedback](feedback.md) — the table that feeds the few-shot exemplars.
|
||||
- [Profiles](profiles.md) — `profile_id` is the scoping key for feedback rows.
|
||||
- [Window management](windows.md) — the task-float window (`tasks-float`) that consumes `list_tasks_cmd`.
|
||||
111
docs/architecture-map/02-tauri-runtime/commands/transcription.md
Normal file
111
docs/architecture-map/02-tauri-runtime/commands/transcription.md
Normal file
@@ -0,0 +1,111 @@
|
||||
---
|
||||
name: Transcription commands
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# `commands::transcription`
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → Transcription
|
||||
|
||||
**Plain English summary.** The non-live transcription paths. Three commands: transcribe a Vec<f32> of PCM samples through Whisper, transcribe the same shape through Parakeet, and transcribe a file from disk by decoding + resampling to 16 kHz first. Both PCM commands emit `transcription-result` events; the file command returns the result inline. All three run inference on a blocking thread, run the formatting pipeline against the output, and respect profile prompt + vocabulary precedence via the shared `build_initial_prompt` helper.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Path: `src-tauri/src/commands/transcription.rs`.
|
||||
- LOC: 413.
|
||||
- Tauri commands exposed:
|
||||
- `transcribe_pcm(window, state, app, samples: Vec<f32>, chunk_id: u32, language, initial_prompt, remove_fillers, british_english, anti_hallucination, format_mode, profile_id) -> Result<(), String>` — main-window only. Whisper PCM. Emits `transcription-result`.
|
||||
- `transcribe_file(window, state, path, engine: Option<String>, model_id: Option<String>, language, initial_prompt, remove_fillers, british_english, anti_hallucination, format_mode, profile_id) -> Result<serde_json::Value, String>` — main-window only. Decodes the file, picks engine (default whisper), returns the result inline.
|
||||
- `transcribe_pcm_parakeet(window, state, app, samples, chunk_id, remove_fillers, british_english, anti_hallucination, format_mode, profile_id) -> Result<(), String>` — main-window only. Parakeet PCM. Emits `transcription-result`.
|
||||
- Events emitted: `transcription-result` (payload: `{ status: "transcription", segments, language, duration, chunk_id, inference_ms, raw_text }`) — fires from `transcribe_pcm` (`src-tauri/src/commands/transcription.rs:208`) and `transcribe_pcm_parakeet` (`:398`).
|
||||
- Depends on: `magnotia_audio::{decode_audio_file_limited, resample_to_16khz, probe_audio_duration_secs}`, `magnotia_core::types::{AudioSamples, Segment, Transcript, TranscriptionOptions}`, `magnotia_transcription::{LocalEngine, TimedTranscript}`, `magnotia_ai_formatting::{post_process_segments, FormatMode, PostProcessOptions}`, `magnotia_storage::{database, DEFAULT_PROFILE_ID}`. Plus `commands::build_initial_prompt`, `commands::models::{default_model_id_for_engine, ensure_model_loaded}`, `commands::security::ensure_main_window`.
|
||||
- Called from frontend at: dictation page (PCM commands when not live), file-import flow (transcribe_file), Settings test page (file command for QA).
|
||||
|
||||
## What's in here
|
||||
|
||||
### Constants (`src-tauri/src/commands/transcription.rs:18`)
|
||||
|
||||
Chunking thresholds:
|
||||
|
||||
- Parakeet: chunk anything over 18 s into 15 s windows with 1 s overlap.
|
||||
- Generic file: chunk anything over 8 minutes into 3-minute windows with 2 s overlap.
|
||||
- Hard cap: `MAX_FILE_TRANSCRIPTION_SECS = 2 * 60 * 60` (2 hours).
|
||||
|
||||
### `pick_engine` (`:31`)
|
||||
|
||||
Maps `"whisper"` and `"parakeet"` to the relevant `Arc<LocalEngine>` from `AppState`.
|
||||
|
||||
### `pick_chunking_strategy` (`:42`)
|
||||
|
||||
Returns the `ChunkingStrategy` to use based on engine + sample count, or `None` for "no chunking needed".
|
||||
|
||||
### `trim_overlap_segments` (`:61`)
|
||||
|
||||
For chunks past the first, drops segments that end before `trim_before_secs` and clamps remaining starts. The same logic appears in `commands::live` for the live-mode path.
|
||||
|
||||
### `transcribe_samples_sync` (`:74`)
|
||||
|
||||
The shared inner that the file path uses. If no chunking is needed, calls `engine.transcribe_sync` once. Otherwise loops over chunks, runs each through inference, trims overlap, offsets timestamps by the chunk start, accumulates segments and inference_ms. Returns a synthesised `TimedTranscript`.
|
||||
|
||||
### `transcribe_pcm` (`:142`)
|
||||
|
||||
Whisper-specific PCM path (no chunking — frontend is expected to keep the buffer reasonable, e.g. ≤ 30 s for the Whisper context window):
|
||||
|
||||
1. `ensure_main_window`.
|
||||
2. Resolve `profile_id` (default `DEFAULT_PROFILE_ID`).
|
||||
3. Fetch `ProfileRow` and profile term list from `magnotia_storage::database`.
|
||||
4. Build effective Whisper prompt via `build_initial_prompt(&caller_prompt, &profile.initial_prompt, &profile_terms)`.
|
||||
5. `spawn_blocking` runs `engine.transcribe_sync` on the samples.
|
||||
6. Run `post_process_segments` (filler removal, British English conversion, anti-hallucination, format mode, dictionary terms, optional LLM cleanup via `state.llm_engine`).
|
||||
7. `app.emit("transcription-result", ...)` with raw_text (pre-post-process) plus the post-processed segments.
|
||||
|
||||
### `transcribe_file` (`:236`)
|
||||
|
||||
1. `ensure_main_window`.
|
||||
2. Resolve profile + terms (same as PCM path).
|
||||
3. Default engine to `"whisper"`, default model id to `default_model_id_for_engine(&engine_name)`.
|
||||
4. `ensure_model_loaded(state, engine, model_id, None)` — None = no sequential-GPU guard.
|
||||
5. Probe audio duration via `magnotia_audio::probe_audio_duration_secs`. If > 2 hours, return a friendly error.
|
||||
6. `spawn_blocking` decodes the file (`decode_audio_file_limited(path, Some(MAX_FILE_TRANSCRIPTION_SECS))`), resamples to 16 kHz mono, then runs `transcribe_samples_sync`.
|
||||
7. Run `post_process_segments`.
|
||||
8. Return a JSON value with `engine`, `modelId`, `segments`, `language`, `duration`, `inference_ms`, `raw_text`.
|
||||
|
||||
### `transcribe_pcm_parakeet` (`:342`)
|
||||
|
||||
Parakeet PCM path. Skips the `initial_prompt` construction (Parakeet doesn't have a Whisper-style prompt) but still validates the profile exists and gathers the dictionary terms for post-processing. Calls `transcribe_samples_sync(parakeet_engine, "parakeet", samples, options)` so the > 18 s chunking kicks in if relevant. Emits `transcription-result` like the Whisper path.
|
||||
|
||||
### `join_segment_text` (`:225`)
|
||||
|
||||
Concatenates segment text with whitespace stripping for the `raw_text` field on the emitted event.
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
frontend invoke('transcribe_pcm', { samples, chunk_id, ..., profile_id })
|
||||
-> ensure_main_window
|
||||
-> get_profile + list_profile_terms (DB)
|
||||
-> build_initial_prompt
|
||||
-> spawn_blocking(engine.transcribe_sync(samples, options)) -> TimedTranscript
|
||||
-> post_process_segments(segments, opts, llm_engine) -- in-place
|
||||
-> app.emit("transcription-result", { segments, raw_text, ... })
|
||||
```
|
||||
|
||||
`transcribe_file` is the same shape with `decode_audio_file_limited` + `resample_to_16khz` + `transcribe_samples_sync` substituted for the inline `transcribe_sync`.
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **`transcribe_pcm` does NOT call `ensure_model_loaded`.** It assumes the model is already loaded (which is true when the frontend has driven the Settings → load flow, and is true after `prewarm_default_model_cmd`). If a third caller invokes `transcribe_pcm` without a load step, you'll get a runtime panic from `engine.transcribe_sync` with no clear message. The file path *does* call `ensure_model_loaded`. Consider adding the same guard to `transcribe_pcm` for symmetry.
|
||||
- **`transcribe_file` returns a `serde_json::Value` rather than a typed DTO.** The shape is fully ad-hoc (`engine`, `modelId`, `segments`, `language`, `duration`, `inference_ms`, `raw_text`). The frontend has to keep this shape in sync by hand. Consider promoting it to a typed struct.
|
||||
- **Chunking does not emit per-chunk events from this command.** The file path returns the entire concatenated result at the end. If a 90-minute file fails three quarters of the way through, the user gets nothing. Live mode (`commands::live`) is the streaming alternative.
|
||||
- **The 2-hour cap is a hard constant.** Document it in the frontend "import audio" flow.
|
||||
- **The Parakeet PCM path does not respect `language` or `initial_prompt`** because Parakeet has no equivalent. The frontend should disable those Settings widgets when the engine is Parakeet, or this command will silently ignore them.
|
||||
|
||||
## See also
|
||||
|
||||
- [Models](models.md) — `ensure_model_loaded`, `default_model_id_for_engine`, the `LocalEngine` cache that `state.whisper_engine` and `state.parakeet_engine` wrap.
|
||||
- [Live transcription](live.md) — the streaming sibling.
|
||||
- [Profiles](profiles.md) — feeds `profile.initial_prompt` and the term list.
|
||||
- [`commands::mod`](mod.md) — `build_initial_prompt` is the prompt assembler.
|
||||
- [Audio capture](audio.md) — supplies the `Vec<f32>` that `transcribe_pcm` consumes.
|
||||
@@ -0,0 +1,83 @@
|
||||
---
|
||||
name: Transcripts CRUD and search
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# `commands::transcripts`
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → Transcripts
|
||||
|
||||
**Plain English summary.** SQLite-backed transcripts: insert, paginated list, count, get, update text/title, patch metadata (starred, manual_tags, template, language, segments_json, llm_tags), delete, and FTS5 search. Replaces the old localStorage cache. Day 4 of the upgrade plan; the History page rename flow had a TODO waiting on `update_transcript` to exist, and now it does. The legacy global-dictionary commands (`list_dictionary_command` and friends) were removed — profile-scoped `profile_terms` is now canonical.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Path: `src-tauri/src/commands/transcripts.rs`.
|
||||
- LOC: 253.
|
||||
- Tauri commands exposed (8 total):
|
||||
- `add_transcript(state, transcript: CreateTranscriptRequest) -> Result<(), String>`.
|
||||
- `list_transcripts(state, limit, offset) -> Result<Vec<TranscriptDto>, String>` — defaults 50 rows, clamp 1..=500.
|
||||
- `count_transcripts_command(state) -> Result<i64, String>`.
|
||||
- `get_transcript(state, id) -> Result<Option<TranscriptDto>, String>`.
|
||||
- `update_transcript(state, id, text: Option<String>, title: Option<String>) -> Result<u64, String>` — returns affected row count (0 if id not found).
|
||||
- `update_transcript_meta_cmd(state, id, patch: UpdateTranscriptMetaRequest) -> Result<TranscriptDto, String>` — patch shape with COALESCE semantics.
|
||||
- `delete_transcript(state, id) -> Result<(), String>`.
|
||||
- `search_transcripts(state, query) -> Result<Vec<TranscriptDto>, String>` — FTS5; up to 50 best-rank.
|
||||
- Events emitted: none.
|
||||
- Depends on: `magnotia_storage::{insert_transcript, list_transcripts_paged, count_transcripts, get_transcript, update_transcript, update_transcript_meta, delete_transcript, search_transcripts, InsertTranscriptParams, TranscriptRow, DEFAULT_PROFILE_ID}`.
|
||||
- Called from frontend at: History page (list / search / get / update / delete), dictation result panel (`add_transcript`), Settings → About (`count_transcripts_command`), History viewer window (`update_transcript_meta_cmd` for star / template / language / llm_tags).
|
||||
|
||||
## What's in here
|
||||
|
||||
### `TranscriptDto` (`src-tauri/src/commands/transcripts.rs:35`)
|
||||
|
||||
camelCase frontend shape. The Task 2.5 fields (`starred`, `manualTags`, `template`, `language`, `segmentsJson`) were added when localStorage was retired. Phase 9 added `llmTags`.
|
||||
|
||||
### `CreateTranscriptRequest` (`:79`)
|
||||
|
||||
Wide shape covering everything an insert needs: id, text, source, profile_id, title, audio_path, duration, engine, model_id, inference_ms, sample_rate, audio_channels, format_mode, plus the post-processing flags (remove_fillers, british_english, anti_hallucination).
|
||||
|
||||
### `UpdateTranscriptMetaRequest` (`:215`)
|
||||
|
||||
Patch shape for Task 2.5 / Phase 9 metadata: each field is `Option`. `None` preserves via COALESCE; `Some` overwrites. `Some("")` explicitly clears (relevant for `manual_tags` and `llm_tags`).
|
||||
|
||||
### Commands
|
||||
|
||||
- `add_transcript` (`:103`) builds an `InsertTranscriptParams` from the wide request and calls `magnotia_storage::insert_transcript`. The FTS5 index is updated automatically by an SQLite trigger inside the storage crate.
|
||||
- `list_transcripts` (`:135`) — paginated with sane defaults.
|
||||
- `count_transcripts_command` (`:150`).
|
||||
- `get_transcript` (`:157`).
|
||||
- `update_transcript` (`:172`) — fixes the long-standing "rename in History never persists" bug per architecture-review.md §13.
|
||||
- `delete_transcript` (`:184`).
|
||||
- `search_transcripts` (`:196`) — empty / whitespace queries return empty results without hitting the DB. FTS5 query syntax (bare words AND together, quoted phrases, OR / NOT / prefix `*`) flows through verbatim.
|
||||
- `update_transcript_meta_cmd` (`:235`) — patch the meta columns and return the updated row.
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
dictation result panel -> add_transcript(req) -> insert_transcript -> FTS5 trigger updates
|
||||
History page mount -> list_transcripts(50, 0)
|
||||
-> count_transcripts_command (for "showing 50 of 1273")
|
||||
History page rename -> update_transcript(id, text=None, title=Some(new))
|
||||
History page delete -> delete_transcript(id)
|
||||
History page search box -> search_transcripts(q)
|
||||
Viewer window star toggle -> update_transcript_meta_cmd(id, { starred: Some(true) })
|
||||
Viewer window LLM tag pass -> update_transcript_meta_cmd(id, { llm_tags: Some(joined) })
|
||||
```
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **No `ensure_main_window` guard.** Both the main window AND the `transcript-viewer` secondary window need to call these commands, and the secondary-windows capability does include `core:event:default`. The DB layer is the only enforcement. If you ever want to hide certain transcripts (private profile?), add a profile-id check at the command layer.
|
||||
- **`segmentsJson` is a JSON string, not a parsed array.** The shape is whatever `commands::transcription` and `commands::live` chose to write at insert time. The frontend re-parses. If you ever change the segment shape, write a migration that touches the existing rows.
|
||||
- **`update_transcript` returns the affected row count (`u64`), not the updated row.** Callers that need the new shape have to follow up with `get_transcript`. `update_transcript_meta_cmd` returns the row directly. Consider unifying.
|
||||
- **FTS5 search caps at 50 results.** History page would need pagination over search if a transcripts library grows beyond a few hundred rows.
|
||||
- **`profile_id` defaults to `DEFAULT_PROFILE_ID` at insert time** if the request omits it. The frontend should be passing the active profile; if it ever silently omits this, all transcripts pile into the default profile.
|
||||
|
||||
## See also
|
||||
|
||||
- [Transcription](transcription.md) — the upstream that produces the segments + raw_text payload that lands in `add_transcript`.
|
||||
- [Live transcription](live.md) — same upstream.
|
||||
- [LLM](llm.md) — `extract_content_tags_cmd` produces the value that goes into `llm_tags`.
|
||||
- [Profiles](profiles.md) — `profile_id` scoping.
|
||||
- [Diagnostics](diagnostics.md) — `count_transcripts_command` is referenced from the Settings → About counts row.
|
||||
95
docs/architecture-map/02-tauri-runtime/commands/tts.md
Normal file
95
docs/architecture-map/02-tauri-runtime/commands/tts.md
Normal file
@@ -0,0 +1,95 @@
|
||||
---
|
||||
name: Text-to-speech
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# `commands::tts`
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → TTS
|
||||
|
||||
**Plain English summary.** Phase 4: Read Page Aloud. Shells out to the platform's built-in TTS binary (`spd-say` with espeak-ng fallback on Linux, `say` on macOS, PowerShell `System.Speech.Synthesis.SpeechSynthesizer` via `-EncodedCommand` on Windows). Stores the spawned child process so a second tap stops in-flight speech cleanly. User text is never interpolated into a shell string — every backend passes it via argv (or, on Windows, inside a here-string delivered as base64 UTF-16-LE).
|
||||
|
||||
## At a glance
|
||||
|
||||
- Path: `src-tauri/src/commands/tts.rs`.
|
||||
- LOC: 444.
|
||||
- Tauri commands exposed:
|
||||
- `tts_speak(state, text: String, rate: f32, voice: Option<String>) -> Result<(), String>`. No window guard (any window can request TTS; the read-aloud feature lives in the main window today but secondary windows are cheap to allowlist for).
|
||||
- `tts_stop(state) -> Result<(), String>`. No window guard.
|
||||
- `tts_list_voices() -> Result<Vec<TtsVoice>, String>`.
|
||||
- Events emitted: none.
|
||||
- Depends on: per-platform TTS binaries (no Rust dependencies beyond `std::process::Command` and `serde`). Windows path uses `base64 = "0.22"` to encode UTF-16-LE for `-EncodedCommand`.
|
||||
- Called from frontend at: dictation result panel ("Read aloud" toggle), Settings → Accessibility → TTS voice picker.
|
||||
|
||||
## What's in here
|
||||
|
||||
### `TtsState` (`src-tauri/src/commands/tts.rs:20`)
|
||||
|
||||
Tauri-managed: `child: Mutex<Option<Child>>`. On Linux `spd-say` returns immediately so the slot is usually empty; macOS `say` and Windows PowerShell speak synchronously, so the handle is retained.
|
||||
|
||||
### `TtsVoice` (`:32`)
|
||||
|
||||
`{ id, name, language: Option<String> }`. Frontend renders the list as a dropdown.
|
||||
|
||||
### Rate clamping and per-platform mapping
|
||||
|
||||
- `clamp_rate(rate)` (`:40`) — clamps to `[0.5, 2.0]`, returns 1.0 for NaN.
|
||||
- Linux: `spd_rate(rate)` maps to `[-100, 100]` (asymmetric: 1.0→0, 2.0→100, 0.5→-50). `espeak_rate(rate)` maps to words-per-minute in `[80, 450]`.
|
||||
- macOS: `say_rate(rate)` maps to wpm.
|
||||
- Windows: `win_rate(rate)` maps to `[-10, 10]` (the SAPI rate scale).
|
||||
|
||||
### Per-platform spawners
|
||||
|
||||
- `spawn_linux` (`:68`) — tries `spd-say -r <rate> [-t voice] -- <text>`. On `NotFound` falls back to `espeak-ng -s <wpm> [-v voice] -- <text>`. Returns `Some(child)` only for espeak-ng (which speaks synchronously); spd-say returns `None`.
|
||||
- `spawn_macos` (`:125`) — `say -r <wpm> [-v voice] -- <text>`. Returns the child.
|
||||
- `spawn_windows` (`:158`) — assembles a PowerShell here-string with the user's text inside, base64-encodes UTF-16-LE, invokes `powershell -NoProfile -EncodedCommand <b64>`. Uses `escape_ps_herestring` (`:153`) to defuse the `'@` here-string terminator if the user's text contains it.
|
||||
|
||||
### Voice listing
|
||||
|
||||
`tts_list_voices` is a thin wrapper. Per-platform implementations (`:197`, `:206`, `:237`, `:296`):
|
||||
|
||||
- Linux: queries `spd-say -L` (or returns `[]` if missing), parses output into `TtsVoice`s.
|
||||
- macOS: runs `say -v ?`, parses each line via `parse_macos_voices` (testable pure helper at `:220`).
|
||||
- Windows: queries the .NET `[System.Speech.Synthesis.SpeechSynthesizer]::new().GetInstalledVoices()` via PowerShell, parses CSV.
|
||||
- Other platforms: returns `[]` with a message.
|
||||
|
||||
### `tts_speak`, `tts_stop`, `tts_list_voices` (`:302`, `:345`, `:362`)
|
||||
|
||||
`tts_speak` trims the text, kills any in-flight child via `kill_child`, dispatches per-platform. Stores a returned child if any. Returns an explicit error on platforms where TTS is not implemented (e.g. Android).
|
||||
|
||||
`tts_stop` calls `kill_child` and additionally calls `stop_linux()` (`:102`) — which fires `spd-say -S` to ask speech-dispatcher to flush its own queue, since spd-say-spawned children are not retained in `state.child`.
|
||||
|
||||
`kill_child` (`:353`) takes the child out of state, kills it, and waits for it. `wait()` is important to avoid zombies.
|
||||
|
||||
### Tests (`:367`)
|
||||
|
||||
Cover rate clamping (NaN, bounds), spd_rate / espeak_rate / say_rate / win_rate per-platform, the PowerShell here-string escape (`ps_herestring_terminator_is_broken`), and the macOS voice parser (`parses_macos_voices`).
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
frontend invoke('tts_speak', { text, rate, voice })
|
||||
-> kill any in-flight child
|
||||
-> per-OS spawn (spd-say / say / powershell)
|
||||
-> stash child handle if backend speaks synchronously
|
||||
frontend invoke('tts_stop')
|
||||
-> kill child
|
||||
-> Linux: also fire spd-say -S
|
||||
```
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **No `ensure_main_window` guard.** Today the read-aloud UI lives in the main window, but the ACL allows any window to invoke. If you decide to lock down secondary windows from triggering speech, add the guard.
|
||||
- **Linux `spd-say` is non-blocking** — `tts_stop` cannot kill its synthesis once it has handed off to speech-dispatcher. The `stop_linux` extra call asks speech-dispatcher to flush its own queue, but that's a soft-stop, not a hard kill.
|
||||
- **Windows path is heavy.** Every speak-call spawns a PowerShell. Acceptable for one-off use; for a streaming TTS pattern you'd want to keep a long-lived child or use the `windows` crate's SAPI bindings directly.
|
||||
- **Voice id semantics differ per platform.** macOS uses the voice name; Linux uses an spd-say `-t` token; Windows uses the SAPI registered voice token. Frontend treats them as opaque strings, but a saved-voice in Settings will not survive a platform switch.
|
||||
- **Brand consistency.** `Magnotia` is being renamed to `Lumenote` (per personal memory `project_lumenote_naming.md`). The TTS module currently embeds the string `"magnotia LLM cleanup"` and `"magnotia"`-prefixed temp filenames; rebrand sweep follow-up.
|
||||
- **No power assertion.** Long read-aloud sessions on macOS could be idled by App Nap. Add `PowerAssertion::begin("magnotia TTS")` to `tts_speak` if longer transcripts ever become a primary use case.
|
||||
|
||||
## See also
|
||||
|
||||
- [Power assertions and security](power-and-security.md) — App Nap pattern.
|
||||
- [LLM](llm.md) — the cleanup pipeline that produces the text fed into TTS.
|
||||
- [Cargo and features](../cargo-and-features.md) — the Windows-only `base64` dependency.
|
||||
86
docs/architecture-map/02-tauri-runtime/commands/windows.md
Normal file
86
docs/architecture-map/02-tauri-runtime/commands/windows.md
Normal file
@@ -0,0 +1,86 @@
|
||||
---
|
||||
name: Window management
|
||||
type: architecture-map-page
|
||||
slice: 02-tauri-runtime
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# `commands::windows`
|
||||
|
||||
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → Window management
|
||||
|
||||
**Plain English summary.** The imperative builder for the three secondary windows: a floating always-on-top task list, a transcript viewer / editor, and a transient transcription preview overlay. Each command is idempotent: an existing window is shown and focused; otherwise a new one is built. Linux uses native KWin/Mutter decorations; macOS and Windows draw the custom frameless chrome. The preview overlay is configured to follow the user across virtual desktops and stay out of the alt-tab list (KWin sticky + GTK Utility hint). Android stubs return a clear error so the frontend can detect and route to in-window routes instead.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Path: `src-tauri/src/commands/windows.rs`.
|
||||
- LOC: 197.
|
||||
- Tauri commands exposed (4 total):
|
||||
- `open_task_window(app) -> Result<(), String>`.
|
||||
- `open_preview_window(app) -> Result<(), String>`.
|
||||
- `close_preview_window(app) -> Result<(), String>`.
|
||||
- `open_viewer_window(app) -> Result<(), String>`.
|
||||
- Events emitted: `task-window-focus` (no payload) when `open_task_window` brings an existing task float forward (`src-tauri/src/commands/windows.rs:49`).
|
||||
- Depends on: `tauri::{Emitter, Manager, WebviewUrl, WebviewWindowBuilder}` (desktop-only). `gdk::WindowTypeHint`, `gtk::prelude::GtkWindowExt` for the Linux preview hint.
|
||||
- Called from frontend at: dictation page (preview open / close around recording), Tasks page header button (task float), History page row click (viewer).
|
||||
|
||||
## What's in here
|
||||
|
||||
### Android stubs (`src-tauri/src/commands/windows.rs:14`)
|
||||
|
||||
`#[cfg(target_os = "android")]` versions of all four commands return `Err("Multi-window is not supported on Android; this command is desktop-only")`. The frontend detects Android via `isAndroid()` and routes the formerly-secondary content into routes inside the main window.
|
||||
|
||||
### `open_task_window` (`:43`)
|
||||
|
||||
If a `tasks-float` window already exists, show + focus and emit `task-window-focus`. Otherwise build a new one at `/float`, 480×520 (min 360×480), `always_on_top: true`, decorations native on Linux / frameless elsewhere, resizable. Inject the `PreferencesScript` if present so prefs land before Svelte mounts.
|
||||
|
||||
### `open_preview_window` (`:88`)
|
||||
|
||||
If a `transcription-preview` exists, show (NOT focus — the overlay must not steal focus from whatever the user is typing into). Otherwise build a new one at `/preview`, 420×200 (min 360×140, max 520×360), `always_on_top: true`, `skip_taskbar: true`, `visible_on_all_workspaces: true` (KWin sticky), `focused: false`, `visible: false` (built hidden so the GTK type-hint can be set pre-realize). Inject prefs.
|
||||
|
||||
After the build:
|
||||
|
||||
- On Linux, fetch the `gtk_window()` and call `set_type_hint(WindowTypeHint::Utility)`. This signals to Hyprland / Sway / GNOME Mutter compositors that the window is auxiliary and should not show in alt-tab. KWin already obeys SKIP_TASKBAR; this is defence in depth for non-KDE compositors.
|
||||
|
||||
Then call `window.show()` to actually surface it.
|
||||
|
||||
### `close_preview_window` (`:158`)
|
||||
|
||||
If the preview window exists, hide it (don't destroy — the next open is instant). No-op if it doesn't exist. Idempotent return Ok.
|
||||
|
||||
### `open_viewer_window` (`:168`)
|
||||
|
||||
If a `transcript-viewer` exists, show + focus. Otherwise build a new one at `/viewer`, 600×700 (min 560×520), Linux native decorations / frameless elsewhere, resizable. Inject prefs. NOT always-on-top — the viewer is a primary surface, not an overlay.
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
Tasks page header button -> invoke('open_task_window')
|
||||
-> if exists: show + focus + emit('task-window-focus')
|
||||
-> else: WebviewWindowBuilder('/float', ..., always_on_top=true)
|
||||
|
||||
Dictation start -> invoke('open_preview_window')
|
||||
-> build hidden, set GTK Utility hint on Linux, show
|
||||
|
||||
Dictation end / paste -> invoke('close_preview_window') -> hide
|
||||
|
||||
History row click -> invoke('open_viewer_window')
|
||||
-> if exists: show + focus
|
||||
-> else: WebviewWindowBuilder('/viewer', ...)
|
||||
```
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **No `ensure_main_window` guard, but the ACL guards effectively.** The main capability gates these commands; the secondary windows do not have permission to invoke them. If you ever expand the secondary-windows capability, double-check.
|
||||
- **Preview overlay focus quirks.** Even with `focused: false`, KWin and Mutter Wayland sometimes route the next keystroke to the overlay. `commands::paste::hide_preview_overlay_for_paste` is the partner workaround that hides the preview before firing a paste.
|
||||
- **The GTK type hint is set *before* `show()`** (`:151`). GTK3 only honours type hints pre-realize. Don't reorder.
|
||||
- **Linux native decorations vs frameless.** The `let use_native_decorations = cfg!(target_os = "linux");` call is the cross-cutting decision: Linux wins native via Tauri's frameless-Wayland resize quirk; macOS and Windows draw the custom Titlebar. If you ever ship a Windows build, audit this.
|
||||
- **`open_task_window` emits `task-window-focus` only on the "already exists" path.** A first-build does not emit it. The Tasks page should listen and re-render the float regardless of which path was taken — but if you have logic that fires only on the event, the first-build will silently miss.
|
||||
- **`always_on_top` plus visible-on-all-workspaces** is the right combination for the preview, but it can feel intrusive. If the user's workflow shifts to a single-workspace setup, consider an opt-out toggle in Settings.
|
||||
|
||||
## See also
|
||||
|
||||
- [App lifecycle](../app-lifecycle.md) — `PreferencesScript` is what gets injected.
|
||||
- [Capabilities and ACL](../capabilities-and-acl.md) — the secondary-windows capability is bound to the labels declared here.
|
||||
- [Paste](paste.md) — `hide_preview_overlay_for_paste` is the partner hide call.
|
||||
- [Tauri config](../tauri-config.md) — the main window is declared statically; the three windows here are imperative.
|
||||
Reference in New Issue
Block a user