a1f3f3f134
Five-slice navigable map of the entire codebase under
docs/architecture-map/. Each slice is a self-contained
breadcrumbed sub-tree:
01-frontend (16) Svelte/SvelteKit UI
02-tauri-runtime (26) src-tauri commands + lifecycle
03-audio-transcription (16) audio + transcription crates
04-llm-formatting-mcp (19) llm, ai-formatting, mcp, cloud
05-core-storage-hotkey-build core, storage, hotkey, workspace,
(26) CI, dev glue
Plus master README.md and data-flow-end-to-end.md tracing
audio bytes from microphone to FTS5 search to MCP read.
Generated by 5 parallel subagents on 2026/05/09 against
HEAD 3c47000. Each page has YAML frontmatter, file:line code
refs, sibling cross-links, plain-English summaries.
Aggregated debt surfaced (full lists in master README):
RB-08 macOS power assertion, schema head drift v14 vs v15,
VAD blocked on ort version conflict, streaming primitives
not wired into live.rs, no prompt versioning, MCP has no
auth, cloud-providers in-memory keystore, SettingsPage
2 484 LOC, commands/live.rs 1 737 LOC, dual theme system,
brand rename to Lumenote pending across the codebase.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
112 lines
8.2 KiB
Markdown
112 lines
8.2 KiB
Markdown
---
|
|
name: Transcription commands
|
|
type: architecture-map-page
|
|
slice: 02-tauri-runtime
|
|
last_verified: 2026/05/09
|
|
---
|
|
|
|
# `commands::transcription`
|
|
|
|
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → Transcription
|
|
|
|
**Plain English summary.** The non-live transcription paths. Three commands: transcribe a Vec<f32> of PCM samples through Whisper, transcribe the same shape through Parakeet, and transcribe a file from disk by decoding + resampling to 16 kHz first. Both PCM commands emit `transcription-result` events; the file command returns the result inline. All three run inference on a blocking thread, run the formatting pipeline against the output, and respect profile prompt + vocabulary precedence via the shared `build_initial_prompt` helper.
|
|
|
|
## At a glance
|
|
|
|
- Path: `src-tauri/src/commands/transcription.rs`.
|
|
- LOC: 413.
|
|
- Tauri commands exposed:
|
|
- `transcribe_pcm(window, state, app, samples: Vec<f32>, chunk_id: u32, language, initial_prompt, remove_fillers, british_english, anti_hallucination, format_mode, profile_id) -> Result<(), String>` — main-window only. Whisper PCM. Emits `transcription-result`.
|
|
- `transcribe_file(window, state, path, engine: Option<String>, model_id: Option<String>, language, initial_prompt, remove_fillers, british_english, anti_hallucination, format_mode, profile_id) -> Result<serde_json::Value, String>` — main-window only. Decodes the file, picks engine (default whisper), returns the result inline.
|
|
- `transcribe_pcm_parakeet(window, state, app, samples, chunk_id, remove_fillers, british_english, anti_hallucination, format_mode, profile_id) -> Result<(), String>` — main-window only. Parakeet PCM. Emits `transcription-result`.
|
|
- Events emitted: `transcription-result` (payload: `{ status: "transcription", segments, language, duration, chunk_id, inference_ms, raw_text }`) — fires from `transcribe_pcm` (`src-tauri/src/commands/transcription.rs:208`) and `transcribe_pcm_parakeet` (`:398`).
|
|
- Depends on: `magnotia_audio::{decode_audio_file_limited, resample_to_16khz, probe_audio_duration_secs}`, `magnotia_core::types::{AudioSamples, Segment, Transcript, TranscriptionOptions}`, `magnotia_transcription::{LocalEngine, TimedTranscript}`, `magnotia_ai_formatting::{post_process_segments, FormatMode, PostProcessOptions}`, `magnotia_storage::{database, DEFAULT_PROFILE_ID}`. Plus `commands::build_initial_prompt`, `commands::models::{default_model_id_for_engine, ensure_model_loaded}`, `commands::security::ensure_main_window`.
|
|
- Called from frontend at: dictation page (PCM commands when not live), file-import flow (transcribe_file), Settings test page (file command for QA).
|
|
|
|
## What's in here
|
|
|
|
### Constants (`src-tauri/src/commands/transcription.rs:18`)
|
|
|
|
Chunking thresholds:
|
|
|
|
- Parakeet: chunk anything over 18 s into 15 s windows with 1 s overlap.
|
|
- Generic file: chunk anything over 8 minutes into 3-minute windows with 2 s overlap.
|
|
- Hard cap: `MAX_FILE_TRANSCRIPTION_SECS = 2 * 60 * 60` (2 hours).
|
|
|
|
### `pick_engine` (`:31`)
|
|
|
|
Maps `"whisper"` and `"parakeet"` to the relevant `Arc<LocalEngine>` from `AppState`.
|
|
|
|
### `pick_chunking_strategy` (`:42`)
|
|
|
|
Returns the `ChunkingStrategy` to use based on engine + sample count, or `None` for "no chunking needed".
|
|
|
|
### `trim_overlap_segments` (`:61`)
|
|
|
|
For chunks past the first, drops segments that end before `trim_before_secs` and clamps remaining starts. The same logic appears in `commands::live` for the live-mode path.
|
|
|
|
### `transcribe_samples_sync` (`:74`)
|
|
|
|
The shared inner that the file path uses. If no chunking is needed, calls `engine.transcribe_sync` once. Otherwise loops over chunks, runs each through inference, trims overlap, offsets timestamps by the chunk start, accumulates segments and inference_ms. Returns a synthesised `TimedTranscript`.
|
|
|
|
### `transcribe_pcm` (`:142`)
|
|
|
|
Whisper-specific PCM path (no chunking — frontend is expected to keep the buffer reasonable, e.g. ≤ 30 s for the Whisper context window):
|
|
|
|
1. `ensure_main_window`.
|
|
2. Resolve `profile_id` (default `DEFAULT_PROFILE_ID`).
|
|
3. Fetch `ProfileRow` and profile term list from `magnotia_storage::database`.
|
|
4. Build effective Whisper prompt via `build_initial_prompt(&caller_prompt, &profile.initial_prompt, &profile_terms)`.
|
|
5. `spawn_blocking` runs `engine.transcribe_sync` on the samples.
|
|
6. Run `post_process_segments` (filler removal, British English conversion, anti-hallucination, format mode, dictionary terms, optional LLM cleanup via `state.llm_engine`).
|
|
7. `app.emit("transcription-result", ...)` with raw_text (pre-post-process) plus the post-processed segments.
|
|
|
|
### `transcribe_file` (`:236`)
|
|
|
|
1. `ensure_main_window`.
|
|
2. Resolve profile + terms (same as PCM path).
|
|
3. Default engine to `"whisper"`, default model id to `default_model_id_for_engine(&engine_name)`.
|
|
4. `ensure_model_loaded(state, engine, model_id, None)` — None = no sequential-GPU guard.
|
|
5. Probe audio duration via `magnotia_audio::probe_audio_duration_secs`. If > 2 hours, return a friendly error.
|
|
6. `spawn_blocking` decodes the file (`decode_audio_file_limited(path, Some(MAX_FILE_TRANSCRIPTION_SECS))`), resamples to 16 kHz mono, then runs `transcribe_samples_sync`.
|
|
7. Run `post_process_segments`.
|
|
8. Return a JSON value with `engine`, `modelId`, `segments`, `language`, `duration`, `inference_ms`, `raw_text`.
|
|
|
|
### `transcribe_pcm_parakeet` (`:342`)
|
|
|
|
Parakeet PCM path. Skips the `initial_prompt` construction (Parakeet doesn't have a Whisper-style prompt) but still validates the profile exists and gathers the dictionary terms for post-processing. Calls `transcribe_samples_sync(parakeet_engine, "parakeet", samples, options)` so the > 18 s chunking kicks in if relevant. Emits `transcription-result` like the Whisper path.
|
|
|
|
### `join_segment_text` (`:225`)
|
|
|
|
Concatenates segment text with whitespace stripping for the `raw_text` field on the emitted event.
|
|
|
|
## Data flow
|
|
|
|
```
|
|
frontend invoke('transcribe_pcm', { samples, chunk_id, ..., profile_id })
|
|
-> ensure_main_window
|
|
-> get_profile + list_profile_terms (DB)
|
|
-> build_initial_prompt
|
|
-> spawn_blocking(engine.transcribe_sync(samples, options)) -> TimedTranscript
|
|
-> post_process_segments(segments, opts, llm_engine) -- in-place
|
|
-> app.emit("transcription-result", { segments, raw_text, ... })
|
|
```
|
|
|
|
`transcribe_file` is the same shape with `decode_audio_file_limited` + `resample_to_16khz` + `transcribe_samples_sync` substituted for the inline `transcribe_sync`.
|
|
|
|
## Watch-outs
|
|
|
|
- **`transcribe_pcm` does NOT call `ensure_model_loaded`.** It assumes the model is already loaded (which is true when the frontend has driven the Settings → load flow, and is true after `prewarm_default_model_cmd`). If a third caller invokes `transcribe_pcm` without a load step, you'll get a runtime panic from `engine.transcribe_sync` with no clear message. The file path *does* call `ensure_model_loaded`. Consider adding the same guard to `transcribe_pcm` for symmetry.
|
|
- **`transcribe_file` returns a `serde_json::Value` rather than a typed DTO.** The shape is fully ad-hoc (`engine`, `modelId`, `segments`, `language`, `duration`, `inference_ms`, `raw_text`). The frontend has to keep this shape in sync by hand. Consider promoting it to a typed struct.
|
|
- **Chunking does not emit per-chunk events from this command.** The file path returns the entire concatenated result at the end. If a 90-minute file fails three quarters of the way through, the user gets nothing. Live mode (`commands::live`) is the streaming alternative.
|
|
- **The 2-hour cap is a hard constant.** Document it in the frontend "import audio" flow.
|
|
- **The Parakeet PCM path does not respect `language` or `initial_prompt`** because Parakeet has no equivalent. The frontend should disable those Settings widgets when the engine is Parakeet, or this command will silently ignore them.
|
|
|
|
## See also
|
|
|
|
- [Models](models.md) — `ensure_model_loaded`, `default_model_id_for_engine`, the `LocalEngine` cache that `state.whisper_engine` and `state.parakeet_engine` wrap.
|
|
- [Live transcription](live.md) — the streaming sibling.
|
|
- [Profiles](profiles.md) — feeds `profile.initial_prompt` and the term list.
|
|
- [`commands::mod`](mod.md) — `build_initial_prompt` is the prompt assembler.
|
|
- [Audio capture](audio.md) — supplies the `Vec<f32>` that `transcribe_pcm` consumes.
|