jake/Lumotia
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: architecture map (initial 5-slice generation, 105 pages)

Browse Source 
Five-slice navigable map of the entire codebase under
docs/architecture-map/. Each slice is a self-contained
breadcrumbed sub-tree:

  01-frontend (16)              Svelte/SvelteKit UI
  02-tauri-runtime (26)         src-tauri commands + lifecycle
  03-audio-transcription (16)   audio + transcription crates
  04-llm-formatting-mcp (19)    llm, ai-formatting, mcp, cloud
  05-core-storage-hotkey-build  core, storage, hotkey, workspace,
                          (26) CI, dev glue

Plus master README.md and data-flow-end-to-end.md tracing
audio bytes from microphone to FTS5 search to MCP read.

Generated by 5 parallel subagents on 2026/05/09 against
HEAD 3c47000. Each page has YAML frontmatter, file:line code
refs, sibling cross-links, plain-English summaries.

Aggregated debt surfaced (full lists in master README):
RB-08 macOS power assertion, schema head drift v14 vs v15,
VAD blocked on ort version conflict, streaming primitives
not wired into live.rs, no prompt versioning, MCP has no
auth, cloud-providers in-memory keystore, SettingsPage
2 484 LOC, commands/live.rs 1 737 LOC, dual theme system,
brand rename to Lumenote pending across the codebase.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jars
2026-05-09 14:04:13 +01:00
parent 3c47000ea9
commit a1f3f3f134
105 changed files with 11266 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

111
docs/architecture-map/02-tauri-runtime/commands/transcription.md Normal file
View File

@@ -0,0 +1,111 @@
---
name: Transcription commands
type: architecture-map-page
slice: 02-tauri-runtime
last_verified: 2026/05/09
---
# `commands::transcription`
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → Transcription
**Plain English summary.** The non-live transcription paths. Three commands: transcribe a Vec<f32> of PCM samples through Whisper, transcribe the same shape through Parakeet, and transcribe a file from disk by decoding + resampling to 16 kHz first. Both PCM commands emit `transcription-result` events; the file command returns the result inline. All three run inference on a blocking thread, run the formatting pipeline against the output, and respect profile prompt + vocabulary precedence via the shared `build_initial_prompt` helper.
## At a glance
- Path: `src-tauri/src/commands/transcription.rs`.
- LOC: 413.
- Tauri commands exposed:
- `transcribe_pcm(window, state, app, samples: Vec<f32>, chunk_id: u32, language, initial_prompt, remove_fillers, british_english, anti_hallucination, format_mode, profile_id) -> Result<(), String>` — main-window only. Whisper PCM. Emits `transcription-result`.
- `transcribe_file(window, state, path, engine: Option<String>, model_id: Option<String>, language, initial_prompt, remove_fillers, british_english, anti_hallucination, format_mode, profile_id) -> Result<serde_json::Value, String>` — main-window only. Decodes the file, picks engine (default whisper), returns the result inline.
- `transcribe_pcm_parakeet(window, state, app, samples, chunk_id, remove_fillers, british_english, anti_hallucination, format_mode, profile_id) -> Result<(), String>` — main-window only. Parakeet PCM. Emits `transcription-result`.
- Events emitted: `transcription-result` (payload: `{ status: "transcription", segments, language, duration, chunk_id, inference_ms, raw_text }`) — fires from `transcribe_pcm` (`src-tauri/src/commands/transcription.rs:208`) and `transcribe_pcm_parakeet` (`:398`).
- Depends on: `magnotia_audio::{decode_audio_file_limited, resample_to_16khz, probe_audio_duration_secs}`, `magnotia_core::types::{AudioSamples, Segment, Transcript, TranscriptionOptions}`, `magnotia_transcription::{LocalEngine, TimedTranscript}`, `magnotia_ai_formatting::{post_process_segments, FormatMode, PostProcessOptions}`, `magnotia_storage::{database, DEFAULT_PROFILE_ID}`. Plus `commands::build_initial_prompt`, `commands::models::{default_model_id_for_engine, ensure_model_loaded}`, `commands::security::ensure_main_window`.
- Called from frontend at: dictation page (PCM commands when not live), file-import flow (transcribe_file), Settings test page (file command for QA).
## What's in here
### Constants (`src-tauri/src/commands/transcription.rs:18`)
Chunking thresholds:
- Parakeet: chunk anything over 18 s into 15 s windows with 1 s overlap.
- Generic file: chunk anything over 8 minutes into 3-minute windows with 2 s overlap.
- Hard cap: `MAX_FILE_TRANSCRIPTION_SECS = 2 * 60 * 60` (2 hours).
### `pick_engine` (`:31`)
Maps `"whisper"` and `"parakeet"` to the relevant `Arc<LocalEngine>` from `AppState`.
### `pick_chunking_strategy` (`:42`)
Returns the `ChunkingStrategy` to use based on engine + sample count, or `None` for "no chunking needed".
### `trim_overlap_segments` (`:61`)
For chunks past the first, drops segments that end before `trim_before_secs` and clamps remaining starts. The same logic appears in `commands::live` for the live-mode path.
### `transcribe_samples_sync` (`:74`)
The shared inner that the file path uses. If no chunking is needed, calls `engine.transcribe_sync` once. Otherwise loops over chunks, runs each through inference, trims overlap, offsets timestamps by the chunk start, accumulates segments and inference_ms. Returns a synthesised `TimedTranscript`.
### `transcribe_pcm` (`:142`)
Whisper-specific PCM path (no chunking — frontend is expected to keep the buffer reasonable, e.g. ≤ 30 s for the Whisper context window):
1. `ensure_main_window`.
2. Resolve `profile_id` (default `DEFAULT_PROFILE_ID`).
3. Fetch `ProfileRow` and profile term list from `magnotia_storage::database`.
4. Build effective Whisper prompt via `build_initial_prompt(&caller_prompt, &profile.initial_prompt, &profile_terms)`.
5. `spawn_blocking` runs `engine.transcribe_sync` on the samples.
6. Run `post_process_segments` (filler removal, British English conversion, anti-hallucination, format mode, dictionary terms, optional LLM cleanup via `state.llm_engine`).
7. `app.emit("transcription-result", ...)` with raw_text (pre-post-process) plus the post-processed segments.
### `transcribe_file` (`:236`)
1. `ensure_main_window`.
2. Resolve profile + terms (same as PCM path).
3. Default engine to `"whisper"`, default model id to `default_model_id_for_engine(&engine_name)`.
4. `ensure_model_loaded(state, engine, model_id, None)` — None = no sequential-GPU guard.
5. Probe audio duration via `magnotia_audio::probe_audio_duration_secs`. If > 2 hours, return a friendly error.
6. `spawn_blocking` decodes the file (`decode_audio_file_limited(path, Some(MAX_FILE_TRANSCRIPTION_SECS))`), resamples to 16 kHz mono, then runs `transcribe_samples_sync`.
7. Run `post_process_segments`.
8. Return a JSON value with `engine`, `modelId`, `segments`, `language`, `duration`, `inference_ms`, `raw_text`.
### `transcribe_pcm_parakeet` (`:342`)
Parakeet PCM path. Skips the `initial_prompt` construction (Parakeet doesn't have a Whisper-style prompt) but still validates the profile exists and gathers the dictionary terms for post-processing. Calls `transcribe_samples_sync(parakeet_engine, "parakeet", samples, options)` so the > 18 s chunking kicks in if relevant. Emits `transcription-result` like the Whisper path.
### `join_segment_text` (`:225`)
Concatenates segment text with whitespace stripping for the `raw_text` field on the emitted event.
## Data flow
```
frontend invoke('transcribe_pcm', { samples, chunk_id, ..., profile_id })
-> ensure_main_window
-> get_profile + list_profile_terms (DB)
-> build_initial_prompt
-> spawn_blocking(engine.transcribe_sync(samples, options)) -> TimedTranscript
-> post_process_segments(segments, opts, llm_engine) -- in-place
-> app.emit("transcription-result", { segments, raw_text, ... })
```
`transcribe_file` is the same shape with `decode_audio_file_limited` + `resample_to_16khz` + `transcribe_samples_sync` substituted for the inline `transcribe_sync`.
## Watch-outs
- **`transcribe_pcm` does NOT call `ensure_model_loaded`.** It assumes the model is already loaded (which is true when the frontend has driven the Settings → load flow, and is true after `prewarm_default_model_cmd`). If a third caller invokes `transcribe_pcm` without a load step, you'll get a runtime panic from `engine.transcribe_sync` with no clear message. The file path *does* call `ensure_model_loaded`. Consider adding the same guard to `transcribe_pcm` for symmetry.
- **`transcribe_file` returns a `serde_json::Value` rather than a typed DTO.** The shape is fully ad-hoc (`engine`, `modelId`, `segments`, `language`, `duration`, `inference_ms`, `raw_text`). The frontend has to keep this shape in sync by hand. Consider promoting it to a typed struct.
- **Chunking does not emit per-chunk events from this command.** The file path returns the entire concatenated result at the end. If a 90-minute file fails three quarters of the way through, the user gets nothing. Live mode (`commands::live`) is the streaming alternative.
- **The 2-hour cap is a hard constant.** Document it in the frontend "import audio" flow.
- **The Parakeet PCM path does not respect `language` or `initial_prompt`** because Parakeet has no equivalent. The frontend should disable those Settings widgets when the engine is Parakeet, or this command will silently ignore them.
## See also
- [Models](models.md) — `ensure_model_loaded`, `default_model_id_for_engine`, the `LocalEngine` cache that `state.whisper_engine` and `state.parakeet_engine` wrap.
- [Live transcription](live.md) — the streaming sibling.
- [Profiles](profiles.md) — feeds `profile.initial_prompt` and the term list.
- [`commands::mod`](mod.md) — `build_initial_prompt` is the prompt assembler.
- [Audio capture](audio.md) — supplies the `Vec<f32>` that `transcribe_pcm` consumes.