docs: architecture map (initial 5-slice generation, 105 pages)
Five-slice navigable map of the entire codebase under
docs/architecture-map/. Each slice is a self-contained
breadcrumbed sub-tree:
01-frontend (16) Svelte/SvelteKit UI
02-tauri-runtime (26) src-tauri commands + lifecycle
03-audio-transcription (16) audio + transcription crates
04-llm-formatting-mcp (19) llm, ai-formatting, mcp, cloud
05-core-storage-hotkey-build core, storage, hotkey, workspace,
(26) CI, dev glue
Plus master README.md and data-flow-end-to-end.md tracing
audio bytes from microphone to FTS5 search to MCP read.
Generated by 5 parallel subagents on 2026/05/09 against
HEAD 3c47000. Each page has YAML frontmatter, file:line code
refs, sibling cross-links, plain-English summaries.
Aggregated debt surfaced (full lists in master README):
RB-08 macOS power assertion, schema head drift v14 vs v15,
VAD blocked on ort version conflict, streaming primitives
not wired into live.rs, no prompt versioning, MCP has no
auth, cloud-providers in-memory keystore, SettingsPage
2 484 LOC, commands/live.rs 1 737 LOC, dual theme system,
brand rename to Lumenote pending across the codebase.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,107 @@
|
||||
---
|
||||
name: Transcription engines overview
|
||||
type: architecture-map-page
|
||||
slice: 03-audio-transcription
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# Transcription engines overview
|
||||
|
||||
> **Where you are:** [Architecture map](../README.md) → [Audio + Transcription](README.md) → Transcription engines overview
|
||||
|
||||
**Plain English summary.** Every speech-to-text backend implements a single `Transcriber` trait. `LocalEngine` owns the currently-loaded backend behind a `Mutex`, exposes `transcribe_sync` for inference, and `load` / `unload` so the sequential-GPU guard can free VRAM before the LLM loads. Two concrete `Transcriber` implementations exist: `WhisperRsBackend` (direct `whisper-rs`, the only one that pipes `initial_prompt`) and `SpeechModelAdapter` (wraps any `transcribe-rs` `SpeechModel`, used today for Parakeet via a thin `ParakeetWordGranularity` shim).
|
||||
|
||||
## At a glance
|
||||
|
||||
- Crate: `magnotia-transcription`
|
||||
- Paths: `crates/transcription/src/transcriber.rs` (62 LOC), `crates/transcription/src/local_engine.rs` (226 LOC).
|
||||
- External deps: `transcribe-rs 0.3` (onnx feature). `whisper-rs 0.16` is feature-gated and lives in the next page.
|
||||
- Internal callers (best effort, slice 2 reconciles): `src-tauri/src/commands/models.rs` calls `load_whisper` / `load_parakeet` and `LocalEngine::load`. `src-tauri/src/commands/transcription.rs` (and `live.rs`) call `LocalEngine::transcribe_sync` indirectly via `concurrency::run_inference`.
|
||||
|
||||
Public surface:
|
||||
|
||||
- `pub trait Transcriber: Send` — `crates/transcription/src/transcriber.rs:35`. Methods: `capabilities()`, `transcribe_sync(&mut self, samples, options)`.
|
||||
- `pub struct TranscriberCapabilities` — `crates/transcription/src/transcriber.rs:23`. Fields: `sample_rate: u32`, `channels: u16`, `supports_initial_prompt: bool`.
|
||||
- `pub struct LocalEngine` — `crates/transcription/src/local_engine.rs:68`.
|
||||
- `pub struct SpeechModelAdapter(pub Box<dyn SpeechModel + Send>)` — `crates/transcription/src/local_engine.rs:26`.
|
||||
- `pub struct TimedTranscript { transcript: Transcript, inference_ms: u64 }` — `crates/transcription/src/local_engine.rs:17`.
|
||||
- `pub fn load_parakeet(model_dir: &Path) -> Result<Box<dyn Transcriber + Send>>` — `crates/transcription/src/local_engine.rs:197`.
|
||||
- `pub fn load_whisper(model_path: &Path) -> Result<Box<dyn Transcriber + Send>>` — `crates/transcription/src/local_engine.rs:208` (gated behind `feature = "whisper"`).
|
||||
- `pub use transcribe_rs::SpeechModel` — re-exported via `lib.rs:18`.
|
||||
|
||||
## What's in here
|
||||
|
||||
### `Transcriber` trait
|
||||
|
||||
`transcriber.rs:35`. `Send` is a supertrait so `Box<dyn Transcriber + Send>` travels through `tokio::task::spawn_blocking` boundaries. Synchronous-only API: async callers wrap a `spawn_blocking`. Inference is `&mut self` so backends with per-call scratch state (whisper-rs `WhisperState`, Parakeet decoder buffers) can mutate without interior-mutability gymnastics.
|
||||
|
||||
`TranscriberCapabilities` carries three fields:
|
||||
|
||||
- `sample_rate: u32` — the rate this backend wants. Live capture's `WavWriter::create` reads this to decide how to format the on-disk file (brief item #19).
|
||||
- `channels: u16` — almost always 1, kept explicit so a stereo-aware backend can declare it.
|
||||
- `supports_initial_prompt: bool` — Whisper says yes, Parakeet says no. Settings UI hides the field accordingly.
|
||||
|
||||
The trait is asserted object-safe by a compile-time witness test (`transcriber.rs:54`).
|
||||
|
||||
### `LocalEngine`
|
||||
|
||||
`local_engine.rs:68`. Holds:
|
||||
|
||||
- `engine: Mutex<Option<Box<dyn Transcriber + Send>>>` — currently-loaded backend or `None`.
|
||||
- `engine_name: EngineName` — display tag (e.g. "whisper", "parakeet"); set at construction.
|
||||
- `loaded_model_id: Mutex<Option<ModelId>>` — what's loaded. Used by the UI and by the model-switch logic.
|
||||
|
||||
API:
|
||||
|
||||
- `new(engine_name)` — create empty.
|
||||
- `load(backend, model_id)` — install a backend; replaces any prior.
|
||||
- `unload()` — drop the backend so its GPU VRAM / mmap'd tensors are freed (sequential-GPU guard, brief item A.1 #28).
|
||||
- `name() -> &EngineName`.
|
||||
- `loaded_model_id() -> Option<ModelId>`.
|
||||
- `is_loaded() -> bool`.
|
||||
- `capabilities() -> Option<TranscriberCapabilities>`.
|
||||
- `transcribe_sync(audio, options) -> Result<TimedTranscript>` — locks the engine mutex, invokes the trait, times it with `Instant::now`, wraps the segments in `Transcript::new(segments, language, duration_secs)`.
|
||||
|
||||
The mutex serialises inference against load / unload. There is no per-call lock contention with the audio thread; all inference is ferried through `concurrency::run_inference` (see [transcription-concurrency.md](transcription-concurrency.md)).
|
||||
|
||||
### `SpeechModelAdapter`
|
||||
|
||||
`local_engine.rs:26`. Adapter from any `transcribe-rs` `SpeechModel` to the `Transcriber` trait. Implements `capabilities()` returning `sample_rate = WHISPER_SAMPLE_RATE`, `channels = 1`, `supports_initial_prompt = false`. `transcribe_sync` constructs a `TranscribeOptions { language, translate: false, leading_silence_ms: None, trailing_silence_ms: None }` and converts the resulting `TranscriptionResult.segments` into `Vec<Segment>`.
|
||||
|
||||
### Loaders
|
||||
|
||||
- `load_whisper` (gated behind `feature = "whisper"`) constructs a `WhisperRsBackend` (see next page).
|
||||
- `load_parakeet` constructs a `ParakeetModel` with `Quantization::Int8`, wraps it in `ParakeetWordGranularity` (so the trait method requests `TimestampGranularity::Word` instead of the per-subword default), then wraps that in `SpeechModelAdapter`.
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
audio: AudioSamples (16 kHz mono)
|
||||
options: TranscriptionOptions (language, initial_prompt)
|
||||
└─ run_inference (concurrency.rs)
|
||||
└─ spawn_blocking
|
||||
└─ LocalEngine::transcribe_sync
|
||||
├─ Mutex::lock(engine)
|
||||
├─ backend.transcribe_sync(samples, options)
|
||||
│ └─ WhisperRsBackend or SpeechModelAdapter
|
||||
├─ Instant::now diff → inference_ms
|
||||
└─ Transcript::new(segments, language, duration_secs)
|
||||
→ TimedTranscript
|
||||
```
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **`engine_name` is not validated.** `LocalEngine::new(EngineName::new("whisper"))` is opaque to the engine itself. Anything that conditionally branches on engine name (e.g. UI selecting the right model list) must agree on the string value with the loader and Tauri command.
|
||||
- **`Mutex::lock().unwrap_or_else(|e| e.into_inner())` is used everywhere.** A poisoned mutex (panicking call) does not propagate — the engine is read in its post-panic state. Acceptable for a single-process desktop app; document this if anyone moves the type into a daemon.
|
||||
- **`Transcript::new` requires the audio duration.** Computed via `audio.duration_secs()`; assumes `AudioSamples` carries the original sample count and rate (it does). If anyone ever passes a pre-resampled vector that has lost the rate field, the duration will be wrong.
|
||||
- **No streaming surface in this trait.** All inference is one-shot synchronous. Live capture turns this into a streaming experience by chunking the audio (see `RmsVadChunker`) and stitching segments via `LocalAgreement`.
|
||||
- **Parakeet wrapper hardcodes `Word` granularity.** `ParakeetWordGranularity::transcribe_raw` (`local_engine.rs:182`) overrides the default `Token` granularity that produced "T Est Ing , One" output. If a future caller wants character-level timestamps, this is where they'd need a flag.
|
||||
- **Adapter discards `initial_prompt`.** `SpeechModelAdapter::transcribe_sync` does not look at `options.initial_prompt`. Whisper is the only path that pipes it; the capability flag exists so the UI does not pretend otherwise.
|
||||
|
||||
## See also
|
||||
|
||||
- [Transcription whisper](transcription-whisper.md) — the whisper-rs backend.
|
||||
- [Transcription parakeet](transcription-parakeet.md) — the transcribe-rs Parakeet wrapper.
|
||||
- [Transcription concurrency](transcription-concurrency.md) — `run_inference` async wrapper.
|
||||
- [Transcription streaming](transcription-streaming.md) — VAD chunker + LocalAgreement stack that drives live capture.
|
||||
- `magnotia_core::types::{Segment, Transcript, TranscriptionOptions, EngineName, ModelId}` (slice 5).
|
||||
Reference in New Issue
Block a user