Jake
26c7307607
Phase 9 of the rebrand cascade. Sweep covers everything the Phase 8
frontend pass deliberately skipped: docs/, root markdown, scripts,
Cargo.toml descriptions, code comments that survived earlier
word-boundary sed, plus a handful of identifiers caught on the final
verify pass.
transcription-app changes:
- README.md, HANDOVER.md, KNOWN-ISSUES.md, run.sh — magnotia/Magnotia
-> lumotia/Lumotia.
- docs/ — sweep across all subdirs except docs/handovers/ (preserved
as immutable audit trail). Includes architecture-map references
to magnotia_core::*, magnotia_storage::*, etc. now pointing at
lumotia_*; dev-setup.md tracing output examples (lumotia_startup
target); brief/ + superpowers/ + issues/ + whisper-ecosystem/ +
audit/.
- Cargo.toml descriptions on 9 crates (core, audio, cloud-providers,
hotkey, llm, mcp, plus referenced others).
- crates/core/src/{error,hardware,recommendation,paths}.rs +
crates/audio/src/wav.rs + crates/llm/src/model_manager.rs +
crates/cloud-providers/src/keystore.rs + crates/mcp/src/lib.rs —
doc comments and a model-manager user-agent string.
- Caught on final pass: BroadcastChannel("magnotia_task_sync") -> ...
("lumotia_task_sync"); magnotia_locale i18n localStorage key
renamed + migration shim added; CSS keyframe names
magnotiaPulse / magnotiaBar / magnotiaFade renamed in the design-
system kit; magnotia_viewer_item / magnotia_viewer_mode handoff
keys renamed in HistoryPage + viewer/+page.svelte; src/assets/
wordmark.svg text.
- src-tauri/src/lib.rs comment cleanup ("magnotia era" was sed'd
to "lumotia era" earlier — restored).
Preserved (intentional):
- crates/core/src/paths.rs — keeps "magnotia" / "Magnotia" / ".magnotia"
legacy detection strings in legacy_and_target_paths() so the
migration shim can still find user data from the magnotia era.
- src/lib/stores/{page,focusTimer}.svelte.ts + src/lib/i18n/index.ts
— migration call sites reference the legacy magnotia keys
deliberately.
- docs/handovers/ — historical audit trail.
cargo build --workspace passes. npm run check: 0 errors / 0 warnings
(3958 files). cargo test --workspace: 339 pass / 0 fail.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
108 lines
7.9 KiB
Markdown
108 lines
7.9 KiB
Markdown
---
|
|
name: Transcription engines overview
|
|
type: architecture-map-page
|
|
slice: 03-audio-transcription
|
|
last_verified: 2026/05/09
|
|
---
|
|
|
|
# Transcription engines overview
|
|
|
|
> **Where you are:** [Architecture map](../README.md) → [Audio + Transcription](README.md) → Transcription engines overview
|
|
|
|
**Plain English summary.** Every speech-to-text backend implements a single `Transcriber` trait. `LocalEngine` owns the currently-loaded backend behind a `Mutex`, exposes `transcribe_sync` for inference, and `load` / `unload` so the sequential-GPU guard can free VRAM before the LLM loads. Two concrete `Transcriber` implementations exist: `WhisperRsBackend` (direct `whisper-rs`, the only one that pipes `initial_prompt`) and `SpeechModelAdapter` (wraps any `transcribe-rs` `SpeechModel`, used today for Parakeet via a thin `ParakeetWordGranularity` shim).
|
|
|
|
## At a glance
|
|
|
|
- Crate: `lumotia-transcription`
|
|
- Paths: `crates/transcription/src/transcriber.rs` (62 LOC), `crates/transcription/src/local_engine.rs` (226 LOC).
|
|
- External deps: `transcribe-rs 0.3` (onnx feature). `whisper-rs 0.16` is feature-gated and lives in the next page.
|
|
- Internal callers (best effort, slice 2 reconciles): `src-tauri/src/commands/models.rs` calls `load_whisper` / `load_parakeet` and `LocalEngine::load`. `src-tauri/src/commands/transcription.rs` (and `live.rs`) call `LocalEngine::transcribe_sync` indirectly via `concurrency::run_inference`.
|
|
|
|
Public surface:
|
|
|
|
- `pub trait Transcriber: Send` — `crates/transcription/src/transcriber.rs:35`. Methods: `capabilities()`, `transcribe_sync(&mut self, samples, options)`.
|
|
- `pub struct TranscriberCapabilities` — `crates/transcription/src/transcriber.rs:23`. Fields: `sample_rate: u32`, `channels: u16`, `supports_initial_prompt: bool`.
|
|
- `pub struct LocalEngine` — `crates/transcription/src/local_engine.rs:68`.
|
|
- `pub struct SpeechModelAdapter(pub Box<dyn SpeechModel + Send>)` — `crates/transcription/src/local_engine.rs:26`.
|
|
- `pub struct TimedTranscript { transcript: Transcript, inference_ms: u64 }` — `crates/transcription/src/local_engine.rs:17`.
|
|
- `pub fn load_parakeet(model_dir: &Path) -> Result<Box<dyn Transcriber + Send>>` — `crates/transcription/src/local_engine.rs:197`.
|
|
- `pub fn load_whisper(model_path: &Path) -> Result<Box<dyn Transcriber + Send>>` — `crates/transcription/src/local_engine.rs:208` (gated behind `feature = "whisper"`).
|
|
- `pub use transcribe_rs::SpeechModel` — re-exported via `lib.rs:18`.
|
|
|
|
## What's in here
|
|
|
|
### `Transcriber` trait
|
|
|
|
`transcriber.rs:35`. `Send` is a supertrait so `Box<dyn Transcriber + Send>` travels through `tokio::task::spawn_blocking` boundaries. Synchronous-only API: async callers wrap a `spawn_blocking`. Inference is `&mut self` so backends with per-call scratch state (whisper-rs `WhisperState`, Parakeet decoder buffers) can mutate without interior-mutability gymnastics.
|
|
|
|
`TranscriberCapabilities` carries three fields:
|
|
|
|
- `sample_rate: u32` — the rate this backend wants. Live capture's `WavWriter::create` reads this to decide how to format the on-disk file (brief item #19).
|
|
- `channels: u16` — almost always 1, kept explicit so a stereo-aware backend can declare it.
|
|
- `supports_initial_prompt: bool` — Whisper says yes, Parakeet says no. Settings UI hides the field accordingly.
|
|
|
|
The trait is asserted object-safe by a compile-time witness test (`transcriber.rs:54`).
|
|
|
|
### `LocalEngine`
|
|
|
|
`local_engine.rs:68`. Holds:
|
|
|
|
- `engine: Mutex<Option<Box<dyn Transcriber + Send>>>` — currently-loaded backend or `None`.
|
|
- `engine_name: EngineName` — display tag (e.g. "whisper", "parakeet"); set at construction.
|
|
- `loaded_model_id: Mutex<Option<ModelId>>` — what's loaded. Used by the UI and by the model-switch logic.
|
|
|
|
API:
|
|
|
|
- `new(engine_name)` — create empty.
|
|
- `load(backend, model_id)` — install a backend; replaces any prior.
|
|
- `unload()` — drop the backend so its GPU VRAM / mmap'd tensors are freed (sequential-GPU guard, brief item A.1 #28).
|
|
- `name() -> &EngineName`.
|
|
- `loaded_model_id() -> Option<ModelId>`.
|
|
- `is_loaded() -> bool`.
|
|
- `capabilities() -> Option<TranscriberCapabilities>`.
|
|
- `transcribe_sync(audio, options) -> Result<TimedTranscript>` — locks the engine mutex, invokes the trait, times it with `Instant::now`, wraps the segments in `Transcript::new(segments, language, duration_secs)`.
|
|
|
|
The mutex serialises inference against load / unload. There is no per-call lock contention with the audio thread; all inference is ferried through `concurrency::run_inference` (see [transcription-concurrency.md](transcription-concurrency.md)).
|
|
|
|
### `SpeechModelAdapter`
|
|
|
|
`local_engine.rs:26`. Adapter from any `transcribe-rs` `SpeechModel` to the `Transcriber` trait. Implements `capabilities()` returning `sample_rate = WHISPER_SAMPLE_RATE`, `channels = 1`, `supports_initial_prompt = false`. `transcribe_sync` constructs a `TranscribeOptions { language, translate: false, leading_silence_ms: None, trailing_silence_ms: None }` and converts the resulting `TranscriptionResult.segments` into `Vec<Segment>`.
|
|
|
|
### Loaders
|
|
|
|
- `load_whisper` (gated behind `feature = "whisper"`) constructs a `WhisperRsBackend` (see next page).
|
|
- `load_parakeet` constructs a `ParakeetModel` with `Quantization::Int8`, wraps it in `ParakeetWordGranularity` (so the trait method requests `TimestampGranularity::Word` instead of the per-subword default), then wraps that in `SpeechModelAdapter`.
|
|
|
|
## Data flow
|
|
|
|
```
|
|
audio: AudioSamples (16 kHz mono)
|
|
options: TranscriptionOptions (language, initial_prompt)
|
|
└─ run_inference (concurrency.rs)
|
|
└─ spawn_blocking
|
|
└─ LocalEngine::transcribe_sync
|
|
├─ Mutex::lock(engine)
|
|
├─ backend.transcribe_sync(samples, options)
|
|
│ └─ WhisperRsBackend or SpeechModelAdapter
|
|
├─ Instant::now diff → inference_ms
|
|
└─ Transcript::new(segments, language, duration_secs)
|
|
→ TimedTranscript
|
|
```
|
|
|
|
## Watch-outs
|
|
|
|
- **`engine_name` is not validated.** `LocalEngine::new(EngineName::new("whisper"))` is opaque to the engine itself. Anything that conditionally branches on engine name (e.g. UI selecting the right model list) must agree on the string value with the loader and Tauri command.
|
|
- **`Mutex::lock().unwrap_or_else(|e| e.into_inner())` is used everywhere.** A poisoned mutex (panicking call) does not propagate — the engine is read in its post-panic state. Acceptable for a single-process desktop app; document this if anyone moves the type into a daemon.
|
|
- **`Transcript::new` requires the audio duration.** Computed via `audio.duration_secs()`; assumes `AudioSamples` carries the original sample count and rate (it does). If anyone ever passes a pre-resampled vector that has lost the rate field, the duration will be wrong.
|
|
- **No streaming surface in this trait.** All inference is one-shot synchronous. Live capture turns this into a streaming experience by chunking the audio (see `RmsVadChunker`) and stitching segments via `LocalAgreement`.
|
|
- **Parakeet wrapper hardcodes `Word` granularity.** `ParakeetWordGranularity::transcribe_raw` (`local_engine.rs:182`) overrides the default `Token` granularity that produced "T Est Ing , One" output. If a future caller wants character-level timestamps, this is where they'd need a flag.
|
|
- **Adapter discards `initial_prompt`.** `SpeechModelAdapter::transcribe_sync` does not look at `options.initial_prompt`. Whisper is the only path that pipes it; the capability flag exists so the UI does not pretend otherwise.
|
|
|
|
## See also
|
|
|
|
- [Transcription whisper](transcription-whisper.md) — the whisper-rs backend.
|
|
- [Transcription parakeet](transcription-parakeet.md) — the transcribe-rs Parakeet wrapper.
|
|
- [Transcription concurrency](transcription-concurrency.md) — `run_inference` async wrapper.
|
|
- [Transcription streaming](transcription-streaming.md) — VAD chunker + LocalAgreement stack that drives live capture.
|
|
- `lumotia_core::types::{Segment, Transcript, TranscriptionOptions, EngineName, ModelId}` (slice 5).
|