Lumotia/docs/architecture-map/03-audio-transcription/transcription-engines-overview.md

name, type, slice, last_verified

name	type	slice	last_verified
Transcription engines overview	architecture-map-page	03-audio-transcription	2026/05/09

Transcription engines overview

Where you are: Architecture map → Audio + Transcription → Transcription engines overview

Plain English summary. Every speech-to-text backend implements a single Transcriber trait. LocalEngine owns the currently-loaded backend behind a Mutex, exposes transcribe_sync for inference, and load / unload so the sequential-GPU guard can free VRAM before the LLM loads. Two concrete Transcriber implementations exist: WhisperRsBackend (direct whisper-rs, the only one that pipes initial_prompt) and SpeechModelAdapter (wraps any transcribe-rs SpeechModel, used today for Parakeet via a thin ParakeetWordGranularity shim).

At a glance

• Crate: lumotia-transcription
• Paths: crates/transcription/src/transcriber.rs (62 LOC), crates/transcription/src/local_engine.rs (226 LOC).
• External deps: transcribe-rs 0.3 (onnx feature). whisper-rs 0.16 is feature-gated and lives in the next page.
• Internal callers (best effort, slice 2 reconciles): src-tauri/src/commands/models.rs calls load_whisper / load_parakeet and LocalEngine::load. src-tauri/src/commands/transcription.rs (and live.rs) call LocalEngine::transcribe_sync indirectly via concurrency::run_inference.

Public surface:

• pub trait Transcriber: Send — crates/transcription/src/transcriber.rs:35. Methods: capabilities(), transcribe_sync(&mut self, samples, options).
• pub struct TranscriberCapabilities — crates/transcription/src/transcriber.rs:23. Fields: sample_rate: u32, channels: u16, supports_initial_prompt: bool.
• pub struct LocalEngine — crates/transcription/src/local_engine.rs:68.
• pub struct SpeechModelAdapter(pub Box<dyn SpeechModel + Send>) — crates/transcription/src/local_engine.rs:26.
• pub struct TimedTranscript { transcript: Transcript, inference_ms: u64 } — crates/transcription/src/local_engine.rs:17.
• pub fn load_parakeet(model_dir: &Path) -> Result<Box<dyn Transcriber + Send>> — crates/transcription/src/local_engine.rs:197.
• pub fn load_whisper(model_path: &Path) -> Result<Box<dyn Transcriber + Send>> — crates/transcription/src/local_engine.rs:208 (gated behind feature = "whisper").
• pub use transcribe_rs::SpeechModel — re-exported via lib.rs:18.

What's in here

Transcriber trait

transcriber.rs:35. Send is a supertrait so Box<dyn Transcriber + Send> travels through tokio::task::spawn_blocking boundaries. Synchronous-only API: async callers wrap a spawn_blocking. Inference is &mut self so backends with per-call scratch state (whisper-rs WhisperState, Parakeet decoder buffers) can mutate without interior-mutability gymnastics.

TranscriberCapabilities carries three fields:

• sample_rate: u32 — the rate this backend wants. Live capture's WavWriter::create reads this to decide how to format the on-disk file (brief item #19).
• channels: u16 — almost always 1, kept explicit so a stereo-aware backend can declare it.
• supports_initial_prompt: bool — Whisper says yes, Parakeet says no. Settings UI hides the field accordingly.

The trait is asserted object-safe by a compile-time witness test (transcriber.rs:54).

LocalEngine

local_engine.rs:68. Holds:

• engine: Mutex<Option<Box<dyn Transcriber + Send>>> — currently-loaded backend or None.
• engine_name: EngineName — display tag (e.g. "whisper", "parakeet"); set at construction.
• loaded_model_id: Mutex<Option<ModelId>> — what's loaded. Used by the UI and by the model-switch logic.

API:

• new(engine_name) — create empty.
• load(backend, model_id) — install a backend; replaces any prior.
• unload() — drop the backend so its GPU VRAM / mmap'd tensors are freed (sequential-GPU guard, brief item A.1 #28).
• name() -> &EngineName.
• loaded_model_id() -> Option<ModelId>.
• is_loaded() -> bool.
• capabilities() -> Option<TranscriberCapabilities>.
• transcribe_sync(audio, options) -> Result<TimedTranscript> — locks the engine mutex, invokes the trait, times it with Instant::now, wraps the segments in Transcript::new(segments, language, duration_secs).

The mutex serialises inference against load / unload. There is no per-call lock contention with the audio thread; all inference is ferried through concurrency::run_inference (see transcription-concurrency.md).

SpeechModelAdapter

local_engine.rs:26. Adapter from any transcribe-rs SpeechModel to the Transcriber trait. Implements capabilities() returning sample_rate = WHISPER_SAMPLE_RATE, channels = 1, supports_initial_prompt = false. transcribe_sync constructs a TranscribeOptions { language, translate: false, leading_silence_ms: None, trailing_silence_ms: None } and converts the resulting TranscriptionResult.segments into Vec<Segment>.

Loaders

• load_whisper (gated behind feature = "whisper") constructs a WhisperRsBackend (see next page).
• load_parakeet constructs a ParakeetModel with Quantization::Int8, wraps it in ParakeetWordGranularity (so the trait method requests TimestampGranularity::Word instead of the per-subword default), then wraps that in SpeechModelAdapter.

Data flow

audio: AudioSamples (16 kHz mono)
options: TranscriptionOptions (language, initial_prompt)
   └─ run_inference (concurrency.rs)
       └─ spawn_blocking
           └─ LocalEngine::transcribe_sync
               ├─ Mutex::lock(engine)
               ├─ backend.transcribe_sync(samples, options)
               │   └─ WhisperRsBackend or SpeechModelAdapter
               ├─ Instant::now diff → inference_ms
               └─ Transcript::new(segments, language, duration_secs)
   → TimedTranscript

Watch-outs

• engine_name is not validated. LocalEngine::new(EngineName::new("whisper")) is opaque to the engine itself. Anything that conditionally branches on engine name (e.g. UI selecting the right model list) must agree on the string value with the loader and Tauri command.
• Mutex::lock().unwrap_or_else(|e| e.into_inner()) is used everywhere. A poisoned mutex (panicking call) does not propagate — the engine is read in its post-panic state. Acceptable for a single-process desktop app; document this if anyone moves the type into a daemon.
• Transcript::new requires the audio duration. Computed via audio.duration_secs(); assumes AudioSamples carries the original sample count and rate (it does). If anyone ever passes a pre-resampled vector that has lost the rate field, the duration will be wrong.
• No streaming surface in this trait. All inference is one-shot synchronous. Live capture turns this into a streaming experience by chunking the audio (see RmsVadChunker) and stitching segments via LocalAgreement.
• Parakeet wrapper hardcodes Word granularity. ParakeetWordGranularity::transcribe_raw (local_engine.rs:182) overrides the default Token granularity that produced "T Est Ing , One" output. If a future caller wants character-level timestamps, this is where they'd need a flag.
• Adapter discards initial_prompt. SpeechModelAdapter::transcribe_sync does not look at options.initial_prompt. Whisper is the only path that pipes it; the capability flag exists so the UI does not pretend otherwise.

See also

• Transcription whisper — the whisper-rs backend.
• Transcription parakeet — the transcribe-rs Parakeet wrapper.
• Transcription concurrency — run_inference async wrapper.
• Transcription streaming — VAD chunker + LocalAgreement stack that drives live capture.
• lumotia_core::types::{Segment, Transcript, TranscriptionOptions, EngineName, ModelId} (slice 5).