Lumotia/docs/architecture-map/03-audio-transcription/audio-resampling.md

name, type, slice, last_verified

name	type	slice	last_verified
Audio resampling (rubato)	architecture-map-page	03-audio-transcription	2026/05/09

Audio resampling (rubato)

Where you are: Architecture map → Audio + Transcription → Audio resampling

Plain English summary. Whisper and Parakeet want 16 kHz mono f32. Microphones and audio files almost never deliver that natively. Two rubato-backed resamplers convert: resample_to_16khz for one-shot file conversion, StreamingResampler for the live session where samples arrive in arbitrary chunks and a flush at the end drains the resampler's tail.

At a glance

• Crate: lumotia-audio
• Paths: crates/audio/src/resample.rs (100 LOC), crates/audio/src/streaming_resample.rs (211 LOC).
• External deps: rubato 0.15 (sinc interpolation).
• Internal callers (best effort, slice 2 reconciles): concurrency::decode_and_resample (file path used by audio.rs and transcription.rs Tauri commands), live.rs Tauri command (live path).

Public surface:

• pub fn resample_to_16khz(audio: &AudioSamples) -> Result<AudioSamples> — crates/audio/src/resample.rs:11.
• pub enum StreamingResampler — crates/audio/src/streaming_resample.rs:37. Variants: Passthrough, Sinc { resampler, residual, ratio }.
• pub fn StreamingResampler::new(from_rate: u32) -> Result<Self> — crates/audio/src/streaming_resample.rs:52.
• pub fn StreamingResampler::push_samples(&mut self, mono: &[f32]) -> Result<Vec<f32>> — crates/audio/src/streaming_resample.rs:93.
• pub fn StreamingResampler::flush(&mut self) -> Result<Vec<f32>> — crates/audio/src/streaming_resample.rs:127.

What's in here

Shared sinc parameters

Both files use the same SincInterpolationParameters:

sinc_len: 256,
f_cutoff: 0.95,
oversampling_factor: 128,
interpolation: SincInterpolationType::Cubic,
window: WindowFunction::Blackman,

max_relative_jitter = 1.1, chunks_in = 1024, channels = 1.

resample_to_16khz (file)

crates/audio/src/resample.rs:11. Pure function over an immutable AudioSamples. Short-circuits the passthrough case (already 16 kHz). Iterates the input in 1024-sample chunks; pads the trailing partial chunk to chunk_size zeros; truncates the output to (input_len * ratio) as usize to discard the padding's residue.

Errors map to MagnotiaError::AudioDecodeFailed (one variant for ratio init failure, one for per-chunk process failure).

StreamingResampler (live)

crates/audio/src/streaming_resample.rs. Two states:

• Passthrough — emitted when from_rate == WHISPER_SAMPLE_RATE. push_samples returns input verbatim; flush returns empty.
• Sinc { resampler, residual, ratio } — wraps a SincFixedIn::<f32>. residual buffers samples that don't yet form a complete INPUT_CHUNK = 1024. Each push_samples call drains as many full chunks as it can.

flush() is the load-bearing tail handler:

1. If residual is empty, return empty (no flush work pending).
2. Pad residual up to INPUT_CHUNK zeros.
3. Run resampler.process() once over the padded chunk.
4. Truncate the output to (leftover_real_samples * ratio).round() as usize so the silence introduced by padding does not leak into the saved WAV.

The flush correctness math is verified by the streaming_48k_to_16k_preserves_duration test (streaming_resample.rs:183).

Data flow

File path:

AudioSamples (any rate, mono)
   └─ resample_to_16khz()
       ├─ if rate == 16k → AudioSamples::mono_16khz(clone)
       └─ else: rubato SincFixedIn loop, padded final chunk, truncate to expected_len
   → AudioSamples (16k, mono)

Live path:

cpal AudioChunk (native rate, possibly stereo, downmixed by caller)
   └─ StreamingResampler::push_samples(&mono_f32)
       ├─ residual.extend(mono)
       ├─ while residual.len() >= 1024: resampler.process(chunk) → out
       └─ return out (zero or more 16 kHz samples)
   ...session ends...
   └─ StreamingResampler::flush()
       └─ pad residual to 1024, process, trim padding-induced output
   → 16 kHz mono Vec<f32>

Watch-outs

• Two implementations, one parameter set. Drift between resample.rs and streaming_resample.rs is currently held by convention only. Any change to SincInterpolationParameters must touch both files.
• Caller is responsible for downmixing. Both functions assume mono input. decode.rs averages channels for files; the live session must do the same before pushing to StreamingResampler.
• from_rate = 0 is rejected in both functions (matches a corrupt WAV header path).
• Truncation policy differs slightly. resample_to_16khz uses (samples.len() as f64 * ratio) as usize (truncating cast), StreamingResampler::flush uses .round() as usize. Off by one sample at most; matters for WAV file length parity if you ever compare them.
• No streaming for file path. resample_to_16khz materialises the whole resampled vec in memory. A 90-minute meeting at 48 kHz pre-resample is ~250 MB of f32; the post-resample buffer is another 350 MB. Acceptable today, brittle for very long imports.
• Passthrough still calls to_vec() on the input samples (streaming_resample.rs:95). Cheap but not free; if push frequency is high consider returning Cow.

See also

• Audio capture pipeline — produces the AudioChunks that feed StreamingResampler.
• Audio file decoding — decode_and_resample chains decode + resample_to_16khz.
• lumotia_core::constants::WHISPER_SAMPLE_RATE (slice 5) — single source of truth for the target rate.