Lumotia/docs/architecture-map/03-audio-transcription/audio-resampling.md

---
name: Audio resampling (rubato)
type: architecture-map-page
slice: 03-audio-transcription
last_verified: 2026/05/09
---

# Audio resampling (rubato)

> **Where you are:** [Architecture map](../README.md) → [Audio + Transcription](README.md) → Audio resampling

**Plain English summary.** Whisper and Parakeet want 16 kHz mono `f32`. Microphones and audio files almost never deliver that natively. Two `rubato`-backed resamplers convert: `resample_to_16khz` for one-shot file conversion, `StreamingResampler` for the live session where samples arrive in arbitrary chunks and a flush at the end drains the resampler's tail.

## At a glance

- Crate: `lumotia-audio`
- Paths: `crates/audio/src/resample.rs` (100 LOC), `crates/audio/src/streaming_resample.rs` (211 LOC).
- External deps: `rubato 0.15` (sinc interpolation).
- Internal callers (best effort, slice 2 reconciles): `concurrency::decode_and_resample` (file path used by `audio.rs` and `transcription.rs` Tauri commands), `live.rs` Tauri command (live path).

Public surface:

- `pub fn resample_to_16khz(audio: &AudioSamples) -> Result<AudioSamples>` — `crates/audio/src/resample.rs:11`.
- `pub enum StreamingResampler` — `crates/audio/src/streaming_resample.rs:37`. Variants: `Passthrough`, `Sinc { resampler, residual, ratio }`.
- `pub fn StreamingResampler::new(from_rate: u32) -> Result<Self>` — `crates/audio/src/streaming_resample.rs:52`.
- `pub fn StreamingResampler::push_samples(&mut self, mono: &[f32]) -> Result<Vec<f32>>` — `crates/audio/src/streaming_resample.rs:93`.
- `pub fn StreamingResampler::flush(&mut self) -> Result<Vec<f32>>` — `crates/audio/src/streaming_resample.rs:127`.

## What's in here

### Shared sinc parameters

Both files use the same `SincInterpolationParameters`:

```rust
sinc_len: 256,
f_cutoff: 0.95,
oversampling_factor: 128,
interpolation: SincInterpolationType::Cubic,
window: WindowFunction::Blackman,
```

`max_relative_jitter = 1.1`, `chunks_in = 1024`, channels = 1.

### `resample_to_16khz` (file)

`crates/audio/src/resample.rs:11`. Pure function over an immutable `AudioSamples`. Short-circuits the passthrough case (already 16 kHz). Iterates the input in 1024-sample chunks; pads the trailing partial chunk to `chunk_size` zeros; truncates the output to `(input_len * ratio) as usize` to discard the padding's residue.

Errors map to `MagnotiaError::AudioDecodeFailed` (one variant for ratio init failure, one for per-chunk process failure).

### `StreamingResampler` (live)

`crates/audio/src/streaming_resample.rs`. Two states:

- `Passthrough` — emitted when `from_rate == WHISPER_SAMPLE_RATE`. `push_samples` returns input verbatim; `flush` returns empty.
- `Sinc { resampler, residual, ratio }` — wraps a `SincFixedIn::<f32>`. `residual` buffers samples that don't yet form a complete `INPUT_CHUNK = 1024`. Each `push_samples` call drains as many full chunks as it can.

`flush()` is the load-bearing tail handler:

1. If `residual` is empty, return empty (no flush work pending).
2. Pad `residual` up to `INPUT_CHUNK` zeros.
3. Run `resampler.process()` once over the padded chunk.
4. Truncate the output to `(leftover_real_samples * ratio).round() as usize` so the silence introduced by padding does not leak into the saved WAV.

The `flush` correctness math is verified by the `streaming_48k_to_16k_preserves_duration` test (`streaming_resample.rs:183`).

## Data flow

File path:

```
AudioSamples (any rate, mono)
   └─ resample_to_16khz()
       ├─ if rate == 16k → AudioSamples::mono_16khz(clone)
       └─ else: rubato SincFixedIn loop, padded final chunk, truncate to expected_len
   → AudioSamples (16k, mono)
```

Live path:

```
cpal AudioChunk (native rate, possibly stereo, downmixed by caller)
   └─ StreamingResampler::push_samples(&mono_f32)
       ├─ residual.extend(mono)
       ├─ while residual.len() >= 1024: resampler.process(chunk) → out
       └─ return out (zero or more 16 kHz samples)
   ...session ends...
   └─ StreamingResampler::flush()
       └─ pad residual to 1024, process, trim padding-induced output
   → 16 kHz mono Vec<f32>
```

## Watch-outs

- **Two implementations, one parameter set.** Drift between `resample.rs` and `streaming_resample.rs` is currently held by convention only. Any change to `SincInterpolationParameters` must touch both files.
- **Caller is responsible for downmixing.** Both functions assume mono input. `decode.rs` averages channels for files; the live session must do the same before pushing to `StreamingResampler`.
- **`from_rate = 0` is rejected** in both functions (matches a corrupt WAV header path).
- **Truncation policy differs slightly.** `resample_to_16khz` uses `(samples.len() as f64 * ratio) as usize` (truncating cast), `StreamingResampler::flush` uses `.round() as usize`. Off by one sample at most; matters for WAV file length parity if you ever compare them.
- **No streaming for file path.** `resample_to_16khz` materialises the whole resampled vec in memory. A 90-minute meeting at 48 kHz pre-resample is ~250 MB of `f32`; the post-resample buffer is another 350 MB. Acceptable today, brittle for very long imports.
- **Passthrough still calls `to_vec()`** on the input samples (`streaming_resample.rs:95`). Cheap but not free; if push frequency is high consider returning `Cow`.

## See also

- [Audio capture pipeline](audio-capture-pipeline.md) — produces the `AudioChunk`s that feed `StreamingResampler`.
- [Audio file decoding](audio-file-decoding.md) — `decode_and_resample` chains decode + `resample_to_16khz`.
- `lumotia_core::constants::WHISPER_SAMPLE_RATE` (slice 5) — single source of truth for the target rate.