Lumotia/docs/architecture-map/03-audio-transcription/audio-capture-pipeline.md

name, type, slice, last_verified

name	type	slice	last_verified
Audio capture pipeline (cpal)	architecture-map-page	03-audio-transcription	2026/05/09

Audio capture pipeline (cpal)

Where you are: Architecture map → Audio + Transcription → Audio capture pipeline

Plain English summary. The microphone-capture path enumerates host input devices, picks one (default first, then non-monitor sources, monitor sources only as a last resort), validates it produces real audio energy via a 350 ms RMS sniff, then opens a cpal stream that converts every supported sample format to f32 and pushes AudioChunks into a bounded mpsc channel. Runtime errors during the stream (mic unplug, audio server crash) are forwarded on a separate channel so the live session can show a toast.

At a glance

• Crate: lumotia-audio
• Path: crates/audio/src/capture.rs
• LOC: 583
• External deps: cpal 0.17, serde 1 (DeviceInfo wire type)
• Internal callers (best effort, slice 2 reconciles): src-tauri/src/commands/audio.rs (device list), src-tauri/src/commands/live.rs (start/stop, chunk + error channels).

Public surface:

• pub struct AudioChunk — crates/audio/src/capture.rs:24 — samples: Vec<f32>, sample_rate: u32, channels: u16.
• pub struct DeviceInfo — crates/audio/src/capture.rs:33 — Serde-derived for the Tauri IPC boundary.
• pub struct CaptureRuntimeError — crates/audio/src/capture.rs:58 — non-fatal stream error reported after start() returned.
• pub struct MicrophoneCapture — crates/audio/src/capture.rs:64.
• pub fn MicrophoneCapture::list_devices() -> Result<Vec<DeviceInfo>> — crates/audio/src/capture.rs:94.
• pub fn MicrophoneCapture::start_with_device(name: &str) -> Result<(Self, mpsc::Receiver<AudioChunk>)> — crates/audio/src/capture.rs:137.
• pub fn MicrophoneCapture::start() -> Result<(Self, mpsc::Receiver<AudioChunk>)> — crates/audio/src/capture.rs:166.
• pub fn MicrophoneCapture::stop(&mut self) — crates/audio/src/capture.rs:240.
• pub fn MicrophoneCapture::dropped_chunks(&self) -> u64 — crates/audio/src/capture.rs:80.
• pub fn MicrophoneCapture::take_error_rx(&mut self) -> Option<mpsc::Receiver<CaptureRuntimeError>> — crates/audio/src/capture.rs:88.

MicrophoneCapture implements Drop (crates/audio/src/capture.rs:247) so a panicked caller still pauses the cpal stream.

What's in here

Constants and tunables

• AUDIO_CHANNEL_CAPACITY = 32 (capture.rs:11) — bounded capacity for the chunk channel.
• DEVICE_VALIDATION_MS = 350 (capture.rs:15) — how long the sniffer listens before deciding a device is real.
• SILENCE_RMS_FLOOR = 1e-5 (capture.rs:21) — lower bound for "produced real audio" during validation.
• DEAD_SILENCE_FLOOR = 1e-7 (capture.rs:475) — even fallback (monitor) sources must clear this; pure dead-zero is rejected.

Device selection (auto)

start() (capture.rs:166) sorts every input device into four tiers, tries each in order, and stops at the first that passes RMS validation:

1. Default + non-monitor (key 0).
2. Any other non-monitor (key 1).
3. Default but is a monitor (key 2, very rare).
4. Monitor source last resort (key 3).

The first pass enforces require_audio = true. If nothing clears the silence floor the loop does a second pass with require_audio = false, which still rejects dead silence but accepts a monitor source as a fallback.

Monitor detection (is_monitor_name, capture.rs:258) catches the standard PulseAudio / PipeWire patterns: .monitor suffix, Monitor of prefix, anything containing loopback.

Device selection (explicit)

start_with_device() (capture.rs:137) takes an exact device name (matched against cpal::Device::description().name()) and refuses to fall back. It still runs the same RMS validation. Returns a clear "open Settings → Audio" message if the requested device is no longer enumerated.

ALSA description enrichment (Linux only)

load_alsa_card_descriptions() (capture.rs:297) parses /proc/asound/cards to map cpal's terse short name (Microphones) to the human-readable product string (Blue Microphones). Empty map on non-Linux. extract_card_id() (capture.rs:278) pulls CARD=... out of an ALSA device string.

Stream construction

open_and_validate() (capture.rs:351) is the single entry point that:

1. Reads default_input_config() for sample rate, channel count, sample format.
2. Creates a mpsc::sync_channel::<AudioChunk>(AUDIO_CHANNEL_CAPACITY).
3. Creates a mpsc::sync_channel::<CaptureRuntimeError>(32) for runtime errors.
4. Dispatches to build_input_stream::<T> for the device's sample format (F32 / I16 / U16). Anything else returns MagnotiaError::AudioCaptureFailed.
5. Calls stream.play().
6. Sniffs samples for DEVICE_VALIDATION_MS, sums squared samples, derives RMS.
7. Rejects below floor (or dead silence). Otherwise hands the validation chunks back to the consumer as a VecDeque pre-roll alongside the live mpsc::Receiver, so downstream consumers do not lose the first 350 ms even on small-buffer hosts that would overflow the bounded channel.

build_input_stream::<T> (capture.rs:505) is generic over T: Sample + SizedSample with f32: FromSample<T> so the same body handles all three sample formats. The data callback maps every sample to f32, packages an AudioChunk, and try_sends on the channel; failure increments the dropped_chunks atomic. The error callback ships a CaptureRuntimeError on err_tx (channel-full path increments dropped_errors and logs to stderr).

Drop counters

Two atomics give the live session visibility:

• dropped_chunks: Arc<AtomicU64> — chunk channel was full. Diagnostic for downstream backpressure.
• dropped_errors: Arc<AtomicU64> (private) — runtime-error channel was full (caller stopped draining). Logged to stderr each time.

Data flow

cpal default_host()
   └─ input_devices()         (enumerate)
       └─ open_and_validate(device)
           ├─ build_input_stream::<f32|i16|u16>(...)
           │     └─ data callback: T → f32 → AudioChunk → mpsc::sync_channel
           │     └─ error callback: cpal::StreamError → CaptureRuntimeError → err_mpsc
           └─ stream.play()
               └─ 350 ms sniff → RMS → accept | reject
                   └─ on accept: return MicrophoneCapture + VecDeque<AudioChunk> pre-roll + Receiver<AudioChunk>

Output: one AudioChunk per cpal callback period at the device's native rate. The live session is responsible for downmixing channels (if channels > 1) and feeding StreamingResampler to reach 16 kHz mono. Native rate is not normalised here.

Watch-outs

• Channel capacity is 32 chunks. At a 1024-frame cpal buffer at 48 kHz that's roughly 700 ms. A blocked consumer for longer than that means dropped audio. dropped_chunks() is the visibility hook; the live-session command reads it on every recv_audio tick (commands/live.rs::poll_capture_drops) and converts the delta into the dropped_audio_ms surfaced to the UI overlay.
• Default-device first works against the safest pick on Linux Pulse setups where the default sink monitor sneaks in. The four-tier sort handles this, but only because monitor names match the patterns in is_monitor_name. New PipeWire schemes that don't include .monitor / loopback would slip through.
• 350 ms validation window adds a startup latency floor. Slice 2 needs to know about this when wiring "click record".
• stop() is pause, not drop. The stream object is kept alive until Drop. A subsequent start() on the same MicrophoneCapture is not supported (signature returns a fresh instance).
• Sample format dispatch is closed-set. Anything not F32 / I16 / U16 is a hard error. cpal can in principle expose I8 / I32 / F64 on exotic devices.
• device_display_name swallows errors. cpal::Device::description() errors silently become None, then <unnamed> downstream. Acceptable for a UI list, surprising for debugging.
• Validation pre-roll is returned out-of-band, not requeued. open_and_validate returns a VecDeque<AudioChunk> alongside the live mpsc::Receiver; the live-session consumer drains the deque before reading the channel. This bypasses the 32-slot cap entirely so small-buffer hosts (WASAPI exclusive, low-latency ALSA at 256 frames) don't lose ~150ms from the head of every recording. Earlier versions used try_send to requeue and silently dropped half of the pre-roll on those hosts.

See also

• Audio resampling — the live session feeds AudioChunk.samples into StreamingResampler to get to 16 kHz.
• Audio VAD — what happens to chunks once they're at 16 kHz.
• Audio WAV I/O — WavWriter is the crash-safe sibling that persists capture audio to disk.
• Audio PCM bridge — the static AudioWorklet that exists for browser-side fallback paths.
• Tests at crates/audio/src/capture.rs:567 cover monitor pattern detection. The validation-window logic is exercised end-to-end through the live integration tests in slice 2.