Jake
26c7307607
Phase 9 of the rebrand cascade. Sweep covers everything the Phase 8
frontend pass deliberately skipped: docs/, root markdown, scripts,
Cargo.toml descriptions, code comments that survived earlier
word-boundary sed, plus a handful of identifiers caught on the final
verify pass.
transcription-app changes:
- README.md, HANDOVER.md, KNOWN-ISSUES.md, run.sh — magnotia/Magnotia
-> lumotia/Lumotia.
- docs/ — sweep across all subdirs except docs/handovers/ (preserved
as immutable audit trail). Includes architecture-map references
to magnotia_core::*, magnotia_storage::*, etc. now pointing at
lumotia_*; dev-setup.md tracing output examples (lumotia_startup
target); brief/ + superpowers/ + issues/ + whisper-ecosystem/ +
audit/.
- Cargo.toml descriptions on 9 crates (core, audio, cloud-providers,
hotkey, llm, mcp, plus referenced others).
- crates/core/src/{error,hardware,recommendation,paths}.rs +
crates/audio/src/wav.rs + crates/llm/src/model_manager.rs +
crates/cloud-providers/src/keystore.rs + crates/mcp/src/lib.rs —
doc comments and a model-manager user-agent string.
- Caught on final pass: BroadcastChannel("magnotia_task_sync") -> ...
("lumotia_task_sync"); magnotia_locale i18n localStorage key
renamed + migration shim added; CSS keyframe names
magnotiaPulse / magnotiaBar / magnotiaFade renamed in the design-
system kit; magnotia_viewer_item / magnotia_viewer_mode handoff
keys renamed in HistoryPage + viewer/+page.svelte; src/assets/
wordmark.svg text.
- src-tauri/src/lib.rs comment cleanup ("magnotia era" was sed'd
to "lumotia era" earlier — restored).
Preserved (intentional):
- crates/core/src/paths.rs — keeps "magnotia" / "Magnotia" / ".magnotia"
legacy detection strings in legacy_and_target_paths() so the
migration shim can still find user data from the magnotia era.
- src/lib/stores/{page,focusTimer}.svelte.ts + src/lib/i18n/index.ts
— migration call sites reference the legacy magnotia keys
deliberately.
- docs/handovers/ — historical audit trail.
cargo build --workspace passes. npm run check: 0 errors / 0 warnings
(3958 files). cargo test --workspace: 339 pass / 0 fail.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
121 lines
8.6 KiB
Markdown
121 lines
8.6 KiB
Markdown
---
|
|
name: Audio capture pipeline (cpal)
|
|
type: architecture-map-page
|
|
slice: 03-audio-transcription
|
|
last_verified: 2026/05/09
|
|
---
|
|
|
|
# Audio capture pipeline (cpal)
|
|
|
|
> **Where you are:** [Architecture map](../README.md) → [Audio + Transcription](README.md) → Audio capture pipeline
|
|
|
|
**Plain English summary.** The microphone-capture path enumerates host input devices, picks one (default first, then non-monitor sources, monitor sources only as a last resort), validates it produces real audio energy via a 350 ms RMS sniff, then opens a `cpal` stream that converts every supported sample format to `f32` and pushes `AudioChunk`s into a bounded `mpsc` channel. Runtime errors during the stream (mic unplug, audio server crash) are forwarded on a separate channel so the live session can show a toast.
|
|
|
|
## At a glance
|
|
|
|
- Crate: `lumotia-audio`
|
|
- Path: `crates/audio/src/capture.rs`
|
|
- LOC: 583
|
|
- External deps: `cpal 0.17`, `serde 1` (DeviceInfo wire type)
|
|
- Internal callers (best effort, slice 2 reconciles): `src-tauri/src/commands/audio.rs` (device list), `src-tauri/src/commands/live.rs` (start/stop, chunk + error channels).
|
|
|
|
Public surface:
|
|
|
|
- `pub struct AudioChunk` — `crates/audio/src/capture.rs:24` — `samples: Vec<f32>`, `sample_rate: u32`, `channels: u16`.
|
|
- `pub struct DeviceInfo` — `crates/audio/src/capture.rs:33` — Serde-derived for the Tauri IPC boundary.
|
|
- `pub struct CaptureRuntimeError` — `crates/audio/src/capture.rs:58` — non-fatal stream error reported after `start()` returned.
|
|
- `pub struct MicrophoneCapture` — `crates/audio/src/capture.rs:64`.
|
|
- `pub fn MicrophoneCapture::list_devices() -> Result<Vec<DeviceInfo>>` — `crates/audio/src/capture.rs:94`.
|
|
- `pub fn MicrophoneCapture::start_with_device(name: &str) -> Result<(Self, mpsc::Receiver<AudioChunk>)>` — `crates/audio/src/capture.rs:137`.
|
|
- `pub fn MicrophoneCapture::start() -> Result<(Self, mpsc::Receiver<AudioChunk>)>` — `crates/audio/src/capture.rs:166`.
|
|
- `pub fn MicrophoneCapture::stop(&mut self)` — `crates/audio/src/capture.rs:240`.
|
|
- `pub fn MicrophoneCapture::dropped_chunks(&self) -> u64` — `crates/audio/src/capture.rs:80`.
|
|
- `pub fn MicrophoneCapture::take_error_rx(&mut self) -> Option<mpsc::Receiver<CaptureRuntimeError>>` — `crates/audio/src/capture.rs:88`.
|
|
|
|
`MicrophoneCapture` implements `Drop` (`crates/audio/src/capture.rs:247`) so a panicked caller still pauses the cpal stream.
|
|
|
|
## What's in here
|
|
|
|
### Constants and tunables
|
|
|
|
- `AUDIO_CHANNEL_CAPACITY = 32` (`capture.rs:11`) — bounded capacity for the chunk channel.
|
|
- `DEVICE_VALIDATION_MS = 350` (`capture.rs:15`) — how long the sniffer listens before deciding a device is real.
|
|
- `SILENCE_RMS_FLOOR = 1e-5` (`capture.rs:21`) — lower bound for "produced real audio" during validation.
|
|
- `DEAD_SILENCE_FLOOR = 1e-7` (`capture.rs:475`) — even fallback (monitor) sources must clear this; pure dead-zero is rejected.
|
|
|
|
### Device selection (auto)
|
|
|
|
`start()` (`capture.rs:166`) sorts every input device into four tiers, tries each in order, and stops at the first that passes RMS validation:
|
|
|
|
1. Default + non-monitor (key 0).
|
|
2. Any other non-monitor (key 1).
|
|
3. Default but is a monitor (key 2, very rare).
|
|
4. Monitor source last resort (key 3).
|
|
|
|
The first pass enforces `require_audio = true`. If nothing clears the silence floor the loop does a second pass with `require_audio = false`, which still rejects dead silence but accepts a monitor source as a fallback.
|
|
|
|
Monitor detection (`is_monitor_name`, `capture.rs:258`) catches the standard PulseAudio / PipeWire patterns: `.monitor` suffix, `Monitor of ` prefix, anything containing `loopback`.
|
|
|
|
### Device selection (explicit)
|
|
|
|
`start_with_device()` (`capture.rs:137`) takes an exact device name (matched against `cpal::Device::description().name()`) and refuses to fall back. It still runs the same RMS validation. Returns a clear "open Settings → Audio" message if the requested device is no longer enumerated.
|
|
|
|
### ALSA description enrichment (Linux only)
|
|
|
|
`load_alsa_card_descriptions()` (`capture.rs:297`) parses `/proc/asound/cards` to map cpal's terse short name (`Microphones`) to the human-readable product string (`Blue Microphones`). Empty map on non-Linux. `extract_card_id()` (`capture.rs:278`) pulls `CARD=...` out of an ALSA device string.
|
|
|
|
### Stream construction
|
|
|
|
`open_and_validate()` (`capture.rs:351`) is the single entry point that:
|
|
|
|
1. Reads `default_input_config()` for sample rate, channel count, sample format.
|
|
2. Creates a `mpsc::sync_channel::<AudioChunk>(AUDIO_CHANNEL_CAPACITY)`.
|
|
3. Creates a `mpsc::sync_channel::<CaptureRuntimeError>(32)` for runtime errors.
|
|
4. Dispatches to `build_input_stream::<T>` for the device's sample format (F32 / I16 / U16). Anything else returns `MagnotiaError::AudioCaptureFailed`.
|
|
5. Calls `stream.play()`.
|
|
6. Sniffs samples for `DEVICE_VALIDATION_MS`, sums squared samples, derives RMS.
|
|
7. Rejects below floor (or dead silence). Otherwise re-queues the validation chunks back into the channel so downstream consumers do not lose the first 350 ms.
|
|
|
|
`build_input_stream::<T>` (`capture.rs:505`) is generic over `T: Sample + SizedSample` with `f32: FromSample<T>` so the same body handles all three sample formats. The data callback maps every sample to `f32`, packages an `AudioChunk`, and `try_send`s on the channel; failure increments the `dropped_chunks` atomic. The error callback ships a `CaptureRuntimeError` on `err_tx` (channel-full path increments `dropped_errors` and logs to stderr).
|
|
|
|
### Drop counters
|
|
|
|
Two atomics give the live session visibility:
|
|
|
|
- `dropped_chunks: Arc<AtomicU64>` — chunk channel was full. Diagnostic for downstream backpressure.
|
|
- `dropped_errors: Arc<AtomicU64>` (private) — runtime-error channel was full (caller stopped draining). Logged to stderr each time.
|
|
|
|
## Data flow
|
|
|
|
```
|
|
cpal default_host()
|
|
└─ input_devices() (enumerate)
|
|
└─ open_and_validate(device)
|
|
├─ build_input_stream::<f32|i16|u16>(...)
|
|
│ └─ data callback: T → f32 → AudioChunk → mpsc::sync_channel
|
|
│ └─ error callback: cpal::StreamError → CaptureRuntimeError → err_mpsc
|
|
└─ stream.play()
|
|
└─ 350 ms sniff → RMS → accept | reject
|
|
└─ on accept: re-queue collected chunks, return MicrophoneCapture + Receiver<AudioChunk>
|
|
```
|
|
|
|
Output: one `AudioChunk` per cpal callback period at the device's native rate. The live session is responsible for downmixing channels (if `channels > 1`) and feeding `StreamingResampler` to reach 16 kHz mono. Native rate is *not* normalised here.
|
|
|
|
## Watch-outs
|
|
|
|
- **Channel capacity is 32 chunks.** At a 1024-frame cpal buffer at 48 kHz that's roughly 700 ms. A blocked consumer for longer than that means dropped audio. `dropped_chunks()` is the visibility hook; the live-session command must surface it.
|
|
- **Default-device first works against the safest pick on Linux Pulse setups** where the default sink monitor sneaks in. The four-tier sort handles this, but only because monitor names match the patterns in `is_monitor_name`. New PipeWire schemes that don't include `.monitor` / `loopback` would slip through.
|
|
- **350 ms validation window adds a startup latency floor.** Slice 2 needs to know about this when wiring "click record".
|
|
- **`stop()` is `pause`, not `drop`.** The stream object is kept alive until `Drop`. A subsequent `start()` on the same `MicrophoneCapture` is not supported (signature returns a fresh instance).
|
|
- **Sample format dispatch is closed-set.** Anything not F32 / I16 / U16 is a hard error. cpal can in principle expose I8 / I32 / F64 on exotic devices.
|
|
- **`device_display_name` swallows errors.** `cpal::Device::description()` errors silently become `None`, then `<unnamed>` downstream. Acceptable for a UI list, surprising for debugging.
|
|
- **Re-queue uses `try_send` on a channel of capacity 32.** If the sniff produced more than 32 chunks (≈64 ms at 48 kHz 256-frame buffers — uncommon but possible), the early ones are dropped against the same `dropped_chunks` counter. Documented at `capture.rs:486`.
|
|
|
|
## See also
|
|
|
|
- [Audio resampling](audio-resampling.md) — the live session feeds `AudioChunk.samples` into `StreamingResampler` to get to 16 kHz.
|
|
- [Audio VAD](audio-vad.md) — what happens to chunks once they're at 16 kHz.
|
|
- [Audio WAV I/O](audio-wav-io.md) — `WavWriter` is the crash-safe sibling that persists capture audio to disk.
|
|
- [Audio PCM bridge](audio-pcm-bridge.md) — the static AudioWorklet that exists for browser-side fallback paths.
|
|
- Tests at `crates/audio/src/capture.rs:567` cover monitor pattern detection. The validation-window logic is exercised end-to-end through the live integration tests in slice 2.
|