Files
Lumotia/docs/architecture-map/02-tauri-runtime/commands/audio.md
jars a1f3f3f134 docs: architecture map (initial 5-slice generation, 105 pages)
Five-slice navigable map of the entire codebase under
docs/architecture-map/. Each slice is a self-contained
breadcrumbed sub-tree:

  01-frontend (16)              Svelte/SvelteKit UI
  02-tauri-runtime (26)         src-tauri commands + lifecycle
  03-audio-transcription (16)   audio + transcription crates
  04-llm-formatting-mcp (19)    llm, ai-formatting, mcp, cloud
  05-core-storage-hotkey-build  core, storage, hotkey, workspace,
                          (26) CI, dev glue

Plus master README.md and data-flow-end-to-end.md tracing
audio bytes from microphone to FTS5 search to MCP read.

Generated by 5 parallel subagents on 2026/05/09 against
HEAD 3c47000. Each page has YAML frontmatter, file:line code
refs, sibling cross-links, plain-English summaries.

Aggregated debt surfaced (full lists in master README):
RB-08 macOS power assertion, schema head drift v14 vs v15,
VAD blocked on ort version conflict, streaming primitives
not wired into live.rs, no prompt versioning, MCP has no
auth, cloud-providers in-memory keystore, SettingsPage
2 484 LOC, commands/live.rs 1 737 LOC, dual theme system,
brand rename to Lumenote pending across the codebase.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 14:04:13 +01:00

8.4 KiB

name, type, slice, last_verified
name type slice last_verified
Audio capture commands architecture-map-page 02-tauri-runtime 2026/05/09

commands::audio

Where you are: Architecture mapTauri runtimeCommands → Audio

Plain English summary. Owns the native cpal microphone capture path used outside live transcription. Exposes "list input devices", "start native capture", "stop native capture", and "save audio to WAV". Streams 16 kHz mono PCM chunks to the frontend via the native-pcm event. Also owns the deterministic recording-filename generator and the helper that resolves a destination WAV path so the live-transcription module can share it.

At a glance

  • Path: src-tauri/src/commands/audio.rs.
  • LOC: 540.
  • Tauri commands exposed:
    • list_audio_devices(window: WebviewWindow) -> Result<Vec<DeviceInfo>, String> — main-window only. Wraps MicrophoneCapture::list_devices in spawn_blocking.
    • start_native_capture(window, app, state, device_name: Option<String>) -> Result<(), String> — main-window only. Starts a cpal stream, downsamples to 16 kHz mono, streams chunks via native-pcm events, also writes to a temp WAV.
    • stop_native_capture(window, state) -> Result<Vec<f32>, String> — main-window only. Awaits the worker join barrier (RB-06) and returns the accumulated samples.
    • save_audio(window, app, samples, output_folder: Option<String>) -> Result<String, String> — main-window only. Writes a fresh WAV to app_local_data_dir/recordings/ (or a user-chosen folder) and returns its absolute path.
  • Events emitted: native-pcm (payload { samples: Vec<f32> }, ~0.5 s of 16 kHz mono per emit) — src-tauri/src/commands/audio.rs:249 and :274.
  • Depends on: magnotia_audio::{DeviceInfo, MicrophoneCapture, WavWriter, write_wav}, magnotia_core::types::AudioSamples, magnotia_core::constants::WHISPER_SAMPLE_RATE. Plus tokio::{mpsc, Mutex, JoinHandle, spawn_blocking, sleep} and std::sync::{Arc, Mutex, atomic::AtomicBool}.
  • Called from frontend at: dictation page (start / stop), Settings → Audio (device list), file-import flow (save_audio).

What's in here

Constants

  • MAX_NATIVE_CAPTURE_RETURN_SAMPLES = WHISPER_SAMPLE_RATE * 60 * 10 (src-tauri/src/commands/audio.rs:14). 10 minutes of 16 kHz mono. Caps the in-memory return buffer; the temp WAV on disk stays authoritative beyond this.

CaptureWorker (src-tauri/src/commands/audio.rs:34)

Internal struct holding stop_tx: tokio_mpsc::Sender<()> and join: JoinHandle<()>. stop_worker(worker).await (src-tauri/src/commands/audio.rs:44) sends the stop signal and awaits the join. Consumes the worker because stop_tx and join are single-use. Errors from the join are logged and swallowed.

NativeCaptureState (src-tauri/src/commands/audio.rs:53)

Tauri-managed state. Fields:

  • worker: AsyncMutex<Option<CaptureWorker>>tokio::sync::Mutex because stop_worker awaits while holding the lock.
  • all_samples: Arc<Mutex<Vec<f32>>> — capped compatibility buffer returned by stop_native_capture.
  • wav_writer: Arc<Mutex<Option<WavWriter>>> — temp WAV writer, finalised when the worker exits.
  • temp_audio_path: Arc<Mutex<Option<PathBuf>>> — destination for the temp WAV.
  • capture_truncated: Arc<AtomicBool> — set when all_samples was capped (the temp WAV still has the full capture).

append_recorded_chunk (src-tauri/src/commands/audio.rs:79)

Helper that pushes a downsampled chunk into both the WAV writer and the in-memory buffer, respecting the buffer cap and flipping the capture_truncated flag when it bumps the ceiling.

start_native_capture (src-tauri/src/commands/audio.rs:113)

Step-by-step:

  1. ensure_main_window(&window) — secondary windows can't start a capture.
  2. Stop any in-flight worker and await its termination (RB-06 — without the join, all_samples.clear() below would race a draining worker and a rapid start→stop→start could leak old-session samples into the new session).
  3. Call MicrophoneCapture::start (or start_with_device(name)) inside spawn_blocking — the underlying cpal probe spends up to DEVICE_VALIDATION_MS * N per pass and would block the async runtime otherwise (Codex review 2026/04/17 D2).
  4. Open a temp WAV via WavWriter::create(&temp_path, WHISPER_SAMPLE_RATE, 1).
  5. Spawn the worker. The worker loop:
    • Polls the stop channel non-blocking.
    • Drains cpal::AudioChunks from the capture mpsc, distinguishing Empty (try again) from Disconnected (stream is dead, exit) — Codex review 2026/04/17 M3.
    • Downmixes to mono if the source is stereo (chunk.samples.chunks(channels).map(|frame| sum / channels)).
    • Decimates to 16 kHz (simple ratio-based decimation, matches the frontend pcm-processor.js pattern).
    • When the buffer ≥ 8000 samples (~0.5 s), drains, calls append_recorded_chunk, and emits native-pcm.
    • On exit, flushes any remaining samples, drops the cpal capture, finalises the WAV.
  6. Stash the worker in state.worker.

stop_native_capture (src-tauri/src/commands/audio.rs:306)

Takes the worker out of state, calls stop_worker(worker).await, then mem::takes all_samples and returns it. Logs a warning if the buffer was truncated.

resolve_recording_path (src-tauri/src/commands/audio.rs:333)

Public helper used by commands::live::start_live_transcription_session. Resolves the recordings dir (user-supplied folder if non-empty, else app_local_data_dir/recordings/), creates it, and returns dir.join(recording_filename()).

recording_filename (src-tauri/src/commands/audio.rs:365)

Deterministic filename: magnotia-<unix_secs>-<nanos:09>-<counter:04>.wav. The counter is a process-lifetime AtomicU64 (RECORDING_COUNTER) bumped on each call. The combination defeats wall-clock collisions both across launches (secs change) and within the same nanosecond (counter changes).

persist_audio_samples and save_audio (src-tauri/src/commands/audio.rs:512, :531)

save_audio is the public command. Calls persist_audio_samples, which resolves the recording path, then calls magnotia_audio::write_wav inside spawn_blocking. Returns the absolute path as a string.

Data flow

  • Start path: start_native_capture(deviceName) → cpal opens stream → worker downmixes/downsamples → emits native-pcm events at 0.5 s cadence → frontend dictation page accumulates samples or feeds them to a transcription command.
  • Stop path: frontend invokes stop_native_capture → worker stop channel signalled → join awaited → samples returned (capped at 10 minutes) → frontend can request save_audio to persist.
  • Live transcription path: commands::live does not use start_native_capture directly; it owns its own MicrophoneCapture::start invocation but reuses resolve_recording_path and recording_filename from this file.

Watch-outs

  • The in-memory buffer is capped at 10 minutes. Anything longer relies on the temp WAV; the frontend cannot just stop_native_capture and treat the returned vec as ground truth for long sessions. Today's UI assumes short captures, but the cap should be surfaced if you build a long-form recorder.
  • The downsampler is naive decimation. Acceptable for speech but not for music. If you ever extend Magnotia to general audio, swap in magnotia_audio::StreamingResampler (which is what commands::live uses).
  • start_native_capture does not engage PowerAssertion (App Nap pinning). commands::live::run_live_session does. If you build a long-form non-live recorder on top of this, add the assertion.
  • The worker emits native-pcm to the entire app handle (not a specific window). Every webview that listens will receive the chunks. If you ever route audio to a non-main window, double-check this is what you want.
  • stop_worker_awaits_full_termination_no_writes_after_join (src-tauri/src/commands/audio.rs:454) is the regression test for RB-06. Do not change stop_worker to a fire-and-forget pattern.

See also

  • Live transcription — uses resolve_recording_path and recording_filename from this file.
  • Transcription — receives Vec<f32> from stop_native_capture and runs Whisper / Parakeet on it.
  • Modelsensure_model_loaded is invoked by the transcription commands before they consume audio.
  • Cargo and featuresarboard and the audio crate dependency live there.