audio: fix mic capture — skip monitor sources, validate by RMS, add device picker

Day 1 of the upgrade plan (output/reports/kon-upgrade-plan-2026-04-17.md
in the CORBEL workspace). Fixes the HANDOVER.md blocker: native live
transcription was capturing silence because PulseAudio/PipeWire monitor
sources (speaker loopback) were winning the device-selection race —
they deliver zero-valued bytes that satisfied the original
"any device that produces data within 350ms" check.

WHAT CHANGED:

crates/audio/src/capture.rs (rewrite):
- New `DeviceInfo` struct (serde-derived) for the Settings device picker
- New `MicrophoneCapture::list_devices()` enumerates inputs with metadata
- New `MicrophoneCapture::start_with_device(name)` for explicit selection
- Refactored `start()` with monitor-source filtering by name pattern
  (.monitor suffix, "Monitor of " prefix, "loopback" substring) and
  RMS-energy validation in a 350ms window
- Two-pass selection: real inputs first, monitor sources only as
  last-resort fallback with explicit warning log
- Drop counter (Arc<AtomicU64>) tracks chunks dropped by `try_send`
  failure under load — Codex review caught this as a silent-failure risk
- `dropped_chunks()` accessor for the live session
- Verbose tracing at every step for diagnosis
- Unit test for monitor-name detection
- `cargo check -p kon-audio` passes clean

crates/audio/src/lib.rs:
- Re-export `DeviceInfo`

crates/audio/Cargo.toml:
- Add serde dependency (for DeviceInfo derives)

src-tauri/src/commands/audio.rs:
- New `list_audio_devices` Tauri command (returns Vec<DeviceInfo>)

src-tauri/src/lib.rs:
- Register `list_audio_devices` in invoke_handler

src/lib/pages/SettingsPage.svelte:
- New "Audio" section with microphone picker dropdown
- Auto-populates on mount via `list_audio_devices`
- Refresh button + clear messaging about monitor sources
- Likely-monitor entries marked disabled in the dropdown
- Auto mode is the default (empty string in settings.microphoneDevice)

src/lib/stores/page.svelte.js:
- New `microphoneDevice` field in defaults (empty = auto-select)

NEXT STEPS (per the upgrade plan):
- Wire `microphoneDevice` from settings into `MicrophoneCapture::start_with_device`
  in the live and standalone capture paths (currently both still call
  the auto-selecting `start()`)
- Test on real hardware (Wayland + multiple input devices)
- Codex sanity-check of this diff is running in parallel; addendum to
  follow if anything substantive comes back

Refs: /home/jake/Documents/CORBEL-Projects/kon/HANDOVER.md
      output/reports/kon-upgrade-plan-2026-04-17.md (CORBEL workspace)
This commit is contained in:
2026-04-17 12:43:13 +01:00
parent fdc4a3cba5
commit 96980c7d5c
7 changed files with 805 additions and 103 deletions

View File

@@ -2,6 +2,7 @@ mod commands;
mod tray;
use std::sync::Arc;
use std::time::Instant;
use sqlx::SqlitePool;
use tauri::Manager;
@@ -47,6 +48,7 @@ fn build_preferences_script(prefs_json: Option<String>) -> String {
if (a.fontSize) el.style.setProperty('--font-size-body', a.fontSize + 'px');
if (a.letterSpacing != null) el.style.setProperty('--letter-spacing-body', a.letterSpacing + 'em');
if (a.lineHeight) el.style.setProperty('--line-height-body', String(a.lineHeight));
if (a.transcriptSize) el.style.setProperty('--text-transcript', a.transcriptSize + 'px');
if (a.bionicReading) el.dataset.bionicReading = 'true';
if (a.reduceMotion === 'on' || (a.reduceMotion === 'system' && window.matchMedia('(prefers-reduced-motion: reduce)').matches)) el.dataset.reduceMotion = 'true';
}}
@@ -75,15 +77,19 @@ pub fn run() {
.setup(|app| {
// Initialise database (blocking in setup — runs once at startup)
let db_path = database_path();
let t0 = Instant::now();
let db = tauri::async_runtime::block_on(async {
init_db(&db_path).await
})
.map_err(|e| Box::new(e) as Box<dyn std::error::Error>)?;
eprintln!("[startup] DB init: {:?}", t0.elapsed());
// Load saved preferences for webview injection
let t1 = Instant::now();
let prefs_json = tauri::async_runtime::block_on(async {
get_setting(&db, "kon_preferences").await.unwrap_or(None)
});
eprintln!("[startup] Preferences load: {:?}", t1.elapsed());
let init_script = build_preferences_script(prefs_json);
// Apply preferences to the main window (defined in tauri.conf.json)
@@ -136,6 +142,8 @@ pub fn run() {
app.manage(PreferencesScript(init_script));
app.manage(commands::hotkey::HotkeyState::new());
app.manage(commands::audio::NativeCaptureState::new());
app.manage(commands::live::LiveTranscriptionState::default());
app.manage(AppState {
whisper_engine: Arc::new(LocalEngine::new(
@@ -162,6 +170,7 @@ pub fn run() {
commands::models::list_models,
commands::models::load_model,
commands::models::check_engine,
commands::models::get_runtime_capabilities,
// Parakeet model management
commands::models::download_parakeet_model,
commands::models::check_parakeet_model,
@@ -174,6 +183,11 @@ pub fn run() {
commands::transcription::transcribe_pcm_parakeet,
// Audio
commands::audio::save_audio,
commands::audio::start_native_capture,
commands::audio::stop_native_capture,
commands::audio::list_audio_devices,
commands::live::start_live_transcription_session,
commands::live::stop_live_transcription_session,
// Windows
commands::windows::open_task_window,
commands::windows::open_viewer_window,