Jake
96980c7d5c
Day 1 of the upgrade plan (output/reports/kon-upgrade-plan-2026-04-17.md
in the CORBEL workspace). Fixes the HANDOVER.md blocker: native live
transcription was capturing silence because PulseAudio/PipeWire monitor
sources (speaker loopback) were winning the device-selection race —
they deliver zero-valued bytes that satisfied the original
"any device that produces data within 350ms" check.
WHAT CHANGED:
crates/audio/src/capture.rs (rewrite):
- New `DeviceInfo` struct (serde-derived) for the Settings device picker
- New `MicrophoneCapture::list_devices()` enumerates inputs with metadata
- New `MicrophoneCapture::start_with_device(name)` for explicit selection
- Refactored `start()` with monitor-source filtering by name pattern
(.monitor suffix, "Monitor of " prefix, "loopback" substring) and
RMS-energy validation in a 350ms window
- Two-pass selection: real inputs first, monitor sources only as
last-resort fallback with explicit warning log
- Drop counter (Arc<AtomicU64>) tracks chunks dropped by `try_send`
failure under load — Codex review caught this as a silent-failure risk
- `dropped_chunks()` accessor for the live session
- Verbose tracing at every step for diagnosis
- Unit test for monitor-name detection
- `cargo check -p kon-audio` passes clean
crates/audio/src/lib.rs:
- Re-export `DeviceInfo`
crates/audio/Cargo.toml:
- Add serde dependency (for DeviceInfo derives)
src-tauri/src/commands/audio.rs:
- New `list_audio_devices` Tauri command (returns Vec<DeviceInfo>)
src-tauri/src/lib.rs:
- Register `list_audio_devices` in invoke_handler
src/lib/pages/SettingsPage.svelte:
- New "Audio" section with microphone picker dropdown
- Auto-populates on mount via `list_audio_devices`
- Refresh button + clear messaging about monitor sources
- Likely-monitor entries marked disabled in the dropdown
- Auto mode is the default (empty string in settings.microphoneDevice)
src/lib/stores/page.svelte.js:
- New `microphoneDevice` field in defaults (empty = auto-select)
NEXT STEPS (per the upgrade plan):
- Wire `microphoneDevice` from settings into `MicrophoneCapture::start_with_device`
in the live and standalone capture paths (currently both still call
the auto-selecting `start()`)
- Test on real hardware (Wayland + multiple input devices)
- Codex sanity-check of this diff is running in parallel; addendum to
follow if anything substantive comes back
Refs: /home/jake/Documents/CORBEL-Projects/kon/HANDOVER.md
output/reports/kon-upgrade-plan-2026-04-17.md (CORBEL workspace)
31 lines
932 B
TOML
31 lines
932 B
TOML
[package]
|
|
name = "kon-audio"
|
|
version = "0.1.0"
|
|
edition = "2021"
|
|
description = "Audio capture (cpal), VAD, resampling (rubato), file decoding (symphonia), WAV I/O (hound) for Kon"
|
|
|
|
[dependencies]
|
|
kon-core = { path = "../core" }
|
|
|
|
# Microphone capture
|
|
cpal = "0.17"
|
|
|
|
# Voice activity detection — deferred until ort version conflict between
|
|
# VAD crates (ort rc.10) and transcribe-rs (ort rc.12) is resolved upstream.
|
|
# silero-vad-rust = { version = "6", default-features = false }
|
|
|
|
# High-quality resampling (sinc interpolation)
|
|
rubato = "0.15"
|
|
|
|
# WAV file I/O
|
|
hound = "3.5"
|
|
|
|
# Audio file decoding (mp3, aac, flac, wav, ogg, etc.)
|
|
symphonia = { version = "0.5", features = ["mp3", "aac", "flac", "pcm", "vorbis", "wav", "ogg", "isomp4"] }
|
|
|
|
# Async runtime for threading
|
|
tokio = { version = "1", features = ["rt", "sync"] }
|
|
|
|
# Serde for DeviceInfo (returned across the Tauri boundary)
|
|
serde = { version = "1", features = ["derive"] }
|