Commit Graph

5 Commits

Author SHA1 Message Date
34fce3cf9e feat: OpenWhispr-inspired transcription polish pass
Some checks failed
check / cargo check (macos-latest) (push) Has been cancelled
check / cargo check (ubuntu-22.04) (push) Has been cancelled
check / cargo check (windows-latest) (push) Has been cancelled
check / svelte build + lint (push) Has been cancelled
Major quality pass on top of Phase 2. Five substantive changes plus
cross-cutting touches across audio, hotkey, transcription, and Tauri
command layers.

  Transcription quality

  - Long-audio chunking in commands/transcription.rs: Parakeet and large
    file transcription now chunk-and-recompose with overlap trimming, so
    the live-path chunking advantage extends to file-based workflows.
  - Stateful live speech gate in commands/live.rs on top of the earlier
    duplicate-boundary filtering — distinguishes start-of-speech from
    mid-speech and holds state across chunks.

  Auto-learning corrections

  - New crates/ai-formatting/src/correction_learning.rs: extracts user
    text corrections from viewer edits and proposes additions to the
    active profile's vocabulary.
  - src-tauri/src/commands/profiles.rs bridge for frontend-driven
    confirmation of learned terms.
  - src/routes/viewer/+page.svelte hooks the learning path into the
    segment-edit flow so corrections feed profile_terms without a
    separate 'train this profile' UX.

  Transcript profile provenance

  - Migration v8 (crates/storage/src/migrations.rs) adds profile_id to
    transcripts, defaulting to DEFAULT_PROFILE_ID so existing rows stay
    valid.
  - crates/storage/src/database.rs: TranscriptRow + CRUD carry profile_id.
  - src-tauri/src/commands/transcripts.rs: add_transcript accepts and
    persists profile_id.
  - DictationPage.svelte + FilesPage.svelte send activeProfileId on
    capture so learned corrections are attributed to the right profile.

  Cleanup prompt contract

  - crates/ai-formatting/src/llm_client.rs hardened: the CLEANUP_PROMPT
    now specifies concrete do/do-not rules, ready for a real model-backed
    cleanup pass. The llm_client is still a stub — kon-llm remains unwired
    — but the prompt shape is final.

  Cross-cutting polish

  - Minor touches in audio (capture/decode/resample), hotkey (lib/linux/stub),
    core, transcription (concurrency/model_manager/local_engine/whisper_rs),
    and the rest of src-tauri/src/commands/*: error-path tightening, log
    clarity, TS-migration follow-ups (@ts-nocheck additions for incremental
    typing).

Verified locally: npm run check, cargo test -p kon-ai-formatting,
cargo test -p kon-storage, cargo test -p kon --lib commands::live::tests,
cargo check — all green.

Scope boundary: kon-llm crate is still a stub; task extraction remains
rule-based. Bundled local-LLM runtime is the next clean step and is not
in this commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 22:39:08 +01:00
19a6b83cb2 audio: Day 2 — Codex follow-up hardening (channel disconnect, spawn_blocking, fallback silence guard, requeue counting, runtime error propagation)
Closes the 6 Codex review findings on the Day 1 mic-capture work
(commits 96980c7 + 41db162). Detail in
output/reports/kon-codex-mic-capture-followup-2026-04-17.md (CORBEL
workspace).

src-tauri/src/commands/audio.rs:
- D1: Sending an explicit stop signal before dropping stop_tx, so the
  accumulator task wakes up immediately rather than waiting for the
  Disconnected detection added below.
- D2: Wrapping MicrophoneCapture::start() in tokio::task::spawn_blocking.
  start() can spend up to 350ms × N_devices × 2 passes; running it on
  the async runtime froze other Tauri commands.
- M3: Match on rx.try_recv() error variants. Empty -> sleep + continue.
  Disconnected -> exit accumulator task immediately. Previous behaviour
  was an infinite loop after the capture stream died.

crates/audio/src/capture.rs:
- D3: Added DEAD_SILENCE_FLOOR (1e-7) gate that applies even in the
  fallback pass. PulseAudio/PipeWire can stream zero-valued bytes from
  a borked device; that is worse than failing fast.
- M1: Validation requeue (`for chunk in collected { try_send }`) now
  counts drops in the same dropped_chunks counter.
- M2: New CaptureRuntimeError type. The cpal stream error callback now
  forwards errors via a separate sync_channel (capacity 16) that the
  live session can drain via take_error_rx() and surface as toasts.
  Re-exported from kon_audio crate root.

cargo check -p kon-audio passes clean.

NOT YET DONE (M4): JACK-specific monitor name patterns. Defers until
testing on a JACK host. Current is_monitor_name() may miss JACK
conventions.

Wiring the new error_rx into the live session and surfacing as toasts
lands with the Day 3 error-toast system commit.
2026-04-17 13:02:51 +01:00
41db162041 audio: wire user's microphone choice through start_native_capture + live session
Day 1 follow-up to 96980c7. The device-picker UI in Settings now
actually takes effect: settings.microphoneDevice flows from the Svelte
store, through the Tauri invoke, into MicrophoneCapture::start_with_device
on the Rust side.

Touched paths (back-to-front):
- src-tauri/src/commands/audio.rs:start_native_capture — new optional
  `device_name: Option<String>` parameter; routes to start_with_device
  when set, falls back to auto-select start() when None or empty.
- src-tauri/src/commands/live.rs:StartLiveTranscriptionConfig — new
  optional `microphone_device: Option<String>` field with same
  semantics (rename_all = "camelCase" maps it to microphoneDevice on
  the wire).
- src-tauri/src/commands/live.rs:run_live_session — picks
  start_with_device when an explicit name is provided.
- src/lib/pages/DictationPage.svelte — passes
  microphoneDevice: settings.microphoneDevice || null in the invoke.

Behaviour:
- "Auto" in the picker (empty string) -> backend auto-selects, skipping
  monitor sources and validating by RMS energy.
- Specific device -> backend opens that device by exact name; if it has
  been disconnected the user gets a clear error pointing them back at
  Settings.

cargo check -p kon-audio passes clean. Tauri-crate cargo check requires
cmake (pre-existing infra dependency for whisper-rs-sys); install via
`sudo dnf install cmake clang-devel`.
2026-04-17 12:45:19 +01:00
96980c7d5c audio: fix mic capture — skip monitor sources, validate by RMS, add device picker
Day 1 of the upgrade plan (output/reports/kon-upgrade-plan-2026-04-17.md
in the CORBEL workspace). Fixes the HANDOVER.md blocker: native live
transcription was capturing silence because PulseAudio/PipeWire monitor
sources (speaker loopback) were winning the device-selection race —
they deliver zero-valued bytes that satisfied the original
"any device that produces data within 350ms" check.

WHAT CHANGED:

crates/audio/src/capture.rs (rewrite):
- New `DeviceInfo` struct (serde-derived) for the Settings device picker
- New `MicrophoneCapture::list_devices()` enumerates inputs with metadata
- New `MicrophoneCapture::start_with_device(name)` for explicit selection
- Refactored `start()` with monitor-source filtering by name pattern
  (.monitor suffix, "Monitor of " prefix, "loopback" substring) and
  RMS-energy validation in a 350ms window
- Two-pass selection: real inputs first, monitor sources only as
  last-resort fallback with explicit warning log
- Drop counter (Arc<AtomicU64>) tracks chunks dropped by `try_send`
  failure under load — Codex review caught this as a silent-failure risk
- `dropped_chunks()` accessor for the live session
- Verbose tracing at every step for diagnosis
- Unit test for monitor-name detection
- `cargo check -p kon-audio` passes clean

crates/audio/src/lib.rs:
- Re-export `DeviceInfo`

crates/audio/Cargo.toml:
- Add serde dependency (for DeviceInfo derives)

src-tauri/src/commands/audio.rs:
- New `list_audio_devices` Tauri command (returns Vec<DeviceInfo>)

src-tauri/src/lib.rs:
- Register `list_audio_devices` in invoke_handler

src/lib/pages/SettingsPage.svelte:
- New "Audio" section with microphone picker dropdown
- Auto-populates on mount via `list_audio_devices`
- Refresh button + clear messaging about monitor sources
- Likely-monitor entries marked disabled in the dropdown
- Auto mode is the default (empty string in settings.microphoneDevice)

src/lib/stores/page.svelte.js:
- New `microphoneDevice` field in defaults (empty = auto-select)

NEXT STEPS (per the upgrade plan):
- Wire `microphoneDevice` from settings into `MicrophoneCapture::start_with_device`
  in the live and standalone capture paths (currently both still call
  the auto-selecting `start()`)
- Test on real hardware (Wayland + multiple input devices)
- Codex sanity-check of this diff is running in parallel; addendum to
  follow if anything substantive comes back

Refs: /home/jake/Documents/CORBEL-Projects/kon/HANDOVER.md
      output/reports/kon-upgrade-plan-2026-04-17.md (CORBEL workspace)
2026-04-17 12:43:13 +01:00
jake
29ff91d3f6 feat(kon): wire Tauri shell — thin commands, ProviderRegistry, tray
- lib.rs rewritten from scaffold to full app setup (~70 lines)
- AppState holds Arc<LocalEngine> for Whisper and Parakeet
- commands/models.rs: download, check, list, load for both engines
  Maps legacy size strings (Tiny/Base/Small/Medium) to ModelId
- commands/transcription.rs: transcribe_pcm, transcribe_file, transcribe_pcm_parakeet
  Delegates to LocalEngine.transcribe_sync() + post_process_segments()
- commands/audio.rs: save_audio via kon_audio::write_wav
- commands/windows.rs: open_task_window, open_viewer_window
- commands/llm.rs: 8 stub commands preserved for frontend compatibility
- tray.rs: extracted system tray setup (show, status, quit, click-to-show)
- Close-to-tray behaviour preserved
- All 25 v0.2 command names preserved for Svelte frontend compatibility
- clippy clean

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 21:15:16 +00:00