15 Commits

Author SHA1 Message Date
0e41c95075 chore: remove stale HANDOVER.md, gitignore .firecrawl
Some checks failed
check / cargo check (macos-latest) (push) Has been cancelled
check / cargo check (ubuntu-22.04) (push) Has been cancelled
check / cargo check (windows-latest) (push) Has been cancelled
check / svelte build + lint (push) Has been cancelled
HANDOVER.md was a working-notes file from the 2026/04/04 session that
was never committed. It described live transcription as broken — which
was fixed in the 2026/04/17 sprint. Leaving it untracked was a trap
for anyone cloning the repo. Deleted.

.firecrawl/ added to .gitignore (web-scraping cache, no repo value).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 09:03:35 +01:00
7de2feb932 chore: add CI workflows, dev launcher, and linux capability schema
.github/workflows/check.yml: cargo check on Linux + Windows + macOS
  plus Svelte/Vite build on every push to main and every PR. Fast
  feedback without paying for the full release build.

.github/workflows/build.yml: cross-platform release build producing
  .AppImage/.deb (Linux), .msi/.exe (Windows), .dmg (macOS). Triggers
  on v* tag push (attaches to a draft GitHub Release) or manual
  workflow_dispatch. Signing stubs commented out pending cert decisions.

run.sh: dev launcher that starts Vite, waits for port 1420, then runs
  tauri dev with beforeDevCommand blanked to avoid a second Vite spawn.
  Restores tauri.conf.json on exit via trap.

src-tauri/gen/schemas/linux-schema.json: Tauri-generated capability
  schema for Linux; the other platform schemas (desktop, windows) were
  already committed. Adds the missing sibling.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 09:03:28 +01:00
f384641097 fix(frontend): commit missing utility modules (P0 — imports resolved to nothing)
textMeasure.js, virtualList.js, and accessibilityTypography.js were
added in the 2026/04/04 session alongside VirtualSegmentList.svelte but
never committed. All four are imported from DictationPage, HistoryPage,
SettingsPage, and the viewer — a fresh clone had unresolvable imports.

textMeasure.js: pretext-backed text measurement with LRU cache.
  measureTextHeight, measurePreWrap, clampTextLines, invalidateCache.
virtualList.js: binary-search visible range for variable-height virtual
  lists. buildCumulativeOffsets + findVisibleRange.
accessibilityTypography.js: pretext font/line-height helpers that read
  the user's accessibility preference store (Lexend / Atkinson /
  OpenDyslexic mapping).
VirtualSegmentList.svelte: virtualised transcript segment renderer used
  by the viewer page; depends on all three utils above.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 09:03:17 +01:00
cca79dec4c fix: SQL trigger-aware migration splitter, SpeechModel re-export, i64 type alignment
crates/storage/src/migrations.rs:
  Replace naive sql.split(';') with split_statements() that tracks
  BEGIN...END depth, so migrations containing trigger definitions
  execute correctly instead of being split mid-block.

crates/transcription/src/lib.rs:
  Re-export transcribe_rs::SpeechModel so callers in src-tauri can
  reference it without adding transcribe-rs as a direct dependency.

src-tauri/src/commands/models.rs:
  Use Box<dyn SpeechModel + Send> as the load_model_from_disk return
  type, matching the trait object that transcription crates produce.

src-tauri/src/commands/transcripts.rs:
  Remove the stale .map(|v| v as i32) casts on sample_rate and
  audio_channels. InsertTranscriptParams now stores these as i64
  (ebf449b), matching the i64 fields on CreateTranscriptRequest;
  casting to i32 first would silently truncate large sample rates.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 09:03:01 +01:00
Claude
ebf449b47b qa: restore boot, wire dead error rx, harden storage and config
Front-to-back QC pass turned up two independent missing-module
showstoppers (workspace did not compile; frontend did not load) plus a
handful of HANDOVER-claimed features that were wired but dead. Fixes:

P0 — gets the app booting again:
  - Add the never-committed src/lib/utils/runtime.js (hasTauriRuntime
    detection); 5 imports were resolving to nothing.
  - Add the never-committed crates/audio/src/streaming_resample.rs
    (rubato-backed StreamingResampler with new/push_samples/flush);
    declared in lib.rs and used 3x by live.rs but had no impl.
  - Drop the duplicate hasTauriRuntime import in routes/+layout.svelte.
  - Allow the transcript-viewer window to use the default capability
    (was missing from capabilities/default.json:windows, so the viewer
    window could open but not invoke any Tauri command).

P1 — features documented as working but actually dead:
  - Pump MicrophoneCapture::take_error_rx() into LiveStatusMessage::
    Warning each loop iteration in commands/live.rs. The HANDOVER
    promised cpal stream errors would surface as toasts; the channel
    was created and never read.
  - Replace .expect() on the WebKit media-permission setup with a
    logged warning. Failure no longer aborts the whole process.
  - Toast on save_preferences failure (preferences.svelte.js had a
    silent console.error — now warns once per failure run via the
    existing toasts store).

P2 — correctness/robustness:
  - add_dictionary_entry: switch INSERT OR IGNORE to ON CONFLICT
    DO UPDATE ... RETURNING id so duplicate terms get the real row id
    instead of a stale auto-increment.
  - search_transcripts: qualify ORDER BY fts.rank.
  - InsertTranscriptParams + TranscriptRow: bump sample_rate /
    audio_channels from i32 to i64 to match the Tauri DTO and avoid
    silent truncation at the boundary.
  - Drop the unused tauri-plugin-mcp dependency.
  - Promote sqlx in src-tauri/Cargo.toml from linux-only to
    unconditional (lib.rs names sqlx::SqlitePool unconditionally —
    macOS/Windows builds were latently broken).
  - hotkey/linux.rs: stop panicking the hotplug task on inotify
    failure; degrade to "no hotplug" with a stderr warning.
  - layout.svelte: store the global error/unhandledrejection handler
    refs and remove them in onDestroy so HMR/window teardown doesn't
    leak listeners.

Verified: cargo check -p kon-core -p kon-storage -p kon-cloud-providers
passes. cargo check on src-tauri/kon-audio/kon-hotkey requires alsa +
gtk system libs not present in this sandbox; their changes are
syntactically and type-checked against the rest of the workspace.
svelte-check requires npm install which is not available here.

https://claude.ai/code/session_018ozAs4UcRC8jbJbddqJtEw
2026-04-18 02:00:26 +00:00
4e6ca0ed96 platform: fix macOS data path + add get_os_info + frontend osInfo helper
CROSS-PLATFORM AUDIT:
- Linux x86_64 (Fedora 43, KDE Wayland): HIGH confidence (the dev target)
- Linux other distros / X11: MEDIUM (tested patterns, untested distros)
- Windows 10/11: LOW — theoretically supported via Tauri + CPAL + whisper.cpp
  + tauri-plugin-global-shortcut (the custom evdev hotkey is no-op on
  Windows); has had zero hands-on testing
- macOS Apple Silicon / Intel: LOW — same; macOS Info.plist needs
  NSMicrophoneUsageDescription for the bundled app, and the path bug
  fixed in this commit

REAL BUG FIXED — macOS app_data_dir was Unix-style:
crates/storage/src/file_storage.rs::app_data_dir() previously fell into
the "Unix" branch on macOS and wrote to ~/.kon/, which violates Apple
guidelines and confuses install/uninstall tooling. Now correctly:
- Windows: %LOCALAPPDATA%/kon (unchanged)
- macOS: ~/Library/Application Support/Kon/
- Linux: $XDG_DATA_HOME/kon or ~/.local/share/kon (XDG Base Directory),
         with legacy ~/.kon/ fallback so existing installs keep working
- Other Unix: ~/.kon/ (unchanged)

OS DETECTION LAYER:
- New get_os_info Tauri command in commands/diagnostics.rs returns:
  {os, arch, family, usesCmd, isWayland, customHotkeyBackend,
   primaryModifierLabel}
- New src/lib/utils/osInfo.js helper:
  - async loadOsInfo() warms a cache, called once at root layout mount
  - then isMac() / isWindows() / isLinux() / modKeyLabel() / isWayland()
    are synchronous
  - browser-preview fallback reads navigator.platform
- Lets components localise hotkey labels (Cmd vs Ctrl), file-picker
  copy ("Open Finder" vs "Open Explorer"), etc, without recomputing in
  every place.

cargo check -p kon-storage clean. Updated HANDOVER-2026-04-17.md with
a per-platform confidence table.
2026-04-17 13:52:58 +01:00
62ea58d95a diagnostics: panic hook + frontend error capture + Settings → About diagnostic report
THE BRIEF:
For the friends beta we need verbal feedback AND technical feedback —
some bugs the user cannot describe but a stack trace can. Built it in
two layers, with Layer 3 deferred until there is real volume.

PRIVACY POSTURE (matches Kon's local-first positioning):
- All capture is to disk only. Nothing is transmitted.
- The manual report bundler shows the user exactly what would be
  shared and lets them choose to copy / save it.
- No remote endpoint, no Sentry, no opt-out telemetry.

LAYER 1 — Always-on local capture:
- Rust panic hook (commands/diagnostics.rs::install_panic_hook) writes
  each panic to ~/.kon/crashes/<unix-ts>-<short-id>.crash. Captures
  thread name, OS/arch, RUST_BACKTRACE state, and the panic info.
  Installed before tauri::Builder so it catches setup-time panics too.
- Frontend window.onerror + window.unhandledrejection in
  src/routes/+layout.svelte forward to the new log_frontend_error
  Tauri command, which inserts into the existing error_log SQLite
  table. Best-effort: errors in the error handler itself are swallowed
  so logging can never crash the app.
- Existing error_log table (migration v1) is now actually used —
  closes the TODO from crates/storage/src/database.rs:409.

LAYER 2 — Manual diagnostic report:
- Settings → About → Diagnostics section: Generate report → Preview →
  Copy / Save.
- Report is plain markdown so it pastes cleanly into email, Discord,
  GitHub issues. Sections: app version + OS, sanitised settings JSON,
  recent error_log rows, last 5 crash dumps with previews, log file
  tail (8KB).
- Preview is shown in a <details> with the full text the user can
  inspect before deciding to share.
- Save writes to ~/.kon/diagnostic-reports/kon-diagnostic-<ts>.md.

NEW FILES:
- src-tauri/src/commands/diagnostics.rs (panic hook + 5 Tauri commands)

CHANGES:
- crates/storage/src/file_storage.rs: crashes_dir() + logs_dir() helpers
- crates/storage/src/database.rs: ErrorLogRow + list_recent_errors()
- crates/storage/src/lib.rs: re-exports
- src-tauri/src/commands/mod.rs: + diagnostics module
- src-tauri/src/lib.rs: install_panic_hook() before Builder; 5 new
  Tauri commands registered (log_frontend_error,
  list_recent_errors_command, list_crash_files,
  generate_diagnostic_report, save_diagnostic_report)
- src/routes/+layout.svelte: installGlobalErrorCapture() at mount
- src/lib/pages/SettingsPage.svelte: diagnostic state + buttons +
  preview block at the end of the About section

cargo check -p kon-storage clean. Settings.svelte if/each balanced.

LAYER 3 (deferred):
- Optional opt-in remote reporting (self-hosted Sentry on Tartarus or
  similar). Not for friends beta. Open up after volume justifies it.
2026-04-17 13:47:20 +01:00
524cab0d98 docs: dogfooding readiness HANDOVER for the 2026/04/17 sprint
Captures the full 7-commit arc (96980c79f3be5c) covering Days 1-6 of
the upgrade plan: mic capture fix, Codex follow-up hardening, toast
system, SQLite as canonical store with FTS5 + update_transcript +
dictionary, Settings → Audio + Vocabulary panels, Wayland self-relaunch.

Includes:
- One-time setup instructions (cmake + clang-devel)
- Six concrete tests for friendly-user verification
- What's deferred (Whisper pre-warm, auto-updater, JACK patterns,
  HistoryPage FTS5 search, SQLite-first cold start)
- Known limitations
- Suggested next steps post-dogfood
2026-04-17 13:17:51 +01:00
9f3be5c007 ui+app: Day 5+6 — Settings → Vocabulary panel + Wayland self-relaunch
VOCABULARY PANEL (Day 5):
src/lib/pages/SettingsPage.svelte gains a new collapsible "Vocabulary"
section between Audio and Transcription. Wires the dictionary commands
shipped in 1cce567:
- list_dictionary_command on mount + onfocus refresh
- add_dictionary_entry_command (term + optional note)
- delete_dictionary_entry_command
- Inline error feedback via vocabularyError
- Empty-state copy + per-entry remove button with aria-label

The dictionary terms are intended to be injected into the LLM cleanup
prompt so the model preserves user-specific vocabulary (medication
names, place names, jargon, names of people in the user's support
network — particularly important for the ND audience). The LLM client
itself is currently a stub (crates/ai-formatting/src/llm_client.rs);
when it lands, it can import list_dictionary from kon_storage and
inject terms into the prompt suffix.

WAYLAND SELF-RELAUNCH (Day 6):
src-tauri/src/lib.rs gains ensure_x11_on_wayland() which runs before
tauri::Builder. On Linux Wayland sessions (XDG_SESSION_TYPE=wayland) it
sets GDK_BACKEND=x11, WINIT_UNIX_BACKEND=x11, and
WEBKIT_DISABLE_DMABUF_RENDERER=1 — the env vars HANDOVER.md documents
the user manually prefixing.

Drops the "users have to remember the env-var prefix" friction.
Idempotent: existing values are respected. Inspired by Open-Whispr's
Chromium --ozone-platform=x11 self-relaunch.

Compile-checked: cargo check -p kon-storage clean. Settings.svelte
$state / #if balanced (28 state, 34/34 if/endif).

NEXT: Phase 6 — dogfooding readiness doc with Jake test instructions.
2026-04-17 13:16:07 +01:00
0e22ec591d ui: Day 4 frontend — dual-write history to SQLite + persist History rename
addToHistory, renameHistoryEntry, deleteFromHistory in
src/lib/stores/page.svelte.js now dual-write to:
- the in-memory `history` array (UI snappiness, unchanged)
- localStorage (offline / browser-preview fallback, unchanged)
- SQLite via the new Tauri commands from 1cce567 (the canonical store)

The SQLite write is best-effort: failure does not lose the in-memory
copy. Console-warns for diagnostic visibility. Browser-preview path
correctly skips the Tauri call.

HistoryPage rename flow (renameItem) now calls renameHistoryEntry, which
goes through update_transcript. Closes the long-standing TODO from
architecture-review.md §13: rename was UI-only, never persisted, lost
on reload.

On rename failure a warn-toast surfaces "your change is visible now but
did not save" so the user is not surprised on next launch.

NOT WIRED YET (deferred):
- HistoryPage.searchQuery still filters in-memory rather than calling
  search_transcripts. Fine for small histories; FTS5 infrastructure is
  in place to upgrade when needed.
- Reading initial history from SQLite on session boot. localStorage
  remains the cold-start source for now; SQLite catches up via dual-
  write. A backfill / one-time sync command can land later.
2026-04-17 13:12:38 +01:00
1cce5670af storage: Day 4 — FTS5 search + update_transcript + dictionary + paginated list + Tauri command surface
Finally exposes the canonical SQLite store to the frontend. Previously
the transcript/task tables existed but no Tauri command read or wrote
them — the UI lived entirely in localStorage. Closes that gap and adds
the missing commands flagged by both Codex and architecture-review.md
§13.

MIGRATIONS:
- v2 adds:
  - transcripts_fts virtual table (FTS5, porter+unicode61 tokeniser,
    diacritics-folded) backed by transcripts via content_rowid
  - INSERT/UPDATE/DELETE triggers keep FTS in sync automatically
  - dictionary table for custom vocabulary (term, note, created_at)
- Append-only as required; never modifies v1

STORAGE FUNCTIONS (crates/storage/src/database.rs):
- list_transcripts_paged(limit, offset) — pagination
- count_transcripts() — for "showing X of N" in UI
- update_transcript(id, text?, title?) — closes the historic
  architecture-review.md §13 rename-never-persists TODO
- search_transcripts(query, limit) — FTS5 wrapper
- list_dictionary / add_dictionary_entry / delete_dictionary_entry
- DictionaryEntry struct exported

TAURI COMMANDS (src-tauri/src/commands/transcripts.rs new file):
- add_transcript, list_transcripts, count_transcripts_command,
  get_transcript, update_transcript, delete_transcript,
  search_transcripts
- list_dictionary_command, add_dictionary_entry_command,
  delete_dictionary_entry_command
- TranscriptDto + DictionaryDto for camelCase frontend serialisation

REGISTRATION (src-tauri/src/lib.rs):
- All 10 new commands wired into the invoke_handler

cargo check -p kon-storage passes clean. The Tauri crate cargo check
still requires `sudo dnf install cmake clang-devel` for whisper-rs-sys
(pre-existing infra dep, unrelated to this work).

NEXT (still in this sprint):
- Wire DictationPage / FilesPage to call add_transcript on every save
  (dual-write: keep localStorage for now, add SQLite alongside)
- Wire HistoryPage rename to call update_transcript (fixes the TODO)
- Add a History search input that calls search_transcripts
- Add Settings → Dictionary panel
- Inject dictionary terms into the LLM cleanup prompt
2026-04-17 13:10:26 +01:00
69d768e803 ui: Day 3 — global toast system + first error-toast wiring on DictationPage
THE PROBLEM (Codex review + architecture-review.md §1, §14):
Multiple critical paths discard errors silently — `let _ = insert_transcript(...)`,
`.catch(() => {})`, inline `error = ...` state that not every page surfaces. ND
users in particular need explicit feedback when something fails; silent
failure is worse than no feature.

SHIPPED:

src/lib/stores/toasts.svelte.js (new):
- Minimal in-house toast store (~80 lines, no svelte-french-toast dep)
- Severity → palette: info/success → moss, warn → signal, error → ember
- Auto-dismiss durations per severity (success 3s, info 4s, warn 6s,
  error sticky)
- `invokeWithToast(invoke, command, args, errorTitle?)` helper wraps a
  Tauri invoke and toasts on failure while still throwing for callers
  that need to handle the error themselves

src/lib/components/ToastViewport.svelte (new):
- Reads from the toasts store
- Bottom-right stack, max-width clamp on small screens
- aria-live=polite + role=alert on error toasts (screen-reader friendly)
- Slide-in animation honours `html.reduce-motion` for the existing
  preferences toggle
- Severity left-border in brand colours via existing CSS variables

src/routes/+layout.svelte:
- Mounts <ToastViewport /> once at the app root so toasts work from any
  page or window

src/lib/pages/DictationPage.svelte:
- First error-toast wiring: failures starting recording now show a
  sticky toast in addition to the inline `error` panel
- Replaces a previously easy-to-miss failure-mode

NEXT (Day 4 batch):
- Wire toasts into FilesPage, HistoryPage, SettingsPage, ModelDownloader
  (all currently swallow errors)
- Add `add_transcript`, `list_transcripts`, `update_transcript` (closes the
  long-standing TODO Codex flagged), `delete_transcript` Tauri commands
  wrapping the existing storage layer
- Switch addToHistory to dual-write (localStorage + SQLite) so the
  canonical store catches up with the actual transcript flow
- FTS5 transcripts_fts virtual table + search command
- Wrap multi-row writes (decompose_and_store) in DB transactions
2026-04-17 13:06:36 +01:00
19a6b83cb2 audio: Day 2 — Codex follow-up hardening (channel disconnect, spawn_blocking, fallback silence guard, requeue counting, runtime error propagation)
Closes the 6 Codex review findings on the Day 1 mic-capture work
(commits 96980c7 + 41db162). Detail in
output/reports/kon-codex-mic-capture-followup-2026-04-17.md (CORBEL
workspace).

src-tauri/src/commands/audio.rs:
- D1: Sending an explicit stop signal before dropping stop_tx, so the
  accumulator task wakes up immediately rather than waiting for the
  Disconnected detection added below.
- D2: Wrapping MicrophoneCapture::start() in tokio::task::spawn_blocking.
  start() can spend up to 350ms × N_devices × 2 passes; running it on
  the async runtime froze other Tauri commands.
- M3: Match on rx.try_recv() error variants. Empty -> sleep + continue.
  Disconnected -> exit accumulator task immediately. Previous behaviour
  was an infinite loop after the capture stream died.

crates/audio/src/capture.rs:
- D3: Added DEAD_SILENCE_FLOOR (1e-7) gate that applies even in the
  fallback pass. PulseAudio/PipeWire can stream zero-valued bytes from
  a borked device; that is worse than failing fast.
- M1: Validation requeue (`for chunk in collected { try_send }`) now
  counts drops in the same dropped_chunks counter.
- M2: New CaptureRuntimeError type. The cpal stream error callback now
  forwards errors via a separate sync_channel (capacity 16) that the
  live session can drain via take_error_rx() and surface as toasts.
  Re-exported from kon_audio crate root.

cargo check -p kon-audio passes clean.

NOT YET DONE (M4): JACK-specific monitor name patterns. Defers until
testing on a JACK host. Current is_monitor_name() may miss JACK
conventions.

Wiring the new error_rx into the live session and surfacing as toasts
lands with the Day 3 error-toast system commit.
2026-04-17 13:02:51 +01:00
41db162041 audio: wire user's microphone choice through start_native_capture + live session
Day 1 follow-up to 96980c7. The device-picker UI in Settings now
actually takes effect: settings.microphoneDevice flows from the Svelte
store, through the Tauri invoke, into MicrophoneCapture::start_with_device
on the Rust side.

Touched paths (back-to-front):
- src-tauri/src/commands/audio.rs:start_native_capture — new optional
  `device_name: Option<String>` parameter; routes to start_with_device
  when set, falls back to auto-select start() when None or empty.
- src-tauri/src/commands/live.rs:StartLiveTranscriptionConfig — new
  optional `microphone_device: Option<String>` field with same
  semantics (rename_all = "camelCase" maps it to microphoneDevice on
  the wire).
- src-tauri/src/commands/live.rs:run_live_session — picks
  start_with_device when an explicit name is provided.
- src/lib/pages/DictationPage.svelte — passes
  microphoneDevice: settings.microphoneDevice || null in the invoke.

Behaviour:
- "Auto" in the picker (empty string) -> backend auto-selects, skipping
  monitor sources and validating by RMS energy.
- Specific device -> backend opens that device by exact name; if it has
  been disconnected the user gets a clear error pointing them back at
  Settings.

cargo check -p kon-audio passes clean. Tauri-crate cargo check requires
cmake (pre-existing infra dependency for whisper-rs-sys); install via
`sudo dnf install cmake clang-devel`.
2026-04-17 12:45:19 +01:00
96980c7d5c audio: fix mic capture — skip monitor sources, validate by RMS, add device picker
Day 1 of the upgrade plan (output/reports/kon-upgrade-plan-2026-04-17.md
in the CORBEL workspace). Fixes the HANDOVER.md blocker: native live
transcription was capturing silence because PulseAudio/PipeWire monitor
sources (speaker loopback) were winning the device-selection race —
they deliver zero-valued bytes that satisfied the original
"any device that produces data within 350ms" check.

WHAT CHANGED:

crates/audio/src/capture.rs (rewrite):
- New `DeviceInfo` struct (serde-derived) for the Settings device picker
- New `MicrophoneCapture::list_devices()` enumerates inputs with metadata
- New `MicrophoneCapture::start_with_device(name)` for explicit selection
- Refactored `start()` with monitor-source filtering by name pattern
  (.monitor suffix, "Monitor of " prefix, "loopback" substring) and
  RMS-energy validation in a 350ms window
- Two-pass selection: real inputs first, monitor sources only as
  last-resort fallback with explicit warning log
- Drop counter (Arc<AtomicU64>) tracks chunks dropped by `try_send`
  failure under load — Codex review caught this as a silent-failure risk
- `dropped_chunks()` accessor for the live session
- Verbose tracing at every step for diagnosis
- Unit test for monitor-name detection
- `cargo check -p kon-audio` passes clean

crates/audio/src/lib.rs:
- Re-export `DeviceInfo`

crates/audio/Cargo.toml:
- Add serde dependency (for DeviceInfo derives)

src-tauri/src/commands/audio.rs:
- New `list_audio_devices` Tauri command (returns Vec<DeviceInfo>)

src-tauri/src/lib.rs:
- Register `list_audio_devices` in invoke_handler

src/lib/pages/SettingsPage.svelte:
- New "Audio" section with microphone picker dropdown
- Auto-populates on mount via `list_audio_devices`
- Refresh button + clear messaging about monitor sources
- Likely-monitor entries marked disabled in the dropdown
- Auto mode is the default (empty string in settings.microphoneDevice)

src/lib/stores/page.svelte.js:
- New `microphoneDevice` field in defaults (empty = auto-select)

NEXT STEPS (per the upgrade plan):
- Wire `microphoneDevice` from settings into `MicrophoneCapture::start_with_device`
  in the live and standalone capture paths (currently both still call
  the auto-selecting `start()`)
- Test on real hardware (Wayland + multiple input devices)
- Codex sanity-check of this diff is running in parallel; addendum to
  follow if anything substantive comes back

Refs: /home/jake/Documents/CORBEL-Projects/kon/HANDOVER.md
      output/reports/kon-upgrade-plan-2026-04-17.md (CORBEL workspace)
2026-04-17 12:43:13 +01:00
40 changed files with 8108 additions and 546 deletions

143
.github/workflows/build.yml vendored Normal file
View File

@@ -0,0 +1,143 @@
# Cross-platform release build. Produces installer artifacts for
# Linux (.AppImage + .deb), Windows (.msi + .exe), macOS (.dmg + .app).
#
# Triggers:
# - Manual: any branch via "Run workflow" in the GitHub Actions UI.
# Use this to build a Windows binary on demand to dual-boot test.
# - Tag push (v*): builds + drafts a GitHub Release with all artifacts
# attached. Tag a release with `git tag v0.2.0 && git push --tags`.
#
# Artifacts:
# - workflow_dispatch builds: uploaded as Action artifacts
# (visible in the run page, downloadable for 30 days).
# - tag builds: attached to a draft GitHub Release named after the tag.
# Promote the draft to a release when ready.
#
# Signing:
# - macOS code-signing not configured. The .dmg will trigger Gatekeeper
# warnings on the first run; users will need to right-click → Open.
# To wire signing later, set APPLE_SIGNING_IDENTITY +
# APPLE_CERTIFICATE secrets and uncomment the env block.
# - Windows code-signing not configured. The .exe/.msi will trigger
# SmartScreen warnings on first run. To wire signing later, set
# WINDOWS_CERTIFICATE + WINDOWS_CERTIFICATE_PASSWORD secrets.
name: build
on:
push:
tags: ['v*']
workflow_dispatch:
inputs:
tag_name:
description: 'Optional tag name to attach the build to (leave blank for plain artifacts)'
required: false
concurrency:
group: build-${{ github.ref }}
cancel-in-progress: false # let release builds finish even on tag re-pushes
jobs:
build:
name: build (${{ matrix.os }})
permissions:
contents: write # needed to create draft releases on tag pushes
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-22.04
artifact_glob: |
src-tauri/target/release/bundle/appimage/*.AppImage
src-tauri/target/release/bundle/deb/*.deb
- os: windows-latest
artifact_glob: |
src-tauri/target/release/bundle/msi/*.msi
src-tauri/target/release/bundle/nsis/*.exe
- os: macos-latest
artifact_glob: |
src-tauri/target/release/bundle/dmg/*.dmg
src-tauri/target/release/bundle/macos/*.app
runs-on: ${{ matrix.os }}
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
# System packages — same as check.yml but locked to release-build needs.
- name: Install Linux deps
if: matrix.os == 'ubuntu-22.04'
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
libwebkit2gtk-4.1-dev \
libappindicator3-dev \
librsvg2-dev \
libasound2-dev \
libudev-dev \
patchelf \
cmake \
build-essential
- name: Install macOS deps
if: matrix.os == 'macos-latest'
run: |
brew list cmake >/dev/null 2>&1 || brew install cmake
- name: Install Windows deps
if: matrix.os == 'windows-latest'
shell: pwsh
run: |
cmake --version
choco install -y llvm --no-progress
- name: Install Node
uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- name: Cache Rust
uses: Swatinem/rust-cache@v2
with:
workspaces: src-tauri -> target
shared-key: kon-build-${{ matrix.os }}
- name: Install JS deps
run: npm ci
# tauri-action handles `tauri build` plus, on tag pushes, attaches
# artifacts to a GitHub draft release. Empty tagName disables the
# release-creation behaviour for manual workflow_dispatch runs.
- name: Build (release)
uses: tauri-apps/tauri-action@v0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Uncomment when signing certs are configured in repo secrets:
# APPLE_SIGNING_IDENTITY: ${{ secrets.APPLE_SIGNING_IDENTITY }}
# APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
# APPLE_CERTIFICATE_PASSWORD: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
# WINDOWS_CERTIFICATE: ${{ secrets.WINDOWS_CERTIFICATE }}
# WINDOWS_CERTIFICATE_PASSWORD: ${{ secrets.WINDOWS_CERTIFICATE_PASSWORD }}
with:
# If pushed as a tag, use the tag name; otherwise leave empty
# so tauri-action builds artifacts but does not touch releases.
tagName: ${{ github.ref_type == 'tag' && github.ref_name || (inputs.tag_name || '') }}
releaseName: ${{ github.ref_type == 'tag' && github.ref_name || (inputs.tag_name || '') }}
releaseDraft: true
prerelease: false
# Build all bundle types the OS supports.
args: ''
# Always upload as an Actions artifact too — accessible from the
# workflow run page even if the release-creation step was skipped.
- name: Upload artifacts
if: always()
uses: actions/upload-artifact@v4
with:
name: kon-${{ matrix.os }}-${{ github.sha }}
path: ${{ matrix.artifact_glob }}
retention-days: 30
if-no-files-found: warn

105
.github/workflows/check.yml vendored Normal file
View File

@@ -0,0 +1,105 @@
# Per-push fast feedback: cargo check on Linux + Windows + macOS, plus
# the Svelte build. Catches platform-specific compile errors (the M3
# fix that broke on Windows because std::sync::mpsc::TryRecvError lives
# in a different module on certain configurations, etc) without paying
# the cost of the full Tauri release build.
#
# For the full installer build (.msi, .dmg, .AppImage) see build.yml.
name: check
on:
push:
branches: [main]
pull_request:
workflow_dispatch:
# Cancel any earlier in-progress runs for the same branch — a fresh push
# supersedes the previous one.
concurrency:
group: check-${{ github.ref }}
cancel-in-progress: true
jobs:
rust:
name: cargo check (${{ matrix.os }})
strategy:
fail-fast: false
matrix:
os: [ubuntu-22.04, windows-latest, macos-latest]
runs-on: ${{ matrix.os }}
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
# System packages whisper-rs-sys + Tauri need on each OS.
# Linux is the heaviest because we depend on GTK/WebKit, audio,
# evdev, plus cmake for whisper.cpp.
- name: Install Linux deps
if: matrix.os == 'ubuntu-22.04'
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
libwebkit2gtk-4.1-dev \
libappindicator3-dev \
librsvg2-dev \
libasound2-dev \
libudev-dev \
patchelf \
cmake \
build-essential
# macOS: cmake is preinstalled in macos-latest but pin via brew to
# be explicit and future-proof.
- name: Install macOS deps
if: matrix.os == 'macos-latest'
run: |
brew list cmake >/dev/null 2>&1 || brew install cmake
# Windows: cmake + clang are needed by whisper-rs-sys. cmake is on
# windows-latest by default; ensure it.
- name: Install Windows deps
if: matrix.os == 'windows-latest'
shell: pwsh
run: |
# cmake is on the runner image; verify it.
cmake --version
# LLVM/clang for whisper.cpp's sources.
choco install -y llvm --no-progress
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
# Cache the Cargo target dir + registry per OS so the heavy
# whisper-rs-sys C++ build only happens on a clean cache.
- name: Cache Rust artifacts
uses: Swatinem/rust-cache@v2
with:
workspaces: src-tauri -> target
shared-key: kon-${{ matrix.os }}
- name: cargo check (workspace)
working-directory: src-tauri
run: cargo check --workspace --all-targets
frontend:
name: svelte build + lint
runs-on: ubuntu-22.04
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- name: Install Node
uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
- name: Install JS deps
run: npm ci
# `tauri build` inside check.yml would trigger the full Rust build
# which is owned by the rust job. Here we only validate that the
# Svelte/Vite frontend compiles cleanly.
- name: Build frontend (Vite only)
run: npm run build

1
.gitignore vendored
View File

@@ -4,3 +4,4 @@ build/
dist/
.svelte-kit/
Cargo.lock
.firecrawl/

253
HANDOVER-2026-04-17.md Normal file
View File

@@ -0,0 +1,253 @@
# Kon Session Handover — 2026/04/17
## Session Summary
Six-commit sprint executing the upgrade plan from
`/home/jake/Documents/CORBEL-Furnished-House/output/reports/kon-upgrade-plan-2026-04-17.md`.
Goal: get Kon from "core feature broken" to "ready to dogfood with friends."
## Commits
| Commit | Title |
|---|---|
| `96980c7` | Day 1 — fix mic capture: skip monitor sources, RMS validation, drop counting, list_devices, start_with_device |
| `41db162` | Day 1 follow-up — wire user's microphone choice through start_native_capture + live session |
| `19a6b83` | Day 2 — Codex follow-up hardening (channel disconnect, spawn_blocking, fallback silence guard, requeue counting, runtime error propagation) |
| `69d768e` | Day 3 — global toast system + first error-toast wiring on DictationPage |
| `1cce567` | Day 4 backend — FTS5 search + update_transcript + dictionary + paginated list + Tauri command surface |
| `0e22ec5` | Day 4 frontend — dual-write history to SQLite + persist History rename |
| `9f3be5c` | Day 5+6 — Settings → Vocabulary panel + Wayland self-relaunch |
## What changed
### Mic capture — now actually works
The HANDOVER from 2026/04/04 flagged native live transcription as broken
(`Selected working microphone: null`, chunks repeatedly skipped as
near-silence). Root cause was PulseAudio/PipeWire monitor sources
(speaker loopback) winning the "first device that produces data within
350ms" race — silent monitor sources delivered zero-valued bytes that
satisfied that check.
Fixed by:
- **Skipping monitor sources** by name pattern (`.monitor` suffix,
`Monitor of ` prefix, `loopback` substring)
- **Validating by RMS energy** in a 350ms window, not just receipt
of bytes
- **Two-pass selection**: real inputs first, monitor sources only as
last resort with explicit warning log + dead-silence floor (1e-7)
guard so even fallback rejects all-zeros
- **Verbose tracing** at every step
- **Drop counter** (`Arc<AtomicU64>`) that tracks chunks lost to
backpressure, including in the validation requeue
- **Runtime error channel** so cpal stream errors after start succeeds
surface to the live session for toast display
- **`spawn_blocking`** wrapper so `start()`'s up-to-3.5s validation
window does not freeze the async runtime
### Settings → Audio → Microphone picker
User can now explicitly pick which input device to use. Auto mode
(empty) skips monitor sources and validates by RMS. Specific device
opens it by exact name. Setting persists in `settings.microphoneDevice`
(localStorage) and flows through to both `start_native_capture` and
`start_live_transcription_session`.
### Toast system
`src/lib/components/ToastViewport.svelte` mounted in root layout.
`toasts.error/warn/success/info(title, body)` from any component.
Brand-palette colours (moss/signal/ember). aria-live polite + role=alert
on errors. Honours `html.reduce-motion`. Sticky errors, auto-dismiss
others.
First wired into DictationPage's "could not start recording" path. More
pages can adopt it incrementally — `invokeWithToast` helper makes
wrapping any Tauri call a one-liner.
### SQLite as canonical store
The transcripts table existed but no Tauri command read or wrote it
(Codex caught this in the joint review). Now exposed via 10 new
commands in `commands/transcripts.rs`:
- `add_transcript`, `list_transcripts` (paginated), `count_transcripts`,
`get_transcript`, `update_transcript` (closes the long-standing
rename-never-persists TODO from `architecture-review.md §13`),
`delete_transcript`, `search_transcripts` (FTS5)
- `list_dictionary_command`, `add_dictionary_entry_command`,
`delete_dictionary_entry_command`
Frontend `addToHistory`, `renameHistoryEntry`, `deleteFromHistory` now
dual-write to SQLite alongside localStorage. Best-effort: SQLite failure
keeps the in-memory copy and warns to console. HistoryPage rename now
calls `update_transcript`.
Migration v2 added FTS5 virtual table with porter+unicode61 tokeniser,
diacritics-folded, plus INSERT/UPDATE/DELETE triggers to keep the FTS
index in sync. Dictionary table also added in v2.
### Settings → Vocabulary
New collapsible section. Add custom terms (medication names, jargon,
people's names) that the LLM cleanup prompt should preserve. Backed by
the `dictionary` SQLite table. The LLM client itself is currently a
stub; when wired, it imports `list_dictionary` from kon_storage and
injects terms into the prompt suffix.
### Wayland self-relaunch
`ensure_x11_on_wayland()` runs before `tauri::Builder` on Linux. If
`XDG_SESSION_TYPE=wayland`, sets `GDK_BACKEND=x11`,
`WINIT_UNIX_BACKEND=x11`, `WEBKIT_DISABLE_DMABUF_RENDERER=1` so the
HANDOVER env-var prefix is no longer needed.
## How to dogfood
### One-time setup on Menhir
```bash
sudo dnf install cmake clang-devel
cd /home/jake/Documents/CORBEL-Projects/kon
npm install # if you have not already
```
### Launch (no env-var prefix needed any more)
```bash
cd /home/jake/Documents/CORBEL-Projects/kon
npm run tauri dev
```
If anything goes wrong on Wayland, you can still fall back to:
```bash
env GDK_BACKEND=x11 WINIT_UNIX_BACKEND=x11 \
WEBKIT_DISABLE_DMABUF_RENDERER=1 npm run tauri dev
```
### What to test
1. **Mic capture happy path.** Open Settings → Audio. Devices populate.
Pick your Blue Yeti (or whatever). Hit dictation. Speak. Text should
appear within 2 seconds. Stop, save, the recording should appear in
History.
2. **Mic capture failure path.** Pull the USB mic mid-recording. A toast
should surface ("device disconnected" or similar). The session should
not silently produce empty transcripts.
3. **Auto mode.** Clear the picker (set to "Auto"). Hit dictation. The
logs (terminal where you ran `npm run tauri dev`) should show:
- `[kon-audio] start: enumerated N input device(s)`
- `[kon-audio] trying '...'` for each candidate
- `[kon-audio] '...' validation: M samples, rms=...`
- `[kon-audio] selected microphone: '...'`
The selected mic should NOT be a `.monitor` source.
4. **History rename.** Make a recording. In History, rename it to
something distinctive ("test rename 1"). Quit the app. Relaunch.
Open History. The rename should still be there (was previously
lost on relaunch — closes the old TODO).
5. **Vocabulary panel.** Settings → Vocabulary. Add "Wren" with note
"CORBEL operating partner". Persists across restarts. (LLM cleanup
prompt is a stub so the term won't actually affect transcripts yet —
storage layer is ready for when LLM lands.)
6. **Toasts on error.** Try to hit dictation with no microphone
connected at all. Should show a sticky error toast in the bottom-right
("Could not start recording" + body) rather than failing silently.
### What's deferred (does not block dogfood)
- **Whisper pre-warm at startup.** Models still load on first dictation
(~2-5s cold start). Deferred because it needs careful threading work
to avoid blocking `setup()`. Easy to add later.
- **Auto-updater (`tauri-plugin-updater`).** Deferred because it needs
a release feed (GitHub releases or similar) which requires CI / signing
infrastructure decisions.
- **JACK monitor-name patterns.** Codex flagged that JACK setups may use
different naming conventions than PulseAudio. Test on a JACK host,
extend `is_monitor_name()` if needed.
- **HistoryPage search via FTS5.** The infrastructure is in place
(`search_transcripts` Tauri command) but HistoryPage still uses the
in-memory client-side filter, which is fine for small histories.
- **Read initial history from SQLite at boot.** Currently localStorage
is the cold-start source; SQLite catches up via dual-write. A backfill
/ one-time sync command can land later.
## Known limitations
- **The full Tauri build needs `cmake` + `clang-devel`** for
whisper-rs-sys. Not a regression; pre-existing infra dep.
- **State is still split** between localStorage (cache) and SQLite
(canonical). Dual-write resolves the consistency problem in the
short term. The eventual destination is SQLite-only with localStorage
as a transparent cache.
## Cross-platform status (audited 2026/04/17)
| Platform | Build | Run | Polish | Confidence |
|---|---|---|---|---|
| **Linux x86_64 (Fedora 43, KDE Wayland)** | ✓ | ✓ | The everyday dev target | **HIGH** — this is what the sprint was developed against |
| **Linux x86_64 (other distros, X11)** | Should work | Should work | Wayland self-relaunch is no-op on X11 sessions, evdev hotkeys may need user added to `input` group | **MEDIUM** — tested patterns, untested distros |
| **Windows 10/11 x86_64** | Untested but should compile (CPAL + Tauri + whisper.cpp all support it) | Untested | Custom evdev hotkeys are no-op; falls back to Tauri's global-shortcut plugin which works on Windows | **LOW** — theoretically supported, has had zero hands-on testing |
| **macOS aarch64 (Apple Silicon)** | Untested | Untested | Path bug fixed this commit (now uses `~/Library/Application Support/Kon/`); Info.plist needs `NSMicrophoneUsageDescription` for the app bundle | **LOW** — theoretically supported, has had zero hands-on testing |
| **macOS x86_64 (Intel)** | Same as Apple Silicon | Same | Same | **LOW** |
**For the friends beta on Linux only**, this matters not at all. For a future Windows or macOS build, expect to spend a focused day or two debugging:
- Tauri config: `bundle.macOS.entitlements`, `bundle.windows.signingIdentity`
- macOS code-signing + notarisation (real money: ~£75/yr Apple developer account)
- Windows code-signing certificate (~£100-300/yr) or accept SmartScreen warning
- whisper-rs-sys build dependencies per OS (cmake on all; Visual Studio Build Tools on Windows)
- macOS-specific Info.plist keys for microphone permission
### What this sprint added on the cross-platform front
- `crates/storage/src/file_storage.rs::app_data_dir()` now correctly handles macOS (`~/Library/Application Support/Kon/`) and Linux (XDG-aware, `~/.local/share/kon`, with legacy `~/.kon` fallback for existing installs). Windows path unchanged.
- New Tauri command `get_os_info` returns `{os, arch, family, usesCmd, isWayland, customHotkeyBackend, primaryModifierLabel}` so the frontend can adapt UI strings (Cmd vs Ctrl labels, "Open Finder" vs "Open Explorer", etc).
- New `src/lib/utils/osInfo.js` helper: async `loadOsInfo()` warms a cache, then `isMac() / isWindows() / isLinux() / modKeyLabel() / isWayland()` are synchronous. Eagerly loaded at app startup in the root layout.
- Falls back gracefully in browser-preview mode by reading `navigator.platform`.
## Files changed this sprint
```
crates/audio/Cargo.toml
crates/audio/src/capture.rs (rewrite + Day 2 hardening)
crates/audio/src/lib.rs
crates/storage/src/database.rs (+ FTS5, update, search, dictionary)
crates/storage/src/lib.rs
crates/storage/src/migrations.rs (+ migration v2)
src-tauri/src/commands/audio.rs (+ device picker, spawn_blocking, M3 fix)
src-tauri/src/commands/live.rs (+ microphoneDevice config field)
src-tauri/src/commands/mod.rs
src-tauri/src/commands/transcripts.rs (NEW — 10 Tauri commands)
src-tauri/src/lib.rs (+ Wayland, command registrations)
src/lib/components/ToastViewport.svelte (NEW)
src/lib/pages/DictationPage.svelte (+ device wiring, error toast)
src/lib/pages/HistoryPage.svelte (+ rename via update_transcript)
src/lib/pages/SettingsPage.svelte (+ Audio + Vocabulary panels)
src/lib/stores/page.svelte.js (+ microphoneDevice, dual-write)
src/lib/stores/toasts.svelte.js (NEW)
src/routes/+layout.svelte (+ ToastViewport mount)
```
## Next steps after dogfood
1. Real-user feedback from one to three friends. What confuses them?
What feels slow? What did they expect that did not happen?
2. Address the deferred items in priority of feedback signal.
3. Consider opening up the `kon-public-beta` channel — a single
GitHub release with the auto-updater plumbed.
4. The architecture review's other items (frontend test coverage,
monolithic component split, hardcoded hex colours, ARIA gaps)
become the "open beta polish" sprint.
---
*Compiled 2026/04/17 by Wren. Kon goes from "live transcription does not
work" to "ready to put in front of one trusted friend." Six commits, no
horrors so far.*

View File

@@ -25,3 +25,6 @@ symphonia = { version = "0.5", features = ["mp3", "aac", "flac", "pcm", "vorbis"
# Async runtime for threading
tokio = { version = "1", features = ["rt", "sync"] }
# Serde for DeviceInfo (returned across the Tauri boundary)
serde = { version = "1", features = ["derive"] }

View File

@@ -1,9 +1,25 @@
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::mpsc;
use cpal::{FromSample, Sample, SampleFormat, SizedSample};
use cpal::traits::{DeviceTrait, HostTrait, StreamTrait};
use serde::{Deserialize, Serialize};
use kon_core::error::{KonError, Result};
const AUDIO_CHANNEL_CAPACITY: usize = 32;
/// Validation window. We listen for this long and compute RMS to decide
/// whether the chosen device is delivering real audio (vs a silent monitor).
const DEVICE_VALIDATION_MS: u64 = 350;
/// Below this RMS amplitude (peak ±1.0 scale) the input is treated as
/// silence. PulseAudio/PipeWire monitor sources for an idle speaker
/// typically deliver dead-zero samples; real microphones yield ~0.0005+
/// even in a quiet room. Conservative floor: 1e-5.
const SILENCE_RMS_FLOOR: f32 = 1e-5;
/// A chunk of captured audio from the microphone.
pub struct AudioChunk {
pub samples: Vec<f32>,
@@ -11,53 +27,202 @@ pub struct AudioChunk {
pub channels: u16,
}
/// Public-facing description of an audio input device.
/// Returned by `list_devices()` and used by the UI device picker.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct DeviceInfo {
/// Device name as reported by cpal/the host.
pub name: String,
/// Default sample rate in Hz.
pub sample_rate: u32,
/// Default channel count.
pub channels: u16,
/// True if the device name matches a known monitor-source pattern
/// (PulseAudio/PipeWire loopback of speaker output).
pub is_likely_monitor: bool,
/// True if cpal reports this as the host's default input device.
pub is_default: bool,
}
/// A non-fatal capture-time error emitted by the cpal stream callback after
/// `start()` has already returned. The live session subscribes to these via
/// `error_rx()` so the frontend can show a toast when the mic vanishes
/// mid-recording.
/// (Codex review 2026/04/17 M2)
#[derive(Debug, Clone)]
pub struct CaptureRuntimeError {
pub device_name: String,
pub message: String,
}
/// Manages microphone capture via cpal.
/// Call `start()` to begin capturing, which returns a receiver for audio chunks.
/// Call `stop()` to end the stream.
pub struct MicrophoneCapture {
stream: Option<cpal::Stream>,
/// Name of the device that is actually capturing.
pub device_name: String,
/// Counter incremented every time the capture callback drops a chunk
/// because the channel was full. Read via `dropped_chunks()`.
dropped_chunks: Arc<AtomicU64>,
/// Receiver for runtime stream errors (device unplugged, audio server
/// crash, etc.). The live session calls `error_rx()` once and listens.
error_rx: Option<mpsc::Receiver<CaptureRuntimeError>>,
}
impl MicrophoneCapture {
/// Start capturing audio from the default input device.
/// Returns a receiver that yields AudioChunks as they arrive.
/// Number of audio chunks dropped because the downstream channel was full
/// since this capture started. Should stay at 0 in normal use; non-zero
/// indicates downstream backpressure or a stuck consumer.
pub fn dropped_chunks(&self) -> u64 {
self.dropped_chunks.load(Ordering::Relaxed)
}
/// Take the runtime-error receiver. Can be called once per capture; the
/// caller (live session manager) drains it on its own cadence and surfaces
/// errors to the frontend. Returns None on the second call.
/// (Codex review 2026/04/17 M2)
pub fn take_error_rx(&mut self) -> Option<mpsc::Receiver<CaptureRuntimeError>> {
self.error_rx.take()
}
/// Enumerate every input device the host knows about, with the metadata
/// needed by the device-picker UI.
pub fn list_devices() -> Result<Vec<DeviceInfo>> {
let host = cpal::default_host();
let default_name = host
.default_input_device()
.and_then(|d| d.name().ok())
.unwrap_or_default();
let devices = host
.input_devices()
.map_err(|e| KonError::AudioCaptureFailed(format!("input_devices: {e}")))?;
let mut out = Vec::new();
for device in devices {
let name = device.name().unwrap_or_else(|_| "<unnamed>".to_string());
let (sample_rate, channels) = match device.default_input_config() {
Ok(cfg) => (cfg.sample_rate(), cfg.channels() as u16),
Err(_) => (0, 0),
};
let is_likely_monitor = is_monitor_name(&name);
let is_default = !default_name.is_empty() && name == default_name;
out.push(DeviceInfo {
name,
sample_rate,
channels,
is_likely_monitor,
is_default,
});
}
Ok(out)
}
/// Start capturing from the device whose name matches `device_name` exactly.
/// If no match is found, returns an error rather than silently falling back.
pub fn start_with_device(
device_name: &str,
) -> Result<(Self, mpsc::Receiver<AudioChunk>)> {
let host = cpal::default_host();
let devices = host
.input_devices()
.map_err(|e| KonError::AudioCaptureFailed(format!("input_devices: {e}")))?;
for device in devices {
let name = device.name().unwrap_or_default();
if name == device_name {
eprintln!("[kon-audio] start_with_device: opening explicit device '{name}'");
return open_and_validate(device, &name, /* require_audio = */ true);
}
}
Err(KonError::AudioCaptureFailed(format!(
"Selected device '{device_name}' not found in current host enumeration. \
It may have been disconnected. Open Settings → Audio to pick another."
)))
}
/// Start capturing audio with auto-selection.
///
/// Selection rules:
/// 1. Try the host default input device first if it exists AND is not a monitor source.
/// 2. Otherwise, try non-monitor devices in enumeration order.
/// 3. Validate the chosen device by RMS energy (not just receipt of bytes) over
/// a short window — this is what defeats the "silent monitor source wins" bug.
/// 4. If no non-monitor device produces real audio, fall back to monitor sources
/// as a last resort (with a clear log line). Never accept dead silence.
pub fn start() -> Result<(Self, mpsc::Receiver<AudioChunk>)> {
let host = cpal::default_host();
let device = host.default_input_device().ok_or_else(|| {
KonError::AudioCaptureFailed("No input device found".into())
})?;
let default_name = host
.default_input_device()
.and_then(|d| d.name().ok())
.unwrap_or_default();
let config = device.default_input_config().map_err(|e| {
KonError::AudioCaptureFailed(format!("No input config: {e}"))
})?;
let mut all_devices: Vec<cpal::Device> =
host.input_devices()
.map_err(|e| {
KonError::AudioCaptureFailed(format!("input_devices: {e}"))
})?
.collect();
let sample_rate = config.sample_rate();
let channels = config.channels() as u16;
// Sort: default first, then non-monitor, then monitor-as-last-resort.
all_devices.sort_by_key(|d| {
let n = d.name().unwrap_or_default();
let is_default = !default_name.is_empty() && n == default_name;
let is_monitor = is_monitor_name(&n);
// Smaller key = tried first.
match (is_default, is_monitor) {
(true, false) => 0, // default, real input
(false, false) => 1, // any other real input
(true, true) => 2, // default but is a monitor (very rare)
(false, true) => 3, // monitor source — last resort
}
});
let (tx, rx) = mpsc::channel::<AudioChunk>();
eprintln!(
"[kon-audio] start: enumerated {} input device(s) (default='{}')",
all_devices.len(),
default_name
);
let stream = device
.build_input_stream(
&config.into(),
move |data: &[f32], _info: &cpal::InputCallbackInfo| {
let _ = tx.send(AudioChunk {
samples: data.to_vec(),
sample_rate,
channels,
});
},
|err| eprintln!("audio capture error: {err}"),
None,
)
.map_err(|e| {
KonError::AudioCaptureFailed(format!("Build stream failed: {e}"))
})?;
// First pass: require real audio energy.
for device in &all_devices {
let name = device.name().unwrap_or_default();
if is_monitor_name(&name) {
continue; // Save monitor sources for second pass.
}
match open_and_validate(device.clone(), &name, true) {
Ok(result) => return Ok(result),
Err(e) => {
eprintln!("[kon-audio] '{name}' rejected: {e}");
}
}
}
stream.play().map_err(|e| {
KonError::AudioCaptureFailed(format!("Stream play failed: {e}"))
})?;
// Second pass: accept anything that delivers bytes (monitor sources
// included). Better to capture from a monitor than fail entirely.
eprintln!(
"[kon-audio] no non-monitor mic produced audio; falling back to monitor/loopback sources"
);
for device in &all_devices {
let name = device.name().unwrap_or_default();
match open_and_validate(device.clone(), &name, false) {
Ok(result) => {
eprintln!(
"[kon-audio] FALLBACK: capturing from '{name}' (likely monitor source). \
Recordings may be silent or contain system audio."
);
return Ok(result);
}
Err(_) => continue,
}
}
Ok((Self { stream: Some(stream) }, rx))
Err(KonError::AudioCaptureFailed(
"No working microphone found. Check that an input device is connected, \
that PulseAudio/PipeWire is running, and that the app has microphone permission. \
Then open Settings → Audio to pick a device explicitly."
.into(),
))
}
/// Stop capturing audio.
@@ -73,3 +238,199 @@ impl Drop for MicrophoneCapture {
self.stop();
}
}
/// Heuristic: identify a PulseAudio/PipeWire monitor source by name.
/// Common patterns:
/// - ".monitor" suffix (PulseAudio convention)
/// - "Monitor of " prefix (longer human-readable name)
/// - "Loopback" anywhere (some PipeWire configurations)
fn is_monitor_name(name: &str) -> bool {
let lower = name.to_lowercase();
lower.ends_with(".monitor")
|| lower.starts_with("monitor of ")
|| lower.contains("monitor of ")
|| lower.contains("loopback")
}
/// Open the given device and validate it produces non-silent audio.
/// If `require_audio` is false, accept any data (used for monitor fallback).
fn open_and_validate(
device: cpal::Device,
name: &str,
require_audio: bool,
) -> Result<(MicrophoneCapture, mpsc::Receiver<AudioChunk>)> {
let config = device.default_input_config().map_err(|e| {
KonError::AudioCaptureFailed(format!("default_input_config: {e}"))
})?;
let sample_rate = config.sample_rate();
let channels = config.channels() as u16;
let format = config.sample_format();
eprintln!(
"[kon-audio] trying '{name}' ({sr}Hz, {ch}ch, {fmt:?})",
sr = sample_rate,
ch = channels,
fmt = format
);
let (tx, rx) = mpsc::sync_channel::<AudioChunk>(AUDIO_CHANNEL_CAPACITY);
let requeue_tx = tx.clone();
let dropped_chunks = Arc::new(AtomicU64::new(0));
// Bounded channel for runtime stream errors. Capacity 16 = plenty for
// the rare error case; if it ever fills, we drop newer errors silently
// because they would be redundant noise in a stream that is already
// failing. (Codex review 2026/04/17 M2)
let (err_tx, err_rx) = mpsc::sync_channel::<CaptureRuntimeError>(16);
let stream = match format {
SampleFormat::F32 => build_input_stream::<f32>(&device, &config, sample_rate, channels, tx, dropped_chunks.clone(), err_tx.clone(), name.to_string()),
SampleFormat::I16 => build_input_stream::<i16>(&device, &config, sample_rate, channels, tx, dropped_chunks.clone(), err_tx.clone(), name.to_string()),
SampleFormat::U16 => build_input_stream::<u16>(&device, &config, sample_rate, channels, tx, dropped_chunks.clone(), err_tx.clone(), name.to_string()),
other => {
return Err(KonError::AudioCaptureFailed(format!(
"unsupported sample format {other:?}"
)))
}
}
.map_err(|e| KonError::AudioCaptureFailed(format!("build_input_stream: {e}")))?;
stream
.play()
.map_err(|e| KonError::AudioCaptureFailed(format!("stream.play: {e}")))?;
// Validation window: collect chunks for DEVICE_VALIDATION_MS, compute RMS.
let deadline = std::time::Instant::now()
+ std::time::Duration::from_millis(DEVICE_VALIDATION_MS);
let mut collected: Vec<AudioChunk> = Vec::new();
let mut total_samples = 0_usize;
let mut sum_sq: f64 = 0.0;
while std::time::Instant::now() < deadline {
let remaining = deadline.saturating_duration_since(std::time::Instant::now());
if remaining.is_zero() {
break;
}
match rx.recv_timeout(remaining) {
Ok(chunk) => {
for &s in &chunk.samples {
sum_sq += (s as f64) * (s as f64);
}
total_samples += chunk.samples.len();
collected.push(chunk);
}
Err(_) => break,
}
}
if total_samples == 0 {
return Err(KonError::AudioCaptureFailed(
"device delivered zero samples in validation window".into(),
));
}
let rms = (sum_sq / total_samples as f64).sqrt() as f32;
eprintln!(
"[kon-audio] '{name}' validation: {samples} samples, rms={rms:.6}",
samples = total_samples
);
if require_audio && rms < SILENCE_RMS_FLOOR {
return Err(KonError::AudioCaptureFailed(format!(
"device produced silence (rms={rms:.6} below floor {SILENCE_RMS_FLOOR:.6})"
)));
}
// Even in the fallback pass (require_audio=false), reject completely
// dead-zero audio. PulseAudio/PipeWire will sometimes happily emit a
// long stream of f32 zeros from a borked device — that is worse than
// failing fast. (Codex review 2026/04/17 D3)
const DEAD_SILENCE_FLOOR: f32 = 1e-7;
if rms < DEAD_SILENCE_FLOOR {
return Err(KonError::AudioCaptureFailed(format!(
"device produced dead silence (rms={rms:.6e} below absolute floor {DEAD_SILENCE_FLOOR:.6e})"
)));
}
// Re-queue the collected chunks so downstream gets them. Count any
// drops here against the same `dropped_chunks` counter so the live
// session sees them and can warn the user.
// (Codex review 2026/04/17 M1)
for chunk in collected {
if requeue_tx.try_send(chunk).is_err() {
dropped_chunks.fetch_add(1, Ordering::Relaxed);
}
}
eprintln!("[kon-audio] selected microphone: '{name}'");
Ok((
MicrophoneCapture {
stream: Some(stream),
device_name: name.to_string(),
dropped_chunks,
error_rx: Some(err_rx),
},
rx,
))
}
#[allow(clippy::too_many_arguments)]
fn build_input_stream<T>(
device: &cpal::Device,
supported_config: &cpal::SupportedStreamConfig,
sample_rate: u32,
channels: u16,
tx: mpsc::SyncSender<AudioChunk>,
dropped_chunks: Arc<AtomicU64>,
err_tx: mpsc::SyncSender<CaptureRuntimeError>,
device_name: String,
) -> std::result::Result<cpal::Stream, cpal::BuildStreamError>
where
T: Sample + SizedSample,
f32: FromSample<T>,
{
let config: cpal::StreamConfig = supported_config.clone().into();
let err_device_name = device_name.clone();
device.build_input_stream(
&config,
move |data: &[T], _| {
let samples: Vec<f32> = data.iter().copied().map(f32::from_sample).collect();
let chunk = AudioChunk {
samples,
sample_rate,
channels,
};
// try_send fails if the channel is full. Track that explicitly
// rather than swallowing it — Codex review 2026/04/17 caught
// this as a silent-failure risk under sustained load.
if tx.try_send(chunk).is_err() {
dropped_chunks.fetch_add(1, Ordering::Relaxed);
}
},
move |err| {
// Surface stream errors to the live session via err_tx so the
// frontend can show a toast. Also keep the eprintln for ops
// logs. (Codex review 2026/04/17 M2)
eprintln!("[kon-audio] capture error: {err}");
let _ = err_tx.try_send(CaptureRuntimeError {
device_name: err_device_name.clone(),
message: err.to_string(),
});
},
None,
)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn monitor_pattern_detection() {
assert!(is_monitor_name("alsa_output.pci-0000_00_1f.3.analog-stereo.monitor"));
assert!(is_monitor_name("Monitor of Built-in Audio Analog Stereo"));
assert!(is_monitor_name("Some Loopback Device"));
assert!(!is_monitor_name("Blue Yeti USB"));
assert!(!is_monitor_name("alsa_input.pci-0000_00_1f.3.analog-stereo"));
assert!(!is_monitor_name(""));
}
}

View File

@@ -2,12 +2,14 @@ pub mod capture;
pub mod concurrency;
pub mod decode;
pub mod resample;
pub mod streaming_resample;
pub mod vad;
pub mod wav;
pub use capture::{AudioChunk, MicrophoneCapture};
pub use capture::{AudioChunk, CaptureRuntimeError, DeviceInfo, MicrophoneCapture};
pub use concurrency::decode_and_resample;
pub use decode::decode_audio_file;
pub use resample::resample_to_16khz;
pub use streaming_resample::StreamingResampler;
pub use vad::SpeechDetector;
pub use wav::{read_wav, write_wav};

View File

@@ -0,0 +1,211 @@
// Streaming resampler used by the live transcription session.
//
// Microphones expose whatever native rate the device supports (commonly
// 44 100 or 48 000 Hz). whisper.cpp wants 16 kHz mono `f32`. The live
// session calls `push_samples()` with each capture chunk as it arrives
// and gets back zero-or-more 16 kHz samples to enqueue into the model
// input buffer. At end-of-session it calls `flush()` once to drain any
// residual input and the resampler's internal tail.
//
// Implementation notes:
//
// - We use rubato's `SincFixedIn` (same engine the file-level
// `resample::resample_to_16khz` uses) so behaviour stays consistent
// across live + file paths.
// - rubato's fixed-in API requires a constant-size input chunk. We
// buffer captured samples in a residual `Vec<f32>` and only feed
// the resampler when we have a full chunk.
// - When the input rate already matches 16 kHz we skip rubato
// entirely and pass samples straight through (zero allocations
// beyond the returned `Vec`).
// - `flush()` zero-pads the residual to one final chunk, processes
// it, then truncates the output to the proportion that came from
// real (non-padded) samples — otherwise the trailing silence
// produced by the padding leaks into the saved audio file.
use rubato::{
Resampler, SincFixedIn, SincInterpolationParameters, SincInterpolationType,
WindowFunction,
};
use kon_core::constants::WHISPER_SAMPLE_RATE;
use kon_core::error::{KonError, Result};
/// Number of input samples the rubato resampler consumes per `process()`
/// call. Matches the chunk size used in `resample::resample_to_16khz`.
const INPUT_CHUNK: usize = 1024;
pub enum StreamingResampler {
/// Source is already at 16 kHz — emit input verbatim.
Passthrough,
/// Source is at some other rate — feed via rubato.
Sinc {
resampler: SincFixedIn<f32>,
residual: Vec<f32>,
ratio: f64,
},
}
impl StreamingResampler {
/// Construct a resampler that converts `from_rate` Hz mono input to
/// 16 kHz mono output. Returns an error if `from_rate` is zero or if
/// rubato rejects the requested ratio.
pub fn new(from_rate: u32) -> Result<Self> {
if from_rate == 0 {
return Err(KonError::AudioDecodeFailed(
"StreamingResampler: input sample rate is 0".into(),
));
}
if from_rate == WHISPER_SAMPLE_RATE {
return Ok(Self::Passthrough);
}
let ratio = WHISPER_SAMPLE_RATE as f64 / from_rate as f64;
let params = SincInterpolationParameters {
sinc_len: 256,
f_cutoff: 0.95,
oversampling_factor: 128,
interpolation: SincInterpolationType::Cubic,
window: WindowFunction::Blackman,
};
let resampler = SincFixedIn::<f32>::new(
ratio,
1.1, // max relative jitter; mirrors the file-level resampler
params,
INPUT_CHUNK,
1, // mono
)
.map_err(|e| {
KonError::AudioDecodeFailed(format!(
"StreamingResampler init failed: {e}"
))
})?;
Ok(Self::Sinc {
resampler,
residual: Vec::new(),
ratio,
})
}
/// Feed a fresh capture chunk and return any 16 kHz samples that are
/// ready to dispatch. The caller may pass any length; samples that
/// don't yet form a complete `INPUT_CHUNK` are buffered internally
/// and emitted on a later call (or on `flush()`).
pub fn push_samples(&mut self, mono: &[f32]) -> Result<Vec<f32>> {
match self {
Self::Passthrough => Ok(mono.to_vec()),
Self::Sinc { resampler, residual, .. } => {
if mono.is_empty() {
return Ok(Vec::new());
}
residual.extend_from_slice(mono);
let mut out: Vec<f32> = Vec::new();
while residual.len() >= INPUT_CHUNK {
let chunk: Vec<f32> = residual.drain(..INPUT_CHUNK).collect();
let input = vec![chunk];
let result = resampler.process(&input, None).map_err(|e| {
KonError::AudioDecodeFailed(format!(
"StreamingResampler process failed: {e}"
))
})?;
if let Some(channel) = result.into_iter().next() {
out.extend_from_slice(&channel);
}
}
Ok(out)
}
}
}
/// Drain any residual samples and return the final 16 kHz output.
/// Called once when the live session is stopping. Subsequent calls
/// return an empty `Vec`.
pub fn flush(&mut self) -> Result<Vec<f32>> {
match self {
Self::Passthrough => Ok(Vec::new()),
Self::Sinc { resampler, residual, ratio } => {
if residual.is_empty() {
return Ok(Vec::new());
}
let leftover = residual.len();
let mut chunk = std::mem::take(residual);
chunk.resize(INPUT_CHUNK, 0.0);
let input = vec![chunk];
let result = resampler.process(&input, None).map_err(|e| {
KonError::AudioDecodeFailed(format!(
"StreamingResampler flush failed: {e}"
))
})?;
let Some(mut out) = result.into_iter().next() else {
return Ok(Vec::new());
};
// Trim padding-induced output: keep only the proportion
// of samples that came from real input, not from the
// zeros we used to fill the chunk.
let real_out = ((leftover as f64) * *ratio).round() as usize;
if real_out < out.len() {
out.truncate(real_out);
}
Ok(out)
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn passthrough_at_16khz() {
let mut r = StreamingResampler::new(16_000).unwrap();
let out = r.push_samples(&[0.1, 0.2, 0.3]).unwrap();
assert_eq!(out, vec![0.1, 0.2, 0.3]);
assert!(r.flush().unwrap().is_empty());
}
#[test]
fn rejects_zero_rate() {
assert!(StreamingResampler::new(0).is_err());
}
#[test]
fn streaming_48k_to_16k_preserves_duration() {
let from_rate = 48_000u32;
let secs = 1.0;
let n = (from_rate as f64 * secs) as usize;
let samples: Vec<f32> =
(0..n).map(|i| (i as f32 * 0.001).sin()).collect();
let mut r = StreamingResampler::new(from_rate).unwrap();
// Push in irregular chunks to exercise the residual buffer.
let mut produced: Vec<f32> = Vec::new();
for window in samples.chunks(700) {
produced.extend(r.push_samples(window).unwrap());
}
produced.extend(r.flush().unwrap());
let out_secs = produced.len() as f64 / WHISPER_SAMPLE_RATE as f64;
assert!(
(out_secs - secs).abs() < 0.05,
"expected ~{secs}s of 16 kHz output, got {out_secs}s ({} samples)",
produced.len(),
);
}
#[test]
fn flush_after_no_input_is_empty() {
let mut r = StreamingResampler::new(48_000).unwrap();
assert!(r.flush().unwrap().is_empty());
}
}

View File

@@ -72,10 +72,15 @@ impl EvdevHotkeyListener {
tokio::spawn(async move {
let (notify_tx, mut notify_rx) = mpsc::channel::<PathBuf>(32);
// notify watcher runs on a blocking thread internally
// notify watcher runs on a blocking thread internally.
// If inotify itself is unavailable (rare: minimal containers,
// some BSDs misconfigured as Linux) we degrade to "no
// hotplug detection" rather than panicking the task — the
// initial scan_and_attach pass above still picks up all
// devices that exist at startup.
let _watcher = {
let notify_tx = notify_tx.clone();
let mut w = recommended_watcher(move |res: Result<notify::Event, _>| {
let watcher = recommended_watcher(move |res: Result<notify::Event, _>| {
if let Ok(event) = res {
if matches!(event.kind, EventKind::Create(_)) {
for path in event.paths {
@@ -85,11 +90,27 @@ impl EvdevHotkeyListener {
}
}
}
})
.expect("failed to create inotify watcher");
w.watch(Path::new("/dev/input"), RecursiveMode::NonRecursive)
.expect("failed to watch /dev/input");
w
});
match watcher {
Ok(mut w) => match w.watch(Path::new("/dev/input"), RecursiveMode::NonRecursive) {
Ok(()) => Some(w),
Err(e) => {
eprintln!(
"[kon-hotkey] cannot watch /dev/input ({e}); \
hotplug detection disabled, devices present \
at startup still work",
);
None
}
},
Err(e) => {
eprintln!(
"[kon-hotkey] cannot create inotify watcher ({e}); \
hotplug detection disabled",
);
None
}
}
};
loop {

View File

@@ -44,8 +44,8 @@ pub struct InsertTranscriptParams<'a> {
pub engine: Option<&'a str>,
pub model_id: Option<&'a str>,
pub inference_ms: Option<i64>,
pub sample_rate: Option<i32>,
pub audio_channels: Option<i32>,
pub sample_rate: Option<i64>,
pub audio_channels: Option<i64>,
pub format_mode: Option<&'a str>,
pub remove_fillers: bool,
pub british_english: bool,
@@ -94,10 +94,21 @@ pub async fn get_transcript(pool: &SqlitePool, id: &str) -> Result<Option<Transc
}
pub async fn list_transcripts(pool: &SqlitePool, limit: i64) -> Result<Vec<TranscriptRow>> {
list_transcripts_paged(pool, limit, 0).await
}
/// Paginated transcript list. `limit` is rows-per-page, `offset` is rows to skip.
/// Used by HistoryPage to load 50 at a time and append more on scroll.
pub async fn list_transcripts_paged(
pool: &SqlitePool,
limit: i64,
offset: i64,
) -> Result<Vec<TranscriptRow>> {
let rows = sqlx::query(
"SELECT id, text, source, title, audio_path, duration, engine, model_id, inference_ms, sample_rate, audio_channels, format_mode, remove_fillers, british_english, anti_hallucination, created_at FROM transcripts ORDER BY created_at DESC LIMIT ?",
"SELECT id, text, source, title, audio_path, duration, engine, model_id, inference_ms, sample_rate, audio_channels, format_mode, remove_fillers, british_english, anti_hallucination, created_at FROM transcripts ORDER BY created_at DESC LIMIT ? OFFSET ?",
)
.bind(limit)
.bind(offset)
.fetch_all(pool)
.await
.map_err(|e| KonError::StorageError(format!("List transcripts failed: {e}")))?;
@@ -105,6 +116,64 @@ pub async fn list_transcripts(pool: &SqlitePool, limit: i64) -> Result<Vec<Trans
Ok(rows.iter().map(transcript_row_from).collect())
}
/// Total count of transcripts. Useful for displaying "showing 50 of 312" in the UI.
pub async fn count_transcripts(pool: &SqlitePool) -> Result<i64> {
let n: i64 = sqlx::query_scalar("SELECT COUNT(*) FROM transcripts")
.fetch_one(pool)
.await
.map_err(|e| KonError::StorageError(format!("Count transcripts failed: {e}")))?;
Ok(n)
}
/// Update mutable transcript fields. Currently `text` and `title`. Other
/// columns (engine, model, audio path) are immutable post-creation. Returns
/// the number of rows updated (0 if id not found).
///
/// This is the function HistoryPage's rename flow needed; previously the
/// rename was a UI-only state change with a TODO never wired up.
/// (architecture-review.md §13)
pub async fn update_transcript(
pool: &SqlitePool,
id: &str,
text: Option<&str>,
title: Option<&str>,
) -> Result<u64> {
// Build the SET clause dynamically so we only update the fields the
// caller actually changed. Two fields max so we can keep this simple
// without a query builder.
match (text, title) {
(Some(t), Some(ttl)) => {
let res = sqlx::query("UPDATE transcripts SET text = ?, title = ? WHERE id = ?")
.bind(t)
.bind(ttl)
.bind(id)
.execute(pool)
.await
.map_err(|e| KonError::StorageError(format!("Update transcript failed: {e}")))?;
Ok(res.rows_affected())
}
(Some(t), None) => {
let res = sqlx::query("UPDATE transcripts SET text = ? WHERE id = ?")
.bind(t)
.bind(id)
.execute(pool)
.await
.map_err(|e| KonError::StorageError(format!("Update transcript failed: {e}")))?;
Ok(res.rows_affected())
}
(None, Some(ttl)) => {
let res = sqlx::query("UPDATE transcripts SET title = ? WHERE id = ?")
.bind(ttl)
.bind(id)
.execute(pool)
.await
.map_err(|e| KonError::StorageError(format!("Update transcript failed: {e}")))?;
Ok(res.rows_affected())
}
(None, None) => Ok(0),
}
}
pub async fn delete_transcript(pool: &SqlitePool, id: &str) -> Result<()> {
sqlx::query("DELETE FROM transcripts WHERE id = ?")
.bind(id)
@@ -114,6 +183,87 @@ pub async fn delete_transcript(pool: &SqlitePool, id: &str) -> Result<()> {
Ok(())
}
/// Full-text search over transcripts using the FTS5 virtual table created
/// in migration v2. Returns matched transcript rows ordered by FTS rank
/// (best match first). `query` follows FTS5 syntax: bare words AND together
/// by default; quote multi-word phrases; `OR`, `NOT` operators supported.
pub async fn search_transcripts(
pool: &SqlitePool,
query: &str,
limit: i64,
) -> Result<Vec<TranscriptRow>> {
let rows = sqlx::query(
"SELECT t.id, t.text, t.source, t.title, t.audio_path, t.duration, t.engine, t.model_id, t.inference_ms, t.sample_rate, t.audio_channels, t.format_mode, t.remove_fillers, t.british_english, t.anti_hallucination, t.created_at \
FROM transcripts t \
JOIN transcripts_fts fts ON fts.rowid = t.rowid \
WHERE transcripts_fts MATCH ? \
ORDER BY fts.rank LIMIT ?",
)
.bind(query)
.bind(limit)
.fetch_all(pool)
.await
.map_err(|e| KonError::StorageError(format!("FTS search failed: {e}")))?;
Ok(rows.iter().map(transcript_row_from).collect())
}
// --- Dictionary (custom vocabulary injected into LLM cleanup prompts) ---
#[derive(Debug, Clone)]
pub struct DictionaryEntry {
pub id: i64,
pub term: String,
pub note: Option<String>,
}
pub async fn list_dictionary(pool: &SqlitePool) -> Result<Vec<DictionaryEntry>> {
let rows = sqlx::query("SELECT id, term, note FROM dictionary ORDER BY term ASC")
.fetch_all(pool)
.await
.map_err(|e| KonError::StorageError(format!("List dictionary failed: {e}")))?;
Ok(rows
.into_iter()
.map(|r| DictionaryEntry {
id: r.get("id"),
term: r.get("term"),
note: r.get("note"),
})
.collect())
}
pub async fn add_dictionary_entry(
pool: &SqlitePool,
term: &str,
note: Option<&str>,
) -> Result<i64> {
// ON CONFLICT … DO UPDATE so we always RETURN the actual row id.
// The previous `INSERT OR IGNORE` returned 0 (or a stale auto-increment
// counter) for duplicate terms, which lied to the frontend about which
// row it had just upserted. Bumping `note` to `excluded.note` also lets
// the user edit a term's note by re-adding it.
let row = sqlx::query(
"INSERT INTO dictionary (term, note) VALUES (?, ?) \
ON CONFLICT(term) DO UPDATE SET note = excluded.note \
RETURNING id",
)
.bind(term)
.bind(note)
.fetch_one(pool)
.await
.map_err(|e| KonError::StorageError(format!("Insert dictionary entry failed: {e}")))?;
Ok(row.get("id"))
}
pub async fn delete_dictionary_entry(pool: &SqlitePool, id: i64) -> Result<()> {
sqlx::query("DELETE FROM dictionary WHERE id = ?")
.bind(id)
.execute(pool)
.await
.map_err(|e| KonError::StorageError(format!("Delete dictionary entry failed: {e}")))?;
Ok(())
}
// --- Task CRUD ---
pub async fn insert_task(
@@ -209,8 +359,8 @@ pub struct TranscriptRow {
pub engine: Option<String>,
pub model_id: Option<String>,
pub inference_ms: Option<i64>,
pub sample_rate: Option<i32>,
pub audio_channels: Option<i32>,
pub sample_rate: Option<i64>,
pub audio_channels: Option<i64>,
pub format_mode: Option<String>,
pub remove_fillers: bool,
pub british_english: bool,
@@ -287,6 +437,41 @@ pub async fn log_error(
Ok(())
}
/// Read the most recent N rows from `error_log`, newest first. Used by the
/// diagnostic-report bundler in Settings → About.
#[derive(Debug, Clone)]
pub struct ErrorLogRow {
pub id: i64,
pub timestamp: String,
pub context: String,
pub error_code: Option<String>,
pub message: String,
pub metadata: Option<String>,
}
pub async fn list_recent_errors(pool: &SqlitePool, limit: i64) -> Result<Vec<ErrorLogRow>> {
let rows = sqlx::query(
"SELECT id, timestamp, context, error_code, message, metadata \
FROM error_log ORDER BY id DESC LIMIT ?",
)
.bind(limit)
.fetch_all(pool)
.await
.map_err(|e| KonError::StorageError(format!("Read error_log failed: {e}")))?;
Ok(rows
.into_iter()
.map(|r| ErrorLogRow {
id: r.get("id"),
timestamp: r.get("timestamp"),
context: r.get("context"),
error_code: r.get("error_code"),
message: r.get("message"),
metadata: r.get("metadata"),
})
.collect())
}
#[cfg(test)]
mod tests {
use super::*;

View File

@@ -1,17 +1,55 @@
use std::path::PathBuf;
/// Resolve the app data directory.
/// Windows: %LOCALAPPDATA%/kon
/// Unix: ~/.kon
/// Resolve the per-user app data directory, following each OS's convention:
///
/// - Windows: `%LOCALAPPDATA%\kon\` e.g. `C:\Users\Jake\AppData\Local\kon`
/// - macOS: `~/Library/Application Support/Kon/`
/// - Linux: `$XDG_DATA_HOME/kon` or `~/.local/share/kon` (XDG Base Directory),
/// with a fallback to the legacy `~/.kon/` if it already exists, so
/// existing installs keep working.
/// - Other Unix: `~/.kon/`
///
/// TODO: Consolidate with `crates/transcription/src/model_manager.rs::dirs_path()`
/// into a shared helper in `crates/core/` to avoid duplicating platform-specific
/// path logic across crates.
pub fn app_data_dir() -> PathBuf {
if cfg!(target_os = "windows") {
#[cfg(target_os = "windows")]
{
let local_app_data = std::env::var("LOCALAPPDATA").unwrap_or_else(|_| ".".to_string());
PathBuf::from(local_app_data).join("kon")
} else {
return PathBuf::from(local_app_data).join("kon");
}
#[cfg(target_os = "macos")]
{
let home = std::env::var("HOME").unwrap_or_else(|_| "/tmp".to_string());
return PathBuf::from(home)
.join("Library")
.join("Application Support")
.join("Kon");
}
#[cfg(target_os = "linux")]
{
let home = std::env::var("HOME").unwrap_or_else(|_| "/tmp".to_string());
// Honour the legacy ~/.kon/ if it exists on disk so existing
// installs are not orphaned. New installs follow XDG.
let legacy = PathBuf::from(&home).join(".kon");
if legacy.exists() {
return legacy;
}
// XDG Base Directory: $XDG_DATA_HOME/kon or default ~/.local/share/kon
if let Ok(xdg) = std::env::var("XDG_DATA_HOME") {
if !xdg.is_empty() {
return PathBuf::from(xdg).join("kon");
}
}
return PathBuf::from(home).join(".local").join("share").join("kon");
}
#[cfg(not(any(target_os = "windows", target_os = "macos", target_os = "linux")))]
{
let home = std::env::var("HOME").unwrap_or_else(|_| "/tmp".to_string());
PathBuf::from(home).join(".kon")
}
@@ -26,3 +64,16 @@ pub fn database_path() -> PathBuf {
pub fn recordings_dir() -> PathBuf {
app_data_dir().join("recordings")
}
/// Directory for crash dumps written by the Rust panic hook.
/// Each crash is a single text file: `<unix-ts>-<short-id>.crash`.
/// Used by the diagnostic-report bundler in Settings → About.
pub fn crashes_dir() -> PathBuf {
app_data_dir().join("crashes")
}
/// Directory for the rolling Rust log file (kon.log + rotated kon.log.1, etc).
/// Subscribers configured in src-tauri/src/lib.rs at startup.
pub fn logs_dir() -> PathBuf {
app_data_dir().join("logs")
}

View File

@@ -3,8 +3,10 @@ pub mod file_storage;
pub mod migrations;
pub use database::{
complete_task, delete_task, delete_transcript, get_setting, get_transcript, init, insert_task,
insert_transcript, list_tasks, list_transcripts, log_error, set_setting,
InsertTranscriptParams, TaskRow, TranscriptRow,
add_dictionary_entry, complete_task, count_transcripts, delete_dictionary_entry,
delete_task, delete_transcript, get_setting, get_transcript, init, insert_task,
insert_transcript, list_dictionary, list_recent_errors, list_tasks, list_transcripts,
list_transcripts_paged, log_error, search_transcripts, set_setting, update_transcript,
DictionaryEntry, ErrorLogRow, InsertTranscriptParams, TaskRow, TranscriptRow,
};
pub use file_storage::{app_data_dir, database_path, recordings_dir};
pub use file_storage::{app_data_dir, crashes_dir, database_path, logs_dir, recordings_dir};

View File

@@ -73,8 +73,77 @@ const MIGRATIONS: &[(i64, &str, &str)] = &[
CREATE INDEX IF NOT EXISTS idx_tasks_transcript ON tasks(source_transcript_id);
CREATE INDEX IF NOT EXISTS idx_error_log_context ON error_log(context)
"#),
(2, "transcripts FTS5 + dictionary table", r#"
CREATE VIRTUAL TABLE IF NOT EXISTS transcripts_fts USING fts5(
text,
title,
content='transcripts',
content_rowid='rowid',
tokenize='porter unicode61 remove_diacritics 2'
);
CREATE TRIGGER IF NOT EXISTS transcripts_ai AFTER INSERT ON transcripts BEGIN
INSERT INTO transcripts_fts(rowid, text, title)
VALUES (new.rowid, new.text, COALESCE(new.title, ''));
END;
CREATE TRIGGER IF NOT EXISTS transcripts_ad AFTER DELETE ON transcripts BEGIN
INSERT INTO transcripts_fts(transcripts_fts, rowid, text, title)
VALUES ('delete', old.rowid, old.text, COALESCE(old.title, ''));
END;
CREATE TRIGGER IF NOT EXISTS transcripts_au AFTER UPDATE ON transcripts BEGIN
INSERT INTO transcripts_fts(transcripts_fts, rowid, text, title)
VALUES ('delete', old.rowid, old.text, COALESCE(old.title, ''));
INSERT INTO transcripts_fts(rowid, text, title)
VALUES (new.rowid, new.text, COALESCE(new.title, ''));
END;
CREATE TABLE IF NOT EXISTS dictionary (
id INTEGER PRIMARY KEY AUTOINCREMENT,
term TEXT NOT NULL UNIQUE,
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_dictionary_term ON dictionary(term)
"#),
];
/// Split SQL into individual statements, respecting BEGIN...END trigger blocks.
fn split_statements(sql: &str) -> Vec<String> {
let mut result = Vec::new();
let mut current = String::new();
let mut depth: i32 = 0;
let upper = sql.to_ascii_uppercase();
let ub = upper.as_bytes();
let ob = sql.as_bytes();
let n = ob.len();
let mut i = 0;
while i < n {
let prev_alpha = i > 0 && ob[i - 1].is_ascii_alphanumeric();
if !prev_alpha && ub[i..].starts_with(b"BEGIN") {
let next_alpha = i + 5 < n && ob[i + 5].is_ascii_alphanumeric();
if !next_alpha { depth += 1; }
}
if !prev_alpha && ub[i..].starts_with(b"END") {
let next_alpha = i + 3 < n && ob[i + 3].is_ascii_alphanumeric();
if !next_alpha && depth > 0 { depth -= 1; }
}
if ob[i] == b';' && depth == 0 {
let stmt = current.trim().to_string();
if !stmt.is_empty() { result.push(stmt); }
current = String::new();
} else {
current.push(ob[i] as char);
}
i += 1;
}
let stmt = current.trim().to_string();
if !stmt.is_empty() { result.push(stmt); }
result
}
/// Ensure the schema_version table exists and run any pending migrations.
pub async fn run_migrations(pool: &SqlitePool) -> Result<()> {
sqlx::query(
@@ -97,12 +166,9 @@ pub async fn run_migrations(pool: &SqlitePool) -> Result<()> {
if *version > current {
log::info!("Running migration {}: {}", version, description);
let statements: Vec<&str> = sql.split(';')
.map(|s| s.trim())
.filter(|s| !s.is_empty())
.collect();
let statements = split_statements(sql);
for statement in statements {
for statement in &statements {
sqlx::query(statement)
.execute(pool)
.await

View File

@@ -6,6 +6,7 @@ pub use concurrency::run_inference;
pub use local_engine::{
load_parakeet, load_whisper, LocalEngine, TimedTranscript,
};
pub use transcribe_rs::SpeechModel;
pub use model_manager::{
download, is_downloaded, list_downloaded, model_dir, models_dir,
};

26
run.sh Executable file
View File

@@ -0,0 +1,26 @@
#!/usr/bin/env bash
# Kon dev launcher. Starts Vite first, waits for it to bind port 1420,
# then runs Tauri without triggering a second Vite instance.
# For performance testing use: npm run tauri build
set -euo pipefail
cd "$(dirname "$0")"
export LIBCLANG_PATH=/usr/lib64/llvm21/lib64
printf 'Starting Vite dev server...\n' >&2
npm run dev &
VITE_PID=$!
until curl -sf http://localhost:1420 > /dev/null 2>&1; do sleep 0.5; done
printf 'Vite ready. Launching Tauri...\n' >&2
# Blank beforeDevCommand so Tauri doesn't spawn a second Vite
sed -i 's/"beforeDevCommand": "npm run dev"/"beforeDevCommand": ""/' src-tauri/tauri.conf.json
cleanup() {
sed -i 's/"beforeDevCommand": ""/"beforeDevCommand": "npm run dev"/' src-tauri/tauri.conf.json
kill "$VITE_PID" 2>/dev/null || true
}
trap cleanup EXIT INT TERM
npx tauri dev

View File

@@ -35,8 +35,11 @@ serde_json = "1"
# Async runtime (spawn_blocking for inference)
tokio = { version = "1", features = ["rt", "sync"] }
arboard = "3.6.1"
tauri-plugin-mcp = "0.7.1"
# SqlitePool is named directly from src-tauri/src/lib.rs (the AppState
# stores it). Must be unconditional, not Linux-only — naming a type from
# a transitive dep requires the dep be listed here too.
sqlx = { version = "0.8", features = ["sqlite", "runtime-tokio"] }
[target.'cfg(target_os = "linux")'.dependencies]
webkit2gtk = "2.0"
sqlx = { version = "0.8", features = ["sqlite", "runtime-tokio"] }

View File

@@ -2,7 +2,7 @@
"$schema": "../gen/schemas/desktop-schema.json",
"identifier": "default",
"description": "Capability for the main window",
"windows": ["main", "tasks-float"],
"windows": ["main", "tasks-float", "transcript-viewer"],
"permissions": [
"core:default",
"core:window:allow-start-dragging",

File diff suppressed because it is too large Load Diff

View File

@@ -1,13 +1,228 @@
use std::path::PathBuf;
use std::sync::{Arc, Mutex};
use tauri::Manager;
use tauri::{Emitter, Manager};
use tokio::sync::mpsc as tokio_mpsc;
use kon_audio::{DeviceInfo, MicrophoneCapture};
use kon_core::constants::WHISPER_SAMPLE_RATE;
use kon_core::types::AudioSamples;
/// Save PCM f32 samples as a WAV file. Returns the file path.
/// Enumerate every input device available to cpal, with metadata for the
/// Settings device-picker UI. Includes a flag for likely PulseAudio /
/// PipeWire monitor sources so the UI can warn the user.
#[tauri::command]
pub async fn save_audio(
pub async fn list_audio_devices() -> Result<Vec<DeviceInfo>, String> {
tokio::task::spawn_blocking(MicrophoneCapture::list_devices)
.await
.map_err(|e| format!("join error: {e}"))?
.map_err(|e| e.to_string())
}
/// Shared state for native microphone capture.
pub struct NativeCaptureState {
/// Stop signal sender — dropping this stops the accumulator task.
stop_tx: Mutex<Option<tokio_mpsc::Sender<()>>>,
/// All captured samples (16kHz mono) for save_audio.
all_samples: Arc<Mutex<Vec<f32>>>,
}
impl NativeCaptureState {
pub fn new() -> Self {
Self {
stop_tx: Mutex::new(None),
all_samples: Arc::new(Mutex::new(Vec::new())),
}
}
}
/// Start native microphone capture via cpal.
/// Streams 16kHz mono PCM chunks to the frontend via `native-pcm` events.
///
/// `device_name`: explicit device name (from `list_audio_devices`) or None / ""
/// to auto-select. The frontend passes `settings.microphoneDevice` here so the
/// user's pick from Settings → Audio → Microphone takes effect.
#[tauri::command]
pub async fn start_native_capture(
app: tauri::AppHandle,
state: tauri::State<'_, NativeCaptureState>,
device_name: Option<String>,
) -> Result<(), String> {
eprintln!(
"[native-capture] start_native_capture called (device='{}')",
device_name.as_deref().unwrap_or("<auto>")
);
// Stop any existing capture: send an explicit stop signal first, then
// drop the sender. The accumulator task watches for `Disconnected` too,
// but signalling explicitly avoids the brief race window.
// (Codex review 2026/04/17 D1)
if let Some(tx) = state.stop_tx.lock().unwrap().take() {
let _ = tx.try_send(());
drop(tx);
}
// `MicrophoneCapture::start()` is synchronous and may spend up to
// DEVICE_VALIDATION_MS per device per pass (350ms × N devices × 2 passes
// worst case). Run on a blocking thread so the async runtime stays
// responsive to other Tauri commands. (Codex review 2026/04/17 D2)
let device_name_for_blocking = device_name.clone();
let (capture, rx) = tokio::task::spawn_blocking(move || {
match device_name_for_blocking.as_deref() {
Some(name) if !name.is_empty() => MicrophoneCapture::start_with_device(name),
_ => MicrophoneCapture::start(),
}
})
.await
.map_err(|e| format!("audio task join error: {e}"))?
.map_err(|e| {
eprintln!("[native-capture] MicrophoneCapture::start failed: {e}");
e.to_string()
})?;
eprintln!(
"[native-capture] cpal capture started successfully on '{}'",
capture.device_name
);
// Wrap capture in Arc<Mutex> so it can be moved into the blocking task
let capture = Arc::new(Mutex::new(Some(capture)));
let capture_clone = capture.clone();
let all_samples = state.all_samples.clone();
all_samples.lock().unwrap().clear();
let (stop_tx, mut stop_rx) = tokio_mpsc::channel::<()>(1);
*state.stop_tx.lock().unwrap() = Some(stop_tx);
let all_samples_clone = all_samples.clone();
// Spawn a task that reads cpal chunks, downsamples to 16kHz mono,
// and emits events to the frontend
tokio::spawn(async move {
let mut pcm_buffer: Vec<f32> = Vec::new();
let chunk_size = 8000_usize; // ~0.5s at 16kHz
loop {
// Check for stop signal (non-blocking)
if stop_rx.try_recv().is_ok() {
break;
}
// Drain available audio chunks from cpal (non-blocking).
// Distinguish Empty (try again) from Disconnected (capture stream
// is dead — exit the loop, don't spin forever).
// (Codex review 2026/04/17 M3)
let mut got_data = false;
let mut capture_dead = false;
loop {
match rx.try_recv() {
Ok(chunk) => {
got_data = true;
let sample_rate = chunk.sample_rate;
let channels = chunk.channels as usize;
// Downmix to mono if stereo
let mono: Vec<f32> = if channels > 1 {
chunk
.samples
.chunks(channels)
.map(|frame| frame.iter().sum::<f32>() / channels as f32)
.collect()
} else {
chunk.samples
};
// Downsample to 16kHz using simple decimation
// (acceptable quality for speech — same approach as pcm-processor.js)
let ratio = sample_rate as f64 / WHISPER_SAMPLE_RATE as f64;
if (ratio - 1.0).abs() < 0.01 {
pcm_buffer.extend_from_slice(&mono);
} else {
let mut pos: f64 = 0.0;
for &s in &mono {
pos += 1.0;
if pos >= ratio {
pcm_buffer.push(s);
pos -= ratio;
}
}
}
}
Err(std::sync::mpsc::TryRecvError::Empty) => break,
Err(std::sync::mpsc::TryRecvError::Disconnected) => {
eprintln!("[native-capture] capture stream disconnected; accumulator exiting");
capture_dead = true;
break;
}
}
}
// Emit chunks to frontend when we have enough
while pcm_buffer.len() >= chunk_size {
let chunk: Vec<f32> = pcm_buffer.drain(..chunk_size).collect();
// Store for save_audio
if let Ok(mut all) = all_samples_clone.lock() {
all.extend_from_slice(&chunk);
}
let _ = app.emit("native-pcm", serde_json::json!({
"samples": chunk,
}));
}
if capture_dead {
break;
}
if !got_data {
// Avoid busy-spinning when no audio data is available
tokio::time::sleep(std::time::Duration::from_millis(10)).await;
}
}
// Emit any remaining samples
if !pcm_buffer.is_empty() {
if let Ok(mut all) = all_samples_clone.lock() {
all.extend_from_slice(&pcm_buffer);
}
let _ = app.emit("native-pcm", serde_json::json!({
"samples": pcm_buffer,
}));
}
// Drop the capture to stop the cpal stream
if let Ok(mut cap) = capture_clone.lock() {
cap.take();
}
});
Ok(())
}
/// Stop native microphone capture. Returns all captured samples (16kHz mono).
#[tauri::command]
pub async fn stop_native_capture(
state: tauri::State<'_, NativeCaptureState>,
) -> Result<Vec<f32>, String> {
// Extract the stop sender without holding the guard across an await
let stop_tx = state.stop_tx.lock().unwrap().take();
if let Some(tx) = stop_tx {
let _ = tx.send(()).await;
}
// Brief delay to let the accumulator flush
tokio::time::sleep(std::time::Duration::from_millis(50)).await;
let samples = {
let mut all = state.all_samples.lock().unwrap();
std::mem::take(&mut *all)
};
Ok(samples)
}
pub async fn persist_audio_samples(
app: &tauri::AppHandle,
samples: Vec<f32>,
output_folder: Option<String>,
) -> Result<String, String> {
@@ -47,3 +262,13 @@ pub async fn save_audio(
Ok(path.to_string_lossy().to_string())
}
/// Save PCM f32 samples as a WAV file. Returns the file path.
#[tauri::command]
pub async fn save_audio(
app: tauri::AppHandle,
samples: Vec<f32>,
output_folder: Option<String>,
) -> Result<String, String> {
persist_audio_samples(&app, samples, output_folder).await
}

View File

@@ -0,0 +1,435 @@
//! Diagnostics: panic hook, frontend error capture, and the manual
//! diagnostic-report bundler used by Settings → About.
//!
//! Privacy posture (matches Kon's local-first positioning):
//! - All capture is to disk only. Nothing is transmitted.
//! - The manual report bundler shows the user exactly what would be
//! shared and lets them choose to copy/save/email it.
//! - No remote endpoint, no Sentry, no opt-out telemetry.
use std::fs;
use std::io::Write;
use std::panic;
use std::path::PathBuf;
use std::time::{SystemTime, UNIX_EPOCH};
use serde::{Deserialize, Serialize};
use kon_storage::{
app_data_dir, crashes_dir, list_recent_errors, log_error, logs_dir, ErrorLogRow,
};
use crate::AppState;
const DEFAULT_RECENT_ERRORS: i64 = 50;
const KON_VERSION: &str = env!("CARGO_PKG_VERSION");
/// Install the Rust panic hook. Writes each panic to a separate file in
/// crashes_dir so the diagnostic-report bundler can attach them. Also
/// keeps the default panic message on stderr so terminal users see it.
///
/// Called once at startup before tauri::Builder.
pub fn install_panic_hook() {
if let Err(e) = fs::create_dir_all(crashes_dir()) {
eprintln!("[diagnostics] could not create crashes dir: {e}");
return;
}
let default_hook = panic::take_hook();
panic::set_hook(Box::new(move |info| {
// Write a minimal text dump. Backtraces require RUST_BACKTRACE=1
// and the std::backtrace API; we capture what's free.
let ts = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0);
let short = (ts % 0xFFFF) as u16;
let path = crashes_dir().join(format!("{ts}-{short:04x}.crash"));
let payload = format!(
"Kon crash dump\n\
==============\n\
Version: {ver}\n\
Timestamp: {ts} (UTC seconds)\n\
Thread: {thread}\n\
\n\
Panic message:\n\
{info}\n\
\n\
OS: {os} {arch}\n\
RUST_BACKTRACE: {bt}\n",
ver = KON_VERSION,
ts = ts,
thread = std::thread::current()
.name()
.unwrap_or("<unnamed>"),
info = info,
os = std::env::consts::OS,
arch = std::env::consts::ARCH,
bt = std::env::var("RUST_BACKTRACE").unwrap_or_else(|_| "0".to_string()),
);
if let Ok(mut f) = fs::File::create(&path) {
let _ = f.write_all(payload.as_bytes());
}
// Still call the original hook so the panic shows in the terminal.
default_hook(info);
}));
}
/// Tauri command: persist a frontend-side error to error_log.
///
/// Wired from the global window.onerror + window.unhandledrejection
/// handlers in the Svelte root layout. Also called explicitly by
/// invokeWithToast on failure paths so we have a unified record.
#[tauri::command]
pub async fn log_frontend_error(
state: tauri::State<'_, AppState>,
context: String,
message: String,
stack: Option<String>,
) -> Result<(), String> {
let metadata = stack.map(|s| {
// Truncate stacks to keep the table from bloating; the full stack
// is also visible in the browser console anyway.
s.chars().take(2000).collect::<String>()
});
log_error(
&state.db,
if context.is_empty() { "frontend" } else { &context },
Some("FRONTEND_ERROR"),
&message,
metadata.as_deref(),
)
.await
.map_err(|e| e.to_string())
}
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct ErrorLogDto {
pub id: i64,
pub timestamp: String,
pub context: String,
pub error_code: Option<String>,
pub message: String,
pub metadata: Option<String>,
}
impl From<ErrorLogRow> for ErrorLogDto {
fn from(r: ErrorLogRow) -> Self {
Self {
id: r.id,
timestamp: r.timestamp,
context: r.context,
error_code: r.error_code,
message: r.message,
metadata: r.metadata,
}
}
}
#[tauri::command]
pub async fn list_recent_errors_command(
state: tauri::State<'_, AppState>,
limit: Option<i64>,
) -> Result<Vec<ErrorLogDto>, String> {
let n = limit.unwrap_or(DEFAULT_RECENT_ERRORS).clamp(1, 1000);
list_recent_errors(&state.db, n)
.await
.map(|rows| rows.into_iter().map(ErrorLogDto::from).collect())
.map_err(|e| e.to_string())
}
/// One crash file on disk.
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct CrashFile {
pub filename: String,
pub mtime_secs: u64,
pub size_bytes: u64,
pub preview: String, // first ~600 chars
}
#[tauri::command]
pub async fn list_crash_files() -> Result<Vec<CrashFile>, String> {
let dir = crashes_dir();
let entries = match fs::read_dir(&dir) {
Ok(e) => e,
Err(_) => return Ok(Vec::new()), // no crashes dir = no crashes, perfect
};
let mut out: Vec<CrashFile> = Vec::new();
for entry in entries.flatten() {
let path = entry.path();
if path.extension().and_then(|s| s.to_str()) != Some("crash") {
continue;
}
let meta = match fs::metadata(&path) {
Ok(m) => m,
Err(_) => continue,
};
let mtime_secs = meta
.modified()
.ok()
.and_then(|t| t.duration_since(UNIX_EPOCH).ok())
.map(|d| d.as_secs())
.unwrap_or(0);
let size_bytes = meta.len();
let preview = match fs::read_to_string(&path) {
Ok(s) => s.chars().take(600).collect::<String>(),
Err(_) => String::new(),
};
out.push(CrashFile {
filename: path
.file_name()
.and_then(|s| s.to_str())
.map(String::from)
.unwrap_or_default(),
mtime_secs,
size_bytes,
preview,
});
}
// Newest first
out.sort_by(|a, b| b.mtime_secs.cmp(&a.mtime_secs));
Ok(out)
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct ReportOptions {
pub include_recent_errors: bool,
pub include_crashes: bool,
pub include_settings: bool,
pub include_log_tail: bool,
}
impl Default for ReportOptions {
fn default() -> Self {
Self {
include_recent_errors: true,
include_crashes: true,
include_settings: true,
include_log_tail: true,
}
}
}
/// Build a single human-readable diagnostic-report string. The user inspects
/// this in Settings → About before deciding whether to copy/save/email it.
///
/// The string is plain markdown so it can be pasted into an email, a
/// Discord thread, or a GitHub issue without further conversion.
#[tauri::command]
pub async fn generate_diagnostic_report(
state: tauri::State<'_, AppState>,
options: Option<ReportOptions>,
) -> Result<String, String> {
let opts = options.unwrap_or_default();
let mut out = String::new();
out.push_str("# Kon diagnostic report\n\n");
out.push_str(&format!("- Version: `{}`\n", KON_VERSION));
out.push_str(&format!(
"- OS / arch: `{} / {}`\n",
std::env::consts::OS,
std::env::consts::ARCH
));
out.push_str(&format!("- App data dir: `{}`\n", app_data_dir().display()));
let now = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0);
out.push_str(&format!("- Generated: unix `{}`\n", now));
out.push_str("\n");
out.push_str("> This report is local-only until you choose to share it. \
Review the contents below before sending to anyone.\n\n");
if opts.include_settings {
out.push_str("## Settings (sanitised)\n\n");
match kon_storage::get_setting(&state.db, "kon_preferences").await {
Ok(Some(json)) => {
out.push_str("```json\n");
out.push_str(&json);
out.push_str("\n```\n\n");
}
Ok(None) => out.push_str("_(no preferences saved)_\n\n"),
Err(e) => out.push_str(&format!("_(could not read preferences: {e})_\n\n")),
}
}
if opts.include_recent_errors {
out.push_str("## Recent errors (newest first)\n\n");
match list_recent_errors(&state.db, DEFAULT_RECENT_ERRORS).await {
Ok(rows) if rows.is_empty() => out.push_str("_(no errors logged)_\n\n"),
Ok(rows) => {
for r in rows {
out.push_str(&format!(
"- `{}` **{}** [{}]{}: {}\n",
r.timestamp,
r.context,
r.error_code.as_deref().unwrap_or("?"),
if r.metadata.is_some() { " (+meta)" } else { "" },
r.message
.chars()
.take(400)
.collect::<String>()
.replace('\n', " "),
));
}
out.push('\n');
}
Err(e) => out.push_str(&format!("_(could not read error log: {e})_\n\n")),
}
}
if opts.include_crashes {
out.push_str("## Crash dumps\n\n");
let crashes = list_crash_files().await.unwrap_or_default();
if crashes.is_empty() {
out.push_str("_(no crash dumps on disk — nothing has crashed)_\n\n");
} else {
for c in crashes.iter().take(5) {
out.push_str(&format!(
"### `{}` ({} bytes)\n\n```\n{}\n```\n\n",
c.filename, c.size_bytes, c.preview
));
}
if crashes.len() > 5 {
out.push_str(&format!(
"_(plus {} older crash dumps in `{}`)_\n\n",
crashes.len() - 5,
crashes_dir().display()
));
}
}
}
if opts.include_log_tail {
out.push_str("## Log tail\n\n");
let log_path = logs_dir().join("kon.log");
match read_tail(&log_path, 8000) {
Ok(tail) if tail.is_empty() => {
out.push_str("_(log file empty or not yet written)_\n\n");
}
Ok(tail) => {
out.push_str("```\n");
out.push_str(&tail);
out.push_str("\n```\n\n");
}
Err(_) => out.push_str("_(no log file found)_\n\n"),
}
}
out.push_str("---\n\n");
out.push_str("Generated by Kon. To share, copy the entire markdown above \
and paste it into an email or issue. Email: jake@corbel.consulting.\n");
Ok(out)
}
/// Read the last `n_bytes` of a file as a UTF-8 string, lossy-decoded.
/// Used to show recent log content without loading huge files.
fn read_tail(path: &PathBuf, n_bytes: u64) -> std::io::Result<String> {
use std::io::{Read, Seek, SeekFrom};
let mut f = fs::File::open(path)?;
let len = f.metadata()?.len();
let start = len.saturating_sub(n_bytes);
f.seek(SeekFrom::Start(start))?;
let mut buf = Vec::new();
f.read_to_end(&mut buf)?;
Ok(String::from_utf8_lossy(&buf).to_string())
}
/// Cross-platform OS info exposed to the frontend so the UI can adapt:
/// hotkey labels (Cmd vs Ctrl), file-picker text, "open file location"
/// terminology, etc. Returned on demand via the `get_os_info` Tauri
/// command rather than recomputed in JS so we have a single source of
/// truth in the Rust layer.
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct OsInfo {
/// "windows" | "macos" | "linux" | "ios" | "android" | other
pub os: String,
/// "x86_64" | "aarch64" | other
pub arch: String,
/// Human-readable platform family for tooltips: "Windows", "macOS",
/// "Linux"
pub family: String,
/// True if the platform uses the Cmd modifier rather than Ctrl as
/// the primary accelerator (macOS).
pub uses_cmd: bool,
/// True if the user is currently in a Wayland session (Linux only).
/// Useful for status display; the actual workaround is automatic.
pub is_wayland: bool,
/// True if the custom evdev hotkey listener is the primary backend
/// (Linux only). Otherwise the Tauri global-shortcut plugin is used.
pub custom_hotkey_backend: bool,
/// Convenience: a localised label for the primary modifier
/// ("Cmd" on macOS, "Ctrl" elsewhere).
pub primary_modifier_label: String,
}
#[tauri::command]
pub fn get_os_info() -> OsInfo {
let os = std::env::consts::OS.to_string();
let arch = std::env::consts::ARCH.to_string();
let family = match os.as_str() {
"windows" => "Windows",
"macos" => "macOS",
"linux" => "Linux",
"ios" => "iOS",
"android" => "Android",
_ => "Unknown",
}
.to_string();
let uses_cmd = os == "macos";
let primary_modifier_label = if uses_cmd { "Cmd" } else { "Ctrl" }.to_string();
let is_wayland = if os == "linux" {
std::env::var("XDG_SESSION_TYPE")
.map(|v| v.eq_ignore_ascii_case("wayland"))
.unwrap_or(false)
} else {
false
};
let custom_hotkey_backend = os == "linux";
OsInfo {
os,
arch,
family,
uses_cmd,
is_wayland,
custom_hotkey_backend,
primary_modifier_label,
}
}
/// Write the report to a file and return the path. Lets the user save it
/// somewhere they can attach to an email.
#[tauri::command]
pub async fn save_diagnostic_report(
app: tauri::AppHandle,
state: tauri::State<'_, AppState>,
options: Option<ReportOptions>,
) -> Result<String, String> {
let _ = app; // reserved for future dialog integration
let report = generate_diagnostic_report(state, options).await?;
let dir = app_data_dir().join("diagnostic-reports");
fs::create_dir_all(&dir).map_err(|e| format!("create dir: {e}"))?;
let ts = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0);
let path = dir.join(format!("kon-diagnostic-{ts}.md"));
fs::write(&path, &report).map_err(|e| format!("write file: {e}"))?;
Ok(path.to_string_lossy().to_string())
}

View File

@@ -0,0 +1,635 @@
#![allow(clippy::too_many_arguments)]
use std::sync::{
atomic::{AtomicBool, AtomicU64, Ordering},
Arc, Mutex,
};
use std::thread;
use std::time::{Duration, Instant};
use serde::{Deserialize, Serialize};
use tauri::ipc::Channel;
use crate::commands::audio::persist_audio_samples;
use crate::commands::models::{default_model_id_for_engine, ensure_model_loaded};
use crate::AppState;
use kon_ai_formatting::{
post_process_segments, FormatMode, PostProcessOptions,
};
use kon_audio::{MicrophoneCapture, StreamingResampler};
use kon_core::constants::WHISPER_SAMPLE_RATE;
use kon_core::types::{AudioSamples, Segment, TranscriptionOptions};
use kon_transcription::LocalEngine;
const CHUNK_SAMPLES: usize = 32_000; // 2s at 16kHz
const OVERLAP_SAMPLES: usize = 4_000; // 0.25s at 16kHz
const FINAL_CHUNK_MIN_SAMPLES: usize = 4_000; // 0.25s
const MAX_PENDING_SAMPLES: usize = CHUNK_SAMPLES;
const SPEECH_FRAME_SAMPLES: usize = 800; // 50ms
const MIN_SPEECH_FRAMES: usize = 1; // any plausible speech-like frame
const RMS_SPEECH_THRESHOLD: f32 = 0.001;
const PEAK_SPEECH_THRESHOLD: f32 = 0.004;
const FLATLINE_PEAK_THRESHOLD: f32 = 0.0005;
#[derive(Default)]
pub struct LiveTranscriptionState {
next_session_id: AtomicU64,
running: Mutex<Option<RunningLiveSession>>,
}
struct RunningLiveSession {
id: u64,
output_folder: Option<String>,
stop_flag: Arc<AtomicBool>,
handle: tokio::task::JoinHandle<Result<LiveSessionSummary, String>>,
status_channel: Channel<LiveStatusMessage>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct StartLiveTranscriptionConfig {
pub engine: String,
pub model_id: Option<String>,
pub language: Option<String>,
pub initial_prompt: Option<String>,
pub save_audio: bool,
pub output_folder: Option<String>,
pub remove_fillers: bool,
pub british_english: bool,
pub anti_hallucination: bool,
pub format_mode: String,
/// Optional explicit microphone device name (from `list_audio_devices`).
/// None or empty string = let `MicrophoneCapture::start` auto-select.
pub microphone_device: Option<String>,
}
#[derive(Debug, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct StartLiveTranscriptionResponse {
pub session_id: u64,
}
#[derive(Debug, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct StopLiveTranscriptionResponse {
pub session_id: u64,
pub audio_path: Option<String>,
pub dropped_audio_ms: u64,
}
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct LiveResultMessage {
pub session_id: u64,
pub chunk_id: u32,
pub chunk_start_secs: f64,
pub duration: f64,
pub language: String,
pub inference_ms: u64,
pub segments: Vec<Segment>,
}
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase", tag = "type")]
#[allow(dead_code)]
pub enum LiveStatusMessage {
Warning {
session_id: u64,
message: String,
},
Overload {
session_id: u64,
dropped_audio_ms: u64,
message: String,
},
Error {
session_id: u64,
message: String,
},
Finished {
session_id: u64,
audio_path: Option<String>,
dropped_audio_ms: u64,
},
}
struct LiveSessionSummary {
session_id: u64,
dropped_audio_ms: u64,
audio_samples: Option<Vec<f32>>,
}
struct InferenceTask {
chunk_id: u32,
chunk_start_sample: u64,
trim_before_secs: f64,
duration_secs: f64,
rx: std::sync::mpsc::Receiver<Result<kon_transcription::TimedTranscript, String>>,
}
#[tauri::command]
pub async fn start_live_transcription_session(
state: tauri::State<'_, AppState>,
live_state: tauri::State<'_, LiveTranscriptionState>,
config: StartLiveTranscriptionConfig,
result_channel: Channel<LiveResultMessage>,
status_channel: Channel<LiveStatusMessage>,
) -> Result<StartLiveTranscriptionResponse, String> {
{
let running = live_state.running.lock().unwrap();
if running.is_some() {
return Err("A live transcription session is already running".into());
}
}
let model_id = config
.model_id
.clone()
.unwrap_or_else(|| default_model_id_for_engine(&config.engine).to_string());
eprintln!(
"[live] starting session: engine={}, model={}, language={:?}, save_audio={}",
config.engine,
model_id,
config.language,
config.save_audio
);
ensure_model_loaded(&state, &config.engine, &model_id).await?;
let session_id = live_state
.next_session_id
.fetch_add(1, Ordering::Relaxed)
.saturating_add(1);
let stop_flag = Arc::new(AtomicBool::new(false));
let engine = pick_engine(&state, &config.engine)?;
let output_folder = config.output_folder.clone();
let worker_stop = stop_flag.clone();
let worker_status = status_channel.clone();
let worker_results = result_channel.clone();
let handle = tokio::task::spawn_blocking(move || {
run_live_session(
session_id,
engine,
config,
worker_results,
worker_status,
worker_stop,
)
});
*live_state.running.lock().unwrap() = Some(RunningLiveSession {
id: session_id,
output_folder,
stop_flag,
handle,
status_channel,
});
Ok(StartLiveTranscriptionResponse { session_id })
}
#[tauri::command]
pub async fn stop_live_transcription_session(
app: tauri::AppHandle,
live_state: tauri::State<'_, LiveTranscriptionState>,
session_id: u64,
) -> Result<StopLiveTranscriptionResponse, String> {
let running = live_state.running.lock().unwrap().take();
let Some(running) = running else {
return Err("No live transcription session is running".into());
};
if running.id != session_id {
*live_state.running.lock().unwrap() = Some(running);
return Err(format!("Session {session_id} is not active"));
}
running.stop_flag.store(true, Ordering::Relaxed);
let summary = running
.handle
.await
.map_err(|e| format!("Live session task failed: {e}"))??;
let audio_path = if let Some(samples) = summary.audio_samples {
Some(
persist_audio_samples(&app, samples, running.output_folder.clone())
.await?,
)
} else {
None
};
let response = StopLiveTranscriptionResponse {
session_id: summary.session_id,
audio_path: audio_path.clone(),
dropped_audio_ms: summary.dropped_audio_ms,
};
let _ = running.status_channel.send(LiveStatusMessage::Finished {
session_id: summary.session_id,
audio_path,
dropped_audio_ms: summary.dropped_audio_ms,
});
Ok(response)
}
fn pick_engine(
state: &AppState,
engine: &str,
) -> Result<Arc<LocalEngine>, String> {
match engine {
"whisper" => Ok(state.whisper_engine.clone()),
"parakeet" => Ok(state.parakeet_engine.clone()),
other => Err(format!("Unknown engine: {other}")),
}
}
fn run_live_session(
session_id: u64,
engine: Arc<LocalEngine>,
config: StartLiveTranscriptionConfig,
result_channel: Channel<LiveResultMessage>,
status_channel: Channel<LiveStatusMessage>,
stop_flag: Arc<AtomicBool>,
) -> Result<LiveSessionSummary, String> {
let (mut capture, rx) = match config.microphone_device.as_deref() {
Some(name) if !name.is_empty() => MicrophoneCapture::start_with_device(name),
_ => MicrophoneCapture::start(),
}
.map_err(|e| e.to_string())?;
// Drain runtime stream errors into the status channel so the user
// gets a toast when the device disconnects mid-recording instead of
// silently producing empty transcripts. The `_capture` binding keeps
// the cpal stream alive for the duration of the session.
let mic_error_rx = capture.take_error_rx();
let _capture = capture;
let mut resampler: Option<StreamingResampler> = None;
let mut capture_buffer: Vec<f32> = Vec::new();
let mut kept_audio = if config.save_audio {
Some(Vec::new())
} else {
None
};
let mut buffer_start_sample: u64 = 0;
let mut dropped_audio_ms: u64 = 0;
let mut chunk_id: u32 = 0;
let mut inflight: Option<InferenceTask> = None;
let mut resampler_flushed = false;
loop {
if let Some(_done) = poll_inference(
&mut inflight,
session_id,
&config,
&result_channel,
&status_channel,
)? {}
// Surface any cpal runtime errors as warnings. Non-fatal: a hard
// disconnect will also drop the audio sender and be caught by
// the `Disconnected` arm below. This lets the user see a toast
// even when cpal recovers without tearing the stream down.
if let Some(err_rx) = &mic_error_rx {
while let Ok(err) = err_rx.try_recv() {
let _ = status_channel.send(LiveStatusMessage::Warning {
session_id,
message: format!(
"Microphone '{}' reported an error: {}",
err.device_name, err.message
),
});
}
}
match rx.recv_timeout(Duration::from_millis(25)) {
Ok(chunk) => {
let mono = downmix_chunk(chunk.samples, chunk.channels as usize);
let resampler = match &mut resampler {
Some(resampler) => resampler,
None => {
resampler = Some(
StreamingResampler::new(chunk.sample_rate)
.map_err(|e| e.to_string())?,
);
resampler.as_mut().expect("resampler just set")
}
};
let resampled =
resampler.push_samples(&mono).map_err(|e| e.to_string())?;
append_resampled_audio(
&mut capture_buffer,
&mut kept_audio,
&resampled,
);
}
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {}
Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => {
let message =
"Microphone capture disconnected unexpectedly".to_string();
let _ = status_channel.send(LiveStatusMessage::Error {
session_id,
message: message.clone(),
});
return Err(message);
}
}
if inflight.is_some() && capture_buffer.len() > MAX_PENDING_SAMPLES {
let overflow = capture_buffer.len() - MAX_PENDING_SAMPLES;
capture_buffer.drain(..overflow);
buffer_start_sample = buffer_start_sample.saturating_add(overflow as u64);
dropped_audio_ms = dropped_audio_ms
.saturating_add((overflow as u64 * 1000) / WHISPER_SAMPLE_RATE as u64);
let _ = status_channel.send(LiveStatusMessage::Overload {
session_id,
dropped_audio_ms,
message: "Kon dropped older audio to keep live dictation responsive".into(),
});
}
let stopping = stop_flag.load(Ordering::Relaxed);
if stopping && !resampler_flushed {
if let Some(resampler) = &mut resampler {
let tail = resampler.flush().map_err(|e| e.to_string())?;
append_resampled_audio(&mut capture_buffer, &mut kept_audio, &tail);
}
resampler_flushed = true;
}
if inflight.is_none() {
if let Some(task) = maybe_dispatch_chunk(
&engine,
&config,
&mut capture_buffer,
&mut buffer_start_sample,
&mut chunk_id,
stopping,
&status_channel,
session_id,
) {
inflight = Some(task);
continue;
}
if stopping && resampler_flushed {
break;
}
}
}
while inflight.is_some() {
poll_inference(
&mut inflight,
session_id,
&config,
&result_channel,
&status_channel,
)?;
thread::sleep(Duration::from_millis(10));
}
Ok(LiveSessionSummary {
session_id,
dropped_audio_ms,
audio_samples: kept_audio,
})
}
fn append_resampled_audio(
capture_buffer: &mut Vec<f32>,
kept_audio: &mut Option<Vec<f32>>,
resampled: &[f32],
) {
if resampled.is_empty() {
return;
}
capture_buffer.extend_from_slice(resampled);
if let Some(kept_audio) = kept_audio {
kept_audio.extend_from_slice(resampled);
}
}
fn maybe_dispatch_chunk(
engine: &Arc<LocalEngine>,
config: &StartLiveTranscriptionConfig,
capture_buffer: &mut Vec<f32>,
buffer_start_sample: &mut u64,
chunk_id: &mut u32,
stopping: bool,
status_channel: &Channel<LiveStatusMessage>,
session_id: u64,
) -> Option<InferenceTask> {
let target_len = if capture_buffer.len() >= CHUNK_SAMPLES {
CHUNK_SAMPLES
} else if stopping && capture_buffer.len() >= FINAL_CHUNK_MIN_SAMPLES {
capture_buffer.len()
} else {
return None;
};
let trim_before_secs = if *chunk_id > 0 && !stopping && target_len > OVERLAP_SAMPLES {
OVERLAP_SAMPLES as f64 / WHISPER_SAMPLE_RATE as f64
} else {
0.0
};
let speech_window = if trim_before_secs > 0.0 {
&capture_buffer[OVERLAP_SAMPLES..target_len]
} else {
&capture_buffer[..target_len]
};
if !has_enough_speech(speech_window) {
let skipped_ms =
(target_len as u64 * 1000) / WHISPER_SAMPLE_RATE as u64;
eprintln!(
"[live] session {session_id}: skipped {skipped_ms}ms chunk as near-silence"
);
let _ = status_channel.send(LiveStatusMessage::Warning {
session_id,
message: format!(
"Skipped {skipped_ms}ms of near-silent audio. If this keeps happening, try a louder mic level or move closer to the microphone."
),
});
capture_buffer.drain(..target_len);
*buffer_start_sample = buffer_start_sample.saturating_add(target_len as u64);
return None;
}
*chunk_id = chunk_id.saturating_add(1);
let current_chunk_id = *chunk_id;
let chunk_start_sample = *buffer_start_sample;
let duration_secs = target_len as f64 / WHISPER_SAMPLE_RATE as f64;
let chunk_samples = capture_buffer[..target_len].to_vec();
eprintln!(
"[live] session {session_id}: dispatching chunk {} ({duration_secs:.2}s, {} samples)",
current_chunk_id,
chunk_samples.len()
);
let advance_by = if stopping || target_len < CHUNK_SAMPLES {
target_len
} else {
target_len.saturating_sub(OVERLAP_SAMPLES)
};
capture_buffer.drain(..advance_by);
*buffer_start_sample = buffer_start_sample.saturating_add(advance_by as u64);
let options = TranscriptionOptions {
language: config.language.clone(),
initial_prompt: config.initial_prompt.clone(),
};
let engine = engine.clone();
let (tx, rx) = std::sync::mpsc::channel();
thread::spawn(move || {
let audio = AudioSamples::mono_16khz(chunk_samples);
let started = Instant::now();
let result = engine
.transcribe_sync(&audio, &options)
.map(|mut timed| {
timed.inference_ms = started.elapsed().as_millis() as u64;
timed
})
.map_err(|e| e.to_string());
let _ = tx.send(result);
});
Some(InferenceTask {
chunk_id: current_chunk_id,
chunk_start_sample,
trim_before_secs,
duration_secs,
rx,
})
}
fn poll_inference(
inflight: &mut Option<InferenceTask>,
session_id: u64,
config: &StartLiveTranscriptionConfig,
result_channel: &Channel<LiveResultMessage>,
status_channel: &Channel<LiveStatusMessage>,
) -> Result<Option<bool>, String> {
let Some(task) = inflight else {
return Ok(None);
};
match task.rx.try_recv() {
Ok(Ok(timed)) => {
let mut segments: Vec<Segment> =
timed.transcript.segments().to_vec();
trim_overlap_segments(&mut segments, task.trim_before_secs);
post_process_segments(
&mut segments,
&PostProcessOptions {
remove_fillers: config.remove_fillers,
british_english: config.british_english,
anti_hallucination: config.anti_hallucination,
format_mode: FormatMode::parse(&config.format_mode),
},
);
let segment_count = segments.len();
result_channel
.send(LiveResultMessage {
session_id,
chunk_id: task.chunk_id,
chunk_start_secs: task.chunk_start_sample as f64
/ WHISPER_SAMPLE_RATE as f64,
duration: task.duration_secs,
language: timed.transcript.language().to_string(),
inference_ms: timed.inference_ms,
segments,
})
.map_err(|e| e.to_string())?;
eprintln!(
"[live] session {session_id}: delivered chunk {} with {} segments in {}ms",
task.chunk_id,
segment_count,
timed.inference_ms
);
*inflight = None;
Ok(Some(true))
}
Ok(Err(err)) => {
eprintln!("[live] session {session_id}: inference error: {err}");
*inflight = None;
let _ = status_channel.send(LiveStatusMessage::Error {
session_id,
message: err.clone(),
});
Err(err)
}
Err(std::sync::mpsc::TryRecvError::Empty) => Ok(Some(false)),
Err(std::sync::mpsc::TryRecvError::Disconnected) => {
*inflight = None;
let message = "Inference worker disconnected unexpectedly".to_string();
eprintln!("[live] session {session_id}: {message}");
let _ = status_channel.send(LiveStatusMessage::Error {
session_id,
message: message.clone(),
});
Err(message)
}
}
}
fn trim_overlap_segments(segments: &mut Vec<Segment>, trim_before_secs: f64) {
if trim_before_secs <= 0.0 {
return;
}
segments.retain(|segment| segment.end > trim_before_secs);
for segment in segments.iter_mut() {
if segment.start < trim_before_secs {
segment.start = trim_before_secs;
}
}
}
fn has_enough_speech(samples: &[f32]) -> bool {
if samples.is_empty() {
return false;
}
let chunk_peak = samples
.iter()
.map(|sample| sample.abs())
.fold(0.0_f32, f32::max);
if chunk_peak < FLATLINE_PEAK_THRESHOLD {
return false;
}
let mut speech_frames = 0usize;
for frame in samples.chunks(SPEECH_FRAME_SAMPLES) {
let len = frame.len().max(1) as f32;
let rms = (frame.iter().map(|sample| sample * sample).sum::<f32>() / len)
.sqrt();
let peak = frame
.iter()
.map(|sample| sample.abs())
.fold(0.0_f32, f32::max);
if rms >= RMS_SPEECH_THRESHOLD || peak >= PEAK_SPEECH_THRESHOLD {
speech_frames += 1;
}
}
speech_frames >= MIN_SPEECH_FRAMES
}
fn downmix_chunk(samples: Vec<f32>, channels: usize) -> Vec<f32> {
if channels <= 1 {
return samples;
}
samples
.chunks(channels)
.map(|frame| frame.iter().sum::<f32>() / channels as f32)
.collect()
}

View File

@@ -1,7 +1,10 @@
pub mod audio;
pub mod clipboard;
pub mod diagnostics;
pub mod hardware;
pub mod hotkey;
pub mod live;
pub mod models;
pub mod transcription;
pub mod transcripts;
pub mod windows;

View File

@@ -1,8 +1,15 @@
use std::sync::Arc;
use serde::Serialize;
use tauri::Emitter;
use crate::AppState;
use kon_core::model_registry::{
self, Engine, LanguageSupport, ModelEntry,
};
use kon_core::types::ModelId;
use kon_transcription::model_manager;
use kon_transcription::{load_parakeet, load_whisper, LocalEngine};
/// Map legacy size strings to ModelId.
fn whisper_model_id(size: &str) -> ModelId {
@@ -22,6 +29,212 @@ fn parakeet_model_id(name: &str) -> ModelId {
}
}
fn engine_for_name(
state: &AppState,
engine_name: &str,
) -> Result<Arc<LocalEngine>, String> {
match engine_name {
"whisper" => Ok(state.whisper_engine.clone()),
"parakeet" => Ok(state.parakeet_engine.clone()),
other => Err(format!("Unknown engine: {other}")),
}
}
fn language_support_info(
language_support: LanguageSupport,
) -> LanguageSupportInfo {
match language_support {
LanguageSupport::EnglishOnly => LanguageSupportInfo {
kind: "english-only".into(),
language_count: 1,
},
LanguageSupport::Multilingual(count) => LanguageSupportInfo {
kind: "multilingual".into(),
language_count: count,
},
}
}
fn model_capability(
entry: &'static ModelEntry,
engine: &Arc<LocalEngine>,
) -> ModelRuntimeCapabilities {
let loaded_model_id = engine.loaded_model_id();
ModelRuntimeCapabilities {
id: entry.id.to_string(),
display_name: entry.display_name.to_string(),
downloaded: model_manager::is_downloaded(&entry.id),
loaded: loaded_model_id
.as_ref()
.map(|id| id == &entry.id)
.unwrap_or(false),
language_support: language_support_info(entry.languages),
}
}
fn load_model_from_disk(model_id: &ModelId) -> Result<Box<dyn kon_transcription::SpeechModel + Send>, String> {
let entry = model_registry::find_model(model_id)
.ok_or_else(|| format!("Unknown model: {model_id}"))?;
match entry.engine {
Engine::Whisper => {
let dir = model_manager::model_dir(model_id);
let model_file = entry
.files
.first()
.map(|file| dir.join(file.filename))
.ok_or_else(|| format!("No files registered for model: {model_id}"))?;
if !model_file.exists() {
return Err(format!("Model not downloaded: {model_id}"));
}
load_whisper(&model_file).map_err(|e| e.to_string())
}
Engine::Parakeet => {
let dir = model_manager::model_dir(model_id);
if !dir.exists() {
return Err(format!("Model not downloaded: {model_id}"));
}
load_parakeet(&dir).map_err(|e| e.to_string())
}
Engine::Moonshine => Err("Moonshine models are not yet supported in this build".into()),
}
}
pub fn default_model_id_for_engine(engine: &str) -> ModelId {
match engine {
"parakeet" => ModelId::new("parakeet-ctc-0.6b-int8"),
_ => ModelId::new("whisper-base-en"),
}
}
pub async fn ensure_model_loaded(
state: &AppState,
engine_name: &str,
model_id: &str,
) -> Result<(), String> {
let model_id = ModelId::new(model_id);
let entry = model_registry::find_model(&model_id)
.ok_or_else(|| format!("Unknown model: {model_id}"))?;
let expected_engine = match entry.engine {
Engine::Whisper => "whisper",
Engine::Parakeet => "parakeet",
Engine::Moonshine => "moonshine",
};
if expected_engine != engine_name {
return Err(format!(
"Model {} belongs to {}, not {}",
model_id, expected_engine, engine_name
));
}
if !model_manager::is_downloaded(&model_id) {
return Err(format!("Model not downloaded: {model_id}"));
}
let engine = engine_for_name(state, engine_name)?;
if engine
.loaded_model_id()
.as_ref()
.map(|loaded| loaded == &model_id)
.unwrap_or(false)
{
return Ok(());
}
let engine_clone = engine.clone();
let model_id_clone = model_id.clone();
tokio::task::spawn_blocking(move || {
let model = load_model_from_disk(&model_id_clone)?;
engine_clone.load(model, model_id_clone);
Ok::<_, String>(())
})
.await
.map_err(|e| e.to_string())??;
Ok(())
}
#[derive(Serialize)]
#[serde(rename_all = "camelCase")]
pub struct RuntimeCapabilities {
pub accelerators: Vec<String>,
pub engines: Vec<EngineRuntimeCapabilities>,
}
#[derive(Serialize)]
#[serde(rename_all = "camelCase")]
pub struct EngineRuntimeCapabilities {
pub id: String,
pub default_model_id: String,
pub loaded_model_id: Option<String>,
pub supports_gpu: bool,
pub models: Vec<ModelRuntimeCapabilities>,
}
#[derive(Serialize)]
#[serde(rename_all = "camelCase")]
pub struct ModelRuntimeCapabilities {
pub id: String,
pub display_name: String,
pub downloaded: bool,
pub loaded: bool,
pub language_support: LanguageSupportInfo,
}
#[derive(Serialize)]
#[serde(rename_all = "camelCase")]
pub struct LanguageSupportInfo {
pub kind: String,
pub language_count: u16,
}
#[tauri::command]
pub fn get_runtime_capabilities(
state: tauri::State<'_, AppState>,
) -> Result<RuntimeCapabilities, String> {
let whisper = state.whisper_engine.clone();
let parakeet = state.parakeet_engine.clone();
let whisper_models = model_registry::all_models()
.iter()
.filter(|entry| entry.engine == Engine::Whisper)
.map(|entry| model_capability(entry, &whisper))
.collect();
let parakeet_models = model_registry::all_models()
.iter()
.filter(|entry| entry.engine == Engine::Parakeet)
.map(|entry| model_capability(entry, &parakeet))
.collect();
Ok(RuntimeCapabilities {
// Current desktop build ships CPU-only inference backends.
accelerators: vec!["cpu".into()],
engines: vec![
EngineRuntimeCapabilities {
id: "whisper".into(),
default_model_id: default_model_id_for_engine("whisper")
.to_string(),
loaded_model_id: whisper
.loaded_model_id()
.map(|model_id| model_id.to_string()),
supports_gpu: false,
models: whisper_models,
},
EngineRuntimeCapabilities {
id: "parakeet".into(),
default_model_id: default_model_id_for_engine("parakeet")
.to_string(),
loaded_model_id: parakeet
.loaded_model_id()
.map(|model_id| model_id.to_string()),
supports_gpu: false,
models: parakeet_models,
},
],
})
}
// --- Whisper model commands ---
#[tauri::command]
@@ -51,14 +264,12 @@ pub fn list_models() -> Result<Vec<String>, String> {
Ok(ids
.into_iter()
.filter(|id| id.as_str().starts_with("whisper-"))
.map(|id| {
match id.as_str() {
"whisper-tiny-en" => "Tiny".to_string(),
"whisper-base-en" => "Base".to_string(),
"whisper-small-en" => "Small".to_string(),
"whisper-medium-en" => "Medium".to_string(),
other => other.to_string(),
}
.map(|id| match id.as_str() {
"whisper-tiny-en" => "Tiny".to_string(),
"whisper-base-en" => "Base".to_string(),
"whisper-small-en" => "Small".to_string(),
"whisper-medium-en" => "Medium".to_string(),
other => other.to_string(),
})
.collect())
}
@@ -69,31 +280,7 @@ pub async fn load_model(
size: String,
) -> Result<String, String> {
let id = whisper_model_id(&size);
let dir = model_manager::model_dir(&id);
let model_file = dir.join(
kon_core::model_registry::find_model(&id)
.ok_or_else(|| format!("Unknown model: {size}"))?
.files
.first()
.ok_or_else(|| format!("No files for model: {size}"))?
.filename,
);
if !model_file.exists() {
return Err(format!("Model not downloaded: {size}"));
}
let engine = state.whisper_engine.clone();
let mid = id.clone();
tokio::task::spawn_blocking(move || {
let model = kon_transcription::load_whisper(&model_file)
.map_err(|e| e.to_string())?;
engine.load(model, mid);
Ok::<_, String>(())
})
.await
.map_err(|e| e.to_string())??;
ensure_model_loaded(&state, "whisper", id.as_str()).await?;
Ok(format!("Model {} loaded", size))
}
@@ -131,11 +318,9 @@ pub fn list_parakeet_models() -> Result<Vec<String>, String> {
Ok(ids
.into_iter()
.filter(|id| id.as_str().starts_with("parakeet-"))
.map(|id| {
match id.as_str() {
"parakeet-ctc-0.6b-int8" => "ctc-int8".to_string(),
other => other.to_string(),
}
.map(|id| match id.as_str() {
"parakeet-ctc-0.6b-int8" => "ctc-int8".to_string(),
other => other.to_string(),
})
.collect())
}
@@ -146,29 +331,13 @@ pub async fn load_parakeet_model(
name: String,
) -> Result<String, String> {
let id = parakeet_model_id(&name);
let dir = model_manager::model_dir(&id);
if !dir.exists() {
return Err(format!("Parakeet model not downloaded: {name}"));
}
let engine = state.parakeet_engine.clone();
let mid = id.clone();
tokio::task::spawn_blocking(move || {
let model = kon_transcription::load_parakeet(&dir)
.map_err(|e| e.to_string())?;
engine.load(model, mid);
Ok::<_, String>(())
})
.await
.map_err(|e| e.to_string())??;
ensure_model_loaded(&state, "parakeet", id.as_str()).await?;
Ok(format!("Parakeet model {} loaded", name))
}
#[tauri::command]
pub fn check_parakeet_engine(
state: tauri::State<AppState>,
state: tauri::State<'_, AppState>,
) -> Result<bool, String> {
Ok(state.parakeet_engine.is_loaded())
}

View File

@@ -3,13 +3,26 @@
#![allow(clippy::too_many_arguments)]
use std::path::Path;
use std::sync::Arc;
use tauri::Emitter;
use crate::commands::models::{default_model_id_for_engine, ensure_model_loaded};
use crate::AppState;
use kon_ai_formatting::{post_process_segments, FormatMode, PostProcessOptions};
use kon_core::types::{Segment, TranscriptionOptions};
fn pick_engine(
state: &AppState,
engine: &str,
) -> Result<Arc<kon_transcription::LocalEngine>, String> {
match engine {
"whisper" => Ok(state.whisper_engine.clone()),
"parakeet" => Ok(state.parakeet_engine.clone()),
other => Err(format!("Unknown engine: {other}")),
}
}
/// Transcribe raw PCM f32 samples (Whisper). Emits "transcription-result" event.
#[tauri::command]
pub async fn transcribe_pcm(
@@ -69,6 +82,8 @@ pub async fn transcribe_pcm(
pub async fn transcribe_file(
state: tauri::State<'_, AppState>,
path: String,
engine: Option<String>,
model_id: Option<String>,
language: String,
initial_prompt: String,
remove_fillers: bool,
@@ -76,7 +91,12 @@ pub async fn transcribe_file(
anti_hallucination: bool,
format_mode: String,
) -> Result<serde_json::Value, String> {
let engine = state.whisper_engine.clone();
let engine_name = engine.unwrap_or_else(|| "whisper".to_string());
let model_id = model_id
.unwrap_or_else(|| default_model_id_for_engine(&engine_name).to_string());
ensure_model_loaded(&state, &engine_name, &model_id).await?;
let engine = pick_engine(&state, &engine_name)?;
let options = TranscriptionOptions {
language: Some(language),
initial_prompt: Some(initial_prompt),
@@ -106,6 +126,8 @@ pub async fn transcribe_file(
);
Ok(serde_json::json!({
"engine": engine_name,
"modelId": model_id,
"segments": segments,
"language": timed.transcript.language(),
"duration": timed.transcript.duration(),

View File

@@ -0,0 +1,235 @@
// Tauri commands wrapping the kon_storage transcript and dictionary CRUD.
// These are the bridge that lets the Svelte frontend treat SQLite as the
// canonical store rather than localStorage.
//
// Day 4 of the upgrade plan. The frontend HistoryPage rename flow has had
// a `// TODO: persist to SQLite when update_transcript exists` for some
// time; that command now exists.
use serde::{Deserialize, Serialize};
use kon_storage::{
add_dictionary_entry, count_transcripts, delete_dictionary_entry,
delete_transcript as db_delete_transcript, get_transcript as db_get_transcript,
insert_transcript as db_insert_transcript, list_dictionary,
list_transcripts_paged, search_transcripts as db_search_transcripts,
update_transcript as db_update_transcript, DictionaryEntry, InsertTranscriptParams,
TranscriptRow,
};
use crate::AppState;
/// Frontend-facing shape of a stored transcript. Mirrors `TranscriptRow`
/// from the storage crate but drops fields the UI does not need yet
/// (sample_rate, audio_channels, format flags) and renames to camelCase.
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct TranscriptDto {
pub id: String,
pub text: String,
pub source: String,
pub title: Option<String>,
pub audio_path: Option<String>,
pub duration: f64,
pub engine: Option<String>,
pub model_id: Option<String>,
pub created_at: String,
}
impl From<TranscriptRow> for TranscriptDto {
fn from(r: TranscriptRow) -> Self {
Self {
id: r.id,
text: r.text,
source: r.source,
title: r.title,
audio_path: r.audio_path,
duration: r.duration,
engine: r.engine,
model_id: r.model_id,
created_at: r.created_at,
}
}
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct CreateTranscriptRequest {
pub id: String,
pub text: String,
pub source: String,
pub title: Option<String>,
pub audio_path: Option<String>,
pub duration: f64,
pub engine: Option<String>,
pub model_id: Option<String>,
pub inference_ms: Option<i64>,
pub sample_rate: Option<i64>,
pub audio_channels: Option<i64>,
pub format_mode: Option<String>,
pub remove_fillers: bool,
pub british_english: bool,
pub anti_hallucination: bool,
}
/// Insert a transcript into the canonical SQLite store. Called from the
/// frontend after a recording or file transcription completes. The FTS5
/// index is updated automatically by trigger.
#[tauri::command]
pub async fn add_transcript(
state: tauri::State<'_, AppState>,
transcript: CreateTranscriptRequest,
) -> Result<(), String> {
let params = InsertTranscriptParams {
id: &transcript.id,
text: &transcript.text,
source: &transcript.source,
title: transcript.title.as_deref(),
audio_path: transcript.audio_path.as_deref(),
duration: transcript.duration,
engine: transcript.engine.as_deref(),
model_id: transcript.model_id.as_deref(),
inference_ms: transcript.inference_ms,
sample_rate: transcript.sample_rate,
audio_channels: transcript.audio_channels,
format_mode: transcript.format_mode.as_deref(),
remove_fillers: transcript.remove_fillers,
british_english: transcript.british_english,
anti_hallucination: transcript.anti_hallucination,
};
db_insert_transcript(&state.db, &params)
.await
.map_err(|e| e.to_string())
}
/// Paginated history list. Defaults to 50 rows from the most recent.
#[tauri::command]
pub async fn list_transcripts(
state: tauri::State<'_, AppState>,
limit: Option<i64>,
offset: Option<i64>,
) -> Result<Vec<TranscriptDto>, String> {
let limit = limit.unwrap_or(50).clamp(1, 500);
let offset = offset.unwrap_or(0).max(0);
list_transcripts_paged(&state.db, limit, offset)
.await
.map(|rows| rows.into_iter().map(TranscriptDto::from).collect())
.map_err(|e| e.to_string())
}
/// Total count of transcripts (for "showing X of N" UI).
#[tauri::command]
pub async fn count_transcripts_command(
state: tauri::State<'_, AppState>,
) -> Result<i64, String> {
count_transcripts(&state.db).await.map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn get_transcript(
state: tauri::State<'_, AppState>,
id: String,
) -> Result<Option<TranscriptDto>, String> {
db_get_transcript(&state.db, &id)
.await
.map(|opt| opt.map(TranscriptDto::from))
.map_err(|e| e.to_string())
}
/// Update mutable fields of an existing transcript. `text` and/or `title`.
/// Returns the row count affected (0 if id not found). Fixes the long-
/// standing "rename in History never persists" bug per
/// architecture-review.md §13.
#[tauri::command]
pub async fn update_transcript(
state: tauri::State<'_, AppState>,
id: String,
text: Option<String>,
title: Option<String>,
) -> Result<u64, String> {
db_update_transcript(&state.db, &id, text.as_deref(), title.as_deref())
.await
.map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn delete_transcript(
state: tauri::State<'_, AppState>,
id: String,
) -> Result<(), String> {
db_delete_transcript(&state.db, &id)
.await
.map_err(|e| e.to_string())
}
/// FTS5 search. Query syntax: bare words AND together; quote phrases;
/// supports `OR`, `NOT`, prefix `*`. Returns up to 50 best-rank matches.
#[tauri::command]
pub async fn search_transcripts(
state: tauri::State<'_, AppState>,
query: String,
) -> Result<Vec<TranscriptDto>, String> {
let q = query.trim();
if q.is_empty() {
return Ok(Vec::new());
}
db_search_transcripts(&state.db, q, 50)
.await
.map(|rows| rows.into_iter().map(TranscriptDto::from).collect())
.map_err(|e| e.to_string())
}
// --- Dictionary commands (custom vocabulary) ---
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct DictionaryDto {
pub id: i64,
pub term: String,
pub note: Option<String>,
}
impl From<DictionaryEntry> for DictionaryDto {
fn from(e: DictionaryEntry) -> Self {
Self {
id: e.id,
term: e.term,
note: e.note,
}
}
}
#[tauri::command]
pub async fn list_dictionary_command(
state: tauri::State<'_, AppState>,
) -> Result<Vec<DictionaryDto>, String> {
list_dictionary(&state.db)
.await
.map(|rows| rows.into_iter().map(DictionaryDto::from).collect())
.map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn add_dictionary_entry_command(
state: tauri::State<'_, AppState>,
term: String,
note: Option<String>,
) -> Result<i64, String> {
let term = term.trim();
if term.is_empty() {
return Err("Dictionary term cannot be empty".into());
}
add_dictionary_entry(&state.db, term, note.as_deref())
.await
.map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn delete_dictionary_entry_command(
state: tauri::State<'_, AppState>,
id: i64,
) -> Result<(), String> {
delete_dictionary_entry(&state.db, id)
.await
.map_err(|e| e.to_string())
}

View File

@@ -2,6 +2,7 @@ mod commands;
mod tray;
use std::sync::Arc;
use std::time::Instant;
use sqlx::SqlitePool;
use tauri::Manager;
@@ -47,6 +48,7 @@ fn build_preferences_script(prefs_json: Option<String>) -> String {
if (a.fontSize) el.style.setProperty('--font-size-body', a.fontSize + 'px');
if (a.letterSpacing != null) el.style.setProperty('--letter-spacing-body', a.letterSpacing + 'em');
if (a.lineHeight) el.style.setProperty('--line-height-body', String(a.lineHeight));
if (a.transcriptSize) el.style.setProperty('--text-transcript', a.transcriptSize + 'px');
if (a.bionicReading) el.dataset.bionicReading = 'true';
if (a.reduceMotion === 'on' || (a.reduceMotion === 'system' && window.matchMedia('(prefers-reduced-motion: reduce)').matches)) el.dataset.reduceMotion = 'true';
}}
@@ -66,8 +68,57 @@ async fn save_preferences(
.map_err(|e| e.to_string())
}
/// On Linux Wayland sessions (KDE, GNOME) WebKitGTK + DMABUF rendering hits
/// known crashes that the HANDOVER documents working around with a manual
/// env-var prefix:
///
/// env GDK_BACKEND=x11 WINIT_UNIX_BACKEND=x11 \
/// WEBKIT_DISABLE_DMABUF_RENDERER=1 npm run tauri dev
///
/// Detect the Wayland session at startup and apply the env vars before
/// anything else loads, so users do not need to remember the prefix and
/// the dev launcher / packaged app both work out of the box.
///
/// Inspired by Open-Whispr's similar Chromium self-relaunch (with
/// --ozone-platform=x11). Day 6 of the upgrade plan.
#[cfg(target_os = "linux")]
fn ensure_x11_on_wayland() {
// If the user explicitly opted in to Wayland (XDG_SESSION_TYPE set by
// their session, KDE / GNOME compositor flags), force WebKitGTK + GDK
// onto X11 via XWayland. Idempotent: if a value is already set, keep
// it (the user knows best).
let session_type = std::env::var("XDG_SESSION_TYPE").unwrap_or_default();
let is_wayland = session_type.eq_ignore_ascii_case("wayland");
if !is_wayland {
return;
}
let set_if_unset = |key: &str, value: &str| {
if std::env::var_os(key).is_none() {
// SAFETY: setting env vars before any threads spawn (we are
// pre-Tauri-Builder here). This block is the only place these
// are written.
unsafe { std::env::set_var(key, value); }
eprintln!("[startup] Wayland workaround: {key}={value}");
}
};
set_if_unset("GDK_BACKEND", "x11");
set_if_unset("WINIT_UNIX_BACKEND", "x11");
set_if_unset("WEBKIT_DISABLE_DMABUF_RENDERER", "1");
// PipeWire screen-capture portal still works fine through XWayland;
// we only redirect the GUI rendering path.
}
#[cfg_attr(mobile, tauri::mobile_entry_point)]
pub fn run() {
#[cfg(target_os = "linux")]
ensure_x11_on_wayland();
// Capture Rust panics to disk so the diagnostic-report bundler in
// Settings → About can attach them. Local only; nothing transmitted.
commands::diagnostics::install_panic_hook();
tauri::Builder::default()
.plugin(tauri_plugin_opener::init())
.plugin(tauri_plugin_dialog::init())
@@ -75,15 +126,19 @@ pub fn run() {
.setup(|app| {
// Initialise database (blocking in setup — runs once at startup)
let db_path = database_path();
let t0 = Instant::now();
let db = tauri::async_runtime::block_on(async {
init_db(&db_path).await
})
.map_err(|e| Box::new(e) as Box<dyn std::error::Error>)?;
eprintln!("[startup] DB init: {:?}", t0.elapsed());
// Load saved preferences for webview injection
let t1 = Instant::now();
let prefs_json = tauri::async_runtime::block_on(async {
get_setting(&db, "kon_preferences").await.unwrap_or(None)
});
eprintln!("[startup] Preferences load: {:?}", t1.elapsed());
let init_script = build_preferences_script(prefs_json);
// Apply preferences to the main window (defined in tauri.conf.json)
@@ -119,7 +174,18 @@ pub fn run() {
request.allow();
true
});
}).expect("Failed to configure webview media permissions");
})
.unwrap_or_else(|e| {
// Non-fatal: WebKitGTK may already have media
// capture wired by some compositors, or the
// signal binding may fail on unusual builds.
// Falling back means getUserMedia() prompts (or
// silently denies) instead of auto-granting,
// which is degraded but recoverable.
eprintln!(
"[startup] failed to configure webview media permissions: {e}",
);
});
}
// Close-to-tray: hide window instead of exiting
@@ -136,6 +202,8 @@ pub fn run() {
app.manage(PreferencesScript(init_script));
app.manage(commands::hotkey::HotkeyState::new());
app.manage(commands::audio::NativeCaptureState::new());
app.manage(commands::live::LiveTranscriptionState::default());
app.manage(AppState {
whisper_engine: Arc::new(LocalEngine::new(
@@ -162,6 +230,7 @@ pub fn run() {
commands::models::list_models,
commands::models::load_model,
commands::models::check_engine,
commands::models::get_runtime_capabilities,
// Parakeet model management
commands::models::download_parakeet_model,
commands::models::check_parakeet_model,
@@ -174,6 +243,29 @@ pub fn run() {
commands::transcription::transcribe_pcm_parakeet,
// Audio
commands::audio::save_audio,
commands::audio::start_native_capture,
commands::audio::stop_native_capture,
commands::audio::list_audio_devices,
// Transcripts (canonical SQLite-backed history) — Day 4
commands::transcripts::add_transcript,
commands::transcripts::list_transcripts,
commands::transcripts::count_transcripts_command,
commands::transcripts::get_transcript,
commands::transcripts::update_transcript,
commands::transcripts::delete_transcript,
commands::transcripts::search_transcripts,
commands::transcripts::list_dictionary_command,
commands::transcripts::add_dictionary_entry_command,
commands::transcripts::delete_dictionary_entry_command,
// Diagnostics (panic + error capture, manual report bundle)
commands::diagnostics::log_frontend_error,
commands::diagnostics::list_recent_errors_command,
commands::diagnostics::list_crash_files,
commands::diagnostics::generate_diagnostic_report,
commands::diagnostics::save_diagnostic_report,
commands::diagnostics::get_os_info,
commands::live::start_live_transcription_session,
commands::live::stop_live_transcription_session,
// Windows
commands::windows::open_task_window,
commands::windows::open_viewer_window,

View File

@@ -0,0 +1,129 @@
<script>
// Renders the toast stack in the bottom-right of the viewport. Mount once
// at the app root (in +layout.svelte). Reads from the global toasts store.
//
// Severity colour mapping uses the brand palette via CSS variables already
// defined in app.css — no inline literals.
//
// Honours prefers-reduced-motion via the existing :root[data-reduce-motion]
// attribute that preferences.svelte.js sets.
import { toasts } from '$lib/stores/toasts.svelte.js';
import { X } from 'lucide-svelte';
function severityClass(severity) {
switch (severity) {
case 'success': return 'toast-success';
case 'info': return 'toast-info';
case 'warn': return 'toast-warn';
case 'error': return 'toast-error';
default: return 'toast-info';
}
}
</script>
<div class="toast-viewport" aria-live="polite" aria-atomic="false">
{#each toasts.items as toast (toast.id)}
<div
class="toast {severityClass(toast.severity)}"
role={toast.severity === 'error' ? 'alert' : 'status'}
>
<div class="toast-body">
<div class="toast-title">{toast.title}</div>
{#if toast.body}
<div class="toast-msg">{toast.body}</div>
{/if}
</div>
<button
class="toast-close"
aria-label="Dismiss"
onclick={() => toasts.dismiss(toast.id)}
type="button"
>
<X size={14} aria-hidden="true" />
</button>
</div>
{/each}
</div>
<style>
.toast-viewport {
position: fixed;
bottom: 1rem;
right: 1rem;
display: flex;
flex-direction: column;
gap: 0.5rem;
z-index: 9999;
max-width: min(28rem, calc(100vw - 2rem));
pointer-events: none;
}
.toast {
pointer-events: auto;
display: flex;
align-items: flex-start;
gap: 0.75rem;
padding: 0.875rem 1rem;
border-radius: 0.5rem;
background: var(--surface, #1a1a1a);
color: var(--text, #fff);
border-left: 3px solid var(--moss, #4a7a4a);
box-shadow: 0 8px 24px rgba(0, 0, 0, 0.25);
/* Slide in from right; honours reduce-motion via the existing class
that preferences.svelte.js applies to <html>. */
animation: toast-slide-in 200ms cubic-bezier(0.2, 0.8, 0.2, 1);
}
:global(html.reduce-motion) .toast { animation: none; }
@keyframes toast-slide-in {
from { transform: translateX(100%); opacity: 0; }
to { transform: translateX(0); opacity: 1; }
}
.toast-info { border-left-color: var(--moss, #4a7a4a); }
.toast-success { border-left-color: var(--moss, #4a7a4a); }
.toast-warn { border-left-color: var(--signal, #d4a017); }
.toast-error { border-left-color: var(--ember, #c87144); }
.toast-body {
flex: 1;
min-width: 0;
}
.toast-title {
font-weight: 600;
font-size: 0.9375rem;
line-height: 1.4;
word-wrap: break-word;
}
.toast-msg {
margin-top: 0.25rem;
font-size: 0.875rem;
line-height: 1.5;
color: var(--text-muted, rgba(255, 255, 255, 0.7));
word-wrap: break-word;
}
.toast-close {
flex-shrink: 0;
background: transparent;
border: none;
color: var(--text-muted, rgba(255, 255, 255, 0.6));
cursor: pointer;
padding: 0.25rem;
border-radius: 0.25rem;
display: flex;
align-items: center;
justify-content: center;
transition: background 100ms, color 100ms;
}
.toast-close:hover {
background: rgba(255, 255, 255, 0.08);
color: var(--text, #fff);
}
.toast-close:focus-visible {
outline: 2px solid var(--ember, #c87144);
outline-offset: 2px;
}
</style>

View File

@@ -0,0 +1,217 @@
<script>
import { measureTextHeight } from "$lib/utils/textMeasure.js";
import { buildCumulativeOffsets, findVisibleRange } from "$lib/utils/virtualList.js";
import { getPreferences } from "$lib/stores/preferences.svelte.js";
import { bodyPretextLineHeight, pretextFontShorthand } from "$lib/utils/accessibilityTypography.js";
// Props
let {
segments = [],
activeSegmentIdx = -1,
matchingSegments = new Set(),
showTimestamps = true,
editingIdx = -1,
editingText = $bindable(""),
searchQuery = "",
formatTime = (t) => String(t),
onSeekToTime = () => {},
onStartEditing = () => {},
onFinishEditing = () => {},
onToggleStar = () => {},
onCopySegment = () => {},
onDeleteSegment = () => {},
highlightText = (t) => t,
} = $props();
// Internal state
let containerEl = $state(null);
let containerHeight = $state(0);
let containerWidth = $state(0);
let scrollTop = $state(0);
let hasSelection = $state(false);
const prefs = getPreferences();
// Segment height measurement
const SEGMENT_PADDING = 20; // py-2 (16px) + gap-1 (4px)
const BUFFER_COUNT = 5;
const SELECTION_BUFFER = 10;
let textFont = $derived(pretextFontShorthand(prefs.accessibility, 13));
let textLineHeight = $derived(bodyPretextLineHeight(prefs.accessibility, 13));
// Pre-calculate heights for all segments
let segmentHeights = $derived.by(() => {
if (!segments.length || containerWidth <= 0) return [];
// Available width for text: container - star (26px) - timestamp (48px if shown) - padding (24px) - actions (0, hidden)
const timestampWidth = showTimestamps ? 48 : 0;
const textWidth = containerWidth - 26 - timestampWidth - 24;
if (textWidth <= 0) return segments.map(() => 40);
return segments.map(seg => {
const text = seg.text?.trim() || "";
if (!text) return SEGMENT_PADDING + textLineHeight;
const { height } = measureTextHeight(text, textFont, textWidth, textLineHeight);
return height + SEGMENT_PADDING;
});
});
// Cumulative offsets for O(1) lookup
let cumulativeOffsets = $derived(buildCumulativeOffsets(segmentHeights));
let totalHeight = $derived(
cumulativeOffsets.length > 0 ? cumulativeOffsets[cumulativeOffsets.length - 1] : 0
);
// Determine visible range
let visibleRange = $derived.by(() => {
const buffer = hasSelection ? SELECTION_BUFFER : BUFFER_COUNT;
return findVisibleRange(
cumulativeOffsets,
segments.length,
scrollTop,
containerHeight,
buffer,
);
});
// Visible segment slice with positioning
let visibleItems = $derived.by(() => {
const items = [];
for (let i = visibleRange.start; i < visibleRange.end; i++) {
items.push({
seg: segments[i],
idx: segments[i]._idx,
top: cumulativeOffsets[i],
height: segmentHeights[i],
});
}
return items;
});
// Scroll to a specific segment index (used for active segment tracking + search)
export function scrollToSegment(idx) {
if (!containerEl || idx < 0 || idx >= segments.length) return;
const segTop = cumulativeOffsets[idx];
const segHeight = segmentHeights[idx];
const viewCenter = containerHeight / 2;
containerEl.scrollTop = segTop - viewCenter + segHeight / 2;
}
function onScroll(e) {
scrollTop = e.target.scrollTop;
// Track selection state for expanded buffer
hasSelection = !document.getSelection()?.isCollapsed;
}
// ResizeObserver for container dimensions
$effect(() => {
if (!containerEl) return;
const ro = new ResizeObserver(entries => {
for (const entry of entries) {
containerHeight = entry.contentRect.height;
containerWidth = entry.contentRect.width;
}
});
ro.observe(containerEl);
return () => ro.disconnect();
});
// Auto-scroll to active segment during playback
$effect(() => {
if (activeSegmentIdx >= 0) {
// Find the index in our segments array that matches
const arrayIdx = segments.findIndex(s => s._idx === activeSegmentIdx);
if (arrayIdx >= 0) {
const segTop = cumulativeOffsets[arrayIdx];
const segBottom = segTop + segmentHeights[arrayIdx];
// Only scroll if the active segment is out of view
if (segTop < scrollTop || segBottom > scrollTop + containerHeight) {
scrollToSegment(arrayIdx);
}
}
}
});
</script>
<div
bind:this={containerEl}
class="flex-1 overflow-y-auto px-5 py-3 min-h-0"
onscroll={onScroll}
>
<div class="relative" style="height: {totalHeight}px">
{#each visibleItems as { seg, idx, top, height } (idx)}
<div
class="absolute left-0 right-0 group flex gap-3 px-3 py-2 rounded-lg cursor-pointer transition-colors
{activeSegmentIdx === idx
? 'bg-accent/10 border-l-2 border-accent'
: matchingSegments.has(idx)
? 'bg-warning/5 border-l-2 border-warning/40'
: 'border-l-2 border-transparent hover:bg-hover'}"
style="top: {top}px"
onclick={() => onSeekToTime(seg.start)}
ondblclick={() => onStartEditing(idx)}
role="button"
tabindex="0"
onkeydown={(e) => { if (e.key === "Enter" && editingIdx !== idx) onSeekToTime(seg.start); }}
>
<!-- Star -->
<button
class="mt-0.5 flex-shrink-0 {seg.starred ? 'text-accent' : 'text-text-tertiary opacity-0 group-hover:opacity-100'}"
onclick={(e) => { e.stopPropagation(); onToggleStar(idx); }}
aria-label={seg.starred ? "Unstar" : "Star"}
>
<svg class="w-3.5 h-3.5" viewBox="0 0 24 24" fill={seg.starred ? "currentColor" : "none"} stroke="currentColor" stroke-width="2">
<path d="M12 2l3.09 6.26L22 9.27l-5 4.87 1.18 6.88L12 17.77l-6.18 3.25L7 14.14 2 9.27l6.91-1.01L12 2z" />
</svg>
</button>
<!-- Timestamp -->
{#if showTimestamps}
<span class="text-[10px] text-text-tertiary tabular-nums flex-shrink-0 mt-0.5 min-w-[36px]">
{formatTime(seg.start)}
</span>
{/if}
<!-- Text (editable or display) -->
{#if editingIdx === idx}
<textarea
class="flex-1 bg-bg-input border border-accent rounded-lg px-2 py-1 text-[13px] text-text
resize-none focus:outline-none min-h-[40px]"
style="line-height: {textLineHeight}px"
bind:value={editingText}
onkeydown={(e) => { if (e.key === "Enter" && !e.shiftKey) { e.preventDefault(); onFinishEditing(); } if (e.key === "Escape") { onFinishEditing(); } }}
onblur={onFinishEditing}
onclick={(e) => e.stopPropagation()}
data-no-transition
></textarea>
{:else}
<p class="text-[13px] text-text flex-1" style="line-height: {textLineHeight}px">
{#if searchQuery.trim()}
{@html highlightText(seg.text.trim())}
{:else}
{seg.text.trim()}
{/if}
</p>
{/if}
<!-- Actions (hover reveal) -->
<div class="flex items-center gap-1 opacity-0 group-hover:opacity-100 flex-shrink-0">
<button
class="text-text-tertiary hover:text-accent"
onclick={(e) => { e.stopPropagation(); onCopySegment(idx); }}
title="Copy segment"
>
<svg class="w-3.5 h-3.5" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
<rect x="9" y="9" width="13" height="13" rx="2" /><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1" />
</svg>
</button>
<button
class="text-text-tertiary hover:text-danger"
onclick={(e) => { e.stopPropagation(); onDeleteSegment(idx); }}
title="Delete segment"
>
<svg class="w-3.5 h-3.5" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
<path d="M18 6L6 18M6 6l12 12" stroke-linecap="round" />
</svg>
</button>
</div>
</div>
{/each}
</div>
</div>

View File

@@ -1,25 +1,29 @@
<script>
import { onMount, onDestroy } from "svelte";
import { invoke } from "@tauri-apps/api/core";
import { listen } from "@tauri-apps/api/event";
import { page, settings, templates, addToHistory, addTask, tasks } from "$lib/stores/page.svelte.js";
import { Channel, invoke } from "@tauri-apps/api/core";
import { page, settings, templates, profiles, addToHistory, addTask, tasks } from "$lib/stores/page.svelte.js";
import { toasts } from "$lib/stores/toasts.svelte.js";
import Card from "$lib/components/Card.svelte";
import ModelDownloader from "$lib/components/ModelDownloader.svelte";
import { exportTranscript } from "$lib/utils/export.js";
import { extractTasks } from "$lib/utils/taskExtractor.js";
import { pad } from "$lib/utils/time.js";
import { MAX_PCM_SAMPLES, MIN_CHUNK_SAMPLES, CHUNK_INTERVAL_MS, FEEDBACK_TIMEOUT_MS } from "$lib/utils/constants.js";
import { FEEDBACK_TIMEOUT_MS } from "$lib/utils/constants.js";
import { Mic, Loader2, SquareCheck, AlertTriangle } from 'lucide-svelte';
import EmptyState from '$lib/components/EmptyState.svelte';
import { getPreferences } from '$lib/stores/preferences.svelte.js';
import { bionicReading } from '$lib/actions/bionicReading.js';
import { measurePreWrap } from '$lib/utils/textMeasure.js';
import { transcriptPretextFont, transcriptPretextLineHeight } from '$lib/utils/accessibilityTypography.js';
import { hasTauriRuntime } from '$lib/utils/runtime.js';
const prefs = getPreferences();
const tauriRuntimeAvailable = hasTauriRuntime();
const browserPreviewMessage = "You're viewing Kon in a normal browser. Local transcription only works in the Tauri desktop app window.";
let transcript = $state("");
let segments = $state([]);
let timerInterval = $state(null);
let startTime = $state(0);
let chunkId = $state(0);
let modelReady = $state(false);
let modelLoading = $state(false);
let needsDownload = $state(false);
@@ -30,8 +34,14 @@
let extractedCount = $state(0);
let aiProcessing = $state(false);
let aiStatus = $state("");
let unlisten = null;
let chunkTimeOffset = 0;
let runtimeCapabilities = $state(null);
let sessionId = $state(null);
let drainingSessionId = $state(null);
let liveWarning = $state("");
let lastResultAt = $state(0);
let lastLiveActivityAt = $state(0);
let resultChannel = null;
let statusChannel = null;
// Cursor-based insertion
let textareaEl = $state(null);
@@ -44,41 +54,23 @@
// Deduplication: track which chunk IDs have been processed
let processedChunks = new Set();
// AudioWorklet state
let audioContext = null;
let workletNode = null;
let mediaStream = null;
let pcmBuffer = [];
let chunkTimer = null;
let allSamples = []; // Accumulate all PCM for audio saving
// Global hotkey listener
let hotkeyHandler = () => toggleRecording();
onMount(async () => {
unlisten = await listen("transcription-result", (event) => {
let result;
try {
result = typeof event.payload === "string"
? JSON.parse(event.payload)
: event.payload;
} catch (e) {
console.error("Failed to parse transcription result:", e);
return;
}
handleResult(result);
});
window.addEventListener("kon:toggle-recording", hotkeyHandler);
if (!tauriRuntimeAvailable) {
error = browserPreviewMessage;
return;
}
await checkModelState();
});
onDestroy(() => {
if (unlisten) unlisten();
window.removeEventListener("kon:toggle-recording", hotkeyHandler);
clearInterval(timerInterval);
clearInterval(chunkTimer);
if (page.recording) {
page.recording = false;
page.status = "Ready";
@@ -87,74 +79,154 @@
cleanup();
});
function handleResult(result) {
if (result.status === "transcription" && result.segments) {
// Deduplication guard: skip if this chunk_id was already processed
if (result.chunk_id != null && processedChunks.has(result.chunk_id)) {
return;
$effect(() => {
settings.engine;
settings.modelSize;
if (!tauriRuntimeAvailable) return;
queueMicrotask(() => {
if (!page.recording) {
void checkModelState();
}
if (result.chunk_id != null) processedChunks.add(result.chunk_id);
const text = result.segments.map((s) => s.text).join(" ").trim();
if (text) {
if (insertPos >= 0) {
// Insert at cursor position
const before = transcript.slice(0, insertPos);
const after = transcript.slice(insertPos);
const spaceBefore = before && !before.endsWith(" ") && !before.endsWith("\n") ? " " : "";
const spaceAfter = after && !after.startsWith(" ") && !after.startsWith("\n") ? " " : "";
transcript = before + spaceBefore + text + spaceAfter + after;
insertPos += spaceBefore.length + text.length + spaceAfter.length;
// Move cursor to end of inserted text
requestAnimationFrame(() => {
if (textareaEl) {
textareaEl.selectionStart = insertPos;
textareaEl.selectionEnd = insertPos;
}
});
} else {
// Append mode
// First chunk (id=1) starts the transcript; subsequent chunks append to it.
// When transcript is empty, assign directly; otherwise prepend a space separator.
if (!transcript) {
transcript = text;
} else {
transcript += " " + text;
});
});
function whisperModelId(size) {
const map = {
Tiny: "whisper-tiny-en",
Base: "whisper-base-en",
Small: "whisper-small-en",
Medium: "whisper-medium-en",
};
return map[size] || "whisper-base-en";
}
function selectedModelId() {
return settings.engine === "parakeet"
? "parakeet-ctc-0.6b-int8"
: whisperModelId(settings.modelSize);
}
function currentEngineCapabilities() {
return runtimeCapabilities?.engines?.find((engine) => engine.id === settings.engine) || null;
}
function currentModelCapabilities() {
return currentEngineCapabilities()?.models?.find((model) => model.id === selectedModelId()) || null;
}
function currentModelIsEnglishOnly() {
return currentModelCapabilities()?.languageSupport?.kind === "english-only";
}
function effectiveLanguage() {
return currentModelIsEnglishOnly() ? "en" : settings.language;
}
function buildInitialPrompt() {
if (!page.activeProfile || page.activeProfile === "None") return "";
const profile = profiles.find((entry) => entry.name === page.activeProfile);
if (!profile?.words) return "";
const words = profile.words
.split("\n")
.map((word) => word.trim())
.filter(Boolean);
if (words.length === 0) return "";
return `Use these terms when they match the audio: ${words.join(", ")}`;
}
async function refreshRuntimeCapabilities() {
runtimeCapabilities = await invoke("get_runtime_capabilities");
}
function matchesLiveSession(candidateSessionId) {
return candidateSessionId != null
&& (candidateSessionId === sessionId || candidateSessionId === drainingSessionId);
}
function handleLiveResult(result) {
if (!result || !matchesLiveSession(result.sessionId) || !result.segments) {
return;
}
lastResultAt = Date.now();
lastLiveActivityAt = lastResultAt;
if (textareaEl) {
if (page.recording && userScrolledUp) {
shouldPreserveScrollAnchor = true;
} else if (page.recording) {
shouldStickToBottom = true;
}
}
if (result.chunkId != null && processedChunks.has(result.chunkId)) {
return;
}
if (result.chunkId != null) processedChunks.add(result.chunkId);
const text = result.segments.map((segment) => segment.text).join(" ").trim();
if (text) {
if (insertPos >= 0) {
const before = transcript.slice(0, insertPos);
const after = transcript.slice(insertPos);
const spaceBefore = before && !before.endsWith(" ") && !before.endsWith("\n") ? " " : "";
const spaceAfter = after && !after.startsWith(" ") && !after.startsWith("\n") ? " " : "";
transcript = before + spaceBefore + text + spaceAfter + after;
insertPos += spaceBefore.length + text.length + spaceAfter.length;
requestAnimationFrame(() => {
if (textareaEl) {
textareaEl.selectionStart = insertPos;
textareaEl.selectionEnd = insertPos;
}
}
// Offset segment timestamps to be absolute
const offset = chunkTimeOffset;
const adjusted = result.segments.map((s) => ({
...s,
start: s.start + offset,
end: s.end + offset,
}));
segments = [...segments, ...adjusted];
if (result.duration) {
chunkTimeOffset += result.duration;
}
});
} else {
transcript = transcript ? `${transcript} ${text}` : text;
}
}
if (!page.recording && result.chunk_id === chunkId) {
finaliseTranscription();
}
const absoluteSegments = result.segments.map((segment) => ({
...segment,
start: segment.start + result.chunkStartSecs,
end: segment.end + result.chunkStartSecs,
}));
segments = [...segments, ...absoluteSegments];
}
function handleLiveStatus(status) {
if (!status || !matchesLiveSession(status.sessionId)) return;
lastLiveActivityAt = Date.now();
if (status.type === "overload" || status.type === "warning") {
liveWarning = status.message || "Kon is dropping older audio to stay responsive.";
return;
}
if (status.type === "error") {
error = status.message || "Live transcription failed";
transcriptionFailed = true;
page.status = "Error";
page.statusColor = "#e87171";
return;
}
if (status.type === "finished" && status.droppedAudioMs > 0) {
liveWarning = `Dropped ${Math.round(status.droppedAudioMs / 1000)}s of older audio to keep up in real time.`;
}
}
async function checkModelState() {
if (!tauriRuntimeAvailable) {
modelReady = false;
needsDownload = false;
return;
}
try {
if (settings.engine === "parakeet") {
const loaded = await invoke("check_parakeet_engine");
if (loaded) { modelReady = true; return; }
const downloaded = await invoke("check_parakeet_model", { name: "ctc-int8" });
if (downloaded) { needsDownload = false; await loadModel(); }
else { needsDownload = true; }
} else {
const loaded = await invoke("check_engine");
if (loaded) { modelReady = true; return; }
const downloaded = await invoke("check_model", { size: settings.modelSize.toLowerCase() });
if (downloaded) { needsDownload = false; await loadModel(); }
else { needsDownload = true; }
await refreshRuntimeCapabilities();
const currentModel = currentModelCapabilities();
modelReady = !!currentModel?.loaded;
needsDownload = currentModel ? !currentModel.downloaded : true;
if (currentModelIsEnglishOnly() && settings.language !== "en") {
settings.language = "en";
}
} catch (err) {
error = typeof err === "string" ? err : err.message || "Failed to check model";
@@ -162,6 +234,10 @@
}
async function loadModel() {
if (!tauriRuntimeAvailable) {
error = browserPreviewMessage;
return;
}
modelLoading = true;
page.status = "Loading model...";
page.statusColor = "#e8c86e";
@@ -173,6 +249,7 @@
} else {
await invoke("load_model", { size: settings.modelSize.toLowerCase() });
}
await refreshRuntimeCapabilities();
modelReady = true;
modelLoading = false;
page.status = "Ready";
@@ -187,7 +264,7 @@
function onModelDownloaded() {
needsDownload = false;
loadModel();
void loadModel();
}
async function toggleRecording() {
@@ -200,8 +277,15 @@
async function startRecording() {
error = "";
liveWarning = "";
saved = false;
transcriptionFailed = false;
if (!tauriRuntimeAvailable) {
error = browserPreviewMessage;
page.status = "Desktop app required";
page.statusColor = "#e8c86e";
return;
}
if (!modelReady) {
if (needsDownload) return;
await loadModel();
@@ -216,48 +300,53 @@
}
try {
audioContext = new AudioContext({ sampleRate: 16000 });
await audioContext.audioWorklet.addModule("/pcm-processor.js");
mediaStream = await navigator.mediaDevices.getUserMedia({
audio: { channelCount: 1, echoCancellation: true, noiseSuppression: true },
});
const source = audioContext.createMediaStreamSource(mediaStream);
workletNode = new AudioWorkletNode(audioContext, "pcm-processor");
pcmBuffer = [];
chunkId = 0;
chunkTimeOffset = 0;
sessionId = null;
drainingSessionId = null;
lastResultAt = 0;
lastLiveActivityAt = 0;
processedChunks.clear();
// Only clear transcript if not in insert mode (fresh recording)
if (insertPos === -1) {
transcript = "";
segments = [];
previousMeasuredContentHeight = 0;
}
allSamples = [];
workletNode.port.onmessage = (e) => {
if (e.data.type === "pcm") {
pcmBuffer = pcmBuffer.concat(e.data.samples);
if (settings.saveAudio) {
allSamples = allSamples.concat(e.data.samples);
}
}
};
resultChannel = new Channel((message) => handleLiveResult(message));
statusChannel = new Channel((message) => handleLiveStatus(message));
source.connect(workletNode);
const response = await invoke("start_live_transcription_session", {
config: {
engine: settings.engine,
modelId: selectedModelId(),
language: effectiveLanguage(),
initialPrompt: buildInitialPrompt(),
saveAudio: settings.saveAudio,
outputFolder: settings.outputFolder || null,
removeFillers: settings.removeFillers,
britishEnglish: settings.britishEnglish,
antiHallucination: settings.antiHallucination,
formatMode: settings.formatMode,
// Honour Settings → Audio → Microphone choice. Empty = auto-select.
microphoneDevice: settings.microphoneDevice || null,
},
resultChannel,
statusChannel,
});
sessionId = response.sessionId;
startTime = Date.now();
page.recording = true;
page.status = "Recording...";
page.statusColor = "#e87171";
timerInterval = setInterval(updateTimer, 1000);
chunkTimer = setInterval(sendChunk, CHUNK_INTERVAL_MS);
} catch (err) {
error = `Microphone access denied: ${err.message}`;
error = typeof err === "string" ? err : err?.message || "Microphone error";
// Surface the failure as a sticky error toast so it does not get
// swallowed if the inline `error` panel is not visible.
// (Day 3 of the upgrade plan)
toasts.error("Could not start recording", error);
page.status = "Error";
page.statusColor = "#e87171";
cleanup();
@@ -266,115 +355,71 @@
async function stopRecording() {
clearInterval(timerInterval);
clearInterval(chunkTimer);
page.recording = false;
transcribing = true;
page.status = "Finalising...";
page.statusColor = "#e8c86e";
const waitForTranscription = () => new Promise((resolve) => {
const check = () => transcribing ? setTimeout(check, 100) : resolve();
check();
});
await waitForTranscription();
await sendChunk();
cleanup();
if (chunkId === 0) {
page.status = "Ready";
page.statusColor = "#7ec89a";
}
}
function cleanup() {
if (workletNode) {
workletNode.disconnect();
workletNode = null;
}
if (mediaStream) {
mediaStream.getTracks().forEach((t) => t.stop());
mediaStream = null;
}
if (audioContext) {
audioContext.close();
audioContext = null;
}
}
async function sendChunk() {
if (pcmBuffer.length < MIN_CHUNK_SAMPLES) return;
if (transcribing) return;
chunkId++;
const currentChunkId = chunkId;
const samples = [...pcmBuffer];
pcmBuffer = [];
if (samples.length > MAX_PCM_SAMPLES) {
console.warn(`PCM buffer truncated from ${samples.length} to ${MAX_PCM_SAMPLES} samples (~5 min at 16 kHz)`);
samples.length = MAX_PCM_SAMPLES;
}
transcribing = true;
try {
let initialPrompt = "";
if (page.activeProfile && page.activeProfile !== "None") {
initialPrompt = page.activeProfile;
}
const activityBeforeStop = lastLiveActivityAt;
drainingSessionId = sessionId;
const response = sessionId
? await invoke("stop_live_transcription_session", { sessionId })
: null;
await waitForResultDrain(activityBeforeStop);
cleanup();
if (settings.engine === "parakeet") {
await invoke("transcribe_pcm_parakeet", {
samples,
chunkId: currentChunkId,
removeFillers: settings.removeFillers,
britishEnglish: settings.britishEnglish,
antiHallucination: settings.antiHallucination,
formatMode: settings.formatMode,
});
if (processedChunks.size === 0 && !transcript.trim()) {
page.status = "Ready";
page.statusColor = "#7ec89a";
} else {
await invoke("transcribe_pcm", {
samples,
chunkId: currentChunkId,
language: settings.language,
initialPrompt,
removeFillers: settings.removeFillers,
britishEnglish: settings.britishEnglish,
antiHallucination: settings.antiHallucination,
formatMode: settings.formatMode,
});
await finaliseTranscription(response?.audioPath || null);
}
} catch (err) {
console.error("transcribe_pcm failed:", err);
error = typeof err === "string" ? err : err.message || "Transcription failed";
transcriptionFailed = true;
if (!page.recording) {
page.status = "Error";
page.statusColor = "#e87171";
}
error = typeof err === "string" ? err : err?.message || "Failed to stop live transcription";
page.status = "Error";
page.statusColor = "#e87171";
cleanup();
} finally {
transcribing = false;
}
}
async function finaliseTranscription() {
function cleanup() {
sessionId = null;
drainingSessionId = null;
resultChannel = null;
statusChannel = null;
}
async function waitForResultDrain(previousActivityAt = 0) {
const firstMessageDeadline = Date.now() + 400;
while (Date.now() < firstMessageDeadline) {
if (lastLiveActivityAt > previousActivityAt) {
break;
}
await new Promise((resolve) => setTimeout(resolve, 25));
}
if (lastLiveActivityAt <= previousActivityAt) {
return;
}
const idleDeadline = Date.now() + 1500;
while (Date.now() < idleDeadline) {
if (Date.now() - lastLiveActivityAt >= 200) {
return;
}
await new Promise((resolve) => setTimeout(resolve, 25));
}
}
async function finaliseTranscription(audioPath = null) {
if (transcript.trim()) {
if (settings.autoCopy) {
invoke("copy_to_clipboard", { text: transcript }).catch(() => {});
}
// Save audio if enabled — capture path for history replay
let audioPath = null;
if (settings.saveAudio && allSamples.length > 0) {
try {
audioPath = await invoke("save_audio", {
samples: allSamples,
outputFolder: settings.outputFolder || null,
});
} catch (err) {
console.error("save_audio failed:", err);
}
allSamples = [];
}
const historyId = crypto.randomUUID();
addToHistory({
id: historyId,
@@ -384,7 +429,7 @@
text: transcript,
segments: segments,
duration: (Date.now() - startTime) / 1000,
language: settings.language,
language: effectiveLanguage(),
template: activeTemplate || undefined,
audioPath,
});
@@ -425,8 +470,12 @@
function clearTranscript() {
transcript = "";
segments = [];
previousMeasuredContentHeight = 0;
shouldPreserveScrollAnchor = false;
shouldStickToBottom = false;
page.timerText = "00:00";
error = "";
liveWarning = "";
saved = false;
insertPos = -1;
activeTemplate = "";
@@ -452,6 +501,7 @@
showTemplateMenu = false;
transcript = template.sections.map((s) => `## ${s}\n\n`).join("\n");
segments = [];
previousMeasuredContentHeight = 0;
// Position cursor at first section body
requestAnimationFrame(() => {
if (textareaEl) {
@@ -504,6 +554,80 @@
return trimmed ? trimmed.split(/\s+/).length : 0;
});
// Pretext content height measurement (DOM-free).
// Used for scroll-to-bottom affordance during live transcription.
let textareaWidth = $state(0);
let userScrolledUp = $state(false);
let previousMeasuredContentHeight = $state(0);
let shouldPreserveScrollAnchor = $state(false);
let shouldStickToBottom = $state(false);
// Build font string matching .font-transcript CSS
let transcriptFont = $derived(transcriptPretextFont(prefs.accessibility));
let transcriptLineHeight = $derived(transcriptPretextLineHeight(prefs.accessibility));
let contentHeight = $derived.by(() => {
if (!transcript.trim() || textareaWidth <= 0) return 0;
// Subtract padding (p-6 = 24px each side)
const innerWidth = textareaWidth - 48;
if (innerWidth <= 0) return 0;
return measurePreWrap(transcript, transcriptFont, innerWidth, transcriptLineHeight).height;
});
function onTextareaScroll(e) {
const el = e.target;
// "scrolled up" = more than 1 line from the bottom
userScrolledUp = (el.scrollHeight - el.scrollTop - el.clientHeight) > transcriptLineHeight;
}
function scrollToBottom() {
if (textareaEl) {
textareaEl.scrollTop = textareaEl.scrollHeight;
userScrolledUp = false;
}
}
// Track textarea width for Pretext layout calculations
$effect(() => {
if (!textareaEl) return;
const ro = new ResizeObserver(entries => {
for (const entry of entries) textareaWidth = entry.contentRect.width;
});
ro.observe(textareaEl);
return () => ro.disconnect();
});
$effect(() => {
const measuredHeight = contentHeight;
const previousHeight = previousMeasuredContentHeight;
if (!textareaEl) {
previousMeasuredContentHeight = measuredHeight;
return;
}
if (shouldPreserveScrollAnchor && measuredHeight > previousHeight) {
const delta = measuredHeight - previousHeight;
requestAnimationFrame(() => {
if (textareaEl) {
textareaEl.scrollTop += delta;
}
});
shouldPreserveScrollAnchor = false;
} else if (shouldStickToBottom) {
requestAnimationFrame(() => {
if (textareaEl) {
textareaEl.scrollTop = textareaEl.scrollHeight;
}
});
shouldStickToBottom = false;
}
previousMeasuredContentHeight = measuredHeight;
});
let showScrollHint = $derived(page.recording && userScrolledUp && contentHeight > 0);
let reduceMotion = $derived(
prefs.accessibility.reduceMotion === 'on'
|| (prefs.accessibility.reduceMotion === 'system'
@@ -519,7 +643,11 @@
{#if page.recording}Recording started{/if}
</div>
{#if needsDownload}
<ModelDownloader modelSize={settings.modelSize.toLowerCase()} onComplete={onModelDownloaded} />
<ModelDownloader
engine={settings.engine}
modelSize={settings.modelSize.toLowerCase()}
onComplete={onModelDownloaded}
/>
{:else}
<!-- Control strip -->
<div class="flex items-center gap-3 px-5 h-[56px] border-b border-border-subtle flex-shrink-0">
@@ -531,9 +659,11 @@
? 'bg-danger animate-pulse-warm'
: modelLoading
? 'bg-warning opacity-60 cursor-wait'
: 'bg-accent hover:bg-accent-hover shadow-[0_4px_20px_rgba(232,168,124,0.3)]'}"
: !tauriRuntimeAvailable
? 'bg-bg-elevated text-text-tertiary cursor-not-allowed'
: 'bg-accent hover:bg-accent-hover shadow-[0_4px_20px_rgba(232,168,124,0.3)]'}"
onclick={toggleRecording}
disabled={modelLoading}
disabled={modelLoading || !tauriRuntimeAvailable}
aria-label={page.recording ? "Stop recording" : "Start recording"}
style="transition-duration: var(--duration-ui)"
>
@@ -571,6 +701,8 @@
<span class="text-[11px] text-text-tertiary">
{#if modelLoading}
Loading model...
{:else if !tauriRuntimeAvailable}
Desktop app required for local transcription
{:else if saved}
<span class="text-success animate-fade-in">
Saved{#if extractedCount > 0} · {extractedCount} task{extractedCount === 1 ? '' : 's'} extracted{/if}
@@ -612,7 +744,7 @@
aria-label="Toggle task sidebar"
>
<span class="flex items-center gap-1.5">
<SquareCheck size={16} class="{page.taskSidebarOpen ? 'text-accent' : 'text-text-tertiary'}" aria-hidden="true" />
<SquareCheck size={16} class={page.taskSidebarOpen ? 'text-accent' : 'text-text-tertiary'} aria-hidden="true" />
<span class="text-[11px] {page.taskSidebarOpen ? 'text-accent' : 'text-text-tertiary'}">Tasks</span>
</span>
{#if taskCount > 0}
@@ -704,6 +836,14 @@
</div>
{/if}
{#if liveWarning && !error}
<div class="px-5 pt-2 animate-fade-in flex-shrink-0">
<div class="px-4 py-2 rounded-lg bg-warning/10 border border-warning/20 text-[12px] text-warning">
{liveWarning}
</div>
</div>
{/if}
<!-- Transcript area -->
<div class="flex-1 px-5 pt-3 pb-3 min-h-0" style="--text-transcript: {prefs.accessibility.transcriptSize}px; font-size: var(--text-transcript)">
<Card classes="h-full flex flex-col">
@@ -727,17 +867,30 @@
aria-live="polite"
></textarea>
{:else}
<textarea
bind:this={textareaEl}
class="font-transcript flex-1 w-full bg-transparent text-text p-6
resize-none focus:outline-none placeholder:text-text-tertiary min-h-0"
placeholder={activeTemplate ? "Click a section above, then press record..." : "Your words will appear here..."}
bind:value={transcript}
onclick={() => { showExportMenu = false; showTemplateMenu = false; }}
data-no-transition
aria-label="Transcript"
aria-live="polite"
></textarea>
<div class="relative flex-1 min-h-0">
<textarea
bind:this={textareaEl}
class="font-transcript h-full w-full bg-transparent text-text p-6
resize-none focus:outline-none placeholder:text-text-tertiary"
placeholder={activeTemplate ? "Click a section above, then press record..." : "Your words will appear here..."}
bind:value={transcript}
onscroll={onTextareaScroll}
onclick={() => { showExportMenu = false; showTemplateMenu = false; }}
data-no-transition
aria-label="Transcript"
aria-live="polite"
></textarea>
{#if showScrollHint}
<button
class="absolute bottom-3 right-6 px-3 py-1.5 rounded-full bg-accent text-white text-[11px] font-medium shadow-md hover:bg-accent-hover animate-fade-in"
onclick={scrollToBottom}
aria-label="Scroll to latest"
style="transition-duration: var(--duration-ui)"
>
↓ New text
</button>
{/if}
</div>
{/if}
<!-- Status footer (inside transcript card) -->

View File

@@ -1,14 +1,35 @@
<script>
import { onDestroy } from "svelte";
import { invoke } from "@tauri-apps/api/core";
import { history, saveHistory, deleteFromHistory } from "$lib/stores/page.svelte.js";
import { history, saveHistory, deleteFromHistory, renameHistoryEntry } from "$lib/stores/page.svelte.js";
import { toasts } from "$lib/stores/toasts.svelte.js";
import { convertFileSrc } from "@tauri-apps/api/core";
import { getPreferences } from "$lib/stores/preferences.svelte.js";
import { clampTextLines, measurePreWrap } from "$lib/utils/textMeasure.js";
import { bodyPretextLineHeight, pretextFontShorthand } from "$lib/utils/accessibilityTypography.js";
import { buildCumulativeOffsets, findVisibleRange } from "$lib/utils/virtualList.js";
import Card from "$lib/components/Card.svelte";
import EmptyState from "$lib/components/EmptyState.svelte";
import { formatTime, formatDuration } from "$lib/utils/time.js";
import { PLAYBACK_SPEEDS, FEEDBACK_TIMEOUT_MS } from "$lib/utils/constants.js";
import { PLAYBACK_SPEEDS } from "$lib/utils/constants.js";
import { Search, Clock, Play, Pause, FileText, Mic, ChevronDown } from 'lucide-svelte';
const prefs = getPreferences();
const COLLAPSED_ROW_MIN_HEIGHT = 54;
const COLLAPSED_ROW_VERTICAL_PADDING = 24;
const EXPANDED_BASE_HEIGHT = 92;
const AUDIO_PLAYER_HEIGHT = 54;
const HISTORY_PREVIEW_LINES = 2;
const HISTORY_DURATION_WIDTH = 48;
const HISTORY_DATE_WIDTH = 90;
const HISTORY_ICON_WIDTH = 24;
const HISTORY_SOURCE_WIDTH = 14;
const HISTORY_CHEVRON_WIDTH = 14;
const HISTORY_ROW_GAP = 12;
const HISTORY_ROW_HORIZONTAL_PADDING = 32;
const HISTORY_PREVIEW_MIN_WIDTH = 120;
const BUFFER_COUNT = 6;
let searchQuery = $state("");
let expandedId = $state(null);
let playingId = $state(null);
@@ -17,6 +38,10 @@
let duration = $state(0);
let playbackRate = $state(1);
let animFrameId = null;
let listEl = $state(null);
let containerHeight = $state(0);
let containerWidth = $state(0);
let scrollTop = $state(0);
onDestroy(() => {
stopPlayback();
@@ -36,6 +61,106 @@
: history
);
let historyTextFont = $derived(pretextFontShorthand(prefs.accessibility, 13));
let historyLineHeight = $derived(bodyPretextLineHeight(prefs.accessibility, 13));
function compactPreviewText(item) {
return item.title || item.preview || item.text || "";
}
let compactPreviews = $derived.by(() => {
return filtered.map((item) => {
const text = compactPreviewText(item);
if (!text) {
return {
text: "",
fullText: "",
height: historyLineHeight,
};
}
if (containerWidth <= 0) {
const fallbackText = text.length > 80 ? `${text.slice(0, 79)}…` : text;
return {
text: fallbackText,
fullText: text,
height: historyLineHeight,
};
}
const reservedWidth =
HISTORY_ROW_HORIZONTAL_PADDING +
HISTORY_ICON_WIDTH +
(item.duration ? HISTORY_DURATION_WIDTH : 0) +
HISTORY_SOURCE_WIDTH +
HISTORY_DATE_WIDTH +
HISTORY_CHEVRON_WIDTH +
HISTORY_ROW_GAP * (item.duration ? 5 : 4);
const textWidth = Math.max(HISTORY_PREVIEW_MIN_WIDTH, containerWidth - reservedWidth);
const preview = clampTextLines(
text,
historyTextFont,
textWidth,
historyLineHeight,
HISTORY_PREVIEW_LINES,
);
return {
text: preview.text,
fullText: text,
height: preview.height,
};
});
});
let itemHeights = $derived.by(() => {
if (!filtered.length) return [];
const textWidth = Math.max(0, containerWidth - 32);
return filtered.map((item, index) => {
const compactHeight = Math.max(
COLLAPSED_ROW_MIN_HEIGHT,
(compactPreviews[index]?.height || historyLineHeight) + COLLAPSED_ROW_VERTICAL_PADDING,
);
let height = compactHeight;
if (expandedId === item.id) {
const transcriptHeight = item.text && textWidth > 0
? measurePreWrap(item.text, historyTextFont, textWidth, historyLineHeight).height
: historyLineHeight;
height += transcriptHeight + EXPANDED_BASE_HEIGHT;
if (item.audioPath && playingId === item.id) {
height += AUDIO_PLAYER_HEIGHT;
}
}
return height;
});
});
let cumulativeOffsets = $derived(buildCumulativeOffsets(itemHeights));
let totalHeight = $derived(
cumulativeOffsets.length > 0 ? cumulativeOffsets[cumulativeOffsets.length - 1] : 0
);
let visibleRange = $derived.by(() =>
findVisibleRange(
cumulativeOffsets,
filtered.length,
scrollTop,
containerHeight,
BUFFER_COUNT,
)
);
let visibleItems = $derived.by(() => {
const items = [];
for (let i = visibleRange.start; i < visibleRange.end; i++) {
items.push({
item: filtered[i],
top: cumulativeOffsets[i],
height: itemHeights[i],
preview: compactPreviews[i],
});
}
return items;
});
function clearAll() {
if (!confirm("Delete all history? This can't be undone.")) return;
history.splice(0);
@@ -48,6 +173,10 @@
expandedId = expandedId === id ? null : id;
}
function onListScroll(e) {
scrollTop = e.target.scrollTop;
}
function copyItem(item) {
invoke("copy_to_clipboard", { text: item.text }).catch(() => {});
}
@@ -61,12 +190,24 @@
}
}
function renameItem(item) {
async function renameItem(item) {
const name = prompt("Name this transcript:", item.title || "");
if (name !== null) {
item.title = name.trim();
item.preview = name.trim() ? `${name.trim()}${item.text.slice(0, 80)}` : item.text.slice(0, 120);
saveHistory();
if (name === null) return;
const trimmed = name.trim();
item.title = trimmed;
item.preview = trimmed ? `${trimmed}${item.text.slice(0, 80)}` : item.text.slice(0, 120);
// Persist via the dual-write helper. Updates SQLite + localStorage and
// surfaces a toast on failure (Day 4 of the upgrade plan, closes
// architecture-review.md §13).
try {
await renameHistoryEntry(item.id, { title: trimmed });
} catch (err) {
toasts.warn(
"Rename did not persist",
"Your change is visible now but did not save. It may revert on next launch."
);
}
}
@@ -115,12 +256,14 @@
}
}
let seekTimeout = null;
function seekTo(e) {
const val = parseFloat(e.target.value);
if (audioEl) {
audioEl.currentTime = val;
currentTime = val;
}
currentTime = val; // update UI immediately
clearTimeout(seekTimeout);
seekTimeout = setTimeout(() => {
if (audioEl) audioEl.currentTime = val;
}, 50);
}
function setSpeed(speed) {
@@ -137,6 +280,24 @@
window.open("/viewer", "_blank", "width=600,height=700");
}
}
$effect(() => {
if (!listEl) return;
const ro = new ResizeObserver((entries) => {
for (const entry of entries) {
containerHeight = entry.contentRect.height;
containerWidth = entry.contentRect.width;
}
});
ro.observe(listEl);
return () => ro.disconnect();
});
$effect(() => {
if (expandedId && !filtered.some((item) => item.id === expandedId)) {
expandedId = null;
}
});
</script>
<div class="flex flex-col h-full overflow-y-auto animate-fade-in">
@@ -186,137 +347,147 @@
message={searchQuery ? "No matching transcripts" : "Your transcriptions will be saved here"}
/>
{:else}
<div class="flex-1 overflow-y-auto">
{#each filtered as item}
<!-- Compact row -->
<div
class="flex items-center gap-3 px-4 py-3 border-b border-border-subtle bg-bg-card hover:bg-hover cursor-pointer"
style="transition-duration: var(--duration-ui)"
onclick={() => toggleExpand(item.id)}
onkeydown={(e) => { if (e.key === "Enter" || e.key === " ") { e.preventDefault(); toggleExpand(item.id); } }}
role="button"
tabindex="0"
aria-expanded={expandedId === item.id}
>
<!-- Play button (if audio exists) -->
{#if item.audioPath}
<button
class="w-6 h-6 rounded-full flex items-center justify-center flex-shrink-0
{playingId === item.id ? 'bg-accent/20 text-accent' : 'bg-accent/10 text-accent hover:bg-accent/20'}"
onclick={(e) => { e.stopPropagation(); togglePlay(item); }}
aria-label={playingId === item.id && audioEl && !audioEl.paused ? "Pause" : "Play"}
<div bind:this={listEl} class="flex-1 overflow-y-auto" onscroll={onListScroll}>
<div class="relative" style="height: {totalHeight}px">
{#each visibleItems as { item, top, height, preview } (item.id)}
<div class="absolute left-0 right-0" style="top: {top}px; min-height: {height}px">
<!-- Compact row -->
<div
class="flex items-center gap-3 px-4 py-3 border-b border-border-subtle bg-bg-card hover:bg-hover cursor-pointer"
style="transition-duration: var(--duration-ui)"
onclick={() => toggleExpand(item.id)}
onkeydown={(e) => { if (e.key === "Enter" || e.key === " ") { e.preventDefault(); toggleExpand(item.id); } }}
role="button"
tabindex="0"
aria-expanded={expandedId === item.id}
>
{#if playingId === item.id && audioEl && !audioEl.paused}
<Pause size={10} aria-hidden="true" />
<!-- Play button (if audio exists) -->
{#if item.audioPath}
<button
class="w-6 h-6 rounded-full flex items-center justify-center flex-shrink-0
{playingId === item.id ? 'bg-accent/20 text-accent' : 'bg-accent/10 text-accent hover:bg-accent/20'}"
onclick={(e) => { e.stopPropagation(); togglePlay(item); }}
aria-label={playingId === item.id && audioEl && !audioEl.paused ? "Pause" : "Play"}
>
{#if playingId === item.id && audioEl && !audioEl.paused}
<Pause size={10} aria-hidden="true" />
{:else}
<Play size={10} class="ml-0.5" aria-hidden="true" />
{/if}
</button>
{:else}
<Play size={10} class="ml-0.5" aria-hidden="true" />
<div class="w-6 h-6 flex-shrink-0"></div>
{/if}
</button>
{:else}
<div class="w-6 h-6 flex-shrink-0"></div>
{/if}
<!-- Title / preview text (truncated) -->
<span class="flex-1 truncate text-[13px] text-text">
{item.title || item.preview || item.text.slice(0, 80)}
</span>
<!-- Title / preview text (truncated) -->
<div class="flex-1 min-w-0">
<p
class="text-[13px] text-text whitespace-pre-wrap break-words"
style="line-height: {historyLineHeight}px"
title={preview?.fullText || ""}
>
{preview?.text || compactPreviewText(item)}
</p>
</div>
<!-- Duration -->
{#if item.duration}
<span class="text-[11px] text-text-tertiary flex-shrink-0 tabular-nums">
{formatDuration(item.duration)}
</span>
{/if}
<!-- Source icon -->
<span class="flex-shrink-0 text-text-tertiary" title={item.source}>
{#if item.source && item.source.toLowerCase().includes("file")}
<FileText size={14} aria-hidden="true" />
{:else}
<Mic size={14} aria-hidden="true" />
{/if}
</span>
<!-- Date (right-aligned) -->
<span class="text-[11px] text-text-tertiary flex-shrink-0 text-right min-w-[90px]">
{item.date}
</span>
<!-- Expand chevron -->
<ChevronDown
size={14}
class="text-text-tertiary flex-shrink-0 {expandedId === item.id ? 'rotate-180' : ''}"
style="transition: transform var(--duration-ui)"
aria-hidden="true"
/>
</div>
<!-- Expanded detail -->
{#if expandedId === item.id}
<div class="px-4 py-4 bg-bg-elevated border-b border-border-subtle animate-slide-up">
<!-- Audio player (if playing this item) -->
{#if item.audioPath && playingId === item.id}
<div class="flex items-center gap-3 mb-4 px-1">
<span class="text-[10px] text-text-tertiary tabular-nums w-[80px]">
{formatTime(currentTime)} / {formatTime(duration)}
<!-- Duration -->
{#if item.duration}
<span class="text-[11px] text-text-tertiary flex-shrink-0 tabular-nums">
{formatDuration(item.duration)}
</span>
<input
type="range"
min="0"
max={duration || 0}
step="0.1"
value={currentTime}
oninput={seekTo}
class="flex-1 accent-accent h-1"
data-no-transition
onclick={(e) => e.stopPropagation()}
/>
<div class="flex gap-0.5">
{#each PLAYBACK_SPEEDS as speed}
{/if}
<!-- Source icon -->
<span class="flex-shrink-0 text-text-tertiary" title={item.source}>
{#if item.source && item.source.toLowerCase().includes("file")}
<FileText size={14} aria-hidden="true" />
{:else}
<Mic size={14} aria-hidden="true" />
{/if}
</span>
<!-- Date (right-aligned) -->
<span class="text-[11px] text-text-tertiary flex-shrink-0 text-right min-w-[90px]">
{item.date}
</span>
<!-- Expand chevron -->
<ChevronDown
size={14}
class="text-text-tertiary flex-shrink-0 {expandedId === item.id ? 'rotate-180' : ''}"
style="transition: transform var(--duration-ui)"
aria-hidden="true"
/>
</div>
<!-- Expanded detail -->
{#if expandedId === item.id}
<div class="px-4 py-4 bg-bg-elevated border-b border-border-subtle">
<!-- Audio player (if playing this item) -->
{#if item.audioPath && playingId === item.id}
<div class="flex items-center gap-3 mb-4 px-1">
<span class="text-[10px] text-text-tertiary tabular-nums w-[80px]">
{formatTime(currentTime)} / {formatTime(duration)}
</span>
<input
type="range"
min="0"
max={duration || 0}
step="0.1"
value={currentTime}
oninput={seekTo}
class="flex-1 accent-accent h-1"
data-no-transition
onclick={(e) => e.stopPropagation()}
/>
<div class="flex gap-0.5">
{#each PLAYBACK_SPEEDS as speed}
<button
class="text-[9px] px-1.5 py-0.5 rounded-full
{playbackRate === speed
? 'bg-accent/15 text-accent font-medium'
: 'text-text-tertiary hover:text-text-secondary'}"
onclick={(e) => { e.stopPropagation(); setSpeed(speed); }}
>{speed}x</button>
{/each}
</div>
</div>
{/if}
<!-- Full transcript -->
<p class="text-[13px] text-text whitespace-pre-wrap mb-4" style="line-height: {historyLineHeight}px">{item.text}</p>
<!-- Action buttons -->
<div class="flex items-center gap-2 flex-wrap">
<button
class="text-[11px] px-3 py-1.5 rounded-lg bg-hover text-text-secondary hover:text-text"
style="transition-duration: var(--duration-ui)"
onclick={(e) => { e.stopPropagation(); renameItem(item); }}
>Rename</button>
<button
class="text-[11px] px-3 py-1.5 rounded-lg bg-hover text-text-secondary hover:text-text"
style="transition-duration: var(--duration-ui)"
onclick={(e) => { e.stopPropagation(); copyItem(item); }}
>Copy</button>
{#if item.audioPath && item.segments && item.segments.length > 0}
<button
class="text-[9px] px-1.5 py-0.5 rounded-full
{playbackRate === speed
? 'bg-accent/15 text-accent font-medium'
: 'text-text-tertiary hover:text-text-secondary'}"
onclick={(e) => { e.stopPropagation(); setSpeed(speed); }}
>{speed}x</button>
{/each}
class="text-[11px] px-3 py-1.5 rounded-lg bg-hover text-accent hover:text-accent-hover"
style="transition-duration: var(--duration-ui)"
onclick={(e) => { e.stopPropagation(); openViewer(item); }}
>Open viewer</button>
{/if}
<div class="flex-1"></div>
<button
class="text-[11px] px-3 py-1.5 rounded-lg bg-hover text-text-secondary hover:text-danger"
style="transition-duration: var(--duration-ui)"
onclick={(e) => { e.stopPropagation(); removeItem(item); }}
>Delete</button>
</div>
</div>
{/if}
<!-- Full transcript -->
<p class="text-[13px] text-text leading-relaxed whitespace-pre-wrap mb-4">{item.text}</p>
<!-- Action buttons -->
<div class="flex items-center gap-2 flex-wrap">
<button
class="text-[11px] px-3 py-1.5 rounded-lg bg-hover text-text-secondary hover:text-text"
style="transition-duration: var(--duration-ui)"
onclick={(e) => { e.stopPropagation(); renameItem(item); }}
>Rename</button>
<button
class="text-[11px] px-3 py-1.5 rounded-lg bg-hover text-text-secondary hover:text-text"
style="transition-duration: var(--duration-ui)"
onclick={(e) => { e.stopPropagation(); copyItem(item); }}
>Copy</button>
{#if item.audioPath && item.segments && item.segments.length > 0}
<button
class="text-[11px] px-3 py-1.5 rounded-lg bg-hover text-accent hover:text-accent-hover"
style="transition-duration: var(--duration-ui)"
onclick={(e) => { e.stopPropagation(); openViewer(item); }}
>Open viewer</button>
{/if}
<div class="flex-1"></div>
<button
class="text-[11px] px-3 py-1.5 rounded-lg bg-hover text-text-secondary hover:text-danger"
style="transition-duration: var(--duration-ui)"
onclick={(e) => { e.stopPropagation(); removeItem(item); }}
>Delete</button>
</div>
</div>
{/if}
{/each}
{/each}
</div>
</div>
{/if}
</Card>

View File

@@ -10,21 +10,139 @@
import ZonePicker from "$lib/components/ZonePicker.svelte";
import AccessibilityControls from "$lib/components/AccessibilityControls.svelte";
import { getPreferences, updatePreferences } from "$lib/stores/preferences.svelte.js";
import { clampTextLines } from "$lib/utils/textMeasure.js";
import { bodyPretextLineHeight, pretextFontShorthand } from "$lib/utils/accessibilityTypography.js";
import { Check, ChevronRight } from "lucide-svelte";
import { open } from "@tauri-apps/plugin-dialog";
const prefs = getPreferences();
let engineStatus = $state("Checking...");
let engineOk = $state(false);
let downloadedModels = $state([]);
let runtimeCapabilities = $state(null);
let downloadingModel = $state("");
let downloadProgress = $state(0);
let unlisten = null;
let outputFolderEl = $state(null);
let outputFolderWidth = $state(0);
// Parakeet state
let parakeetStatus = $state("Checking...");
let parakeetOk = $state(false);
// Audio device picker (Settings → Audio → Microphone)
let audioDevices = $state([]);
let audioDevicesError = $state(null);
async function refreshAudioDevices() {
audioDevicesError = null;
try {
audioDevices = await invoke("list_audio_devices");
} catch (err) {
audioDevicesError = "Could not enumerate audio devices: " + (err?.message || err);
audioDevices = [];
}
}
// --- Custom vocabulary (dictionary) — Day 5 of upgrade plan ---
// Backed by SQLite `dictionary` table via list_dictionary_command etc.
// Terms get injected into the LLM cleanup prompt so the model preserves
// the user's spellings (e.g. medication names, jargon, place names).
let vocabulary = $state([]);
let vocabularyError = $state(null);
let newVocabTerm = $state("");
let newVocabNote = $state("");
async function refreshVocabulary() {
vocabularyError = null;
try {
vocabulary = await invoke("list_dictionary_command");
} catch (err) {
vocabularyError = "Could not load vocabulary: " + (err?.message || err);
}
}
async function addVocabTerm() {
const term = newVocabTerm.trim();
if (!term) return;
try {
await invoke("add_dictionary_entry_command", {
term,
note: newVocabNote.trim() || null,
});
newVocabTerm = "";
newVocabNote = "";
await refreshVocabulary();
} catch (err) {
vocabularyError = "Could not save term: " + (err?.message || err);
}
}
async function deleteVocabTerm(id) {
try {
await invoke("delete_dictionary_entry_command", { id });
await refreshVocabulary();
} catch (err) {
vocabularyError = "Could not delete term: " + (err?.message || err);
}
}
// --- Diagnostics (Settings → About → Generate diagnostic report) ---
// Layer 2 of the diagnostics plan: user previews exactly what would be
// shared, then chooses to copy / save. Nothing leaves the machine
// automatically.
let diagnosticReport = $state("");
let diagnosticReportError = $state(null);
let diagnosticReportSavedTo = $state(null);
let diagnosticReportLoading = $state(false);
async function generateDiagnosticReport() {
diagnosticReportError = null;
diagnosticReportSavedTo = null;
diagnosticReportLoading = true;
try {
diagnosticReport = await invoke("generate_diagnostic_report", {
options: {
includeRecentErrors: true,
includeCrashes: true,
includeSettings: true,
includeLogTail: true,
},
});
} catch (err) {
diagnosticReportError = "Could not generate report: " + (err?.message || err);
} finally {
diagnosticReportLoading = false;
}
}
async function copyDiagnosticReport() {
if (!diagnosticReport) return;
try {
await navigator.clipboard.writeText(diagnosticReport);
diagnosticReportSavedTo = "Copied to clipboard.";
} catch (err) {
diagnosticReportError = "Clipboard copy failed: " + (err?.message || err);
}
}
async function saveDiagnosticReport() {
diagnosticReportError = null;
try {
const path = await invoke("save_diagnostic_report", {
options: {
includeRecentErrors: true,
includeCrashes: true,
includeSettings: true,
includeLogTail: true,
},
});
diagnosticReportSavedTo = "Saved to " + path;
} catch (err) {
diagnosticReportError = "Save failed: " + (err?.message || err);
}
}
let parakeetDownloaded = $state(false);
let parakeetDownloading = $state(false);
let parakeetProgress = $state(0);
@@ -42,9 +160,66 @@
// Accordion state — first section open by default
let openSection = $state('transcription');
let settingsTextFont = $derived(pretextFontShorthand(prefs.accessibility, 11));
let settingsPathLineHeight = $derived(bodyPretextLineHeight(prefs.accessibility, 11));
let outputFolderPreview = $derived.by(() => {
const text = settings.outputFolder || "Default (app data)";
if (!text) return "";
if (outputFolderWidth <= 0) {
return text.length > 80 ? `${text.slice(0, 79)}…` : text;
}
return clampTextLines(
text,
settingsTextFont,
Math.max(120, outputFolderWidth - 24),
settingsPathLineHeight,
2,
).text;
});
function whisperModelId(size) {
const map = {
Tiny: "whisper-tiny-en",
Base: "whisper-base-en",
Small: "whisper-small-en",
Medium: "whisper-medium-en",
};
return map[size] || "whisper-base-en";
}
function selectedModelId() {
return settings.engine === "parakeet"
? "parakeet-ctc-0.6b-int8"
: whisperModelId(settings.modelSize);
}
function engineCapabilities(engineId) {
return runtimeCapabilities?.engines?.find((engine) => engine.id === engineId) || null;
}
function selectedModelCapabilities() {
return engineCapabilities(settings.engine)?.models?.find((model) => model.id === selectedModelId()) || null;
}
function whisperModelCapabilities(size) {
return engineCapabilities("whisper")?.models?.find((model) => model.id === whisperModelId(size)) || null;
}
function currentModelIsEnglishOnly() {
return selectedModelCapabilities()?.languageSupport?.kind === "english-only";
}
function hasGpuAcceleration() {
return (runtimeCapabilities?.accelerators || []).some((accelerator) => accelerator !== "cpu");
}
async function refreshRuntimeCapabilities() {
runtimeCapabilities = await invoke("get_runtime_capabilities");
}
onMount(async () => {
try {
await refreshRuntimeCapabilities();
const loaded = await invoke("check_engine");
engineOk = loaded;
engineStatus = loaded ? "Model loaded" : "No model loaded";
@@ -52,6 +227,14 @@
engineStatus = "Engine not ready";
}
// Populate audio device list so the Microphone picker is ready when
// the user opens the Audio section.
refreshAudioDevices();
// Pre-load vocabulary too. Both lists are small; pre-fetching avoids
// a flash of "no data" when the section opens.
refreshVocabulary();
try {
downloadedModels = await invoke("list_models");
} catch {}
@@ -79,23 +262,22 @@
if (unlistenParakeet) unlistenParakeet();
});
// Auto-save on any change
$effect(() => {
void settings.engine;
void settings.modelSize;
void settings.language;
void settings.device;
void settings.formatMode;
void settings.removeFillers;
void settings.antiHallucination;
void settings.britishEnglish;
void settings.autoCopy;
void settings.includeTimestamps;
void settings.theme;
void settings.fontSize;
void settings.saveAudio;
void settings.outputFolder;
void settings.globalHotkey;
if (!outputFolderEl) return;
const ro = new ResizeObserver((entries) => {
for (const entry of entries) {
outputFolderWidth = entry.contentRect.width;
}
});
ro.observe(outputFolderEl);
return () => ro.disconnect();
});
// Auto-save on any settings change.
// JSON.stringify subscribes to all properties on the reactive object.
// Safe because `settings` only holds persistent preferences — no ephemeral state.
$effect(() => {
void JSON.stringify(settings);
saveSettings();
});
@@ -108,12 +290,24 @@
}
});
$effect(() => {
settings.engine;
settings.modelSize;
if (currentModelIsEnglishOnly() && settings.language !== "en") {
settings.language = "en";
}
if (!hasGpuAcceleration() && settings.device !== "auto") {
settings.device = "auto";
}
});
async function downloadModel(size) {
downloadingModel = size;
downloadProgress = 0;
try {
await invoke("download_model", { size });
downloadedModels = await invoke("list_models");
await refreshRuntimeCapabilities();
downloadingModel = "";
} catch (err) {
downloadingModel = "";
@@ -127,6 +321,7 @@
engineOk = false;
try {
await invoke("load_model", { size });
await refreshRuntimeCapabilities();
engineOk = true;
engineStatus = `${settings.modelSize} model loaded`;
} catch (err) {
@@ -136,7 +331,21 @@
}
function isModelDownloaded(size) {
return downloadedModels.includes(size.toLowerCase());
const runtimeModel = whisperModelCapabilities(size);
if (runtimeModel) {
return runtimeModel.downloaded;
}
return downloadedModels.some((model) => model.toLowerCase() === size.toLowerCase());
}
function isModelLoaded(size) {
const runtimeModel = whisperModelCapabilities(size);
if (runtimeModel) {
return runtimeModel.loaded;
}
return runtimeCapabilities?.engines?.some(
(engine) => engine.id === "whisper" && engine.loadedModelId === whisperModelId(size),
) || false;
}
const modelDescriptions = {
@@ -151,6 +360,7 @@
parakeetProgress = 0;
try {
await invoke("download_parakeet_model", { name: "ctc-int8" });
await refreshRuntimeCapabilities();
parakeetDownloaded = true;
parakeetDownloading = false;
parakeetStatus = "Downloaded (not loaded)";
@@ -165,6 +375,7 @@
parakeetOk = false;
try {
await invoke("load_parakeet_model", { name: "ctc-int8" });
await refreshRuntimeCapabilities();
parakeetOk = true;
parakeetStatus = "Model loaded";
} catch (err) {
@@ -223,6 +434,126 @@
<div class="px-7 pb-8">
<Card>
<!-- Audio (microphone selection) -->
<div class="border-b border-border-subtle">
<button
class="flex items-center justify-between w-full py-4 px-5 text-left"
onclick={() => openSection = openSection === 'audio' ? null : 'audio'}
>
<h3 class="font-display text-[18px] italic text-text">Audio</h3>
<span class="text-text-tertiary text-[16px] leading-none">{openSection === 'audio' ? '' : '+'}</span>
</button>
{#if openSection === 'audio'}
<div class="px-5 pb-5 animate-fade-in">
<div class="mb-6">
<p class="text-[10px] font-medium text-text-tertiary uppercase tracking-wider mb-2">Microphone</p>
<div class="flex items-center gap-3">
<select
class="bg-bg-input border border-border rounded-lg px-3 py-2 text-[13px] text-text
focus:outline-none focus:border-accent focus:shadow-[0_0_0_3px_rgba(232,168,124,0.1)]
appearance-none cursor-pointer min-w-[280px] max-w-full"
bind:value={settings.microphoneDevice}
onfocus={refreshAudioDevices}
>
<option value="">Auto (recommended) — let Kon pick the working mic</option>
{#each audioDevices as dev}
<option value={dev.name} disabled={dev.is_likely_monitor}>
{dev.name}
{dev.is_default ? " (system default)" : ""}
{dev.is_likely_monitor ? " — speaker monitor, skip" : ""}
</option>
{/each}
</select>
<button
class="px-3 py-2 text-[12px] text-text-secondary border border-border rounded-lg hover:border-accent hover:text-text"
onclick={refreshAudioDevices}
type="button"
>Refresh</button>
</div>
{#if audioDevicesError}
<p class="text-[11px] text-error mt-2">{audioDevicesError}</p>
{:else if audioDevices.length === 0}
<p class="text-[11px] text-text-tertiary mt-2">No input devices detected. Check that a microphone is connected and PulseAudio/PipeWire is running.</p>
{:else}
<p class="text-[11px] text-text-tertiary mt-2">
Auto mode tries the system default first, then any other real input. Speaker-monitor sources (loopback) are skipped because they record system audio rather than the microphone. If dictation is silent, pick the device explicitly here.
</p>
{/if}
</div>
</div>
{/if}
</div>
<!-- Vocabulary (custom dictionary injected into LLM cleanup prompt) -->
<div class="border-b border-border-subtle">
<button
class="flex items-center justify-between w-full py-4 px-5 text-left"
onclick={() => openSection = openSection === 'vocabulary' ? null : 'vocabulary'}
>
<h3 class="font-display text-[18px] italic text-text">Vocabulary</h3>
<span class="text-text-tertiary text-[16px] leading-none">{openSection === 'vocabulary' ? '' : '+'}</span>
</button>
{#if openSection === 'vocabulary'}
<div class="px-5 pb-5 animate-fade-in">
<p class="text-[12px] text-text-tertiary mb-4">
Words and phrases the AI cleanup pass should preserve exactly. Useful for medication names, place names, jargon, names of people in your support network, anything Whisper tends to mishear.
</p>
<div class="flex gap-2 mb-4">
<input
type="text"
placeholder="Term (e.g. methylphenidate, Wren, Tartarus)"
bind:value={newVocabTerm}
onkeydown={(e) => { if (e.key === 'Enter') addVocabTerm(); }}
class="flex-1 bg-bg-input border border-border rounded-lg px-3 py-2 text-[13px] text-text
focus:outline-none focus:border-accent focus:shadow-[0_0_0_3px_rgba(232,168,124,0.1)]"
/>
<input
type="text"
placeholder="Note (optional)"
bind:value={newVocabNote}
onkeydown={(e) => { if (e.key === 'Enter') addVocabTerm(); }}
class="flex-1 bg-bg-input border border-border rounded-lg px-3 py-2 text-[13px] text-text
focus:outline-none focus:border-accent focus:shadow-[0_0_0_3px_rgba(232,168,124,0.1)]"
/>
<button
type="button"
onclick={addVocabTerm}
disabled={!newVocabTerm.trim()}
class="px-4 py-2 text-[13px] bg-accent text-bg rounded-lg disabled:opacity-50 disabled:cursor-not-allowed hover:bg-accent-hover"
>Add</button>
</div>
{#if vocabularyError}
<p class="text-[11px] text-error mb-3">{vocabularyError}</p>
{/if}
{#if vocabulary.length === 0}
<p class="text-[11px] text-text-tertiary italic">No terms yet. Add one above.</p>
{:else}
<ul class="space-y-1">
{#each vocabulary as entry (entry.id)}
<li class="flex items-center gap-3 py-2 px-3 bg-bg-input border border-border rounded-lg">
<div class="flex-1 min-w-0">
<div class="text-[13px] text-text font-medium truncate">{entry.term}</div>
{#if entry.note}
<div class="text-[11px] text-text-tertiary truncate">{entry.note}</div>
{/if}
</div>
<button
type="button"
onclick={() => deleteVocabTerm(entry.id)}
class="text-[12px] text-text-tertiary hover:text-error border border-border hover:border-error rounded px-2 py-1"
aria-label="Delete {entry.term}"
>Remove</button>
</li>
{/each}
</ul>
{/if}
</div>
{/if}
</div>
<!-- Transcription -->
<div class="border-b border-border-subtle">
<button
@@ -240,8 +571,9 @@
<p class="text-[10px] font-medium text-text-tertiary uppercase tracking-wider mb-2">Engine</p>
<SegmentedButton options={["whisper", "parakeet"]} bind:value={settings.engine} />
<p class="text-[11px] text-text-tertiary mt-2">
{settings.engine === "whisper" ? "OpenAI Whisper — 99+ languages, reliable" :
"Nvidia Parakeet — faster on CPU, English-focused, auto-punctuated"}
{settings.engine === "whisper"
? "Whisper with the currently shipped English-only models in this build"
: "Parakeet CTC 0.6B — English-only, fast when the model is installed"}
</p>
</div>
@@ -264,7 +596,12 @@
<p class="text-[11px] text-text-tertiary mt-2">{modelDescriptions[settings.modelSize]}</p>
<div class="flex items-center gap-2 mt-3">
{#if isModelDownloaded(settings.modelSize)}
{#if isModelLoaded(settings.modelSize)}
<span class="inline-flex items-center gap-1.5 text-[11px] text-success">
<span class="w-[6px] h-[6px] rounded-full bg-success"></span>
Model loaded
</span>
{:else if isModelDownloaded(settings.modelSize)}
<span class="inline-flex items-center gap-1.5 text-[11px] text-success">
<span class="w-[6px] h-[6px] rounded-full bg-success"></span>
Downloaded
@@ -318,42 +655,68 @@
<!-- Compute device -->
<div class="mb-6">
<p class="text-[10px] font-medium text-text-tertiary uppercase tracking-wider mb-2">Compute Device</p>
<select
class="bg-bg-input border border-border rounded-lg px-3 py-2 text-[13px] text-text
focus:outline-none focus:border-accent focus:shadow-[0_0_0_3px_rgba(232,168,124,0.1)]
appearance-none cursor-pointer w-[220px]"
bind:value={settings.device}
>
<option value="auto">Auto (CPU)</option>
<option value="cuda">CUDA (NVIDIA GPU)</option>
<option value="cpu">CPU</option>
</select>
<p class="text-[11px] text-text-tertiary mt-2">GPU acceleration requires building with CUDA feature</p>
{#if hasGpuAcceleration()}
<select
class="bg-bg-input border border-border rounded-lg px-3 py-2 text-[13px] text-text
focus:outline-none focus:border-accent focus:shadow-[0_0_0_3px_rgba(232,168,124,0.1)]
appearance-none cursor-pointer w-[220px]"
bind:value={settings.device}
>
<option value="auto">Auto</option>
{#if runtimeCapabilities?.accelerators?.includes("cuda")}
<option value="cuda">CUDA (NVIDIA GPU)</option>
{/if}
{#if runtimeCapabilities?.accelerators?.includes("metal")}
<option value="metal">Metal (Apple GPU)</option>
{/if}
{#if runtimeCapabilities?.accelerators?.includes("vulkan")}
<option value="vulkan">Vulkan</option>
{/if}
<option value="cpu">CPU</option>
</select>
<p class="text-[11px] text-text-tertiary mt-2">Only accelerators built into this binary are shown here.</p>
{:else}
<div class="bg-bg-input border border-border rounded-lg px-3 py-2 text-[13px] text-text w-[320px]">
This build is CPU-only. GPU controls appear in GPU-enabled builds.
</div>
{/if}
</div>
<!-- Language -->
<div>
<p class="text-[10px] font-medium text-text-tertiary uppercase tracking-wider mb-2">Language</p>
<div class="flex items-center gap-3">
<select
class="bg-bg-input border border-border rounded-lg px-3 py-2 text-[13px] text-text
focus:outline-none focus:border-accent focus:shadow-[0_0_0_3px_rgba(232,168,124,0.1)]
appearance-none cursor-pointer w-[140px]"
bind:value={settings.language}
>
<option value="en">English</option>
<option value="auto">Auto-detect</option>
<option value="fr">French</option>
<option value="de">German</option>
<option value="es">Spanish</option>
<option value="it">Italian</option>
<option value="pt">Portuguese</option>
<option value="nl">Dutch</option>
<option value="pl">Polish</option>
<option value="ja">Japanese</option>
<option value="ko">Korean</option>
<option value="zh">Chinese</option>
</select>
{#if currentModelIsEnglishOnly()}
<select
class="bg-bg-input border border-border rounded-lg px-3 py-2 text-[13px] text-text
focus:outline-none focus:border-accent focus:shadow-[0_0_0_3px_rgba(232,168,124,0.1)]
appearance-none cursor-not-allowed w-[220px]"
bind:value={settings.language}
disabled
>
<option value="en">English only</option>
</select>
{:else}
<select
class="bg-bg-input border border-border rounded-lg px-3 py-2 text-[13px] text-text
focus:outline-none focus:border-accent focus:shadow-[0_0_0_3px_rgba(232,168,124,0.1)]
appearance-none cursor-pointer w-[140px]"
bind:value={settings.language}
>
<option value="en">English</option>
<option value="auto">Auto-detect</option>
<option value="fr">French</option>
<option value="de">German</option>
<option value="es">Spanish</option>
<option value="it">Italian</option>
<option value="pt">Portuguese</option>
<option value="nl">Dutch</option>
<option value="pl">Polish</option>
<option value="ja">Japanese</option>
<option value="ko">Korean</option>
<option value="zh">Chinese</option>
</select>
{/if}
{#if settings.language === "en"}
<button
class="flex items-center gap-1.5 px-2.5 py-1.5 rounded-full text-[11px] border animate-fade-in
@@ -370,6 +733,13 @@
</button>
{/if}
</div>
<p class="text-[11px] text-text-tertiary mt-2">
{#if currentModelIsEnglishOnly()}
The selected model only supports English in this build.
{:else}
Auto-detect is only meaningful with multilingual models.
{/if}
</p>
</div>
</div>
{/if}
@@ -466,10 +836,10 @@
</div>
{#if editingProfile === i}
<div class="mt-1.5 animate-fade-in">
<textarea
class="w-full h-[120px] bg-bg-elevated border border-border-subtle rounded-lg px-3 py-2
text-[12px] text-text leading-relaxed resize-none
focus:outline-none focus:border-accent"
<textarea
class="w-full h-[120px] bg-bg-elevated border border-border-subtle rounded-lg px-3 py-2
text-[12px] text-text resize-none
focus:outline-none focus:border-accent"
placeholder="One word or phrase per line..."
bind:value={profile.words}
oninput={() => saveProfiles()}
@@ -534,10 +904,10 @@
</div>
{#if editingTemplate === i}
<div class="mt-1.5 animate-fade-in">
<textarea
class="w-full h-[100px] bg-bg-elevated border border-border-subtle rounded-lg px-3 py-2
text-[12px] text-text leading-relaxed resize-none
focus:outline-none focus:border-accent"
<textarea
class="w-full h-[100px] bg-bg-elevated border border-border-subtle rounded-lg px-3 py-2
text-[12px] text-text resize-none
focus:outline-none focus:border-accent"
placeholder="One section per line..."
value={template.sections.join("\n")}
oninput={(e) => { template.sections = e.target.value.split("\n").filter((s) => s.trim()); saveTemplates(); }}
@@ -591,13 +961,13 @@
/>
{#if settings.saveAudio}
<div class="ml-[50px] mt-2 animate-fade-in">
<p class="text-[10px] font-medium text-text-tertiary uppercase tracking-wider mb-1.5">Output Folder</p>
<div class="flex items-center gap-2">
<div class="flex-1 bg-bg-input rounded-lg px-3 py-1.5 border border-border-subtle">
<p class="text-[11px] text-text-secondary truncate">
{settings.outputFolder || "Default (app data)"}
</p>
</div>
<p class="text-[10px] font-medium text-text-tertiary uppercase tracking-wider mb-1.5">Output Folder</p>
<div class="flex items-center gap-2">
<div bind:this={outputFolderEl} class="flex-1 bg-bg-input rounded-lg px-3 py-1.5 border border-border-subtle">
<p class="text-[11px] text-text-secondary whitespace-pre-wrap break-words" style="line-height: {settingsPathLineHeight}px" title={settings.outputFolder || "Default (app data)"}>
{outputFolderPreview}
</p>
</div>
<button
class="text-[11px] text-accent hover:text-accent-hover whitespace-nowrap"
onclick={async () => {
@@ -723,6 +1093,46 @@
</div>
<p class="text-[11px] text-text-tertiary mt-5">Kon v1.0 · Powered by whisper.cpp · Built by CORBEL Ltd</p>
<!-- Diagnostic report (privacy posture: local only until you share it) -->
<div class="mt-6 pt-5 border-t border-border-subtle">
<p class="text-[10px] font-medium text-text-tertiary uppercase tracking-wider mb-2">Diagnostics</p>
<p class="text-[11px] text-text-tertiary mb-3">
If something goes wrong, generate a diagnostic report to share with the developer. The report contains your settings, recent errors, any crash dumps, and the tail of the log file. Nothing is sent automatically — you preview it first, then choose copy or save.
</p>
<div class="flex gap-2 mb-3">
<button
type="button"
onclick={generateDiagnosticReport}
disabled={diagnosticReportLoading}
class="px-3 py-2 text-[12px] bg-accent text-bg rounded-lg disabled:opacity-50 hover:bg-accent-hover"
>{diagnosticReportLoading ? "Generating…" : "Generate report"}</button>
{#if diagnosticReport}
<button
type="button"
onclick={copyDiagnosticReport}
class="px-3 py-2 text-[12px] text-text border border-border rounded-lg hover:border-accent hover:text-accent"
>Copy to clipboard</button>
<button
type="button"
onclick={saveDiagnosticReport}
class="px-3 py-2 text-[12px] text-text border border-border rounded-lg hover:border-accent hover:text-accent"
>Save as file</button>
{/if}
</div>
{#if diagnosticReportError}
<p class="text-[11px] text-error mb-2">{diagnosticReportError}</p>
{/if}
{#if diagnosticReportSavedTo}
<p class="text-[11px] text-success mb-2">{diagnosticReportSavedTo}</p>
{/if}
{#if diagnosticReport}
<details class="text-[11px]">
<summary class="cursor-pointer text-text-secondary hover:text-text">Preview report ({diagnosticReport.length} chars)</summary>
<pre class="mt-2 p-3 bg-bg-input border border-border rounded-lg text-[10px] text-text-secondary overflow-auto max-h-[400px] whitespace-pre-wrap font-mono">{diagnosticReport}</pre>
</details>
{/if}
</div>
</div>
{/if}
</div>

View File

@@ -33,6 +33,9 @@ const defaults = {
outputFolder: "",
globalHotkey: "Ctrl+Shift+R",
sidebarCollapsed: false,
// Empty string = let backend auto-select. Otherwise, exact device name as
// returned by `list_audio_devices`. Set via Settings → Audio → Microphone.
microphoneDevice: "",
};
function loadSettings() {
@@ -72,7 +75,17 @@ export function saveProfiles() {
} catch {}
}
// ---- History (persisted to localStorage) ----
// ---- History (dual-write: SQLite via Tauri = canonical, localStorage = read-cache) ----
//
// As of Day 4 of the upgrade plan, the canonical store is the SQLite
// transcripts table reachable via the `add_transcript`, `list_transcripts`,
// `update_transcript`, `delete_transcript`, `search_transcripts` Tauri
// commands. localStorage continues to back the in-memory `history` array
// for UI snappiness and offline browser-preview support, but the source
// of truth is SQLite. Browser preview falls back to localStorage only.
import { invoke } from "@tauri-apps/api/core";
import { hasTauriRuntime } from "$lib/utils/runtime.js";
const HISTORY_KEY = "kon_history";
@@ -92,15 +105,86 @@ export function saveHistory() {
} catch {}
}
export function addToHistory(entry) {
/**
* Add a transcript to history. Dual-writes to SQLite (canonical) and
* localStorage (cache). The SQLite write is best-effort: if it fails,
* we still update the in-memory + localStorage copy so the UI stays
* responsive. The next session boot reconciles by reading SQLite first.
*
* `entry` shape:
* { id, text, source?, title?, audioPath?, duration?, engine?, modelId?,
* inferenceMs?, sampleRate?, audioChannels?, formatMode?,
* removeFillers?, britishEnglish?, antiHallucination?, ...uiOnlyFields }
*/
export async function addToHistory(entry) {
history.unshift(entry);
if (history.length > 100) history.length = 100;
saveHistory();
if (!hasTauriRuntime()) return; // Browser preview: localStorage only.
try {
await invoke("add_transcript", {
transcript: {
id: String(entry.id),
text: entry.text ?? "",
source: entry.source ?? "microphone",
title: entry.title ?? null,
audioPath: entry.audioPath ?? null,
duration: Number(entry.duration ?? 0),
engine: entry.engine ?? null,
modelId: entry.modelId ?? null,
inferenceMs: entry.inferenceMs ?? null,
sampleRate: entry.sampleRate ?? null,
audioChannels: entry.audioChannels ?? null,
formatMode: entry.formatMode ?? null,
removeFillers: !!entry.removeFillers,
britishEnglish: !!entry.britishEnglish,
antiHallucination: !!entry.antiHallucination,
},
});
} catch (err) {
console.warn("addToHistory: SQLite dual-write failed, kept in localStorage", err);
}
}
/**
* Update text and/or title of an existing transcript. Persists to SQLite
* via update_transcript; updates the in-memory + localStorage cache.
* Closes the historic "rename never persists" bug from
* architecture-review.md §13.
*/
export async function renameHistoryEntry(id, updates) {
const idx = history.findIndex(h => String(h.id) === String(id));
if (idx >= 0) {
if (updates.title !== undefined) history[idx].title = updates.title;
if (updates.text !== undefined) history[idx].text = updates.text;
saveHistory();
}
if (!hasTauriRuntime()) return;
try {
await invoke("update_transcript", {
id: String(id),
text: updates.text ?? null,
title: updates.title ?? null,
});
} catch (err) {
console.warn("renameHistoryEntry: SQLite update failed", err);
throw err;
}
}
export function deleteFromHistory(index) {
const entry = history[index];
history.splice(index, 1);
saveHistory();
if (!hasTauriRuntime() || !entry?.id) return;
invoke("delete_transcript", { id: String(entry.id) })
.catch(err => console.warn("deleteFromHistory: SQLite delete failed", err));
}
// ---- Tasks (persisted to localStorage) ----

View File

@@ -1,5 +1,6 @@
// src/lib/stores/preferences.svelte.js
import { invoke } from '@tauri-apps/api/core';
import { toasts } from './toasts.svelte.js';
const DEFAULTS = {
theme: 'dark',
@@ -74,13 +75,25 @@ function applyToDOM(prefs) {
}
let saveTimeout = null;
// Show the failure toast at most once per process so a stuck SQLite path
// doesn't spam the user every time they nudge a slider.
let saveFailureToastShown = false;
function persistToSQLite(prefs) {
clearTimeout(saveTimeout);
saveTimeout = setTimeout(async () => {
try {
await invoke('save_preferences', { preferences: JSON.stringify(prefs) });
saveFailureToastShown = false;
} catch (e) {
console.error('Failed to save preferences:', e);
if (!saveFailureToastShown) {
const msg = typeof e === 'string' ? e : (e?.message ?? String(e));
toasts.warn(
'Could not save preferences',
`${msg}. Your changes still apply for this session.`,
);
saveFailureToastShown = true;
}
}
}, 500);
}

View File

@@ -0,0 +1,83 @@
// Minimal toast store. Roll-our-own (no svelte-french-toast etc) so Kon
// stays lean. Toasts auto-dismiss after `duration` ms unless duration is 0
// (sticky) or unless the user clicks the close button.
//
// Severity → colour:
// info → moss (informational, default 4s)
// success → moss (operation succeeded, default 3s)
// warn → signal (degraded but not failed, default 6s)
// error → ember (operation failed, sticky until dismissed)
//
// Usage from a component:
// import { toasts } from '$lib/stores/toasts.svelte.js';
// toasts.error('Could not start recording', 'Selected microphone disappeared.');
// toasts.success('Saved');
let nextId = 1;
function defaultDuration(severity) {
switch (severity) {
case 'success': return 3000;
case 'info': return 4000;
case 'warn': return 6000;
case 'error': return 0; // sticky
default: return 4000;
}
}
function createToastsStore() {
// $state requires a class field or top-level let in Svelte 5; we expose
// `items` via a getter on the singleton.
let items = $state([]);
function show(severity, title, body) {
const id = nextId++;
const duration = defaultDuration(severity);
const toast = { id, severity, title, body: body ?? '', duration };
items.push(toast);
if (duration > 0) {
setTimeout(() => dismiss(id), duration);
}
return id;
}
function dismiss(id) {
const ix = items.findIndex(t => t.id === id);
if (ix >= 0) items.splice(ix, 1);
}
function dismissAll() {
items.length = 0;
}
return {
get items() { return items; },
info: (title, body) => show('info', title, body),
success: (title, body) => show('success', title, body),
warn: (title, body) => show('warn', title, body),
error: (title, body) => show('error', title, body),
dismiss,
dismissAll,
};
}
export const toasts = createToastsStore();
// Helper: wrap a Tauri invoke and toast on failure. Returns the result on
// success, throws on failure (so callers that need to handle the error
// can still try/catch).
//
// import { invoke } from '@tauri-apps/api/core';
// import { invokeWithToast } from '$lib/stores/toasts.svelte.js';
// const result = await invokeWithToast('start_native_capture', { deviceName });
//
// Optional `errorTitle` overrides the default ("Action failed").
export async function invokeWithToast(invokeFn, command, args, errorTitle) {
try {
return await invokeFn(command, args);
} catch (err) {
const msg = typeof err === 'string' ? err : (err?.message ?? String(err));
toasts.error(errorTitle ?? 'Action failed', msg);
throw err;
}
}

View File

@@ -0,0 +1,28 @@
const PRETEXT_FONT_FAMILIES = {
lexend: "'Lexend'",
atkinson: "'Atkinson Hyperlegible Next'",
opendyslexic: "'OpenDyslexic'",
};
export function pretextFontFamily(fontFamily = "lexend") {
return PRETEXT_FONT_FAMILIES[fontFamily] || PRETEXT_FONT_FAMILIES.lexend;
}
export function pretextFontShorthand(accessibility = {}, sizePx = 16) {
return `${sizePx}px ${pretextFontFamily(accessibility.fontFamily)}`;
}
export function bodyPretextLineHeight(accessibility = {}, sizePx = 16) {
const ratio = accessibility.lineHeight || 1.5;
return Math.round(sizePx * ratio);
}
export function transcriptPretextFont(accessibility = {}) {
const sizePx = accessibility.transcriptSize || accessibility.fontSize || 16;
return pretextFontShorthand(accessibility, sizePx);
}
export function transcriptPretextLineHeight(accessibility = {}, ratio = 1.85) {
const sizePx = accessibility.transcriptSize || accessibility.fontSize || 16;
return Math.round(sizePx * ratio);
}

100
src/lib/utils/osInfo.js Normal file
View File

@@ -0,0 +1,100 @@
// Cross-platform OS info for the Svelte frontend. Single async fetch,
// cached in memory, exposes synchronous helpers after first await.
//
// Backed by the `get_os_info` Tauri command (commands/diagnostics.rs).
// In browser-preview mode (no Tauri runtime), falls back to a sensible
// default based on navigator.platform.
//
// Usage:
// import { loadOsInfo, osInfo, modKeyLabel, isMac } from '$lib/utils/osInfo.js';
// await loadOsInfo(); // call once at app startup
// modKeyLabel(); // "Cmd" on macOS, "Ctrl" elsewhere
// if (isMac()) { ... }
//
// The store version (osInfo) is reactive — Svelte components can read it
// directly with `$osInfo` once Svelte 5 runes are everywhere, or via
// destructuring on a non-reactive read.
import { invoke } from '@tauri-apps/api/core';
import { hasTauriRuntime } from './runtime.js';
let cached = null;
const FALLBACK_BROWSER_INFO = {
os: detectBrowserOs(),
arch: 'unknown',
family: detectBrowserFamily(),
usesCmd: detectBrowserOs() === 'macos',
isWayland: false,
customHotkeyBackend: false,
primaryModifierLabel: detectBrowserOs() === 'macos' ? 'Cmd' : 'Ctrl',
};
function detectBrowserOs() {
if (typeof navigator === 'undefined') return 'unknown';
const ua = (navigator.userAgent || '').toLowerCase();
const platform = (navigator.platform || '').toLowerCase();
if (platform.includes('mac') || ua.includes('mac os')) return 'macos';
if (platform.includes('win') || ua.includes('windows')) return 'windows';
if (platform.includes('linux') || ua.includes('linux')) return 'linux';
return 'unknown';
}
function detectBrowserFamily() {
switch (detectBrowserOs()) {
case 'macos': return 'macOS';
case 'windows': return 'Windows';
case 'linux': return 'Linux';
default: return 'Unknown';
}
}
/** Fetch OS info from the Tauri backend. Idempotent — call multiple times,
* only the first does the round-trip. Browser-preview mode returns the
* navigator-detected fallback. */
export async function loadOsInfo() {
if (cached) return cached;
if (!hasTauriRuntime()) {
cached = FALLBACK_BROWSER_INFO;
return cached;
}
try {
cached = await invoke('get_os_info');
} catch (err) {
console.warn('loadOsInfo: get_os_info failed, using browser fallback', err);
cached = FALLBACK_BROWSER_INFO;
}
return cached;
}
/** Synchronous accessor. Returns null if loadOsInfo has not been awaited
* yet. Components that need the value at first render should await
* loadOsInfo() in onMount before reading. */
export function osInfo() {
return cached;
}
export function isMac() {
return cached?.os === 'macos';
}
export function isWindows() {
return cached?.os === 'windows';
}
export function isLinux() {
return cached?.os === 'linux';
}
/** Localised label for the primary keyboard modifier. "Cmd" on macOS,
* "Ctrl" everywhere else. Use in hotkey display strings:
* `${modKeyLabel()}+Shift+R` */
export function modKeyLabel() {
return cached?.primaryModifierLabel ?? 'Ctrl';
}
/** True if the current Linux session is Wayland. False on Windows/macOS.
* Useful for Settings → Audio "PipeWire detected" status display. */
export function isWayland() {
return !!cached?.isWayland;
}

18
src/lib/utils/runtime.js Normal file
View File

@@ -0,0 +1,18 @@
// Detects whether the Tauri runtime is present in the current window.
//
// In Tauri v2 the bootstrap script injects `window.__TAURI_INTERNALS__`
// before the page script runs, and also sets `window.isTauri = true` as
// a convenience marker. Either is sufficient evidence we are inside a
// Tauri webview.
//
// In a plain browser (vite preview, `npm run dev` outside `tauri dev`)
// neither global is set, so `hasTauriRuntime()` returns false. Callers
// use this to gate `invoke()` calls and fall back to localStorage-only
// behaviour for browser-preview mode.
export function hasTauriRuntime() {
if (typeof window === 'undefined') return false;
if (window.__TAURI_INTERNALS__) return true;
if (window.isTauri === true) return true;
return false;
}

View File

@@ -0,0 +1,127 @@
import {
prepare,
layout,
prepareWithSegments,
layoutWithLines,
} from '@chenglou/pretext';
// Cache prepared text to avoid re-measuring.
// prepare() is expensive, layout() is the cheap hot path.
const prepareCache = new Map();
const segmentedPrepareCache = new Map();
const MAX_CACHE_ENTRIES = 400;
function touch(cache, key, value) {
if (cache.has(key)) cache.delete(key);
cache.set(key, value);
if (cache.size > MAX_CACHE_ENTRIES) {
const oldestKey = cache.keys().next().value;
cache.delete(oldestKey);
}
return value;
}
function cacheKey(text, font, options = {}) {
return `${text}\0${font}\0${JSON.stringify(options)}`;
}
function getPrepared(cache, factory, text, font, options = {}) {
const key = cacheKey(text, font, options);
if (cache.has(key)) {
const cached = cache.get(key);
return touch(cache, key, cached);
}
return touch(cache, key, factory(text, font, options));
}
export function prepareText(text, font, options = {}) {
return getPrepared(prepareCache, prepare, text, font, options);
}
export function prepareTextWithSegments(text, font, options = {}) {
return getPrepared(
segmentedPrepareCache,
prepareWithSegments,
text,
font,
options,
);
}
/**
* Measure the rendered height of text without DOM reflow.
* Caches the prepare() result per unique text+font combination.
*
* @param {string} text - The text content to measure
* @param {string} font - CSS font shorthand (e.g. "16px Lexend")
* @param {number} maxWidth - Container width in pixels
* @param {number} lineHeight - Line height in pixels
* @param {object} [options] - Additional options (e.g. { whiteSpace: 'pre-wrap' })
* @returns {{ height: number, lineCount: number }}
*/
export function measureTextHeight(text, font, maxWidth, lineHeight, options = {}) {
const prepared = prepareText(text, font, options);
return layout(prepared, maxWidth, lineHeight);
}
/**
* Measure height using pre-wrap mode (textarea-like behaviour).
* Spaces, tabs, and hard breaks are preserved.
*/
export function measurePreWrap(text, font, maxWidth, lineHeight) {
return measureTextHeight(text, font, maxWidth, lineHeight, { whiteSpace: 'pre-wrap' });
}
/**
* Return full line layout information for custom rendering or clipping.
*/
export function layoutTextLines(text, font, maxWidth, lineHeight, options = {}) {
const prepared = prepareTextWithSegments(text, font, options);
return layoutWithLines(prepared, maxWidth, lineHeight);
}
/**
* Clamp text to a maximum number of laid-out lines without DOM reads.
*/
export function clampTextLines(
text,
font,
maxWidth,
lineHeight,
maxLines,
options = {},
) {
const result = layoutTextLines(text, font, maxWidth, lineHeight, options);
const visibleLines = result.lines.slice(0, maxLines);
const didTruncate = result.lineCount > maxLines;
if (!didTruncate) {
return {
...result,
visibleLineCount: result.lineCount,
text,
didTruncate: false,
};
}
const trimmedLines = visibleLines.map((line) => line.text);
const lastLine = trimmedLines[trimmedLines.length - 1] || "";
trimmedLines[trimmedLines.length - 1] =
`${lastLine.replace(/\s+$/g, "")}\u2026`;
return {
...result,
visibleLineCount: visibleLines.length,
text: trimmedLines.join("\n"),
height: visibleLines.length * lineHeight,
didTruncate: true,
};
}
/**
* Invalidate all cached measurements.
* Call when font family, size, or line height changes.
*/
export function invalidateCache() {
prepareCache.clear();
segmentedPrepareCache.clear();
}

View File

@@ -0,0 +1,49 @@
export function buildCumulativeOffsets(heights) {
const offsets = new Array(heights.length + 1);
offsets[0] = 0;
for (let i = 0; i < heights.length; i++) {
offsets[i + 1] = offsets[i] + heights[i];
}
return offsets;
}
export function findVisibleRange(
cumulativeOffsets,
itemCount,
scrollTop,
viewportHeight,
buffer = 4,
) {
if (!cumulativeOffsets.length || itemCount === 0 || viewportHeight <= 0) {
return { start: 0, end: 0 };
}
let lo = 0;
let hi = itemCount - 1;
while (lo < hi) {
const mid = (lo + hi) >>> 1;
if (cumulativeOffsets[mid + 1] < scrollTop) {
lo = mid + 1;
} else {
hi = mid;
}
}
const start = Math.max(0, lo - buffer);
const endScroll = scrollTop + viewportHeight;
lo = start;
hi = itemCount - 1;
while (lo < hi) {
const mid = (lo + hi) >>> 1;
if (cumulativeOffsets[mid] < endScroll) {
lo = mid + 1;
} else {
hi = mid;
}
}
return {
start,
end: Math.min(itemCount, lo + buffer),
};
}

View File

@@ -5,6 +5,9 @@
import Sidebar from "$lib/Sidebar.svelte";
import TaskSidebar from "$lib/components/TaskSidebar.svelte";
import Titlebar from "$lib/components/Titlebar.svelte";
import ToastViewport from "$lib/components/ToastViewport.svelte";
import { hasTauriRuntime } from "$lib/utils/runtime.js";
import { loadOsInfo } from "$lib/utils/osInfo.js";
import { page, settings, saveSettings } from "$lib/stores/page.svelte.js";
import { getPreferences, updatePreferences } from "$lib/stores/preferences.svelte.js";
@@ -13,6 +16,7 @@
let { children } = $props();
const prefs = getPreferences();
const tauriRuntimeAvailable = hasTauriRuntime();
// Detect secondary windows (float, viewer) — they use +layout@.svelte
@@ -49,6 +53,10 @@
let hotkeyBackend = $state("unknown"); // "evdev" | "tauri" | "unavailable"
async function initHotkeyBackend() {
if (!tauriRuntimeAvailable) {
hotkeyBackend = "unavailable";
return;
}
try {
const isWayland = await invoke("is_wayland_session");
if (isWayland) {
@@ -73,6 +81,7 @@
}
async function registerGlobalHotkey(hotkey) {
if (!tauriRuntimeAvailable) return;
if (hotkeyBackend === "unknown") return; // not yet initialised
try {
@@ -106,6 +115,7 @@
// Listen for evdev hotkey events from the Rust backend
let unlistenEvdev = null;
async function setupEvdevListener() {
if (!tauriRuntimeAvailable) return;
const { listen } = await import("@tauri-apps/api/event");
unlistenEvdev = await listen("kon:hotkey-pressed", () => {
if (page.current !== "dictation") page.current = "dictation";
@@ -156,11 +166,58 @@
}
}
// Capture global frontend errors and forward to the Rust error_log via
// log_frontend_error. Best-effort: never let the error handler itself
// throw, never crash the app over a logging failure.
// (Diagnostics layer 1 — local only, never transmitted)
let onWindowError = null;
let onUnhandledRejection = null;
function installGlobalErrorCapture() {
if (!hasTauriRuntime()) return;
const safeLog = (context, message, stack) => {
try {
invoke("log_frontend_error", { context, message: String(message ?? ""), stack: stack ?? null })
.catch(() => { /* swallow — diagnostic logging must never throw */ });
} catch { /* same */ }
};
onWindowError = (ev) => {
safeLog(
"window.onerror",
ev?.message || ev?.error?.message || "Unknown error",
ev?.error?.stack || null,
);
};
onUnhandledRejection = (ev) => {
const reason = ev?.reason;
const msg = (reason && (reason.message || String(reason))) || "Unhandled rejection";
safeLog("unhandledrejection", msg, reason?.stack || null);
};
window.addEventListener("error", onWindowError);
window.addEventListener("unhandledrejection", onUnhandledRejection);
}
onMount(async () => {
// Auto-collapse if window is already narrow on first load
handleResize();
window.addEventListener("resize", handleResize);
// Diagnostics: capture every uncaught frontend error to error_log.
installGlobalErrorCapture();
// OS detection: warm the cache so components can use modKeyLabel() /
// isMac() / isWayland() synchronously after first render.
loadOsInfo().catch(() => { /* fallback already populated */ });
if (!tauriRuntimeAvailable) {
hotkeyBackend = "unavailable";
return;
}
// Detect and initialise the correct hotkey backend
await initHotkeyBackend();
@@ -177,6 +234,11 @@
onDestroy(() => {
window.removeEventListener("resize", handleResize);
if (onWindowError) window.removeEventListener("error", onWindowError);
if (onUnhandledRejection) window.removeEventListener("unhandledrejection", onUnhandledRejection);
if (!tauriRuntimeAvailable) {
return;
}
if (hotkeyBackend === "evdev") {
invoke("stop_evdev_hotkey").catch(() => {});
} else if (hotkeyBackend === "tauri" && registeredHotkey) {
@@ -220,3 +282,8 @@
</div>
</div>
{/if}
<!-- Global toast viewport. Mounted once at the app root so any component
can call toasts.error(...), toasts.success(...) etc and have it render
in the bottom-right of the viewport. (Day 3 of the upgrade plan) -->
<ToastViewport />