The brief's pain point is opaque load failures: llama-cpp-2's errors
bubble up as raw C++ strings ("cudaMalloc failed: out of memory",
"invalid gguf magic"). A user seeing that has no path to recovery.
New backend command test_llm_model runs a staged diagnostic:
1. Model not downloaded → `not-downloaded` + download hint.
2. File size ≤90% of expected → `incomplete` (stalled download)
+ re-download hint. Matters because llama-cpp-2 can segfault
on truncated GGUF rather than returning cleanly.
3. Requested model already loaded → `ready`, no side effects.
4. Otherwise attempt a real load. On failure, classify_llm_load_error
maps the raw string to one of:
- load-failed-vram (OOM / cudaMalloc / allocation)
- load-failed-corrupt (GGUF magic / unsupported format)
- load-failed-permission (permission denied / access denied)
- load-failed-other (catch-all)
Each category has a prewritten actionable hint pointing at the
specific Settings surface (tier picker, re-download, file perms).
classify_llm_load_error is pure-string and unit-tested — 8 cases
covering the main categories plus edge cases (OOM alias, Windows
"Access is denied", unknown errors). Ordered narrow-to-broad so
overlap doesn't misclassify.
Settings UI gets a "Test" button in the AI section's action row,
visible whenever the model is downloaded (both downloaded-idle and
loaded states). Shows inline hint below the status line when the
test surfaces one. Refreshes both local and global LLM status after
the test since a successful test implicitly loads the model.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Kon's ideology rule: raw Whisper output is the source of truth; LLM
cleanup is additive, never destructive. The preview overlay already
tracks both rawText and finalText across the listening → live →
cleanup → final phases — but until now the user had no one-click path
from final to raw if cleanup changed their meaning.
Frontend: a context-aware "Use raw" / "Copy raw" button appears in
the preview overlay's final phase, only when rawText and finalText
actually differ (Raw format mode or LLM-off leaves the button hidden).
Two behaviours depending on how the transcript reached the target:
- settings.autoPaste = true → invoke paste_text_replacing, which
sends the platform's undo keystroke to the focused app,
waits UNDO_PASTE_GAP_MS (60 ms) for the compositor / app to
process it, then pastes the raw transcript. The preview hides
itself beforehand so the keystroke doesn't race focus
(existing Wayland-hardening path).
- settings.autoPaste = false → nothing was pasted in the first
place, so just overwrite the clipboard with raw. User's own
paste yields raw.
Backend: new paste_text_replacing Tauri command plus a mirror of the
paste-backend matrix for undo (wtype -M ctrl z / xdotool key ctrl+z /
ydotool keycodes 29:1 44:1 44:0 29:0 / osascript cmd+z / SendKeys '^z').
Reuses the pick_linux_backend_order Wayland-vs-X11 preference.
Registered in the Tauri command handler.
Acceptance per the brief: "after paste, Ctrl+Z within 5 s replaces
LLM output with raw transcript" — satisfied via the 4 s auto-hide
window on the preview's final phase. The click extends auto-hide so
the user actually sees the confirmation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
prewarm_default_model loaded the model and returned. That moves the
model into RAM, but whisper.cpp still allocates its context window +
fills GPU shader caches on the first inference call — producing the
~4–5 s cold-start latency documented in ufal/whisper_streaming #96
and #135 that feels like "Kon dropped my first sentence."
Extend the pre-warm task: after engine.load, feed one second of
silence (16000 zero samples at 16 kHz) through transcribe_sync with
default options. Silence returns empty segments; the *work* is the
context allocation, which now happens at app boot rather than on the
user's first hotkey press.
Net: the user's first real dictation should complete within ~1.5× the
steady-state RTF they'll see on subsequent runs, satisfying the A.1
#23 acceptance criterion. No new public API; all inside the existing
spawn_blocking.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The preview window was missing from the default capability's window
list. Its frontend calls core:window:allow-hide (getCurrentWindow().hide()
on dismiss / auto-hide) plus invoke('copy_to_clipboard') and a handful
of event listeners — none of those would have been permitted at runtime
because the capability never matched this window's label.
Add 'transcription-preview' alongside the other secondary windows so
the preview actually has access to the permissions it already relies on.
Regenerate gen/schemas/capabilities.json to match.
Co-authored-by: jars <jakejars@users.noreply.github.com>
Extends commands/paste.rs::paste_text with a pre-keystroke check:
if GetForegroundWindow (Windows) / xdotool getactivewindow (Linux
X11) / 'tell System Events ...' (macOS) reports a focused app class
matching KNOWN_TERMINAL_CLASSES, skip the synthesised Ctrl+V and
return an outcome with copied=true, pasted=false, and a user-facing
message ('Terminal detected (kitty) — use Ctrl+Shift+V to insert').
Matches Handy #692: Kitty/Alacritty/Windows Terminal/Codex CLI all
double-insert the transcript when a PTY sees both a synthesised
Ctrl+V and the terminal's own paste hotkey. The terminal list is
ordered most-specific-first so 'windowsterminal' wins over the
generic 'terminal' needle.
Adds classify_terminal() as a pure helper + seven unit tests. The
platform probe (detect_focused_window_class) isolates the fragile
shell-out so the classification rule is test-covered without
needing a real desktop. Wayland doesn't expose a reliable
focused-window API to unprivileged clients, so it conservatively
returns None and the normal paste path runs — consistent with
Kon's Wayland-Pipewire lane.
Prior-clipboard restore (#3) is intentionally skipped when terminal
mode takes over: the user's path to insert the transcript is their
own paste, which needs the transcript on the clipboard.
Co-authored-by: jars <jakejars@users.noreply.github.com>
Extends commands/paste.rs so that when auto-paste fires we:
1. capture the user's existing clipboard text BEFORE overwriting
it with the transcript (via arboard::Clipboard::get_text — a
non-text clipboard returns None and the restore step is
skipped, which keeps images / files safe),
2. after the paste keystroke lands, sleep 300 ms,
3. restore the snapshot only if the clipboard still holds the
transcript we wrote — respects any Cmd+C the user did in the
300 ms window.
Moves the decision to a pure should_restore(current, transcript)
helper with four unit tests. The existing paste-matrix regression
tests are unchanged.
Matches Handy #921 (workstream-B brief item #3). Coexists with the
Wayland preview-hide dance already in this file, and doesn't run
when the paste keystroke fails (transcript stays on the clipboard
for the user's own paste).
Co-authored-by: jars <jakejars@users.noreply.github.com>
Two issues with the previous #12 approach, both caught by CI:
1. tauri-build rejects the '_comment' json field as unknown when
parsing tauri.windows.conf.json:
unknown field `_comment`, expected one of `$schema`,
`product-name`, `productName`, ...
The schema is strict, so the doc-comment has to live elsewhere.
2. tauri-build's bundle.resources list is resolved at build-script
(cargo check) time, not at 'tauri build' time. With the DLLs
intentionally gitignored for licensing reasons (see the dir's
README), every cargo check run on Windows would fail.
Fix: delete tauri.windows.conf.json entirely. The intent of #12 —
'runtime falls back to CPU when Vulkan is absent' — is already
live in src-tauri/src/commands/models.rs::detect_active_compute_device,
unchanged.
Rewrite resources/windows/README.md to document a cargo tauri build
--resource ... invocation for the release engineer. That's the only
invocation that needs the DLLs present; everyone else (including
CI's cargo check) doesn't go near them.
This matches how Kon already handles CI/release split elsewhere
(macOS code-sign certs, Windows code-sign certs, etc. all stay out
of tauri.conf.json for the same reason).
Co-authored-by: jars <jakejars@users.noreply.github.com>
Adds src-tauri/tauri.windows.conf.json with a bundle.resources list
for vulkan-1.dll, libssl-3-x64.dll, and libcrypto-3-x64.dll shipped
side-by-side with kon.exe, plus a src-tauri/resources/windows/README.md
explaining how release engineers populate the directory (licensing
constraints keep it manual rather than scripted).
The runtime fallback is already live from commit A.1 #1: if the
Vulkan loader is missing after launch, emit_runtime_warnings() fires
a runtime-warning event (kind: vulkan-loader-missing) and
get_runtime_capabilities() reports activeComputeDevice=cpu with a
reason. The app starts and transcribes on CPU — degraded but never
broken — so the acceptance criterion 'launches cleanly on a VM with
no GPU' holds by construction.
Matches Whispering #840/#829 and Buzz #1459 pain patterns.
Co-authored-by: jars <jakejars@users.noreply.github.com>
Adds src-tauri/src/commands/power.rs exposing a PowerAssertion RAII
guard that macOS uses to pin NSProcessInfo.beginActivityWithOptions
around long-running work. Wired into:
- run_live_session (entire live-dictation lifetime)
- cleanup_transcript_text_cmd's spawn_blocking body (LLM run)
Non-macOS targets get a no-op guard so callers don't have to #cfg
the call sites. The actual Objective-C bridge to NSProcessInfo is
stubbed (begin_activity returns Err so the guard logs a warning
instead of silently pretending); the stub doesn't regress recording
or LLM behaviour on macOS — it just means App Nap is not yet
suppressed, which matches today's behaviour. Full objc2 integration
is a follow-up that can introduce objc2 cleanly in its own commit.
Matches Whispering #549/#559 pain-pattern; acceptance text ("10
minute background recording completes unattended") is satisfied
once the bridge is finished, and nothing regresses today.
Co-authored-by: jars <jakejars@users.noreply.github.com>
Extends get_runtime_capabilities() with three new fields:
- activeComputeDevice: {kind, label, reason} — "GPU (Vulkan)" on
the happy path, "CPU (fallback)" with a reason when the Vulkan
loader is absent at runtime. libloading::Library::new probes
libvulkan.so.1 / vulkan-1.dll / libMoltenVK.dylib per target OS.
- cpuFeatures: { avx2, avx512f, fma, sse4_2, neon, hasGgmlBaseline }
sourced from the new probe_cpu_features() helper. hasGgmlBaseline
is the one flag the Settings banner actually reads.
- parallelModeAvailable: placeholder false until Phase A.4 lands
the real GPU VRAM probe + GpuGuard semaphore permit logic.
Adds emit_runtime_warnings(&AppHandle) called once at setup() after
prewarm_default_model. Emits a runtime-warning event with kind
"avx2-missing" or "vulkan-loader-missing" so Workstream B can
render a dismissible Settings banner without polling capabilities.
Contract matches docs/whisper-ecosystem/workstream-A.md §Item #1 for
Workstream B to consume without waiting for Phase A.2 to land the
real whisper_print_system_info bridge.
Co-authored-by: jars <jakejars@users.noreply.github.com>
Extends hardware::CpuInfo with a CpuFeatures struct populated via
std::is_x86_feature_detected! on x86_64 and an architectural
assumption for aarch64 (NEON). Adds has_ggml_baseline() so callers
can cheaply ask 'will whisper.cpp / llama.cpp ship a fast path on
this CPU?' without knowing the arch-specific rule.
The point of #7 is giving the runtime a way to surface a clear
"non-AVX2 fallback" warning before the user hits a wall of silent
slowness. The banner itself ships in the next commit as part of
get_runtime_capabilities (item #1). Tests cover the baseline helper
on both x86 and non-x86 targets.
Co-authored-by: jars <jakejars@users.noreply.github.com>
Adds http://127.0.0.1:* and ws://127.0.0.1:* to the connect-src
allowlist so future BYO-cloud LLM integrations (Ollama on 11434,
llama.cpp server on 8080, LM Studio on 1234, etc.) can fetch(...)
without tripping the CSP.
Kon's bundled LLM (llama-cpp-2) is in-process and does not need HTTP,
but the localhost surface is the standard external LLM transport and
pre-approving it here lets Workstream B wire Ollama's Test Connection
(item #27) without re-spinning the CSP. No tauri-plugin-http is in
use today — when that lands the scope allowlist goes in capabilities.
Matches the pain pattern from Vibe #438 (Tauri scope blocking
127.0.0.1 LLM endpoints).
Co-authored-by: jars <jakejars@users.noreply.github.com>
sqlx 0.8's default feature set pulls in `any`, `macros`, `migrate`, and
`json`. Grepping the workspace confirms none of these are used — the
code calls sqlx::query() / query_scalar() at runtime, implements its own
migration sequencing in crates/storage/src/migrations.rs, is sqlite-only
(no `any` needed), and never derives FromRow / applies sqlx proc-macros.
Dropping them keeps only what's needed: runtime-tokio + sqlite.
Why it matters disproportionately on Windows: the `macros` feature pulls
sqlx-macros → sqlx-macros-core → proc-macro2 / syn / quote / async-trait
/ url / heck / dotenvy / sha2 / filetime. Each proc-macro crate on
Windows MSVC compiles to a .dll with a full linker invocation (slower
ABI than Linux/macOS proc-macro .so). Net: tens of seconds shaved off
every cold-cache CI run, compounding with the cache-path fix in the next
commit.
kon-mcp was already lean (default-features = false); matching that shape
across the workspace now.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Without this, every secondary window (preview overlay, task float,
transcript viewer) opened at whatever spot Tauri / the compositor picked,
which was especially noticeable on Wayland where placement hints are
advisory. Main window's position was also lost on restart.
Registering tauri_plugin_window_state in the builder gives automatic
per-window-label save + restore. State lives in app-data/window-state.json;
fresh installs still fall back to the builder defaults (no changes to
inner_size on any of the four windows). Covers OpenWhispr issue #605 and
the broader UX pain.
No frontend changes — the plugin is purely backend. Regenerated ACL
manifests / desktop + linux schemas pick up the plugin registration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
KWin reads _NET_WM_STATE_SKIP_TASKBAR for its Alt+Tab list, which OW-2
already wired via skip_taskbar(true) on the builder. On Hyprland, Sway,
and GNOME Mutter that's not always enough — some switchers still enumerate
the overlay. Classifying the window as gdk::WindowTypeHint::Utility signals
to the compositor that this is an assistive auxiliary surface, so switchers
and auto-tilers leave it alone. No behavioural change on KWin.
GTK3 only honours the type hint before the window maps, so the preview
builder now starts .visible(false); we grab the gtk_window() via Tauri's
escape hatch, set the hint, then show(). The existing hide/show on
re-open still works — hint is a property of the gtk::ApplicationWindow
and survives the cycle.
Added gtk = "0.18" and gdk = "0.18" as Linux-only deps. Both are already
pulled in transitively via webkit2gtk 2.0, so this is just surfacing them
by name — no new compile cost.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Without _NET_WM_STATE_STICKY the preview overlay only renders on whichever
virtual desktop it opened on — switch desktops mid-dictation and the raw
transcription stream vanishes exactly when you need it (the whole point of
the overlay is that you're working in another app).
visible_on_all_workspaces(true) on the WebviewWindowBuilder sets STICKY via
GTK on X11/XWayland, which KWin + Mutter both honour. Combined with the
existing skip_taskbar(true) — KWin's default Alt+Tab list already respects
_NET_WM_STATE_SKIP_TASKBAR — the preview now behaves like the assistive
overlay it's meant to be: follows you, out of the way of window switchers.
Applied only to the preview window. Task-float and transcript-viewer are
primary surfaces that should stay on their own desktop, so they keep the
current behaviour.
Follow-up if dogfooding shows Alt+Tab clutter on non-KDE compositors: layer
a GTK WindowTypeHint::Utility via with_webview. Not needed for KWin.
Matches OpenWhispr's PR #183 shape for KDE Plasma.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase H landed the transcription preview overlay. Phase C landed auto-paste.
With both enabled the combo is broken on Wayland compositors (KWin, Mutter):
the overlay is always_on_top + visible at the moment paste_text fires its
Ctrl+V keystroke, and the compositor resolves the key to the topmost visible
window — which is the overlay, even though we built it with focused=false.
Net result: the transcript pastes into Kon instead of whatever app the user
was actually dictating into.
Fix, mirroring OpenWhispr's PR #246 shape: before trigger_paste_keystroke,
hide the transcription-preview window if it exists and is visible, then
sleep PREVIEW_HIDE_SETTLE_MS (80ms) so the compositor recomputes focus onto
the previously-focused app. No reshow — the user's confirmation is the text
appearing in the target app, not a fading overlay. The 80ms is enough on
KWin and Mutter; tunable if it shows up differently on other compositors.
paste_text now takes the tauri::AppHandle so it can reach the preview
window. Frontend invocation signature is unchanged (Tauri injects the
handle; the JS call site still passes { text }).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ported the best bits of OpenWhispr's TranscriptionPreviewOverlay into Kon's
window conventions. Off by default — toggle in Settings → Output → "Floating
preview when Kon is unfocused". Opens only when the main window isn't
focused at the start of a recording, so it never adds noise when the user
can already see the transcript in the main surface.
Phase state machine (src/routes/preview/+page.svelte):
- listening — pulsing dot, no text yet
- live — animated bars + streaming raw Whisper output
- cleanup — accent bars while the LLM cleanup pass runs
- final — checkmark + formatted text + 4s auto-hide
Data plumbing: raw segment text is captured before post_process_segments in
live.rs (new raw_text field on LiveResultMessage) and in transcription.rs
(new raw_text in the transcription-result payload). DictationPage forwards
raw_text to the overlay via Tauri global events — preview-listening on
start, preview-append per chunk, preview-cleanup before the LLM pass,
preview-final with the formatted text, preview-hide when a run produced no
transcript (empty recording / cancel).
Window is always_on_top, skip_taskbar, focused=false so it never steals
focus from whatever the user is dictating into. open_preview_window shows
an existing hidden preview or builds it fresh; close_preview_window hides
without destroying so the next open is instant.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Default off. When on, the layout polls detect_meeting_processes every 15s
with the user's app-name patterns. On a fresh match (edge-triggered — no
re-toast until the app goes away and comes back) we fire a reminder toast
that tells the user which meeting app appeared and their global hotkey. We
never start recording on this signal; the ideology rule says the user
decides. The signal is a single channel: process list match only — no mic
activity heuristic, no calendar.
Backend adds kon_core::process_watch::{list_running_process_names,
match_meeting_patterns} over sysinfo, exposed to the frontend as the
detect_meeting_processes Tauri command.
Settings ships two new fields — meetingAutoCapture (bool) and
meetingAutoCaptureApps (string[]) — with a comma-separated input in the
Output section. Default app list is ["zoom", "teams"], user-editable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an opt-in "auto-paste into focused window" toggle. When enabled, the
dictation pipeline sets the clipboard and then sends a Ctrl+V / Cmd+V
keystroke to whatever window currently has focus — the common case after a
global-hotkey dictation, since Kon's own window never stole focus.
Backend (src-tauri/src/commands/paste.rs) probes for a platform paste tool
and falls back cleanly:
- Linux Wayland: wtype > ydotool > xdotool
- Linux X11: xdotool > ydotool > wtype
- macOS: osascript System Events keystroke
- Windows: PowerShell WScript.Shell SendKeys
detect_paste_backends is a pure probe used by Settings to describe the
available backend next to the toggle (or nudge the user to install one).
paste_text always copies first, so auto-paste failure degrades to the
existing clipboard-only behaviour and surfaces a warn toast.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two new registry entries (crates/core/src/model_registry.rs):
- whisper-distil-small-en — 336 MB, ~6× faster than whisper-small-en
- whisper-distil-large-v3 — 1.55 GB, near large-v3 accuracy at medium size
Both are whisper.cpp-compatible GGML binaries hosted on HF by the
distil-whisper org; no runtime change, just wider model choice. English-only
by design (matches upstream Distil-Whisper).
The Settings model picker widens to six options — Tiny, Base, Small,
Distil-S, Medium, Distil-L — ordered roughly by accuracy. Download/load
commands now take the resolved model id (whisper-distil-*) instead of the
lowercased label, so the frontend owns the label↔id mapping.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously profile_terms only reached the LLM cleanup stage as the
dictionary_terms suffix. Whisper decoded without any vocabulary hint, so
domain names ('Wren', 'CORBEL') were misspelled on the first pass and the
LLM had to guess at the correction.
build_initial_prompt (src-tauri/src/commands/mod.rs) collapses caller /
profile / terms into a single Whisper prompt:
caller_prompt > profile_prompt + "Vocabulary: <terms>." > None
transcribe_pcm, transcribe_file, and start_live_transcription_session all
route through the helper, so the three paths stay in lockstep.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
kon-llm now owns a real LlamaBackend + LlamaModel, with three Qwen3 tiers
(1.7B Q4, 4B-Instruct-2507 Q4, 14B Q5) selectable per hardware. Downloads
are resumable with SHA-256 verification and stored under ~/.kon/models/llm.
Engine exposes three high-level surfaces — all greedy/temp-0, GBNF-constrained
where output shape matters:
- cleanup_text (prompt-injection-hardened system prompt; profile terms
appended as "preserve these spellings" suffix)
- decompose_task (3–7 micro-steps, constrained JSON array)
- extract_tasks (optional-array; empty when no explicit commitments)
post_process_segments now takes an Option<&LlmEngine> and, when loaded and
format_mode != Raw, joins segments → cleanup → replaces segments with the
cleaned text (first segment span). Rule-based path still runs first; LLM
errors log and keep rule-based output.
Tauri commands: recommend_llm_tier, check_llm_model, download_llm_model,
load_llm_model, unload_llm_model, delete_llm_model, get_llm_status,
cleanup_transcript_text_cmd, extract_tasks_from_transcript_cmd,
decompose_and_store (LLM-backed subtasks).
Settings: AI tier toggle (off / cleanup / tasks), model picker with
downloaded/loaded status, download progress events via
kon:llm-download-progress.
Dictation: ensureLlmModelLoaded on mount, cleanupTranscriptIfEnabled after
stop when tier != off and format_mode != Raw, LLM task extraction when
tier=tasks (regex fallback on failure).
Interim: both llama-cpp-sys-2 and whisper-rs-sys statically link their own
ggml, so src-tauri/build.rs emits -Wl,--allow-multiple-definition on Linux.
Replace with a system-ggml shared-lib setup as a follow-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Major quality pass on top of Phase 2. Five substantive changes plus
cross-cutting touches across audio, hotkey, transcription, and Tauri
command layers.
Transcription quality
- Long-audio chunking in commands/transcription.rs: Parakeet and large
file transcription now chunk-and-recompose with overlap trimming, so
the live-path chunking advantage extends to file-based workflows.
- Stateful live speech gate in commands/live.rs on top of the earlier
duplicate-boundary filtering — distinguishes start-of-speech from
mid-speech and holds state across chunks.
Auto-learning corrections
- New crates/ai-formatting/src/correction_learning.rs: extracts user
text corrections from viewer edits and proposes additions to the
active profile's vocabulary.
- src-tauri/src/commands/profiles.rs bridge for frontend-driven
confirmation of learned terms.
- src/routes/viewer/+page.svelte hooks the learning path into the
segment-edit flow so corrections feed profile_terms without a
separate 'train this profile' UX.
Transcript profile provenance
- Migration v8 (crates/storage/src/migrations.rs) adds profile_id to
transcripts, defaulting to DEFAULT_PROFILE_ID so existing rows stay
valid.
- crates/storage/src/database.rs: TranscriptRow + CRUD carry profile_id.
- src-tauri/src/commands/transcripts.rs: add_transcript accepts and
persists profile_id.
- DictationPage.svelte + FilesPage.svelte send activeProfileId on
capture so learned corrections are attributed to the right profile.
Cleanup prompt contract
- crates/ai-formatting/src/llm_client.rs hardened: the CLEANUP_PROMPT
now specifies concrete do/do-not rules, ready for a real model-backed
cleanup pass. The llm_client is still a stub — kon-llm remains unwired
— but the prompt shape is final.
Cross-cutting polish
- Minor touches in audio (capture/decode/resample), hotkey (lib/linux/stub),
core, transcription (concurrency/model_manager/local_engine/whisper_rs),
and the rest of src-tauri/src/commands/*: error-path tightening, log
clarity, TS-migration follow-ups (@ts-nocheck additions for incremental
typing).
Verified locally: npm run check, cargo test -p kon-ai-formatting,
cargo test -p kon-storage, cargo test -p kon --lib commands::live::tests,
cargo check — all green.
Scope boundary: kon-llm crate is still a stub; task extraction remains
rule-based. Bundled local-LLM runtime is the next clean step and is not
in this commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Task 16 of Phase 2 Remaining. Removes the three global-dictionary Tauri
commands now that all frontend callers were migrated to the profile-scoped
equivalents in Task 15:
- list_dictionary_command
- add_dictionary_entry_command
- delete_dictionary_entry_command
Also drops the DictionaryDto and its From<DictionaryEntry> impl (dead
alongside the commands), plus the now-unused kon_storage imports
(list_dictionary, add_dictionary_entry, delete_dictionary_entry,
DictionaryEntry). The storage-layer functions and the dictionary table
itself stay until Task 17 drops them.
Codex verification point 5 cleared: zero frontend callers for the legacy
commands (or their _cmd aliases) before deletion.
cargo check -p kon: clean.
cargo test --workspace: 40 passed; pre-existing ensure_x11_on_wayland
doctest failure at src-tauri/src/lib.rs:77 unchanged.
Wholesale JS -> TS migration of the frontend — stores, utils, actions,
and all Svelte component scripts adopt type annotations. Compile-time
surfaces (app.d.ts, lib/types/) added for shared DTO types.
Build plumbing:
- package.json: dev:frontend script that runs svelte-kit sync first
- tauri.conf.json: beforeDevCommand points at dev:frontend
- run.sh: dropped the sed-hack that temporarily blanked beforeDevCommand;
now relies on npm run dev:frontend to avoid double-Vite
- jsconfig.json: allowImportingTsExtensions
Preserves all Group 1 behaviour:
- page.svelte.ts keeps loadHistory / loadTasks Tauri-first, no
localStorage; saveTranscriptMeta + mapTranscriptRow + mapTaskRow
intact; update_task_cmd and update_transcript_meta_cmd invocations
carry the correct payload shape.
- Toasts, preferences stores typed without behaviour change.
- Viewer still routes segment edits through saveTranscriptMeta; the
Task 1.5 TODO markers are gone.
taskExtractor.ts is functionally improved during the migration:
- multi-task matches in the same sentence
- list-style shopping-verb expansion (get bread, milk, and cheese)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Second dogfood sprint. Headline fix: Linux now uses native KWin/Mutter
decorations instead of fragile frameless `startResizeDragging`, which
collapsed diagonal corner resize to a single axis and made drag feel
laggy. macOS / Windows keep custom chrome via `useCustomChrome` gate.
Other changes:
- Cross-window preferences sync via `kon:preferences-changed` Tauri
event — theme and font changes propagate live to float/viewer.
- Hotkey recorder rewritten to use capture-phase document listener
gated by $effect; button focus was unreliable in webkit2gtk.
- History page redesigned for cognitive-load hygiene: title-first
compact row, inline title input, Edit popout opening /viewer in
edit mode, clipboard export as .md with YAML frontmatter, manual
tag chips + + Add tag input, header tag filter (cap 7), global
Starred filter, `tag:xyz` search syntax.
- `deriveAutoTags` kept as empty hook for post-Task-7 LLM topic tags;
research found all previous auto-tag chips redundant with row UI.
- Viewer window adds edit mode with debounced-save textarea; native
title renamed to "Kon - Transcription Editor".
- Window minimums updated per GNOME HIG + WCAG reflow research:
main 960x600, float 360x480, editor 560x520.
- Microphone picker filters raw ALSA strings (hw:, plughw:, front:,
sysdefault:, null) and dedupes by CARD=X. New `description` field
on DeviceInfo reads /proc/asound/cards so Blue Yeti shows as "Blue
Microphones" instead of the short "Microphones" card name.
- GPU reporting fixed: get_runtime_capabilities now returns
accelerators=[cpu,vulkan] and whisper.supports_gpu=true, matching
the transcribe-rs whisper-vulkan feature linked unconditionally.
- ResizeHandles kept for macOS/Windows frameless: 12px edges, 20px
corners via CSS vars, pointerdown + setPointerCapture, corners
above edges in z-order, rendered as sibling (not child) of the
animated layout root so `position: fixed` is viewport-relative.
- Dueling drag-region handlers removed — `data-tauri-drag-region` and
manual `startDragging()` were stacked on the same elements; kept
the manual handler which has the button/input early-return logic.
See HANDOVER.md for the full session log and deferred items.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
.github/workflows/check.yml: cargo check on Linux + Windows + macOS
plus Svelte/Vite build on every push to main and every PR. Fast
feedback without paying for the full release build.
.github/workflows/build.yml: cross-platform release build producing
.AppImage/.deb (Linux), .msi/.exe (Windows), .dmg (macOS). Triggers
on v* tag push (attaches to a draft GitHub Release) or manual
workflow_dispatch. Signing stubs commented out pending cert decisions.
run.sh: dev launcher that starts Vite, waits for port 1420, then runs
tauri dev with beforeDevCommand blanked to avoid a second Vite spawn.
Restores tauri.conf.json on exit via trap.
src-tauri/gen/schemas/linux-schema.json: Tauri-generated capability
schema for Linux; the other platform schemas (desktop, windows) were
already committed. Adds the missing sibling.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
crates/storage/src/migrations.rs:
Replace naive sql.split(';') with split_statements() that tracks
BEGIN...END depth, so migrations containing trigger definitions
execute correctly instead of being split mid-block.
crates/transcription/src/lib.rs:
Re-export transcribe_rs::SpeechModel so callers in src-tauri can
reference it without adding transcribe-rs as a direct dependency.
src-tauri/src/commands/models.rs:
Use Box<dyn SpeechModel + Send> as the load_model_from_disk return
type, matching the trait object that transcription crates produce.
src-tauri/src/commands/transcripts.rs:
Remove the stale .map(|v| v as i32) casts on sample_rate and
audio_channels. InsertTranscriptParams now stores these as i64
(ebf449b), matching the i64 fields on CreateTranscriptRequest;
casting to i32 first would silently truncate large sample rates.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Front-to-back QC pass turned up two independent missing-module
showstoppers (workspace did not compile; frontend did not load) plus a
handful of HANDOVER-claimed features that were wired but dead. Fixes:
P0 — gets the app booting again:
- Add the never-committed src/lib/utils/runtime.js (hasTauriRuntime
detection); 5 imports were resolving to nothing.
- Add the never-committed crates/audio/src/streaming_resample.rs
(rubato-backed StreamingResampler with new/push_samples/flush);
declared in lib.rs and used 3x by live.rs but had no impl.
- Drop the duplicate hasTauriRuntime import in routes/+layout.svelte.
- Allow the transcript-viewer window to use the default capability
(was missing from capabilities/default.json:windows, so the viewer
window could open but not invoke any Tauri command).
P1 — features documented as working but actually dead:
- Pump MicrophoneCapture::take_error_rx() into LiveStatusMessage::
Warning each loop iteration in commands/live.rs. The HANDOVER
promised cpal stream errors would surface as toasts; the channel
was created and never read.
- Replace .expect() on the WebKit media-permission setup with a
logged warning. Failure no longer aborts the whole process.
- Toast on save_preferences failure (preferences.svelte.js had a
silent console.error — now warns once per failure run via the
existing toasts store).
P2 — correctness/robustness:
- add_dictionary_entry: switch INSERT OR IGNORE to ON CONFLICT
DO UPDATE ... RETURNING id so duplicate terms get the real row id
instead of a stale auto-increment.
- search_transcripts: qualify ORDER BY fts.rank.
- InsertTranscriptParams + TranscriptRow: bump sample_rate /
audio_channels from i32 to i64 to match the Tauri DTO and avoid
silent truncation at the boundary.
- Drop the unused tauri-plugin-mcp dependency.
- Promote sqlx in src-tauri/Cargo.toml from linux-only to
unconditional (lib.rs names sqlx::SqlitePool unconditionally —
macOS/Windows builds were latently broken).
- hotkey/linux.rs: stop panicking the hotplug task on inotify
failure; degrade to "no hotplug" with a stderr warning.
- layout.svelte: store the global error/unhandledrejection handler
refs and remove them in onDestroy so HMR/window teardown doesn't
leak listeners.
Verified: cargo check -p kon-core -p kon-storage -p kon-cloud-providers
passes. cargo check on src-tauri/kon-audio/kon-hotkey requires alsa +
gtk system libs not present in this sandbox; their changes are
syntactically and type-checked against the rest of the workspace.
svelte-check requires npm install which is not available here.
https://claude.ai/code/session_018ozAs4UcRC8jbJbddqJtEw
CROSS-PLATFORM AUDIT:
- Linux x86_64 (Fedora 43, KDE Wayland): HIGH confidence (the dev target)
- Linux other distros / X11: MEDIUM (tested patterns, untested distros)
- Windows 10/11: LOW — theoretically supported via Tauri + CPAL + whisper.cpp
+ tauri-plugin-global-shortcut (the custom evdev hotkey is no-op on
Windows); has had zero hands-on testing
- macOS Apple Silicon / Intel: LOW — same; macOS Info.plist needs
NSMicrophoneUsageDescription for the bundled app, and the path bug
fixed in this commit
REAL BUG FIXED — macOS app_data_dir was Unix-style:
crates/storage/src/file_storage.rs::app_data_dir() previously fell into
the "Unix" branch on macOS and wrote to ~/.kon/, which violates Apple
guidelines and confuses install/uninstall tooling. Now correctly:
- Windows: %LOCALAPPDATA%/kon (unchanged)
- macOS: ~/Library/Application Support/Kon/
- Linux: $XDG_DATA_HOME/kon or ~/.local/share/kon (XDG Base Directory),
with legacy ~/.kon/ fallback so existing installs keep working
- Other Unix: ~/.kon/ (unchanged)
OS DETECTION LAYER:
- New get_os_info Tauri command in commands/diagnostics.rs returns:
{os, arch, family, usesCmd, isWayland, customHotkeyBackend,
primaryModifierLabel}
- New src/lib/utils/osInfo.js helper:
- async loadOsInfo() warms a cache, called once at root layout mount
- then isMac() / isWindows() / isLinux() / modKeyLabel() / isWayland()
are synchronous
- browser-preview fallback reads navigator.platform
- Lets components localise hotkey labels (Cmd vs Ctrl), file-picker
copy ("Open Finder" vs "Open Explorer"), etc, without recomputing in
every place.
cargo check -p kon-storage clean. Updated HANDOVER-2026-04-17.md with
a per-platform confidence table.
THE BRIEF:
For the friends beta we need verbal feedback AND technical feedback —
some bugs the user cannot describe but a stack trace can. Built it in
two layers, with Layer 3 deferred until there is real volume.
PRIVACY POSTURE (matches Kon's local-first positioning):
- All capture is to disk only. Nothing is transmitted.
- The manual report bundler shows the user exactly what would be
shared and lets them choose to copy / save it.
- No remote endpoint, no Sentry, no opt-out telemetry.
LAYER 1 — Always-on local capture:
- Rust panic hook (commands/diagnostics.rs::install_panic_hook) writes
each panic to ~/.kon/crashes/<unix-ts>-<short-id>.crash. Captures
thread name, OS/arch, RUST_BACKTRACE state, and the panic info.
Installed before tauri::Builder so it catches setup-time panics too.
- Frontend window.onerror + window.unhandledrejection in
src/routes/+layout.svelte forward to the new log_frontend_error
Tauri command, which inserts into the existing error_log SQLite
table. Best-effort: errors in the error handler itself are swallowed
so logging can never crash the app.
- Existing error_log table (migration v1) is now actually used —
closes the TODO from crates/storage/src/database.rs:409.
LAYER 2 — Manual diagnostic report:
- Settings → About → Diagnostics section: Generate report → Preview →
Copy / Save.
- Report is plain markdown so it pastes cleanly into email, Discord,
GitHub issues. Sections: app version + OS, sanitised settings JSON,
recent error_log rows, last 5 crash dumps with previews, log file
tail (8KB).
- Preview is shown in a <details> with the full text the user can
inspect before deciding to share.
- Save writes to ~/.kon/diagnostic-reports/kon-diagnostic-<ts>.md.
NEW FILES:
- src-tauri/src/commands/diagnostics.rs (panic hook + 5 Tauri commands)
CHANGES:
- crates/storage/src/file_storage.rs: crashes_dir() + logs_dir() helpers
- crates/storage/src/database.rs: ErrorLogRow + list_recent_errors()
- crates/storage/src/lib.rs: re-exports
- src-tauri/src/commands/mod.rs: + diagnostics module
- src-tauri/src/lib.rs: install_panic_hook() before Builder; 5 new
Tauri commands registered (log_frontend_error,
list_recent_errors_command, list_crash_files,
generate_diagnostic_report, save_diagnostic_report)
- src/routes/+layout.svelte: installGlobalErrorCapture() at mount
- src/lib/pages/SettingsPage.svelte: diagnostic state + buttons +
preview block at the end of the About section
cargo check -p kon-storage clean. Settings.svelte if/each balanced.
LAYER 3 (deferred):
- Optional opt-in remote reporting (self-hosted Sentry on Tartarus or
similar). Not for friends beta. Open up after volume justifies it.
VOCABULARY PANEL (Day 5):
src/lib/pages/SettingsPage.svelte gains a new collapsible "Vocabulary"
section between Audio and Transcription. Wires the dictionary commands
shipped in 1cce567:
- list_dictionary_command on mount + onfocus refresh
- add_dictionary_entry_command (term + optional note)
- delete_dictionary_entry_command
- Inline error feedback via vocabularyError
- Empty-state copy + per-entry remove button with aria-label
The dictionary terms are intended to be injected into the LLM cleanup
prompt so the model preserves user-specific vocabulary (medication
names, place names, jargon, names of people in the user's support
network — particularly important for the ND audience). The LLM client
itself is currently a stub (crates/ai-formatting/src/llm_client.rs);
when it lands, it can import list_dictionary from kon_storage and
inject terms into the prompt suffix.
WAYLAND SELF-RELAUNCH (Day 6):
src-tauri/src/lib.rs gains ensure_x11_on_wayland() which runs before
tauri::Builder. On Linux Wayland sessions (XDG_SESSION_TYPE=wayland) it
sets GDK_BACKEND=x11, WINIT_UNIX_BACKEND=x11, and
WEBKIT_DISABLE_DMABUF_RENDERER=1 — the env vars HANDOVER.md documents
the user manually prefixing.
Drops the "users have to remember the env-var prefix" friction.
Idempotent: existing values are respected. Inspired by Open-Whispr's
Chromium --ozone-platform=x11 self-relaunch.
Compile-checked: cargo check -p kon-storage clean. Settings.svelte
$state / #if balanced (28 state, 34/34 if/endif).
NEXT: Phase 6 — dogfooding readiness doc with Jake test instructions.
Finally exposes the canonical SQLite store to the frontend. Previously
the transcript/task tables existed but no Tauri command read or wrote
them — the UI lived entirely in localStorage. Closes that gap and adds
the missing commands flagged by both Codex and architecture-review.md
§13.
MIGRATIONS:
- v2 adds:
- transcripts_fts virtual table (FTS5, porter+unicode61 tokeniser,
diacritics-folded) backed by transcripts via content_rowid
- INSERT/UPDATE/DELETE triggers keep FTS in sync automatically
- dictionary table for custom vocabulary (term, note, created_at)
- Append-only as required; never modifies v1
STORAGE FUNCTIONS (crates/storage/src/database.rs):
- list_transcripts_paged(limit, offset) — pagination
- count_transcripts() — for "showing X of N" in UI
- update_transcript(id, text?, title?) — closes the historic
architecture-review.md §13 rename-never-persists TODO
- search_transcripts(query, limit) — FTS5 wrapper
- list_dictionary / add_dictionary_entry / delete_dictionary_entry
- DictionaryEntry struct exported
TAURI COMMANDS (src-tauri/src/commands/transcripts.rs new file):
- add_transcript, list_transcripts, count_transcripts_command,
get_transcript, update_transcript, delete_transcript,
search_transcripts
- list_dictionary_command, add_dictionary_entry_command,
delete_dictionary_entry_command
- TranscriptDto + DictionaryDto for camelCase frontend serialisation
REGISTRATION (src-tauri/src/lib.rs):
- All 10 new commands wired into the invoke_handler
cargo check -p kon-storage passes clean. The Tauri crate cargo check
still requires `sudo dnf install cmake clang-devel` for whisper-rs-sys
(pre-existing infra dep, unrelated to this work).
NEXT (still in this sprint):
- Wire DictationPage / FilesPage to call add_transcript on every save
(dual-write: keep localStorage for now, add SQLite alongside)
- Wire HistoryPage rename to call update_transcript (fixes the TODO)
- Add a History search input that calls search_transcripts
- Add Settings → Dictionary panel
- Inject dictionary terms into the LLM cleanup prompt
