Kon

Think out loud. Keep working.

Kon is a local-first, cognitive-load-aware dictation and task-capture desktop app. Every transcription, LLM cleanup, and task extraction runs on the user's machine. No telemetry, no analytics, no cloud dependency. The app is designed around a single observation: people who think in bursts lose ideas faster than they can type, and the tool's job is to get out of the way.

---

Status

Pre-alpha. Actively dogfooded on Linux (KDE Plasma 6 on Wayland). macOS and Windows targets are in scope and exercised by CI, but not yet beta-ready. One primary user; open source-intent with licence TBD before public beta.

• Current main: see commit log
• 245 automated lib tests across 10 crates, all passing
• Cross-platform CI (Linux / macOS / Windows) via GitHub Actions

---

Design principles (non-negotiable)

1. Local-first is the floor, not a feature. No voice, transcript, or task ever leaves the user's machine unless they explicitly send it. No telemetry.
2. Cognitive load is the limiting resource. Every new setting must earn its mental real estate. Every interaction should reduce, not add, decisions.
3. Composable, not monolithic. Kon is a dictation primitive: via MCP, CLI, and filesystem export, it slots into whatever workflow the user already has (Obsidian, Claude Desktop, Cline, any text field).
4. LLM scope is narrow. The in-app LLM does transcription cleanup and task extraction. It is not a wake-word agent, not a chat UI, not a multi-provider cloud fan-out.
5. Raw transcript is always recoverable. Cleanup is additive, never destructive. The user can always see and revert to what Whisper heard.

These are enforced in the codebase (where practical) and in the docs under docs/whisper-ecosystem/kon-context.md.

---

What Kon does today

Speech-to-text

• Vulkan-accelerated local Whisper inference via whisper-rs 0.16 + whisper.cpp. Works on NVIDIA, AMD, Intel Arc, Apple (via MoltenVK), and integrated graphics.
• Vulkan / CUDA-accelerated Parakeet inference via sherpa-onnx (NVIDIA's English-only model; lower latency than Whisper-Large on English).
• Six Whisper variants shipped: Tiny, Base, Small, Distil-Small, Medium, Distil-Large v3.
• Parakeet-as-default for English when hardware supports it; first-run hardware probe picks the fastest-accurate pair.
• Resumable downloads with SHA-256 verification; retains audio if transcription fails.
• Per-profile custom vocabulary fed to Whisper as initial_prompt plus to the LLM cleanup prompt; bulk import via paste.
• Live streaming transcription with speech-gated chunking, hallucination filtering, and duplicate-boundary detection.

LLM formatting (local only)

• Local LLM runtime via llama-cpp-2 0.1.144 with Vulkan.
• Three Qwen3 tiers (1.7B, 4B-Instruct-2507, 14B) auto-selected by hardware probe.
• GBNF grammar-constrained output for task extraction (always-parseable JSON).
• System prompt hardened against voice-delivered prompt injection.

Task capture

• Automatic task extraction from any transcript.
• MicroSteps — one-tap "break this task into 3–7 concrete physical actions."
• Profile-scoped task lists with inbox / today / soon / later buckets.
• Tasks back-link to their source transcript.

Input, paste, and window management

• Global hotkey — evdev-based on Linux (Wayland-compatible out of the box), tauri-plugin-global-shortcut on macOS / Windows. Per-OS capability matrix rejects invalid key combinations.
• Platform-aware paste matrix — wtype / xdotool / ydotool on Linux, AppleScript on macOS, SendKeys on Windows. Clipboard snapshot + 300 ms restore after paste.
• Wayland-hardened transcription preview overlay (/preview): pinned across virtual desktops, hidden from Alt+Tab via WindowTypeHint::Utility, never steals focus, focus-gated open.
• Meeting auto-capture (opt-in, default off): single-signal process-list watcher, user-editable app list, surfaces a non-modal reminder. No mic-activity heuristics, no calendar integration.

History and search

• FTS5-indexed transcript search over SQLite.
• YAML-frontmatter markdown export one-click into Obsidian vault.
• Per-transcript metadata: starred, manual tags, template, language, duration.
• Transcript editor window (/viewer) with debounced autosave.

External integration

• MCP stdio server (kon-mcp) exposing read-only transcripts and tasks to any Model Context Protocol client (Claude Desktop, Cline, Cursor, etc.). No authentication, read-only, local-only.

Accessibility

• Dyslexia-friendly fonts bundled: Lexend, Atkinson Hyperlegible Next, OpenDyslexic.
• Bionic reading mode.
• Per-region font size, letter spacing, line height, transcript-specific sizing.
• System-aware reduce-motion.
• i18n: English, Spanish, German (svelte-i18n scaffold).

Privacy, deployment, reliability

• Zero telemetry. Zero analytics. No crash reports leave the machine unless explicitly bundled.
• Auto-update via Tauri updater plugin (signed, user-approved).
• Per-window size + position persistence (tauri-plugin-window-state).
• Crash + panic capture stored locally; user-bundleable for support.

---

Architecture

Kon is a Tauri 2 desktop app with three layers:

┌─────────────────────────────────────────────────────────────────┐
│  Svelte 5 frontend (src/)                                       │
│  Routes: /, /float, /viewer, /preview                           │
│  Stores, i18n, Tailwind CSS                                     │
├─────────────────────────────────────────────────────────────────┤
│  Tauri 2 runtime (src-tauri/)                                   │
│  Commands: audio, clipboard, diagnostics, hotkey, live, llm,    │
│            meeting, models, paste, power, profiles, tasks,      │
│            transcription, transcripts, update, windows          │
│  Plugins: global-shortcut, dialog, opener, updater,             │
│           window-state                                          │
├─────────────────────────────────────────────────────────────────┤
│  Rust workspace (crates/)                                       │
│  kon-core, kon-audio, kon-transcription, kon-llm,               │
│  kon-ai-formatting, kon-storage, kon-hotkey,                    │
│  kon-cloud-providers, kon-mcp                                   │
└─────────────────────────────────────────────────────────────────┘

The Rust workspace is the brain; Tauri is the OS integration surface; Svelte is the UI. The MCP server (kon-mcp) is a separate binary that opens Kon's SQLite store read-only — it's Kon-as-primitive for external agents.

Repository layout

kon/
├── Cargo.toml              # workspace root
├── src-tauri/              # Tauri app (main binary + commands)
│   ├── src/
│   │   ├── commands/       # 18 Tauri command modules
│   │   ├── lib.rs          # app entry, setup, command registration
│   │   ├── tray.rs
│   │   └── main.rs
│   ├── capabilities/       # Tauri ACL capability files
│   ├── gen/schemas/        # auto-generated ACL schemas
│   ├── tauri.conf.json     # base Tauri config
│   ├── tauri.linux.conf.json  # Linux overlay (native decorations)
│   └── resources/windows/  # Windows-specific bundled assets
├── crates/                 # workspace Rust crates
│   ├── ai-formatting/      # post-processing pipeline + LLM cleanup client
│   ├── audio/              # capture, resampling, decoding, WAV I/O
│   ├── cloud-providers/    # BYOK cloud STT stubs (empty scaffolding)
│   ├── core/               # types, hardware probe, model registry, process watch
│   ├── hotkey/             # Linux evdev hotkey listener
│   ├── llm/                # llama-cpp-2 engine + model manager
│   ├── mcp/                # MCP stdio server binary
│   ├── storage/            # SQLite + FTS5 + file storage
│   └── transcription/      # Whisper + Parakeet wrappers, model mgmt
├── src/                    # Svelte frontend
│   ├── routes/             # SvelteKit routes
│   │   ├── +page.svelte    # main dictation UI
│   │   ├── +layout.svelte  # shell (sidebar, tray sync, hotkey wiring)
│   │   ├── float/          # tasks float window
│   │   ├── viewer/         # transcript editor window
│   │   └── preview/        # transcription preview overlay
│   ├── lib/
│   │   ├── pages/          # DictationPage, SettingsPage, HistoryPage, TasksPage, FilesPage, FirstRunPage
│   │   ├── components/     # reusable Svelte components
│   │   ├── stores/         # $state stores (page, preferences, profiles, toasts)
│   │   ├── actions/        # Svelte actions (bionic reading, etc.)
│   │   ├── utils/          # frontmatter, textMeasure, errors, storage helpers
│   │   ├── types/          # TS type definitions
│   │   └── i18n/           # svelte-i18n setup + en/es/de locales
│   ├── fonts/              # bundled accessibility fonts
│   ├── design-system/      # design tokens + UI kit references (not live code)
│   └── app.css
├── docs/                   # all project documentation (see below)
├── .github/workflows/      # CI (check.yml, build.yml)
├── package.json
├── HANDOVER.md             # latest session handover
└── run.sh                  # dev launcher (starts Vite then Tauri)

Rust crates

Crate	Responsibility
kon-core	Shared types (Segment, Transcript, Megabytes, ModelId), constants, the Engine / SpeedTier / AccuracyTier enums, hardware probe (sysinfo-based), model registry (Whisper + Parakeet + Moonshine entries), hardware-aware recommendation scoring, process_watch for meeting detection.
kon-audio	cpal-based microphone capture with device hotplug + error forwarding, VAD, rubato streaming resampler to 16 kHz mono, symphonia file decoding, hound WAV I/O.
kon-transcription	whisper-rs backend (WhisperRsBackend) that owns a WhisperContext and supports set_initial_prompt. LocalEngine wraps both Whisper and Parakeet (via transcribe-rs ONNX) behind a common Transcriber trait. Streaming primitives (VadChunker, LocalAgreement, buffer trim) live in the streaming/ module. Model manager handles downloads, paths, and disk checks.
kon-llm	llama-cpp-2 engine with Qwen3 model manager. Three high-level surfaces: cleanup_text (formatting), decompose_task (3–7 micro-steps, GBNF-constrained JSON array), extract_tasks (optional-array, GBNF-constrained). Resumable HTTP downloads with SHA-256 verify.
kon-ai-formatting	Post-processing pipeline: filler removal, British English conversion, anti-hallucination filter, smart paragraph breaks on long pauses, optional LLM cleanup. Also hosts the llm_client::CLEANUP_PROMPT constant (prompt-injection-hardened).
kon-storage	SQLite via sqlx 0.8. Migrations, CRUD for transcripts / tasks / subtasks / profiles / profile terms / settings / error log, FTS5 search, file-storage paths.
kon-hotkey	Linux evdev hotkey listener with device hotplug. Parses Tauri-style hotkey strings (Ctrl+Shift+R), emits Pressed / Released events. Works natively on Wayland (no X11 dependency). Checks /dev/input/event* access on startup; surfaces a clear "add yourself to the input group" error when missing.
kon-cloud-providers	BYOK cloud-STT provider stubs. Currently empty scaffolding. When populated: OpenAI-compatible endpoint + Anthropic (ceiling for scope).
kon-mcp	Standalone kon-mcp binary implementing the MCP stdio protocol (2024-11-05). Read-only tools: list_transcripts, get_transcript, search_transcripts, list_tasks. Opens Kon's SQLite store.

Tauri commands (src-tauri/src/commands/)

Module	What it exposes
audio	Device enumeration, native capture start/stop, audio-samples persistence
clipboard	Cross-platform clipboard write (arboard)
diagnostics	Panic hook, frontend error log, crash file listing, diagnostic report bundler
hardware	probe_system, rank_models
hotkey	start_evdev_hotkey, update_evdev_hotkey, stop_evdev_hotkey, check_hotkey_access, is_wayland_session
live	Live streaming transcription session lifecycle + speech-gate tuning
llm	Tier recommend, model check / download / load / unload / delete, status, cleanup_transcript_text_cmd, extract_tasks_from_transcript_cmd
meeting	detect_meeting_processes (process-list poll)
models	Whisper + Parakeet model download / load / check / default-id resolution, runtime capabilities API, pre-warm
paste	paste_text (copy + keystroke), detect_paste_backends, Wayland focus-race mitigation against the preview overlay
power	macOS PowerAssertion guard during long sessions (blocks App Nap)
profiles	Profile CRUD, profile-terms CRUD, learn-terms-from-edit
tasks	Task CRUD, subtask CRUD, decompose_and_store, extract_tasks_from_transcript_cmd
transcription	transcribe_pcm, transcribe_file, transcribe_pcm_parakeet
transcripts	Transcript CRUD + FTS5 search
update	Tauri-plugin-updater check / install
windows	open_task_window, open_viewer_window, open_preview_window, close_preview_window

Frontend (src/)

• SvelteKit + Svelte 5 runes ($state, $derived, $effect).
• Tailwind CSS 4 for styling, with a Lexend/Atkinson/OpenDyslexic type system.
• Secondary windows (/float, /viewer, /preview) use named layouts (+layout@.svelte) to skip the main shell and run chrome-free.
• Reactive stores (src/lib/stores/page.svelte.ts): settings, profiles, tasks, history, taskLists, templates, page, toasts, preferences.
• i18n: svelte-i18n with en/es/de locales at src/lib/i18n/locales/. Scaffolding only — strings migrate to translation keys incrementally.

---

Runtime stack

Layer	Technology	Version
Desktop framework	Tauri	2.10.3
Frontend	Svelte 5 + SvelteKit + Vite	latest
Styling	Tailwind CSS	4.x
Speech-to-text (primary)	whisper.cpp via whisper-rs	0.16 (Vulkan feature)
Speech-to-text (Parakeet)	sherpa-onnx via transcribe-rs	0.3
Local LLM	llama-cpp-2	0.1.144 (openmp + vulkan)
Database	SQLite via sqlx	0.8
Async runtime	tokio	1.x
Audio capture	cpal	current
Resampling	rubato	current
File decode	symphonia	current

---

Platform support

Platform	Status	Notes
Linux Wayland (KDE Plasma, GNOME Mutter, Hyprland, Sway)	Primary target, daily-dogfooded on KDE	evdev hotkey, GTK 3 via webkit2gtk, Vulkan, all paste backends
Linux X11	Supported	xdotool paste path, GTK 3
macOS	In CI, untested runtime	osascript paste, Metal via MoltenVK, App Nap guard
Windows	In CI, untested runtime	SendKeys paste, Vulkan-first GPU path, bundled DLLs for CPU fallback

CI runs cargo check --workspace --all-targets + svelte-check on all three on every push and PR.

---

Build + development

Prerequisites

Linux (Fedora/RHEL listed; adjust for your distro):

sudo dnf install libclang-devel clang \
    webkit2gtk4.1-devel libappindicator-gtk3-devel librsvg2-devel \
    alsa-lib-devel systemd-devel cmake \
    vulkan-headers vulkan-loader-devel glslc

macOS:

brew install cmake llvm vulkan-headers vulkan-loader molten-vk shaderc

Windows:

choco install cmake llvm vulkan-sdk

See docs/dev-setup.md for the authoritative per-platform dependency list and for how LIBCLANG_PATH should be set.

Dev launch

The fast path — starts Vite, waits for port 1420, then launches Tauri:

./run.sh

Or manually:

# Terminal 1
npm run dev:frontend

# Terminal 2
npm run tauri dev

Build

npm run tauri build    # release build, produces .AppImage / .deb / .dmg / .msi / .exe

CI also builds release installers on tag push (see .github/workflows/build.yml).

Testing

cargo test --workspace --lib    # 245 tests across 10 crates
npm run check                   # svelte-check (type-checks .svelte files)
cargo check --workspace --all-targets

---

Project documentation

Beyond this README, the repo ships extensive internal documentation:

Product + strategy — docs/brief/

Research briefs, competitive analysis, and strategic framing. Start with:

• what-kon-is.md — product thesis
• why-current-tools-fail.md — market gap
• design-principles.md — full principle list
• target-audience.md, market-size-demographics.md
• Appendices on cognitive ergonomics, AI body doubling, evolutionary psychology, implementation intentions, HITL scaffolding, voice interfaces

Brand — docs/brand/

• kon-brand-guidelines.md
• kon-brand-platform.md

Technical research — docs/whisper-ecosystem/

Cross-repo survey of 10 OSS Whisper projects, the Kon-specific atomic task backlog, and the two Cursor workstream plans.

• brief.md — 31-item task backlog (the canonical research spec)
• kon-context.md — ideology, shipped state, file-ownership fence for cloud AI agents
• workstream-A.md, workstream-B.md — executed workstream plans

GPU tuning — docs/gpu-tuning/

• plan.md — MVP plan for GGML env-var panel + kon-bench auto-tuner + kon-configs community repo

Session handovers

• HANDOVER.md — latest session summary
• Dated historical handovers: HANDOVER-2026-04-17.md, HANDOVER-2026-04-18.md

Dev reference

• docs/dev-setup.md — dependency + launch reference
• docs/icon-mapping.md — icon conventions

---

Roadmap

The shipped code represents Phases 1–3 and a partial Phase 4.

Pinned roadmap items (scoped in docs and session memory):

• Phase 4 — remaining items from workstream-A.md + workstream-B.md
• Voice calibration — three-tier plan replacing the hardcoded speech-gate with per-user baselines
• GPU community tuning — see docs/gpu-tuning/plan.md; five-phase roadmap from settings panel to agentic auto-tuner + community config repo
• Cloud endpoint contract test — when kon-cloud-providers grows a real provider
• ggml dedup — replace the interim -Wl,--allow-multiple-definition link flag with a proper shared-lib setup; unblocks custom shader / backend work
• Mobile (iOS / Android) — long-horizon, gated on the single-binary Rust stack scaling

Explicitly shelved (not coming without specific community signal):

• Wake-word / always-listening agent
• Chat-style LLM UI
• Multi-provider cloud fan-out beyond OpenAI-compatible + Anthropic
• Second notes-editing surface (transcripts leave Kon via frontmatter to Obsidian)
• Speaker diarization
• Dragon-style passage-based speaker fine-tuning (Whisper has no speaker adaptation)

---

Contributing

Pre-alpha status; contribution process TBD before public beta. For now:

• Every Tauri command change must register in both src-tauri/src/lib.rs (invoke handler) and in the invoking frontend code.
• Every Settings-visible setting must have a type field in src/lib/types/app.ts and a default in src/lib/stores/page.svelte.ts.
• Every new workspace crate needs a description in its Cargo.toml.
• Tests: add at least a smoke test per new Tauri command or crate module. The workspace test floor is "no regressions on main."
• Wayland compatibility is a first-class concern — don't assume X11. The preview overlay and paste matrix live-document what this looks like in practice.

---

Licence

To be finalised before public beta. Current intent: MIT or similar permissive licence, with Corbel Consulting offering optional commercial support / managed services as the revenue path.

---

Contact

Jake Sames — jakeadriansames@gmail.com Repo: github.com/jakejars/kon · git.corbel.consulting/jake/kon