docs(readme): add project README with architectural profile

Previously the repo had no root README. Pitch + architecture + crate-level breakdown + routes + runtime stack + roadmap all lived in dated HANDOVER docs + docs/brief/ + memory. This pulls the canonical overview into the single place a new contributor / investor / cursory browser will actually land first. Sections: design principles, what Kon does today (shipped capabilities), architecture (3-layer: Rust workspace → Tauri commands → Svelte UI), crate-by-crate responsibilities, Tauri command surface, frontend shape, runtime stack with pinned versions, platform support matrix, build + dev setup, testing, pointers into docs/, roadmap, contributing notes, licence placeholder, contact. All claims are grounded in current code state (136 tests, 9 crates, 18 Tauri command modules, Tauri 2.10.3, whisper-rs 0.16, llama-cpp-2 0.1.144, sqlx 0.8.6). No marketing puff; preserves the ideology-firm stance from memory. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 16:11:16 +01:00
parent e75f676fc1
commit 3790fa0c91
1 changed files with 377 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,377 @@
+# Kon
+
+*Think out loud. Keep working.*
+
+Kon is a local-first, cognitive-load-aware dictation and task-capture desktop app. Every transcription, LLM cleanup, and task extraction runs on the user's machine. No telemetry, no analytics, no cloud dependency. The app is designed around a single observation: people who think in bursts lose ideas faster than they can type, and the tool's job is to get out of the way.
+
+---
+
+## Status
+
+**Pre-alpha.** Actively dogfooded on Linux (KDE Plasma 6 on Wayland). macOS and Windows targets are in scope and exercised by CI, but not yet beta-ready. One primary user; open source-intent with licence TBD before public beta.
+
+- Current `main`: see commit log
+- 136 automated lib tests across 10 crates, all passing
+- Cross-platform CI (Linux / macOS / Windows) via GitHub Actions
+
+---
+
+## Design principles (non-negotiable)
+
+1. **Local-first is the floor, not a feature.** No voice, transcript, or task ever leaves the user's machine unless they explicitly send it. No telemetry.
+2. **Cognitive load is the limiting resource.** Every new setting must earn its mental real estate. Every interaction should reduce, not add, decisions.
+3. **Composable, not monolithic.** Kon is a dictation primitive: via MCP, CLI, and filesystem export, it slots into whatever workflow the user already has (Obsidian, Claude Desktop, Cline, any text field).
+4. **LLM scope is narrow.** The in-app LLM does transcription cleanup and task extraction. It is not a wake-word agent, not a chat UI, not a multi-provider cloud fan-out.
+5. **Raw transcript is always recoverable.** Cleanup is additive, never destructive. The user can always see and revert to what Whisper heard.
+
+These are enforced in the codebase (where practical) and in the docs under [`docs/whisper-ecosystem/kon-context.md`](docs/whisper-ecosystem/kon-context.md).
+
+---
+
+## What Kon does today
+
+### Speech-to-text
+- Vulkan-accelerated local **Whisper** inference via [whisper-rs](https://github.com/tazz4843/whisper-rs) 0.16 + whisper.cpp. Works on NVIDIA, AMD, Intel Arc, Apple (via MoltenVK), and integrated graphics.
+- Vulkan / CUDA-accelerated **Parakeet** inference via sherpa-onnx (NVIDIA's English-only model; lower latency than Whisper-Large on English).
+- **Six Whisper variants** shipped: Tiny, Base, Small, Distil-Small, Medium, Distil-Large v3.
+- **Parakeet-as-default for English** when hardware supports it; first-run hardware probe picks the fastest-accurate pair.
+- **Resumable downloads with SHA-256 verification**; retains audio if transcription fails.
+- **Per-profile custom vocabulary** fed to Whisper as `initial_prompt` plus to the LLM cleanup prompt; bulk import via paste.
+- **Live streaming transcription** with speech-gated chunking, hallucination filtering, and duplicate-boundary detection.
+
+### LLM formatting (local only)
+- Local LLM runtime via [llama-cpp-2](https://github.com/utilityai/llama-cpp-rs) 0.1.144 with Vulkan.
+- Three Qwen3 tiers (1.7B, 4B-Instruct-2507, 14B) auto-selected by hardware probe.
+- GBNF grammar-constrained output for task extraction (always-parseable JSON).
+- System prompt hardened against voice-delivered prompt injection.
+
+### Task capture
+- Automatic task extraction from any transcript.
+- **MicroSteps** — one-tap "break this task into 3–7 concrete physical actions."
+- Profile-scoped task lists with inbox / today / soon / later buckets.
+- Tasks back-link to their source transcript.
+
+### Input, paste, and window management
+- **Global hotkey** — evdev-based on Linux (Wayland-compatible out of the box), `tauri-plugin-global-shortcut` on macOS / Windows. Per-OS capability matrix rejects invalid key combinations.
+- **Platform-aware paste matrix** — `wtype` / `xdotool` / `ydotool` on Linux, AppleScript on macOS, SendKeys on Windows. Clipboard snapshot + 300 ms restore after paste.
+- **Wayland-hardened transcription preview overlay** (`/preview`): pinned across virtual desktops, hidden from Alt+Tab via `WindowTypeHint::Utility`, never steals focus, focus-gated open.
+- **Meeting auto-capture** (opt-in, default off): single-signal process-list watcher, user-editable app list, surfaces a non-modal reminder. No mic-activity heuristics, no calendar integration.
+
+### History and search
+- **FTS5-indexed transcript search** over SQLite.
+- **YAML-frontmatter markdown export** one-click into Obsidian vault.
+- Per-transcript metadata: starred, manual tags, template, language, duration.
+- Transcript editor window (`/viewer`) with debounced autosave.
+
+### External integration
+- **MCP stdio server** (`kon-mcp`) exposing read-only transcripts and tasks to any Model Context Protocol client (Claude Desktop, Cline, Cursor, etc.). No authentication, read-only, local-only.
+
+### Accessibility
+- Dyslexia-friendly fonts bundled: Lexend, Atkinson Hyperlegible Next, OpenDyslexic.
+- Bionic reading mode.
+- Per-region font size, letter spacing, line height, transcript-specific sizing.
+- System-aware reduce-motion.
+- **i18n**: English, Spanish, German (svelte-i18n scaffold).
+
+### Privacy, deployment, reliability
+- Zero telemetry. Zero analytics. No crash reports leave the machine unless explicitly bundled.
+- Auto-update via Tauri updater plugin (signed, user-approved).
+- Per-window size + position persistence (`tauri-plugin-window-state`).
+- Crash + panic capture stored locally; user-bundleable for support.
+
+---
+
+## Architecture
+
+Kon is a Tauri 2 desktop app with three layers:
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│  Svelte 5 frontend (src/)                                       │
+│  Routes: /, /float, /viewer, /preview                           │
+│  Stores, i18n, Tailwind CSS                                     │
+├─────────────────────────────────────────────────────────────────┤
+│  Tauri 2 runtime (src-tauri/)                                   │
+│  Commands: audio, clipboard, diagnostics, hotkey, live, llm,    │
+│            meeting, models, paste, power, profiles, tasks,      │
+│            transcription, transcripts, update, windows          │
+│  Plugins: global-shortcut, dialog, opener, updater,             │
+│           window-state                                          │
+├─────────────────────────────────────────────────────────────────┤
+│  Rust workspace (crates/)                                       │
+│  kon-core, kon-audio, kon-transcription, kon-llm,               │
+│  kon-ai-formatting, kon-storage, kon-hotkey,                    │
+│  kon-cloud-providers, kon-mcp                                   │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+The Rust workspace is the brain; Tauri is the OS integration surface; Svelte is the UI. The MCP server (`kon-mcp`) is a separate binary that opens Kon's SQLite store read-only — it's Kon-as-primitive for external agents.
+
+### Repository layout
+
+```
+kon/
+├── Cargo.toml              # workspace root
+├── src-tauri/              # Tauri app (main binary + commands)
+│   ├── src/
+│   │   ├── commands/       # 18 Tauri command modules
+│   │   ├── lib.rs          # app entry, setup, command registration
+│   │   ├── tray.rs
+│   │   └── main.rs
+│   ├── capabilities/       # Tauri ACL capability files
+│   ├── gen/schemas/        # auto-generated ACL schemas
+│   ├── tauri.conf.json     # base Tauri config
+│   ├── tauri.linux.conf.json  # Linux overlay (native decorations)
+│   └── resources/windows/  # Windows-specific bundled assets
+├── crates/                 # workspace Rust crates
+│   ├── ai-formatting/      # post-processing pipeline + LLM cleanup client
+│   ├── audio/              # capture, resampling, decoding, WAV I/O
+│   ├── cloud-providers/    # BYOK cloud STT stubs (empty scaffolding)
+│   ├── core/               # types, hardware probe, model registry, process watch
+│   ├── hotkey/             # Linux evdev hotkey listener
+│   ├── llm/                # llama-cpp-2 engine + model manager
+│   ├── mcp/                # MCP stdio server binary
+│   ├── storage/            # SQLite + FTS5 + file storage
+│   └── transcription/      # Whisper + Parakeet wrappers, model mgmt
+├── src/                    # Svelte frontend
+│   ├── routes/             # SvelteKit routes
+│   │   ├── +page.svelte    # main dictation UI
+│   │   ├── +layout.svelte  # shell (sidebar, tray sync, hotkey wiring)
+│   │   ├── float/          # tasks float window
+│   │   ├── viewer/         # transcript editor window
+│   │   └── preview/        # transcription preview overlay
+│   ├── lib/
+│   │   ├── pages/          # DictationPage, SettingsPage, HistoryPage, TasksPage, FilesPage, FirstRunPage
+│   │   ├── components/     # reusable Svelte components
+│   │   ├── stores/         # $state stores (page, preferences, profiles, toasts)
+│   │   ├── actions/        # Svelte actions (bionic reading, etc.)
+│   │   ├── utils/          # frontmatter, textMeasure, errors, storage helpers
+│   │   ├── types/          # TS type definitions
+│   │   └── i18n/           # svelte-i18n setup + en/es/de locales
+│   ├── fonts/              # bundled accessibility fonts
+│   ├── design-system/      # design tokens + UI kit references (not live code)
+│   └── app.css
+├── docs/                   # all project documentation (see below)
+├── .github/workflows/      # CI (check.yml, build.yml)
+├── package.json
+├── HANDOVER.md             # latest session handover
+└── run.sh                  # dev launcher (starts Vite then Tauri)
+```
+
+### Rust crates
+
+| Crate | Responsibility |
+|---|---|
+| **`kon-core`** | Shared types (`Segment`, `Transcript`, `Megabytes`, `ModelId`), constants, the `Engine` / `SpeedTier` / `AccuracyTier` enums, hardware probe (`sysinfo`-based), model registry (Whisper + Parakeet + Moonshine entries), hardware-aware recommendation scoring, `process_watch` for meeting detection. |
+| **`kon-audio`** | `cpal`-based microphone capture with device hotplug + error forwarding, VAD, `rubato` streaming resampler to 16 kHz mono, `symphonia` file decoding, `hound` WAV I/O. |
+| **`kon-transcription`** | `whisper-rs` backend (`WhisperRsBackend`) that owns a `WhisperContext` and supports `set_initial_prompt`. `LocalEngine` wraps both Whisper and Parakeet (via `transcribe-rs` ONNX) behind a common `SpeechToText` trait. Model manager handles downloads, paths, and disk checks. |
+| **`kon-llm`** | `llama-cpp-2` engine with Qwen3 model manager. Three high-level surfaces: `cleanup_text` (formatting), `decompose_task` (3–7 micro-steps, GBNF-constrained JSON array), `extract_tasks` (optional-array, GBNF-constrained). Resumable HTTP downloads with SHA-256 verify. |
+| **`kon-ai-formatting`** | Post-processing pipeline: filler removal, British English conversion, anti-hallucination filter, smart paragraph breaks on long pauses, optional LLM cleanup. Also hosts the `llm_client::CLEANUP_PROMPT` constant (prompt-injection-hardened). |
+| **`kon-storage`** | SQLite via `sqlx` 0.8. Migrations, CRUD for transcripts / tasks / subtasks / profiles / profile terms / settings / error log, FTS5 search, file-storage paths. |
+| **`kon-hotkey`** | Linux `evdev` hotkey listener with device hotplug. Parses Tauri-style hotkey strings (`Ctrl+Shift+R`), emits Pressed / Released events. Works natively on Wayland (no X11 dependency). Checks `/dev/input/event*` access on startup; surfaces a clear "add yourself to the `input` group" error when missing. |
+| **`kon-cloud-providers`** | BYOK cloud-STT provider stubs. Currently empty scaffolding. When populated: OpenAI-compatible endpoint + Anthropic (ceiling for scope). |
+| **`kon-mcp`** | Standalone `kon-mcp` binary implementing the MCP stdio protocol (2024-11-05). Read-only tools: `list_transcripts`, `get_transcript`, `search_transcripts`, `list_tasks`. Opens Kon's SQLite store. |
+
+### Tauri commands (src-tauri/src/commands/)
+
+| Module | What it exposes |
+|---|---|
+| `audio` | Device enumeration, native capture start/stop, audio-samples persistence |
+| `clipboard` | Cross-platform clipboard write (arboard) |
+| `diagnostics` | Panic hook, frontend error log, crash file listing, diagnostic report bundler |
+| `hardware` | `probe_system`, `rank_models` |
+| `hotkey` | `start_evdev_hotkey`, `update_evdev_hotkey`, `stop_evdev_hotkey`, `check_hotkey_access`, `is_wayland_session` |
+| `live` | Live streaming transcription session lifecycle + speech-gate tuning |
+| `llm` | Tier recommend, model check / download / load / unload / delete, status, `cleanup_transcript_text_cmd`, `extract_tasks_from_transcript_cmd` |
+| `meeting` | `detect_meeting_processes` (process-list poll) |
+| `models` | Whisper + Parakeet model download / load / check / default-id resolution, runtime capabilities API, pre-warm |
+| `paste` | `paste_text` (copy + keystroke), `detect_paste_backends`, Wayland focus-race mitigation against the preview overlay |
+| `power` | macOS `PowerAssertion` guard during long sessions (blocks App Nap) |
+| `profiles` | Profile CRUD, profile-terms CRUD, learn-terms-from-edit |
+| `tasks` | Task CRUD, subtask CRUD, `decompose_and_store`, `extract_tasks_from_transcript_cmd` |
+| `transcription` | `transcribe_pcm`, `transcribe_file`, `transcribe_pcm_parakeet` |
+| `transcripts` | Transcript CRUD + FTS5 search |
+| `update` | Tauri-plugin-updater check / install |
+| `windows` | `open_task_window`, `open_viewer_window`, `open_preview_window`, `close_preview_window` |
+
+### Frontend (src/)
+
+- **SvelteKit + Svelte 5 runes** (`$state`, `$derived`, `$effect`).
+- **Tailwind CSS 4** for styling, with a Lexend/Atkinson/OpenDyslexic type system.
+- **Secondary windows** (`/float`, `/viewer`, `/preview`) use named layouts (`+layout@.svelte`) to skip the main shell and run chrome-free.
+- **Reactive stores** (`src/lib/stores/page.svelte.ts`): `settings`, `profiles`, `tasks`, `history`, `taskLists`, `templates`, `page`, `toasts`, `preferences`.
+- **i18n**: `svelte-i18n` with en/es/de locales at `src/lib/i18n/locales/`. Scaffolding only — strings migrate to translation keys incrementally.
+
+---
+
+## Runtime stack
+
+| Layer | Technology | Version |
+|---|---|---|
+| Desktop framework | [Tauri](https://tauri.app) | 2.10.3 |
+| Frontend | Svelte 5 + SvelteKit + Vite | latest |
+| Styling | Tailwind CSS | 4.x |
+| Speech-to-text (primary) | whisper.cpp via [`whisper-rs`](https://github.com/tazz4843/whisper-rs) | 0.16 (Vulkan feature) |
+| Speech-to-text (Parakeet) | sherpa-onnx via `transcribe-rs` | 0.3 |
+| Local LLM | [`llama-cpp-2`](https://github.com/utilityai/llama-cpp-rs) | 0.1.144 (openmp + vulkan) |
+| Database | SQLite via [`sqlx`](https://github.com/launchbadge/sqlx) | 0.8 |
+| Async runtime | [`tokio`](https://tokio.rs/) | 1.x |
+| Audio capture | [`cpal`](https://github.com/RustAudio/cpal) | current |
+| Resampling | [`rubato`](https://github.com/HEnquist/rubato) | current |
+| File decode | [`symphonia`](https://github.com/pdeljanov/Symphonia) | current |
+
+---
+
+## Platform support
+
+| Platform | Status | Notes |
+|---|---|---|
+| Linux Wayland (KDE Plasma, GNOME Mutter, Hyprland, Sway) | **Primary target**, daily-dogfooded on KDE | evdev hotkey, GTK 3 via webkit2gtk, Vulkan, all paste backends |
+| Linux X11 | Supported | xdotool paste path, GTK 3 |
+| macOS | In CI, untested runtime | osascript paste, Metal via MoltenVK, App Nap guard |
+| Windows | In CI, untested runtime | SendKeys paste, Vulkan-first GPU path, bundled DLLs for CPU fallback |
+
+CI runs `cargo check --workspace --all-targets` + `svelte-check` on all three on every push and PR.
+
+---
+
+## Build + development
+
+### Prerequisites
+
+Linux (Fedora/RHEL listed; adjust for your distro):
+```
+sudo dnf install libclang-devel clang \
+    webkit2gtk4.1-devel libappindicator-gtk3-devel librsvg2-devel \
+    alsa-lib-devel systemd-devel cmake \
+    vulkan-headers vulkan-loader-devel glslc
+```
+
+macOS:
+```
+brew install cmake llvm vulkan-headers vulkan-loader molten-vk shaderc
+```
+
+Windows:
+```
+choco install cmake llvm vulkan-sdk
+```
+
+See [`docs/dev-setup.md`](docs/dev-setup.md) for the authoritative per-platform dependency list and for how `LIBCLANG_PATH` should be set.
+
+### Dev launch
+
+The fast path — starts Vite, waits for port 1420, then launches Tauri:
+
+```bash
+./run.sh
+```
+
+Or manually:
+
+```bash
+# Terminal 1
+npm run dev:frontend
+
+# Terminal 2
+npm run tauri dev
+```
+
+### Build
+
+```bash
+npm run tauri build    # release build, produces .AppImage / .deb / .dmg / .msi / .exe
+```
+
+CI also builds release installers on tag push (see `.github/workflows/build.yml`).
+
+### Testing
+
+```bash
+cargo test --workspace --lib    # 136 tests across 10 crates
+npm run check                   # svelte-check (type-checks .svelte files)
+cargo check --workspace --all-targets
+```
+
+---
+
+## Project documentation
+
+Beyond this README, the repo ships extensive internal documentation:
+
+### Product + strategy — `docs/brief/`
+Research briefs, competitive analysis, and strategic framing. Start with:
+- [`what-kon-is.md`](docs/brief/what-kon-is.md) — product thesis
+- [`why-current-tools-fail.md`](docs/brief/why-current-tools-fail.md) — market gap
+- [`design-principles.md`](docs/brief/design-principles.md) — full principle list
+- [`target-audience.md`](docs/brief/target-audience.md), [`market-size-demographics.md`](docs/brief/market-size-demographics.md)
+- Appendices on cognitive ergonomics, AI body doubling, evolutionary psychology, implementation intentions, HITL scaffolding, voice interfaces
+
+### Brand — `docs/brand/`
+- [`kon-brand-guidelines.md`](docs/brand/kon-brand-guidelines.md)
+- [`kon-brand-platform.md`](docs/brand/kon-brand-platform.md)
+
+### Technical research — `docs/whisper-ecosystem/`
+Cross-repo survey of 10 OSS Whisper projects, the Kon-specific atomic task backlog, and the two Cursor workstream plans.
+- [`brief.md`](docs/whisper-ecosystem/brief.md) — 31-item task backlog (the canonical research spec)
+- [`kon-context.md`](docs/whisper-ecosystem/kon-context.md) — ideology, shipped state, file-ownership fence for cloud AI agents
+- [`workstream-A.md`](docs/whisper-ecosystem/workstream-A.md), [`workstream-B.md`](docs/whisper-ecosystem/workstream-B.md) — executed workstream plans
+
+### GPU tuning — `docs/gpu-tuning/`
+- [`plan.md`](docs/gpu-tuning/plan.md) — MVP plan for GGML env-var panel + `kon-bench` auto-tuner + `kon-configs` community repo
+
+### Session handovers
+- [`HANDOVER.md`](HANDOVER.md) — latest session summary
+- Dated historical handovers: `HANDOVER-2026-04-17.md`, `HANDOVER-2026-04-18.md`
+
+### Dev reference
+- [`docs/dev-setup.md`](docs/dev-setup.md) — dependency + launch reference
+- [`docs/icon-mapping.md`](docs/icon-mapping.md) — icon conventions
+
+---
+
+## Roadmap
+
+The shipped code represents Phases 1–3 and a partial Phase 4.
+
+Pinned roadmap items (scoped in docs and session memory):
+
+- **Phase 4** — remaining items from [`workstream-A.md`](docs/whisper-ecosystem/workstream-A.md) + [`workstream-B.md`](docs/whisper-ecosystem/workstream-B.md)
+- **Voice calibration** — three-tier plan replacing the hardcoded speech-gate with per-user baselines
+- **GPU community tuning** — see [`docs/gpu-tuning/plan.md`](docs/gpu-tuning/plan.md); five-phase roadmap from settings panel to agentic auto-tuner + community config repo
+- **Cloud endpoint contract test** — when `kon-cloud-providers` grows a real provider
+- **`ggml` dedup** — replace the interim `-Wl,--allow-multiple-definition` link flag with a proper shared-lib setup; unblocks custom shader / backend work
+- **Mobile (iOS / Android)** — long-horizon, gated on the single-binary Rust stack scaling
+
+Explicitly shelved (not coming without specific community signal):
+- Wake-word / always-listening agent
+- Chat-style LLM UI
+- Multi-provider cloud fan-out beyond OpenAI-compatible + Anthropic
+- Second notes-editing surface (transcripts leave Kon via frontmatter to Obsidian)
+- Speaker diarization
+- Dragon-style passage-based speaker fine-tuning (Whisper has no speaker adaptation)
+
+---
+
+## Contributing
+
+Pre-alpha status; contribution process TBD before public beta. For now:
+
+- Every Tauri command change must register in both [`src-tauri/src/lib.rs`](src-tauri/src/lib.rs) (invoke handler) and in the invoking frontend code.
+- Every Settings-visible setting must have a type field in [`src/lib/types/app.ts`](src/lib/types/app.ts) and a default in [`src/lib/stores/page.svelte.ts`](src/lib/stores/page.svelte.ts).
+- Every new workspace crate needs a `description` in its `Cargo.toml`.
+- Tests: add at least a smoke test per new Tauri command or crate module. The workspace test floor is "no regressions on main."
+- Wayland compatibility is a first-class concern — don't assume X11. The preview overlay and paste matrix live-document what this looks like in practice.
+
+---
+
+## Licence
+
+To be finalised before public beta. Current intent: MIT or similar permissive licence, with Corbel Consulting offering optional commercial support / managed services as the revenue path.
+
+---
+
+## Contact
+
+**Jake Sames** — [jakeadriansames@gmail.com](mailto:jakeadriansames@gmail.com)
+Repo: [github.com/jakejars/kon](https://github.com/jakejars/kon) · [git.corbel.consulting/jake/kon](https://git.corbel.consulting/jake/kon)