Both `kon-transcription` and `kon-llm` previously hardcoded their native
acceleration features in Cargo.toml — `whisper-rs` with `vulkan`,
`llama-cpp-2` with `openmp` + `vulkan`. That worked everywhere desktop
ships (Linux/macOS/Windows all have Vulkan via MoltenVK on Mac), but it
made an Android build structurally impossible: NDK builds against drivers
that vary wildly across SoCs (Adreno OK, Mali patchy, PowerVR worse), and
some older devices have no Vulkan at all.
Roadmap step 0 from the Android plan: make the GPU acceleration
opt-in so a CPU-only target compiles. Reuses the existing pattern that
README's "future Windows non-AVX2 build" comment hinted at.
- kon-transcription: new `whisper-vulkan` feature gates `whisper-rs/vulkan`
via the optional-syntax `whisper-rs?/vulkan`. Default features stay as
`["whisper", "whisper-vulkan"]` so desktop is unchanged.
- kon-llm: new `gpu-vulkan` and `openmp` features each gate the matching
`llama-cpp-2` feature. Default stays `["gpu-vulkan", "openmp"]`. They are
independent so an Android Vulkan build can opt into vulkan without
openmp (NDK OpenMP linking has known cross-version fragility).
CPU-only build invocations:
cargo build -p kon-transcription --no-default-features --features whisper
cargo build -p kon-llm --no-default-features
Verified: all 91 tests in the buildable-in-sandbox crates still pass.
The two crates whose Cargo.toml changed (kon-transcription, kon-llm)
can't be compiled in this sandbox (ort-sys CDN + cmake-built llama.cpp);
CI's Linux/macOS/Windows builders will exercise the default-feature path
exactly as before.
https://claude.ai/code/session_0189xUb6ie6t9qHkzatGZ9Rb
Added as a method on LlmEngine alongside cleanup_text and
extract_tasks; same render_chat_prompt -> generate -> parse pattern.
Truncates the transcript to its trailing 2000 chars on a UTF-8 char
boundary, runs at temperature 0.0 with the CONTENT_TAGS_GRAMMAR GBNF,
and re-validates intent against INTENT_CLOSED_SET to catch the
unlikely grammar bypass case. max_tokens 96 is enough for the JSON
envelope. Smoke test gated on KON_LLM_TEST_MODEL like the existing
smoke.rs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ContentTags serde-serialisable. CONTENT_TAGS_SYSTEM is the system
message rendered at extraction time; INTENT_CLOSED_SET is the single
source of truth for the enum values the grammar restricts. Grammar is
strict: lowercase hyphen-joined topic 3+ chars (max enforced by
max_tokens at call site), intent from the closed set, JSON-only
output. Recursive topic-rest matches the existing GBNF style in this
file.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the human-in-the-loop gap from docs/brief/feature-set.md and
Phase 2 of the 2026-04-23 feature-complete roadmap.
Storage (kon-storage):
- Migration v10 adds the `feedback` table: (target_type, target_id,
rating, original_text, corrected_text, context_json, profile_id,
created_at) with CHECK constraints on target_type and rating, plus
indexes on (target_type, rating, created_at DESC) for prompt-time
retrieval and (profile_id, target_type, created_at DESC) for
per-profile scoping.
- New public API: `FeedbackTargetType`, `RecordFeedbackParams`,
`FeedbackRow`, `record_feedback`, `list_feedback_examples`.
- Tests updated — the RB-02 rollback regression now discovers the
real max version at runtime instead of hard-coding v10 for its
poison migration.
LLM (kon-llm):
- `prompts::FeedbackExample` — local shape for few-shot exemplars so
kon-llm stays independent of kon-storage.
- `prompts::build_conditioned_system_prompt` — appends a "here is
the style this user prefers" block to the base system prompt
when examples are available; returns the base prompt unchanged
when empty, so new users and early sessions see generic output.
- `LlmEngine::decompose_task_with_feedback` and
`LlmEngine::extract_tasks_with_feedback` thread examples through
to the builder. The old one-arg variants are preserved and now
call through with an empty slice.
- 4 unit tests covering empty, empty-input-skip, correction-wins,
and thumbs-up-only fallback.
Tauri (src-tauri):
- New commands::feedback module: `record_feedback`,
`list_feedback_examples_cmd`.
- `decompose_and_store` and `extract_tasks_from_transcript_cmd`
now fetch the last 5 positive/neutral feedback rows for their
target type and pass them through to the LLM, wiring the
learning loop end-to-end.
- Shared `to_llm_examples` helper parses the `context_json.input`
field (where the recorder stashes the parent task text / transcript
chunk) back into the exemplar shape.
Frontend (MicroSteps.svelte):
- Thumbs-up and thumbs-down buttons on every micro-step row.
Hover-revealed; the vote recolours the icon; clicking again
clears the local highlight (the row itself stays in the audit
trail).
- Pencil icon + double-click to edit step text. Save flows through
update_task_cmd for persistence and records a correction feedback
row with (original_text, corrected_text) — the highest-value
training signal.
- Parent task text is captured in context_json.input at record time
so the prompt builder can reconstruct the (input, preferred-output)
pair on subsequent decompositions.
- Feedback capture is best-effort — a record_feedback failure never
interrupts the primary action.
What's deferred to a later phase:
- Thumbs + corrections on extracted tasks (same pipeline, different
surface — probably TasksPage after the AI-extraction path)
- Thumbs on transcript cleanup output
- Semantic retrieval over the feedback corpus (once there is enough
data to justify embedding infrastructure; the storage shape is
already ready for it)
kon-llm now owns a real LlamaBackend + LlamaModel, with three Qwen3 tiers
(1.7B Q4, 4B-Instruct-2507 Q4, 14B Q5) selectable per hardware. Downloads
are resumable with SHA-256 verification and stored under ~/.kon/models/llm.
Engine exposes three high-level surfaces — all greedy/temp-0, GBNF-constrained
where output shape matters:
- cleanup_text (prompt-injection-hardened system prompt; profile terms
appended as "preserve these spellings" suffix)
- decompose_task (3–7 micro-steps, constrained JSON array)
- extract_tasks (optional-array; empty when no explicit commitments)
post_process_segments now takes an Option<&LlmEngine> and, when loaded and
format_mode != Raw, joins segments → cleanup → replaces segments with the
cleaned text (first segment span). Rule-based path still runs first; LLM
errors log and keep rule-based output.
Tauri commands: recommend_llm_tier, check_llm_model, download_llm_model,
load_llm_model, unload_llm_model, delete_llm_model, get_llm_status,
cleanup_transcript_text_cmd, extract_tasks_from_transcript_cmd,
decompose_and_store (LLM-backed subtasks).
Settings: AI tier toggle (off / cleanup / tasks), model picker with
downloaded/loaded status, download progress events via
kon:llm-download-progress.
Dictation: ensureLlmModelLoaded on mount, cleanupTranscriptIfEnabled after
stop when tier != off and format_mode != Raw, LLM task extraction when
tier=tasks (regex fallback on failure).
Interim: both llama-cpp-sys-2 and whisper-rs-sys statically link their own
ggml, so src-tauri/build.rs emits -Wl,--allow-multiple-definition on Linux.
Replace with a system-ggml shared-lib setup as a follow-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Major quality pass on top of Phase 2. Five substantive changes plus
cross-cutting touches across audio, hotkey, transcription, and Tauri
command layers.
Transcription quality
- Long-audio chunking in commands/transcription.rs: Parakeet and large
file transcription now chunk-and-recompose with overlap trimming, so
the live-path chunking advantage extends to file-based workflows.
- Stateful live speech gate in commands/live.rs on top of the earlier
duplicate-boundary filtering — distinguishes start-of-speech from
mid-speech and holds state across chunks.
Auto-learning corrections
- New crates/ai-formatting/src/correction_learning.rs: extracts user
text corrections from viewer edits and proposes additions to the
active profile's vocabulary.
- src-tauri/src/commands/profiles.rs bridge for frontend-driven
confirmation of learned terms.
- src/routes/viewer/+page.svelte hooks the learning path into the
segment-edit flow so corrections feed profile_terms without a
separate 'train this profile' UX.
Transcript profile provenance
- Migration v8 (crates/storage/src/migrations.rs) adds profile_id to
transcripts, defaulting to DEFAULT_PROFILE_ID so existing rows stay
valid.
- crates/storage/src/database.rs: TranscriptRow + CRUD carry profile_id.
- src-tauri/src/commands/transcripts.rs: add_transcript accepts and
persists profile_id.
- DictationPage.svelte + FilesPage.svelte send activeProfileId on
capture so learned corrections are attributed to the right profile.
Cleanup prompt contract
- crates/ai-formatting/src/llm_client.rs hardened: the CLEANUP_PROMPT
now specifies concrete do/do-not rules, ready for a real model-backed
cleanup pass. The llm_client is still a stub — kon-llm remains unwired
— but the prompt shape is final.
Cross-cutting polish
- Minor touches in audio (capture/decode/resample), hotkey (lib/linux/stub),
core, transcription (concurrency/model_manager/local_engine/whisper_rs),
and the rest of src-tauri/src/commands/*: error-path tightening, log
clarity, TS-migration follow-ups (@ts-nocheck additions for incremental
typing).
Verified locally: npm run check, cargo test -p kon-ai-formatting,
cargo test -p kon-storage, cargo test -p kon --lib commands::live::tests,
cargo check — all green.
Scope boundary: kon-llm crate is still a stub; task extraction remains
rule-based. Bundled local-LLM runtime is the next clean step and is not
in this commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
