feat(llm): wire Phase 3 local LLM runtime via llama-cpp-2

kon-llm now owns a real LlamaBackend + LlamaModel, with three Qwen3 tiers
(1.7B Q4, 4B-Instruct-2507 Q4, 14B Q5) selectable per hardware. Downloads
are resumable with SHA-256 verification and stored under ~/.kon/models/llm.

Engine exposes three high-level surfaces — all greedy/temp-0, GBNF-constrained
where output shape matters:
- cleanup_text (prompt-injection-hardened system prompt; profile terms
  appended as "preserve these spellings" suffix)
- decompose_task (3–7 micro-steps, constrained JSON array)
- extract_tasks (optional-array; empty when no explicit commitments)

post_process_segments now takes an Option<&LlmEngine> and, when loaded and
format_mode != Raw, joins segments → cleanup → replaces segments with the
cleaned text (first segment span). Rule-based path still runs first; LLM
errors log and keep rule-based output.

Tauri commands: recommend_llm_tier, check_llm_model, download_llm_model,
load_llm_model, unload_llm_model, delete_llm_model, get_llm_status,
cleanup_transcript_text_cmd, extract_tasks_from_transcript_cmd,
decompose_and_store (LLM-backed subtasks).

Settings: AI tier toggle (off / cleanup / tasks), model picker with
downloaded/loaded status, download progress events via
kon:llm-download-progress.

Dictation: ensureLlmModelLoaded on mount, cleanupTranscriptIfEnabled after
stop when tier != off and format_mode != Raw, LLM task extraction when
tier=tasks (regex fallback on failure).

Interim: both llama-cpp-sys-2 and whisper-rs-sys statically link their own
ggml, so src-tauri/build.rs emits -Wl,--allow-multiple-definition on Linux.
Replace with a system-ggml shared-lib setup as a follow-up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-21 07:31:51 +01:00
parent 34fce3cf9e
commit d1eb56fac9
21 changed files with 1598 additions and 43 deletions

View File

@@ -4,6 +4,8 @@ export type ReduceMotion = "system" | "on" | "off";
export type RecordingEngine = "whisper" | "parakeet";
export type FormatMode = "Raw" | "Clean" | "Smart";
export type WhisperModelSize = "Tiny" | "Base" | "Small" | "Medium";
export type AiTier = "off" | "cleanup" | "tasks";
export type LlmModelIdStr = "qwen3_1_7b" | "qwen3_4b_instruct_2507" | "qwen3_14b";
export type TaskBucket = "inbox" | "today" | "soon" | "later";
export type ToastSeverity = "info" | "success" | "warn" | "error";
@@ -31,8 +33,8 @@ export interface SettingsState {
includeTimestamps: boolean;
theme: "Dark" | "Light" | "System";
fontSize: number;
llmModelSize: string;
llmEnabled: boolean;
aiTier: AiTier;
llmModelId: LlmModelIdStr | null;
saveAudio: boolean;
outputFolder: string;
globalHotkey: string;