feat(llm): wire Phase 3 local LLM runtime via llama-cpp-2
kon-llm now owns a real LlamaBackend + LlamaModel, with three Qwen3 tiers (1.7B Q4, 4B-Instruct-2507 Q4, 14B Q5) selectable per hardware. Downloads are resumable with SHA-256 verification and stored under ~/.kon/models/llm. Engine exposes three high-level surfaces — all greedy/temp-0, GBNF-constrained where output shape matters: - cleanup_text (prompt-injection-hardened system prompt; profile terms appended as "preserve these spellings" suffix) - decompose_task (3–7 micro-steps, constrained JSON array) - extract_tasks (optional-array; empty when no explicit commitments) post_process_segments now takes an Option<&LlmEngine> and, when loaded and format_mode != Raw, joins segments → cleanup → replaces segments with the cleaned text (first segment span). Rule-based path still runs first; LLM errors log and keep rule-based output. Tauri commands: recommend_llm_tier, check_llm_model, download_llm_model, load_llm_model, unload_llm_model, delete_llm_model, get_llm_status, cleanup_transcript_text_cmd, extract_tasks_from_transcript_cmd, decompose_and_store (LLM-backed subtasks). Settings: AI tier toggle (off / cleanup / tasks), model picker with downloaded/loaded status, download progress events via kon:llm-download-progress. Dictation: ensureLlmModelLoaded on mount, cleanupTranscriptIfEnabled after stop when tier != off and format_mode != Raw, LLM task extraction when tier=tasks (regex fallback on failure). Interim: both llama-cpp-sys-2 and whisper-rs-sys statically link their own ggml, so src-tauri/build.rs emits -Wl,--allow-multiple-definition on Linux. Replace with a system-ggml shared-lib setup as a follow-up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -176,7 +176,12 @@ pub async fn decompose_and_store(
|
||||
.map_err(|e| e.to_string())?
|
||||
.ok_or_else(|| format!("Task {parent_task_id} not found"))?;
|
||||
|
||||
let steps = state.llm_engine.decompose_task(&parent.text)?;
|
||||
let engine = state.llm_engine.clone();
|
||||
let parent_text = parent.text.clone();
|
||||
let steps = tokio::task::spawn_blocking(move || engine.decompose_task(&parent_text))
|
||||
.await
|
||||
.map_err(|e| e.to_string())?
|
||||
.map_err(|e| e.to_string())?;
|
||||
|
||||
let mut created = Vec::new();
|
||||
for text in steps {
|
||||
@@ -195,6 +200,18 @@ pub async fn decompose_and_store(
|
||||
Ok(created)
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
pub async fn extract_tasks_from_transcript_cmd(
|
||||
state: tauri::State<'_, AppState>,
|
||||
transcript: String,
|
||||
) -> Result<Vec<String>, String> {
|
||||
let engine = state.llm_engine.clone();
|
||||
tokio::task::spawn_blocking(move || engine.extract_tasks(&transcript))
|
||||
.await
|
||||
.map_err(|e| e.to_string())?
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
pub async fn list_subtasks_cmd(
|
||||
state: tauri::State<'_, AppState>,
|
||||
|
||||
Reference in New Issue
Block a user