feat(llm B.1 #27): Test LLM button with classified error diagnostics
Some checks failed
check / cargo check (macos-latest) (push) Has been cancelled
check / cargo check (ubuntu-22.04) (push) Has been cancelled
check / cargo check (windows-latest) (push) Has been cancelled
check / svelte build + lint (push) Has been cancelled

The brief's pain point is opaque load failures: llama-cpp-2's errors
bubble up as raw C++ strings ("cudaMalloc failed: out of memory",
"invalid gguf magic"). A user seeing that has no path to recovery.

New backend command test_llm_model runs a staged diagnostic:
  1. Model not downloaded → `not-downloaded` + download hint.
  2. File size ≤90% of expected → `incomplete` (stalled download)
     + re-download hint. Matters because llama-cpp-2 can segfault
     on truncated GGUF rather than returning cleanly.
  3. Requested model already loaded → `ready`, no side effects.
  4. Otherwise attempt a real load. On failure, classify_llm_load_error
     maps the raw string to one of:
       - load-failed-vram         (OOM / cudaMalloc / allocation)
       - load-failed-corrupt      (GGUF magic / unsupported format)
       - load-failed-permission   (permission denied / access denied)
       - load-failed-other        (catch-all)
     Each category has a prewritten actionable hint pointing at the
     specific Settings surface (tier picker, re-download, file perms).

classify_llm_load_error is pure-string and unit-tested — 8 cases
covering the main categories plus edge cases (OOM alias, Windows
"Access is denied", unknown errors). Ordered narrow-to-broad so
overlap doesn't misclassify.

Settings UI gets a "Test" button in the AI section's action row,
visible whenever the model is downloaded (both downloaded-idle and
loaded states). Shows inline hint below the status line when the
test surfaces one. Refreshes both local and global LLM status after
the test since a successful test implicitly loads the model.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-21 17:04:11 +01:00
parent 70b97c5273
commit a57da0feb5
3 changed files with 257 additions and 0 deletions

View File

@@ -120,6 +120,216 @@ pub fn get_llm_status(state: State<'_, AppState>) -> Result<bool, String> {
Ok(state.llm_engine.is_loaded())
}
/// Diagnostic result for the Settings "Test LLM" button (brief item
/// B.1 #27). Classifies LLM setup failures into actionable categories
/// instead of surfacing a raw llama.cpp error string.
#[derive(Debug, serde::Serialize)]
#[serde(rename_all = "camelCase")]
pub struct LlmTestResult {
/// One of: "ready", "not-downloaded", "incomplete",
/// "load-failed-vram", "load-failed-corrupt",
/// "load-failed-permission", "load-failed-other".
pub category: String,
/// `true` when the LLM is healthy and usable after the test.
pub ok: bool,
/// One-line status copy for the Settings chip ("Qwen3 4B ready").
pub message: String,
/// Optional actionable next step ("Click Download", "Delete and
/// re-download", "Pick a smaller tier"). Absent when the state is
/// healthy.
pub hint: Option<String>,
}
/// Best-effort LLM health check. Behaviour:
/// 1. Model not downloaded → reports `not-downloaded` with a
/// download hint.
/// 2. File present but size is ≤90% of expected → reports
/// `incomplete` (stalled download) with a re-download hint.
/// 3. Same model already loaded → returns `ready` without disturbing
/// the engine.
/// 4. Otherwise attempts `engine.load_model(...)` and classifies any
/// error string via `classify_llm_load_error` — VRAM exhaustion,
/// GGUF magic mismatch, filesystem permissions, or
/// everything-else. Success returns `ready`.
///
/// The point is that the user sees "Not enough GPU memory — pick a
/// smaller tier" rather than a raw C++ exception bubbled up from
/// llama.cpp. Mirrors OpenWhispr's "Test connection" UX for cloud
/// LLMs, adapted to Kon's local stack.
#[tauri::command]
pub async fn test_llm_model(
state: State<'_, AppState>,
model_id: String,
) -> Result<LlmTestResult, String> {
let id = parse_model_id(model_id)?;
let info = model_info(id);
let path = model_manager::model_path(id);
if !path.exists() {
return Ok(LlmTestResult {
category: "not-downloaded".into(),
ok: false,
message: format!("{} is not downloaded.", info.display_name),
hint: Some(format!(
"Click Download in Settings → AI (~{} MB).",
info.size_bytes / 1_000_000
)),
});
}
// Partial-download detection: llama-cpp-2 will segfault or panic
// on a truncated GGUF rather than returning a clean error, so
// catch it here before we attempt a load. 10% tolerance because
// the expected size is rounded in model_manager.
if let Ok(metadata) = std::fs::metadata(&path) {
let actual = metadata.len();
let minimum = info.size_bytes.saturating_sub(info.size_bytes / 10);
if actual < minimum {
return Ok(LlmTestResult {
category: "incomplete".into(),
ok: false,
message: format!(
"{} file is incomplete ({} MB of expected {} MB).",
info.display_name,
actual / 1_000_000,
info.size_bytes / 1_000_000
),
hint: Some("Delete and re-download from Settings → AI.".into()),
});
}
}
// Already loaded — no need to disturb the engine just to confirm.
if state.llm_engine.loaded_model_id().as_deref() == Some(id.as_str()) {
return Ok(LlmTestResult {
category: "ready".into(),
ok: true,
message: format!("{} loaded and ready.", info.display_name),
hint: None,
});
}
// Not currently loaded. Attempt a real load (with GPU by default
// — matches load_llm_model's default) and classify any failure.
let engine = state.llm_engine.clone();
let load_result = tokio::task::spawn_blocking(move || engine.load_model(id, &path, true))
.await
.map_err(|e| e.to_string())?;
match load_result {
Ok(()) => Ok(LlmTestResult {
category: "ready".into(),
ok: true,
message: format!("{} loaded and ready.", info.display_name),
hint: None,
}),
Err(err) => {
let raw = err.to_string();
let (category, hint) = classify_llm_load_error(&raw);
Ok(LlmTestResult {
category: category.into(),
ok: false,
message: format!("Load failed: {raw}"),
hint: Some(hint.into()),
})
}
}
}
/// Pure string classifier so the test_llm_model command stays
/// unit-testable without spinning up an actual LlmEngine. Order of
/// checks matters — permission errors can contain the word "failed"
/// too, so we check narrower categories before the catch-all.
fn classify_llm_load_error(raw: &str) -> (&'static str, &'static str) {
let lower = raw.to_lowercase();
if lower.contains("out of memory")
|| lower.contains("oom")
|| lower.contains("allocation failed")
|| lower.contains("vram")
|| lower.contains("cudamalloc")
{
(
"load-failed-vram",
"Not enough GPU memory. Pick a smaller tier in Settings → AI, or disable GPU acceleration (Advanced → GPU Tuning).",
)
} else if lower.contains("magic")
|| lower.contains("invalid gguf")
|| lower.contains("unsupported file format")
|| lower.contains("tensor shape")
{
(
"load-failed-corrupt",
"Model file appears corrupt or unsupported. Delete and re-download from Settings → AI.",
)
} else if lower.contains("permission denied") || lower.contains("access is denied") {
(
"load-failed-permission",
"Permission denied reading the model file. Check ownership of ~/.kon/models/llm/.",
)
} else {
(
"load-failed-other",
"Unexpected load error. See Settings → About → Diagnostics bundle.",
)
}
}
#[cfg(test)]
mod tests {
use super::classify_llm_load_error;
#[test]
fn classifies_vram_exhaustion() {
let (category, hint) = classify_llm_load_error("cudaMalloc failed: out of memory");
assert_eq!(category, "load-failed-vram");
assert!(hint.contains("smaller tier"));
}
#[test]
fn classifies_oom_alias() {
let (category, _) = classify_llm_load_error("OOM while allocating 4096 MB");
assert_eq!(category, "load-failed-vram");
}
#[test]
fn classifies_generic_allocation_failure_as_vram() {
let (category, _) = classify_llm_load_error("allocation failed at step 7");
assert_eq!(category, "load-failed-vram");
}
#[test]
fn classifies_gguf_magic_mismatch() {
let (category, hint) = classify_llm_load_error("invalid gguf magic bytes");
assert_eq!(category, "load-failed-corrupt");
assert!(hint.contains("re-download"));
}
#[test]
fn classifies_unsupported_format() {
let (category, _) = classify_llm_load_error("Unsupported file format for model");
assert_eq!(category, "load-failed-corrupt");
}
#[test]
fn classifies_permission_denied() {
let (category, hint) = classify_llm_load_error("os error 13: Permission denied");
assert_eq!(category, "load-failed-permission");
assert!(hint.contains("ownership"));
}
#[test]
fn classifies_windows_access_denied() {
let (category, _) = classify_llm_load_error("Access is denied. (os error 5)");
assert_eq!(category, "load-failed-permission");
}
#[test]
fn classifies_unknown_error_as_other() {
let (category, _) = classify_llm_load_error("Quantum entanglement disrupted");
assert_eq!(category, "load-failed-other");
}
}
#[tauri::command]
pub async fn cleanup_transcript_text_cmd(
state: State<'_, AppState>,