Files
Lumotia/src-tauri/src/commands/llm.rs
Jake 3770815fbf
Some checks failed
check / cargo check (macos-latest) (push) Has been cancelled
check / cargo check (ubuntu-22.04) (push) Has been cancelled
check / cargo check (windows-latest) (push) Has been cancelled
check / svelte build + lint (push) Has been cancelled
agent: lumotia — v0.1 release-completion run
Closes the code-side v0.1 ship gate. All quality gates green:
cargo fmt/clippy/test (~327 tests), npm check (0/0), vitest 13/13,
scripts/dogfood-rebrand-drill.sh 8/8.

Phase F — first-run onboarding promoted to v0.1
- FirstRunPage with skip-to-main + failure recovery + event recording
- Six onboarding commands (record/list/has-completed + lumotia_events)
- Storage migration v17 (onboarding_events + lumotia_events tables)

UI hardening (in-scope items from v0.1-ui-hardening.md)
- StatusPill + PostCaptureCard components, 21st preview entry
- Sidebar recording-as-sacred-state (opacity + aria-disabled, reduced-motion)
- Settings 6-section regroup + Help section + Activation log + Privacy toggle
- Error-state copy sweep (DictationPage + SettingsPage, plain-language)
- Global :focus-visible rule, textarea outlines restored
- Ctrl+K / Ctrl+, / Escape bindings in +layout

LLM resilience
- rule_based_extract_tasks (regex-free imperative-verb extractor) +
  extract_tasks_with_fallback wrapper — task extraction never returns zero
- tokio::time::timeout(120s) wraps cleanup/tags/tasks commands

Release artefacts
- LICENSE (canonical AGPL-3.0), CHANGELOG (Keep-a-Changelog format)
- v0.1-release-notes, privacy-and-ai-use, install-warnings,
  tester-onboarding-kit, tester-acceptance-runbook, code-signing-setup,
  apple-silicon-rb08-runbook, virtual-audio-setup, v0.1-contrast-audit
- Workspace versioning + AGPL spdx; npm exact-pin (10 ranges removed)
- AppImage SHA-256 sidecar in build.yml
- README v0.1 section + Reporting-issues; canonical repo slug

Closure pass — items moved from human-required to code-complete
- KI-02 Linux idle inhibit: zbus 5 → org.freedesktop.login1.Manager.Inhibit
- KI-03 Windows sleep prevention: SetThreadExecutionState(ES_CONTINUOUS|...)
- acquire/release_idle_inhibit Tauri commands, wired in DictationPage
- Diagnostic-bundle frontend wire-up (Settings → Help button)
- WCAG-AA contrast fix via .btn-filled-text utility (no token changes)
- 8 destructive-action sites wrapped in plain-language confirm() guards
- KNOWN-ISSUES.md + v0.1-known-limitations.md updated (KI-02/03 fixed)

Scripts
- pre-tag-verify.sh, tag-day.sh, smoke-linux + driver
- parse-diagnostic-bundle.sh, parse-activation-log.py

Per-item audit trail: docs/release/v0.1-completion-status.md
Remaining: W-01…W-08 (signing certs, hardware probes, smoke matrix,
tester recruitment) — see docs/release/v0.1-known-limitations.md.
2026-05-15 06:59:08 +01:00

459 lines
16 KiB
Rust

use tauri::{Emitter, State};
use tokio::time::{timeout, Duration};
use crate::commands::power::PowerAssertion;
use crate::commands::security::ensure_main_window;
use crate::AppState;
use lumotia_ai_formatting::{llm_cleanup_text, LlmPromptPreset};
use lumotia_core::hardware;
use lumotia_llm::model_manager::{self, model_info};
use lumotia_llm::{ContentTags, LlmModelId};
const LLM_TIMEOUT: Duration = Duration::from_secs(120);
#[derive(Debug, serde::Serialize)]
#[serde(rename_all = "camelCase")]
pub struct LlmModelStatusDto {
pub id: String,
pub display_name: String,
pub downloaded: bool,
pub loaded: bool,
pub size_bytes: u64,
pub description: String,
pub minimum_ram_bytes: u64,
pub recommended_vram_bytes: Option<u64>,
}
fn parse_model_id(model_id: String) -> Result<LlmModelId, String> {
model_id.parse()
}
#[tauri::command]
pub fn recommend_llm_tier() -> Result<String, String> {
let profile = hardware::probe_system();
let ram_bytes = profile.ram.0.saturating_mul(1024 * 1024);
let vram_bytes = profile
.gpu
.map(|gpu| gpu.vram.0.saturating_mul(1024 * 1024));
Ok(model_manager::recommend_tier(ram_bytes, vram_bytes)
.as_str()
.to_string())
}
#[tauri::command]
pub fn check_llm_model(
state: State<'_, AppState>,
model_id: String,
) -> Result<LlmModelStatusDto, String> {
let id = parse_model_id(model_id)?;
let info = model_info(id);
let loaded_model_id = state.llm_engine.loaded_model_id();
Ok(LlmModelStatusDto {
id: info.id,
display_name: info.display_name.to_string(),
downloaded: model_manager::is_downloaded(id),
loaded: loaded_model_id.as_deref() == Some(id.as_str()),
size_bytes: info.size_bytes,
description: info.description.to_string(),
minimum_ram_bytes: info.minimum_ram_bytes,
recommended_vram_bytes: info.recommended_vram_bytes,
})
}
#[tauri::command]
pub async fn download_llm_model(
window: tauri::WebviewWindow,
app: tauri::AppHandle,
model_id: String,
) -> Result<(), String> {
ensure_main_window(&window)?;
let id = parse_model_id(model_id)?;
let app_clone = app.clone();
model_manager::download_model(id, move |done, total| {
let percent = if total > 0 {
((done as f64 / total as f64) * 100.0).round() as u8
} else {
0
};
let _ = app_clone.emit(
"lumotia:llm-download-progress",
serde_json::json!({
"modelId": id.as_str(),
"done": done,
"total": total,
"percent": percent,
}),
);
})
.await
.map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn load_llm_model(
window: tauri::WebviewWindow,
state: State<'_, AppState>,
model_id: String,
use_gpu: Option<bool>,
concurrent: Option<bool>,
) -> Result<(), String> {
ensure_main_window(&window)?;
let id = parse_model_id(model_id)?;
let path = model_manager::model_path(id);
if !path.exists() {
return Err("Model not downloaded — call download_llm_model first".to_string());
}
// Sequential-GPU guard (brief item A.1 #28): when the user has opted
// out of concurrent GPU residency, free the transcription engines
// before loading the LLM. Prevents VRAM OOM on tight-GPU setups.
// concurrent=None or Some(true) preserves legacy parallel behaviour.
if concurrent == Some(false) {
state.whisper_engine.unload();
state.parakeet_engine.unload();
}
let engine = state.llm_engine.clone();
let use_gpu = use_gpu.unwrap_or(true);
tokio::task::spawn_blocking(move || engine.load_model(id, &path, use_gpu))
.await
.map_err(|e| e.to_string())?
.map_err(|e| e.to_string())
}
#[tauri::command]
pub fn unload_llm_model(
window: tauri::WebviewWindow,
state: State<'_, AppState>,
) -> Result<(), String> {
ensure_main_window(&window)?;
state.llm_engine.unload().map_err(|e| e.to_string())
}
#[tauri::command]
pub fn delete_llm_model(
window: tauri::WebviewWindow,
state: State<'_, AppState>,
model_id: String,
) -> Result<(), String> {
ensure_main_window(&window)?;
let id = parse_model_id(model_id)?;
if state.llm_engine.loaded_model_id().as_deref() == Some(id.as_str()) {
state.llm_engine.unload().map_err(|e| e.to_string())?;
}
model_manager::delete_model(id).map_err(|e| e.to_string())
}
#[tauri::command]
pub fn get_llm_status(state: State<'_, AppState>) -> Result<bool, String> {
Ok(state.llm_engine.is_loaded())
}
/// Diagnostic result for the Settings "Test LLM" button (brief item
/// B.1 #27). Classifies LLM setup failures into actionable categories
/// instead of surfacing a raw llama.cpp error string.
#[derive(Debug, serde::Serialize)]
#[serde(rename_all = "camelCase")]
pub struct LlmTestResult {
/// One of: "ready", "not-downloaded", "incomplete",
/// "load-failed-vram", "load-failed-corrupt",
/// "load-failed-permission", "load-failed-other".
pub category: String,
/// `true` when the LLM is healthy and usable after the test.
pub ok: bool,
/// One-line status copy for the Settings chip ("Qwen3.5 4B ready").
pub message: String,
/// Optional actionable next step ("Click Download", "Delete and
/// re-download", "Pick a smaller tier"). Absent when the state is
/// healthy.
pub hint: Option<String>,
}
/// Best-effort LLM health check. Behaviour:
/// 1. Model not downloaded → reports `not-downloaded` with a
/// download hint.
/// 2. File present but size is ≤90% of expected → reports
/// `incomplete` (stalled download) with a re-download hint.
/// 3. Same model already loaded → returns `ready` without disturbing
/// the engine.
/// 4. Otherwise attempts `engine.load_model(...)` and classifies any
/// error string via `classify_llm_load_error` — VRAM exhaustion,
/// GGUF magic mismatch, filesystem permissions, or
/// everything-else. Success returns `ready`.
///
/// The point is that the user sees "Not enough GPU memory — pick a
/// smaller tier" rather than a raw C++ exception bubbled up from
/// llama.cpp. Mirrors OpenWhispr's "Test connection" UX for cloud
/// LLMs, adapted to Lumotia's local stack.
#[tauri::command]
pub async fn test_llm_model(
window: tauri::WebviewWindow,
state: State<'_, AppState>,
model_id: String,
use_gpu: Option<bool>,
) -> Result<LlmTestResult, String> {
ensure_main_window(&window)?;
let id = parse_model_id(model_id)?;
let info = model_info(id);
let path = model_manager::model_path(id);
if !path.exists() {
return Ok(LlmTestResult {
category: "not-downloaded".into(),
ok: false,
message: format!("{} is not downloaded.", info.display_name),
hint: Some(format!(
"Click Download in Settings → AI (~{} MB).",
info.size_bytes / 1_000_000
)),
});
}
// Partial-download detection: llama-cpp-2 will segfault or panic
// on a truncated GGUF rather than returning a clean error, so
// catch it here before we attempt a load. 10% tolerance because
// the expected size is rounded in model_manager.
if let Ok(metadata) = std::fs::metadata(&path) {
let actual = metadata.len();
let minimum = info.size_bytes.saturating_sub(info.size_bytes / 10);
if actual < minimum {
return Ok(LlmTestResult {
category: "incomplete".into(),
ok: false,
message: format!(
"{} file is incomplete ({} MB of expected {} MB).",
info.display_name,
actual / 1_000_000,
info.size_bytes / 1_000_000
),
hint: Some("Delete and re-download from Settings → AI.".into()),
});
}
}
// Already loaded — no need to disturb the engine just to confirm.
if state.llm_engine.loaded_model_id().as_deref() == Some(id.as_str()) {
return Ok(LlmTestResult {
category: "ready".into(),
ok: true,
message: format!("{} loaded and ready.", info.display_name),
hint: None,
});
}
// Not currently loaded. Attempt a real load with the caller's
// GPU preference (matching load_llm_model's default of `true` when
// unspecified). Hard-coding `true` here used to race with a parallel
// load_llm_model(use_gpu=false) — LlmEngine::load_model decides
// "same model?" via (id, path, use_gpu) triple-equality and tears
// the engine down on mismatch, silently flipping the user's GPU
// mode underneath the in-flight test.
let resolved_use_gpu = use_gpu.unwrap_or(true);
let engine = state.llm_engine.clone();
let load_result =
tokio::task::spawn_blocking(move || engine.load_model(id, &path, resolved_use_gpu))
.await
.map_err(|e| e.to_string())?;
match load_result {
Ok(()) => Ok(LlmTestResult {
category: "ready".into(),
ok: true,
message: format!("{} loaded and ready.", info.display_name),
hint: None,
}),
Err(err) => {
let raw = err.to_string();
let (category, hint) = classify_llm_load_error(&raw);
Ok(LlmTestResult {
category: category.into(),
ok: false,
message: format!("Load failed: {raw}"),
hint: Some(hint.into()),
})
}
}
}
/// Pure string classifier so the test_llm_model command stays
/// unit-testable without spinning up an actual LlmEngine. Order of
/// checks matters — permission errors can contain the word "failed"
/// too, so we check narrower categories before the catch-all.
fn classify_llm_load_error(raw: &str) -> (&'static str, &'static str) {
let lower = raw.to_lowercase();
if lower.contains("out of memory")
|| lower.contains("oom")
|| lower.contains("allocation failed")
|| lower.contains("vram")
|| lower.contains("cudamalloc")
{
(
"load-failed-vram",
"Not enough GPU memory. Pick a smaller tier in Settings → AI, or disable GPU acceleration (Advanced → GPU Tuning).",
)
} else if lower.contains("magic")
|| lower.contains("invalid gguf")
|| lower.contains("unsupported file format")
|| lower.contains("tensor shape")
{
(
"load-failed-corrupt",
"Model file appears corrupt or unsupported. Delete and re-download from Settings → AI.",
)
} else if lower.contains("permission denied") || lower.contains("access is denied") {
(
"load-failed-permission",
"Permission denied reading the model file. Check ownership of ~/.lumotia/models/llm/.",
)
} else {
(
"load-failed-other",
"Unexpected load error. See Settings → About → Diagnostics bundle.",
)
}
}
#[cfg(test)]
mod tests {
use super::classify_llm_load_error;
#[test]
fn classifies_vram_exhaustion() {
let (category, hint) = classify_llm_load_error("cudaMalloc failed: out of memory");
assert_eq!(category, "load-failed-vram");
assert!(hint.contains("smaller tier"));
}
#[test]
fn classifies_oom_alias() {
let (category, _) = classify_llm_load_error("OOM while allocating 4096 MB");
assert_eq!(category, "load-failed-vram");
}
#[test]
fn classifies_generic_allocation_failure_as_vram() {
let (category, _) = classify_llm_load_error("allocation failed at step 7");
assert_eq!(category, "load-failed-vram");
}
#[test]
fn classifies_gguf_magic_mismatch() {
let (category, hint) = classify_llm_load_error("invalid gguf magic bytes");
assert_eq!(category, "load-failed-corrupt");
assert!(hint.contains("re-download"));
}
#[test]
fn classifies_unsupported_format() {
let (category, _) = classify_llm_load_error("Unsupported file format for model");
assert_eq!(category, "load-failed-corrupt");
}
#[test]
fn classifies_permission_denied() {
let (category, hint) = classify_llm_load_error("os error 13: Permission denied");
assert_eq!(category, "load-failed-permission");
assert!(hint.contains("ownership"));
}
#[test]
fn classifies_windows_access_denied() {
let (category, _) = classify_llm_load_error("Access is denied. (os error 5)");
assert_eq!(category, "load-failed-permission");
}
#[test]
fn classifies_unknown_error_as_other() {
let (category, _) = classify_llm_load_error("Quantum entanglement disrupted");
assert_eq!(category, "load-failed-other");
}
}
#[tauri::command]
pub async fn cleanup_transcript_text_cmd(
window: tauri::WebviewWindow,
state: State<'_, AppState>,
transcript: String,
profile_id: Option<String>,
preset: Option<String>,
) -> Result<String, String> {
ensure_main_window(&window)?;
let resolved_profile_id =
profile_id.unwrap_or_else(|| lumotia_storage::DEFAULT_PROFILE_ID.to_string());
let profile_terms: Vec<String> =
lumotia_storage::database::list_profile_terms(&state.db, &resolved_profile_id)
.await
.map_err(|e| e.to_string())?
.into_iter()
.map(|term| term.term)
.collect();
// Named preset (brief item B.1 #15): Email / Notes / Code shape the
// output tone + structure without changing the translator-not-editor
// contract. None or unknown → Default (no additional guidance).
let resolved_preset = preset
.as_deref()
.map(LlmPromptPreset::parse)
.unwrap_or(LlmPromptPreset::Default);
let engine = state.llm_engine.clone();
let cleanup_future = tokio::task::spawn_blocking(move || {
// macOS: pin a power assertion for the duration of the LLM
// generation so App Nap can't decide to throttle us mid-token.
// No-op on every other OS. Item #9.
let _power_guard = PowerAssertion::begin("lumotia LLM cleanup");
llm_cleanup_text(&engine, &transcript, &profile_terms, resolved_preset)
});
match timeout(LLM_TIMEOUT, cleanup_future).await {
Ok(Ok(value)) => value.map_err(|e| e.to_string()),
Ok(Err(e)) => Err(e.to_string()),
Err(_elapsed) => {
tracing::warn!(
"LLM cleanup_transcript_text_cmd timed out after {:?}",
LLM_TIMEOUT
);
Err("Cleanup took too long. The raw transcript is preserved — try again, or continue without cleanup.".to_string())
}
}
}
/// Phase 9 LLM-powered content tags. On-demand from the History page;
/// never auto-runs. Heavy work (LlmEngine::extract_content_tags is
/// synchronous llama-cpp inference) is wrapped in spawn_blocking so it
/// does not stall the Tauri runtime, with the same App-Nap power
/// assertion the other LLM commands use.
///
/// Trust-4 (conf 85, code-atomiser-fix 2026-05-12): added the
/// `ensure_main_window` guard that every other command in this file
/// already enforces. The `window: tauri::WebviewWindow` parameter is
/// injected by Tauri automatically — frontend invoke call sites
/// (HistoryPage) do not need updating.
#[tauri::command]
pub async fn extract_content_tags_cmd(
window: tauri::WebviewWindow,
state: State<'_, AppState>,
transcript: String,
) -> Result<ContentTags, String> {
ensure_main_window(&window)?;
if !state.llm_engine.is_loaded() {
return Err("LLM not loaded. Download an AI model in Settings.".to_string());
}
let engine = state.llm_engine.clone();
let tag_future = tokio::task::spawn_blocking(move || {
let _power_guard = PowerAssertion::begin("lumotia LLM content-tag extraction");
engine.extract_content_tags(&transcript)
});
match timeout(LLM_TIMEOUT, tag_future).await {
Ok(Ok(value)) => value.map_err(|e| e.to_string()),
Ok(Err(e)) => Err(e.to_string()),
Err(_elapsed) => {
tracing::warn!(
"LLM extract_content_tags_cmd timed out after {:?}",
LLM_TIMEOUT
);
Err("Tagging took too long. Your transcript is unchanged — you can retry from the post-capture card.".to_string())
}
}
}