Jake
26c7307607
Phase 9 of the rebrand cascade. Sweep covers everything the Phase 8
frontend pass deliberately skipped: docs/, root markdown, scripts,
Cargo.toml descriptions, code comments that survived earlier
word-boundary sed, plus a handful of identifiers caught on the final
verify pass.
transcription-app changes:
- README.md, HANDOVER.md, KNOWN-ISSUES.md, run.sh — magnotia/Magnotia
-> lumotia/Lumotia.
- docs/ — sweep across all subdirs except docs/handovers/ (preserved
as immutable audit trail). Includes architecture-map references
to magnotia_core::*, magnotia_storage::*, etc. now pointing at
lumotia_*; dev-setup.md tracing output examples (lumotia_startup
target); brief/ + superpowers/ + issues/ + whisper-ecosystem/ +
audit/.
- Cargo.toml descriptions on 9 crates (core, audio, cloud-providers,
hotkey, llm, mcp, plus referenced others).
- crates/core/src/{error,hardware,recommendation,paths}.rs +
crates/audio/src/wav.rs + crates/llm/src/model_manager.rs +
crates/cloud-providers/src/keystore.rs + crates/mcp/src/lib.rs —
doc comments and a model-manager user-agent string.
- Caught on final pass: BroadcastChannel("magnotia_task_sync") -> ...
("lumotia_task_sync"); magnotia_locale i18n localStorage key
renamed + migration shim added; CSS keyframe names
magnotiaPulse / magnotiaBar / magnotiaFade renamed in the design-
system kit; magnotia_viewer_item / magnotia_viewer_mode handoff
keys renamed in HistoryPage + viewer/+page.svelte; src/assets/
wordmark.svg text.
- src-tauri/src/lib.rs comment cleanup ("magnotia era" was sed'd
to "lumotia era" earlier — restored).
Preserved (intentional):
- crates/core/src/paths.rs — keeps "magnotia" / "Magnotia" / ".magnotia"
legacy detection strings in legacy_and_target_paths() so the
migration shim can still find user data from the magnotia era.
- src/lib/stores/{page,focusTimer}.svelte.ts + src/lib/i18n/index.ts
— migration call sites reference the legacy magnotia keys
deliberately.
- docs/handovers/ — historical audit trail.
cargo build --workspace passes. npm run check: 0 errors / 0 warnings
(3958 files). cargo test --workspace: 339 pass / 0 fail.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
119 lines
8.7 KiB
Markdown
119 lines
8.7 KiB
Markdown
---
|
|
name: Local LLM commands
|
|
type: architecture-map-page
|
|
slice: 02-tauri-runtime
|
|
last_verified: 2026/05/09
|
|
---
|
|
|
|
# `commands::llm`
|
|
|
|
> **Where you are:** [Architecture map](../../README.md) → [Tauri runtime](../README.md) → [Commands](README.md) → LLM
|
|
|
|
**Plain English summary.** Owns the local LLM lifecycle (recommend tier, check, download with progress events, load with sequential-GPU coordination, unload, delete, status, test) plus two inference commands the frontend uses on demand: cleanup-transcript-text (the "fix this" pass that runs after a transcription) and extract-content-tags (Phase 9, surface topic / intent tags on the History page). Diagnostic test command classifies load failures into VRAM / corrupt / permission / other so Settings shows actionable hints rather than raw llama.cpp exceptions.
|
|
|
|
## At a glance
|
|
|
|
- Path: `src-tauri/src/commands/llm.rs`.
|
|
- LOC: 423.
|
|
- Tauri commands exposed (10 total):
|
|
- `recommend_llm_tier() -> Result<String, String>`. No window guard — pure hardware probe.
|
|
- `check_llm_model(state, model_id) -> Result<LlmModelStatusDto, String>`.
|
|
- `download_llm_model(window, app, model_id) -> Result<(), String>`. Main-window only. Emits `lumotia:llm-download-progress`.
|
|
- `load_llm_model(window, state, model_id, use_gpu, concurrent) -> Result<(), String>`. Main-window only.
|
|
- `unload_llm_model(window, state) -> Result<(), String>`. Main-window only.
|
|
- `delete_llm_model(window, state, model_id) -> Result<(), String>`. Main-window only.
|
|
- `get_llm_status(state) -> Result<bool, String>`.
|
|
- `test_llm_model(window, state, model_id) -> Result<LlmTestResult, String>`. Main-window only.
|
|
- `cleanup_transcript_text_cmd(window, state, transcript, profile_id, preset) -> Result<String, String>`. Main-window only.
|
|
- `extract_content_tags_cmd(state, transcript) -> Result<ContentTags, String>`.
|
|
- Events emitted: `lumotia:llm-download-progress` (`{ modelId, done, total, percent }`) — `src-tauri/src/commands/llm.rs:76`.
|
|
- Depends on: `lumotia_llm::{LlmEngine, LlmModelId, ContentTags, model_manager::{download_model, model_path, model_info, recommend_tier, is_downloaded, delete_model}}`, `lumotia_ai_formatting::{llm_cleanup_text, LlmPromptPreset}`, `lumotia_core::hardware`, `lumotia_storage::database::list_profile_terms`. Plus `commands::power::PowerAssertion`, `commands::security::ensure_main_window`.
|
|
- Called from frontend at: Settings → AI page (download / load / unload / delete / test), dictation result panel (`cleanup_transcript_text_cmd`), History page (`extract_content_tags_cmd`).
|
|
|
|
## What's in here
|
|
|
|
### `LlmModelStatusDto` (`src-tauri/src/commands/llm.rs:11`)
|
|
|
|
Frontend-facing status DTO: id, displayName, downloaded, loaded, sizeBytes, description, minimumRamBytes, recommendedVramBytes.
|
|
|
|
### `parse_model_id` (`:24`)
|
|
|
|
Wraps `model_id.parse()` (which goes through the `LlmModelId` parser).
|
|
|
|
### `recommend_llm_tier` (`:28`)
|
|
|
|
Probes RAM via `lumotia_core::hardware::probe_system`, multiplies the MB to bytes, then defers to `lumotia_llm::model_manager::recommend_tier(ram, vram)`. No window guard — Settings calls this at first paint to populate the default tier suggestion.
|
|
|
|
### `check_llm_model` (`:40`)
|
|
|
|
Combines `model_info(id)` (static metadata), `model_manager::is_downloaded(id)`, and `state.llm_engine.loaded_model_id()` into a `LlmModelStatusDto`. Used by Settings to render per-tier status chips.
|
|
|
|
### `download_llm_model` (`:61`)
|
|
|
|
`ensure_main_window`. Calls `model_manager::download_model(id, progress_cb)` with a closure that emits `lumotia:llm-download-progress` on each chunk. The percent calculation rounds against `total > 0` (avoids division by zero on a server that doesn't return a content-length).
|
|
|
|
### `load_llm_model` (`:90`)
|
|
|
|
`ensure_main_window`, `parse_model_id`, check the file exists. Implements the inverse Phase A.1 sequential-GPU guard: when `concurrent == Some(false)`, unloads BOTH `state.whisper_engine` and `state.parakeet_engine` before loading the LLM. Default `use_gpu = true`. Loads inside `spawn_blocking`.
|
|
|
|
### `unload_llm_model` (`:122`), `delete_llm_model` (`:131`), `get_llm_status` (`:145`)
|
|
|
|
Thin wrappers. `delete_llm_model` unloads first if the model is currently loaded.
|
|
|
|
### `LlmTestResult` (`:155`) and `test_llm_model` (`:186`)
|
|
|
|
Settings → AI's "Test LLM" button. Brief item B.1 #27. Decision tree:
|
|
|
|
1. File missing → `not-downloaded` with a "Click Download" hint.
|
|
2. File present but ≤ 90% of expected size → `incomplete` (truncated download) with a re-download hint. 10% tolerance because `info.size_bytes` is rounded.
|
|
3. Same model already loaded → `ready` without disturbing the engine.
|
|
4. Otherwise attempt `engine.load_model(id, &path, use_gpu=true)` inside `spawn_blocking` and pass any error string to `classify_llm_load_error`.
|
|
|
|
### `classify_llm_load_error` (`:272`)
|
|
|
|
Pure string classifier. Tested independently. Buckets: `load-failed-vram` (looks for `out of memory`, `oom`, `allocation failed`, `vram`, `cudamalloc`), `load-failed-corrupt` (looks for `magic`, `invalid gguf`, `unsupported file format`, `tensor shape`), `load-failed-permission` (looks for `permission denied`, `access is denied`), and `load-failed-other` for everything else. Each bucket pairs with a user-facing hint string.
|
|
|
|
### `cleanup_transcript_text_cmd` (`:362`)
|
|
|
|
`ensure_main_window`. Resolves `profile_id`. Fetches profile_terms from storage. Resolves the `preset` (Brief item B.1 #15: `Default`, `Email`, `Notes`, `Code`). `spawn_blocking` runs `llm_cleanup_text(&engine, &transcript, &profile_terms, resolved_preset)` inside a `PowerAssertion::begin("lumotia LLM cleanup")` so macOS App Nap doesn't throttle mid-token.
|
|
|
|
### `extract_content_tags_cmd` (`:407`)
|
|
|
|
Phase 9 LLM tagging. NO window guard (the History page lives in main but the command would be safe even without it; if you ever expose the History page in a secondary window, add the guard). Errors out cleanly if no LLM is loaded. `spawn_blocking` runs `engine.extract_content_tags(&transcript)` under the same App Nap power-assertion pattern.
|
|
|
|
### Tests (`:306`)
|
|
|
|
Eight `classify_llm_load_error` cases cover the four classification buckets and edge cases (cudaMalloc, OOM, generic allocation failure, GGUF magic mismatch, unsupported file format, permission denied, Windows access denied, unknown error).
|
|
|
|
## Data flow
|
|
|
|
```
|
|
Settings -> recommend_llm_tier() -> hardware probe -> tier string
|
|
Settings -> download_llm_model(model_id) -> model_manager::download_model
|
|
-> per-chunk progress events
|
|
-> file lands in ~/.lumotia/models/llm/
|
|
Settings -> load_llm_model(model_id, use_gpu, concurrent)
|
|
-> if concurrent=false: unload whisper + parakeet
|
|
-> spawn_blocking(engine.load_model)
|
|
Settings -> test_llm_model(model_id) -> tiered checks -> LlmTestResult
|
|
History page -> extract_content_tags_cmd(transcript) -> ContentTags { topics, intents }
|
|
Dictation result -> cleanup_transcript_text_cmd(transcript, profile_id, preset) -> cleaned text
|
|
```
|
|
|
|
## Watch-outs
|
|
|
|
- **Sequential-GPU guard is the inverse here.** `load_llm_model` unloads transcription engines when `concurrent=false`. `commands::models::ensure_model_loaded` unloads the LLM. Settings owns the toggle; live transcription explicitly bypasses it (passes `None`). Confirm both halves stay in sync if you ever change the toggle's semantics.
|
|
- **`extract_content_tags_cmd` lacks `ensure_main_window`.** Intentional today, but if you change the ACL to expose this to a non-main window, add the guard.
|
|
- **`PowerAssertion` is a no-op outside macOS.** Long LLM cleanup on Linux can be idled by the compositor. See [Power assertions and security](power-and-security.md).
|
|
- **Partial-download detection uses 10% tolerance.** A model that ships exactly 90% of expected size will be incorrectly flagged as incomplete. The expected size is rounded by `model_manager`, so empirically this is fine; if you tighten the rounding, also tighten the threshold.
|
|
- **`download_llm_model` emits a percent rounded to `u8`.** A frontend that wants smoother progress bars needs to use the raw `done / total` fields.
|
|
- **The cleanup command silently passes an unrecognised preset as `Default`** (`LlmPromptPreset::parse` returns `Default` for unknown strings). Consider erroring out instead so frontend bugs surface faster.
|
|
|
|
## See also
|
|
|
|
- [Models](models.md) — the inverse sequential-GPU guard.
|
|
- [Tasks](tasks.md) — also calls `engine.decompose_task_with_feedback` / `extract_tasks_with_feedback` under the same App Nap pattern.
|
|
- [Power assertions and security](power-and-security.md) — `PowerAssertion::begin` is shared.
|
|
- [Diagnostics](diagnostics.md) — the test_llm_model failure messages cite "see Settings → About → Diagnostics bundle".
|
|
- [Profiles](profiles.md) — `cleanup_transcript_text_cmd` reads profile_terms from storage.
|