agent: lumotia-rebrand — docs, scripts, root config, residuals
Phase 9 of the rebrand cascade. Sweep covers everything the Phase 8
frontend pass deliberately skipped: docs/, root markdown, scripts,
Cargo.toml descriptions, code comments that survived earlier
word-boundary sed, plus a handful of identifiers caught on the final
verify pass.
transcription-app changes:
- README.md, HANDOVER.md, KNOWN-ISSUES.md, run.sh — magnotia/Magnotia
-> lumotia/Lumotia.
- docs/ — sweep across all subdirs except docs/handovers/ (preserved
as immutable audit trail). Includes architecture-map references
to magnotia_core::*, magnotia_storage::*, etc. now pointing at
lumotia_*; dev-setup.md tracing output examples (lumotia_startup
target); brief/ + superpowers/ + issues/ + whisper-ecosystem/ +
audit/.
- Cargo.toml descriptions on 9 crates (core, audio, cloud-providers,
hotkey, llm, mcp, plus referenced others).
- crates/core/src/{error,hardware,recommendation,paths}.rs +
crates/audio/src/wav.rs + crates/llm/src/model_manager.rs +
crates/cloud-providers/src/keystore.rs + crates/mcp/src/lib.rs —
doc comments and a model-manager user-agent string.
- Caught on final pass: BroadcastChannel("magnotia_task_sync") -> ...
("lumotia_task_sync"); magnotia_locale i18n localStorage key
renamed + migration shim added; CSS keyframe names
magnotiaPulse / magnotiaBar / magnotiaFade renamed in the design-
system kit; magnotia_viewer_item / magnotia_viewer_mode handoff
keys renamed in HistoryPage + viewer/+page.svelte; src/assets/
wordmark.svg text.
- src-tauri/src/lib.rs comment cleanup ("magnotia era" was sed'd
to "lumotia era" earlier — restored).
Preserved (intentional):
- crates/core/src/paths.rs — keeps "magnotia" / "Magnotia" / ".magnotia"
legacy detection strings in legacy_and_target_paths() so the
migration shim can still find user data from the magnotia era.
- src/lib/stores/{page,focusTimer}.svelte.ts + src/lib/i18n/index.ts
— migration call sites reference the legacy magnotia keys
deliberately.
- docs/handovers/ — historical audit trail.
cargo build --workspace passes. npm run check: 0 errors / 0 warnings
(3958 files). cargo test --workspace: 339 pass / 0 fail.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -9,11 +9,11 @@ last_verified: 2026/05/09
|
||||
|
||||
> **Where you are:** [Architecture map](../README.md) → [LLM, Formatting, MCP](README.md) → LLM engine
|
||||
|
||||
**Plain English summary.** `LlmEngine` is the cloneable handle every part of Magnotia uses to talk to a local Qwen model. It owns the llama-cpp-2 backend, holds a single loaded model behind a mutex, and exposes one low-level `generate` plus four high-level surfaces. Everything else in the LLM crate (prompts, grammars, model manager) feeds into this type.
|
||||
**Plain English summary.** `LlmEngine` is the cloneable handle every part of Lumotia uses to talk to a local Qwen model. It owns the llama-cpp-2 backend, holds a single loaded model behind a mutex, and exposes one low-level `generate` plus four high-level surfaces. Everything else in the LLM crate (prompts, grammars, model manager) feeds into this type.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Crate: `magnotia-llm`
|
||||
- Crate: `lumotia-llm`
|
||||
- Path: `crates/llm/src/lib.rs`
|
||||
- LOC: 570
|
||||
- Public surface (from `lib.rs`):
|
||||
@@ -22,7 +22,7 @@ last_verified: 2026/05/09
|
||||
- `pub struct GenerationConfig` (`crates/llm/src/lib.rs:50`).
|
||||
- `pub struct LoadedModelState` (`crates/llm/src/lib.rs:70`).
|
||||
- Re-exports: `CONTENT_TAGS_GRAMMAR`, `recommend_tier`, `LlmModelId`, `LlmModelInfo`, `is_valid_intent`, `ContentTags`, `CONTENT_TAGS_SYSTEM`, `INTENT_CLOSED_SET`.
|
||||
- External deps that matter: `llama-cpp-2 = 0.1.144`, `tokio` (only for the async download path in `model_manager`; `generate` itself is sync), `serde`, `tracing`, `magnotia-core` for thread tuning.
|
||||
- External deps that matter: `llama-cpp-2 = 0.1.144`, `tokio` (only for the async download path in `model_manager`; `generate` itself is sync), `serde`, `tracing`, `lumotia-core` for thread tuning.
|
||||
- Tauri command that calls this (slice 2, best guess):
|
||||
- Lifecycle: `llm_load_model`, `llm_unload_model`, `llm_load_recommended_model`, `llm_is_loaded` etc. defined in `src-tauri/src/commands/llm.rs`. Observed call sites at `src-tauri/src/commands/llm.rs:116` (`engine.load_model`), `:128` (`engine.unload`), `:147` (`engine.is_loaded`), `:244` (`load_model` again from the recommended-tier path).
|
||||
|
||||
@@ -60,7 +60,7 @@ The sync, low-level inference primitive. Steps:
|
||||
1. **Tokenise the prompt.** `model.str_to_token(prompt, AddBos::Never)` — the `AddBos::Never` matters: `render_chat_prompt` already emits the BOS token via the Qwen chat template.
|
||||
2. **Empty-prompt short-circuit.** If tokenisation produced zero tokens, return `Ok(String::new())` without touching the GPU.
|
||||
3. **Preflight context window.** `preflight_context_window(prompt_tokens.len(), config.max_tokens)` (`:436`) errors with `EngineError::PromptTooLong { ... }` when `prompt_tokens + max_tokens + 64 reserve` exceeds the 8192 cap. Fixed sizing — see the watch-out about MAX_CONTEXT_TOKENS below. This was RB-10 from the 2026-04-22 review (`docs/issues/llm-prompt-preflight.md`).
|
||||
4. **Compute thread count.** `gpu_offloaded = use_gpu && gpu_layers >= model.n_layer()`. The compiler can prove this is trivially true today because `gpu_layers` is `u32::MAX` whenever `use_gpu` is set. The redundant check is documented inline (`:169-173`) as a placeholder for future per-layer residency parsing of llama.cpp's log output. `inference_thread_count(Workload::Llm, gpu_offloaded)` from `magnotia_core::tuning` returns the physical core count adjusted for the workload class.
|
||||
4. **Compute thread count.** `gpu_offloaded = use_gpu && gpu_layers >= model.n_layer()`. The compiler can prove this is trivially true today because `gpu_layers` is `u32::MAX` whenever `use_gpu` is set. The redundant check is documented inline (`:169-173`) as a placeholder for future per-layer residency parsing of llama.cpp's log output. `inference_thread_count(Workload::Llm, gpu_offloaded)` from `lumotia_core::tuning` returns the physical core count adjusted for the workload class.
|
||||
5. **Build context params.** `n_ctx` from preflight, `n_batch` and `n_ubatch` clamped to `[max(prompt_tokens, 512), n_ctx]`, `n_threads` and `n_threads_batch` both set to the computed thread count.
|
||||
6. **Prefill.** A single `LlamaBatch` is built with every prompt token, the last token marked as the only logits-bearing position, then `ctx.decode(&mut batch)`.
|
||||
7. **Sample loop.** A custom sampler chain (`build_sampler`, `:400`) is built from `config.grammar` (optional GBNF), `temperature`, and a fixed `GENERATION_SEED = 0`. For temperature 0.0 (the only value the high-level surfaces use) we attach `LlamaSampler::greedy()` after the optional grammar. For non-zero temperatures we attach `temp` then `dist` with the seed.
|
||||
@@ -132,7 +132,7 @@ This file does not hold prompts or grammars itself. See [`llm-prompts-and-gramma
|
||||
|
||||
- **Mutex-protected single model.** `LlmEngine` allows only one model loaded at a time. Two concurrent `generate` calls serialise on the underlying llama context (each call builds its own `new_context` from the shared `LlamaModel`, so the model weights are shared but the KV cache is per-call). The Tauri layer wraps each high-level call in `tokio::task::spawn_blocking` because `generate` is sync and blocks the executor for hundreds of milliseconds at minimum.
|
||||
- **`MAX_CONTEXT_TOKENS = 8192` is a process-wide cap** regardless of which tier is loaded. The 27B tier's native context is much larger; we are deliberately leaving headroom on the table to keep the preflight predictable. Surfaced in the slice README's debt section.
|
||||
- **`u32::MAX` GPU offload.** The engine has no concept of partial offload. On a low-VRAM machine that cannot fit all layers, llama.cpp will emit warnings and fall back to mixed CPU/GPU automatically, but our `gpu_offloaded` boolean tells `inference_thread_count` we are fully GPU-resident. When this matters (battery, throttling), the consumer is `magnotia_core::tuning` — it picks a bigger thread count when it thinks the CPU is idle. Trivial-true today; tracked as observability gap (commit `052265b`).
|
||||
- **`u32::MAX` GPU offload.** The engine has no concept of partial offload. On a low-VRAM machine that cannot fit all layers, llama.cpp will emit warnings and fall back to mixed CPU/GPU automatically, but our `gpu_offloaded` boolean tells `inference_thread_count` we are fully GPU-resident. When this matters (battery, throttling), the consumer is `lumotia_core::tuning` — it picks a bigger thread count when it thinks the CPU is idle. Trivial-true today; tracked as observability gap (commit `052265b`).
|
||||
- **GBNF parser quirks.** llama-cpp-2's GBNF is strict about whitespace. Each grammar in [`llm-prompts-and-grammars.md`](llm-prompts-and-grammars.md) carries an explicit `ws` rule and `r#""#` raw strings — refactors that try to "tidy" the grammar literal by stripping the trailing newline have, in the past, broken `LlamaSampler::grammar` with cryptic parse errors.
|
||||
- **Stop sequences are post-detokenisation substring matches.** They run on the running `String`, not on token ids. A multi-byte stop string that splits across a token boundary still matches because the UTF-8 decoder buffers partial bytes. A stop string that contains characters the chat template re-emits as part of normal output (e.g. a literal `<|`) will trigger early termination — only use the EOG sentinels we already use.
|
||||
- **Chat template fallback to ChatML.** If `model.chat_template(None)` errors, we warn-log and use `LlamaChatTemplate::new("chatml")`. The warn-log fires once per `generate` call, not once per session — keep an eye on log volume if a non-Qwen model is ever loaded.
|
||||
|
||||
Reference in New Issue
Block a user