a1f3f3f134
Five-slice navigable map of the entire codebase under
docs/architecture-map/. Each slice is a self-contained
breadcrumbed sub-tree:
01-frontend (16) Svelte/SvelteKit UI
02-tauri-runtime (26) src-tauri commands + lifecycle
03-audio-transcription (16) audio + transcription crates
04-llm-formatting-mcp (19) llm, ai-formatting, mcp, cloud
05-core-storage-hotkey-build core, storage, hotkey, workspace,
(26) CI, dev glue
Plus master README.md and data-flow-end-to-end.md tracing
audio bytes from microphone to FTS5 search to MCP read.
Generated by 5 parallel subagents on 2026/05/09 against
HEAD 3c47000. Each page has YAML frontmatter, file:line code
refs, sibling cross-links, plain-English summaries.
Aggregated debt surfaced (full lists in master README):
RB-08 macOS power assertion, schema head drift v14 vs v15,
VAD blocked on ort version conflict, streaming primitives
not wired into live.rs, no prompt versioning, MCP has no
auth, cloud-providers in-memory keystore, SettingsPage
2 484 LOC, commands/live.rs 1 737 LOC, dual theme system,
brand rename to Lumenote pending across the codebase.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
75 lines
5.2 KiB
Markdown
75 lines
5.2 KiB
Markdown
---
|
|
name: LLM Cargo features
|
|
type: architecture-map-page
|
|
slice: 04-llm-formatting-mcp
|
|
last_verified: 2026/05/09
|
|
---
|
|
|
|
# LLM Cargo features
|
|
|
|
> **Where you are:** [Architecture map](../README.md) → [LLM, Formatting, MCP](README.md) → Cargo features
|
|
|
|
**Plain English summary.** Two independent build-time switches gate llama-cpp-2's GPU and threading acceleration: `gpu-vulkan` and `openmp`. Both ship enabled by default. A mobile or CPU-only build can drop one or both with `--no-default-features`. The features are independent so an Android Vulkan build can opt into Vulkan without OpenMP, where the NDK toolchain configuration is fragile.
|
|
|
|
## At a glance
|
|
|
|
- Crate: `magnotia-llm`
|
|
- Path: `crates/llm/Cargo.toml:7-16`
|
|
- LOC: relevant section is 10 lines
|
|
- Public surface: build-time only — no runtime API change.
|
|
- External deps that matter: `llama-cpp-2 = { version = "0.1.144", default-features = false }`. Magnotia's features are forwarders.
|
|
- Tauri command that calls this (slice 2, best guess): n/a — features are picked at build time. The Tauri build (`src-tauri/Cargo.toml`) inherits whatever set was active when the workspace built.
|
|
|
|
## What's in here
|
|
|
|
### Feature definitions (`crates/llm/Cargo.toml:7`)
|
|
|
|
```toml
|
|
[features]
|
|
# Default desktop build keeps the existing openmp + vulkan acceleration.
|
|
# Mobile / CPU-only targets can drop one or both via:
|
|
# cargo build -p magnotia-llm --no-default-features
|
|
# These are independent so an Android Vulkan build can opt into vulkan
|
|
# without openmp (the NDK ships OpenMP libs but the toolchain configuration
|
|
# is fragile across NDK versions).
|
|
default = ["gpu-vulkan", "openmp"]
|
|
gpu-vulkan = ["llama-cpp-2/vulkan"]
|
|
openmp = ["llama-cpp-2/openmp"]
|
|
```
|
|
|
|
### Default profile
|
|
|
|
The default profile is the desktop developer build:
|
|
|
|
- `gpu-vulkan` → Vulkan backend in llama.cpp. Works across Linux, Windows, and macOS (via MoltenVK) with a single binary; no per-vendor SDK at build time. The runtime requires Vulkan 1.1+ and a compatible driver.
|
|
- `openmp` → OpenMP-parallel CPU paths in llama.cpp. Effective on big-core CPU inference; on a fully-GPU-offloaded run the gain is small but non-zero (KV cache management runs on CPU).
|
|
|
|
### Mobile / CPU-only toolchain combinations
|
|
|
|
- `cargo build -p magnotia-llm --no-default-features` — neither feature. CPU-only single-threaded llama.cpp. Useful for environments where neither Vulkan nor OpenMP is workable. Not used by any current target.
|
|
- `cargo build -p magnotia-llm --no-default-features --features gpu-vulkan` — Vulkan without OpenMP. The intended Android target per the comment block. The NDK ships OpenMP runtime libs, but `cargo` builds can struggle to find them across NDK versions, so this combination keeps the GPU acceleration without the toolchain risk.
|
|
- `cargo build -p magnotia-llm --no-default-features --features openmp` — OpenMP without Vulkan. CPU-only on a multi-core machine where Vulkan is either unavailable (server with no GPU) or undesirable (Steam Deck running in a sandbox).
|
|
|
|
### How features cross to llama-cpp-2
|
|
|
|
Each Magnotia feature directly toggles a `llama-cpp-2` feature:
|
|
|
|
- `gpu-vulkan` → `llama-cpp-2/vulkan`. The crate's Vulkan feature pulls in `vulkan-headers` and configures the C++ `LLAMA_VULKAN` build flag.
|
|
- `openmp` → `llama-cpp-2/openmp`. The crate's OpenMP feature configures the C++ `LLAMA_OPENMP` build flag and links against the system OpenMP runtime.
|
|
|
|
`llama-cpp-2 = { version = "0.1.144", default-features = false }` is the import line: we explicitly disable the upstream defaults and re-enable only what we declare here. That keeps the surface area small and the build deterministic.
|
|
|
|
## Watch-outs
|
|
|
|
- **`use_gpu` is orthogonal to `gpu-vulkan`.** A binary built without `gpu-vulkan` still accepts `use_gpu: true` at runtime (in `LlmEngine::load_model`); llama.cpp will warn that no GPU backend was compiled in and fall back to CPU. The Tauri layer should hide the GPU toggle when the feature is off, but the engine does not enforce.
|
|
- **Feature drift across the workspace.** The Tauri binary at `src-tauri/Cargo.toml` depends on `magnotia-llm` and inherits its features unless overridden. If a CI matrix builds `--no-default-features` for `magnotia-llm` alone, the Tauri build will still pull defaults. Verify via `cargo tree -e features -p magnotia-llm` when changing.
|
|
- **Build-time only.** None of these are runtime-toggleable. A user cannot disable Vulkan after the fact; they need a different binary. We do not currently ship two binaries — only the desktop default.
|
|
- **`MAX_CONTEXT_TOKENS` and threading code paths are independent of features.** The same `LlmEngine::generate` runs whether OpenMP is in or not; `inference_thread_count(Workload::Llm, gpu_offloaded)` decides thread count from physical cores, not from compile-time information.
|
|
- **Vulkan headroom.** Vulkan on macOS requires MoltenVK at runtime. The build does not ship MoltenVK. A macOS `magnotia-llm` build with `gpu-vulkan` works on a Mac that has the Vulkan SDK or MoltenVK installed; without it, llama.cpp will fail to initialise the backend and fall back to CPU.
|
|
|
|
## See also
|
|
|
|
- [LLM engine](llm-engine.md) — runtime side of the GPU and threading story
|
|
- [LLM model manager](llm-model-manager.md) — tier sizing accounts for VRAM at recommendation time
|
|
- [Slice README](README.md)
|