Lumotia/docs/architecture-map/04-llm-formatting-mcp/llm-cargo-features.md

name, type, slice, last_verified

name	type	slice	last_verified
LLM Cargo features	architecture-map-page	04-llm-formatting-mcp	2026/05/09

LLM Cargo features

Where you are: Architecture map → LLM, Formatting, MCP → Cargo features

Plain English summary. Two independent build-time switches gate llama-cpp-2's GPU and threading acceleration: gpu-vulkan and openmp. Both ship enabled by default. A mobile or CPU-only build can drop one or both with --no-default-features. The features are independent so an Android Vulkan build can opt into Vulkan without OpenMP, where the NDK toolchain configuration is fragile.

At a glance

• Crate: lumotia-llm
• Path: crates/llm/Cargo.toml:7-16
• LOC: relevant section is 10 lines
• Public surface: build-time only — no runtime API change.
• External deps that matter: llama-cpp-2 = { version = "0.1.144", default-features = false }. Lumotia's features are forwarders.
• Tauri command that calls this (slice 2, best guess): n/a — features are picked at build time. The Tauri build (src-tauri/Cargo.toml) inherits whatever set was active when the workspace built.

What's in here

Feature definitions (crates/llm/Cargo.toml:7)

[features]
# Default desktop build keeps the existing openmp + vulkan acceleration.
# Mobile / CPU-only targets can drop one or both via:
#   cargo build -p lumotia-llm --no-default-features
# These are independent so an Android Vulkan build can opt into vulkan
# without openmp (the NDK ships OpenMP libs but the toolchain configuration
# is fragile across NDK versions).
default = ["gpu-vulkan", "openmp"]
gpu-vulkan = ["llama-cpp-2/vulkan"]
openmp = ["llama-cpp-2/openmp"]

Default profile

The default profile is the desktop developer build:

• gpu-vulkan → Vulkan backend in llama.cpp. Works across Linux, Windows, and macOS (via MoltenVK) with a single binary; no per-vendor SDK at build time. The runtime requires Vulkan 1.1+ and a compatible driver.
• openmp → OpenMP-parallel CPU paths in llama.cpp. Effective on big-core CPU inference; on a fully-GPU-offloaded run the gain is small but non-zero (KV cache management runs on CPU).

Mobile / CPU-only toolchain combinations

• cargo build -p lumotia-llm --no-default-features — neither feature. CPU-only single-threaded llama.cpp. Useful for environments where neither Vulkan nor OpenMP is workable. Not used by any current target.
• cargo build -p lumotia-llm --no-default-features --features gpu-vulkan — Vulkan without OpenMP. The intended Android target per the comment block. The NDK ships OpenMP runtime libs, but cargo builds can struggle to find them across NDK versions, so this combination keeps the GPU acceleration without the toolchain risk.
• cargo build -p lumotia-llm --no-default-features --features openmp — OpenMP without Vulkan. CPU-only on a multi-core machine where Vulkan is either unavailable (server with no GPU) or undesirable (Steam Deck running in a sandbox).

How features cross to llama-cpp-2

Each Lumotia feature directly toggles a llama-cpp-2 feature:

• gpu-vulkan → llama-cpp-2/vulkan. The crate's Vulkan feature pulls in vulkan-headers and configures the C++ LLAMA_VULKAN build flag.
• openmp → llama-cpp-2/openmp. The crate's OpenMP feature configures the C++ LLAMA_OPENMP build flag and links against the system OpenMP runtime.

llama-cpp-2 = { version = "0.1.144", default-features = false } is the import line: we explicitly disable the upstream defaults and re-enable only what we declare here. That keeps the surface area small and the build deterministic.

Watch-outs

• use_gpu is orthogonal to gpu-vulkan. A binary built without gpu-vulkan still accepts use_gpu: true at runtime (in LlmEngine::load_model); llama.cpp will warn that no GPU backend was compiled in and fall back to CPU. The Tauri layer should hide the GPU toggle when the feature is off, but the engine does not enforce.
• Feature drift across the workspace. The Tauri binary at src-tauri/Cargo.toml depends on lumotia-llm and inherits its features unless overridden. If a CI matrix builds --no-default-features for lumotia-llm alone, the Tauri build will still pull defaults. Verify via cargo tree -e features -p lumotia-llm when changing.
• Build-time only. None of these are runtime-toggleable. A user cannot disable Vulkan after the fact; they need a different binary. We do not currently ship two binaries — only the desktop default.
• MAX_CONTEXT_TOKENS and threading code paths are independent of features. The same LlmEngine::generate runs whether OpenMP is in or not; inference_thread_count(Workload::Llm, gpu_offloaded) decides thread count from physical cores, not from compile-time information.
• Vulkan headroom. Vulkan on macOS requires MoltenVK at runtime. The build does not ship MoltenVK. A macOS lumotia-llm build with gpu-vulkan works on a Mac that has the Vulkan SDK or MoltenVK installed; without it, llama.cpp will fail to initialise the backend and fall back to CPU.

See also

• LLM engine — runtime side of the GPU and threading story
• LLM model manager — tier sizing accounts for VRAM at recommendation time
• Slice README