chore(llm): document trivial-true cross-check as observability gap

Per the final reviewer's suggestion: the gpu_offloaded cross-check
(gpu_layers >= model.n_layer()) is trivially true when use_gpu since
we always pass u32::MAX. The check documents intent and is future-
proofed if we ever pass specific N, but a future reader might
simplify it away as dead code. Inline comment points at the spec's
out-of-scope follow-up for true residency observability.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jars
2026-05-09 12:47:28 +01:00
parent 4f3c063008
commit 052265b3c2

View File

@@ -166,6 +166,11 @@ impl LlmEngine {
let n_ctx = preflight_context_window(prompt_tokens.len(), config.max_tokens)?;
let use_gpu = self.loaded_model().map(|s| s.use_gpu).unwrap_or(false);
let gpu_layers = if use_gpu { u32::MAX } else { 0u32 };
// Trivially true today (gpu_layers = u32::MAX when use_gpu), but the
// explicit comparison documents intent. True residency observability
// (parsing llama.cpp's "offloaded N/M layers" log) is tracked as a
// follow-up in docs/superpowers/specs/2026-05-09-battery-gpu-aware-
// thread-tuning-design.md (§ Out of scope).
let gpu_offloaded = use_gpu && gpu_layers >= model.n_layer();
let thread_count = i32::try_from(inference_thread_count(Workload::Llm, gpu_offloaded))
.unwrap_or(4);