chore(llm): document trivial-true cross-check as observability gap
Per the final reviewer's suggestion: the gpu_offloaded cross-check (gpu_layers >= model.n_layer()) is trivially true when use_gpu since we always pass u32::MAX. The check documents intent and is future- proofed if we ever pass specific N, but a future reader might simplify it away as dead code. Inline comment points at the spec's out-of-scope follow-up for true residency observability. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -166,6 +166,11 @@ impl LlmEngine {
|
||||
let n_ctx = preflight_context_window(prompt_tokens.len(), config.max_tokens)?;
|
||||
let use_gpu = self.loaded_model().map(|s| s.use_gpu).unwrap_or(false);
|
||||
let gpu_layers = if use_gpu { u32::MAX } else { 0u32 };
|
||||
// Trivially true today (gpu_layers = u32::MAX when use_gpu), but the
|
||||
// explicit comparison documents intent. True residency observability
|
||||
// (parsing llama.cpp's "offloaded N/M layers" log) is tracked as a
|
||||
// follow-up in docs/superpowers/specs/2026-05-09-battery-gpu-aware-
|
||||
// thread-tuning-design.md (§ Out of scope).
|
||||
let gpu_offloaded = use_gpu && gpu_layers >= model.n_layer();
|
||||
let thread_count = i32::try_from(inference_thread_count(Workload::Llm, gpu_offloaded))
|
||||
.unwrap_or(4);
|
||||
|
||||
Reference in New Issue
Block a user