jars
a58c2f0994
test(transcription): thread_sweep prints power-aware 4-panel table
...
Extends the JFK thread-count sweep to print four labelled panels
(AC+CPU, AC+GPU, battery+CPU, battery+GPU) driven by
MAGNOTIA_POWER_STATE_OVERRIDE. Each panel shows the helper's
predicted thread count for its (power, gpu) combination alongside
the empirical RTF table for n_threads = [1, 2, 4, physical, logical,
maybe 8].
The actual whisper context is initialised once (Vulkan if compiled
in and resolvable, CPU otherwise), so the RTF rows themselves are
produced by the same backend across panels. The CPU vs GPU axis
controls only the helper-pick column, which is the empirical
question we want answered: is the LLM=2 / Whisper=4 GPU floor right?
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
2026-05-09 12:43:04 +01:00
jars
4cb954ece4
perf: route whisper + llm n_threads through physical-core helper
...
check / cargo check (macos-latest) (push) Has been cancelled
check / cargo check (ubuntu-22.04) (push) Has been cancelled
check / cargo check (windows-latest) (push) Has been cancelled
check / svelte build + lint (push) Has been cancelled
Both inference call sites previously called `num_cpus::get()` (logical
thread count). Established whisper.cpp / llama.cpp guidance is that
SMT siblings contend for shared FPU resources during heavy F16/F32
matmul, so going past physical core count anti-scales. Empirical
sweep on Whisper Tiny / 11s JFK clip / Ryzen 5 4650U (6c12t):
n_threads | xc_time | RTF | speedup_vs_1
----------|---------|--------|-------------
1 | 0.33s | 0.030 | 1.00x
2 | 0.33s | 0.030 | 1.00x
4 | 0.30s | 0.028 | 1.09x
6 | 0.32s | 0.029 | 1.04x
8 | 0.31s | 0.028 | 1.06x
12 | 0.32s | 0.029 | 1.03x
Tiny doesn't scale (work dominated by overhead) but the larger
Whisper variants and Qwen LLMs do. Sources: whisper.cpp #200 , #1033 ,
#1252 , #403 ; llama.cpp #3167 , #572 .
Changes:
- crates/core/src/constants.rs:
* MIN_INFERENCE_THREADS lowered 4 → 2 (research-derived floor)
* MAX_INFERENCE_THREADS = 8 added (research-derived ceiling)
* inference_thread_count() rewritten:
- reads MAGNOTIA_INFERENCE_THREADS env var (override)
- num_cpus::get_physical() with available_parallelism fallback
- clamped to [MIN_INFERENCE_THREADS, MAX_INFERENCE_THREADS]
- crates/core/Cargo.toml: + num_cpus = "1"
- crates/transcription/src/whisper_rs_backend.rs: call site uses helper.
- crates/llm/src/lib.rs: call site uses helper.
- crates/transcription/Cargo.toml: drop num_cpus from [features] +
[dependencies] (production no longer needs it). Move to
[dev-dependencies] for tests/thread_sweep.rs only.
- crates/llm/Cargo.toml: drop num_cpus = "1" (no longer used directly).
Per-machine maps (after this patch):
Ryzen 5 4650U 6c12t → 6
big-iron 12c24t → 8 (clamp; users can override)
cheap 2c2t laptop → 2
1c container/VM → 2
Adds crates/transcription/tests/thread_sweep.rs — env-gated like
jfk_bench, prints the table above against any model + WAV. Useful for
re-baselining on new hardware or when tuning the clamp values.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
2026-05-09 09:26:23 +01:00
