feat(llm): bump model registry to Qwen3.5 + Qwen3.6 family (4 tiers)
Replaces the three older Qwen3 variants with a four-tier ladder spanning a wider hardware range: - Qwen3_5_2B_Q4 (Minimal, 8 GB RAM, ~1.3 GB download) - Qwen3_5_4B_Q4 (Standard, 16 GB RAM / 6 GB VRAM, ~2.7 GB) — DEFAULT - Qwen3_5_9B_Q4 (High, 32 GB RAM / 12 GB VRAM, ~5.7 GB) - Qwen3_6_27B_Q4 (Maximum, 64 GB RAM / 24 GB VRAM, ~17 GB) All four GGUFs sourced from unsloth's HF org with pinned commit SHAs. Sizes and SHA256 hashes verified against the live X-Linked-Etag / X-Linked-Size headers on the LFS CDN. Q4_K_M quantisation throughout (common sweet-spot for cleanup + task extraction). recommend_tier rewritten to span four bands; default_tier moves from the old 4B-Instruct-2507 to Qwen3.5 4B. The 27B Maximum tier honestly needs 64 GB RAM to run without partial offload — surfaced in the description string so the Settings UI can warn realistically. In-tree smoke tests (smoke.rs, content_tags_smoke.rs) updated to reference the new smallest tier so a developer's MAGNOTIA_LLM_TEST_MODEL points at the cheapest GGUF to download. Crate description in crates/llm/Cargo.toml refreshed to mention the new family. NOTE (out of scope; not fixed): the size_bytes / sha256 / hf_url methods could collapse into a single LlmModelMetadata table to remove four parallel match arms. Layer 2 cleanup, separate session. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -24,7 +24,7 @@ fn extract_content_tags_returns_valid_pair() {
|
||||
|
||||
let engine = LlmEngine::new();
|
||||
engine
|
||||
.load_model(LlmModelId::Qwen3_1_7B_Q4, &model_path, true)
|
||||
.load_model(LlmModelId::Qwen3_5_2B_Q4, &model_path, true)
|
||||
.expect("load model");
|
||||
|
||||
let transcript = "Tomorrow I need to run through the grant application one more time \
|
||||
|
||||
@@ -26,7 +26,7 @@ fn llama_cpp_2_smoke_generates_and_wraps() {
|
||||
|
||||
let engine = LlmEngine::new();
|
||||
engine
|
||||
.load_model(LlmModelId::Qwen3_1_7B_Q4, &model_path, true)
|
||||
.load_model(LlmModelId::Qwen3_5_2B_Q4, &model_path, true)
|
||||
.expect("load model");
|
||||
|
||||
let completion = engine
|
||||
|
||||
Reference in New Issue
Block a user