Jake
699cb7e08e
Replaces the three older Qwen3 variants with a four-tier ladder spanning a wider hardware range: - Qwen3_5_2B_Q4 (Minimal, 8 GB RAM, ~1.3 GB download) - Qwen3_5_4B_Q4 (Standard, 16 GB RAM / 6 GB VRAM, ~2.7 GB) — DEFAULT - Qwen3_5_9B_Q4 (High, 32 GB RAM / 12 GB VRAM, ~5.7 GB) - Qwen3_6_27B_Q4 (Maximum, 64 GB RAM / 24 GB VRAM, ~17 GB) All four GGUFs sourced from unsloth's HF org with pinned commit SHAs. Sizes and SHA256 hashes verified against the live X-Linked-Etag / X-Linked-Size headers on the LFS CDN. Q4_K_M quantisation throughout (common sweet-spot for cleanup + task extraction). recommend_tier rewritten to span four bands; default_tier moves from the old 4B-Instruct-2507 to Qwen3.5 4B. The 27B Maximum tier honestly needs 64 GB RAM to run without partial offload — surfaced in the description string so the Settings UI can warn realistically. In-tree smoke tests (smoke.rs, content_tags_smoke.rs) updated to reference the new smallest tier so a developer's MAGNOTIA_LLM_TEST_MODEL points at the cheapest GGUF to download. Crate description in crates/llm/Cargo.toml refreshed to mention the new family. NOTE (out of scope; not fixed): the size_bytes / sha256 / hf_url methods could collapse into a single LlmModelMetadata table to remove four parallel match arms. Layer 2 cleanup, separate session. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
63 lines
1.8 KiB
Rust
63 lines
1.8 KiB
Rust
//! Smoke test: load a GGUF model and exercise the high-level wrappers.
|
|
//!
|
|
//! Verified against llama-cpp-2 `0.1.144` using:
|
|
//! - `llama_backend::LlamaBackend`
|
|
//! - `model::LlamaModel`
|
|
//! - `context::params::LlamaContextParams`
|
|
//! - `sampling::LlamaSampler`
|
|
//!
|
|
//! The test is gated behind `MAGNOTIA_LLM_TEST_MODEL`.
|
|
|
|
use std::env;
|
|
use std::path::PathBuf;
|
|
|
|
use magnotia_llm::LlmEngine;
|
|
use magnotia_llm::LlmModelId;
|
|
|
|
#[test]
|
|
fn llama_cpp_2_smoke_generates_and_wraps() {
|
|
let model_path = match env::var("MAGNOTIA_LLM_TEST_MODEL") {
|
|
Ok(path) => PathBuf::from(path),
|
|
Err(_) => {
|
|
eprintln!("MAGNOTIA_LLM_TEST_MODEL not set — skipping");
|
|
return;
|
|
}
|
|
};
|
|
|
|
let engine = LlmEngine::new();
|
|
engine
|
|
.load_model(LlmModelId::Qwen3_5_2B_Q4, &model_path, true)
|
|
.expect("load model");
|
|
|
|
let completion = engine
|
|
.generate(
|
|
"Write exactly one short greeting.",
|
|
&magnotia_llm::GenerationConfig {
|
|
max_tokens: 32,
|
|
temperature: 0.0,
|
|
stop_sequences: vec!["\n".to_string()],
|
|
grammar: None,
|
|
},
|
|
)
|
|
.expect("generate");
|
|
assert!(!completion.trim().is_empty());
|
|
|
|
let cleaned = engine
|
|
.cleanup_text(
|
|
"You are a transcript cleanup assistant. Remove fillers and output only cleaned text.",
|
|
"um hello there like general kenobi",
|
|
)
|
|
.expect("cleanup_text");
|
|
assert!(!cleaned.trim().is_empty());
|
|
|
|
let tasks = engine
|
|
.extract_tasks("I need to call the plumber tomorrow and buy milk.")
|
|
.expect("extract_tasks");
|
|
assert!(!tasks.is_empty());
|
|
|
|
let steps = engine
|
|
.decompose_task("Plan a weekend trip to the coast")
|
|
.expect("decompose_task");
|
|
assert!((3..=7).contains(&steps.len()));
|
|
}
|