Lumotia/docs/architecture-map/03-audio-transcription/transcription-model-manager.md

name, type, slice, last_verified

name	type	slice	last_verified
Model manager (download, verify, locate)	architecture-map-page	03-audio-transcription	2026/05/09

Model manager (download, verify, locate)

Where you are: Architecture map → Audio + Transcription → Model manager

Plain English summary. Speech models live on disk under a per-platform path. model_manager resolves those paths, checks "is this model fully downloaded", lists what is downloaded, and downloads missing files with HTTP Range resume + per-chunk SHA256 verification. A single in-process HashSet reservation prevents concurrent downloads of the same model. After a successful download a .lumotia-verified manifest is written so a subsequent process restart can short-circuit re-hashing.

At a glance

• Crate: lumotia-transcription
• Path: crates/transcription/src/model_manager.rs
• LOC: 617 (production code + extensive in-tree download server tests)
• External deps: reqwest 0.12 (rustls-tls, stream), futures-util 0.3 (StreamExt), sha2 0.10. Path resolution comes from lumotia_core::paths::app_paths. Catalogue from lumotia_core::model_registry.
• Internal callers (best effort, slice 2 reconciles): the models Tauri command consumes all five public functions. The frontend Settings → Models view drives download with a progress callback.

Public surface:

• pub fn models_dir() -> PathBuf — crates/transcription/src/model_manager.rs:42. Resolves to %LOCALAPPDATA%/lumotia/models on Windows, ~/.lumotia/models on Unix (delegates to lumotia_core::paths::app_paths).
• pub fn model_dir(id: &ModelId) -> PathBuf — crates/transcription/src/model_manager.rs:47.
• pub fn is_downloaded(id: &ModelId) -> bool — crates/transcription/src/model_manager.rs:52.
• pub fn list_downloaded() -> Vec<ModelId> — crates/transcription/src/model_manager.rs:63.
• pub async fn download(id: &ModelId, progress: impl Fn(DownloadProgress) + Send + 'static) -> Result<()> — crates/transcription/src/model_manager.rs:78.

Private helpers:

• DownloadReservation — RAII guard (model_manager.rs:12).
• verified_manifest_path / verified_manifest_matches / write_verified_manifest (model_manager.rs:115–:153).
• sha256_of_file (model_manager.rs:157).
• download_file (model_manager.rs:178).

What's in here

Path resolution

models_dir and model_dir defer entirely to lumotia_core::paths::app_paths(). Anything platform-specific lives in slice 5; this crate just consumes the resolution.

Download reservation

DownloadReservation::acquire(id) (model_manager.rs:17) inserts the model id into a process-wide HashSet guarded by a LazyLock<Mutex<...>>. Returns Err(MagnotiaError::DownloadFailed("download already in progress for {id}")) if the id is already present. Drop removes the entry. This protects against re-entry from the same process; cross-process races are still possible but the atomic .part → dest rename closes the most damaging window.

is_downloaded

model_manager.rs:52. Two-step:

1. Every file declared in the registry entry must exist at model_dir(id).join(filename).
2. The verified-manifest at model_dir.join(".lumotia-verified") must record the same filename\tsha256\tsize triple per file.

A missing or stale manifest causes is_downloaded to return false even if the bytes are correct. The next download call will revalidate and rewrite the manifest. A truncated or tampered file fails the existence + size check, so a subsequent download redownloads it.

list_downloaded

model_manager.rs:63. Filters the static lumotia_core::model_registry::all_models() catalogue by is_downloaded.

download (the orchestrator)

model_manager.rs:78. For each ModelFile in the registry entry:

1. If dest exists, hash it with sha256_of_file. Match → skip the download. Mismatch → delete and re-fetch. Hash IO error → propagate as MagnotiaError::DownloadFailed.
2. Otherwise call download_file(file, dest, id, &progress).

After all files are present and verified, write_verified_manifest writes the version\t1 header and per-file filename\tsha256\tsize lines.

download_file (single file with resume + hash)

model_manager.rs:178. Implements the Buzz-style resume pattern:

1. Build a reqwest::Client with a 30 s connect timeout.
2. If dest.with_extension("part") exists, send a Range: bytes={existing}- header.
3. Inspect the response status:
   • 206 Partial Content → server is resuming. Continue appending.
   • 200 OK after a Range request → server ignored Range. Discard the stale .part and start fresh (regression-tested at model_manager.rs:494).
   • 4xx/5xx (non-resume path) → MagnotiaError::DownloadFailed("download returned HTTP {status} for {filename}"). Crucial because reqwest does not auto-error on bad status; without this check, a 404 body would be streamed into .part and renamed over the destination (regression at model_manager.rs:556).
   • Other status on resume → propagate as unexpected.
4. Determine total_bytes from Content-Range (resume) or Content-Length (fresh).
5. Hasher state: when resuming, hash the on-disk .part first to catch the hash up to where the network stream is about to resume. Otherwise start fresh.
6. Stream the body in chunks, writing each to the file, updating the running SHA256, emitting DownloadProgress per percent step.
7. On stream end, finalise the hash. Mismatch → remove .part and MagnotiaError::DownloadFailed("SHA256 mismatch for {filename}: expected ..., got ...").
8. std::fs::rename(.part, dest) — atomic finalise.

sha256_of_file

model_manager.rs:157. 8 KiB buffer streaming hash. Used by download to validate an existing complete file before trusting it.

Data flow

ModelId
  └─ models_dir / model_dir → on-disk directory
  └─ is_downloaded:
       all files exist AND verified-manifest matches
         → true / false
  └─ list_downloaded:
       filter all_models() by is_downloaded
  └─ download:
       acquire reservation
       for file in entry.files:
         if dest exists:
           sha256_of_file → match? skip : delete + re-fetch
         else:
           download_file:
             [.part exists? Range request : full GET]
             stream body → file + hasher + progress callback
             match SHA256 → atomic rename
             else → cleanup, error
       write_verified_manifest
       drop reservation

Watch-outs

• Reservation is per-process. Two Lumotia instances (dev + release, two cargo runs) racing on the same model dir can collide. The .part → dest atomic rename is the only thing standing between them and a torn file.
• DownloadProgress is emitted per percent step, not per byte. A 10 MB file at 4G download speeds emits ~100 events; a 5 GB file emits the same 100 events. UI must not assume tight cadence.
• Manifest schema is version\t1 + filename\tsha256\tsize lines. Bumping the version is a manual operation; a stale manifest after a registry update will simply trigger a redownload (waste, not corruption).
• Cross-process file locking is not used. A user opening Settings → Models in two windows of the same desktop session will trip the in-process reservation, but two physical machines syncing the same network home would not be detected.
• Reqwest connect timeout is 30 s, no read timeout. A stalled mid-stream connection blocks indefinitely. Acceptable for a foreground UI flow today; would need a tokio::time::timeout wrap if the download ever became headless.
• ModelFile::sha256 is &'static str. The leak helper in tests reflects that production code reads a static catalogue compiled into the binary. Tampering with the catalogue requires a code change, not a runtime config.
• reqwest is configured default-features = false, features = ["rustls-tls", "stream"]. No native-tls fallback. Linux without ca-certificates will fail at connect.
• models_dir is the user-data dir, not a cache dir. A "clear cache" button in the OS would not nuke models. Documented behaviour.

See also

• Transcription engines overview — load_whisper / load_parakeet are called against model_dir(id) outputs.
• Tests and fixtures — in-tree TcpListener-backed servers that exercise the resume + sha-mismatch + 5xx paths.
• lumotia_core::model_registry::{find_model, all_models, ModelEntry, ModelFile} (slice 5) — the source of URLs and hashes.
• lumotia_core::paths::app_paths (slice 5) — platform path resolution.
• lumotia_core::types::DownloadProgress (slice 5) — the progress event shape.