Lumotia/docs/brief/technology-map.md

Building Magnotia: a complete technology map for local-first, voice-first desktop AI

Magnotia's entire stack -- from audio capture through LLM inference to neurodivergent-friendly UI -- can be built from actively maintained, production-tested open-source components. The Rust + Tauri v2 + Svelte 5 ecosystem has matured dramatically through 2024-2026, with reference applications like Handy (13.8k stars, Tauri + Whisper + real-time audio) and Whispering (Svelte 5 + Tauri transcription) proving the core architecture viable. The most critical finding: no existing app combines all of Magnotia's pieces, making this a genuinely novel integration -- but every individual subsystem has battle-tested implementations to learn from.

Ingested from: input/inbox/backlinksforfree on 2026/03/20 Used in: docs/superpowers/specs/2026-03-20-magnotia-mvp-design.md

---

Area 1: Core MVP features

1. Audio capture pipeline

The real-time audio path from microphone to Whisper requires three crates: cpal (v0.15.x, Apache 2.0) for cross-platform audio capture, rubato (v0.16.2, MIT) for SIMD-accelerated resampling to 16kHz, and a VAD layer. Recommended architecture: three dedicated threads connected by ring buffers.

The voice-stream crate (v0.4.0) wraps the entire pipeline (cpal + rubato + Silero VAD) into a single library. Fastest path to working audio, though forking allows finer control.

For VAD: whisper-rs v0.16's built-in VAD (simplest), silero-vad-rust (MIT, streaming-ready), voice_activity_detector (used by Handy), webrtc-vad (lightweight but lower accuracy).

Reference apps: Handy (13.8k stars, exact pipeline), Whispering (4.2k stars, Svelte 5 + Tauri), Vibe (v3.0.19, model management patterns).

2. Whisper integration

whisper-rs (v0.16.0, 183k+ downloads) is the primary recommendation. transcribe-rs (v0.3.0) abstracts over multiple STT engines (whisper.cpp, Parakeet, Moonshine, SenseVoice). whisper-cpp-plus adds WhisperStream for real-time streaming with integrated Silero VAD.

Two transcription patterns: chunked-VAD (simpler, 1-5s latency, used by Handy) vs overlapping-window streaming (3.3s latency, more complex). Chunked-VAD is sufficient for voice-first task capture.

3. Local LLM integration

llama-cpp-2 (MIT/Apache-2.0) provides safe Rust bindings. Does not follow semver -- pin exact versions.

Three architectures: Direct embedding via Tauri Channels (recommended -- faster, ordered delivery), sidecar (fault isolation but process management complexity), tauri-plugin-llm (PolyForm licence -- evaluate carefully).

Higher-level alternatives: kalosm (type-safe structured generation via #[derive(Parse)]), mistral.rs (pure Rust, PagedAttention).

Model lifecycle: load at first inference, keep during session, unload on background/close (simpler than Ollama's 5-minute idle timeout).

4. sqlite-vec + fastembed RAG pipeline

sqlite-vec (~7.2k stars, MIT/Apache-2.0) adds vector search via vec0 virtual table. Sub-10ms latency for tens of thousands of vectors. Uses rusqlite with bundled feature.

fastembed-rs (v5.x, Apache-2.0, Qdrant team) generates embeddings via ONNX Runtime. Recommended: BGESmallENV15Q (quantised, ~17MB, 384 dims) or AllMiniLML6V2 (~23MB).

Hybrid search: FTS5 + sqlite-vec with Reciprocal Rank Fusion (documented by Alex Garcia). <3ms total retrieval on Raspberry Pi Zero 2 W.

No published project combines sqlite-vec + fastembed-rs -- Magnotia's implementation is novel.

5. Time-block visualisation

Schedule-X (@schedule-x/svelte, v3.0.0, MIT) for day/week calendar views. Frappe Gantt (MIT, SVG-based) for timeline. Custom CSS Grid for maximum control.

Design references: Tiimo (circular countdown, sensory-friendly), Structured (vertical timeline, energy monitor), Llama Life (single-task focus with countdown), Sunsama (guided daily planning).

6. Task decomposition

GBNF grammar constraints ensure valid JSON output (~25% accuracy improvement). kalosm's #[derive(Parse)] eliminates JSON parsing entirely.

Goblin Tools provides the best UX reference -- "spiciness slider" for decomposition depth. Each step: single concrete physical action, verb-first, 2-15 minutes, energy-level tagged, 20% overestimation buffer, first step highlighted prominently.

---

Area 2: Optimisation patterns

7. Fractional indexing

fractional_index crate (v2.x, MIT) for Rust. fractional-indexing (CC0, ~535k weekly npm) for JS. Reordering updates exactly one row.

Pairs with svelte-dnd-action (MIT, accessible, keyboard/screen reader) or @dnd-kit/svelte (official port, Svelte 5.29+).

8. Session state restoration

tauri-plugin-store for persistent key-value. tauri-plugin-window-state for window position/size. Timer persistence: { startedAt, accumulatedMs, lastResumedAt, state } with absolute timestamps.

9. Model downloading

reqwest with bytes_stream, HTTP Range headers for resume, incremental SHA256 via ring/sha2. Progress via Tauri Channels (not events). trauma crate for resume support.

10. Tauri v2 local-first patterns

tauri-plugin-sql for standard SQLite. rusqlite with bundled for sqlite-vec. State management: commands for CRUD, events for push notifications, channels for streaming.

cr-sqlite (Apache-2.0) for future CRDT-based sync (~2.5x write overhead).

Reference apps: Screenpipe, GitButler, Musicat, Duckling.

11. WIP limits

Soft limits with progressive visual warning (green to yellow to red). Start with WIP limit of 3, let users adjust per energy/context. "Stop starting, start finishing."

12. Neurodivergent-first design

No open-source component library exists for neurodivergent users -- ecosystem gap and differentiation opportunity.

Foundation: shadcn-svelte + Bits UI for ARIA/keyboard accessibility. Layer neurodivergent styling on top. OKLCH colour system with locked Lightness. Reduced motion as default (opt-in, not opt-out). Progressive disclosure below 3 levels. Literal labels always.

Essential references: W3C COGA, Microsoft Inclusive Design for Cognition Guidebook.