# Building Lumotia: a complete technology map for local-first, voice-first desktop AI

**Lumotia's entire stack -- from audio capture through LLM inference to neurodivergent-friendly UI -- can be built from actively maintained, production-tested open-source components.** The Rust + Tauri v2 + Svelte 5 ecosystem has matured dramatically through 2024-2026, with reference applications like Handy (13.8k stars, Tauri + Whisper + real-time audio) and Whispering (Svelte 5 + Tauri transcription) proving the core architecture viable. The most critical finding: **no existing app combines all of Lumotia's pieces**, making this a genuinely novel integration -- but every individual subsystem has battle-tested implementations to learn from.

**Ingested from:** `input/inbox/backlinksforfree` on 2026/03/20
**Used in:** `docs/superpowers/specs/2026-03-20-lumotia-mvp-design.md`

---

## Area 1: Core MVP features

### 1. Audio capture pipeline

The real-time audio path from microphone to Whisper requires three crates: **cpal** (v0.15.x, Apache 2.0) for cross-platform audio capture, **rubato** (v0.16.2, MIT) for SIMD-accelerated resampling to 16kHz, and a VAD layer. Recommended architecture: three dedicated threads connected by ring buffers.

The **voice-stream** crate (v0.4.0) wraps the entire pipeline (cpal + rubato + Silero VAD) into a single library. Fastest path to working audio, though forking allows finer control.

For VAD: whisper-rs v0.16's **built-in VAD** (simplest), **silero-vad-rust** (MIT, streaming-ready), voice_activity_detector (used by Handy), **webrtc-vad** (lightweight but lower accuracy).

**Reference apps:** Handy (13.8k stars, exact pipeline), Whispering (4.2k stars, Svelte 5 + Tauri), Vibe (v3.0.19, model management patterns).

### 2. Whisper integration

**whisper-rs** (v0.16.0, 183k+ downloads) is the primary recommendation. **transcribe-rs** (v0.3.0) abstracts over multiple STT engines (whisper.cpp, Parakeet, Moonshine, SenseVoice). **whisper-cpp-plus** adds WhisperStream for real-time streaming with integrated Silero VAD.

Two transcription patterns: **chunked-VAD** (simpler, 1-5s latency, used by Handy) vs **overlapping-window streaming** (3.3s latency, more complex). Chunked-VAD is sufficient for voice-first task capture.

### 3. Local LLM integration

**llama-cpp-2** (MIT/Apache-2.0) provides safe Rust bindings. Does not follow semver -- pin exact versions.

Three architectures: **Direct embedding via Tauri Channels** (recommended -- faster, ordered delivery), **sidecar** (fault isolation but process management complexity), **tauri-plugin-llm** (PolyForm licence -- evaluate carefully).

Higher-level alternatives: **kalosm** (type-safe structured generation via `#[derive(Parse)]`), **mistral.rs** (pure Rust, PagedAttention).

Model lifecycle: load at first inference, keep during session, unload on background/close (simpler than Ollama's 5-minute idle timeout).

### 4. sqlite-vec + fastembed RAG pipeline

**sqlite-vec** (~7.2k stars, MIT/Apache-2.0) adds vector search via vec0 virtual table. Sub-10ms latency for tens of thousands of vectors. Uses rusqlite with bundled feature.

**fastembed-rs** (v5.x, Apache-2.0, Qdrant team) generates embeddings via ONNX Runtime. Recommended: **BGESmallENV15Q** (quantised, ~17MB, 384 dims) or **AllMiniLML6V2** (~23MB).

Hybrid search: FTS5 + sqlite-vec with **Reciprocal Rank Fusion** (documented by Alex Garcia). <3ms total retrieval on Raspberry Pi Zero 2 W.

**No published project combines sqlite-vec + fastembed-rs** -- Lumotia's implementation is novel.

### 5. Time-block visualisation

**Schedule-X** (@schedule-x/svelte, v3.0.0, MIT) for day/week calendar views. **Frappe Gantt** (MIT, SVG-based) for timeline. Custom CSS Grid for maximum control.

Design references: Tiimo (circular countdown, sensory-friendly), Structured (vertical timeline, energy monitor), Llama Life (single-task focus with countdown), Sunsama (guided daily planning).

### 6. Task decomposition

GBNF grammar constraints ensure valid JSON output (~25% accuracy improvement). kalosm's `#[derive(Parse)]` eliminates JSON parsing entirely.

**Goblin Tools** provides the best UX reference -- "spiciness slider" for decomposition depth. Each step: single concrete physical action, verb-first, 2-15 minutes, energy-level tagged, 20% overestimation buffer, first step highlighted prominently.

---

## Area 2: Optimisation patterns

### 7. Fractional indexing

**fractional_index** crate (v2.x, MIT) for Rust. **fractional-indexing** (CC0, ~535k weekly npm) for JS. Reordering updates exactly one row.

Pairs with **svelte-dnd-action** (MIT, accessible, keyboard/screen reader) or **@dnd-kit/svelte** (official port, Svelte 5.29+).

### 8. Session state restoration

**tauri-plugin-store** for persistent key-value. **tauri-plugin-window-state** for window position/size. Timer persistence: `{ startedAt, accumulatedMs, lastResumedAt, state }` with absolute timestamps.

### 9. Model downloading

reqwest with bytes_stream, HTTP Range headers for resume, incremental SHA256 via ring/sha2. Progress via Tauri Channels (not events). **trauma** crate for resume support.

### 10. Tauri v2 local-first patterns

**tauri-plugin-sql** for standard SQLite. **rusqlite** with bundled for sqlite-vec. State management: commands for CRUD, events for push notifications, channels for streaming.

**cr-sqlite** (Apache-2.0) for future CRDT-based sync (~2.5x write overhead).

Reference apps: Screenpipe, GitButler, Musicat, Duckling.

### 11. WIP limits

Soft limits with progressive visual warning (green to yellow to red). Start with WIP limit of 3, let users adjust per energy/context. "Stop starting, start finishing."

### 12. Neurodivergent-first design

**No open-source component library exists for neurodivergent users** -- ecosystem gap and differentiation opportunity.

Foundation: **shadcn-svelte** + Bits UI for ARIA/keyboard accessibility. Layer neurodivergent styling on top. **OKLCH colour system** with locked Lightness. Reduced motion as default (opt-in, not opt-out). Progressive disclosure below 3 levels. Literal labels always.

Essential references: W3C COGA, Microsoft Inclusive Design for Cognition Guidebook.