Jake
26c7307607
Phase 9 of the rebrand cascade. Sweep covers everything the Phase 8
frontend pass deliberately skipped: docs/, root markdown, scripts,
Cargo.toml descriptions, code comments that survived earlier
word-boundary sed, plus a handful of identifiers caught on the final
verify pass.
transcription-app changes:
- README.md, HANDOVER.md, KNOWN-ISSUES.md, run.sh — magnotia/Magnotia
-> lumotia/Lumotia.
- docs/ — sweep across all subdirs except docs/handovers/ (preserved
as immutable audit trail). Includes architecture-map references
to magnotia_core::*, magnotia_storage::*, etc. now pointing at
lumotia_*; dev-setup.md tracing output examples (lumotia_startup
target); brief/ + superpowers/ + issues/ + whisper-ecosystem/ +
audit/.
- Cargo.toml descriptions on 9 crates (core, audio, cloud-providers,
hotkey, llm, mcp, plus referenced others).
- crates/core/src/{error,hardware,recommendation,paths}.rs +
crates/audio/src/wav.rs + crates/llm/src/model_manager.rs +
crates/cloud-providers/src/keystore.rs + crates/mcp/src/lib.rs —
doc comments and a model-manager user-agent string.
- Caught on final pass: BroadcastChannel("magnotia_task_sync") -> ...
("lumotia_task_sync"); magnotia_locale i18n localStorage key
renamed + migration shim added; CSS keyframe names
magnotiaPulse / magnotiaBar / magnotiaFade renamed in the design-
system kit; magnotia_viewer_item / magnotia_viewer_mode handoff
keys renamed in HistoryPage + viewer/+page.svelte; src/assets/
wordmark.svg text.
- src-tauri/src/lib.rs comment cleanup ("magnotia era" was sed'd
to "lumotia era" earlier — restored).
Preserved (intentional):
- crates/core/src/paths.rs — keeps "magnotia" / "Magnotia" / ".magnotia"
legacy detection strings in legacy_and_target_paths() so the
migration shim can still find user data from the magnotia era.
- src/lib/stores/{page,focusTimer}.svelte.ts + src/lib/i18n/index.ts
— migration call sites reference the legacy magnotia keys
deliberately.
- docs/handovers/ — historical audit trail.
cargo build --workspace passes. npm run check: 0 errors / 0 warnings
(3958 files). cargo test --workspace: 339 pass / 0 fail.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
97 lines
6.1 KiB
Markdown
97 lines
6.1 KiB
Markdown
# Building Lumotia: a complete technology map for local-first, voice-first desktop AI
|
|
|
|
**Lumotia's entire stack -- from audio capture through LLM inference to neurodivergent-friendly UI -- can be built from actively maintained, production-tested open-source components.** The Rust + Tauri v2 + Svelte 5 ecosystem has matured dramatically through 2024-2026, with reference applications like Handy (13.8k stars, Tauri + Whisper + real-time audio) and Whispering (Svelte 5 + Tauri transcription) proving the core architecture viable. The most critical finding: **no existing app combines all of Lumotia's pieces**, making this a genuinely novel integration -- but every individual subsystem has battle-tested implementations to learn from.
|
|
|
|
**Ingested from:** `input/inbox/backlinksforfree` on 2026/03/20
|
|
**Used in:** `docs/superpowers/specs/2026-03-20-lumotia-mvp-design.md`
|
|
|
|
---
|
|
|
|
## Area 1: Core MVP features
|
|
|
|
### 1. Audio capture pipeline
|
|
|
|
The real-time audio path from microphone to Whisper requires three crates: **cpal** (v0.15.x, Apache 2.0) for cross-platform audio capture, **rubato** (v0.16.2, MIT) for SIMD-accelerated resampling to 16kHz, and a VAD layer. Recommended architecture: three dedicated threads connected by ring buffers.
|
|
|
|
The **voice-stream** crate (v0.4.0) wraps the entire pipeline (cpal + rubato + Silero VAD) into a single library. Fastest path to working audio, though forking allows finer control.
|
|
|
|
For VAD: whisper-rs v0.16's **built-in VAD** (simplest), **silero-vad-rust** (MIT, streaming-ready), voice_activity_detector (used by Handy), **webrtc-vad** (lightweight but lower accuracy).
|
|
|
|
**Reference apps:** Handy (13.8k stars, exact pipeline), Whispering (4.2k stars, Svelte 5 + Tauri), Vibe (v3.0.19, model management patterns).
|
|
|
|
### 2. Whisper integration
|
|
|
|
**whisper-rs** (v0.16.0, 183k+ downloads) is the primary recommendation. **transcribe-rs** (v0.3.0) abstracts over multiple STT engines (whisper.cpp, Parakeet, Moonshine, SenseVoice). **whisper-cpp-plus** adds WhisperStream for real-time streaming with integrated Silero VAD.
|
|
|
|
Two transcription patterns: **chunked-VAD** (simpler, 1-5s latency, used by Handy) vs **overlapping-window streaming** (3.3s latency, more complex). Chunked-VAD is sufficient for voice-first task capture.
|
|
|
|
### 3. Local LLM integration
|
|
|
|
**llama-cpp-2** (MIT/Apache-2.0) provides safe Rust bindings. Does not follow semver -- pin exact versions.
|
|
|
|
Three architectures: **Direct embedding via Tauri Channels** (recommended -- faster, ordered delivery), **sidecar** (fault isolation but process management complexity), **tauri-plugin-llm** (PolyForm licence -- evaluate carefully).
|
|
|
|
Higher-level alternatives: **kalosm** (type-safe structured generation via `#[derive(Parse)]`), **mistral.rs** (pure Rust, PagedAttention).
|
|
|
|
Model lifecycle: load at first inference, keep during session, unload on background/close (simpler than Ollama's 5-minute idle timeout).
|
|
|
|
### 4. sqlite-vec + fastembed RAG pipeline
|
|
|
|
**sqlite-vec** (~7.2k stars, MIT/Apache-2.0) adds vector search via vec0 virtual table. Sub-10ms latency for tens of thousands of vectors. Uses rusqlite with bundled feature.
|
|
|
|
**fastembed-rs** (v5.x, Apache-2.0, Qdrant team) generates embeddings via ONNX Runtime. Recommended: **BGESmallENV15Q** (quantised, ~17MB, 384 dims) or **AllMiniLML6V2** (~23MB).
|
|
|
|
Hybrid search: FTS5 + sqlite-vec with **Reciprocal Rank Fusion** (documented by Alex Garcia). <3ms total retrieval on Raspberry Pi Zero 2 W.
|
|
|
|
**No published project combines sqlite-vec + fastembed-rs** -- Lumotia's implementation is novel.
|
|
|
|
### 5. Time-block visualisation
|
|
|
|
**Schedule-X** (@schedule-x/svelte, v3.0.0, MIT) for day/week calendar views. **Frappe Gantt** (MIT, SVG-based) for timeline. Custom CSS Grid for maximum control.
|
|
|
|
Design references: Tiimo (circular countdown, sensory-friendly), Structured (vertical timeline, energy monitor), Llama Life (single-task focus with countdown), Sunsama (guided daily planning).
|
|
|
|
### 6. Task decomposition
|
|
|
|
GBNF grammar constraints ensure valid JSON output (~25% accuracy improvement). kalosm's `#[derive(Parse)]` eliminates JSON parsing entirely.
|
|
|
|
**Goblin Tools** provides the best UX reference -- "spiciness slider" for decomposition depth. Each step: single concrete physical action, verb-first, 2-15 minutes, energy-level tagged, 20% overestimation buffer, first step highlighted prominently.
|
|
|
|
---
|
|
|
|
## Area 2: Optimisation patterns
|
|
|
|
### 7. Fractional indexing
|
|
|
|
**fractional_index** crate (v2.x, MIT) for Rust. **fractional-indexing** (CC0, ~535k weekly npm) for JS. Reordering updates exactly one row.
|
|
|
|
Pairs with **svelte-dnd-action** (MIT, accessible, keyboard/screen reader) or **@dnd-kit/svelte** (official port, Svelte 5.29+).
|
|
|
|
### 8. Session state restoration
|
|
|
|
**tauri-plugin-store** for persistent key-value. **tauri-plugin-window-state** for window position/size. Timer persistence: `{ startedAt, accumulatedMs, lastResumedAt, state }` with absolute timestamps.
|
|
|
|
### 9. Model downloading
|
|
|
|
reqwest with bytes_stream, HTTP Range headers for resume, incremental SHA256 via ring/sha2. Progress via Tauri Channels (not events). **trauma** crate for resume support.
|
|
|
|
### 10. Tauri v2 local-first patterns
|
|
|
|
**tauri-plugin-sql** for standard SQLite. **rusqlite** with bundled for sqlite-vec. State management: commands for CRUD, events for push notifications, channels for streaming.
|
|
|
|
**cr-sqlite** (Apache-2.0) for future CRDT-based sync (~2.5x write overhead).
|
|
|
|
Reference apps: Screenpipe, GitButler, Musicat, Duckling.
|
|
|
|
### 11. WIP limits
|
|
|
|
Soft limits with progressive visual warning (green to yellow to red). Start with WIP limit of 3, let users adjust per energy/context. "Stop starting, start finishing."
|
|
|
|
### 12. Neurodivergent-first design
|
|
|
|
**No open-source component library exists for neurodivergent users** -- ecosystem gap and differentiation opportunity.
|
|
|
|
Foundation: **shadcn-svelte** + Bits UI for ARIA/keyboard accessibility. Layer neurodivergent styling on top. **OKLCH colour system** with locked Lightness. Reduced motion as default (opt-in, not opt-out). Progressive disclosure below 3 levels. Literal labels always.
|
|
|
|
Essential references: W3C COGA, Microsoft Inclusive Design for Cognition Guidebook.
|