Lumotia/docs/brief/tech-stack.md

3. Tech Stack

Core framework

• Framework: Tauri v2.10+ (Rust backend, Svelte 5 frontend)
• Database: SQLite via rusqlite v0.31 (bundled, with load_extension support)
• Platforms: Windows, macOS, Linux (primary), Android and iOS (secondary — Tauri v2 mobile support)
• Testing device: Pixel 9 Pro XL (Android)

AI transcription

• Engine: whisper-rs v0.16.0 (Rust bindings to whisper.cpp). Supports CUDA, Vulkan, Metal, OpenBLAS, and CoreML acceleration. Built-in Voice Activity Detection via Silero for automatic silence trimming.
• Desktop model: ggml-base.en (~142MB). Processes 5 minutes of audio in ~10–15 seconds on a modern CPU.
• Mobile model: ggml-tiny.en (~75MB). Lighter footprint for constrained devices.
• Audio format: 16kHz mono f32 PCM. Use Tauri's media APIs to capture and convert.

AI reasoning (local LLM)

• Inference engine: llama-cpp-2 crate (utilityai/llama-cpp-rs) — safe Rust wrappers around llama.cpp with GGUF format support, CUDA/Vulkan/Metal backends via feature flags, tool-calling support.
• Hardware tiers:

Hardware	RAM	Model	Quantisation	Size	CPU Speed
Minimum	8GB	Phi-4-mini (3.8B)	Q4_K_M	~2.3GB	15–25 tok/s
Recommended	16GB	Qwen 3 7B	Q4_K_M	~4.5GB	10–20 tok/s
Optimal	32GB	Llama 3.3 8B	Q5_K_M	~5.5GB	10–20 tok/s
Mobile	4–6GB	Llama 3.2 1B	Q4_K_M	~0.8GB	30–50 tok/s

• Benchmarks: Ryzen 5700G (DDR4) achieves ~11 tok/s on 7B Q4_K_M. Apple M3 base achieves ~26 tok/s. For Kon's use case (50–200 token responses for task decomposition), 10–15 tok/s is perfectly usable (1–10 seconds per response).
• Minimum published spec: 8GB RAM, any CPU from 2020+. Below 8GB is not supported.

Local RAG pipeline

• Vector search: sqlite-vec v0.1.0 (Alex Garcia). Pure C SQLite extension, zero external dependencies. Creates vec0 virtual tables alongside regular tables. Brute-force KNN completes in ~20ms for 100,000 vectors at 384 dimensions. Everything lives in one .db file — no second data store.
• Embeddings: fastembed v5.12.0 (wraps ONNX Runtime). Default model: BGE-small-en-v1.5 quantised — 33M parameters, 384 dimensions, ~35MB model file, ~20ms per 1,000 tokens on CPU. For 16GB+ machines: nomic-embed-text-v1.5 (768 dimensions, 8,192 token context).
• Chunking strategy: Recursive character splitting at 400–512 tokens with 15% overlap. Split on sentence boundaries first (natural speech has clear breaks), then fall back to recursive splitting. Research (Vectara, NAACL 2025) confirms fixed-size chunking outperforms semantic chunking for retrieval accuracy.
• RAG pipeline stages: Voice → Whisper transcription → Chunking → Embedding via fastembed → Vector storage in sqlite-vec → KNN retrieval on query → Context assembly → LLM inference → Response.

AI agent framework (MCP)

• Protocol: Model Context Protocol (MCP) via rmcp v0.16.0 (official Rust SDK). JSON-RPC 2.0 with STDIO transport — runs entirely in-process, no network, no cloud.
• Core tools defined:
  • create_task — creates a new task with title (must start with a verb), priority, and project
  • search_history — embeds query → sqlite-vec KNN → returns relevant transcription chunks
  • set_reminder — creates a time-based or context-based reminder
  • decompose_task — sends abstract task to local LLM with micro-stepping system prompt, returns 3–7 concrete steps
• Autonomous loop: Background agent runs every 30 minutes (or on new transcription). Observe recent activity → Analyse patterns via embedding search → Generate 1–2 proactive suggestions → Present as non-intrusive badges. All suggestions require explicit user confirmation — never auto-execute.

Cross-device sync (post-MVP)

• CRDT layer: cr-sqlite (vlcn.io, ~3,500 GitHub stars, core Rust). Operates at the SQL level — SELECT crsql_as_crr('tasks') converts any table to a Conflict-free Replicated Relation. Normal SQL continues working. Metadata overhead: ~50–100 bytes per modified cell.
• Networking: iroh (n0-computer/iroh, ~7,900 GitHub stars, pure Rust, v0.96+). Dials peers by Ed25519 public key. Auto-selects best path: direct QUIC on LAN, NAT hole-punching on WAN, or encrypted relay fallback. QUIC with TLS 1.3. Relays are zero-knowledge.
• Local discovery: mdns-sd crate v0.13.11. Registers _kon-sync._tcp.local. via multicast DNS.
• Device pairing: QR code + Noise XX handshake (snow crate v0.9.x) with OTP pre-shared key. No server required.
• Relay fallback: Self-host with cargo install iroh-relay on a £4/month VPS. n0 also operates free public relays (rate-limited).
• Conflict resolution: Last-Writer-Wins per field (highest lamport timestamp, site_id tiebreaker). Edits to different fields merge cleanly. Extended offline: changeset size proportional to number of changes, not duration.
• Risk note: cr-sqlite development pace has slowed since late 2024. Fallback plan: Automerge + SQLite BLOB storage, reusing the entire iroh/mDNS networking stack unchanged.

Context management for long-term memory

Layer	Content	Token Budget
Immediate	Current query + last 2–3 exchanges	~500
Retrieved	Top-5 semantically relevant chunks from sqlite-vec	~1,500
Session	Running summary of current session	~300
Long-term	Compressed summaries of older transcriptions	~200

• Progressive summarisation: Transcriptions >7 days old get LLM-generated summaries. >30 days: merge into monthly digests. Original chunks remain vector-searchable. Summaries used for context injection.

Core Rust dependencies

[dependencies]
tauri = "2.10"
rusqlite = { version = "0.31", features = ["bundled", "load_extension"] }
whisper-rs = "0.16"
llama-cpp-2 = { version = "0.1", features = ["vulkan"] }
fastembed = "5"
sqlite-vec = "0.1"
rmcp = { version = "0.16", features = ["server", "transport-io", "macros"] }
iroh = "0.96"
mdns-sd = "0.13"
snow = "0.9"
ed25519-dalek = "2.1"
tokio = { version = "1", features = ["full"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
uuid = { version = "1", features = ["v4"] }
chrono = "0.4"
tauri-plugin-store = "2"
tauri-plugin-notification = "2"
tauri-plugin-window-state = "2"