Lumotia/docs/architecture-map/05-core-storage-hotkey-build/storage-overview.md

name, type, slice, last_verified

name	type	slice	last_verified
Storage overview	architecture-map-page	05-core-storage-hotkey-build	2026/05/09

Storage overview

Where you are: Architecture map → Core, Storage, Hotkey, Build → Storage overview

Plain English summary. The storage crate owns Lumotia's SQLite database, its FTS5 transcript index, the file-system paths it writes to, and the migration machinery that brings a fresh DB up to head. It uses sqlx 0.8 in a slimmed-down configuration that strips out features the crate does not need.

At a glance

• Crate: lumotia-storage.
• LOC: 3,771 (database 2,534, migrations 1,185, file_storage 28, lib 24).
• External deps: sqlx 0.8 (runtime-tokio, sqlite; no default features), tokio 1, serde 1, log 0.4, uuid 1 (v4), lumotia-core (path).
• Public surface: 46 pub async fn (every CRUD verb listed in crates/storage/src/lib.rs), one pub fn (as_str), 9 pub structs (param types + row types), 1 pub enum (FeedbackTargetType), 1 pub const (DEFAULT_PROFILE_ID), plus the file-storage path re-exports.
• Consumers: every Tauri command that persists or reads (slice 2); the MCP server (slice 4) opens the same database read-only via init_readonly.

sqlx configuration

sqlx = { version = "0.8", default-features = false, features = ["runtime-tokio", "sqlite"] }

default-features = false strips sqlx's any, macros, migrate, and json features. None of these are used:

• any — query builder for "any database backend"; we are SQLite-only.
• macros — compile-time-checked queries; we use runtime queries via sqlx::query() and sqlx::query_scalar().
• migrate — sqlx's bundled migration runner; we run our own custom migration machinery (see storage-schema-and-migrations.md).
• json — JSON column adapters; we serialise JSON into TEXT columns via serde_json::to_string at the call site.

This cuts ~40% of sqlx's compile graph, most visibly on Windows MSVC where every proc-macro crate (which macros pulls in) becomes a slow .dll link. Documented inline at crates/storage/Cargo.toml.

Init flow

init(db_path) — crates/storage/src/database.rs:9

The standard read-write entry point. Steps:

1. std::fs::create_dir_all(parent) so a fresh install has the parent directory.
2. SqliteConnectOptions::new().filename(db_path).create_if_missing(true).
3. SqlitePoolOptions::new().max_connections(5).connect_with(options).
4. PRAGMA foreign_keys = ON — without this, sqlite ignores REFERENCES clauses at runtime even though they parse.
5. run_migrations(&pool) — applies any pending migrations. See storage-schema-and-migrations.md.

Returns a SqlitePool that is held in tauri::State for the rest of the process.

init_readonly(db_path) — crates/storage/src/database.rs:40

The MCP server's entry point. Same options minus create_if_missing(true) and plus read_only(true). max_connections = 2 because the MCP server is single-purpose.

The structural read-only constraint matters: it makes "the MCP server cannot mutate user data" a property of the connection rather than a property of the dispatcher being well-behaved. Even if a future MCP tool tries to INSERT, sqlite refuses at the connection level.

Public surface (lib.rs re-exports)

The complete public CRUD surface is re-exported from crates/storage/src/lib.rs:

• Transcripts: insert_transcript, get_transcript, list_transcripts, list_transcripts_paged, count_transcripts, update_transcript, update_transcript_meta, delete_transcript, search_transcripts.
• Tasks: insert_task, list_tasks, get_task_by_id, update_task, set_task_energy, complete_task, uncomplete_task, delete_task. Plus subtasks: insert_subtask, list_subtasks, complete_subtask_and_check_parent. Plus completion analytics: list_recent_completions.
• Profiles: list_profiles, get_profile, create_profile, update_profile, delete_profile. Plus profile terms: list_profile_terms, add_profile_term, delete_profile_term.
• Settings: set_setting, get_setting.
• Error log: log_error, prune_error_log, list_recent_errors.
• Feedback (HITL): record_feedback, list_feedback_examples.
• Implementation rules: insert_implementation_rule, list_implementation_rules, get_implementation_rule, set_implementation_rule_enabled, mark_implementation_rule_fired, delete_implementation_rule.
• Init: init, init_readonly.
• File paths (re-exported from file_storage.rs): app_data_dir, database_path, recordings_dir, crashes_dir, logs_dir.
• Constants: DEFAULT_PROFILE_ID = "00000000-0000-0000-0000-000000000001".

Per-table CRUD is split across the per-page docs in this slice. See:

• Transcripts CRUD
• Tasks CRUD
• Profiles CRUD
• Settings, error log, feedback, implementation rules
• FTS5 search
• Schema and migrations
• File paths

Watch-outs

• PRAGMA foreign_keys = ON is per-connection, not per-database. The pool's max_connections = 5 means we run the pragma once at init on the first connection. SQLite re-applies the pragma on each new pool connection because we set it via the connect options... but actually we don't, we set it after connect_with. This is a latent issue worth verifying: if a second pool connection opens later, foreign keys may not be enforced on it. Audit candidate.
• No connection-level retry on locked DB. SQLITE_BUSY propagates as lumotia_storage::Error::Query { ... } (flattened to MagnotiaError::Storage { kind: Query, ... } at the boundary). With WAL mode + 5 max connections this is rare, but a long-running write under a slow filesystem could trigger it.
• Custom migration runner. sqlx's bundled migrate! macro is not used. The custom runner is documented in storage-schema-and-migrations.md and was the subject of the C3 critical-issue write-up at docs/issues/c3-migrations-atomicity.md.

Existing in-repo docs

• docs/issues/c3-migrations-atomicity.md — drove the transactional migration design.
• docs/issues/c4-transcript-profile-fk.md — drove migration v9.

See also

• Schema and migrations
• File storage paths
• Slice 2 Tauri startup — the caller of init.
• Slice 4 MCP server — the caller of init_readonly.