Lumotia/docs/audit/phases-1-8-playbook.md

# Audit Playbook — Phases 1 through 8

*Companion to [`phase0-cartography.md`](phase0-cartography.md). Pick up from any phase.*

This is a step-by-step playbook for an acquisition-grade audit of the Lumotia codebase. Phase 0 (Cartography) is complete; this document describes Phases 1–8.

**How to use this doc.** Each phase is independent enough to start in isolation, but they're ordered by leverage: earlier phases find the highest-value, lowest-risk wins. Don't skip phases without a reason.

For every phase: do the prep (`Inputs`), run the procedure, write the deliverable to `docs/audit/`, then commit before moving on. The deliverable is the audit trail.

---

## Conventions

- All commands assume `cwd = /home/user/lumotia` (or wherever the repo lives).
- All deliverables live under `docs/audit/`. Naming: `phaseN-<short-name>.md`.
- Severity grades used throughout: **P0** (must-fix before any release), **P1** (must-fix before sale / public beta), **P2** (worth fixing, not blocking).
- "Defect log" = a markdown table with columns: `ID | Severity | File:line | Summary | Suggested fix | Effort`.
- Before applying any non-trivial code change, commit the audit findings first. Audit and remediation are separate operations.

---

## Phase 1 — Lean-pass

**Goal.** Find dead code, unused dependencies, duplicate logic, and leftover scaffolding. Apply low-risk deletions; log higher-risk ones for Phase 2.

**Time:** 1 working day.

**Inputs:** Phase 0 §2 (largest files), §9 Tier C (structural smells).

### Procedure

1. **Unused Rust dependencies.**
   ```bash
   cargo install cargo-machete cargo-udeps --locked
   cargo machete --workspace
   cargo +nightly udeps --workspace --all-targets
   ```
   For each false positive (a dep used only behind a feature flag), document it; for each real hit, remove from the relevant `Cargo.toml`.

2. **Unused frontend modules.**
   ```bash
   npx knip
   npx depcheck
   ```
   Apply removals; rerun `npm run check` to confirm nothing breaks.

3. **Dead Rust code.**
   ```bash
   cargo +nightly rustc -p lumotia-core -- -W dead_code -W unused 2>&1 | grep -E "warning|note"
   ```
   Repeat for every crate. Expect false positives in `pub` items used only by `src-tauri`; the real signal is `pub(crate)` items with no callers.

4. **Tech-debt grep.**
   ```bash
   grep -rnE "TODO|FIXME|HACK|XXX|unimplemented!\(\)|todo!\(\)" \
     --include="*.rs" --include="*.svelte" --include="*.ts" \
     --exclude-dir=node_modules --exclude-dir=target . > /tmp/tech-debt.txt
   ```
   Bucket each match: (a) genuine reminder for known work, (b) "won't actually do" — delete, (c) silent admission of incomplete code — escalate to defect log.

5. **`unwrap()` / `expect()` outside tests.**
   ```bash
   grep -rnE "\.(unwrap|expect)\(" crates/ src-tauri/src/ \
     --include="*.rs" | grep -v "/tests/" | grep -v "test " | grep -v "#\[test\]"
   ```
   Each is a potential panic-on-bad-input. For each, prove it can't panic on user data, or replace with `?`/`map_err`.

6. **Duplicate logic in 1k+ LOC files.** Manually walk:
   - `src/lib/pages/SettingsPage.svelte` (2,250 LOC) — already flagged for the seven-group split.
   - `crates/storage/src/database.rs` (2,534 LOC) — split by domain (Phase 0 §9 C2).
   - `src-tauri/src/commands/live.rs` (1,737 LOC) — look for mixed concerns (session lifecycle vs. tuning).
   - `crates/storage/src/migrations.rs` (1,185 LOC) — confirm migrations are append-only and v-numbered.
   - `src/lib/pages/DictationPage.svelte` (1,081 LOC) — extract child components for any block >150 lines.

7. **Cross-file duplicate detection.**
   ```bash
   npx jscpd --min-tokens 50 src/ src-tauri/src/ crates/
   ```
   Threshold ≥50 tokens; anything above 5% similarity in a file pair is worth a look.

### Deliverable

`docs/audit/phase1-lean-pass.md` — three sections:
- **Removed:** what was deleted, with line-count savings.
- **Kept with reason:** items that look unused but aren't (with the reason).
- **Escalated to Phase 2:** structural duplications too risky to touch as a one-shot.

### Acceptance criteria

- `cargo build --workspace` passes.
- `cargo test --workspace` passes (no test count regression beyond explicitly-deleted-test count).
- `npm run check` passes.
- Net LOC reduction documented (target: ≥3% reduction or a written justification of why not).

---

## Phase 2 — Architecture conformance

**Goal.** Verify the 10-crate boundary is real, not aspirational. Tighten public API surfaces. Restructure files >1k LOC where the split is obvious.

**Time:** 1 working day.

**Inputs:** Phase 0 §4 (dependency graph), §3 (pub item counts), Phase 1 escalations.

### Procedure

1. **No upward dependencies.** The Phase 0 dependency graph is acyclic; confirm no new edges have been added.
   ```bash
   for d in crates/*/Cargo.toml; do
     name=$(grep -m1 '^name' "$d" | sed 's/.*"\(.*\)"/\1/')
     deps=$(grep -E "^lumotia[-_]" "$d" | sed 's/ *=.*$//')
     echo "$name -> $deps"
   done
   ```
   If any leaf crate now imports `lumotia` (the Tauri app crate), that's a P0.

2. **Boundary conformance — no SQL outside `lumotia-storage`.**
   ```bash
   grep -rE "sqlx::|sqlite::|sql_query|\\.execute\\(|\\.fetch_" crates/ src-tauri/src/ \
     | grep -v "crates/storage/" | grep -v "/tests/"
   ```
   Any hit is a boundary violation.

3. **Boundary conformance — no `cpal` / `whisper` / `llama` outside their owners.**
   ```bash
   grep -rnE "use cpal" crates/ src-tauri/src/ | grep -v "crates/audio/"
   grep -rnE "use whisper_rs|use whisper-rs" crates/ src-tauri/src/ | grep -v "crates/transcription/"
   grep -rnE "use llama_cpp_2" crates/ src-tauri/src/ | grep -v "crates/llm/"
   ```

4. **Boundary conformance — no business logic in Tauri commands.** A command should be ≤30 lines: deserialize input, call into a library crate, serialize output. Anything else is leakage.
   ```bash
   for f in src-tauri/src/commands/*.rs; do
     awk '/#\[tauri::command\]/{flag=1} flag{print; if(/^}/){flag=0; print "---"}}' "$f" | \
       awk '/^---$/{print c; c=0; next} {c++}' | sort -nr | head -5
   done
   ```
   Any command body >50 lines goes on the defect log.

5. **Reduce `lumotia-core` public surface.** It exports 104 items (Phase 0 §3). For each, run a workspace-wide reverse search:
   ```bash
   grep -rnE "lumotia_core::ITEM_NAME" crates/ src-tauri/src/
   ```
   If the only hits are inside `lumotia-core` itself, demote to `pub(crate)`. Expected outcome: 30–60% reduction.

6. **Apply Phase 0 §9 Tier C structural fixes (C1 and C2 are in scope here).**
   - C1: tighten `lumotia-core` exports.
   - C2: split `crates/storage/src/database.rs` into `database/{transcripts,tasks,profiles,…}.rs`. Re-export from `database/mod.rs` so the public API doesn't move.

7. **`lumotia-cloud-providers` decision.** Phase 0 §9 C4 — fold into `lumotia-core::keystore` or grow it. Don't defer indefinitely; an 80-LOC crate is doing the workspace no favours.

### Deliverable

`docs/audit/phase2-architecture.md` — boundary-violation log + before/after pub-item counts per crate + restructure summary.

### Acceptance criteria

- Zero hits on the SQL / `cpal` / `whisper` / `llama` boundary greps.
- `lumotia-core` public-item count reduced (target: ≤60).
- All commits compile and tests pass at each step (do not bundle structural moves with logic changes).

---

## Phase 3 — Correctness audit (the expensive one)

**Goal.** Walk every public function and every error path. Eliminate panics on bad input. Justify or remove every `unsafe` block.

**Time:** 3–5 working days. Single biggest investment in the audit.

**Inputs:** Phase 1 unwrap log, Phase 2 reduced public surface.

### Procedure

1. **Lints with teeth.**
   ```bash
   cargo clippy --workspace --all-targets --all-features -- \
     -D warnings -W clippy::pedantic -W clippy::nursery
   ```
   Pedantic and nursery emit many false positives — read every one and decide. The yield from `clippy::pedantic` on a real codebase is high.

2. **`unsafe` audit.** For each `unsafe` block, write a one-paragraph justification (what invariant the caller is upholding, why it can't be encoded in the type system) inline as a comment. If you can't write the justification, the `unsafe` is suspect.
   ```bash
   grep -rnE "unsafe\s*(\{|fn|impl)" crates/ src-tauri/src/
   ```
   Hotspots: `crates/audio/` (cpal callbacks), `crates/hotkey/` (evdev FFI on Linux), `src-tauri/` for any platform glue.

3. **Panic surface.** Every `.unwrap()`, `.expect()`, `panic!()`, `unreachable!()`, `assert!()`, slice indexing `[i]`, integer arithmetic that can overflow.
   ```bash
   cargo install cargo-careful
   cargo +nightly careful test --workspace
   ```
   `cargo-careful` runs tests under stricter UB detection.

4. **Property tests on parsing/format functions.**
   - `crates/hotkey/src/lib.rs` — Tauri-style hotkey string parser. `proptest!` with arbitrary modifier sets + key codes; assert round-trip.
   - `crates/audio/src/wav.rs` — WAV decode. Fuzz with `cargo-fuzz` or `libfuzzer-sys` against malformed headers.
   - `src/lib/utils/frontmatter.ts` — YAML frontmatter parse/emit. Fast-check (npm) for round-trip.
   - `crates/storage/src/database.rs` — FTS5 query escaping. Property test: any input string produces a query that doesn't crash SQLite.

5. **Miri on storage and audio.**
   ```bash
   cargo +nightly miri test -p lumotia-storage --lib
   cargo +nightly miri test -p lumotia-audio --lib
   ```
   Catches UB and aliasing bugs that `cargo test` misses.

6. **Manual public-API walk.** For each public function in each crate:
   - What are the preconditions? Are they enforced or assumed?
   - What are the error variants? Are any absorbed silently (`let _ = …`)?
   - Are any return types `Result<…, String>`? — that's a smell; prefer typed errors.

7. **`tracing` audit.** Run with `RUST_LOG=trace` for one full dictation → cleanup → save cycle. Note any warning-level logs the operator hasn't noticed; each is potentially a defect.

### Deliverable

`docs/audit/phase3-correctness.md` — defect log graded P0/P1/P2, plus an `unsafe` justification appendix.

### Acceptance criteria

- Zero `cargo clippy -D warnings` errors.
- Every `unsafe` block has an inline justification comment.
- All P0 defects fixed before phase close; P1 defects logged with an owner.
- Miri tests for storage and audio pass.

---

## Phase 4 — Security & trust boundaries

**Goal.** Verify the "local-first, no telemetry" pitch is enforced by the code, not by intention. Audit every Tauri command and MCP tool as a trust boundary.

**Time:** 2 working days.

**Inputs:** Phase 0 §5.1 (102 Tauri commands), §5.2 (MCP tools).

### Procedure

1. **Network egress audit (the big one).**
   ```bash
   sudo tcpdump -i any -w /tmp/lumotia-egress.pcap host not 127.0.0.1 &
   # …run the app for 30 minutes covering: dictation, cleanup, save, MCP query…
   sudo kill %1
   tshark -r /tmp/lumotia-egress.pcap -q -z conv,ip
   ```
   Allowed: model downloads from huggingface.co (only on user click). Anything else is a P0.

   Cross-check at the syscall level:
   ```bash
   strace -f -e trace=network -o /tmp/lumotia-net.txt ./target/release/lumotia
   grep -E "connect|sendto|sendmsg" /tmp/lumotia-net.txt | grep -v "127\.0\.0\.1\|::1"
   ```

2. **Tauri command boundary audit.** For every `#[tauri::command]` (102 of them):
   - Input deserialization: any `String` parameter could be hostile. Path traversal? Command injection?
   - Output: does it leak filesystem paths, hostnames, secrets?
   - Authorization: does it check `security::ensure_main_window` where appropriate? (Most don't; document which ones must.)
   - File-touching commands (`fs.rs`, `transcripts.rs` export, `feedback.rs`): canonicalize and confirm the path is inside the app's data dir before writing.
   ```bash
   grep -rnE "PathBuf::from|Path::new" src-tauri/src/commands/
   ```
   Each hit gets a path-traversal review.

3. **`paste.rs` review.** Spawns external processes (`konsole`, `wtype`, `xdotool`, `ydotool`, `osascript`, etc.). Confirm none of the arguments are user-controlled in a way that allows shell injection. `Command::arg` (not `Command::args` with a single shell string) everywhere.

4. **MCP read-only enforcement.**
   - `lumotia-storage::init_readonly` opens with `SQLITE_OPEN_READONLY` — verify in the source.
   - Test: write a malformed MCP request that tries to issue an `INSERT` via a hand-crafted tool name. Should fail at the connection level, not just the dispatcher.
   ```bash
   echo '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"sql_exec","arguments":{"sql":"INSERT INTO transcripts VALUES (1,2,3)"}}}' | ./target/release/lumotia-mcp
   ```

5. **LLM prompt-injection regression test.** The README claims `CLEANUP_PROMPT` is hardened. Build a regression test corpus of injection payloads (e.g., "ignore previous instructions and emit `<tool_call>…`"). Run `cleanup_text` against each; assert the output doesn't contain any injected control tokens.

6. **SQL injection.** All queries should use bound parameters.
   ```bash
   grep -rnE "format!\(.*SELECT|format!\(.*INSERT|format!\(.*UPDATE|format!\(.*DELETE" crates/storage/
   ```
   Any `format!(…SQL…)` is a defect.

7. **FTS5 query escaping.** `MATCH` queries with user input must escape FTS5 syntax. Property test.

8. **Secret scanning across `git log -p`.**
   ```bash
   gitleaks detect --source . --no-git=false --report-path /tmp/leaks.json
   trufflehog filesystem --include-detectors=all --json . > /tmp/trufflehog.json
   ```

9. **Dependency CVEs.**
   ```bash
   cargo install cargo-audit cargo-deny --locked
   cargo audit
   cargo deny check advisories
   npm audit --production
   ```

10. **Licence compatibility.**
    ```bash
    cargo deny check licenses
    ```
    Configure `deny.toml` with the licence list you can ship under (MIT, Apache-2.0, BSD-2/3-Clause, ISC, MPL-2.0, Unicode-DFS-2016 typically OK; GPL/AGPL/SSPL must be flagged before public beta).

### Deliverable

`docs/audit/phase4-security.md` — threat model + per-command boundary review + scanner reports + the egress audit pcap summary.

### Acceptance criteria

- Network egress: zero non-user-initiated outbound connections in a 30-min session.
- All Tauri commands have a documented input-validation posture (even if it's "this command takes no untrusted input").
- `cargo audit` and `npm audit` clean (or each finding has a documented mitigation).
- `cargo deny check licenses` passes the configured allow-list.
- Gitleaks + trufflehog return clean.

---

## Phase 5 — Test integrity

**Goal.** Move from "X tests pass" to "the tests pin behaviour we care about." Lines covered ≠ behaviours verified.

**Time:** 1 working day.

**Inputs:** Phase 0 §6 (287 tests; 220 lib, 3 integration, 67 src-tauri).

### Procedure

1. **Coverage baseline.**
   ```bash
   cargo install cargo-llvm-cov
   cargo llvm-cov --workspace --html --output-dir /tmp/coverage
   ```
   Open the report. Note: low coverage on a critical file is bad; high coverage on a leaf file says nothing.

2. **Mutation testing on the heavy crates.**
   ```bash
   cargo install cargo-mutants
   cargo mutants -p lumotia-storage --timeout 60
   cargo mutants -p lumotia-transcription --timeout 60
   cargo mutants -p lumotia-llm --timeout 60
   cargo mutants -p lumotia-audio --timeout 60
   ```
   Surviving mutants = code paths whose tests don't actually verify behaviour. Each survivor either deserves a new test or a deletion.

3. **Tests-as-theatre check.** For a sample of 20 random tests (`shuf -n 20 tests-list.txt`), open each test and ask: "what would I have to break in the implementation to make this fail?" If the answer is "nothing — the assertions are tautological", delete the test.

4. **Regression tests for the audit-grade invariants.** Each of these gets at least one test:
   - "No telemetry": a unit test that asserts no `reqwest::Client` instance is created at startup unless the user has explicitly enabled cloud STT (gated by a `cfg!` or feature flag check).
   - "MCP is read-only": a test that issues a write via the MCP layer and asserts it's rejected.
   - "Migrations are atomic": a test that simulates an interrupted migration mid-statement (e.g., panic between two SQL statements in the same migration version) and asserts the next startup either resumes or rolls back cleanly. (See `docs/issues/c3-migrations-atomicity.md` for context.)
   - "Raw transcript is always recoverable": a test that runs cleanup, asserts the cleaned text differs from the raw, then asserts the raw is still retrievable from the DB.
   - "FTS5 query escaping": property test (see Phase 3 step 4).

5. **CI must enforce coverage.** Add a coverage floor (say, 70% for each crate) to `.github/workflows/check.yml`. New PRs that drop a crate below the floor fail CI.

### Deliverable

`docs/audit/phase5-test-integrity.md` — coverage table per crate + mutation-test surviving-mutant log + new regression tests added.

### Acceptance criteria

- Mutation-testing kill rate ≥80% on `lumotia-storage`, `lumotia-transcription`, `lumotia-llm`.
- Each audit-grade invariant has at least one passing regression test.
- Coverage floor enforced in CI.

---

## Phase 6 — Performance & resource profile

**Goal.** Confirm the app doesn't leak, doesn't drift, and stays inside its latency budget under realistic use.

**Time:** 1 working day.

**Inputs:** none specific — use realistic dictation workloads.

### Procedure

1. **Long-session leak check.**
   ```bash
   # Linux:
   ./target/release/lumotia & PID=$!
   while sleep 60; do
     ps -p $PID -o rss,vsz,nlwp,fd | tee -a /tmp/lumotia-rss.csv
   done
   ```
   Run for 1 hour with periodic dictation. Plot RSS vs. time. A monotonic upward slope is a leak.

2. **File-descriptor count.**
   ```bash
   ls /proc/$PID/fd | wc -l   # repeat over time
   ```
   FD count should be bounded.

3. **Heap profile.**
   ```bash
   heaptrack ./target/release/lumotia
   # …run one full dictate → cleanup → save cycle…
   heaptrack_print heaptrack.lumotia.*.zst | head -100
   ```
   Look for allocators in the cleanup path that aren't freed.

4. **CPU hot path.**
   ```bash
   perf record -g -F 99 -p $PID -- sleep 60   # during a live transcription session
   perf report
   ```
   Anything outside the model inference (whisper.cpp, llama.cpp) using >5% CPU is a candidate finding.

5. **Cold-start budget.**
   ```bash
   time ./target/release/lumotia --headless-startup-test    # add this entrypoint if missing
   ```
   Target: < 2s from launch to "recording-ready". Anything slower → profile with `samply`.

6. **Audio device hot-plug stress.** Plug/unplug USB mic 20 times during a dictation session. Count: leaks, panics, dropped frames. (cpal hotplug is the documented hotspot.)

### Deliverable

`docs/audit/phase6-performance.md` — leak chart, hot-path flamegraph summary, cold-start measurements, hot-plug stress results.

### Acceptance criteria

- RSS plateaus within 10 minutes of dictation start (no monotonic growth).
- FD count bounded.
- Cold start < 2s on the reference machine (document the machine).
- No panics in the hot-plug stress test.

---

## Phase 7 — Build & release reproducibility

**Goal.** Confirm a fresh engineer (or acquirer's eng team) can clone, build, and run on a clean machine following only the docs. If they can't, the deal stalls.

**Time:** ½ working day.

**Inputs:** `docs/dev-setup.md`, `.github/workflows/build.yml`.

### Procedure

1. **Fresh container build.** Use a Docker container matching one supported OS at a time.
   ```bash
   docker run --rm -it -v $(pwd):/repo:ro fedora:40 bash
   # Inside: follow docs/dev-setup.md literally, time each step.
   ```
   Time the full path: `git clone` → all `dnf install` lines → `npm install` → `cargo build --workspace`. Document every step that's missing or wrong in the docs.

2. **CI parity.** Compare local build with `.github/workflows/build.yml`. Any drift between local and CI is a P1 reproducibility risk.

3. **Bundle build.**
   ```bash
   npm run tauri build
   ```
   Confirm the resulting `.AppImage` / `.deb` / `.dmg` / `.msi` runs on a clean target OS.

4. **Bundle ID + signing transferability.** Confirm:
   - `uk.co.corbel.lumotia` bundle ID is owned, not squatted.
   - Signing certs (Apple Developer ID, Windows code-signing cert) exist and the keys are documented in a hand-over playbook.
   - Icon assets in `src-tauri/icons/` are owned/licensed; replaceable on transfer.

5. **`run.sh` works as documented.** Run on a fresh checkout; it should JustWork.

### Deliverable

`docs/audit/phase7-reproducibility.md` — fresh-build walkthrough log, missing-step list for `dev-setup.md`, bundle-build evidence, transferability checklist.

### Acceptance criteria

- Fresh-container build succeeds following `dev-setup.md` verbatim. (Update the docs if not.)
- All three bundle targets build successfully in CI.
- Transferability checklist signed off.

---

## Phase 8 — Documentation truth

**Goal.** Re-walk every public doc against the post-audit code. Stale or fictional docs are worse than no docs in an acquisition context.

**Time:** ½ working day.

**Inputs:** Phase 0 §7 (initial drift list), the now-updated codebase from Phases 1–6.

### Procedure

1. **Re-run Phase 0 §7 checks.** Phases 1–4 will have moved things; the README needs another pass.
   - Test count
   - Crate count
   - Tauri command module list
   - Stores list
   - Model-registry contents

2. **`README.md` ↔ source-of-truth pairings.** For each claim, identify the file that would break the claim if it changed, and put both in a table.

3. **Archive HANDOVER files.** Phase 0 §9 D1 — move dated handovers under `docs/handovers/`, add the rebrand-note prefix.

4. **`docs/brief/` and `docs/whisper-ecosystem/` re-read.** Any roadmap claim that's now shipped → move to a `done.md` archive. Any claim that's now de-scoped → mark as such with the date.

5. **`docs/issues/` triage.** Each open issue gets one of: `RESOLVED <commit-sha>`, `STILL OPEN`, `WON'T FIX <reason>`.

6. **Add an `AUDIT.md` at repo root.** Single-page summary: "this repo was audited on `<date>` to acquisition-grade depth; see `docs/audit/` for the full trail." Future maintainers (and acquirers) need this signpost.

### Deliverable

`docs/audit/phase8-docs-truth.md` — diff log of doc changes, archive moves, and the new `AUDIT.md`.

### Acceptance criteria

- Zero stale claims in `README.md` (re-verified).
- All `docs/issues/` items triaged.
- `AUDIT.md` exists at repo root.

---

## Closing the audit

After Phase 8, the deliverables in `docs/audit/` should read as a coherent, sequential story:

```
docs/audit/
├── phase0-cartography.md         (done — survey + drift log + fix areas)
├── phase1-lean-pass.md           (kill list, applied)
├── phase2-architecture.md        (boundary log + restructure)
├── phase3-correctness.md         (defect log + unsafe justifications)
├── phase4-security.md            (threat model + scanner reports + egress audit)
├── phase5-test-integrity.md      (coverage + mutation results + new tests)
├── phase6-performance.md         (leak chart, hot path, cold start)
├── phase7-reproducibility.md     (fresh-build walkthrough + transfer checklist)
└── phase8-docs-truth.md          (post-audit doc reconciliation)
```

With `AUDIT.md` at the root pointing into the directory.

That's the artefact an acquirer's engineering team gets. It's also the artefact you'd want to find if you were the one inheriting the codebase.

---

*End of playbook.*