feat(plan): add Phase 2 functional MVP implementation plan
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
jake
902
docs/superpowers/plans/2026-03-21-phase2-functional-mvp.md
Normal file
902
docs/superpowers/plans/2026-03-21-phase2-functional-mvp.md
Normal file
@@ -0,0 +1,902 @@
|
||||
# Kon Phase 2: Functional MVP — Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Transform Kon from a branded shell into a functional voice → text → tasks pipeline with local LLM intelligence, delivering a shippable closed-beta desktop app.
|
||||
|
||||
**Architecture:** The existing codebase has a working audio capture → Whisper transcription → text display pipeline via browser AudioWorklet + Tauri IPC. Phase 2 migrates persistence from localStorage to SQLite (backend already has schema + CRUD), adds FTS5 search, wires llama-cpp-2 for local LLM task extraction and micro-stepping, connects the VisualTimer to tasks, and polishes first-run + settings + export.
|
||||
|
||||
**Tech Stack:** Svelte 5, SvelteKit 2, Tailwind CSS 4.2, Tauri 2, Rust, sqlx (SQLite), whisper-rs (via transcribe-rs), llama-cpp-2, lucide-svelte
|
||||
|
||||
**Branch:** `phase-2/functional-mvp`
|
||||
|
||||
**Commit format:** `feat(scope): description`
|
||||
|
||||
---
|
||||
|
||||
## Existing State Summary
|
||||
|
||||
### Already Working
|
||||
- Microphone capture via browser AudioWorklet → 16kHz mono PCM
|
||||
- Whisper + Parakeet transcription via transcribe-rs (streaming chunks)
|
||||
- Model download/load/cache management
|
||||
- Text post-processing (filler removal, British English, anti-hallucination)
|
||||
- Rule-based task extraction (frontend JS — `taskExtractor.js`)
|
||||
- Task CRUD in localStorage with BroadcastChannel multi-window sync
|
||||
- History in localStorage with playback
|
||||
- File transcription (drag-drop, multi-format)
|
||||
- Preferences store with SQLite persistence
|
||||
- Full brand token system, accessibility controls, sensory zones
|
||||
|
||||
### Needs Building
|
||||
1. **SQLite migration v2**: Add `priority`, `project`, `status`, `updated_at` to tasks; add FTS5 virtual table for transcripts
|
||||
2. **Tauri commands for task CRUD**: Replace localStorage task management with SQLite backend
|
||||
3. **Tauri commands for transcript persistence**: Save transcriptions to SQLite (currently only localStorage)
|
||||
4. **FTS5 full-text search**: Backend search across transcriptions
|
||||
5. **llama-cpp-2 integration**: Wire LLM inference engine for task extraction + micro-stepping
|
||||
6. **LLM model management**: Download/cache GGUF models (Phi-4-mini, Qwen 3 7B)
|
||||
7. **Micro-stepping UI**: Inline micro-steps below parent tasks with "Just Start" timer
|
||||
8. **VisualTimer wiring**: Connect timer to tasks, add notifications
|
||||
9. **Export to Obsidian**: Markdown with YAML frontmatter
|
||||
10. **Global hotkey update**: Change default from Ctrl+Shift+R to Ctrl+Shift+Space
|
||||
11. **Settings backend wiring**: Migrate remaining settings to SQLite preferences
|
||||
|
||||
---
|
||||
|
||||
## File Map
|
||||
|
||||
### New files to create
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `crates/ai-formatting/src/llm_client.rs` | llama-cpp-2 inference wrapper (rewrite from placeholder) |
|
||||
| `crates/ai-formatting/src/task_extraction.rs` | LLM-based task extraction with fallback to rule-based |
|
||||
| `crates/ai-formatting/src/micro_stepping.rs` | Task decomposition into micro-steps |
|
||||
| `crates/llm/Cargo.toml` | New crate for LLM model management |
|
||||
| `crates/llm/src/lib.rs` | LLM engine wrapper |
|
||||
| `crates/llm/src/model_manager.rs` | GGUF model download/cache |
|
||||
| `crates/llm/src/inference.rs` | Token streaming inference |
|
||||
| `src-tauri/src/commands/tasks.rs` | Task CRUD Tauri commands |
|
||||
| `src-tauri/src/commands/history.rs` | Transcript persistence + FTS5 search commands |
|
||||
| `src-tauri/src/commands/llm.rs` | LLM model management + inference commands |
|
||||
| `src/lib/components/MicroSteps.svelte` | Micro-step display + "Just Start" button |
|
||||
| `src/lib/components/TaskTimer.svelte` | Timer wired to specific task |
|
||||
| `src/lib/stores/tasks.svelte.js` | Task store backed by SQLite via Tauri commands |
|
||||
| `src/lib/stores/history.svelte.js` | History store backed by SQLite |
|
||||
| `src/lib/utils/obsidianExport.js` | Obsidian vault export logic |
|
||||
|
||||
### Files to modify
|
||||
|
||||
| File | Changes |
|
||||
|---|---|
|
||||
| `crates/storage/src/migrations.rs` | Add migration v2 (FTS5, task columns, timer state) |
|
||||
| `crates/storage/src/database.rs` | Add task CRUD with new columns, FTS5 search, timer persistence |
|
||||
| `crates/ai-formatting/Cargo.toml` | Add serde, serde_json dependencies |
|
||||
| `src-tauri/Cargo.toml` | Add llama-cpp-2, tauri-plugin-notification |
|
||||
| `src-tauri/src/lib.rs` | Register new commands, add LLM state |
|
||||
| `src-tauri/src/commands/mod.rs` | Add new command modules |
|
||||
| `src/lib/pages/DictationPage.svelte` | Wire SQLite transcript persistence |
|
||||
| `src/lib/pages/TasksPage.svelte` | Wire SQLite task CRUD, add micro-steps |
|
||||
| `src/lib/pages/HistoryPage.svelte` | Wire FTS5 search, SQLite history |
|
||||
| `src/lib/pages/FilesPage.svelte` | Wire SQLite persistence for file transcriptions |
|
||||
| `src/lib/pages/FirstRunPage.svelte` | Add LLM model download step |
|
||||
| `src/lib/pages/SettingsPage.svelte` | Wire remaining settings to backend |
|
||||
| `src/lib/stores/page.svelte.js` | Remove localStorage task/history stores (migrate to new stores) |
|
||||
| `src/lib/components/WipTaskList.svelte` | Add micro-step expansion, timer button |
|
||||
| `src/lib/components/VisualTimer.svelte` | Add countdown logic, notifications |
|
||||
| `src/lib/components/ModelDownloader.svelte` | Support LLM model downloads |
|
||||
| `Cargo.toml` | Add crates/llm to workspace |
|
||||
|
||||
---
|
||||
|
||||
## Phase 2A — Core Pipeline
|
||||
|
||||
### Task 1: SQLite Migration v2 — Schema Extensions
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/storage/src/migrations.rs`
|
||||
- Modify: `crates/storage/src/database.rs`
|
||||
- Modify: `crates/storage/Cargo.toml`
|
||||
|
||||
**Why first:** Everything else depends on the database schema being right.
|
||||
|
||||
- [ ] **Step 1: Add migration v2 to migrations.rs**
|
||||
|
||||
Add after the existing migration v1 entry in the `MIGRATIONS` array:
|
||||
|
||||
```rust
|
||||
(2, "phase 2 — task fields, FTS5, timer state", r#"
|
||||
ALTER TABLE tasks ADD COLUMN priority TEXT NOT NULL DEFAULT 'medium';
|
||||
ALTER TABLE tasks ADD COLUMN project TEXT;
|
||||
ALTER TABLE tasks ADD COLUMN status TEXT NOT NULL DEFAULT 'pending';
|
||||
ALTER TABLE tasks ADD COLUMN updated_at TEXT NOT NULL DEFAULT (datetime('now'));
|
||||
ALTER TABLE tasks ADD COLUMN sort_order INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE tasks ADD COLUMN notes TEXT NOT NULL DEFAULT '';
|
||||
|
||||
CREATE VIRTUAL TABLE IF NOT EXISTS transcripts_fts USING fts5(
|
||||
text,
|
||||
title,
|
||||
content='transcripts',
|
||||
content_rowid='rowid'
|
||||
);
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS transcripts_ai AFTER INSERT ON transcripts BEGIN
|
||||
INSERT INTO transcripts_fts(rowid, text, title)
|
||||
VALUES (new.rowid, new.text, new.title);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS transcripts_ad AFTER DELETE ON transcripts BEGIN
|
||||
INSERT INTO transcripts_fts(transcripts_fts, rowid, text, title)
|
||||
VALUES ('delete', old.rowid, old.text, old.title);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS transcripts_au AFTER UPDATE ON transcripts BEGIN
|
||||
INSERT INTO transcripts_fts(transcripts_fts, rowid, text, title)
|
||||
VALUES ('delete', old.rowid, old.text, old.title);
|
||||
INSERT INTO transcripts_fts(rowid, text, title)
|
||||
VALUES (new.rowid, new.text, new.title);
|
||||
END;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS timer_state (
|
||||
id TEXT PRIMARY KEY DEFAULT 'active',
|
||||
task_id TEXT NOT NULL,
|
||||
total_seconds INTEGER NOT NULL,
|
||||
remaining_seconds INTEGER NOT NULL,
|
||||
started_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
paused INTEGER NOT NULL DEFAULT 0
|
||||
)
|
||||
"#),
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add new database functions to database.rs**
|
||||
|
||||
Add task functions with new columns:
|
||||
|
||||
```rust
|
||||
// Task CRUD with extended fields
|
||||
pub async fn insert_task_v2(pool, id, text, priority, project, status, bucket, effort, source_transcript_id, sort_order) -> Result<()>
|
||||
pub async fn update_task_v2(pool, id, text, priority, project, status, bucket, effort, notes) -> Result<()>
|
||||
pub async fn reorder_tasks(pool, task_ids: &[String]) -> Result<()>
|
||||
pub async fn list_tasks_by_status(pool, status, limit) -> Result<Vec<TaskRow>>
|
||||
pub async fn search_transcripts(pool, query: &str, limit: i64) -> Result<Vec<TranscriptRow>>
|
||||
|
||||
// Timer state persistence
|
||||
pub async fn save_timer_state(pool, task_id, total_seconds, remaining_seconds, paused) -> Result<()>
|
||||
pub async fn get_timer_state(pool) -> Result<Option<TimerStateRow>>
|
||||
pub async fn clear_timer_state(pool) -> Result<()>
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add FTS5 search function**
|
||||
|
||||
```rust
|
||||
pub async fn search_transcripts(pool: &SqlitePool, query: &str, limit: i64) -> Result<Vec<TranscriptRow>> {
|
||||
let rows = sqlx::query(
|
||||
"SELECT t.id, t.text, t.source, t.title, t.audio_path, t.duration, t.engine, t.model_id, t.inference_ms, t.sample_rate, t.audio_channels, t.format_mode, t.remove_fillers, t.british_english, t.anti_hallucination, t.created_at
|
||||
FROM transcripts t
|
||||
JOIN transcripts_fts fts ON t.rowid = fts.rowid
|
||||
WHERE transcripts_fts MATCH ?1
|
||||
ORDER BY rank
|
||||
LIMIT ?2"
|
||||
)
|
||||
.bind(query)
|
||||
.bind(limit)
|
||||
.fetch_all(pool)
|
||||
.await
|
||||
.map_err(|e| KonError::StorageError(format!("FTS search failed: {e}")))?;
|
||||
Ok(rows.iter().map(transcript_row_from).collect())
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests**
|
||||
|
||||
```bash
|
||||
cd crates/storage && cargo test
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Verify Tauri app compiles**
|
||||
|
||||
```bash
|
||||
cd src-tauri && cargo check
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/storage/
|
||||
git commit -m "feat(storage): add migration v2 — task fields, FTS5 search, timer state"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Tauri Commands for Transcript Persistence
|
||||
|
||||
**Files:**
|
||||
- Create: `src-tauri/src/commands/history.rs`
|
||||
- Modify: `src-tauri/src/commands/mod.rs`
|
||||
- Modify: `src-tauri/src/lib.rs`
|
||||
- Modify: `src-tauri/src/commands/transcription.rs`
|
||||
|
||||
- [ ] **Step 1: Create history.rs with transcript CRUD commands**
|
||||
|
||||
```rust
|
||||
// save_transcript — persist completed transcription to SQLite
|
||||
// get_transcript — fetch by ID
|
||||
// list_transcripts — paginated list, newest first
|
||||
// delete_transcript — remove by ID
|
||||
// search_transcripts — FTS5 search
|
||||
// save_segments — batch insert segments for a transcript
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Register commands in mod.rs and lib.rs**
|
||||
|
||||
- [ ] **Step 3: Modify transcription.rs to auto-persist**
|
||||
|
||||
After successful transcription, auto-save the transcript + segments to SQLite (in addition to emitting the event).
|
||||
|
||||
- [ ] **Step 4: Verify compilation**
|
||||
|
||||
```bash
|
||||
cd src-tauri && cargo check
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src-tauri/
|
||||
git commit -m "feat(history): add Tauri commands for transcript persistence and FTS5 search"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Tauri Commands for Task CRUD
|
||||
|
||||
**Files:**
|
||||
- Create: `src-tauri/src/commands/tasks.rs`
|
||||
- Modify: `src-tauri/src/commands/mod.rs`
|
||||
- Modify: `src-tauri/src/lib.rs`
|
||||
|
||||
- [ ] **Step 1: Create tasks.rs**
|
||||
|
||||
Commands:
|
||||
```rust
|
||||
#[tauri::command] async fn create_task(state, text, priority, project, bucket, effort, source_transcript_id) -> Result<TaskResponse, String>
|
||||
#[tauri::command] async fn update_task(state, id, text, priority, project, status, bucket, effort, notes) -> Result<(), String>
|
||||
#[tauri::command] async fn delete_task(state, id) -> Result<(), String>
|
||||
#[tauri::command] async fn list_tasks(state, status, limit) -> Result<Vec<TaskResponse>, String>
|
||||
#[tauri::command] async fn reorder_tasks(state, task_ids: Vec<String>) -> Result<(), String>
|
||||
#[tauri::command] async fn complete_task(state, id) -> Result<(), String>
|
||||
```
|
||||
|
||||
TaskResponse struct:
|
||||
```rust
|
||||
#[derive(Serialize)]
|
||||
struct TaskResponse {
|
||||
id: String,
|
||||
text: String,
|
||||
priority: String,
|
||||
project: Option<String>,
|
||||
status: String,
|
||||
bucket: String,
|
||||
effort: Option<String>,
|
||||
done: bool,
|
||||
done_at: Option<String>,
|
||||
created_at: String,
|
||||
updated_at: String,
|
||||
sort_order: i64,
|
||||
notes: String,
|
||||
source_transcript_id: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Register commands in mod.rs and lib.rs**
|
||||
|
||||
- [ ] **Step 3: Verify compilation**
|
||||
|
||||
```bash
|
||||
cd src-tauri && cargo check
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add src-tauri/
|
||||
git commit -m "feat(tasks): add Tauri commands for full task CRUD with priority, project, status"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Frontend Task Store Migration (localStorage → SQLite)
|
||||
|
||||
**Files:**
|
||||
- Create: `src/lib/stores/tasks.svelte.js`
|
||||
- Modify: `src/lib/pages/TasksPage.svelte`
|
||||
- Modify: `src/lib/components/WipTaskList.svelte`
|
||||
- Modify: `src/lib/stores/page.svelte.js`
|
||||
|
||||
- [ ] **Step 1: Create tasks.svelte.js**
|
||||
|
||||
New store that wraps Tauri commands instead of localStorage:
|
||||
|
||||
```javascript
|
||||
import { invoke } from '@tauri-apps/api/core';
|
||||
|
||||
let tasks = $state([]);
|
||||
let loading = $state(false);
|
||||
|
||||
export async function loadTasks() { ... }
|
||||
export async function createTask(text, opts = {}) { ... }
|
||||
export async function updateTask(id, updates) { ... }
|
||||
export async function deleteTask(id) { ... }
|
||||
export async function completeTask(id) { ... }
|
||||
export async function reorderTasks(ids) { ... }
|
||||
export function getTasks() { return tasks; }
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update TasksPage.svelte to use new store**
|
||||
|
||||
Replace all `tasks` imports from page.svelte.js with the new SQLite-backed store.
|
||||
|
||||
- [ ] **Step 3: Update WipTaskList.svelte**
|
||||
|
||||
Wire to new task store.
|
||||
|
||||
- [ ] **Step 4: Keep page.svelte.js tasks for backwards compat during migration**
|
||||
|
||||
Add a bridge that loads from SQLite on mount, falls back to localStorage.
|
||||
|
||||
- [ ] **Step 5: Verify build**
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add src/
|
||||
git commit -m "feat(tasks): migrate task store from localStorage to SQLite backend"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Frontend History Store Migration
|
||||
|
||||
**Files:**
|
||||
- Create: `src/lib/stores/history.svelte.js`
|
||||
- Modify: `src/lib/pages/HistoryPage.svelte`
|
||||
- Modify: `src/lib/pages/DictationPage.svelte`
|
||||
|
||||
- [ ] **Step 1: Create history.svelte.js**
|
||||
|
||||
```javascript
|
||||
import { invoke } from '@tauri-apps/api/core';
|
||||
|
||||
let transcripts = $state([]);
|
||||
|
||||
export async function loadHistory(limit = 100) { ... }
|
||||
export async function saveTranscript(transcript) { ... }
|
||||
export async function deleteTranscript(id) { ... }
|
||||
export async function searchTranscripts(query) { ... }
|
||||
export function getHistory() { return transcripts; }
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update HistoryPage.svelte**
|
||||
|
||||
Replace localStorage-based history with SQLite search. Wire FTS5 search to the search input.
|
||||
|
||||
- [ ] **Step 3: Update DictationPage.svelte**
|
||||
|
||||
After transcription completes, call `saveTranscript()` from the new store (in addition to existing behaviour).
|
||||
|
||||
- [ ] **Step 4: Verify build**
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/
|
||||
git commit -m "feat(history): migrate history to SQLite with FTS5 search"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 2B — Intelligence Layer
|
||||
|
||||
### Task 6: LLM Crate + llama-cpp-2 Integration
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/llm/Cargo.toml`
|
||||
- Create: `crates/llm/src/lib.rs`
|
||||
- Create: `crates/llm/src/inference.rs`
|
||||
- Create: `crates/llm/src/model_manager.rs`
|
||||
- Modify: `Cargo.toml` (workspace members)
|
||||
- Modify: `src-tauri/Cargo.toml` (add dependency)
|
||||
|
||||
**Note:** llama-cpp-2 requires CMake and a C++ compiler. On Windows this means MSVC build tools.
|
||||
|
||||
- [ ] **Step 1: Create crates/llm/Cargo.toml**
|
||||
|
||||
```toml
|
||||
[package]
|
||||
name = "kon-llm"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
description = "Local LLM inference via llama.cpp for Kon"
|
||||
|
||||
[dependencies]
|
||||
kon-core = { path = "../core" }
|
||||
llama-cpp-2 = { version = "0.1", features = ["vulkan"] }
|
||||
tokio = { version = "1", features = ["rt", "sync"] }
|
||||
reqwest = { version = "0.12", features = ["stream"] }
|
||||
futures-util = "0.3"
|
||||
serde = { version = "1", features = ["derive"] }
|
||||
serde_json = "1"
|
||||
log = "0.4"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Create lib.rs with LlmEngine struct**
|
||||
|
||||
```rust
|
||||
pub struct LlmEngine {
|
||||
model: Mutex<Option<LlamaModel>>,
|
||||
loaded_model_path: Mutex<Option<PathBuf>>,
|
||||
}
|
||||
|
||||
impl LlmEngine {
|
||||
pub fn new() -> Self { ... }
|
||||
pub fn load(&self, model_path: &Path) -> Result<()> { ... }
|
||||
pub fn is_loaded(&self) -> bool { ... }
|
||||
pub fn generate(&self, prompt: &str, max_tokens: u32) -> Result<String> { ... }
|
||||
pub fn generate_streaming(&self, prompt: &str, max_tokens: u32, callback: impl Fn(&str)) -> Result<String> { ... }
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Create model_manager.rs for GGUF downloads**
|
||||
|
||||
Reuse the pattern from crates/transcription/model_manager.rs — streaming download with progress callback, atomic rename.
|
||||
|
||||
Model catalog:
|
||||
```rust
|
||||
const LLM_MODELS: &[LlmModelEntry] = &[
|
||||
LlmModelEntry {
|
||||
id: "phi-4-mini-q4",
|
||||
display_name: "Phi-4 Mini (8GB RAM)",
|
||||
url: "https://huggingface.co/...",
|
||||
disk_size: Megabytes(2300),
|
||||
ram_required: Megabytes(4000),
|
||||
filename: "phi-4-mini-q4_k_m.gguf",
|
||||
},
|
||||
LlmModelEntry {
|
||||
id: "qwen3-7b-q4",
|
||||
display_name: "Qwen 3 7B (16GB RAM)",
|
||||
url: "https://huggingface.co/...",
|
||||
disk_size: Megabytes(4500),
|
||||
ram_required: Megabytes(8000),
|
||||
filename: "qwen3-7b-q4_k_m.gguf",
|
||||
},
|
||||
];
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Create inference.rs with async wrapper**
|
||||
|
||||
```rust
|
||||
pub async fn run_llm_inference(
|
||||
engine: Arc<LlmEngine>,
|
||||
prompt: String,
|
||||
max_tokens: u32,
|
||||
) -> Result<String> {
|
||||
tokio::task::spawn_blocking(move || {
|
||||
engine.generate(&prompt, max_tokens)
|
||||
}).await.map_err(|e| KonError::Other(e.to_string()))?
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Add workspace member and verify compilation**
|
||||
|
||||
```bash
|
||||
cargo check -p kon-llm
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/llm/ Cargo.toml
|
||||
git commit -m "feat(llm): add kon-llm crate with llama-cpp-2 inference engine"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 7: LLM Tauri Commands + Model Download UI
|
||||
|
||||
**Files:**
|
||||
- Create: `src-tauri/src/commands/llm.rs`
|
||||
- Modify: `src-tauri/src/commands/mod.rs`
|
||||
- Modify: `src-tauri/src/lib.rs`
|
||||
- Modify: `src/lib/pages/SettingsPage.svelte`
|
||||
- Modify: `src/lib/components/ModelDownloader.svelte`
|
||||
- Modify: `src/lib/pages/FirstRunPage.svelte`
|
||||
|
||||
- [ ] **Step 1: Create llm.rs with commands**
|
||||
|
||||
```rust
|
||||
#[tauri::command] async fn list_llm_models() -> Vec<LlmModelInfo>
|
||||
#[tauri::command] async fn download_llm_model(app, id) -> Result<(), String> // emits "llm-download-progress"
|
||||
#[tauri::command] async fn load_llm_model(state, id) -> Result<(), String>
|
||||
#[tauri::command] async fn check_llm_engine(state) -> bool
|
||||
#[tauri::command] async fn llm_generate(state, prompt, max_tokens) -> Result<String, String>
|
||||
#[tauri::command] async fn extract_tasks_llm(state, transcript_text) -> Result<Vec<TaskSuggestion>, String>
|
||||
#[tauri::command] async fn decompose_task(state, task_text) -> Result<Vec<MicroStep>, String>
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add LlmEngine to AppState**
|
||||
|
||||
```rust
|
||||
pub struct AppState {
|
||||
pub whisper_engine: Arc<LocalEngine>,
|
||||
pub parakeet_engine: Arc<LocalEngine>,
|
||||
pub llm_engine: Arc<LlmEngine>,
|
||||
pub db: SqlitePool,
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Register commands in lib.rs**
|
||||
|
||||
- [ ] **Step 4: Update ModelDownloader.svelte to support LLM models**
|
||||
|
||||
Add a `modelType` prop ("whisper" | "llm") and listen to appropriate download events.
|
||||
|
||||
- [ ] **Step 5: Add LLM model section to FirstRunPage.svelte**
|
||||
|
||||
After STT model download, offer optional LLM model download: "Download AI assistant for task extraction? (optional, {size})"
|
||||
|
||||
- [ ] **Step 6: Add LLM section to SettingsPage.svelte**
|
||||
|
||||
In the "AI Assistant" accordion: model selection, download button, status indicator.
|
||||
|
||||
- [ ] **Step 7: Verify build**
|
||||
|
||||
```bash
|
||||
cd src-tauri && cargo check && cd .. && npm run build
|
||||
```
|
||||
|
||||
- [ ] **Step 8: Commit**
|
||||
|
||||
```bash
|
||||
git add src-tauri/ src/ crates/llm/
|
||||
git commit -m "feat(llm): add LLM Tauri commands, model download UI, FirstRun integration"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Task Extraction — LLM + Rule-Based Fallback
|
||||
|
||||
**Files:**
|
||||
- Rewrite: `crates/ai-formatting/src/llm_client.rs` (replace placeholder)
|
||||
- Create: `crates/ai-formatting/src/task_extraction.rs`
|
||||
- Modify: `crates/ai-formatting/Cargo.toml`
|
||||
- Modify: `crates/ai-formatting/src/lib.rs`
|
||||
- Modify: `src-tauri/src/commands/llm.rs`
|
||||
- Modify: `src/lib/pages/DictationPage.svelte`
|
||||
|
||||
- [ ] **Step 1: Create task_extraction.rs**
|
||||
|
||||
```rust
|
||||
pub struct ExtractedTask {
|
||||
pub title: String,
|
||||
pub priority: String,
|
||||
pub project: Option<String>,
|
||||
}
|
||||
|
||||
const EXTRACTION_SYSTEM_PROMPT: &str = r#"Extract actionable tasks from the following voice transcription. Each task must start with a concrete verb. Return as JSON array of {"title": "...", "priority": "high|medium|low", "project": "..."}.
|
||||
Only extract genuine tasks — not observations or comments. If no tasks found, return empty array []."#;
|
||||
|
||||
pub fn extract_tasks_with_llm(engine: &LlmEngine, transcript: &str) -> Result<Vec<ExtractedTask>> { ... }
|
||||
pub fn extract_tasks_rule_based(transcript: &str) -> Vec<ExtractedTask> { ... }
|
||||
pub fn extract_tasks(engine: Option<&LlmEngine>, transcript: &str) -> Vec<ExtractedTask> { ... }
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Wire into extract_tasks_llm command**
|
||||
|
||||
The Tauri command tries LLM first, falls back to rule-based.
|
||||
|
||||
- [ ] **Step 3: Update DictationPage.svelte**
|
||||
|
||||
Replace the JS `extractTasks()` call with `invoke('extract_tasks_llm', { transcriptText })`.
|
||||
|
||||
- [ ] **Step 4: Verify build**
|
||||
|
||||
```bash
|
||||
cd src-tauri && cargo check && cd .. && npm run build
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/ai-formatting/ src-tauri/ src/
|
||||
git commit -m "feat(extraction): add LLM task extraction with rule-based fallback"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 9: Micro-Stepping
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/ai-formatting/src/micro_stepping.rs`
|
||||
- Create: `src/lib/components/MicroSteps.svelte`
|
||||
- Modify: `src/lib/components/WipTaskList.svelte`
|
||||
- Modify: `src-tauri/src/commands/llm.rs`
|
||||
|
||||
- [ ] **Step 1: Create micro_stepping.rs**
|
||||
|
||||
```rust
|
||||
const MICRO_STEP_PROMPT: &str = r#"Break this task into 3-7 micro-steps. Each step MUST start with a specific physical verb (e.g. 'Open', 'Type', 'Click', 'Pick up'). Each step must be completable in under 5 minutes. Never use abstract verbs like 'organise', 'plan', 'consider'. Return as JSON array of strings."#;
|
||||
|
||||
pub fn decompose_task(engine: &LlmEngine, task_text: &str) -> Result<Vec<String>> { ... }
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Wire into decompose_task Tauri command**
|
||||
|
||||
- [ ] **Step 3: Create MicroSteps.svelte**
|
||||
|
||||
```svelte
|
||||
<script>
|
||||
import { invoke } from '@tauri-apps/api/core';
|
||||
import { Play } from 'lucide-svelte';
|
||||
let { taskId, taskText } = $props();
|
||||
let steps = $state([]);
|
||||
let loading = $state(false);
|
||||
// ...
|
||||
</script>
|
||||
```
|
||||
|
||||
Shows expandable micro-steps below a task. Each step has a "Just Start" button that launches a 2min or 5min timer.
|
||||
|
||||
- [ ] **Step 4: Wire MicroSteps into WipTaskList**
|
||||
|
||||
Add expand/collapse per task that loads micro-steps on demand.
|
||||
|
||||
- [ ] **Step 5: Verify build**
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/ai-formatting/ src-tauri/ src/
|
||||
git commit -m "feat(microsteps): add LLM task decomposition with Just Start timer"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 10: Visual Timer Wiring + Notifications
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/lib/components/VisualTimer.svelte`
|
||||
- Create: `src/lib/components/TaskTimer.svelte`
|
||||
- Modify: `src-tauri/Cargo.toml` (add tauri-plugin-notification)
|
||||
- Modify: `src-tauri/src/lib.rs` (register notification plugin)
|
||||
- Modify: `src-tauri/tauri.conf.json` (add notification permission)
|
||||
|
||||
- [ ] **Step 1: Add tauri-plugin-notification**
|
||||
|
||||
```bash
|
||||
cd src-tauri && cargo add tauri-plugin-notification@2
|
||||
```
|
||||
|
||||
Update lib.rs: `.plugin(tauri_plugin_notification::init())`
|
||||
|
||||
Update tauri.conf.json capabilities.
|
||||
|
||||
- [ ] **Step 2: Create TaskTimer.svelte**
|
||||
|
||||
Wraps VisualTimer with countdown logic, persists timer state to SQLite, shows OS notification on complete:
|
||||
|
||||
```svelte
|
||||
<script>
|
||||
import VisualTimer from './VisualTimer.svelte';
|
||||
import { invoke } from '@tauri-apps/api/core';
|
||||
import { sendNotification } from '@tauri-apps/plugin-notification';
|
||||
// Timer countdown, pause/resume, persist state
|
||||
</script>
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Wire timer persistence**
|
||||
|
||||
On start: `invoke('save_timer_state', { taskId, totalSeconds, remainingSeconds })`
|
||||
On tick: Update remaining (debounced, every 5s)
|
||||
On complete: `invoke('clear_timer_state')` + notification
|
||||
On app restart: `invoke('get_timer_state')` → resume timer
|
||||
|
||||
- [ ] **Step 4: Respect reduce-motion preference**
|
||||
|
||||
When reduce motion is on, VisualTimer shows static fill state instead of animated ring.
|
||||
|
||||
- [ ] **Step 5: Verify build**
|
||||
|
||||
```bash
|
||||
cd src-tauri && cargo check && cd .. && npm run build
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add src-tauri/ src/
|
||||
git commit -m "feat(timer): wire VisualTimer to tasks with notifications and persistence"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 2C — Data & Polish
|
||||
|
||||
### Task 11: Export and Open Data
|
||||
|
||||
**Files:**
|
||||
- Create: `src/lib/utils/obsidianExport.js`
|
||||
- Modify: `src/lib/pages/DictationPage.svelte`
|
||||
- Modify: `src/lib/pages/HistoryPage.svelte`
|
||||
- Modify: `src/lib/pages/TasksPage.svelte`
|
||||
|
||||
- [ ] **Step 1: Create obsidianExport.js**
|
||||
|
||||
```javascript
|
||||
export function exportTranscriptToObsidian(transcript, segments, tasks) {
|
||||
const frontmatter = `---
|
||||
title: "${transcript.title || 'Voice Note'}"
|
||||
date: ${transcript.created_at}
|
||||
source: ${transcript.source}
|
||||
duration: ${transcript.duration}s
|
||||
engine: ${transcript.engine}
|
||||
tags: [kon, transcription]
|
||||
---\n\n`;
|
||||
// ... body with text + optional task list
|
||||
}
|
||||
|
||||
export function exportTasksToJSON(tasks) { ... }
|
||||
export function exportTasksToCSV(tasks) { ... }
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add "Export to Obsidian" button to HistoryPage**
|
||||
|
||||
Uses `@tauri-apps/plugin-dialog` to pick output directory, then writes markdown files.
|
||||
|
||||
- [ ] **Step 3: Add task export to TasksPage**
|
||||
|
||||
JSON and CSV export buttons.
|
||||
|
||||
- [ ] **Step 4: Verify build**
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/
|
||||
git commit -m "feat(export): add Obsidian export, task JSON/CSV export"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 12: First Run Polish
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/lib/pages/FirstRunPage.svelte`
|
||||
- Modify: `src/lib/stores/page.svelte.js`
|
||||
|
||||
- [ ] **Step 1: Add microphone permission request step**
|
||||
|
||||
Before model download, request mic permission via `navigator.mediaDevices.getUserMedia()`.
|
||||
|
||||
- [ ] **Step 2: Add test recording step**
|
||||
|
||||
After model loads, show a quick 5-second test recording: "Say something..." → display result → "You're ready!"
|
||||
|
||||
- [ ] **Step 3: Wire optional LLM download**
|
||||
|
||||
After STT model: "Want smarter task extraction? Download AI assistant ({size}, optional)"
|
||||
|
||||
- [ ] **Step 4: Time the flow — target under 90 seconds**
|
||||
|
||||
Add performance instrumentation to log total onboarding time.
|
||||
|
||||
- [ ] **Step 5: Verify build**
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add src/
|
||||
git commit -m "feat(firstrun): add mic permission, test recording, LLM download step"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 13: Settings Wiring + Global Hotkey Update
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/lib/pages/SettingsPage.svelte`
|
||||
- Modify: `src/lib/stores/page.svelte.js`
|
||||
- Modify: `src/routes/+layout.svelte`
|
||||
|
||||
- [ ] **Step 1: Change default hotkey to Ctrl+Shift+Space**
|
||||
|
||||
In `page.svelte.js`, change `globalHotkey: "Ctrl+Shift+R"` to `globalHotkey: "Ctrl+Shift+Space"`.
|
||||
|
||||
- [ ] **Step 2: Add microphone selection setting**
|
||||
|
||||
Use `navigator.mediaDevices.enumerateDevices()` to list audio input devices. Display as dropdown in Settings. Pass selected device ID to AudioContext.
|
||||
|
||||
- [ ] **Step 3: Wire export directory setting**
|
||||
|
||||
Use `@tauri-apps/plugin-dialog` for directory picker.
|
||||
|
||||
- [ ] **Step 4: Migrate remaining localStorage settings to preferences store**
|
||||
|
||||
The `settings` object in page.svelte.js currently uses localStorage. Add a `$effect` that syncs key settings to the SQLite-backed preferences store.
|
||||
|
||||
- [ ] **Step 5: Verify build**
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add src/
|
||||
git commit -m "feat(settings): wire mic selection, export directory, update default hotkey"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 14: Final Validation
|
||||
|
||||
- [ ] **Step 1: Full build check**
|
||||
|
||||
```bash
|
||||
npm run build && cd src-tauri && cargo check
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Keyboard navigation**
|
||||
|
||||
Tab through every page. Verify focus rings visible.
|
||||
|
||||
- [ ] **Step 3: Context restoration test**
|
||||
|
||||
Set non-default preferences → close app → relaunch. Verify state preserved.
|
||||
|
||||
- [ ] **Step 4: Reduce motion test**
|
||||
|
||||
Toggle reduce motion on → verify all animations stopped, timer shows static state.
|
||||
|
||||
- [ ] **Step 5: Commit any fixes**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "fix(validation): final validation pass corrections"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Phase | Tasks | Key Deliverable |
|
||||
|---|---|---|
|
||||
| 2A: Core Pipeline (1–5) | Schema migration, transcript persistence, task CRUD, FTS5 search, frontend store migration | Working voice → text → SQLite pipeline |
|
||||
| 2B: Intelligence (6–10) | LLM crate, model management, task extraction, micro-stepping, visual timer | AI-powered task decomposition with timer |
|
||||
| 2C: Polish (11–14) | Export, first run, settings, validation | Ship-ready for closed beta |
|
||||
|
||||
**Total:** 14 tasks. Schema first. Backend commands before frontend. LLM after core pipeline works. Polish last.
|
||||
|
||||
**Critical path:** Task 1 (schema) → Task 2-3 (commands) → Task 4-5 (frontend migration) → Task 6-7 (LLM) → everything else.
|
||||
|
||||
**Risk:** llama-cpp-2 compilation on Windows requires MSVC + CMake. If it fails, Tasks 6-9 scope down to rule-based extraction only (already works).
|
||||
Reference in New Issue
Block a user