feat(feedback): Phase 2 — HITL thumbs + correction capture with prompt-conditioning loop
Closes the human-in-the-loop gap from docs/brief/feature-set.md and Phase 2 of the 2026-04-23 feature-complete roadmap. Storage (kon-storage): - Migration v10 adds the `feedback` table: (target_type, target_id, rating, original_text, corrected_text, context_json, profile_id, created_at) with CHECK constraints on target_type and rating, plus indexes on (target_type, rating, created_at DESC) for prompt-time retrieval and (profile_id, target_type, created_at DESC) for per-profile scoping. - New public API: `FeedbackTargetType`, `RecordFeedbackParams`, `FeedbackRow`, `record_feedback`, `list_feedback_examples`. - Tests updated — the RB-02 rollback regression now discovers the real max version at runtime instead of hard-coding v10 for its poison migration. LLM (kon-llm): - `prompts::FeedbackExample` — local shape for few-shot exemplars so kon-llm stays independent of kon-storage. - `prompts::build_conditioned_system_prompt` — appends a "here is the style this user prefers" block to the base system prompt when examples are available; returns the base prompt unchanged when empty, so new users and early sessions see generic output. - `LlmEngine::decompose_task_with_feedback` and `LlmEngine::extract_tasks_with_feedback` thread examples through to the builder. The old one-arg variants are preserved and now call through with an empty slice. - 4 unit tests covering empty, empty-input-skip, correction-wins, and thumbs-up-only fallback. Tauri (src-tauri): - New commands::feedback module: `record_feedback`, `list_feedback_examples_cmd`. - `decompose_and_store` and `extract_tasks_from_transcript_cmd` now fetch the last 5 positive/neutral feedback rows for their target type and pass them through to the LLM, wiring the learning loop end-to-end. - Shared `to_llm_examples` helper parses the `context_json.input` field (where the recorder stashes the parent task text / transcript chunk) back into the exemplar shape. Frontend (MicroSteps.svelte): - Thumbs-up and thumbs-down buttons on every micro-step row. Hover-revealed; the vote recolours the icon; clicking again clears the local highlight (the row itself stays in the audit trail). - Pencil icon + double-click to edit step text. Save flows through update_task_cmd for persistence and records a correction feedback row with (original_text, corrected_text) — the highest-value training signal. - Parent task text is captured in context_json.input at record time so the prompt builder can reconstruct the (input, preferred-output) pair on subsequent decompositions. - Feedback capture is best-effort — a record_feedback failure never interrupts the primary action. What's deferred to a later phase: - Thumbs + corrections on extracted tasks (same pipeline, different surface — probably TasksPage after the AI-extraction path) - Thumbs on transcript cleanup output - Semantic retrieval over the feedback corpus (once there is enough data to justify embedding infrastructure; the storage shape is already ready for it)
This commit is contained in:
@@ -240,11 +240,30 @@ impl LlmEngine {
|
||||
}
|
||||
|
||||
pub fn decompose_task(&self, task_text: &str) -> Result<Vec<String>, EngineError> {
|
||||
self.decompose_task_with_feedback(task_text, &[])
|
||||
}
|
||||
|
||||
/// Same as `decompose_task` but allows callers to pass recent HITL
|
||||
/// feedback rows so the system prompt gets conditioned on the
|
||||
/// user's preferred decomposition style. The `examples` vec is
|
||||
/// rendered into a few-shot block appended to the base system
|
||||
/// prompt by `prompts::build_conditioned_system_prompt`.
|
||||
///
|
||||
/// Callers should pass most-recent-first; older examples still
|
||||
/// participate but weigh less because of their position in the
|
||||
/// prompt. Empty slice keeps behaviour identical to `decompose_task`.
|
||||
pub fn decompose_task_with_feedback(
|
||||
&self,
|
||||
task_text: &str,
|
||||
examples: &[prompts::FeedbackExample],
|
||||
) -> Result<Vec<String>, EngineError> {
|
||||
let model = self.loaded_model_arc()?;
|
||||
let system =
|
||||
prompts::build_conditioned_system_prompt(prompts::DECOMPOSE_TASK_SYSTEM, examples);
|
||||
let prompt = render_chat_prompt(
|
||||
&model,
|
||||
&[
|
||||
("system", prompts::DECOMPOSE_TASK_SYSTEM),
|
||||
("system", system.as_str()),
|
||||
("user", &format!("Task: {task_text}")),
|
||||
],
|
||||
)?;
|
||||
@@ -261,15 +280,27 @@ impl LlmEngine {
|
||||
}
|
||||
|
||||
pub fn extract_tasks(&self, transcript: &str) -> Result<Vec<String>, EngineError> {
|
||||
self.extract_tasks_with_feedback(transcript, &[])
|
||||
}
|
||||
|
||||
/// Feedback-conditioned variant of `extract_tasks`. See
|
||||
/// `decompose_task_with_feedback` for the `examples` semantics.
|
||||
pub fn extract_tasks_with_feedback(
|
||||
&self,
|
||||
transcript: &str,
|
||||
examples: &[prompts::FeedbackExample],
|
||||
) -> Result<Vec<String>, EngineError> {
|
||||
if transcript.trim().is_empty() {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
|
||||
let model = self.loaded_model_arc()?;
|
||||
let system =
|
||||
prompts::build_conditioned_system_prompt(prompts::EXTRACT_TASKS_SYSTEM, examples);
|
||||
let prompt = render_chat_prompt(
|
||||
&model,
|
||||
&[
|
||||
("system", prompts::EXTRACT_TASKS_SYSTEM),
|
||||
("system", system.as_str()),
|
||||
("user", &format!("Transcript:\n{transcript}")),
|
||||
],
|
||||
)?;
|
||||
|
||||
@@ -10,3 +10,113 @@ output a JSON array of action items the speaker committed to. Each item must \
|
||||
be a short imperative sentence. Omit observations, wishes, and background \
|
||||
context that are not explicit commitments. Output an empty array if there are \
|
||||
no action items.";
|
||||
|
||||
/// Compact representation of a human-in-the-loop feedback example used
|
||||
/// for few-shot prompt conditioning. Built by kon-storage and fed to the
|
||||
/// prompt builder below; we keep this struct local to the LLM crate so
|
||||
/// kon-llm does not depend on kon-storage.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct FeedbackExample {
|
||||
/// What the AI was given as input (e.g. the parent task text, or
|
||||
/// the transcript chunk). Kept verbatim.
|
||||
pub input: String,
|
||||
/// What the AI produced originally. `None` if the user only
|
||||
/// gave a thumbs-up without a prior edit (positive signal
|
||||
/// without a paired correction).
|
||||
pub original_output: Option<String>,
|
||||
/// What the user changed it to. `None` for thumbs-only rows.
|
||||
/// This is the highest-value signal — when present, inject it
|
||||
/// as the "good" output in the few-shot example.
|
||||
pub corrected_output: Option<String>,
|
||||
}
|
||||
|
||||
/// Render a feedback example into the exemplar block used in prompt
|
||||
/// conditioning. Returns `None` for rows that carry no usable pairing
|
||||
/// (e.g. a thumbs-up with no input context).
|
||||
fn render_feedback_exemplar(ex: &FeedbackExample) -> Option<String> {
|
||||
if ex.input.trim().is_empty() {
|
||||
return None;
|
||||
}
|
||||
let good = ex
|
||||
.corrected_output
|
||||
.as_deref()
|
||||
.or(ex.original_output.as_deref())?;
|
||||
let good = good.trim();
|
||||
if good.is_empty() {
|
||||
return None;
|
||||
}
|
||||
Some(format!("Input: {}\nGood output: {}", ex.input.trim(), good))
|
||||
}
|
||||
|
||||
/// Build a system prompt that combines the base task system prompt
|
||||
/// with a few-shot block assembled from recent HITL examples. If no
|
||||
/// usable examples are available, returns the base prompt unchanged
|
||||
/// so early users see the generic behaviour and the LLM is not
|
||||
/// confused by an empty exemplar section.
|
||||
///
|
||||
/// The exemplars are ordered most-recent-first (caller's order is
|
||||
/// preserved) so the LLM weights the user's current style over
|
||||
/// earlier noise, mirroring what a human reviewer would do.
|
||||
pub fn build_conditioned_system_prompt(base: &str, examples: &[FeedbackExample]) -> String {
|
||||
let rendered: Vec<String> = examples
|
||||
.iter()
|
||||
.filter_map(render_feedback_exemplar)
|
||||
.collect();
|
||||
if rendered.is_empty() {
|
||||
return base.to_string();
|
||||
}
|
||||
let block = rendered
|
||||
.iter()
|
||||
.map(|s| format!("- {s}"))
|
||||
.collect::<Vec<_>>()
|
||||
.join("\n");
|
||||
format!(
|
||||
"{base}\n\nHere are examples of the style this user prefers, in the \
|
||||
user's own words. Match this style closely when producing your output:\n{block}"
|
||||
)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn builds_plain_prompt_when_no_examples() {
|
||||
let out = build_conditioned_system_prompt(DECOMPOSE_TASK_SYSTEM, &[]);
|
||||
assert_eq!(out, DECOMPOSE_TASK_SYSTEM);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn skips_empty_input_examples() {
|
||||
let examples = vec![FeedbackExample {
|
||||
input: String::new(),
|
||||
original_output: None,
|
||||
corrected_output: Some("ignored".into()),
|
||||
}];
|
||||
let out = build_conditioned_system_prompt(DECOMPOSE_TASK_SYSTEM, &examples);
|
||||
assert_eq!(out, DECOMPOSE_TASK_SYSTEM);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn prefers_corrected_over_original() {
|
||||
let examples = vec![FeedbackExample {
|
||||
input: "Clean room".into(),
|
||||
original_output: Some("Organise your bedroom".into()),
|
||||
corrected_output: Some("Pick up one shirt from the floor".into()),
|
||||
}];
|
||||
let out = build_conditioned_system_prompt(DECOMPOSE_TASK_SYSTEM, &examples);
|
||||
assert!(out.contains("Pick up one shirt from the floor"));
|
||||
assert!(!out.contains("Organise your bedroom"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn falls_back_to_original_when_no_correction() {
|
||||
let examples = vec![FeedbackExample {
|
||||
input: "Write report".into(),
|
||||
original_output: Some("Open a blank document".into()),
|
||||
corrected_output: None,
|
||||
}];
|
||||
let out = build_conditioned_system_prompt(DECOMPOSE_TASK_SYSTEM, &examples);
|
||||
assert!(out.contains("Open a blank document"));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -889,6 +889,151 @@ pub async fn list_recent_errors(pool: &SqlitePool, limit: i64) -> Result<Vec<Err
|
||||
.collect())
|
||||
}
|
||||
|
||||
// --- Feedback (HITL) -------------------------------------------------------
|
||||
//
|
||||
// Phase 2 of the feature-complete roadmap: capture thumbs + corrections on
|
||||
// AI-generated output so the prompt builder can inject recent examples as
|
||||
// few-shot exemplars. Storage-only here; the prompt-conditioning logic lives
|
||||
// in kon-llm. Retrieval returns the most recent rows, narrowed to the
|
||||
// active profile when provided so feedback does not cross profiles.
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum FeedbackTargetType {
|
||||
MicroStep,
|
||||
TaskExtraction,
|
||||
Cleanup,
|
||||
}
|
||||
|
||||
impl FeedbackTargetType {
|
||||
pub fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
FeedbackTargetType::MicroStep => "microstep",
|
||||
FeedbackTargetType::TaskExtraction => "task_extraction",
|
||||
FeedbackTargetType::Cleanup => "cleanup",
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse the database `target_type` string back into the enum.
|
||||
/// Named `parse` rather than `from_str` so it does not collide with
|
||||
/// the `std::str::FromStr` trait — the trait is overkill here
|
||||
/// because callers never want a `FromStr::Err` and already know the
|
||||
/// set of valid values at the call site.
|
||||
pub fn parse(s: &str) -> Option<Self> {
|
||||
match s {
|
||||
"microstep" => Some(FeedbackTargetType::MicroStep),
|
||||
"task_extraction" => Some(FeedbackTargetType::TaskExtraction),
|
||||
"cleanup" => Some(FeedbackTargetType::Cleanup),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct RecordFeedbackParams {
|
||||
pub target_type: FeedbackTargetType,
|
||||
pub target_id: Option<String>,
|
||||
/// -1 = thumbs down, 0 = correction (neutral), +1 = thumbs up.
|
||||
pub rating: i8,
|
||||
pub original_text: Option<String>,
|
||||
pub corrected_text: Option<String>,
|
||||
pub context_json: Option<String>,
|
||||
pub profile_id: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct FeedbackRow {
|
||||
pub id: i64,
|
||||
pub target_type: String,
|
||||
pub target_id: Option<String>,
|
||||
pub rating: i64,
|
||||
pub original_text: Option<String>,
|
||||
pub corrected_text: Option<String>,
|
||||
pub context_json: Option<String>,
|
||||
pub profile_id: String,
|
||||
pub created_at: String,
|
||||
}
|
||||
|
||||
pub async fn record_feedback(pool: &SqlitePool, params: RecordFeedbackParams) -> Result<i64> {
|
||||
if !matches!(params.rating, -1..=1) {
|
||||
return Err(KonError::StorageError(format!(
|
||||
"invalid feedback rating {} (must be -1, 0, or 1)",
|
||||
params.rating
|
||||
)));
|
||||
}
|
||||
let profile_id = params
|
||||
.profile_id
|
||||
.unwrap_or_else(|| crate::DEFAULT_PROFILE_ID.to_string());
|
||||
let row = sqlx::query(
|
||||
"INSERT INTO feedback (
|
||||
target_type, target_id, rating,
|
||||
original_text, corrected_text, context_json, profile_id
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
RETURNING id",
|
||||
)
|
||||
.bind(params.target_type.as_str())
|
||||
.bind(params.target_id)
|
||||
.bind(params.rating as i64)
|
||||
.bind(params.original_text)
|
||||
.bind(params.corrected_text)
|
||||
.bind(params.context_json)
|
||||
.bind(profile_id)
|
||||
.fetch_one(pool)
|
||||
.await
|
||||
.map_err(|e| KonError::StorageError(format!("record_feedback failed: {e}")))?;
|
||||
Ok(row.get::<i64, _>("id"))
|
||||
}
|
||||
|
||||
/// Fetch the most recent feedback rows for a given target type, scoped to
|
||||
/// the active profile. Used by the prompt builder to gather few-shot
|
||||
/// exemplars. Orders by `created_at DESC` so the most recent corrections
|
||||
/// outweigh older ones — the user's style drifts, and we want the LLM
|
||||
/// to track the current preference.
|
||||
///
|
||||
/// `min_rating` filters out thumbs-down examples when the caller only
|
||||
/// wants positive reinforcement; pass `-1` to include everything.
|
||||
pub async fn list_feedback_examples(
|
||||
pool: &SqlitePool,
|
||||
target_type: FeedbackTargetType,
|
||||
limit: i64,
|
||||
min_rating: i8,
|
||||
profile_id: Option<&str>,
|
||||
) -> Result<Vec<FeedbackRow>> {
|
||||
let pid = profile_id.unwrap_or(crate::DEFAULT_PROFILE_ID);
|
||||
let rows = sqlx::query(
|
||||
"SELECT id, target_type, target_id, rating,
|
||||
original_text, corrected_text, context_json,
|
||||
profile_id, created_at
|
||||
FROM feedback
|
||||
WHERE target_type = ?
|
||||
AND profile_id = ?
|
||||
AND rating >= ?
|
||||
ORDER BY created_at DESC
|
||||
LIMIT ?",
|
||||
)
|
||||
.bind(target_type.as_str())
|
||||
.bind(pid)
|
||||
.bind(min_rating as i64)
|
||||
.bind(limit)
|
||||
.fetch_all(pool)
|
||||
.await
|
||||
.map_err(|e| KonError::StorageError(format!("list_feedback_examples failed: {e}")))?;
|
||||
|
||||
Ok(rows
|
||||
.into_iter()
|
||||
.map(|r| FeedbackRow {
|
||||
id: r.get("id"),
|
||||
target_type: r.get("target_type"),
|
||||
target_id: r.get("target_id"),
|
||||
rating: r.get("rating"),
|
||||
original_text: r.get("original_text"),
|
||||
corrected_text: r.get("corrected_text"),
|
||||
context_json: r.get("context_json"),
|
||||
profile_id: r.get("profile_id"),
|
||||
created_at: r.get("created_at"),
|
||||
})
|
||||
.collect())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
@@ -10,10 +10,11 @@ pub use database::{
|
||||
add_profile_term, complete_subtask_and_check_parent, complete_task, count_transcripts,
|
||||
create_profile, delete_profile, delete_profile_term, delete_task, delete_transcript,
|
||||
get_profile, get_setting, get_task_by_id, get_transcript, init, insert_subtask, insert_task,
|
||||
insert_transcript, list_profile_terms, list_profiles, list_recent_errors, list_subtasks,
|
||||
list_tasks, list_transcripts, list_transcripts_paged, log_error, search_transcripts,
|
||||
set_setting, uncomplete_task, update_profile, update_task, update_transcript,
|
||||
update_transcript_meta, ErrorLogRow, InsertTranscriptParams, ProfileRow, ProfileTermRow,
|
||||
insert_transcript, list_feedback_examples, list_profile_terms, list_profiles,
|
||||
list_recent_errors, list_subtasks, list_tasks, list_transcripts, list_transcripts_paged,
|
||||
log_error, record_feedback, search_transcripts, set_setting, uncomplete_task, update_profile,
|
||||
update_task, update_transcript, update_transcript_meta, ErrorLogRow, FeedbackRow,
|
||||
FeedbackTargetType, InsertTranscriptParams, ProfileRow, ProfileTermRow, RecordFeedbackParams,
|
||||
TaskRow, TranscriptRow,
|
||||
};
|
||||
pub use file_storage::{app_data_dir, crashes_dir, database_path, logs_dir, recordings_dir};
|
||||
|
||||
@@ -334,6 +334,49 @@ const MIGRATIONS: &[(i64, &str, &str)] = &[
|
||||
FROM transcripts;
|
||||
"#,
|
||||
),
|
||||
(
|
||||
10,
|
||||
"feedback: HITL thumbs + correction capture",
|
||||
r#"
|
||||
-- Feedback rows capture human-in-the-loop signal on AI-generated
|
||||
-- output. Two flavours bundled into one table:
|
||||
-- - thumbs (rating = -1 | +1, original_text optional, corrected_text NULL)
|
||||
-- - correction (rating defaults to +1, original_text + corrected_text present)
|
||||
--
|
||||
-- `target_type` names the producing surface:
|
||||
-- 'microstep' — subtask decomposition from DECOMPOSE_TASK_SYSTEM
|
||||
-- 'task_extraction' — tasks lifted from a transcript (EXTRACT_TASKS_SYSTEM)
|
||||
-- 'cleanup' — transcript cleanup output
|
||||
--
|
||||
-- `target_id` is the surface-specific identifier where one exists
|
||||
-- (subtask id, task id, transcript id). NULL is allowed because
|
||||
-- not every feedback event has a stable target id yet.
|
||||
--
|
||||
-- `context_json` carries the input the AI was conditioned on
|
||||
-- (parent task text, transcript chunk, etc.) so future prompt
|
||||
-- builders can reconstruct the original I/O pair for few-shot
|
||||
-- injection or semantic retrieval.
|
||||
CREATE TABLE feedback (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
target_type TEXT NOT NULL
|
||||
CHECK (target_type IN ('microstep', 'task_extraction', 'cleanup')),
|
||||
target_id TEXT,
|
||||
rating INTEGER NOT NULL
|
||||
CHECK (rating IN (-1, 0, 1)),
|
||||
original_text TEXT,
|
||||
corrected_text TEXT,
|
||||
context_json TEXT,
|
||||
profile_id TEXT NOT NULL DEFAULT '00000000-0000-0000-0000-000000000001'
|
||||
REFERENCES profiles(id) ON DELETE RESTRICT,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
CREATE INDEX idx_feedback_target_type_rating
|
||||
ON feedback(target_type, rating, created_at DESC);
|
||||
CREATE INDEX idx_feedback_profile
|
||||
ON feedback(profile_id, target_type, created_at DESC);
|
||||
"#,
|
||||
),
|
||||
];
|
||||
|
||||
/// Split SQL into individual statements, respecting BEGIN...END trigger blocks.
|
||||
@@ -483,7 +526,7 @@ mod tests {
|
||||
.fetch_one(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(count, 9);
|
||||
assert_eq!(count, 10);
|
||||
|
||||
sqlx::query("INSERT INTO settings (key, value) VALUES ('test', 'value')")
|
||||
.execute(&pool)
|
||||
@@ -502,7 +545,7 @@ mod tests {
|
||||
.fetch_one(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(count, 9);
|
||||
assert_eq!(count, 10);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
@@ -859,8 +902,11 @@ mod tests {
|
||||
// The poisoned migration below first creates `poison_marker`
|
||||
// (syntactically valid, would succeed against any SQLite) and then
|
||||
// runs a guaranteed-invalid function call. Under the new atomic
|
||||
// implementation, neither `poison_marker` nor the v9 row should
|
||||
// implementation, neither `poison_marker` nor the poison row should
|
||||
// survive the failed call.
|
||||
//
|
||||
// Version number must sit above the real MIGRATIONS max so the
|
||||
// baseline migrate cleanly finishes first.
|
||||
#[tokio::test]
|
||||
async fn multi_statement_migration_rolls_back_on_failure() {
|
||||
let pool = SqlitePoolOptions::new()
|
||||
@@ -871,8 +917,18 @@ mod tests {
|
||||
|
||||
run_migrations(&pool).await.expect("baseline migrate");
|
||||
|
||||
const POISON: &[(i64, &str, &str)] = &[(
|
||||
10,
|
||||
// Discover the real max version so the poison migration is
|
||||
// always exactly one past the end of MIGRATIONS, regardless of
|
||||
// how many real migrations we add in future.
|
||||
let real_max: i64 =
|
||||
sqlx::query_scalar("SELECT COALESCE(MAX(version), 0) FROM schema_version")
|
||||
.fetch_one(&pool)
|
||||
.await
|
||||
.expect("read schema_version");
|
||||
let poison_version = real_max + 1;
|
||||
|
||||
let poison: &[(i64, &str, &str)] = &[(
|
||||
poison_version,
|
||||
"rb-02 atomicity poison",
|
||||
r#"
|
||||
CREATE TABLE poison_marker (id INTEGER PRIMARY KEY);
|
||||
@@ -880,7 +936,7 @@ mod tests {
|
||||
"#,
|
||||
)];
|
||||
|
||||
let result = run_migrations_slice(&pool, POISON).await;
|
||||
let result = run_migrations_slice(&pool, poison).await;
|
||||
assert!(
|
||||
result.is_err(),
|
||||
"poisoned migration must return Err, got: {result:?}"
|
||||
@@ -896,14 +952,14 @@ mod tests {
|
||||
"poison_marker must not exist; got: {marker:?}"
|
||||
);
|
||||
|
||||
// `schema_version` must not include v10 — version insert is part
|
||||
// of the same transaction that rolled back.
|
||||
// `schema_version` must not include the poison version — version
|
||||
// insert is part of the same transaction that rolled back.
|
||||
let max: i64 = sqlx::query_scalar("SELECT COALESCE(MAX(version), 0) FROM schema_version")
|
||||
.fetch_one(&pool)
|
||||
.await
|
||||
.expect("read schema_version");
|
||||
assert_eq!(
|
||||
max, 9,
|
||||
max, real_max,
|
||||
"schema_version must not advance past the failed migration"
|
||||
);
|
||||
}
|
||||
|
||||
110
src-tauri/src/commands/feedback.rs
Normal file
110
src-tauri/src/commands/feedback.rs
Normal file
@@ -0,0 +1,110 @@
|
||||
// Tauri commands for human-in-the-loop feedback capture and retrieval.
|
||||
// Phase 2 of the feature-complete roadmap: thumbs + correction capture
|
||||
// on AI-generated output feeds a few-shot loop that conditions future
|
||||
// prompts on the user's preferred style.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use kon_storage::{
|
||||
list_feedback_examples as db_list_feedback_examples, record_feedback as db_record_feedback,
|
||||
FeedbackRow, FeedbackTargetType, RecordFeedbackParams,
|
||||
};
|
||||
|
||||
use crate::AppState;
|
||||
|
||||
#[derive(Debug, Clone, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct RecordFeedbackInput {
|
||||
/// One of "microstep", "task_extraction", "cleanup".
|
||||
pub target_type: String,
|
||||
/// Optional surface-specific id (subtask id, task id, transcript id).
|
||||
#[serde(default)]
|
||||
pub target_id: Option<String>,
|
||||
/// -1 = thumbs down, 0 = correction (neutral), +1 = thumbs up.
|
||||
pub rating: i8,
|
||||
#[serde(default)]
|
||||
pub original_text: Option<String>,
|
||||
#[serde(default)]
|
||||
pub corrected_text: Option<String>,
|
||||
/// Freeform JSON context: e.g. the parent task text, the transcript
|
||||
/// chunk the AI was given, etc. Used later by the prompt builder
|
||||
/// to reconstruct the (input, preferred-output) pair.
|
||||
#[serde(default)]
|
||||
pub context_json: Option<String>,
|
||||
#[serde(default)]
|
||||
pub profile_id: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct FeedbackDto {
|
||||
pub id: i64,
|
||||
pub target_type: String,
|
||||
pub target_id: Option<String>,
|
||||
pub rating: i64,
|
||||
pub original_text: Option<String>,
|
||||
pub corrected_text: Option<String>,
|
||||
pub context_json: Option<String>,
|
||||
pub profile_id: String,
|
||||
pub created_at: String,
|
||||
}
|
||||
|
||||
impl From<FeedbackRow> for FeedbackDto {
|
||||
fn from(r: FeedbackRow) -> Self {
|
||||
Self {
|
||||
id: r.id,
|
||||
target_type: r.target_type,
|
||||
target_id: r.target_id,
|
||||
rating: r.rating,
|
||||
original_text: r.original_text,
|
||||
corrected_text: r.corrected_text,
|
||||
context_json: r.context_json,
|
||||
profile_id: r.profile_id,
|
||||
created_at: r.created_at,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn parse_target_type(raw: &str) -> Result<FeedbackTargetType, String> {
|
||||
FeedbackTargetType::parse(raw).ok_or_else(|| format!("unknown feedback target_type: {raw}"))
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
pub async fn record_feedback(
|
||||
state: tauri::State<'_, AppState>,
|
||||
input: RecordFeedbackInput,
|
||||
) -> Result<i64, String> {
|
||||
let target_type = parse_target_type(&input.target_type)?;
|
||||
db_record_feedback(
|
||||
&state.db,
|
||||
RecordFeedbackParams {
|
||||
target_type,
|
||||
target_id: input.target_id,
|
||||
rating: input.rating,
|
||||
original_text: input.original_text,
|
||||
corrected_text: input.corrected_text,
|
||||
context_json: input.context_json,
|
||||
profile_id: input.profile_id,
|
||||
},
|
||||
)
|
||||
.await
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
pub async fn list_feedback_examples_cmd(
|
||||
state: tauri::State<'_, AppState>,
|
||||
target_type: String,
|
||||
limit: Option<i64>,
|
||||
min_rating: Option<i8>,
|
||||
profile_id: Option<String>,
|
||||
) -> Result<Vec<FeedbackDto>, String> {
|
||||
let target = parse_target_type(&target_type)?;
|
||||
let limit = limit.unwrap_or(8).clamp(1, 64);
|
||||
let min_rating = min_rating.unwrap_or(0).clamp(-1, 1);
|
||||
let rows =
|
||||
db_list_feedback_examples(&state.db, target, limit, min_rating, profile_id.as_deref())
|
||||
.await
|
||||
.map_err(|e| e.to_string())?;
|
||||
Ok(rows.into_iter().map(FeedbackDto::from).collect())
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
pub mod audio;
|
||||
pub mod clipboard;
|
||||
pub mod diagnostics;
|
||||
pub mod feedback;
|
||||
pub mod hardware;
|
||||
pub mod hotkey;
|
||||
pub mod live;
|
||||
|
||||
@@ -6,12 +6,14 @@ use serde::{Deserialize, Serialize};
|
||||
|
||||
use uuid::Uuid;
|
||||
|
||||
use kon_llm::prompts::FeedbackExample as LlmFeedbackExample;
|
||||
use kon_storage::{
|
||||
complete_subtask_and_check_parent as db_complete_subtask, complete_task as db_complete_task,
|
||||
delete_task as db_delete_task, get_task_by_id as db_get_task,
|
||||
insert_subtask as db_insert_subtask, insert_task as db_insert_task,
|
||||
list_subtasks as db_list_subtasks, list_tasks as db_list_tasks,
|
||||
uncomplete_task as db_uncomplete_task, update_task as db_update_task, TaskRow,
|
||||
list_feedback_examples as db_list_feedback_examples, list_subtasks as db_list_subtasks,
|
||||
list_tasks as db_list_tasks, uncomplete_task as db_uncomplete_task,
|
||||
update_task as db_update_task, FeedbackRow, FeedbackTargetType, TaskRow,
|
||||
};
|
||||
|
||||
use crate::AppState;
|
||||
@@ -166,6 +168,34 @@ pub async fn uncomplete_task_cmd(
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
/// Convert HITL feedback rows fetched from storage into the few-shot
|
||||
/// exemplar shape the LLM crate consumes. We reconstruct the `input`
|
||||
/// (parent task text, transcript chunk) from `context_json` where the
|
||||
/// recorder has stored it. Rows without usable input are dropped —
|
||||
/// the prompt builder filters them too, but doing it here keeps the
|
||||
/// exemplar list tight and the prompt budget predictable.
|
||||
fn to_llm_examples(rows: Vec<FeedbackRow>) -> Vec<LlmFeedbackExample> {
|
||||
rows.into_iter()
|
||||
.filter_map(|r| {
|
||||
let ctx: serde_json::Value =
|
||||
serde_json::from_str(r.context_json.as_deref().unwrap_or("{}")).ok()?;
|
||||
let input = ctx
|
||||
.get("input")
|
||||
.and_then(|v| v.as_str())
|
||||
.map(str::to_string)
|
||||
.unwrap_or_default();
|
||||
if input.trim().is_empty() {
|
||||
return None;
|
||||
}
|
||||
Some(LlmFeedbackExample {
|
||||
input,
|
||||
original_output: r.original_text,
|
||||
corrected_output: r.corrected_text,
|
||||
})
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
pub async fn decompose_and_store(
|
||||
state: tauri::State<'_, AppState>,
|
||||
@@ -176,12 +206,23 @@ pub async fn decompose_and_store(
|
||||
.map_err(|e| e.to_string())?
|
||||
.ok_or_else(|| format!("Task {parent_task_id} not found"))?;
|
||||
|
||||
// Pull recent micro-step feedback so the system prompt gets
|
||||
// conditioned on the user's preferred decomposition style. We
|
||||
// cap at 5 examples to keep the prompt under budget regardless
|
||||
// of how much feedback has been captured.
|
||||
let examples = db_list_feedback_examples(&state.db, FeedbackTargetType::MicroStep, 5, 0, None)
|
||||
.await
|
||||
.map(to_llm_examples)
|
||||
.unwrap_or_default();
|
||||
|
||||
let engine = state.llm_engine.clone();
|
||||
let parent_text = parent.text.clone();
|
||||
let steps = tokio::task::spawn_blocking(move || engine.decompose_task(&parent_text))
|
||||
.await
|
||||
.map_err(|e| e.to_string())?
|
||||
.map_err(|e| e.to_string())?;
|
||||
let steps = tokio::task::spawn_blocking(move || {
|
||||
engine.decompose_task_with_feedback(&parent_text, &examples)
|
||||
})
|
||||
.await
|
||||
.map_err(|e| e.to_string())?
|
||||
.map_err(|e| e.to_string())?;
|
||||
|
||||
let mut created = Vec::new();
|
||||
for text in steps {
|
||||
@@ -205,8 +246,14 @@ pub async fn extract_tasks_from_transcript_cmd(
|
||||
state: tauri::State<'_, AppState>,
|
||||
transcript: String,
|
||||
) -> Result<Vec<String>, String> {
|
||||
let examples =
|
||||
db_list_feedback_examples(&state.db, FeedbackTargetType::TaskExtraction, 5, 0, None)
|
||||
.await
|
||||
.map(to_llm_examples)
|
||||
.unwrap_or_default();
|
||||
|
||||
let engine = state.llm_engine.clone();
|
||||
tokio::task::spawn_blocking(move || engine.extract_tasks(&transcript))
|
||||
tokio::task::spawn_blocking(move || engine.extract_tasks_with_feedback(&transcript, &examples))
|
||||
.await
|
||||
.map_err(|e| e.to_string())?
|
||||
.map_err(|e| e.to_string())
|
||||
|
||||
@@ -288,6 +288,9 @@ pub fn run() {
|
||||
commands::tasks::extract_tasks_from_transcript_cmd,
|
||||
commands::tasks::list_subtasks_cmd,
|
||||
commands::tasks::complete_subtask_cmd,
|
||||
// HITL feedback (Phase 2 roadmap)
|
||||
commands::feedback::record_feedback,
|
||||
commands::feedback::list_feedback_examples_cmd,
|
||||
// Profiles + profile terms (canonical SQLite-backed profile CRUD) — Task 12
|
||||
commands::profiles::list_profiles_cmd,
|
||||
commands::profiles::get_profile_cmd,
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
<script lang="ts">
|
||||
import { invoke } from '@tauri-apps/api/core';
|
||||
import { ListTree, Check, Timer, Loader2 } from 'lucide-svelte';
|
||||
import { ListTree, Check, Timer, Loader2, ThumbsUp, ThumbsDown, Pencil } from 'lucide-svelte';
|
||||
|
||||
let { parentTaskId, reduceMotion = false } = $props();
|
||||
let { parentTaskId, parentTaskText = '', reduceMotion = false } = $props();
|
||||
|
||||
interface Subtask {
|
||||
id: string;
|
||||
@@ -15,6 +15,15 @@
|
||||
let error = $state('');
|
||||
let decomposing = $state(false);
|
||||
|
||||
// Per-step UI state. Keyed by subtask id so we never lose state when
|
||||
// the list reorders. Values:
|
||||
// rating[id] — 1 | -1 — the thumbs vote the user gave this session
|
||||
// editing[id] — true while the user is editing the step text
|
||||
// draft[id] — the in-flight edit value before save
|
||||
let rating = $state<Record<string, 1 | -1 | undefined>>({});
|
||||
let editing = $state<Record<string, boolean>>({});
|
||||
let draft = $state<Record<string, string>>({});
|
||||
|
||||
async function loadSubtasks() {
|
||||
loading = true;
|
||||
error = '';
|
||||
@@ -53,6 +62,88 @@
|
||||
}));
|
||||
}
|
||||
|
||||
// --- HITL feedback --------------------------------------------------------
|
||||
//
|
||||
// All three paths (thumbs up, thumbs down, correction-via-edit) route
|
||||
// into the same `record_feedback` command. The parent task text is the
|
||||
// "input" the AI was given, so it travels in context_json so the prompt
|
||||
// builder can reconstruct the (input, good-output) pair.
|
||||
|
||||
function feedbackContextJson() {
|
||||
return JSON.stringify({ input: parentTaskText ?? '' });
|
||||
}
|
||||
|
||||
async function recordThumb(step: Subtask, ratingValue: 1 | -1) {
|
||||
// Toggle: if the user already voted the same way, clear it (record
|
||||
// rating 0 means correction, not a thumb-off — we just skip the
|
||||
// re-record and drop the local highlight). Unvoting isn't stored;
|
||||
// the audit trail stays immutable.
|
||||
if (rating[step.id] === ratingValue) {
|
||||
const next = { ...rating };
|
||||
delete next[step.id];
|
||||
rating = next;
|
||||
return;
|
||||
}
|
||||
rating = { ...rating, [step.id]: ratingValue };
|
||||
try {
|
||||
await invoke('record_feedback', {
|
||||
input: {
|
||||
targetType: 'microstep',
|
||||
targetId: step.id,
|
||||
rating: ratingValue,
|
||||
originalText: step.text,
|
||||
correctedText: null,
|
||||
contextJson: feedbackContextJson(),
|
||||
},
|
||||
});
|
||||
} catch (_) { /* feedback capture is best-effort, never fatal */ }
|
||||
}
|
||||
|
||||
function startEdit(step: Subtask) {
|
||||
editing = { ...editing, [step.id]: true };
|
||||
draft = { ...draft, [step.id]: step.text };
|
||||
}
|
||||
|
||||
function cancelEdit(stepId: string) {
|
||||
const nextE = { ...editing }; delete nextE[stepId]; editing = nextE;
|
||||
const nextD = { ...draft }; delete nextD[stepId]; draft = nextD;
|
||||
}
|
||||
|
||||
async function saveEdit(step: Subtask) {
|
||||
const next = (draft[step.id] ?? '').trim();
|
||||
cancelEdit(step.id);
|
||||
if (!next || next === step.text) return;
|
||||
const original = step.text;
|
||||
// Update in-memory first so the UI is snappy; roll back if the
|
||||
// persistence call fails so we never show stale-but-different text.
|
||||
const idx = subtasks.findIndex(s => s.id === step.id);
|
||||
if (idx >= 0) subtasks[idx] = { ...subtasks[idx], text: next };
|
||||
try {
|
||||
await invoke('update_task_cmd', {
|
||||
id: step.id,
|
||||
patch: { text: next },
|
||||
});
|
||||
// Record correction as the highest-value feedback signal.
|
||||
await invoke('record_feedback', {
|
||||
input: {
|
||||
targetType: 'microstep',
|
||||
targetId: step.id,
|
||||
rating: 0,
|
||||
originalText: original,
|
||||
correctedText: next,
|
||||
contextJson: feedbackContextJson(),
|
||||
},
|
||||
}).catch(() => {});
|
||||
} catch (_) {
|
||||
if (idx >= 0) subtasks[idx] = { ...subtasks[idx], text: original };
|
||||
}
|
||||
}
|
||||
|
||||
function handleEditKeydown(evt: KeyboardEvent, step: Subtask) {
|
||||
if (evt.key === 'Enter') { evt.preventDefault(); saveEdit(step); }
|
||||
else if (evt.key === 'Escape') { evt.preventDefault(); cancelEdit(step.id); }
|
||||
}
|
||||
|
||||
$effect(() => {
|
||||
if (parentTaskId) loadSubtasks();
|
||||
});
|
||||
@@ -97,10 +188,64 @@
|
||||
<Check size={9} aria-hidden="true" />
|
||||
{/if}
|
||||
</button>
|
||||
<span class="text-[12px] flex-1 min-w-0 {step.done ? 'line-through text-text-tertiary' : 'text-text-secondary'} truncate">
|
||||
{step.text}
|
||||
</span>
|
||||
{#if !step.done}
|
||||
|
||||
{#if editing[step.id]}
|
||||
<!-- svelte-ignore a11y_autofocus — deliberate: inline edit
|
||||
is user-initiated and focus must land on the input to
|
||||
match the UX pattern users expect from any task app. -->
|
||||
<input
|
||||
type="text"
|
||||
bind:value={draft[step.id]}
|
||||
onkeydown={(e) => handleEditKeydown(e, step)}
|
||||
onblur={() => saveEdit(step)}
|
||||
class="text-[12px] flex-1 min-w-0 bg-bg-input border border-accent rounded px-1.5 py-0.5 text-text focus:outline-none"
|
||||
autofocus
|
||||
data-no-transition
|
||||
/>
|
||||
{:else}
|
||||
<button
|
||||
type="button"
|
||||
class="text-[12px] flex-1 min-w-0 {step.done ? 'line-through text-text-tertiary' : 'text-text-secondary'} truncate text-left cursor-text bg-transparent border-0 p-0"
|
||||
ondblclick={() => !step.done && startEdit(step)}
|
||||
disabled={step.done}
|
||||
aria-label="Double-click to edit this step"
|
||||
title="Double-click to edit"
|
||||
>{step.text}</button>
|
||||
{/if}
|
||||
|
||||
{#if !step.done && !editing[step.id]}
|
||||
<!-- HITL feedback: thumbs vote + pencil edit. All three
|
||||
route into record_feedback and feed the prompt-conditioning
|
||||
loop. See docs/roadmap/2026-04-23-... Phase 2. -->
|
||||
<button
|
||||
class="opacity-0 group-hover:opacity-100 p-0.5 text-text-tertiary hover:text-success
|
||||
{rating[step.id] === 1 ? '!opacity-100 text-success' : ''}"
|
||||
onclick={() => recordThumb(step, 1)}
|
||||
aria-label={rating[step.id] === 1 ? 'Remove thumbs up' : 'Thumbs up — this is a good step'}
|
||||
title="Thumbs up — train the model on this style"
|
||||
style={reduceMotion ? '' : 'transition: opacity var(--duration-ui), color var(--duration-ui)'}
|
||||
>
|
||||
<ThumbsUp size={10} aria-hidden="true" />
|
||||
</button>
|
||||
<button
|
||||
class="opacity-0 group-hover:opacity-100 p-0.5 text-text-tertiary hover:text-danger
|
||||
{rating[step.id] === -1 ? '!opacity-100 text-danger' : ''}"
|
||||
onclick={() => recordThumb(step, -1)}
|
||||
aria-label={rating[step.id] === -1 ? 'Remove thumbs down' : 'Thumbs down — this misses the mark'}
|
||||
title="Thumbs down — avoid this style"
|
||||
style={reduceMotion ? '' : 'transition: opacity var(--duration-ui), color var(--duration-ui)'}
|
||||
>
|
||||
<ThumbsDown size={10} aria-hidden="true" />
|
||||
</button>
|
||||
<button
|
||||
class="opacity-0 group-hover:opacity-100 p-0.5 text-text-tertiary hover:text-accent"
|
||||
onclick={() => startEdit(step)}
|
||||
aria-label="Edit this step (the correction trains future suggestions)"
|
||||
title="Edit — this is the strongest training signal"
|
||||
style={reduceMotion ? '' : 'transition: opacity var(--duration-ui)'}
|
||||
>
|
||||
<Pencil size={10} aria-hidden="true" />
|
||||
</button>
|
||||
<button
|
||||
class="opacity-0 group-hover:opacity-100 flex items-center gap-1 text-[10px] text-text-tertiary hover:text-accent"
|
||||
onclick={() => startTimer(step.id)}
|
||||
|
||||
@@ -107,7 +107,7 @@
|
||||
</div>
|
||||
<!-- Micro-steps panel (expanded) -->
|
||||
{#if expandedTaskIds.has(task.id)}
|
||||
<MicroSteps parentTaskId={task.id} />
|
||||
<MicroSteps parentTaskId={task.id} parentTaskText={task.text} />
|
||||
{/if}
|
||||
</div>
|
||||
{/each}
|
||||
|
||||
Reference in New Issue
Block a user