feat(feedback): Phase 2 — HITL thumbs + correction capture with prompt-conditioning loop
Some checks failed
check / cargo check (macos-latest) (push) Has been cancelled
check / cargo check (ubuntu-22.04) (push) Has been cancelled
check / cargo check (windows-latest) (push) Has been cancelled
check / svelte build + lint (push) Has been cancelled

Closes the human-in-the-loop gap from docs/brief/feature-set.md and
Phase 2 of the 2026-04-23 feature-complete roadmap.

Storage (kon-storage):
- Migration v10 adds the `feedback` table: (target_type, target_id,
  rating, original_text, corrected_text, context_json, profile_id,
  created_at) with CHECK constraints on target_type and rating, plus
  indexes on (target_type, rating, created_at DESC) for prompt-time
  retrieval and (profile_id, target_type, created_at DESC) for
  per-profile scoping.
- New public API: `FeedbackTargetType`, `RecordFeedbackParams`,
  `FeedbackRow`, `record_feedback`, `list_feedback_examples`.
- Tests updated — the RB-02 rollback regression now discovers the
  real max version at runtime instead of hard-coding v10 for its
  poison migration.

LLM (kon-llm):
- `prompts::FeedbackExample` — local shape for few-shot exemplars so
  kon-llm stays independent of kon-storage.
- `prompts::build_conditioned_system_prompt` — appends a "here is
  the style this user prefers" block to the base system prompt
  when examples are available; returns the base prompt unchanged
  when empty, so new users and early sessions see generic output.
- `LlmEngine::decompose_task_with_feedback` and
  `LlmEngine::extract_tasks_with_feedback` thread examples through
  to the builder. The old one-arg variants are preserved and now
  call through with an empty slice.
- 4 unit tests covering empty, empty-input-skip, correction-wins,
  and thumbs-up-only fallback.

Tauri (src-tauri):
- New commands::feedback module: `record_feedback`,
  `list_feedback_examples_cmd`.
- `decompose_and_store` and `extract_tasks_from_transcript_cmd`
  now fetch the last 5 positive/neutral feedback rows for their
  target type and pass them through to the LLM, wiring the
  learning loop end-to-end.
- Shared `to_llm_examples` helper parses the `context_json.input`
  field (where the recorder stashes the parent task text / transcript
  chunk) back into the exemplar shape.

Frontend (MicroSteps.svelte):
- Thumbs-up and thumbs-down buttons on every micro-step row.
  Hover-revealed; the vote recolours the icon; clicking again
  clears the local highlight (the row itself stays in the audit
  trail).
- Pencil icon + double-click to edit step text. Save flows through
  update_task_cmd for persistence and records a correction feedback
  row with (original_text, corrected_text) — the highest-value
  training signal.
- Parent task text is captured in context_json.input at record time
  so the prompt builder can reconstruct the (input, preferred-output)
  pair on subsequent decompositions.
- Feedback capture is best-effort — a record_feedback failure never
  interrupts the primary action.

What's deferred to a later phase:
- Thumbs + corrections on extracted tasks (same pipeline, different
  surface — probably TasksPage after the AI-extraction path)
- Thumbs on transcript cleanup output
- Semantic retrieval over the feedback corpus (once there is enough
  data to justify embedding infrastructure; the storage shape is
  already ready for it)
This commit is contained in:
2026-04-24 12:53:51 +01:00
parent f25f8db818
commit 46be0a5aca
11 changed files with 678 additions and 29 deletions

View File

@@ -889,6 +889,151 @@ pub async fn list_recent_errors(pool: &SqlitePool, limit: i64) -> Result<Vec<Err
.collect())
}
// --- Feedback (HITL) -------------------------------------------------------
//
// Phase 2 of the feature-complete roadmap: capture thumbs + corrections on
// AI-generated output so the prompt builder can inject recent examples as
// few-shot exemplars. Storage-only here; the prompt-conditioning logic lives
// in kon-llm. Retrieval returns the most recent rows, narrowed to the
// active profile when provided so feedback does not cross profiles.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum FeedbackTargetType {
MicroStep,
TaskExtraction,
Cleanup,
}
impl FeedbackTargetType {
pub fn as_str(self) -> &'static str {
match self {
FeedbackTargetType::MicroStep => "microstep",
FeedbackTargetType::TaskExtraction => "task_extraction",
FeedbackTargetType::Cleanup => "cleanup",
}
}
/// Parse the database `target_type` string back into the enum.
/// Named `parse` rather than `from_str` so it does not collide with
/// the `std::str::FromStr` trait — the trait is overkill here
/// because callers never want a `FromStr::Err` and already know the
/// set of valid values at the call site.
pub fn parse(s: &str) -> Option<Self> {
match s {
"microstep" => Some(FeedbackTargetType::MicroStep),
"task_extraction" => Some(FeedbackTargetType::TaskExtraction),
"cleanup" => Some(FeedbackTargetType::Cleanup),
_ => None,
}
}
}
#[derive(Debug, Clone)]
pub struct RecordFeedbackParams {
pub target_type: FeedbackTargetType,
pub target_id: Option<String>,
/// -1 = thumbs down, 0 = correction (neutral), +1 = thumbs up.
pub rating: i8,
pub original_text: Option<String>,
pub corrected_text: Option<String>,
pub context_json: Option<String>,
pub profile_id: Option<String>,
}
#[derive(Debug, Clone)]
pub struct FeedbackRow {
pub id: i64,
pub target_type: String,
pub target_id: Option<String>,
pub rating: i64,
pub original_text: Option<String>,
pub corrected_text: Option<String>,
pub context_json: Option<String>,
pub profile_id: String,
pub created_at: String,
}
pub async fn record_feedback(pool: &SqlitePool, params: RecordFeedbackParams) -> Result<i64> {
if !matches!(params.rating, -1..=1) {
return Err(KonError::StorageError(format!(
"invalid feedback rating {} (must be -1, 0, or 1)",
params.rating
)));
}
let profile_id = params
.profile_id
.unwrap_or_else(|| crate::DEFAULT_PROFILE_ID.to_string());
let row = sqlx::query(
"INSERT INTO feedback (
target_type, target_id, rating,
original_text, corrected_text, context_json, profile_id
) VALUES (?, ?, ?, ?, ?, ?, ?)
RETURNING id",
)
.bind(params.target_type.as_str())
.bind(params.target_id)
.bind(params.rating as i64)
.bind(params.original_text)
.bind(params.corrected_text)
.bind(params.context_json)
.bind(profile_id)
.fetch_one(pool)
.await
.map_err(|e| KonError::StorageError(format!("record_feedback failed: {e}")))?;
Ok(row.get::<i64, _>("id"))
}
/// Fetch the most recent feedback rows for a given target type, scoped to
/// the active profile. Used by the prompt builder to gather few-shot
/// exemplars. Orders by `created_at DESC` so the most recent corrections
/// outweigh older ones — the user's style drifts, and we want the LLM
/// to track the current preference.
///
/// `min_rating` filters out thumbs-down examples when the caller only
/// wants positive reinforcement; pass `-1` to include everything.
pub async fn list_feedback_examples(
pool: &SqlitePool,
target_type: FeedbackTargetType,
limit: i64,
min_rating: i8,
profile_id: Option<&str>,
) -> Result<Vec<FeedbackRow>> {
let pid = profile_id.unwrap_or(crate::DEFAULT_PROFILE_ID);
let rows = sqlx::query(
"SELECT id, target_type, target_id, rating,
original_text, corrected_text, context_json,
profile_id, created_at
FROM feedback
WHERE target_type = ?
AND profile_id = ?
AND rating >= ?
ORDER BY created_at DESC
LIMIT ?",
)
.bind(target_type.as_str())
.bind(pid)
.bind(min_rating as i64)
.bind(limit)
.fetch_all(pool)
.await
.map_err(|e| KonError::StorageError(format!("list_feedback_examples failed: {e}")))?;
Ok(rows
.into_iter()
.map(|r| FeedbackRow {
id: r.get("id"),
target_type: r.get("target_type"),
target_id: r.get("target_id"),
rating: r.get("rating"),
original_text: r.get("original_text"),
corrected_text: r.get("corrected_text"),
context_json: r.get("context_json"),
profile_id: r.get("profile_id"),
created_at: r.get("created_at"),
})
.collect())
}
#[cfg(test)]
mod tests {
use super::*;

View File

@@ -10,10 +10,11 @@ pub use database::{
add_profile_term, complete_subtask_and_check_parent, complete_task, count_transcripts,
create_profile, delete_profile, delete_profile_term, delete_task, delete_transcript,
get_profile, get_setting, get_task_by_id, get_transcript, init, insert_subtask, insert_task,
insert_transcript, list_profile_terms, list_profiles, list_recent_errors, list_subtasks,
list_tasks, list_transcripts, list_transcripts_paged, log_error, search_transcripts,
set_setting, uncomplete_task, update_profile, update_task, update_transcript,
update_transcript_meta, ErrorLogRow, InsertTranscriptParams, ProfileRow, ProfileTermRow,
insert_transcript, list_feedback_examples, list_profile_terms, list_profiles,
list_recent_errors, list_subtasks, list_tasks, list_transcripts, list_transcripts_paged,
log_error, record_feedback, search_transcripts, set_setting, uncomplete_task, update_profile,
update_task, update_transcript, update_transcript_meta, ErrorLogRow, FeedbackRow,
FeedbackTargetType, InsertTranscriptParams, ProfileRow, ProfileTermRow, RecordFeedbackParams,
TaskRow, TranscriptRow,
};
pub use file_storage::{app_data_dir, crashes_dir, database_path, logs_dir, recordings_dir};

View File

@@ -334,6 +334,49 @@ const MIGRATIONS: &[(i64, &str, &str)] = &[
FROM transcripts;
"#,
),
(
10,
"feedback: HITL thumbs + correction capture",
r#"
-- Feedback rows capture human-in-the-loop signal on AI-generated
-- output. Two flavours bundled into one table:
-- - thumbs (rating = -1 | +1, original_text optional, corrected_text NULL)
-- - correction (rating defaults to +1, original_text + corrected_text present)
--
-- `target_type` names the producing surface:
-- 'microstep' — subtask decomposition from DECOMPOSE_TASK_SYSTEM
-- 'task_extraction' — tasks lifted from a transcript (EXTRACT_TASKS_SYSTEM)
-- 'cleanup' — transcript cleanup output
--
-- `target_id` is the surface-specific identifier where one exists
-- (subtask id, task id, transcript id). NULL is allowed because
-- not every feedback event has a stable target id yet.
--
-- `context_json` carries the input the AI was conditioned on
-- (parent task text, transcript chunk, etc.) so future prompt
-- builders can reconstruct the original I/O pair for few-shot
-- injection or semantic retrieval.
CREATE TABLE feedback (
id INTEGER PRIMARY KEY AUTOINCREMENT,
target_type TEXT NOT NULL
CHECK (target_type IN ('microstep', 'task_extraction', 'cleanup')),
target_id TEXT,
rating INTEGER NOT NULL
CHECK (rating IN (-1, 0, 1)),
original_text TEXT,
corrected_text TEXT,
context_json TEXT,
profile_id TEXT NOT NULL DEFAULT '00000000-0000-0000-0000-000000000001'
REFERENCES profiles(id) ON DELETE RESTRICT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX idx_feedback_target_type_rating
ON feedback(target_type, rating, created_at DESC);
CREATE INDEX idx_feedback_profile
ON feedback(profile_id, target_type, created_at DESC);
"#,
),
];
/// Split SQL into individual statements, respecting BEGIN...END trigger blocks.
@@ -483,7 +526,7 @@ mod tests {
.fetch_one(&pool)
.await
.unwrap();
assert_eq!(count, 9);
assert_eq!(count, 10);
sqlx::query("INSERT INTO settings (key, value) VALUES ('test', 'value')")
.execute(&pool)
@@ -502,7 +545,7 @@ mod tests {
.fetch_one(&pool)
.await
.unwrap();
assert_eq!(count, 9);
assert_eq!(count, 10);
}
#[tokio::test]
@@ -859,8 +902,11 @@ mod tests {
// The poisoned migration below first creates `poison_marker`
// (syntactically valid, would succeed against any SQLite) and then
// runs a guaranteed-invalid function call. Under the new atomic
// implementation, neither `poison_marker` nor the v9 row should
// implementation, neither `poison_marker` nor the poison row should
// survive the failed call.
//
// Version number must sit above the real MIGRATIONS max so the
// baseline migrate cleanly finishes first.
#[tokio::test]
async fn multi_statement_migration_rolls_back_on_failure() {
let pool = SqlitePoolOptions::new()
@@ -871,8 +917,18 @@ mod tests {
run_migrations(&pool).await.expect("baseline migrate");
const POISON: &[(i64, &str, &str)] = &[(
10,
// Discover the real max version so the poison migration is
// always exactly one past the end of MIGRATIONS, regardless of
// how many real migrations we add in future.
let real_max: i64 =
sqlx::query_scalar("SELECT COALESCE(MAX(version), 0) FROM schema_version")
.fetch_one(&pool)
.await
.expect("read schema_version");
let poison_version = real_max + 1;
let poison: &[(i64, &str, &str)] = &[(
poison_version,
"rb-02 atomicity poison",
r#"
CREATE TABLE poison_marker (id INTEGER PRIMARY KEY);
@@ -880,7 +936,7 @@ mod tests {
"#,
)];
let result = run_migrations_slice(&pool, POISON).await;
let result = run_migrations_slice(&pool, poison).await;
assert!(
result.is_err(),
"poisoned migration must return Err, got: {result:?}"
@@ -896,14 +952,14 @@ mod tests {
"poison_marker must not exist; got: {marker:?}"
);
// `schema_version` must not include v10 — version insert is part
// of the same transaction that rolled back.
// `schema_version` must not include the poison version — version
// insert is part of the same transaction that rolled back.
let max: i64 = sqlx::query_scalar("SELECT COALESCE(MAX(version), 0) FROM schema_version")
.fetch_one(&pool)
.await
.expect("read schema_version");
assert_eq!(
max, 9,
max, real_max,
"schema_version must not advance past the failed migration"
);
}