feat(feedback): Phase 2 — HITL thumbs + correction capture with prompt-conditioning loop
Closes the human-in-the-loop gap from docs/brief/feature-set.md and Phase 2 of the 2026-04-23 feature-complete roadmap. Storage (kon-storage): - Migration v10 adds the `feedback` table: (target_type, target_id, rating, original_text, corrected_text, context_json, profile_id, created_at) with CHECK constraints on target_type and rating, plus indexes on (target_type, rating, created_at DESC) for prompt-time retrieval and (profile_id, target_type, created_at DESC) for per-profile scoping. - New public API: `FeedbackTargetType`, `RecordFeedbackParams`, `FeedbackRow`, `record_feedback`, `list_feedback_examples`. - Tests updated — the RB-02 rollback regression now discovers the real max version at runtime instead of hard-coding v10 for its poison migration. LLM (kon-llm): - `prompts::FeedbackExample` — local shape for few-shot exemplars so kon-llm stays independent of kon-storage. - `prompts::build_conditioned_system_prompt` — appends a "here is the style this user prefers" block to the base system prompt when examples are available; returns the base prompt unchanged when empty, so new users and early sessions see generic output. - `LlmEngine::decompose_task_with_feedback` and `LlmEngine::extract_tasks_with_feedback` thread examples through to the builder. The old one-arg variants are preserved and now call through with an empty slice. - 4 unit tests covering empty, empty-input-skip, correction-wins, and thumbs-up-only fallback. Tauri (src-tauri): - New commands::feedback module: `record_feedback`, `list_feedback_examples_cmd`. - `decompose_and_store` and `extract_tasks_from_transcript_cmd` now fetch the last 5 positive/neutral feedback rows for their target type and pass them through to the LLM, wiring the learning loop end-to-end. - Shared `to_llm_examples` helper parses the `context_json.input` field (where the recorder stashes the parent task text / transcript chunk) back into the exemplar shape. Frontend (MicroSteps.svelte): - Thumbs-up and thumbs-down buttons on every micro-step row. Hover-revealed; the vote recolours the icon; clicking again clears the local highlight (the row itself stays in the audit trail). - Pencil icon + double-click to edit step text. Save flows through update_task_cmd for persistence and records a correction feedback row with (original_text, corrected_text) — the highest-value training signal. - Parent task text is captured in context_json.input at record time so the prompt builder can reconstruct the (input, preferred-output) pair on subsequent decompositions. - Feedback capture is best-effort — a record_feedback failure never interrupts the primary action. What's deferred to a later phase: - Thumbs + corrections on extracted tasks (same pipeline, different surface — probably TasksPage after the AI-extraction path) - Thumbs on transcript cleanup output - Semantic retrieval over the feedback corpus (once there is enough data to justify embedding infrastructure; the storage shape is already ready for it)
This commit is contained in:
@@ -10,3 +10,113 @@ output a JSON array of action items the speaker committed to. Each item must \
|
||||
be a short imperative sentence. Omit observations, wishes, and background \
|
||||
context that are not explicit commitments. Output an empty array if there are \
|
||||
no action items.";
|
||||
|
||||
/// Compact representation of a human-in-the-loop feedback example used
|
||||
/// for few-shot prompt conditioning. Built by kon-storage and fed to the
|
||||
/// prompt builder below; we keep this struct local to the LLM crate so
|
||||
/// kon-llm does not depend on kon-storage.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct FeedbackExample {
|
||||
/// What the AI was given as input (e.g. the parent task text, or
|
||||
/// the transcript chunk). Kept verbatim.
|
||||
pub input: String,
|
||||
/// What the AI produced originally. `None` if the user only
|
||||
/// gave a thumbs-up without a prior edit (positive signal
|
||||
/// without a paired correction).
|
||||
pub original_output: Option<String>,
|
||||
/// What the user changed it to. `None` for thumbs-only rows.
|
||||
/// This is the highest-value signal — when present, inject it
|
||||
/// as the "good" output in the few-shot example.
|
||||
pub corrected_output: Option<String>,
|
||||
}
|
||||
|
||||
/// Render a feedback example into the exemplar block used in prompt
|
||||
/// conditioning. Returns `None` for rows that carry no usable pairing
|
||||
/// (e.g. a thumbs-up with no input context).
|
||||
fn render_feedback_exemplar(ex: &FeedbackExample) -> Option<String> {
|
||||
if ex.input.trim().is_empty() {
|
||||
return None;
|
||||
}
|
||||
let good = ex
|
||||
.corrected_output
|
||||
.as_deref()
|
||||
.or(ex.original_output.as_deref())?;
|
||||
let good = good.trim();
|
||||
if good.is_empty() {
|
||||
return None;
|
||||
}
|
||||
Some(format!("Input: {}\nGood output: {}", ex.input.trim(), good))
|
||||
}
|
||||
|
||||
/// Build a system prompt that combines the base task system prompt
|
||||
/// with a few-shot block assembled from recent HITL examples. If no
|
||||
/// usable examples are available, returns the base prompt unchanged
|
||||
/// so early users see the generic behaviour and the LLM is not
|
||||
/// confused by an empty exemplar section.
|
||||
///
|
||||
/// The exemplars are ordered most-recent-first (caller's order is
|
||||
/// preserved) so the LLM weights the user's current style over
|
||||
/// earlier noise, mirroring what a human reviewer would do.
|
||||
pub fn build_conditioned_system_prompt(base: &str, examples: &[FeedbackExample]) -> String {
|
||||
let rendered: Vec<String> = examples
|
||||
.iter()
|
||||
.filter_map(render_feedback_exemplar)
|
||||
.collect();
|
||||
if rendered.is_empty() {
|
||||
return base.to_string();
|
||||
}
|
||||
let block = rendered
|
||||
.iter()
|
||||
.map(|s| format!("- {s}"))
|
||||
.collect::<Vec<_>>()
|
||||
.join("\n");
|
||||
format!(
|
||||
"{base}\n\nHere are examples of the style this user prefers, in the \
|
||||
user's own words. Match this style closely when producing your output:\n{block}"
|
||||
)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn builds_plain_prompt_when_no_examples() {
|
||||
let out = build_conditioned_system_prompt(DECOMPOSE_TASK_SYSTEM, &[]);
|
||||
assert_eq!(out, DECOMPOSE_TASK_SYSTEM);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn skips_empty_input_examples() {
|
||||
let examples = vec![FeedbackExample {
|
||||
input: String::new(),
|
||||
original_output: None,
|
||||
corrected_output: Some("ignored".into()),
|
||||
}];
|
||||
let out = build_conditioned_system_prompt(DECOMPOSE_TASK_SYSTEM, &examples);
|
||||
assert_eq!(out, DECOMPOSE_TASK_SYSTEM);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn prefers_corrected_over_original() {
|
||||
let examples = vec![FeedbackExample {
|
||||
input: "Clean room".into(),
|
||||
original_output: Some("Organise your bedroom".into()),
|
||||
corrected_output: Some("Pick up one shirt from the floor".into()),
|
||||
}];
|
||||
let out = build_conditioned_system_prompt(DECOMPOSE_TASK_SYSTEM, &examples);
|
||||
assert!(out.contains("Pick up one shirt from the floor"));
|
||||
assert!(!out.contains("Organise your bedroom"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn falls_back_to_original_when_no_correction() {
|
||||
let examples = vec![FeedbackExample {
|
||||
input: "Write report".into(),
|
||||
original_output: Some("Open a blank document".into()),
|
||||
corrected_output: None,
|
||||
}];
|
||||
let out = build_conditioned_system_prompt(DECOMPOSE_TASK_SYSTEM, &examples);
|
||||
assert!(out.contains("Open a blank document"));
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user