Closes the human-in-the-loop gap from docs/brief/feature-set.md and Phase 2 of the 2026-04-23 feature-complete roadmap. Storage (kon-storage): - Migration v10 adds the `feedback` table: (target_type, target_id, rating, original_text, corrected_text, context_json, profile_id, created_at) with CHECK constraints on target_type and rating, plus indexes on (target_type, rating, created_at DESC) for prompt-time retrieval and (profile_id, target_type, created_at DESC) for per-profile scoping. - New public API: `FeedbackTargetType`, `RecordFeedbackParams`, `FeedbackRow`, `record_feedback`, `list_feedback_examples`. - Tests updated — the RB-02 rollback regression now discovers the real max version at runtime instead of hard-coding v10 for its poison migration. LLM (kon-llm): - `prompts::FeedbackExample` — local shape for few-shot exemplars so kon-llm stays independent of kon-storage. - `prompts::build_conditioned_system_prompt` — appends a "here is the style this user prefers" block to the base system prompt when examples are available; returns the base prompt unchanged when empty, so new users and early sessions see generic output. - `LlmEngine::decompose_task_with_feedback` and `LlmEngine::extract_tasks_with_feedback` thread examples through to the builder. The old one-arg variants are preserved and now call through with an empty slice. - 4 unit tests covering empty, empty-input-skip, correction-wins, and thumbs-up-only fallback. Tauri (src-tauri): - New commands::feedback module: `record_feedback`, `list_feedback_examples_cmd`. - `decompose_and_store` and `extract_tasks_from_transcript_cmd` now fetch the last 5 positive/neutral feedback rows for their target type and pass them through to the LLM, wiring the learning loop end-to-end. - Shared `to_llm_examples` helper parses the `context_json.input` field (where the recorder stashes the parent task text / transcript chunk) back into the exemplar shape. Frontend (MicroSteps.svelte): - Thumbs-up and thumbs-down buttons on every micro-step row. Hover-revealed; the vote recolours the icon; clicking again clears the local highlight (the row itself stays in the audit trail). - Pencil icon + double-click to edit step text. Save flows through update_task_cmd for persistence and records a correction feedback row with (original_text, corrected_text) — the highest-value training signal. - Parent task text is captured in context_json.input at record time so the prompt builder can reconstruct the (input, preferred-output) pair on subsequent decompositions. - Feedback capture is best-effort — a record_feedback failure never interrupts the primary action. What's deferred to a later phase: - Thumbs + corrections on extracted tasks (same pipeline, different surface — probably TasksPage after the AI-extraction path) - Thumbs on transcript cleanup output - Semantic retrieval over the feedback corpus (once there is enough data to justify embedding infrastructure; the storage shape is already ready for it)
123 lines
4.7 KiB
Rust
123 lines
4.7 KiB
Rust
pub const DECOMPOSE_TASK_SYSTEM: &str = "\
|
|
You are a task-decomposition assistant. Given a task description, produce \
|
|
between 3 and 7 concrete, physical micro-steps. Each step must be a short \
|
|
imperative sentence, actionable today, with no commentary. Output ONLY a \
|
|
JSON array of strings.";
|
|
|
|
pub const EXTRACT_TASKS_SYSTEM: &str = "\
|
|
You are a task-extraction assistant. Given a transcript of spoken notes, \
|
|
output a JSON array of action items the speaker committed to. Each item must \
|
|
be a short imperative sentence. Omit observations, wishes, and background \
|
|
context that are not explicit commitments. Output an empty array if there are \
|
|
no action items.";
|
|
|
|
/// Compact representation of a human-in-the-loop feedback example used
|
|
/// for few-shot prompt conditioning. Built by kon-storage and fed to the
|
|
/// prompt builder below; we keep this struct local to the LLM crate so
|
|
/// kon-llm does not depend on kon-storage.
|
|
#[derive(Debug, Clone)]
|
|
pub struct FeedbackExample {
|
|
/// What the AI was given as input (e.g. the parent task text, or
|
|
/// the transcript chunk). Kept verbatim.
|
|
pub input: String,
|
|
/// What the AI produced originally. `None` if the user only
|
|
/// gave a thumbs-up without a prior edit (positive signal
|
|
/// without a paired correction).
|
|
pub original_output: Option<String>,
|
|
/// What the user changed it to. `None` for thumbs-only rows.
|
|
/// This is the highest-value signal — when present, inject it
|
|
/// as the "good" output in the few-shot example.
|
|
pub corrected_output: Option<String>,
|
|
}
|
|
|
|
/// Render a feedback example into the exemplar block used in prompt
|
|
/// conditioning. Returns `None` for rows that carry no usable pairing
|
|
/// (e.g. a thumbs-up with no input context).
|
|
fn render_feedback_exemplar(ex: &FeedbackExample) -> Option<String> {
|
|
if ex.input.trim().is_empty() {
|
|
return None;
|
|
}
|
|
let good = ex
|
|
.corrected_output
|
|
.as_deref()
|
|
.or(ex.original_output.as_deref())?;
|
|
let good = good.trim();
|
|
if good.is_empty() {
|
|
return None;
|
|
}
|
|
Some(format!("Input: {}\nGood output: {}", ex.input.trim(), good))
|
|
}
|
|
|
|
/// Build a system prompt that combines the base task system prompt
|
|
/// with a few-shot block assembled from recent HITL examples. If no
|
|
/// usable examples are available, returns the base prompt unchanged
|
|
/// so early users see the generic behaviour and the LLM is not
|
|
/// confused by an empty exemplar section.
|
|
///
|
|
/// The exemplars are ordered most-recent-first (caller's order is
|
|
/// preserved) so the LLM weights the user's current style over
|
|
/// earlier noise, mirroring what a human reviewer would do.
|
|
pub fn build_conditioned_system_prompt(base: &str, examples: &[FeedbackExample]) -> String {
|
|
let rendered: Vec<String> = examples
|
|
.iter()
|
|
.filter_map(render_feedback_exemplar)
|
|
.collect();
|
|
if rendered.is_empty() {
|
|
return base.to_string();
|
|
}
|
|
let block = rendered
|
|
.iter()
|
|
.map(|s| format!("- {s}"))
|
|
.collect::<Vec<_>>()
|
|
.join("\n");
|
|
format!(
|
|
"{base}\n\nHere are examples of the style this user prefers, in the \
|
|
user's own words. Match this style closely when producing your output:\n{block}"
|
|
)
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn builds_plain_prompt_when_no_examples() {
|
|
let out = build_conditioned_system_prompt(DECOMPOSE_TASK_SYSTEM, &[]);
|
|
assert_eq!(out, DECOMPOSE_TASK_SYSTEM);
|
|
}
|
|
|
|
#[test]
|
|
fn skips_empty_input_examples() {
|
|
let examples = vec![FeedbackExample {
|
|
input: String::new(),
|
|
original_output: None,
|
|
corrected_output: Some("ignored".into()),
|
|
}];
|
|
let out = build_conditioned_system_prompt(DECOMPOSE_TASK_SYSTEM, &examples);
|
|
assert_eq!(out, DECOMPOSE_TASK_SYSTEM);
|
|
}
|
|
|
|
#[test]
|
|
fn prefers_corrected_over_original() {
|
|
let examples = vec![FeedbackExample {
|
|
input: "Clean room".into(),
|
|
original_output: Some("Organise your bedroom".into()),
|
|
corrected_output: Some("Pick up one shirt from the floor".into()),
|
|
}];
|
|
let out = build_conditioned_system_prompt(DECOMPOSE_TASK_SYSTEM, &examples);
|
|
assert!(out.contains("Pick up one shirt from the floor"));
|
|
assert!(!out.contains("Organise your bedroom"));
|
|
}
|
|
|
|
#[test]
|
|
fn falls_back_to_original_when_no_correction() {
|
|
let examples = vec![FeedbackExample {
|
|
input: "Write report".into(),
|
|
original_output: Some("Open a blank document".into()),
|
|
corrected_output: None,
|
|
}];
|
|
let out = build_conditioned_system_prompt(DECOMPOSE_TASK_SYSTEM, &examples);
|
|
assert!(out.contains("Open a blank document"));
|
|
}
|
|
}
|