Files
Lumotia/crates/llm/src/prompts.rs
Jake 46be0a5aca
Some checks failed
check / cargo check (macos-latest) (push) Has been cancelled
check / cargo check (ubuntu-22.04) (push) Has been cancelled
check / cargo check (windows-latest) (push) Has been cancelled
check / svelte build + lint (push) Has been cancelled
feat(feedback): Phase 2 — HITL thumbs + correction capture with prompt-conditioning loop
Closes the human-in-the-loop gap from docs/brief/feature-set.md and
Phase 2 of the 2026-04-23 feature-complete roadmap.

Storage (kon-storage):
- Migration v10 adds the `feedback` table: (target_type, target_id,
  rating, original_text, corrected_text, context_json, profile_id,
  created_at) with CHECK constraints on target_type and rating, plus
  indexes on (target_type, rating, created_at DESC) for prompt-time
  retrieval and (profile_id, target_type, created_at DESC) for
  per-profile scoping.
- New public API: `FeedbackTargetType`, `RecordFeedbackParams`,
  `FeedbackRow`, `record_feedback`, `list_feedback_examples`.
- Tests updated — the RB-02 rollback regression now discovers the
  real max version at runtime instead of hard-coding v10 for its
  poison migration.

LLM (kon-llm):
- `prompts::FeedbackExample` — local shape for few-shot exemplars so
  kon-llm stays independent of kon-storage.
- `prompts::build_conditioned_system_prompt` — appends a "here is
  the style this user prefers" block to the base system prompt
  when examples are available; returns the base prompt unchanged
  when empty, so new users and early sessions see generic output.
- `LlmEngine::decompose_task_with_feedback` and
  `LlmEngine::extract_tasks_with_feedback` thread examples through
  to the builder. The old one-arg variants are preserved and now
  call through with an empty slice.
- 4 unit tests covering empty, empty-input-skip, correction-wins,
  and thumbs-up-only fallback.

Tauri (src-tauri):
- New commands::feedback module: `record_feedback`,
  `list_feedback_examples_cmd`.
- `decompose_and_store` and `extract_tasks_from_transcript_cmd`
  now fetch the last 5 positive/neutral feedback rows for their
  target type and pass them through to the LLM, wiring the
  learning loop end-to-end.
- Shared `to_llm_examples` helper parses the `context_json.input`
  field (where the recorder stashes the parent task text / transcript
  chunk) back into the exemplar shape.

Frontend (MicroSteps.svelte):
- Thumbs-up and thumbs-down buttons on every micro-step row.
  Hover-revealed; the vote recolours the icon; clicking again
  clears the local highlight (the row itself stays in the audit
  trail).
- Pencil icon + double-click to edit step text. Save flows through
  update_task_cmd for persistence and records a correction feedback
  row with (original_text, corrected_text) — the highest-value
  training signal.
- Parent task text is captured in context_json.input at record time
  so the prompt builder can reconstruct the (input, preferred-output)
  pair on subsequent decompositions.
- Feedback capture is best-effort — a record_feedback failure never
  interrupts the primary action.

What's deferred to a later phase:
- Thumbs + corrections on extracted tasks (same pipeline, different
  surface — probably TasksPage after the AI-extraction path)
- Thumbs on transcript cleanup output
- Semantic retrieval over the feedback corpus (once there is enough
  data to justify embedding infrastructure; the storage shape is
  already ready for it)
2026-04-24 12:53:51 +01:00

123 lines
4.7 KiB
Rust

pub const DECOMPOSE_TASK_SYSTEM: &str = "\
You are a task-decomposition assistant. Given a task description, produce \
between 3 and 7 concrete, physical micro-steps. Each step must be a short \
imperative sentence, actionable today, with no commentary. Output ONLY a \
JSON array of strings.";
pub const EXTRACT_TASKS_SYSTEM: &str = "\
You are a task-extraction assistant. Given a transcript of spoken notes, \
output a JSON array of action items the speaker committed to. Each item must \
be a short imperative sentence. Omit observations, wishes, and background \
context that are not explicit commitments. Output an empty array if there are \
no action items.";
/// Compact representation of a human-in-the-loop feedback example used
/// for few-shot prompt conditioning. Built by kon-storage and fed to the
/// prompt builder below; we keep this struct local to the LLM crate so
/// kon-llm does not depend on kon-storage.
#[derive(Debug, Clone)]
pub struct FeedbackExample {
/// What the AI was given as input (e.g. the parent task text, or
/// the transcript chunk). Kept verbatim.
pub input: String,
/// What the AI produced originally. `None` if the user only
/// gave a thumbs-up without a prior edit (positive signal
/// without a paired correction).
pub original_output: Option<String>,
/// What the user changed it to. `None` for thumbs-only rows.
/// This is the highest-value signal — when present, inject it
/// as the "good" output in the few-shot example.
pub corrected_output: Option<String>,
}
/// Render a feedback example into the exemplar block used in prompt
/// conditioning. Returns `None` for rows that carry no usable pairing
/// (e.g. a thumbs-up with no input context).
fn render_feedback_exemplar(ex: &FeedbackExample) -> Option<String> {
if ex.input.trim().is_empty() {
return None;
}
let good = ex
.corrected_output
.as_deref()
.or(ex.original_output.as_deref())?;
let good = good.trim();
if good.is_empty() {
return None;
}
Some(format!("Input: {}\nGood output: {}", ex.input.trim(), good))
}
/// Build a system prompt that combines the base task system prompt
/// with a few-shot block assembled from recent HITL examples. If no
/// usable examples are available, returns the base prompt unchanged
/// so early users see the generic behaviour and the LLM is not
/// confused by an empty exemplar section.
///
/// The exemplars are ordered most-recent-first (caller's order is
/// preserved) so the LLM weights the user's current style over
/// earlier noise, mirroring what a human reviewer would do.
pub fn build_conditioned_system_prompt(base: &str, examples: &[FeedbackExample]) -> String {
let rendered: Vec<String> = examples
.iter()
.filter_map(render_feedback_exemplar)
.collect();
if rendered.is_empty() {
return base.to_string();
}
let block = rendered
.iter()
.map(|s| format!("- {s}"))
.collect::<Vec<_>>()
.join("\n");
format!(
"{base}\n\nHere are examples of the style this user prefers, in the \
user's own words. Match this style closely when producing your output:\n{block}"
)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn builds_plain_prompt_when_no_examples() {
let out = build_conditioned_system_prompt(DECOMPOSE_TASK_SYSTEM, &[]);
assert_eq!(out, DECOMPOSE_TASK_SYSTEM);
}
#[test]
fn skips_empty_input_examples() {
let examples = vec![FeedbackExample {
input: String::new(),
original_output: None,
corrected_output: Some("ignored".into()),
}];
let out = build_conditioned_system_prompt(DECOMPOSE_TASK_SYSTEM, &examples);
assert_eq!(out, DECOMPOSE_TASK_SYSTEM);
}
#[test]
fn prefers_corrected_over_original() {
let examples = vec![FeedbackExample {
input: "Clean room".into(),
original_output: Some("Organise your bedroom".into()),
corrected_output: Some("Pick up one shirt from the floor".into()),
}];
let out = build_conditioned_system_prompt(DECOMPOSE_TASK_SYSTEM, &examples);
assert!(out.contains("Pick up one shirt from the floor"));
assert!(!out.contains("Organise your bedroom"));
}
#[test]
fn falls_back_to_original_when_no_correction() {
let examples = vec![FeedbackExample {
input: "Write report".into(),
original_output: Some("Open a blank document".into()),
corrected_output: None,
}];
let out = build_conditioned_system_prompt(DECOMPOSE_TASK_SYSTEM, &examples);
assert!(out.contains("Open a blank document"));
}
}