feat(ai B.1 #15 + A.1 #28): cleanup presets and sequential-GPU guard
Some checks failed
check / cargo check (macos-latest) (push) Has been cancelled
check / cargo check (ubuntu-22.04) (push) Has been cancelled
check / cargo check (windows-latest) (push) Has been cancelled
check / svelte build + lint (push) Has been cancelled

Two new Settings → AI knobs that compose cleanly with what already
shipped (aiTier, LLM model, translator prompt framing).

**B.1 #15 — Named cleanup presets.** LlmPromptPreset enum
(Default / Email / Notes / Code) appends a short context hint onto
the CLEANUP_PROMPT just before generation. Presets shape tone and
structure ("email paragraph", "bulleted meeting notes", "preserve
technical terms") without licensing the content-editing the
translator-not-editor framing forbids. cleanup_transcript_text_cmd
now takes `preset: Option<String>` which runs through the new
LlmPromptPreset::parse (normalises aliases like "meeting-notes",
collapses unknown values to Default).

**A.1 #28 — Sequential-GPU guard.** New LocalEngine::unload drops
the backend + model_id so a subsequent load actually reclaims VRAM.
load_llm_model, load_model, and load_parakeet_model Tauri commands
grow an optional `concurrent: bool` argument. When concurrent is
Some(false), loading LLM first unloads whisper+parakeet, and vice
versa — prevents VRAM OOM on tight-VRAM setups. Default is the
previous parallel behaviour so nothing changes for multi-GB cards.
Transcribe-in-progress paths (transcribe_pcm, transcribe_file, live)
pass None, so mid-dictation model loads don't accidentally tear
down the LLM.

Settings UI (AI section):
- Cleanup preset segmented button + descriptive copy for each option.
- GPU concurrency segmented button with explicit trade-off text
  ("faster transitions vs fits in tight VRAM").

Frontend wiring:
- settings.llmPromptPreset flows from DictationPage's
  cleanupTranscriptIfEnabled into the Tauri command.
- settings.aiGpuConcurrency flows from both DictationPage (auto-load
  on record) and SettingsPage (manual load/unload buttons) as
  `concurrent: "parallel" === true` to the load commands.

Tests: three new preset cases in crates/ai-formatting/src/llm_client.rs
(parse aliases, suffix non-empty for non-default, default suffix
empty). All 139 existing lib tests still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-21 17:12:48 +01:00
parent a57da0feb5
commit ce2b4fdac6
12 changed files with 254 additions and 18 deletions

View File

@@ -4,6 +4,6 @@ pub mod pipeline;
pub mod rule_based;
pub use correction_learning::extract_corrections;
pub use llm_client::cleanup_text as llm_cleanup_text;
pub use llm_client::{cleanup_text as llm_cleanup_text, LlmPromptPreset};
pub use pipeline::{post_process_segments, FormatMode, PostProcessOptions};
pub use rule_based::{format_text, is_hallucination, remove_fillers, to_british_english};

View File

@@ -67,19 +67,95 @@ pub fn format_dictionary_suffix(terms: &[String]) -> String {
)
}
/// Named cleanup-style presets (brief item B.1 #15). Each preset adds a
/// short additional instruction to the translation contract so the same
/// underlying translator behaviour produces output appropriate for the
/// user's current context (email vs. meeting notes vs. code).
///
/// Deliberately narrow set — four presets is small enough to pick from a
/// dropdown without becoming its own cognitive load. Users wanting more
/// nuance edit `profile.initial_prompt` instead; presets layer on top of
/// whatever the active profile specifies.
///
/// The translator-not-editor framing from CLEANUP_PROMPT still governs —
/// presets shape tone and structure, never licence content editing.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum LlmPromptPreset {
/// No additional guidance beyond the profile's initial_prompt.
Default,
/// Format as an email paragraph — tight sentences, natural
/// paragraph breaks at topic shifts, no markdown.
Email,
/// Format as bulleted meeting notes. Lead action items with an
/// imperative verb; keep informational sentences as prose.
Notes,
/// Software-dictation mode. Preserve technical terms, variable
/// names, file paths, and symbols exactly as spoken. Do not reword
/// technical phrasing.
Code,
}
impl LlmPromptPreset {
/// Parse a frontend-serialised preset identifier. Unknown or empty
/// strings collapse to Default so an outdated frontend can never
/// produce an unhandled enum variant — the user just sees baseline
/// behaviour.
pub fn parse(value: &str) -> Self {
match value.trim().to_ascii_lowercase().as_str() {
"email" => Self::Email,
"notes" | "meeting" | "meeting-notes" => Self::Notes,
"code" | "software" => Self::Code,
_ => Self::Default,
}
}
/// Extra instruction appended to the system prompt. Empty string
/// for Default — no whitespace or leading newline — so the concat
/// with the dictionary suffix stays clean.
pub fn suffix(self) -> &'static str {
match self {
Self::Default => "",
Self::Email => concat!(
"\n\n",
"Context: the speaker is dictating an email. Produce a single ",
"coherent email paragraph (or two if the topic clearly shifts). ",
"Tight sentences, no markdown, no salutation or signature unless ",
"the speaker explicitly dictates one.",
),
Self::Notes => concat!(
"\n\n",
"Context: the speaker is dictating meeting notes. Where the text ",
"contains a list of items or action items, render them as a ",
"markdown bullet list ('- '). Action items should lead with an ",
"imperative verb. Preserve prose informational sentences as prose; ",
"don't force bullets where narrative is clearer.",
),
Self::Code => concat!(
"\n\n",
"Context: the speaker is dictating about software. Preserve ",
"technical terms, variable names, file paths, CLI flags, and ",
"symbols exactly as spoken. Do not reword technical phrasing or ",
"'translate' identifiers into natural English.",
),
}
}
}
pub fn cleanup_text(
engine: &LlmEngine,
transcript: &str,
dictionary_terms: &[String],
preset: LlmPromptPreset,
) -> Result<String, EngineError> {
if transcript.trim().is_empty() {
return Ok(String::new());
}
let system_prompt = format!(
"{}{}",
"{}{}{}",
CLEANUP_PROMPT,
format_dictionary_suffix(dictionary_terms),
preset.suffix(),
);
engine.cleanup_text(&system_prompt, transcript)
}
@@ -134,14 +210,39 @@ mod tests {
#[test]
fn cleanup_empty_returns_empty_string() {
let engine = LlmEngine::new();
let result = cleanup_text(&engine, "", &[]);
let result = cleanup_text(&engine, "", &[], LlmPromptPreset::Default);
assert!(matches!(result, Ok(cleaned) if cleaned.is_empty()));
}
#[test]
fn cleanup_unloaded_returns_not_loaded_error() {
let engine = LlmEngine::new();
let result = cleanup_text(&engine, "um hi there", &[]);
let result = cleanup_text(&engine, "um hi there", &[], LlmPromptPreset::Default);
assert!(matches!(result, Err(EngineError::NotLoaded)));
}
#[test]
fn preset_parse_normalises_aliases() {
assert_eq!(LlmPromptPreset::parse("email"), LlmPromptPreset::Email);
assert_eq!(LlmPromptPreset::parse("EMAIL"), LlmPromptPreset::Email);
assert_eq!(LlmPromptPreset::parse("notes"), LlmPromptPreset::Notes);
assert_eq!(LlmPromptPreset::parse("meeting"), LlmPromptPreset::Notes);
assert_eq!(LlmPromptPreset::parse("meeting-notes"), LlmPromptPreset::Notes);
assert_eq!(LlmPromptPreset::parse("code"), LlmPromptPreset::Code);
assert_eq!(LlmPromptPreset::parse("software"), LlmPromptPreset::Code);
// Unknown values and explicit default fall back safely.
assert_eq!(LlmPromptPreset::parse("default"), LlmPromptPreset::Default);
assert_eq!(LlmPromptPreset::parse(""), LlmPromptPreset::Default);
assert_eq!(LlmPromptPreset::parse("random-unknown"), LlmPromptPreset::Default);
}
#[test]
fn preset_suffix_shapes_tone_without_editing_licence() {
// Each non-default preset must add something; the Default must
// be empty so it composes cleanly with dictionary suffix.
assert!(LlmPromptPreset::Default.suffix().is_empty());
assert!(LlmPromptPreset::Email.suffix().contains("email"));
assert!(LlmPromptPreset::Notes.suffix().to_lowercase().contains("bullet"));
assert!(LlmPromptPreset::Code.suffix().contains("technical"));
}
}

View File

@@ -76,7 +76,17 @@ pub fn post_process_segments(
.join(" ");
if !joined.is_empty() {
match llm_client::cleanup_text(engine, &joined, &options.dictionary_terms) {
// Pipeline-internal cleanup (used by file-based + live
// transcribe paths) runs with the Default preset. The
// named-preset UX (B.1 #15) flows through the explicit
// cleanup_transcript_text_cmd path instead, where the
// frontend decides which preset the user has selected.
match llm_client::cleanup_text(
engine,
&joined,
&options.dictionary_terms,
llm_client::LlmPromptPreset::Default,
) {
Ok(cleaned) if !cleaned.trim().is_empty() => {
replace_segments_with_cleaned(segments, cleaned.trim());
}

View File

@@ -56,6 +56,23 @@ impl LocalEngine {
*id_guard = Some(model_id);
}
/// Drop the loaded model and free its backing resources (GPU VRAM,
/// CPU memory, mmap'd GGML tensors). Used by the sequential-GPU
/// guard (brief item A.1 #28) so loading the LLM on a tight-VRAM
/// system first frees the transcription engine, and vice versa.
///
/// No-op when nothing is loaded. Thread-safe — the internal Mutex
/// serialises against concurrent transcribe_sync calls.
pub fn unload(&self) {
let mut guard = self.engine.lock().unwrap_or_else(|e| e.into_inner());
*guard = None;
let mut id_guard = self
.loaded_model_id
.lock()
.unwrap_or_else(|e| e.into_inner());
*id_guard = None;
}
pub fn name(&self) -> &EngineName {
&self.engine_name
}