feat(ai B.1 #15 + A.1 #28): cleanup presets and sequential-GPU guard
Some checks failed
check / cargo check (macos-latest) (push) Has been cancelled
check / cargo check (ubuntu-22.04) (push) Has been cancelled
check / cargo check (windows-latest) (push) Has been cancelled
check / svelte build + lint (push) Has been cancelled

Two new Settings → AI knobs that compose cleanly with what already
shipped (aiTier, LLM model, translator prompt framing).

**B.1 #15 — Named cleanup presets.** LlmPromptPreset enum
(Default / Email / Notes / Code) appends a short context hint onto
the CLEANUP_PROMPT just before generation. Presets shape tone and
structure ("email paragraph", "bulleted meeting notes", "preserve
technical terms") without licensing the content-editing the
translator-not-editor framing forbids. cleanup_transcript_text_cmd
now takes `preset: Option<String>` which runs through the new
LlmPromptPreset::parse (normalises aliases like "meeting-notes",
collapses unknown values to Default).

**A.1 #28 — Sequential-GPU guard.** New LocalEngine::unload drops
the backend + model_id so a subsequent load actually reclaims VRAM.
load_llm_model, load_model, and load_parakeet_model Tauri commands
grow an optional `concurrent: bool` argument. When concurrent is
Some(false), loading LLM first unloads whisper+parakeet, and vice
versa — prevents VRAM OOM on tight-VRAM setups. Default is the
previous parallel behaviour so nothing changes for multi-GB cards.
Transcribe-in-progress paths (transcribe_pcm, transcribe_file, live)
pass None, so mid-dictation model loads don't accidentally tear
down the LLM.

Settings UI (AI section):
- Cleanup preset segmented button + descriptive copy for each option.
- GPU concurrency segmented button with explicit trade-off text
  ("faster transitions vs fits in tight VRAM").

Frontend wiring:
- settings.llmPromptPreset flows from DictationPage's
  cleanupTranscriptIfEnabled into the Tauri command.
- settings.aiGpuConcurrency flows from both DictationPage (auto-load
  on record) and SettingsPage (manual load/unload buttons) as
  `concurrent: "parallel" === true` to the load commands.

Tests: three new preset cases in crates/ai-formatting/src/llm_client.rs
(parse aliases, suffix non-empty for non-default, default suffix
empty). All 139 existing lib tests still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-21 17:12:48 +01:00
parent a57da0feb5
commit ce2b4fdac6
12 changed files with 254 additions and 18 deletions

View File

@@ -2,7 +2,7 @@ use tauri::{Emitter, State};
use crate::commands::power::PowerAssertion;
use crate::AppState;
use kon_ai_formatting::llm_cleanup_text;
use kon_ai_formatting::{llm_cleanup_text, LlmPromptPreset};
use kon_core::hardware;
use kon_llm::model_manager::{self, model_info};
use kon_llm::LlmModelId;
@@ -86,6 +86,7 @@ pub async fn load_llm_model(
state: State<'_, AppState>,
model_id: String,
use_gpu: Option<bool>,
concurrent: Option<bool>,
) -> Result<(), String> {
let id = parse_model_id(model_id)?;
let path = model_manager::model_path(id);
@@ -93,6 +94,15 @@ pub async fn load_llm_model(
return Err("Model not downloaded — call download_llm_model first".to_string());
}
// Sequential-GPU guard (brief item A.1 #28): when the user has opted
// out of concurrent GPU residency, free the transcription engines
// before loading the LLM. Prevents VRAM OOM on tight-GPU setups.
// concurrent=None or Some(true) preserves legacy parallel behaviour.
if concurrent == Some(false) {
state.whisper_engine.unload();
state.parakeet_engine.unload();
}
let engine = state.llm_engine.clone();
let use_gpu = use_gpu.unwrap_or(true);
tokio::task::spawn_blocking(move || engine.load_model(id, &path, use_gpu))
@@ -335,6 +345,7 @@ pub async fn cleanup_transcript_text_cmd(
state: State<'_, AppState>,
transcript: String,
profile_id: Option<String>,
preset: Option<String>,
) -> Result<String, String> {
let resolved_profile_id =
profile_id.unwrap_or_else(|| kon_storage::DEFAULT_PROFILE_ID.to_string());
@@ -346,13 +357,21 @@ pub async fn cleanup_transcript_text_cmd(
.map(|term| term.term)
.collect();
// Named preset (brief item B.1 #15): Email / Notes / Code shape the
// output tone + structure without changing the translator-not-editor
// contract. None or unknown → Default (no additional guidance).
let resolved_preset = preset
.as_deref()
.map(LlmPromptPreset::parse)
.unwrap_or(LlmPromptPreset::Default);
let engine = state.llm_engine.clone();
tokio::task::spawn_blocking(move || {
// macOS: pin a power assertion for the duration of the LLM
// generation so App Nap can't decide to throttle us mid-token.
// No-op on every other OS. Item #9.
let _power_guard = PowerAssertion::begin("kon LLM cleanup");
llm_cleanup_text(&engine, &transcript, &profile_terms)
llm_cleanup_text(&engine, &transcript, &profile_terms, resolved_preset)
})
.await
.map_err(|e| e.to_string())?