Two new Settings → AI knobs that compose cleanly with what already shipped (aiTier, LLM model, translator prompt framing). **B.1 #15 — Named cleanup presets.** LlmPromptPreset enum (Default / Email / Notes / Code) appends a short context hint onto the CLEANUP_PROMPT just before generation. Presets shape tone and structure ("email paragraph", "bulleted meeting notes", "preserve technical terms") without licensing the content-editing the translator-not-editor framing forbids. cleanup_transcript_text_cmd now takes `preset: Option<String>` which runs through the new LlmPromptPreset::parse (normalises aliases like "meeting-notes", collapses unknown values to Default). **A.1 #28 — Sequential-GPU guard.** New LocalEngine::unload drops the backend + model_id so a subsequent load actually reclaims VRAM. load_llm_model, load_model, and load_parakeet_model Tauri commands grow an optional `concurrent: bool` argument. When concurrent is Some(false), loading LLM first unloads whisper+parakeet, and vice versa — prevents VRAM OOM on tight-VRAM setups. Default is the previous parallel behaviour so nothing changes for multi-GB cards. Transcribe-in-progress paths (transcribe_pcm, transcribe_file, live) pass None, so mid-dictation model loads don't accidentally tear down the LLM. Settings UI (AI section): - Cleanup preset segmented button + descriptive copy for each option. - GPU concurrency segmented button with explicit trade-off text ("faster transitions vs fits in tight VRAM"). Frontend wiring: - settings.llmPromptPreset flows from DictationPage's cleanupTranscriptIfEnabled into the Tauri command. - settings.aiGpuConcurrency flows from both DictationPage (auto-load on record) and SettingsPage (manual load/unload buttons) as `concurrent: "parallel" === true` to the load commands. Tests: three new preset cases in crates/ai-formatting/src/llm_client.rs (parse aliases, suffix non-empty for non-default, default suffix empty). All 139 existing lib tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -2,7 +2,7 @@ use tauri::{Emitter, State};
|
||||
|
||||
use crate::commands::power::PowerAssertion;
|
||||
use crate::AppState;
|
||||
use kon_ai_formatting::llm_cleanup_text;
|
||||
use kon_ai_formatting::{llm_cleanup_text, LlmPromptPreset};
|
||||
use kon_core::hardware;
|
||||
use kon_llm::model_manager::{self, model_info};
|
||||
use kon_llm::LlmModelId;
|
||||
@@ -86,6 +86,7 @@ pub async fn load_llm_model(
|
||||
state: State<'_, AppState>,
|
||||
model_id: String,
|
||||
use_gpu: Option<bool>,
|
||||
concurrent: Option<bool>,
|
||||
) -> Result<(), String> {
|
||||
let id = parse_model_id(model_id)?;
|
||||
let path = model_manager::model_path(id);
|
||||
@@ -93,6 +94,15 @@ pub async fn load_llm_model(
|
||||
return Err("Model not downloaded — call download_llm_model first".to_string());
|
||||
}
|
||||
|
||||
// Sequential-GPU guard (brief item A.1 #28): when the user has opted
|
||||
// out of concurrent GPU residency, free the transcription engines
|
||||
// before loading the LLM. Prevents VRAM OOM on tight-GPU setups.
|
||||
// concurrent=None or Some(true) preserves legacy parallel behaviour.
|
||||
if concurrent == Some(false) {
|
||||
state.whisper_engine.unload();
|
||||
state.parakeet_engine.unload();
|
||||
}
|
||||
|
||||
let engine = state.llm_engine.clone();
|
||||
let use_gpu = use_gpu.unwrap_or(true);
|
||||
tokio::task::spawn_blocking(move || engine.load_model(id, &path, use_gpu))
|
||||
@@ -335,6 +345,7 @@ pub async fn cleanup_transcript_text_cmd(
|
||||
state: State<'_, AppState>,
|
||||
transcript: String,
|
||||
profile_id: Option<String>,
|
||||
preset: Option<String>,
|
||||
) -> Result<String, String> {
|
||||
let resolved_profile_id =
|
||||
profile_id.unwrap_or_else(|| kon_storage::DEFAULT_PROFILE_ID.to_string());
|
||||
@@ -346,13 +357,21 @@ pub async fn cleanup_transcript_text_cmd(
|
||||
.map(|term| term.term)
|
||||
.collect();
|
||||
|
||||
// Named preset (brief item B.1 #15): Email / Notes / Code shape the
|
||||
// output tone + structure without changing the translator-not-editor
|
||||
// contract. None or unknown → Default (no additional guidance).
|
||||
let resolved_preset = preset
|
||||
.as_deref()
|
||||
.map(LlmPromptPreset::parse)
|
||||
.unwrap_or(LlmPromptPreset::Default);
|
||||
|
||||
let engine = state.llm_engine.clone();
|
||||
tokio::task::spawn_blocking(move || {
|
||||
// macOS: pin a power assertion for the duration of the LLM
|
||||
// generation so App Nap can't decide to throttle us mid-token.
|
||||
// No-op on every other OS. Item #9.
|
||||
let _power_guard = PowerAssertion::begin("kon LLM cleanup");
|
||||
llm_cleanup_text(&engine, &transcript, &profile_terms)
|
||||
llm_cleanup_text(&engine, &transcript, &profile_terms, resolved_preset)
|
||||
})
|
||||
.await
|
||||
.map_err(|e| e.to_string())?
|
||||
|
||||
Reference in New Issue
Block a user