Two new Settings → AI knobs that compose cleanly with what already shipped (aiTier, LLM model, translator prompt framing). **B.1 #15 — Named cleanup presets.** LlmPromptPreset enum (Default / Email / Notes / Code) appends a short context hint onto the CLEANUP_PROMPT just before generation. Presets shape tone and structure ("email paragraph", "bulleted meeting notes", "preserve technical terms") without licensing the content-editing the translator-not-editor framing forbids. cleanup_transcript_text_cmd now takes `preset: Option<String>` which runs through the new LlmPromptPreset::parse (normalises aliases like "meeting-notes", collapses unknown values to Default). **A.1 #28 — Sequential-GPU guard.** New LocalEngine::unload drops the backend + model_id so a subsequent load actually reclaims VRAM. load_llm_model, load_model, and load_parakeet_model Tauri commands grow an optional `concurrent: bool` argument. When concurrent is Some(false), loading LLM first unloads whisper+parakeet, and vice versa — prevents VRAM OOM on tight-VRAM setups. Default is the previous parallel behaviour so nothing changes for multi-GB cards. Transcribe-in-progress paths (transcribe_pcm, transcribe_file, live) pass None, so mid-dictation model loads don't accidentally tear down the LLM. Settings UI (AI section): - Cleanup preset segmented button + descriptive copy for each option. - GPU concurrency segmented button with explicit trade-off text ("faster transitions vs fits in tight VRAM"). Frontend wiring: - settings.llmPromptPreset flows from DictationPage's cleanupTranscriptIfEnabled into the Tauri command. - settings.aiGpuConcurrency flows from both DictationPage (auto-load on record) and SettingsPage (manual load/unload buttons) as `concurrent: "parallel" === true` to the load commands. Tests: three new preset cases in crates/ai-formatting/src/llm_client.rs (parse aliases, suffix non-empty for non-default, default suffix empty). All 139 existing lib tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -239,7 +239,13 @@
|
||||
try {
|
||||
const status = await invoke("check_llm_model", { modelId: settings.llmModelId });
|
||||
if (status?.downloaded && !status.loaded) {
|
||||
await invoke("load_llm_model", { modelId: settings.llmModelId });
|
||||
await invoke("load_llm_model", {
|
||||
modelId: settings.llmModelId,
|
||||
// Sequential-GPU guard (brief item A.1 #28): frees whisper
|
||||
// before loading the LLM on tight-VRAM setups. "parallel"
|
||||
// keeps both resident (default, safe on multi-GB cards).
|
||||
concurrent: settings.aiGpuConcurrency === "parallel",
|
||||
});
|
||||
}
|
||||
} catch (err) {
|
||||
console.warn("ensureLlmModelLoaded failed", err);
|
||||
@@ -257,10 +263,15 @@
|
||||
error = "";
|
||||
|
||||
try {
|
||||
// concurrent flag mirrors settings.aiGpuConcurrency (brief A.1 #28).
|
||||
const concurrent = settings.aiGpuConcurrency === "parallel";
|
||||
if (settings.engine === "parakeet") {
|
||||
await invoke("load_parakeet_model", { name: "ctc-int8" });
|
||||
await invoke("load_parakeet_model", { name: "ctc-int8", concurrent });
|
||||
} else {
|
||||
await invoke("load_model", { size: settings.modelSize.toLowerCase() });
|
||||
await invoke("load_model", {
|
||||
size: settings.modelSize.toLowerCase(),
|
||||
concurrent,
|
||||
});
|
||||
}
|
||||
await refreshRuntimeCapabilities();
|
||||
modelReady = true;
|
||||
@@ -460,6 +471,7 @@
|
||||
const cleaned = await invoke("cleanup_transcript_text_cmd", {
|
||||
transcript: text,
|
||||
profileId: profilesStore.activeProfileId,
|
||||
preset: settings.llmPromptPreset,
|
||||
});
|
||||
markGenerationDone(true);
|
||||
return cleaned?.trim() ? cleaned.trim() : text;
|
||||
|
||||
Reference in New Issue
Block a user