feat(ai B.1 #15 + A.1 #28): cleanup presets and sequential-GPU guard
Some checks failed
check / cargo check (macos-latest) (push) Has been cancelled
check / cargo check (ubuntu-22.04) (push) Has been cancelled
check / cargo check (windows-latest) (push) Has been cancelled
check / svelte build + lint (push) Has been cancelled

Two new Settings → AI knobs that compose cleanly with what already
shipped (aiTier, LLM model, translator prompt framing).

**B.1 #15 — Named cleanup presets.** LlmPromptPreset enum
(Default / Email / Notes / Code) appends a short context hint onto
the CLEANUP_PROMPT just before generation. Presets shape tone and
structure ("email paragraph", "bulleted meeting notes", "preserve
technical terms") without licensing the content-editing the
translator-not-editor framing forbids. cleanup_transcript_text_cmd
now takes `preset: Option<String>` which runs through the new
LlmPromptPreset::parse (normalises aliases like "meeting-notes",
collapses unknown values to Default).

**A.1 #28 — Sequential-GPU guard.** New LocalEngine::unload drops
the backend + model_id so a subsequent load actually reclaims VRAM.
load_llm_model, load_model, and load_parakeet_model Tauri commands
grow an optional `concurrent: bool` argument. When concurrent is
Some(false), loading LLM first unloads whisper+parakeet, and vice
versa — prevents VRAM OOM on tight-VRAM setups. Default is the
previous parallel behaviour so nothing changes for multi-GB cards.
Transcribe-in-progress paths (transcribe_pcm, transcribe_file, live)
pass None, so mid-dictation model loads don't accidentally tear
down the LLM.

Settings UI (AI section):
- Cleanup preset segmented button + descriptive copy for each option.
- GPU concurrency segmented button with explicit trade-off text
  ("faster transitions vs fits in tight VRAM").

Frontend wiring:
- settings.llmPromptPreset flows from DictationPage's
  cleanupTranscriptIfEnabled into the Tauri command.
- settings.aiGpuConcurrency flows from both DictationPage (auto-load
  on record) and SettingsPage (manual load/unload buttons) as
  `concurrent: "parallel" === true` to the load commands.

Tests: three new preset cases in crates/ai-formatting/src/llm_client.rs
(parse aliases, suffix non-empty for non-default, default suffix
empty). All 139 existing lib tests still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-21 17:12:48 +01:00
parent a57da0feb5
commit ce2b4fdac6
12 changed files with 254 additions and 18 deletions

View File

@@ -239,7 +239,13 @@
try {
const status = await invoke("check_llm_model", { modelId: settings.llmModelId });
if (status?.downloaded && !status.loaded) {
await invoke("load_llm_model", { modelId: settings.llmModelId });
await invoke("load_llm_model", {
modelId: settings.llmModelId,
// Sequential-GPU guard (brief item A.1 #28): frees whisper
// before loading the LLM on tight-VRAM setups. "parallel"
// keeps both resident (default, safe on multi-GB cards).
concurrent: settings.aiGpuConcurrency === "parallel",
});
}
} catch (err) {
console.warn("ensureLlmModelLoaded failed", err);
@@ -257,10 +263,15 @@
error = "";
try {
// concurrent flag mirrors settings.aiGpuConcurrency (brief A.1 #28).
const concurrent = settings.aiGpuConcurrency === "parallel";
if (settings.engine === "parakeet") {
await invoke("load_parakeet_model", { name: "ctc-int8" });
await invoke("load_parakeet_model", { name: "ctc-int8", concurrent });
} else {
await invoke("load_model", { size: settings.modelSize.toLowerCase() });
await invoke("load_model", {
size: settings.modelSize.toLowerCase(),
concurrent,
});
}
await refreshRuntimeCapabilities();
modelReady = true;
@@ -460,6 +471,7 @@
const cleaned = await invoke("cleanup_transcript_text_cmd", {
transcript: text,
profileId: profilesStore.activeProfileId,
preset: settings.llmPromptPreset,
});
markGenerationDone(true);
return cleaned?.trim() ? cleaned.trim() : text;

View File

@@ -531,7 +531,11 @@
const modelId = selectedLlmModelId();
llmStatus = "Loading...";
try {
await invoke("load_llm_model", { modelId });
await invoke("load_llm_model", {
modelId,
// Sequential-GPU guard (brief item A.1 #28).
concurrent: settings.aiGpuConcurrency === "parallel",
});
await refreshLlmStatus();
await refreshGlobalLlmStatus(settings.aiTier);
} catch (err) {
@@ -740,7 +744,10 @@
engineStatus = "Loading...";
engineOk = false;
try {
await invoke("load_model", { size });
await invoke("load_model", {
size,
concurrent: settings.aiGpuConcurrency === "parallel",
});
await refreshRuntimeCapabilities();
engineOk = true;
engineStatus = `${settings.modelSize} model loaded`;
@@ -796,7 +803,10 @@
parakeetStatus = "Loading...";
parakeetOk = false;
try {
await invoke("load_parakeet_model", { name: "ctc-int8" });
await invoke("load_parakeet_model", {
name: "ctc-int8",
concurrent: settings.aiGpuConcurrency === "parallel",
});
await refreshRuntimeCapabilities();
parakeetOk = true;
parakeetStatus = "Model loaded";
@@ -1505,6 +1515,48 @@
{/if}
</p>
</div>
<!-- Cleanup preset (brief item B.1 #15). Shapes the tone /
structure of LLM cleanup output; composes on top of the
active profile's initial prompt. -->
<div class="mt-5">
<p class="text-[10px] font-medium text-text-tertiary uppercase tracking-wider mb-2">Cleanup preset</p>
<SegmentedButton
options={["default", "email", "notes", "code"]}
bind:value={settings.llmPromptPreset}
/>
<p class="text-[11px] text-text-tertiary mt-2">
{#if settings.llmPromptPreset === "email"}
Formats output as an email paragraph — tight sentences, no markdown, no auto-added greeting or signoff.
{:else if settings.llmPromptPreset === "notes"}
Formats action items as a bullet list led by imperative verbs; keeps prose informational sentences as prose.
{:else if settings.llmPromptPreset === "code"}
Preserves technical terms, variable names, file paths, and symbols exactly as spoken. No translation of identifiers.
{:else}
No preset — the active profile's prompt governs tone alone.
{/if}
</p>
</div>
<!-- Sequential-GPU guard (brief item A.1 #28). On a tight-
VRAM single-GPU system, loading LLM + Whisper together
can OOM. Sequential mode unloads one before loading the
other; adds reload latency between transcribe and
cleanup phases. -->
<div class="mt-5">
<p class="text-[10px] font-medium text-text-tertiary uppercase tracking-wider mb-2">GPU concurrency</p>
<SegmentedButton
options={["parallel", "sequential"]}
bind:value={settings.aiGpuConcurrency}
/>
<p class="text-[11px] text-text-tertiary mt-2">
{#if settings.aiGpuConcurrency === "sequential"}
On tight-VRAM cards (≤6 GB), loading Whisper + LLM together OOMs. Sequential mode frees the other model before loading; adds a small reload pause between transcribe and cleanup.
{:else}
Both models stay resident in GPU memory. Faster transitions, but needs enough VRAM to hold both at once.
{/if}
</p>
</div>
</div>
{/if}
</div>

View File

@@ -61,6 +61,8 @@ const defaults: SettingsState = {
fontSize: 14,
aiTier: "cleanup",
llmModelId: null,
llmPromptPreset: "default",
aiGpuConcurrency: "parallel",
saveAudio: false,
outputFolder: "",
globalHotkey: "Ctrl+Shift+R",

View File

@@ -12,6 +12,8 @@ export type WhisperModelSize =
| "Distil-L";
export type AiTier = "off" | "cleanup" | "tasks";
export type LlmModelIdStr = "qwen3_1_7b" | "qwen3_4b_instruct_2507" | "qwen3_14b";
export type LlmPromptPreset = "default" | "email" | "notes" | "code";
export type AiGpuConcurrency = "parallel" | "sequential";
export type TaskBucket = "inbox" | "today" | "soon" | "later";
export type ToastSeverity = "info" | "success" | "warn" | "error";
@@ -47,6 +49,8 @@ export interface SettingsState {
fontSize: number;
aiTier: AiTier;
llmModelId: LlmModelIdStr | null;
llmPromptPreset: LlmPromptPreset;
aiGpuConcurrency: AiGpuConcurrency;
saveAudio: boolean;
outputFolder: string;
globalHotkey: string;