Lumotia/docs/architecture-map/04-llm-formatting-mcp/llm-cleanup-text.md

name, type, slice, last_verified

name	type	slice	last_verified
LLM cleanup_text surface	architecture-map-page	04-llm-formatting-mcp	2026/05/09

LLM cleanup_text surface

Where you are: Architecture map → LLM, Formatting, MCP → cleanup_text

Plain English summary. cleanup_text is the freeform LLM call. Given any system prompt and a transcript, it returns the LLM's cleaned version as plain text. No grammar constraint, no JSON parsing. The formatting crate's llm_client::cleanup_text is the only canonical caller and supplies the prompt-injection-hardened system prompt.

At a glance

• Crate: lumotia-llm
• Path: crates/llm/src/lib.rs:232
• LOC: 21 lines for this method
• Public surface: pub fn cleanup_text(&self, system_prompt: &str, transcript: &str) -> Result<String, EngineError>
• External deps that matter: none beyond what LlmEngine::generate already pulls in
• Tauri command that calls this (slice 2, best guess): not called directly by Tauri. The chain is commands::llm::cleanup_transcript_text_cmd (src-tauri/src/commands/llm.rs:363) → lumotia_ai_formatting::llm_cleanup_text (src-tauri/src/commands/llm.rs:395) → this method. Also reached from commands::transcription::* and commands::live::* via the formatting pipeline at crates/ai-formatting/src/pipeline.rs:84.

What's in here

pub fn cleanup_text(&self, system_prompt: &str, transcript: &str) -> Result<String, EngineError>

Behaviour:

1. If transcript.trim().is_empty(), return Ok(String::new()) immediately. No model touch.
2. Borrow the loaded model via loaded_model_arc(). Returns EngineError::NotLoaded if no model is loaded.
3. Render a chat prompt with two messages — ("system", system_prompt) and ("user", transcript) — through render_chat_prompt, which applies the model's tokenizer-bundled chat template and falls back to ChatML if missing.
4. Call generate with:
   • max_tokens: 1024
   • temperature: 0.0
   • stop_sequences: ["<|im_end|>", "<|im_end_of_text|>"]
   • grammar: None

Returns the trimmed, post-stop-sequence-truncated output verbatim.

Data flow

(system_prompt: &str, transcript: &str)
  → empty-transcript short-circuit (returns "")
  → loaded_model_arc()  (NotLoaded error if absent)
  → render_chat_prompt([(system, system_prompt), (user, transcript)])
  → generate(prompt, GenerationConfig { max_tokens: 1024, temp: 0.0, stops: [<|im_end|>, <|im_end_of_text|>], grammar: None })
  → trimmed String

No JSON parse, no GBNF, no closed set. The contract with the caller is "do whatever the system prompt tells you to do".

Prompts and grammars

cleanup_text itself does not own a prompt — the system prompt is a parameter. The canonical caller is lumotia_ai_formatting::llm_client::cleanup_text which composes:

CLEANUP_PROMPT  +  format_dictionary_suffix(dictionary_terms)  +  preset.suffix()

See formatting-llm-cleanup-bridge.md for the full composition logic and prompt-injection-hardening rationale, and llm-prompts-and-grammars.md for the prompt text in full.

Watch-outs

• No grammar means the model can output anything. This is by design — cleanup is freeform — but it means the prompt is the only line of defence against prompt injection. The llm_client bridge handles this; do not call LlmEngine::cleanup_text from anywhere else without porting that hardening.
• max_tokens: 1024 is a hard ceiling on output length. A long dictation that compresses well is fine; one that compresses poorly will be cut off mid-sentence. The pipeline does not detect or retry truncated output. If we ever get reports of mid-sentence drops on long transcripts, raise this constant in tandem with the preflight cap.
• temperature: 0.0 plus the fixed seed makes output deterministic for a given prompt and loaded model. Switching tier (e.g. 4B → 9B) will change the output even with the same input.
• Stop sequences are Qwen-specific. Both <|im_end|> and <|im_end_of_text|> are emitted by Qwen3.5 / 3.6 chat templates. A future model from a different family would need its own stop set.
• Empty transcript returns empty string, not an error. Callers that want to distinguish "nothing to clean" from "model not loaded" should check is_loaded() first.

See also

• LLM engine
• LLM cleanup bridge in formatting crate
• Prompts and grammars catalogue
• Slice README