Lumotia/docs/architecture-map/04-llm-formatting-mcp/formatting-anti-hallucination.md

name, type, slice, last_verified

name	type	slice	last_verified
Anti-hallucination filter	architecture-map-page	04-llm-formatting-mcp	2026/05/09

Anti-hallucination filter

Where you are: Architecture map → LLM, Formatting, MCP → Anti-hallucination

Plain English summary. Whisper hallucinates on silence. It produces things like [blank_audio], Thanks for watching!, ♪♪♪, or a single token cascading 8 times in a row. is_hallucination returns true on any of those, and the pipeline drops the segment entirely. Three independent passes — bracketed markers, exact-match subtitle leakage, and a token-repetition detector.

At a glance

• Crate: magnotia-ai-formatting
• Path: crates/ai-formatting/src/rule_based.rs:374
• LOC: ~50 for is_hallucination and the helper, plus ~70 lines of pattern tables
• Public surface: pub fn is_hallucination(text: &str) -> bool (crates/ai-formatting/src/rule_based.rs:374)
• External deps that matter: none — pure str work
• Tauri command that calls this (slice 2, best guess): not called directly. Reaches Tauri only via post_process_segments's anti-hallucination retain-loop (crates/ai-formatting/src/pipeline.rs:43).

What's in here

Three passes (the function body)

1. Empty-after-trim → true. A blank segment is, by convention, treated as a hallucination so it gets dropped.

2. Contains-match on HALLUCINATION_MARKERS (crates/ai-formatting/src/rule_based.rs:282). Substring match on the lowercased trimmed text, so [Music], [MUSIC], and [blank_audio] all hit. Markers covered:

   • Bracketed annotations: [blank_audio], [blank audio], [silence], [music], [applause], [laughter], [laughs], [inaudible], [background noise], [sounds], (music), (silence), (applause), (laughter).
   • Musical notation: ♪, ♫ — Whisper interprets sustained room tone as song.
   • The contains-match catches ♪♪♪ thanks for watching ♪♪♪ even though neither half alone is exact.

3. Exact-match on HALLUCINATION_TRAIL_PHRASES (crates/ai-formatting/src/rule_based.rs:312). The full lowercased trimmed text must equal one of the phrases. Used for the YouTube / subtitle-training leakage that Whisper imports from its training data:

   • Minimalist false-positives on silence: thank you., thank you, thanks., thanks, you., you, bye., bye.
   • YouTube subtitle sign-offs: thank you for watching., thanks for watching!, thanks for watching, bye., thanks for listening., please subscribe., please subscribe to our channel., don't forget to subscribe., don't forget to like and subscribe., like and subscribe., see you in the next video., see you next time..
   • Subtitle-credit leakage: subtitles by the amara.org community, subtitles by the, subtitled by, subtitles by, translated by.
   • Non-English sign-offs: Japanese ご視聴ありがとうございました, 字幕作成者, 字幕by, 字幕, Korean mbc 뉴스 김수영입니다. Lowercase exact-match consistency is preserved across scripts.

   Exact-match is deliberate: a real sentence containing "thanks for the heads up on the migration" must pass.

4. Consecutive-repetition detector (crates/ai-formatting/src/rule_based.rs:403). Whisper's prompt-loop failure mode (ufal/whisper_streaming #161) is a single token cascading 5–10+ times. Threshold is 4 — caught at REPETITION_RUN_THRESHOLD = 4 (:358). Three-in-a-row is common in natural speech ("no no no, that's wrong"), four-in-a-row almost never is. Case-insensitive token comparison.

Provenance of the pattern lists

The trail phrases trace back to specific upstream issues:

• WhisperLive #185 and #246 — silence triggering Thank you for watching and similar.
• ufal/whisper_streaming #121 — caption-dataset leakage on room tone.
• ufal/whisper_streaming #161 — prompt-loop cascade.

Comments on each pattern list cite the exact source so a future contributor knows where each entry came from and why removing it might let the failure mode return.

has_consecutive_repetition (crates/ai-formatting/src/rule_based.rs:403)

Linear pass. Walk whitespace-separated tokens, lowercase each, increment a run counter when the current matches the previous, reset when it does not. Return true the moment run >= min_run.

Tests at crates/ai-formatting/src/rule_based.rs:551 cover:

• The cascade case: "I I I I I I I I I", "hello hello hello hello world", "the the the the quick brown fox".
• The case-insensitive case: "Hello HELLO hello hello".
• The legitimate-triple case: "no no no, that's wrong" (returns false — three is below threshold).
• Alternating patterns: "I am I am I am I am" (returns false — never four-in-a-row).

Data flow

text: &str
  → trimmed = text.trim().to_lowercase()
  → empty? → true
  → for marker in HALLUCINATION_MARKERS:
       if trimmed.contains(marker) → true
  → for phrase in HALLUCINATION_TRAIL_PHRASES:
       if trimmed == phrase → true
  → has_consecutive_repetition(&trimmed, 4) → true if any run >= 4
  → otherwise false

post_process_segments behaviour: segments.retain(|s| !is_hallucination(&s.text))
  — segments returning true are dropped from the output list entirely.

Watch-outs

• Drop is permanent. A segment removed by anti-hallucination is gone before any other filter or LLM cleanup runs. If a real-world transcript ever has a legitimate segment that exactly matches "Thanks." (e.g. in a meeting where someone said only "Thanks." in response to a question), it gets dropped. The exact-match policy on HALLUCINATION_TRAIL_PHRASES is the trade-off — substring-match would have a much larger false-positive rate.
• Threshold of 4 for repetition is conservative. Some Whisper failures cascade to dozens of tokens, well past the threshold; the detector catches those easily. The risk is on the other side: legitimate four-in-a-row chants ("go go go go", "yes yes yes yes!") get dropped. Acceptable for dictation; would be wrong for music transcription, but Magnotia's scope is dictation.
• Multi-token phrase repetition is not yet detected. "thank you thank you thank you thank you thank you" (five thank you in a row) does not trigger the detector — the comparison is per-token, not per n-gram. The test comment at crates/ai-formatting/src/rule_based.rs:553 calls this out explicitly as a future enhancement requiring sliding n-gram matching.
• Non-English sign-offs are lowercased trimmed exact match. A future Japanese ASR engine that uses different sign-off phrasing would slip through. Update the table when new ASR backends are added.
• No alphabet-class detection. A long burst of mojibake (<EFBFBD> <20> <20> <20> <20>) where every char is the replacement codepoint would not trigger any of the three passes. Whisper does not produce this in practice; if a future codec change made it possible, a fourth pass would be needed.
• HALLUCINATION_MARKERS is contains-match. A real meeting transcript containing the phrase "the team will [music]play in October" would be dropped. Markers are deliberately niche enough that real text containing them is improbable; the cost of substring match is accepted.

See also

• Pipeline overview — anti-halluc as a drop step
• Filler removal and British English (sibling functions in same file)
• Slice README