docs: architecture map (initial 5-slice generation, 105 pages)
Five-slice navigable map of the entire codebase under
docs/architecture-map/. Each slice is a self-contained
breadcrumbed sub-tree:
01-frontend (16) Svelte/SvelteKit UI
02-tauri-runtime (26) src-tauri commands + lifecycle
03-audio-transcription (16) audio + transcription crates
04-llm-formatting-mcp (19) llm, ai-formatting, mcp, cloud
05-core-storage-hotkey-build core, storage, hotkey, workspace,
(26) CI, dev glue
Plus master README.md and data-flow-end-to-end.md tracing
audio bytes from microphone to FTS5 search to MCP read.
Generated by 5 parallel subagents on 2026/05/09 against
HEAD 3c47000. Each page has YAML frontmatter, file:line code
refs, sibling cross-links, plain-English summaries.
Aggregated debt surfaced (full lists in master README):
RB-08 macOS power assertion, schema head drift v14 vs v15,
VAD blocked on ort version conflict, streaming primitives
not wired into live.rs, no prompt versioning, MCP has no
auth, cloud-providers in-memory keystore, SettingsPage
2 484 LOC, commands/live.rs 1 737 LOC, dual theme system,
brand rename to Lumenote pending across the codebase.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jars
@@ -0,0 +1,93 @@
|
||||
---
|
||||
name: Anti-hallucination filter
|
||||
type: architecture-map-page
|
||||
slice: 04-llm-formatting-mcp
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# Anti-hallucination filter
|
||||
|
||||
> **Where you are:** [Architecture map](../README.md) → [LLM, Formatting, MCP](README.md) → Anti-hallucination
|
||||
|
||||
**Plain English summary.** Whisper hallucinates on silence. It produces things like `[blank_audio]`, `Thanks for watching!`, `♪♪♪`, or a single token cascading 8 times in a row. `is_hallucination` returns true on any of those, and the pipeline drops the segment entirely. Three independent passes — bracketed markers, exact-match subtitle leakage, and a token-repetition detector.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Crate: `magnotia-ai-formatting`
|
||||
- Path: `crates/ai-formatting/src/rule_based.rs:374`
|
||||
- LOC: ~50 for `is_hallucination` and the helper, plus ~70 lines of pattern tables
|
||||
- Public surface: `pub fn is_hallucination(text: &str) -> bool` (`crates/ai-formatting/src/rule_based.rs:374`)
|
||||
- External deps that matter: none — pure `str` work
|
||||
- Tauri command that calls this (slice 2, best guess): not called directly. Reaches Tauri only via `post_process_segments`'s anti-hallucination retain-loop (`crates/ai-formatting/src/pipeline.rs:43`).
|
||||
|
||||
## What's in here
|
||||
|
||||
### Three passes (the function body)
|
||||
|
||||
1. **Empty-after-trim → true.** A blank segment is, by convention, treated as a hallucination so it gets dropped.
|
||||
2. **Contains-match on `HALLUCINATION_MARKERS`** (`crates/ai-formatting/src/rule_based.rs:282`). Substring match on the lowercased trimmed text, so `[Music]`, `[MUSIC]`, and `[blank_audio]` all hit. Markers covered:
|
||||
- Bracketed annotations: `[blank_audio]`, `[blank audio]`, `[silence]`, `[music]`, `[applause]`, `[laughter]`, `[laughs]`, `[inaudible]`, `[background noise]`, `[sounds]`, `(music)`, `(silence)`, `(applause)`, `(laughter)`.
|
||||
- Musical notation: `♪`, `♫` — Whisper interprets sustained room tone as song.
|
||||
- The contains-match catches `♪♪♪ thanks for watching ♪♪♪` even though neither half alone is exact.
|
||||
3. **Exact-match on `HALLUCINATION_TRAIL_PHRASES`** (`crates/ai-formatting/src/rule_based.rs:312`). The full lowercased trimmed text must equal one of the phrases. Used for the YouTube / subtitle-training leakage that Whisper imports from its training data:
|
||||
- Minimalist false-positives on silence: `thank you.`, `thank you`, `thanks.`, `thanks`, `you.`, `you`, `bye.`, `bye`.
|
||||
- YouTube subtitle sign-offs: `thank you for watching.`, `thanks for watching!`, `thanks for watching, bye.`, `thanks for listening.`, `please subscribe.`, `please subscribe to our channel.`, `don't forget to subscribe.`, `don't forget to like and subscribe.`, `like and subscribe.`, `see you in the next video.`, `see you next time.`.
|
||||
- Subtitle-credit leakage: `subtitles by the amara.org community`, `subtitles by the`, `subtitled by`, `subtitles by`, `translated by`.
|
||||
- Non-English sign-offs: Japanese `ご視聴ありがとうございました`, `字幕作成者`, `字幕by`, `字幕`, Korean `mbc 뉴스 김수영입니다`. Lowercase exact-match consistency is preserved across scripts.
|
||||
|
||||
Exact-match is deliberate: a real sentence containing "thanks for the heads up on the migration" must pass.
|
||||
|
||||
4. **Consecutive-repetition detector** (`crates/ai-formatting/src/rule_based.rs:403`). Whisper's prompt-loop failure mode (ufal/whisper_streaming #161) is a single token cascading 5–10+ times. Threshold is 4 — caught at `REPETITION_RUN_THRESHOLD = 4` (`:358`). Three-in-a-row is common in natural speech ("no no no, that's wrong"), four-in-a-row almost never is. Case-insensitive token comparison.
|
||||
|
||||
### Provenance of the pattern lists
|
||||
|
||||
The trail phrases trace back to specific upstream issues:
|
||||
|
||||
- WhisperLive #185 and #246 — silence triggering `Thank you for watching` and similar.
|
||||
- ufal/whisper_streaming #121 — caption-dataset leakage on room tone.
|
||||
- ufal/whisper_streaming #161 — prompt-loop cascade.
|
||||
|
||||
Comments on each pattern list cite the exact source so a future contributor knows where each entry came from and why removing it might let the failure mode return.
|
||||
|
||||
### `has_consecutive_repetition` (`crates/ai-formatting/src/rule_based.rs:403`)
|
||||
|
||||
Linear pass. Walk whitespace-separated tokens, lowercase each, increment a `run` counter when the current matches the previous, reset when it does not. Return true the moment `run >= min_run`.
|
||||
|
||||
Tests at `crates/ai-formatting/src/rule_based.rs:551` cover:
|
||||
|
||||
- The cascade case: `"I I I I I I I I I"`, `"hello hello hello hello world"`, `"the the the the quick brown fox"`.
|
||||
- The case-insensitive case: `"Hello HELLO hello hello"`.
|
||||
- The legitimate-triple case: `"no no no, that's wrong"` (returns false — three is below threshold).
|
||||
- Alternating patterns: `"I am I am I am I am"` (returns false — never four-in-a-row).
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
text: &str
|
||||
→ trimmed = text.trim().to_lowercase()
|
||||
→ empty? → true
|
||||
→ for marker in HALLUCINATION_MARKERS:
|
||||
if trimmed.contains(marker) → true
|
||||
→ for phrase in HALLUCINATION_TRAIL_PHRASES:
|
||||
if trimmed == phrase → true
|
||||
→ has_consecutive_repetition(&trimmed, 4) → true if any run >= 4
|
||||
→ otherwise false
|
||||
|
||||
post_process_segments behaviour: segments.retain(|s| !is_hallucination(&s.text))
|
||||
— segments returning true are dropped from the output list entirely.
|
||||
```
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **Drop is permanent.** A segment removed by anti-hallucination is gone before any other filter or LLM cleanup runs. If a real-world transcript ever has a legitimate segment that exactly matches "Thanks." (e.g. in a meeting where someone said only "Thanks." in response to a question), it gets dropped. The exact-match policy on `HALLUCINATION_TRAIL_PHRASES` is the trade-off — substring-match would have a much larger false-positive rate.
|
||||
- **Threshold of 4 for repetition is conservative.** Some Whisper failures cascade to dozens of tokens, well past the threshold; the detector catches those easily. The risk is on the other side: legitimate four-in-a-row chants ("go go go go", "yes yes yes yes!") get dropped. Acceptable for dictation; would be wrong for music transcription, but Magnotia's scope is dictation.
|
||||
- **Multi-token phrase repetition is not yet detected.** "thank you thank you thank you thank you thank you" (five `thank you` in a row) does not trigger the detector — the comparison is per-token, not per n-gram. The test comment at `crates/ai-formatting/src/rule_based.rs:553` calls this out explicitly as a future enhancement requiring sliding n-gram matching.
|
||||
- **Non-English sign-offs are lowercased trimmed exact match.** A future Japanese ASR engine that uses different sign-off phrasing would slip through. Update the table when new ASR backends are added.
|
||||
- **No alphabet-class detection.** A long burst of mojibake (`<60> <20> <20> <20> <20>`) where every char is the replacement codepoint would not trigger any of the three passes. Whisper does not produce this in practice; if a future codec change made it possible, a fourth pass would be needed.
|
||||
- **`HALLUCINATION_MARKERS` is contains-match.** A real meeting transcript containing the phrase "the team will [music]play in October" would be dropped. Markers are deliberately niche enough that real text containing them is improbable; the cost of substring match is accepted.
|
||||
|
||||
## See also
|
||||
|
||||
- [Pipeline overview — anti-halluc as a drop step](formatting-pipeline.md)
|
||||
- [Filler removal and British English (sibling functions in same file)](formatting-filler-and-british.md)
|
||||
- [Slice README](README.md)
|
||||
Reference in New Issue
Block a user