feat(ai-formatting A.1 #26): detect prompt-loop repetition cascades

ufal/whisper_streaming #161 documents the classic Whisper streaming
failure: on ambiguous audio the model falls into a prompt loop,
cascading a single token for 10+ words ("I I I I I I I I I I I…").
The chunk-boundary duplicate detector in live.rs doesn't catch
this — the repeat is within a single chunk, and the text is
technically novel so FTS is happy to keep it.

Fold the detection into is_hallucination as a third pass (after
HALLUCINATION_MARKERS substring-match and HALLUCINATION_TRAIL_PHRASES
exact-match). has_consecutive_repetition walks the token stream
(whitespace-split, lowercased) and returns true when any run of
≥REPETITION_RUN_THRESHOLD (4) identical tokens is found.

Threshold chosen deliberately: three consecutive matches appear in
normal speech ("no no no, that's wrong"), four almost never does.
Tests pin both sides — "I I I I I" detected, "no no no" allowed,
alternating patterns ("I am I am I am I am") allowed regardless of
length.

Phrase-level repetition ("thank you thank you thank you thank you")
is a documented companion failure mode but needs a sliding n-gram
matcher — deferred with a code comment flagging it.

No caller changes: post_process_segments already drops
is_hallucination hits when anti_hallucination is enabled.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

crates/ai-formatting/src/rule_based.rs

@@ -349,9 +349,17 @@ static HALLUCINATION_TRAIL_PHRASES: &[&str] = &[
     "mbc 뉴스 김수영입니다",
 ];
 
+/// Minimum run length for the token-repetition detector (brief item
+/// A.1 #26). Whisper's prompt-loop failure mode (ufal #161) typically
+/// produces 5–10+ consecutive identical tokens; requiring 4 catches
+/// those cleanly while leaving natural dialogue alone — three-in-a-row
+/// is common speech ("no no no, that's wrong"), four-in-a-row almost
+/// never is.
+const REPETITION_RUN_THRESHOLD: usize = 4;
+
 /// Returns true if a segment's text looks like a hallucination.
 ///
-/// Two passes:
+/// Three passes:
 /// - **Contains-match on HALLUCINATION_MARKERS** — catches bracketed
 ///   and musical markers even when Whisper surrounds them with other
 ///   noise ("♪♪♪ thanks for watching ♪♪♪").
@@ -359,6 +367,10 @@ static HALLUCINATION_TRAIL_PHRASES: &[&str] = &[
 ///   well-documented subtitle-training leakage without false-positiving
 ///   on legitimate dialogue that happens to mention "thanks" or
 ///   "subscribe" mid-sentence.
+/// - **Consecutive-repetition detector** — Whisper occasionally enters
+///   a prompt-loop where a single token cascades for dozens of words.
+///   Flagging it here lets the existing anti_hallucination pipeline
+///   drop the chunk rather than emitting "I I I I I I I I I …".
 pub fn is_hallucination(text: &str) -> bool {
     let trimmed = text.trim().to_lowercase();
     if trimmed.is_empty() {
@@ -374,6 +386,38 @@ pub fn is_hallucination(text: &str) -> bool {
             return true;
         }
     }
+    if has_consecutive_repetition(&trimmed, REPETITION_RUN_THRESHOLD) {
+        return true;
+    }
+    false
+}
+
+/// Returns true when `text` contains at least `min_run` consecutive
+/// identical whitespace-separated tokens (case-insensitive).
+///
+/// Detects the prompt-loop failure mode that Whisper falls into on
+/// ambiguous audio (ufal #161) without flagging normal triple-repeats
+/// that appear in everyday speech ("no no no, that's wrong"). The
+/// threshold is deliberately conservative — four-in-a-row is almost
+/// never organic.
+fn has_consecutive_repetition(text: &str, min_run: usize) -> bool {
+    if min_run < 2 {
+        return false;
+    }
+    let mut run: usize = 1;
+    let mut last: Option<String> = None;
+    for token in text.split_whitespace() {
+        let token_lower = token.to_lowercase();
+        if last.as_deref() == Some(token_lower.as_str()) {
+            run += 1;
+            if run >= min_run {
+                return true;
+            }
+        } else {
+            run = 1;
+            last = Some(token_lower);
+        }
+    }
     false
 }
 
@@ -499,4 +543,27 @@ mod tests {
         assert!(!is_hallucination("Thanks for the heads up on the migration"));
         assert!(!is_hallucination("Please subscribe to the RSS feed and tell me when it updates"));
     }
+
+    #[test]
+    fn is_hallucination_detects_prompt_loop_repetition() {
+        // ufal #161: Whisper prompt-loop cascade, the classic
+        // streaming failure mode. Single-token runs only for now —
+        // multi-token phrase repetition ("thank you thank you thank
+        // you...") is a documented companion failure mode but needs
+        // sliding n-gram matching, which is a future enhancement.
+        assert!(is_hallucination("I I I I I I I I I"));
+        assert!(is_hallucination("hello hello hello hello world"));
+        assert!(is_hallucination("the the the the quick brown fox"));
+        // Case-insensitive.
+        assert!(is_hallucination("Hello HELLO hello hello"));
+    }
+
+    #[test]
+    fn is_hallucination_allows_natural_triple_repeats() {
+        // Threshold is 4, so natural speech patterns pass.
+        assert!(!is_hallucination("no no no, that's wrong"));
+        assert!(!is_hallucination("do do do the thing"));
+        // Alternating patterns never trigger regardless of length.
+        assert!(!is_hallucination("I am I am I am I am"));
+    }
 }