---
name: Correction learning (HITL → custom dictionary)
type: architecture-map-page
slice: 04-llm-formatting-mcp
last_verified: 2026/05/09
---

# Correction learning (HITL → custom dictionary)

> **Where you are:** [Architecture map](../README.md) → [LLM, Formatting, MCP](README.md) → Correction learning

**Plain English summary.** When a user edits a transcript, `extract_corrections` infers which words were swapped and surfaces them as candidates for the user's custom-vocabulary dictionary. Conservative — only short, low-edit-distance, non-existing-term substitutions count. Large rewrites are ignored. The result feeds back into the LLM cleanup prompt's dictionary suffix.

## At a glance

- Crate: `lumotia-ai-formatting`
- Path: `crates/ai-formatting/src/correction_learning.rs`
- LOC: 229
- Public surface:
  - `pub fn extract_corrections(original_text: &str, edited_text: &str, existing_terms: &[String]) -> Vec<String>` (`crates/ai-formatting/src/correction_learning.rs:131`)
  - Re-exported at crate root as `lumotia_ai_formatting::extract_corrections` (`crates/ai-formatting/src/lib.rs:7`).
- External deps that matter: none — pure CPU, pure stdlib.
- Tauri command that calls this (slice 2, best guess): `commands::profiles.rs:14` imports it; the actual call is at `src-tauri/src/commands/profiles.rs:165`. The wrapper command is named in `profiles.rs` (likely `learn_corrections_*_cmd` — check slice 2's profiles page).

## What's in here

### Constants (`crates/ai-formatting/src/correction_learning.rs:3`)

- `MAX_REWRITE_RATIO = 0.5` — if more than half the original words got substituted, treat as a rewrite, not corrections.
- `MIN_CORRECTION_LEN = 3` — corrected words shorter than 3 chars are dropped (avoids noise on "a", "of", "to").
- `MAX_DISTANCE_RATIO = 0.65` — Levenshtein distance over max-length must be at most 0.65; anything more is a different word, not a correction.
- `MAX_CORRECTIONS_PER_EDIT = 8` — cap on how many correction candidates a single edit can yield.

### Levenshtein distance (`crates/ai-formatting/src/correction_learning.rs:8`)

Standard two-row dynamic programming distance over `Vec<char>`. Used to decide whether `Shunade → Sinead` is "phonetic correction" (low distance ratio) or "different word" (high).

### `tokenize` and `trim_non_word_edges` (`:29`, `:33`)

Whitespace split, then strip non-alphanumeric edges (keeping `_` for code). `"hello,"` → `"hello"`. Empty results filtered out.

### `find_edited_region` (`:42`)

If the edit is significantly larger than the original (`field_value.len() > original.len() * 1.5`), the user might have appended notes around the original. The function tries to locate the region in the edited text that aligns with the original via a sliding-window word-match, returning just that region. If the alignment score is below 30% of the window size, the function gives up and returns the full edited text. The original-text-contained-as-substring case short-circuits: if the original appears verbatim, no corrections were made — return the original.

This guards against the failure mode where a user dictates a paragraph, then later appends 500 words of new content; we do not want to "learn" every word of the appended content as a correction.

### `find_substitutions` (`:81`)

LCS-based alignment between original and edited word lists. Walks back through the DP table to produce an aligned `Vec<(Option<String>, Option<String>)>`. A pair `(Some(orig), None)` is a deletion; `(None, Some(edit))` is an insertion. A `(deletion, insertion)` adjacent pair is a *substitution* — the only shape this function emits as a `(orig, corrected)` tuple.

The substitution-detection condition (`:118`) is precise: `(Some(orig_word), None, None, Some(corrected_word))` across two consecutive aligned pairs. Adjacent inserts and adjacent deletes do not become substitutions.

### `extract_corrections` (`:131`)

The pipeline:

1. **Empty / unchanged guard.** Return empty when either side is whitespace-only or when texts are identical.
2. **Locate edited region** via `find_edited_region`. If the region equals the original, return empty (nothing was edited in this region).
3. **Tokenise both sides.** Empty token lists → return empty.
4. **Find substitutions** via the LCS aligner.
5. **Rewrite-ratio gate.** If substitutions count exceeds `MAX_REWRITE_RATIO` of original word count, return empty. Catches the "user threw away the transcript and rewrote it" case.
6. **Build the existing-terms set** (lowercased) for the dedupe step.
7. **For each substitution**:
   - Skip if `orig.lower() == corrected.lower()` (same word, just casing).
   - Skip if `corrected.len() < MIN_CORRECTION_LEN`.
   - Skip if `corrected.lower()` is already in `existing_terms`.
   - Skip if we already produced this correction in this batch.
   - Compute edit distance, skip if `distance / max(orig.len, corrected.len) > MAX_DISTANCE_RATIO`.
   - Push the corrected word (preserving its casing) onto the result.
   - Stop at `MAX_CORRECTIONS_PER_EDIT`.

Returns `Vec<String>` of new terms the caller should consider adding to the user's custom-vocabulary dictionary.

### Tests (`crates/ai-formatting/src/correction_learning.rs:193`)

- `extracts_phonetic_corrections_for_profile_learning` (`:197`) — `"Shunade"` → `"Sinead"` is detected.
- `ignores_large_rewrites` (`:208`) — total rewrite returns empty.
- `skips_terms_already_in_profile_dictionary` (`:219`) — `"Corble"` → `"CORBEL"` is dropped because `CORBEL` is already in `existing_terms`.

## Data flow

```
(original_text, edited_text, existing_terms)
  → empty / unchanged guard
  → edited_region = find_edited_region(original_text, edited_text)
       (handles "user appended new content" case)
  → tokenise both
  → substitutions = find_substitutions(original_words, edited_words)  (LCS-based)
  → rewrite-ratio gate (drop if > 50% substituted)
  → for each substitution:
       skip same-casing, skip too-short, skip existing, skip duplicate-this-batch,
       skip if distance/max_len > 0.65
  → cap at 8 results
  → return Vec<String> of corrected words (case preserved)

caller (Tauri profiles command) merges these into the user's profile.dictionary,
which then appears as PostProcessOptions.dictionary_terms in the next pipeline run.
```

## Watch-outs

- **Conservative by design.** Many false positives is worse than a few false negatives, because every "learned" word goes into the cleanup prompt and biases the LLM. The four constants combine to keep the noise floor low. Tuning them down (looser distance ratio, smaller min length) is safe to experiment with but should be A/B-tested on real edits.
- **Levenshtein on words.** Distance is computed character-by-character on the lowercased forms, not phoneme-by-phoneme. `Shunade` → `Sinead` (distance 4, max-len 7, ratio 0.57) makes it under the 0.65 cap. `Run` → `Walk` (distance 4, max-len 4, ratio 1.0) does not.
- **Casing is preserved in the output.** The dedupe set is lowercased (so two corrections that differ only in casing collapse to one), but the kept variant is whatever case the user typed. If a user types `Sinead` and `SINEAD` in the same edit, the first one wins.
- **`existing_terms` should pass the user's full custom dictionary.** The function lowercases internally; the caller can pass any casing. Filling this in correctly is the difference between "we keep suggesting CORBEL the user already added" and "we only suggest new terms".
- **Substitutions only — no insertions or deletions.** A user inserting a new word that the ASR did not pick up at all is not a "correction" to anything; it is content. The aligner correctly leaves it as `(None, Some(_))` and the subsequent matcher ignores those.
- **No language awareness.** The function does not know about morphological variants. `"organize" → "organise"` is correctly detected as a 1-character substitution (because `to_british_english` would have done it earlier in the pipeline), but if a user edits a transcript Lumotia did not Britanise, the function will happily learn `organise` as a new term — duplicating work.
- **`find_edited_region` heuristic floor is 30%.** Below that match-score, the function gives up alignment and returns the whole edited text, which is then very likely to fail the rewrite-ratio gate. Effectively a graceful "I don't know what changed, skip it" path.

## See also

- [LLM cleanup bridge — where dictionary terms get used](formatting-llm-cleanup-bridge.md)
- [Pipeline overview — where dictionary terms enter the pipeline](formatting-pipeline.md)
- [Slice README](README.md)