---
name: Storage FTS5 transcript search
type: architecture-map-page
slice: 05-core-storage-hotkey-build
last_verified: 2026/05/09
---

# Storage FTS5 transcript search

> **Where you are:** [Architecture map](../README.md) → [Core, Storage, Hotkey, Build](README.md) → Storage FTS5 transcript search

**Plain English summary.** Full-text search over transcripts. SQLite's built-in FTS5 module gives us tokenisation, stemming, and BM25 ranking for free. The `transcripts_fts` virtual table is kept in lock-step with `transcripts` by three triggers; `search_transcripts` is the only public read path.

## At a glance

- Function: `search_transcripts(pool, query, limit) -> Result<Vec<TranscriptRow>>` at `crates/storage/src/database.rs:270`.
- Schema: virtual table + three triggers, all created by migration v2 and rebuilt in migration v9 (table rebuild).
- Indexed columns: `text`, `title`. Other columns (engine, model, language) are not searchable today.
- Tokenizer: `porter unicode61 remove_diacritics 2` — Porter stemmer, Unicode-normalised, diacritics stripped.
- Consumers: slice 2 history command (the search box on the History page).

## Schema

Created in migration v2, rebuilt in migration v9 against the new (FK-bearing) `transcripts` table:

```sql
CREATE VIRTUAL TABLE transcripts_fts USING fts5(
    text, title,
    content='transcripts',
    content_rowid='rowid',
    tokenize='porter unicode61 remove_diacritics 2'
);

CREATE TRIGGER transcripts_ai AFTER INSERT ON transcripts BEGIN
    INSERT INTO transcripts_fts(rowid, text, title)
    VALUES (new.rowid, new.text, COALESCE(new.title, ''));
END;

CREATE TRIGGER transcripts_ad AFTER DELETE ON transcripts BEGIN
    INSERT INTO transcripts_fts(transcripts_fts, rowid, text, title)
    VALUES ('delete', old.rowid, old.text, COALESCE(old.title, ''));
END;

CREATE TRIGGER transcripts_au AFTER UPDATE ON transcripts BEGIN
    INSERT INTO transcripts_fts(transcripts_fts, rowid, text, title)
    VALUES ('delete', old.rowid, old.text, COALESCE(old.title, ''));
    INSERT INTO transcripts_fts(rowid, text, title)
    VALUES (new.rowid, new.text, COALESCE(new.title, ''));
END;
```

`content='transcripts'` makes `transcripts_fts` a "contentless" external-content index — the FTS table stores tokens but not the data itself, which lives in `transcripts`. This halves on-disk size compared with a self-contained FTS5 table.

## Query shape — `crates/storage/src/database.rs:275`

```sql
SELECT t.<all columns>
FROM transcripts t
JOIN transcripts_fts fts ON fts.rowid = t.rowid
WHERE transcripts_fts MATCH ?
ORDER BY fts.rank
LIMIT ?
```

`fts.rank` is FTS5's negative BM25 score (most-relevant first). `MATCH` accepts standard FTS5 syntax: bare words AND together; `OR`, `NOT`, parentheses; quoted multi-word phrases; column-scoped queries (`text:foo title:bar`).

## Watch-outs

- **`MATCH` requires the leading word to be a real query, not a malformed token.** A user typing a single quote into the search box will produce a `SQL: malformed match` error from sqlite. The frontend should sanitise or fall back; today it does not, so the error propagates to the toast.
- **The triggers fire on every transcript insert / delete / update.** A bulk import of 1,000 transcripts walks the FTS path 1,000 times. Acceptable today (transcripts are user-authored, not imported). If bulk-import lands, consider a `transcripts_fts(transcripts_fts) VALUES('rebuild')` after the bulk insert.
- **Rebuild on schema change is expensive.** Migration v9 rebuilt the FTS table and re-tokenised every existing transcript. On a heavy user's database this can take several seconds. v9 ran inside a transaction so a failed rebuild rolls back cleanly.
- **`title` is `COALESCE`-d to `''`.** A null title becomes an empty FTS document, not a missing column.
- **No FTS over `manual_tags` or `llm_tags`.** Those columns hold comma-joined values and are filtered with `LIKE` queries today. A future migration could add them to the FTS5 index.

## Tests

No dedicated FTS test in `database.rs`. Coverage is via the migration tests (which assert the table and triggers exist post-migration) and via the integration test pattern `tx.execute("INSERT INTO transcripts ..."); search_transcripts("phrase")` used in the slice-2 command tests.

Worth adding a unit test that asserts BM25 ranking actually picks the better match — today a regression in the `tokenize=` clause would not break compile or migration tests.

## See also

- [Storage overview](storage-overview.md)
- [Schema and migrations](storage-schema-and-migrations.md) — v2 and v9 are the relevant points.
- [Transcripts CRUD](storage-crud-transcripts.md)