Jake
592b894790
Captures the 12 items from docs/code-review-2026-04-22.md that must land before v0.1 ships. One markdown file per issue with: severity, path:line, problem description, acceptance criteria, fix scope, and dependency graph. Split by severity: - 3 CRITICAL: live-session race, migration atomicity, transcript- profile FK - 9 MAJOR: monolith refactor, channel-fatality, capture worker join, runtime capabilities, macOS App Nap, decoder error prop, LLM prompt preflight, keystore thread-safety, hotkey device filter README.md indexes them with a fix-order dependency graph and a fish-shell script for bulk-converting to GitHub issues once `gh` CLI is installed and authed. Deferred step by user decision — markdown tracker is authoritative until then.
25 lines
1.3 KiB
Markdown
25 lines
1.3 KiB
Markdown
# RB-10 MAJOR: LLM prompts not preflighted against context window
|
||
|
||
**Severity:** MAJOR
|
||
**Path:** `crates/llm/src/lib.rs:143-166`, `:317-321`
|
||
**Source:** [2026-04-22 code review](../code-review-2026-04-22.md)
|
||
**Labels:** release-blocker, major, llm
|
||
|
||
## Problem
|
||
|
||
`generate` tokenises and batches the full prompt at runtime. `context_window_size` hard-caps context at 8192 tokens. Long transcripts (a 30-minute dictation session is easily 4000–6000 tokens after segment joining) reach inference with prompts already bigger than the available context — causing late runtime failure instead of a controlled early-exit path.
|
||
|
||
## Acceptance
|
||
|
||
- Before inference begins, the prompt token count is compared against the available context window (minus the expected response budget).
|
||
- Oversized prompts either (a) surface a typed error the caller can handle gracefully, or (b) are truncated with a logged warning — decide during the fix.
|
||
- Regression test: synthesise a transcript whose tokenised form exceeds 8192 tokens, assert the chosen behaviour (early error or truncated input).
|
||
|
||
## Fix scope
|
||
|
||
Medium. Tokeniser access is already on the LLM path; the check is cheap. Decision work is in what to do when a prompt is too long (fail hard vs truncate).
|
||
|
||
## Dependencies
|
||
|
||
- None — standalone fix.
|