Jake
592b894790
Captures the 12 items from docs/code-review-2026-04-22.md that must land before v0.1 ships. One markdown file per issue with: severity, path:line, problem description, acceptance criteria, fix scope, and dependency graph. Split by severity: - 3 CRITICAL: live-session race, migration atomicity, transcript- profile FK - 9 MAJOR: monolith refactor, channel-fatality, capture worker join, runtime capabilities, macOS App Nap, decoder error prop, LLM prompt preflight, keystore thread-safety, hotkey device filter README.md indexes them with a fix-order dependency graph and a fish-shell script for bulk-converting to GitHub issues once `gh` CLI is installed and authed. Deferred step by user decision — markdown tracker is authoritative until then.
1.3 KiB
1.3 KiB
RB-10 MAJOR: LLM prompts not preflighted against context window
Severity: MAJOR
Path: crates/llm/src/lib.rs:143-166, :317-321
Source: 2026-04-22 code review
Labels: release-blocker, major, llm
Problem
generate tokenises and batches the full prompt at runtime. context_window_size hard-caps context at 8192 tokens. Long transcripts (a 30-minute dictation session is easily 4000–6000 tokens after segment joining) reach inference with prompts already bigger than the available context — causing late runtime failure instead of a controlled early-exit path.
Acceptance
- Before inference begins, the prompt token count is compared against the available context window (minus the expected response budget).
- Oversized prompts either (a) surface a typed error the caller can handle gracefully, or (b) are truncated with a logged warning — decide during the fix.
- Regression test: synthesise a transcript whose tokenised form exceeds 8192 tokens, assert the chosen behaviour (early error or truncated input).
Fix scope
Medium. Tokeniser access is already on the LLM path; the check is cheap. Decision work is in what to do when a prompt is too long (fail hard vs truncate).
Dependencies
- None — standalone fix.