Phase B.9 audit of commit 1d71e8e (replace GBNF grammar with manual
brace-counting JSON-envelope extractor). Existing coverage:
* parse_string_array_trims_and_dedupes
* json_envelope_complete_detects_finished_{object,array}
* json_envelope_complete_ignores_braces_inside_strings
* json_envelope_complete_rejects_prefixes_and_trailing_text
* extract_json_envelope_skips_qwen_thinking_prefix (EMPTY think block)
* extract_json_envelope_handles_arrays_and_trailing_stop_text
Solid for the cases tested. One real residual.
The `_skips_qwen_thinking_prefix` regression uses an EMPTY <think></think>
block: `"<think>\n\n</think>\n\n{...}"`. Qwen3.5's reasoning mode emits
non-empty reasoning when enabled (and reasoning is a documented Qwen
feature, surfaced in the model name family the engine targets). The
naive "find the first '{' or '[' in the whole text" extractor breaks in
two ways once the reasoning is non-empty:
1. **JSON-looking text in thinking.** The model thinks out loud about
the schema: "the answer should look like {\"topic\":\"x\",\"intent\":\"y\"}".
The extractor sees the FIRST '{' (inside the reasoning), scans for
its matching '}', and returns the reasoning literal as the
envelope. The actual answer after </think> is dropped.
2. **Unbalanced braces in thinking.** The model writes "I wonder
about {something unfinished" inside <think>. The extractor starts
its brace-stack on that unbalanced '{', never finds a matching
'}', scans past </think> picking up the real answer's '{' (stack
now has TWO '}' targets), eventually finds one '}' which pops the
thinking's, then end of input — returns None. The actual answer
is lost entirely.
Fix: split on the FIRST `</think>` and scan only the substring after.
Anything before `</think>` is reasoning, anything after is the answer
proper. Falls back to the whole text when no `</think>` is present
(covers non-reasoning models AND the empty-thinking case the existing
test pins).
Backwards-compatible:
* Empty thinking — split_once returns ("", "\n\n{...}"); scan
finds the '{' and returns the answer. Existing test passes.
* No thinking tags at all — split_once returns None; fall back to
full text. Existing tests pass.
* Trailing stop tokens (`<|im_end|>` etc.) — unchanged behaviour;
they sit after the envelope and don't affect the scan.
New regression tests:
* extract_json_envelope_skips_thinking_block_with_json_looking_content
— thinking with a JSON literal followed by the real answer. Pre-fix
would return the thinking's literal; post-fix returns the answer.
* extract_json_envelope_survives_unbalanced_braces_in_thinking — the
unbalanced-brace-in-thinking case. Pre-fix returns None; post-fix
returns the real answer.
Verification:
* cargo test -p lumotia-llm --lib
→ 28/28 pass including the two new tests.
* cargo fmt --check → clean.
* cargo clippy -p lumotia-llm --all-targets -- -D warnings → clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Jake
401b6c3654