The claude-cli backend passed the extraction schema via --system-prompt with
only the raw file dump in the user turn, assuming a replacement system prompt
is the model's sole authority. Claude Code >= ~2.1 (verified on 2.1.197) does
not honour that: it still layers in the local coding-agent context
(CLAUDE.md/AGENTS.md in cwd, skills, MCP) and, given a user turn that is just a
file with no request, replies conversationally ("I see the file, but there's no
actual request attached"). That prose parses to zero nodes/edges, so
_response_is_hollow flags it as truncation and the adaptive-retry path bisects
the chunk indefinitely (94 -> 47 -> 23 -> ...), never converging and never
writing graph.json.
Move the full extraction schema plus an explicit imperative into the user turn
and drop --system-prompt, so the CLI emits the JSON object directly. The
<untrusted_source> prompt-injection guardrails are carried verbatim; model
override, --add-dir image handling, timeout, and token accounting are untouched.
Three compounding bugs caused ~30-50% of semantic chunks to come back
as 'hollow responses' on the claude-cli backend, triggering adaptive
bisection that doubled or tripled the number of subprocess calls.
Root causes
-----------
1. _parse_llm_json only stripped markdown fences when raw.startswith('```').
Claude frequently prepends a short preamble before the fence
('Here are the extracted entities:\n\n```json\n{...}```'), making
the check fail. json.loads then drops the chunk. Each bisected half
may exhibit the same failure, so cost compounds.
2. _call_claude_cli used --append-system-prompt, which layers graphify's
extraction prompt on top of Claude Code's default interactive-agent
prompt ('use markdown formatting', 'output text to communicate with
the user'). The conflicting instructions explain ~50% of the
preambles and fences from (1). Switching to --system-prompt (replace)
eliminates the conflict at the source.
3. claude-cli defaults to Opus, which is overkill for the structured
JSON extraction graphify performs. New GRAPHIFY_CLAUDE_CLI_MODEL env
var lets users opt into haiku / sonnet for big builds. Default
behaviour unchanged when the env var is unset.
Fix
---
- Robust _parse_llm_json: strips fences regardless of position, with a
balanced-brace fallback that scans for the first complete JSON object
in the response. Handles preambles, trailing prose, prose-wrapped
JSON without fences. Diagnostic log on terminal failure includes the
first 200 chars of the response.
- _call_claude_cli switches to --system-prompt.
- _call_claude_cli respects GRAPHIFY_CLAUDE_CLI_MODEL when set.
Tests (tests/test_llm_parser.py)
--------------------------------
- The four PR-body failure modes: preamble+fence, prose+JSON, raw JSON,
total refusal.
- Bonus: uppercase fence tag, unclosed fence, empty response.
- argv shape: --system-prompt present, --append-system-prompt absent.
- argv shape: --model added iff GRAPHIFY_CLAUDE_CLI_MODEL is set.
19/19 tests pass (9 pre-existing in test_claude_cli_backend.py +
10 new). Verified end-to-end on a 800-file repo: 0 hollow responses
after, vs ~30-50% before; output tokens -93%; wall time 44 min -> 4 min.
Jeisson