mirror of
https://github.com/safishamsi/graphify.git
synced 2026-07-15 11:57:12 +00:00
tmaeder
f88567b114
Adaptive retry only recovered from `finish_reason="length"` (output truncation). It did not handle the other shape of overflow: the API rejecting the prompt outright with a 400 because the input plus `max_completion_tokens` doesn't fit in the model's context window. This shows up immediately on local OpenAI-compatible servers (LM Studio, llama.cpp, vLLM) where the default context is small (4K-32K) and a 60K-token chunk packed for cloud Kimi/Claude blows past it. Without retry the whole chunk fails with no output, even though the two halves would each fit cleanly. Catch a heuristic set of context-overflow exception messages, classify them as the same kind of recoverable failure as `finish_reason="length"`, and split-recurse on the same path. Single- file overflow returns an empty fragment so the rest of the corpus keeps running. Unrelated errors (rate limit, auth, etc.) still propagate. Tested with qwen3.5-9b on LM Studio (32K ctx) against a 215-file corpus where chunks 4-12 of 12 previously failed; with this change the overflowing chunks self-heal by splitting in half.