fix(cache): resolve FileSlice via unit_path in checkpoint allowlist (#1870)

The #1757 batch-scoping followup built the per-chunk allowlist by reading
FileSlice.rel, which does not exist (a FileSlice carries its parent file
in .path). So every chunk containing a sliced oversized document leaked
the FileSlice object into the allowlist, save_semantic_cache raised
TypeError on Path(FileSlice), and the best-effort except swallowed it:
extraction finished but those chunks were never checkpointed, so a
re-run or a crash/rate-limit resume re-billed them.

Resolve each unit through the canonical unit_path() helper so a slice
maps to its parent file. Adds a regression test that slices a real
oversized .md and asserts the checkpoint writes without swallowing a
TypeError.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
safishamsi
2026-07-14 00:25:34 +01:00
co-authored by Claude Opus 4.8
parent 961b78e57a
commit cfc7cf2c93
4 changed files with 49 additions and 4 deletions
+6 -3
View File
@@ -1918,9 +1918,12 @@ def extract_corpus_parallel(
# (#1757). The model can attribute a node's source_file to another
# corpus file; without this bound, that stray node would clobber the
# other file's complete cache entry (or, with merge_existing, pollute
# it). A FileSlice reports its file via `.rel`; a bare Path is the
# relative source_file itself.
allowed = [getattr(item, "rel", None) or item for item in chunk]
# it). Use unit_path so a FileSlice (one slice of an oversized doc)
# resolves to its parent file; a bare Path passes through. (#1870: the
# old `.rel` attribute does not exist on FileSlice, so every sliced
# chunk leaked the FileSlice object into the allowlist and the write
# raised TypeError, silently defeating the checkpoint.)
allowed = [unit_path(item) for item in chunk]
_scs(
result.get("nodes", []),
result.get("edges", []),