mirror of
https://github.com/safishamsi/graphify.git
synced 2026-09-26 23:45:44 +00:00
fix(cache): resolve FileSlice via unit_path in checkpoint allowlist (#1870)
The #1757 batch-scoping followup built the per-chunk allowlist by reading FileSlice.rel, which does not exist (a FileSlice carries its parent file in .path). So every chunk containing a sliced oversized document leaked the FileSlice object into the allowlist, save_semantic_cache raised TypeError on Path(FileSlice), and the best-effort except swallowed it: extraction finished but those chunks were never checkpointed, so a re-run or a crash/rate-limit resume re-billed them. Resolve each unit through the canonical unit_path() helper so a slice maps to its parent file. Adds a regression test that slices a real oversized .md and asserts the checkpoint writes without swallowing a TypeError. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
961b78e57a
commit
cfc7cf2c93
+6
-3
@@ -1918,9 +1918,12 @@ def extract_corpus_parallel(
|
||||
# (#1757). The model can attribute a node's source_file to another
|
||||
# corpus file; without this bound, that stray node would clobber the
|
||||
# other file's complete cache entry (or, with merge_existing, pollute
|
||||
# it). A FileSlice reports its file via `.rel`; a bare Path is the
|
||||
# relative source_file itself.
|
||||
allowed = [getattr(item, "rel", None) or item for item in chunk]
|
||||
# it). Use unit_path so a FileSlice (one slice of an oversized doc)
|
||||
# resolves to its parent file; a bare Path passes through. (#1870: the
|
||||
# old `.rel` attribute does not exist on FileSlice, so every sliced
|
||||
# chunk leaked the FileSlice object into the allowlist and the write
|
||||
# raised TypeError, silently defeating the checkpoint.)
|
||||
allowed = [unit_path(item) for item in chunk]
|
||||
_scs(
|
||||
result.get("nodes", []),
|
||||
result.get("edges", []),
|
||||
|
||||
Reference in New Issue
Block a user