fix(cache): anchor semantic cache writes to cache_root so --out round-trips (#1990, #1991)

With `graphify extract --out <dir>`, the semantic cache write and read
sides disagreed on both location and key anchoring, breaking the cache
round-trip in two ways:

- Checkpoints (#1990): `_checkpoint_chunk` called `save_semantic_cache`
  with only `root=target`, so per-chunk recovery checkpoints were written
  under `<corpus>/graphify-out/` while the reader consulted
  `<out>/graphify-out/` — creating an unwanted graphify-out/ inside the
  analyzed source tree and making every interrupted run re-extract (and
  re-bill) completed chunks.

- Final save (#1991): cli.py passed `root=out_root`, so corpus-relative
  `source_file` paths resolved against the --out directory, failed
  `p.is_file()`, and every result group was silently skipped — the cache
  the reader would consult was never populated at all, with no warning.

Fix, following the split the AST cache already uses (#1774):

- `save_semantic_cache` and `check_semantic_cache` gain a `cache_root`
  parameter mirroring `load_cached`/`save_cached`: `root` stays the
  source-key anchor (content-hash keys, source_file resolution and
  relativization), `cache_root` selects where cache files live. Omitting
  it keeps `root` for both, so existing callers are unchanged.
- `extract_corpus_parallel` plumbs `cache_root` into `_checkpoint_chunk`.
- cli.py extract passes `root=target, cache_root=out_root` at the cache
  read, the checkpoint path, and the final save, and re-anchors the
  prune sweep's live hashes to `target` (keys anchored to out_root would
  mismatch every entry and sweep the fresh cache as orphaned).
- `save_semantic_cache` now warns loudly when every result group is
  dropped because its source_file does not resolve to a real file — the
  silent-0-writes failure mode #1991 asked to surface.

Regression tests cover: checkpoint written under cache_root (not the
corpus, no corpus graphify-out/ created), recovery read finds the
checkpoint via the same root/cache_root split, the final-save call shape
writes entries where the reader looks, the all-groups-dropped warning,
and backward compatibility when cache_root is omitted.

Fixes #1990
Fixes #1991
This commit is contained in:
shazeb
2026-07-18 19:09:53 +01:00
committed by safishamsi
parent 0224bcaea4
commit 08166306ba
4 changed files with 281 additions and 10 deletions
+10
View File
@@ -2044,6 +2044,7 @@ def extract_corpus_parallel(
max_concurrency: int = 4,
max_retry_depth: int = 3,
deep_mode: bool = False,
cache_root: "Path | None" = None,
) -> dict:
"""Extract a corpus in chunks, merging results.
@@ -2079,6 +2080,14 @@ def extract_corpus_parallel(
output_tokens. Failed chunks are logged to stderr and skipped — one bad
chunk does not abort the run.
``cache_root`` (when given) is where per-chunk checkpoint cache entries are
written, decoupled from ``root`` which anchors content-hash keys and
``source_file`` resolution — the same split the AST cache uses (#1774).
With ``--out``, cli.py passes the corpus as ``root`` and the output
directory as ``cache_root`` so checkpoints land where the recovery read
looks, instead of creating an unwanted ``graphify-out/`` inside the
analyzed source tree (#1990).
Accepts ``str`` paths as well as ``Path``; string entries are coerced up
front so packing/slicing helpers can rely on ``Path`` semantics (#1386).
"""
@@ -2154,6 +2163,7 @@ def extract_corpus_parallel(
result.get("edges", []),
result.get("hyperedges", []),
root=root,
cache_root=cache_root,
merge_existing=True,
allowed_source_files=allowed,
mode="deep" if deep_mode else None,