lee/graphify
1
0
Fork 0
mirror of https://github.com/safishamsi/graphify.git synced 2026-09-23 05:55:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(extract,watch): C# assembly-aware partial merge, incremental edge preservation, extract crash recovery; bump to 0.9.33

Browse Source 
#2411 (thanks @JensD-git): key the C# partial-class merge on assembly
(nearest ancestor .csproj/.fsproj/.vbproj) in addition to namespace and
name, so same-name partial classes in different assemblies stay distinct
while genuine partial halves within one project still merge. Fixes a
0.9.32 regression from #2332.

#2437/#2438 (thanks @aryanbonigala, builds on PR #2439): incremental
rebuilds no longer drop member-call and indirect_call edges from a
changed file into an unchanged target. Re-resolution now sees the
unchanged corpus (nodes, contains/method edges, and _callable markers,
which now persist to graph.json like _origin); edges to a genuinely
removed target are still evicted.

#2444/#2445 (thanks @Baziar, builds on PRs #2461/#2458): a
BrokenProcessPool triggers the sequential fallback instead of being
swallowed per future, a failed worker file is retried sequentially
rather than merged as empty, and a whole-pass AST failure on a fresh
build exits non-zero instead of writing a zero-node graph
(--allow-partial opts into a best-effort partial graph).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
safishamsi
2026-08-05 01:41:12 +01:00
co-authored by Claude Opus 4.8
parent a426d63414
commit 4e7e6b1f7e
10 changed files with 1399 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+8 -1
View File
@@ -2,7 +2,14 @@
Full release notes with details on each version: [GitHub Releases](https://github.com/safishamsi/graphify/releases)
## 0.9.32 (unreleased)
## 0.9.33 (unreleased)
- Fix: the C# `partial class` merge (#2332) no longer conflates two same-named classes that live in different assemblies (#2411, thanks @JensD-git). The merge now keys on assembly (nearest ancestor directory containing a `.csproj`/`.fsproj`/`.vbproj`) in addition to namespace and name, so genuine partial halves within one project still merge while same-name types in separate projects stay distinct. A corpus with no project file keeps merging by namespace and name as before.
- Fix: `graphify update` no longer drops member-call and `indirect_call` edges from a changed file into an unchanged target (#2437, #2438, thanks @aryanbonigala). Incremental re-resolution now sees the unchanged corpus (its nodes, `contains`/`method` edges, and the `_callable` markers, which now persist to `graph.json` like `_origin`), so cross-file calls survive an incremental rebuild while edges to a genuinely removed target are still evicted.
- Fix: `graphify extract` no longer silently substitutes an empty result when a worker crashes (#2444, #2445, thanks @Baziar). A `BrokenProcessPool` now triggers the sequential fallback instead of being swallowed per future, a failed worker file is retried sequentially rather than merged as empty, and a whole-pass AST failure on a fresh build exits non-zero instead of writing a zero-node graph (use `--allow-partial` to opt into a best-effort partial graph).
- `graphify install` now prints a one-time pointer to the hosted platform (early access is open free before the public v1 launch) after the setup summary.
## 0.9.32 (2026-08-01)
- Fix: incremental extraction and `_rebuild_code` no longer drop a file's other tier (#2333, #2334, #2336). Node/edge ownership was keyed on `source_file` alone, so a semantic re-extract deleted a doc's AST headings and a full rebuild deleted document AST nodes. Merge is now tier-aware (an AST re-extract replaces only AST nodes and keeps the semantic layer, and vice versa), the `_origin` provenance marker is backfilled on load so old graphs self-heal, and the full-rebuild drop is scoped to sources actually regenerated.
- Fix: `graphify update` preserves the graph's `directed` flag instead of rebuilding it undirected (#2342, thanks @Rishet11), so God-node / path ranking keeps its direction on both the clustered and `--no-cluster` rebuild paths.
graphify/cli.py
+89
View File
@@ -3091,11 +3091,100 @@ def dispatch_command(cmd: str) -> None:
ast_kwargs: dict = {"cache_root": out_root, "root": target}
if cli_max_workers is not None:
ast_kwargs["max_workers"] = cli_max_workers
# #2437/#2438 (the `graphify update` twin of watch's #2406 fix): an
# incremental re-scan extracts only the changed code files, so the
# cross-file resolvers cannot see a callee living in an unchanged
# file and every changed->unchanged call edge silently vanished on
# merge. Hand extract() read-only resolution context from the
# persisted graph: its AST-tier nodes (with their `_callable`/
# `_callable_class` markers, #2438) plus the contains/method edges
# the member-call resolvers walk (#2437), scoped to the UNCHANGED
# live corpus — never a re-extracted, deleted, or excluded file, so
# stale symbols cannot resurrect. Fails open (changed-batch-only
# resolution, the pre-fix behavior) on an unreadable graph.
if incremental_mode and existing_graph_path.exists():
_ctx_nodes: list[dict] = []
_ctx_edges: list[dict] = []
try:
from graphify.build import _is_ast_tier as _ctx_is_ast_tier
from graphify.security import (
check_graph_file_size_cap as _ctx_size_cap,
)
_ctx_size_cap(existing_graph_path)
_ctx_graph = json.loads(
existing_graph_path.read_text(encoding="utf-8")
)
_ctx_root = Path(os.path.abspath(target))
def _ctx_identity(source_file) -> str | None:
# graph.json source_file values are relative to the
# scanned root (`root=target` above); detect's
# unchanged_files keep their scan-time form. Compare
# both as absolute posix paths.
if not source_file:
return None
_p = Path(str(source_file))
if not _p.is_absolute():
_p = _ctx_root / _p
return Path(os.path.abspath(_p)).as_posix()
_ctx_live = {
_ctx_identity(f)
for _flist in detection.get("unchanged_files", {}).values()
for f in _flist
}
_ctx_live.discard(None)
for _node in _ctx_graph.get("nodes", []):
if not _node.get("id") or not _ctx_is_ast_tier(_node):
continue
_sf = _node.get("source_file")
if not _sf or _ctx_identity(_sf) not in _ctx_live:
continue
_ctx_node = {
"id": _node["id"],
"label": _node.get("label"),
"source_file": _sf,
"file_type": _node.get("file_type"),
"type": _node.get("type"),
}
for _marker in ("_callable", "_callable_class"):
if _node.get(_marker):
_ctx_node[_marker] = _node[_marker]
_ctx_nodes.append(_ctx_node)
for _edge in _ctx_graph.get(
"links", _ctx_graph.get("edges", [])
):
if _edge.get("relation") not in ("contains", "method"):
continue
if not _ctx_is_ast_tier(_edge):
continue
_sf = _edge.get("source_file")
if not _sf or _ctx_identity(_sf) not in _ctx_live:
continue
_ctx_edges.append({
"source": _edge.get("source"),
"target": _edge.get("target"),
"relation": _edge.get("relation"),
"source_file": _sf,
})
except Exception:
_ctx_nodes, _ctx_edges = [], []
if _ctx_nodes:
ast_kwargs["resolution_context_nodes"] = _ctx_nodes
if _ctx_edges:
ast_kwargs["resolution_context_edges"] = _ctx_edges
print(f"[graphify extract] AST extraction on {len(code_files)} code files...")
try:
ast_result = _ast_extract(code_files, **ast_kwargs)
except Exception as exc:
print(f"[graphify extract] AST extraction failed: {exc}", file=sys.stderr)
# #2445: losing the whole AST pass is fatal by default. The
# empty stand-in only reaches the shrink guard when an existing
# graph is larger — on a fresh build it used to be written as a
# 0-node graph with exit 0, indistinguishable from success.
# --allow-partial opts back into the best-effort continuation.
if not cli_allow_partial:
sys.exit(1)
ast_result = {"nodes": [], "edges": [], "input_tokens": 0, "output_tokens": 0}
_extraction_incomplete = True # the whole AST pass was lost
stages.mark("AST extract")
graphify/extract.py
+210 -30
View File
@@ -2086,21 +2086,31 @@ def _merge_csharp_partial_class_nodes(
per_file: list[dict],
all_nodes: list[dict],
all_edges: list[dict],
paths: list[Path],
root: Path,
) -> None:
"""Collapse C# `partial class Foo` halves split across files into ONE node
(#2332).
(#2332), without crossing assembly boundaries (#2411).
The per-file extractor mints class ids with the file stem, so each file
declaring `partial class Foo` produces its own `Foo` node: members split
across the halves and cross-half calls don't resolve (two candidate types
make every receiver-typed lookup bail as ambiguous). Group partial-stamped
type nodes by (namespace, label) — same-named types in different namespaces
are distinct types, non-partial same-named types are separate declarations,
and nested partials are excluded (their ids omit the enclosing type, so a
same-named nested pair under different outers would falsely merge). The
canonical node is the sorted-first half by (source_file, source_location,
id); every edge endpoint and raw-call caller is remapped onto it. Member
node ids are left untouched — only the class-level nodes collapse.
type nodes by (assembly, namespace, label) — same-named types in different
namespaces are distinct types, non-partial same-named types are separate
declarations, and nested partials are excluded (their ids omit the
enclosing type, so a same-named nested pair under different outers would
falsely merge). The `partial` keyword only fuses declarations compiled into
the SAME assembly, so the key also carries the nearest ancestor directory
holding a `*.csproj`/`*.fsproj`/`*.vbproj` — same-named halves under
different project dirs are genuinely distinct types and stay apart. Halves
with NO project file on any ancestor (up to the scan root) all key to ""
and still merge together, so single-project/snippet corpora behave exactly
as before; the probe runs only for groups that are otherwise ambiguous.
The canonical node is the sorted-first half by (source_file,
source_location, id); every edge endpoint and raw-call caller is remapped
onto it. Member node ids are left untouched — only the class-level nodes
collapse.
Must run BEFORE _disambiguate_colliding_node_ids / _rewire_unique_stub_nodes /
_resolve_csharp_type_references and the resolver registry, so every later
@@ -2120,19 +2130,90 @@ def _merge_csharp_partial_class_nodes(
continue
groups.setdefault((str(md.get("namespace", "")), str(label)), []).append(n)
if not any(len(members) >= 2 for members in groups.values()):
return
# Assembly probe (#2411). A node's `source_file` can be a bare filename at
# this point (ambiguous across project dirs), so map nid -> scanned path
# via per_file, which aligns 1:1 with `paths`.
nid_to_path: dict[str, Path] = {}
for result, path in zip(per_file, paths):
for pn in result.get("nodes") or []:
nid_to_path.setdefault(pn["id"], path)
proj_exts = (".csproj", ".fsproj", ".vbproj")
project_dirs: set[Path] = set()
for p in paths:
if p.suffix.lower() in proj_exts:
try:
project_dirs.add(p.resolve().parent)
except OSError:
pass
try:
stop = root.resolve()
except OSError:
stop = root
dir_assembly: dict[Path, str] = {}
def _assembly_of_dir(d: Path) -> str:
"""Nearest ancestor dir (self included) holding a project file, "" if
none up to the scan root; memoized along the walked chain."""
chain: list[Path] = []
key = ""
while True:
cached = dir_assembly.get(d)
if cached is not None:
key = cached
break
chain.append(d)
if d in project_dirs:
key = str(d)
break
try:
has_project = any(
c.suffix.lower() in proj_exts for c in d.iterdir()
)
except OSError:
has_project = False
if has_project:
key = str(d)
break
if d == stop or d.parent == d:
break
d = d.parent
for c in chain:
dir_assembly[c] = key
return key
def _assembly_of_node(nid: str) -> str:
path = nid_to_path.get(nid)
if path is None:
return ""
try:
d = path.resolve().parent
except OSError:
return ""
return _assembly_of_dir(d)
remap: dict[str, str] = {}
for members in groups.values():
if len(members) < 2:
continue
members.sort(key=lambda n: (
str(n.get("source_file", "")),
str(n.get("source_location", "")),
str(n.get("id", "")),
))
canonical_nid = members[0]["id"]
for other in members[1:]:
if other["id"] != canonical_nid:
remap[other["id"]] = canonical_nid
by_assembly: dict[str, list[dict]] = {}
for n in members:
by_assembly.setdefault(_assembly_of_node(n["id"]), []).append(n)
for halves in by_assembly.values():
if len(halves) < 2:
continue
halves.sort(key=lambda n: (
str(n.get("source_file", "")),
str(n.get("source_location", "")),
str(n.get("id", "")),
))
canonical_nid = halves[0]["id"]
for other in halves[1:]:
if other["id"] != canonical_nid:
remap[other["id"]] = canonical_nid
if not remap:
return
@@ -4491,6 +4572,7 @@ def _extract_parallel(
work_items = [(idx, str(path), root_str, cache_loc_str) for idx, path in uncached_work]
done_count = 0
failed: list[int] = [] # positions into uncached_work whose future failed
_PROGRESS_INTERVAL = 100
try:
with concurrent.futures.ProcessPoolExecutor(max_workers=max_workers) as pool:
@@ -4502,12 +4584,21 @@ def _extract_parallel(
try:
idx, result = future.result()
per_file[idx] = result
except concurrent.futures.process.BrokenProcessPool:
# #2444: a pool that dies while results are being consumed
# raises BrokenProcessPool from every pending future. It
# must reach the pool-level handler below (which returns
# False so the caller falls back to sequential), not be
# swallowed here per-future — that left the remaining
# per_file slots empty and silently dropped the files.
raise
except Exception as exc:
pos = futures[future]
print(
f" warning: worker failed for {work_items[pos][1]}: {exc}",
file=sys.stderr, flush=True,
)
failed.append(pos)
done_count += 1
if (
total_files >= _PROGRESS_INTERVAL
@@ -4532,6 +4623,16 @@ def _extract_parallel(
flush=True,
)
return False
if failed:
# #2445: retry per-future failures once, in-process, instead of leaving
# their per_file slots None (which the defensive fill downstream turned
# into well-formed empties — silent data loss). This is bounded, not a
# loop: _extract_sequential goes through _safe_extract, which converts
# a second failure into an error-carrying result.
_extract_sequential(
[uncached_work[pos] for pos in failed],
per_file, root, total_files, cache_location,
)
if total_files >= _PROGRESS_INTERVAL:
# Report the same denominator the intermediate lines used (uncached files
# actually processed this run), not total_files — switching to the full
@@ -4591,6 +4692,8 @@ def extract(
root: Path | None = None,
parallel: bool = True,
max_workers: int | None = None,
resolution_context_nodes: list[dict] | None = None,
resolution_context_edges: list[dict] | None = None,
) -> dict:
"""Extract AST nodes and edges from a list of code files.
@@ -4613,6 +4716,24 @@ def extract(
use ProcessPoolExecutor for multi-core extraction.
max_workers: max subprocess count. Defaults to cpu_count (or the
value of GRAPHIFY_MAX_WORKERS if set), bounded by len(uncached_work).
resolution_context_nodes: read-only AST nodes from files that are NOT
being extracted this run (an incremental rebuild's unchanged
corpus, #2406). They extend the cross-file resolution indexes —
the shared direct-call pass's label/file indexes, the
indirect_call callable guard (via the persisted `_callable` /
`_callable_class` markers, #2438), and the member-call resolvers
run by `run_language_resolvers` (#2437) — so a changed caller can
still bind `foo()`, `obj.method()`, or `submit(handler)` to an
unchanged callee. They are never parsed, mutated, or returned;
raw_calls come only from `paths`, so only edges sourced by the
re-extracted files are emitted.
resolution_context_edges: the `contains`/`method` edges of the same
unchanged corpus (#2437). The member-call resolvers walk these to
map a receiver type to the single class owning the called method;
without them an unchanged callee's class never passes the
single-definition guard. Read-only, same contract as
resolution_context_nodes: they widen the resolvers' view but only
fresh results are appended to the returned nodes/edges.
"""
paths = [Path(p) for p in paths]
anchor_root = Path(root) if root is not None else None
@@ -4682,12 +4803,24 @@ def extract(
uncached_work, per_file, root, max_workers, total, cache_location
)
if not ran_parallel:
_extract_sequential(uncached_work, per_file, root, total, cache_location)
# #2444: only re-extract what the pool didn't finish. A pool that
# breaks mid-run has already filled some per_file slots; redoing
# the whole batch would throw that work away.
_extract_sequential(
[(i, p) for (i, p) in uncached_work if per_file[i] is None],
per_file, root, total, cache_location,
)
# Fill any remaining None slots (shouldn't happen, but defensive)
# Fill any remaining None slots. With the #2444/#2445 handling above this
# is unreachable; the error marker keeps any regression loud (and out of
# the caches/#1666 paths) instead of letting a dropped file masquerade as
# a legitimately-empty one.
for i in range(total):
if per_file[i] is None:
per_file[i] = {"nodes": [], "edges": []}
per_file[i] = {
"nodes": [], "edges": [],
"error": "internal: no extraction result produced",
}
# #1666: surface any source file an extractor accepted but that produced zero
# nodes (not even a file node). Such a file is silently absent from the graph,
@@ -5175,7 +5308,7 @@ def extract(
# graph is identical regardless of scan root (#2072).
_repoint_python_package_imports(paths, all_nodes, all_edges, root)
_merge_swift_extensions(per_file, all_nodes, all_edges)
_merge_csharp_partial_class_nodes(per_file, all_nodes, all_edges)
_merge_csharp_partial_class_nodes(per_file, all_nodes, all_edges, paths, root)
_disambiguate_colliding_node_ids(all_nodes, all_edges, all_raw_calls, root)
_canonicalize_csharp_namespace_nodes(all_nodes, all_edges)
# PHP namespace/use disambiguation must run BEFORE the unique-stub rewire:
@@ -5296,7 +5429,28 @@ def extract(
# identifiers, and they were polluting matches for short names — #563).
global_label_to_nids: dict[str, list[str]] = {} # exact-case (all languages)
global_label_to_nids_ci: dict[str, list[str]] = {} # case-INSENSITIVE-language nodes
for n in all_nodes:
# #2406: on an incremental rebuild only the CHANGED files are parsed, so
# `all_nodes` alone cannot see a callee that lives in an unchanged file and
# every changed->unchanged DIRECT call silently vanished (while the file-level
# `imports` edge survived, because the JS/Python symbol-resolution pass
# reads the import TARGET off disk instead of off the node list). Extend the
# resolution indexes — and ONLY the indexes — with the caller-supplied
# unchanged-corpus nodes. Fresh nodes win on id collision, nothing is
# appended to `all_nodes`, and raw_calls still come solely from `paths`, so
# the emitted edges remain sourced by the re-extracted files.
#
# Scope: this list feeds the shared direct-call loop below, the
# indirect_call callable guard (#2438, via the persisted `_callable` /
# `_callable_class` markers), and — together with resolution_context_edges —
# the member-call resolvers run by run_language_resolvers (#2437).
resolution_nodes = all_nodes
if resolution_context_nodes:
_fresh_ids = {n["id"] for n in all_nodes}
resolution_nodes = all_nodes + [
n for n in resolution_context_nodes
if n.get("id") and n["id"] not in _fresh_ids
]
for n in resolution_nodes:
if n.get("file_type") == "rationale" or n.get("type") == "namespace":
continue
raw = n.get("label", "")
@@ -5313,12 +5467,15 @@ def extract(
# Callable-def ids for the indirect_call callable guard, read from the `_callable`
# marker on the FINAL (post-remap) nodes — so a callback resolves only to a real
# function/method/class, never a same-named data symbol, and the guard never goes
# stale when node ids were relativized/disambiguated above (#1566).
callable_nids = {n["id"] for n in all_nodes if n.get("_callable")}
# stale when node ids were relativized/disambiguated above (#1566). Read from
# `resolution_nodes`, not `all_nodes` (#2438): an unchanged callee's context node
# carries the marker persisted in graph.json, so an incremental rebuild keeps
# resolving callbacks into unchanged files while data symbols stay excluded.
callable_nids = {n["id"] for n in resolution_nodes if n.get("_callable")}
# Class defs are callable only via their constructor; they are frequently passed
# as descriptive values (`select(Model)`, exception tuples), not invoked. Exclude
# them from the indirect_call guard below to avoid false edges (#2137).
class_nids = {n["id"] for n in all_nodes if n.get("_callable_class")}
class_nids = {n["id"] for n in resolution_nodes if n.get("_callable_class")}
# Build evidence index from import edges so cross-file calls backed by an
# explicit import statement can be promoted from INFERRED to EXTRACTED.
@@ -5344,7 +5501,7 @@ def extract(
# absolute-derived id — which would spuriously fail import evidence and (with
# the #1659 JS/TS gate below) drop a legitimately-imported call.
sf_to_file_nid: dict[str, str] = {}
for n in all_nodes:
for n in resolution_nodes:
sf = n.get("source_file")
if sf and n.get("label") == Path(str(sf)).name:
sf_to_file_nid.setdefault(str(sf), n["id"])
@@ -5353,7 +5510,7 @@ def extract(
# (test/non-test classification + path proximity). Kept separate from the
# file-node-id map because tie-breaking compares the actual file paths.
nid_to_source_file: dict[str, str] = {}
for n in all_nodes:
for n in resolution_nodes:
sf = n.get("source_file")
if not sf:
continue
@@ -5567,7 +5724,24 @@ def extract(
# receiver-typed/qualified calls the shared pass skipped) with its own
# single-definition god-node guard. Registered in graphify.resolver_registry so
# a new language plugs in without editing this body (#1356 Swift, #1446 Python).
run_language_resolvers(paths, per_file, all_nodes, all_edges)
#
# #2437: on an incremental rebuild the resolvers must also see the unchanged
# corpus — its nodes (types/methods, from resolution_nodes above) and its
# persisted contains/method edges (resolution_context_edges) — or the
# single-definition guards bail on every changed->unchanged member call. Run
# them over SCRATCH lists that include the context, then keep only the fresh
# results: raw_calls come solely from `paths`, so nothing sourced by an
# unchanged file is ever emitted, and the ambiguity guards count the same
# candidates a full build would (the context is the whole unchanged corpus).
if resolution_context_nodes or resolution_context_edges:
_rl_nodes = list(resolution_nodes)
_rl_edges = all_edges + list(resolution_context_edges or [])
_n0, _e0 = len(_rl_nodes), len(_rl_edges)
run_language_resolvers(paths, per_file, _rl_nodes, _rl_edges)
all_nodes.extend(_rl_nodes[_n0:])
all_edges.extend(_rl_edges[_e0:])
else:
run_language_resolvers(paths, per_file, all_nodes, all_edges)
# Relativize source_file fields so paths are portable across machines (#555).
# When the node's id was itself minted from the absolute path, remap it to a
@@ -5695,8 +5869,14 @@ def extract(
# cache keeps its own copy, which is what the colliding-id pass reads on a cache hit.
for n in all_nodes:
n.pop("origin_file", None)
n.pop("_callable", None) # internal indirect_call marker — never ships to graph.json
n.pop("_callable_class", None) # internal #2137 marker — never ships to graph.json
# `_callable` / `_callable_class` are deliberately NOT popped (#2438): they
# persist into graph.json — the same underscore-provenance precedent as
# `_origin` below — so an incremental rebuild can hand them back as
# resolution context and the indirect_call callable guard keeps working for
# targets in unchanged files. Callability is never inferred from a persisted
# label (that would reintroduce the #1566/#2137 data-symbol false positives);
# a graph written before the markers existed simply fails closed until its
# files are re-extracted.
# local_alias is a transient import-resolution hint (#2082), same shape as
# target_file (#1814): it exists only so the module arm of
graphify/watch.py
+89 -1
View File
@@ -1157,8 +1157,96 @@ def _rebuild_code(
# AST heading layer intact alongside the semantic layer.
extract_targets = [p for p in code_files if p not in semantic_doc_files]
# #2406: an incremental rebuild parses only the changed files, so the
# cross-file resolvers could not see a callee living in an unchanged
# file and every changed->unchanged `calls` edge disappeared (reconcile
# evicts the old one as AST-tier output of a re-extracted source, and
# nothing regenerates it). Hand extract() read-only resolution context:
# the persisted AST nodes of files this run is NOT re-extracting —
# including their `_callable`/`_callable_class` markers, so the
# indirect_call guard keeps working (#2438) — plus their contains/method
# edges, which the member-call resolvers walk (#2437).
#
# Scoping rules, in order of importance:
# * AST-tier only — semantic/LLM nodes are not symbol definitions.
# * never a file in extract_targets (its fresh nodes are authoritative)
# nor a deleted one (its persisted symbols are gone).
# * only sources still in the scanned corpus, so a renamed/removed file
# cannot stay a resolver target.
# extract() uses these purely to widen the resolvers' indexes; nothing
# is parsed, mutated, or emitted from them (see extract()'s docstring).
resolution_context_nodes: list[dict] = []
resolution_context_edges: list[dict] = []
if changed_paths is not None and existing_graph.exists():
try:
check_graph_file_size_cap(existing_graph)
ctx_graph = json.loads(existing_graph.read_text(encoding="utf-8"))
ctx_paths = _StoredSourcePaths(
ctx_graph,
out=out,
project_root=project_root,
watch_root=watch_root,
normalize_source=_nsf,
)
ctx_live = {
ctx_paths.absolute_identity(str(p), project_root) for p in code_files
}
ctx_live -= {
ctx_paths.absolute_identity(str(p), project_root) for p in extract_targets
}
ctx_live -= deleted_source_identities
ctx_live.discard(None)
for node in ctx_graph.get("nodes", []):
if not node.get("id") or not _is_ast_tier(node):
continue
source_file = node.get("source_file")
if not source_file or ctx_paths.identity(source_file) not in ctx_live:
continue
ctx_node = {
"id": node["id"],
"label": node.get("label"),
"source_file": source_file,
"file_type": node.get("file_type"),
"type": node.get("type"),
}
# #2438: the persisted callability markers are the only
# thing that lets an unchanged target pass the
# indirect_call guard — never re-derived from the label.
for marker in ("_callable", "_callable_class"):
if node.get(marker):
ctx_node[marker] = node[marker]
resolution_context_nodes.append(ctx_node)
# #2437: the member-call resolvers map receiver type -> owning
# class -> method through contains/method edges; hand over the
# unchanged corpus's, scoped exactly like the nodes above so a
# deleted/re-extracted file's edges can never resurrect.
for edge in ctx_graph.get("links", ctx_graph.get("edges", [])):
if edge.get("relation") not in ("contains", "method"):
continue
if not _is_ast_tier(edge):
continue
source_file = edge.get("source_file")
if not source_file or ctx_paths.identity(source_file) not in ctx_live:
continue
resolution_context_edges.append({
"source": edge.get("source"),
"target": edge.get("target"),
"relation": edge.get("relation"),
"source_file": source_file,
})
except Exception:
# Unreadable/oversized graph: resolve with the changed batch only
# (pre-#2406 behavior). Reconcile below still fails closed on it.
resolution_context_nodes = []
resolution_context_edges = []
commit = _git_head(cwd=watch_root)
result = extract(extract_targets, cache_root=watch_root) if extract_targets else {
result = extract(
extract_targets,
cache_root=watch_root,
resolution_context_nodes=resolution_context_nodes or None,
resolution_context_edges=resolution_context_edges or None,
) if extract_targets else {
"nodes": [], "edges": [], "hyperedges": [],
"input_tokens": 0, "output_tokens": 0,
}
pyproject.toml
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "graphifyy"
version = "0.9.32"
version = "0.9.33"
description = "AI coding assistant skill (Claude Code, CodeBuddy, Codex, OpenCode, Kilo Code, Cursor, Gemini CLI, Aider, OpenClaw, Factory Droid, Trae, Hermes, Kiro, Pi, Devin CLI, Google Antigravity) - turn any folder of code, docs, papers, images, or videos into a queryable knowledge graph"
readme = "README.md"
license = "Apache-2.0"
tests/test_csharp_partial_classes.py
+109 -2
View File
@@ -4,8 +4,11 @@
carries the per-file stem), so the type's members split across the halves and
every receiver-typed lookup on `Foo` bailed as ambiguous — cross-half calls
never resolved. `_merge_csharp_partial_class_nodes` collapses the halves onto
one canonical node, keyed by (namespace, label); same-named types in other
namespaces, non-partial declarations, and nested partial types are left alone.
one canonical node, keyed by (assembly, namespace, label); same-named types in
other namespaces, non-partial declarations, and nested partial types are left
alone. The assembly key is the nearest ancestor dir holding a `*.csproj`/
`*.fsproj`/`*.vbproj` (#2411: `partial` never fuses across assemblies), with
"" — halves under no project at all, which still merge — as the sentinel.
"""
from __future__ import annotations
@@ -149,3 +152,107 @@ def test_nested_partial_not_merged(tmp_path):
assert len(outers) == 1, "top-level partial halves still merge"
assert len(inners) == 2, \
f"nested partial types must NOT merge (id has no outer qualifier): {inners}"
_TWO_ASSEMBLIES = {
"src/AsmOne/AsmOne.csproj": "<Project Sdk=\"Microsoft.NET.Sdk\"></Project>\n",
"src/AsmOne/Widget.cs": (
"namespace Shared {\n"
" public partial class Widget {\n"
" public void OnlyInAssemblyOne() {}\n"
" }\n"
"}\n"
),
"src/AsmOne/Widget.Part2.cs": (
"namespace Shared {\n"
" public partial class Widget {\n"
" public void AlsoInAssemblyOne() {}\n"
" }\n"
"}\n"
),
"src/AsmTwo/AsmTwo.csproj": "<Project Sdk=\"Microsoft.NET.Sdk\"></Project>\n",
"src/AsmTwo/Widget.cs": (
"namespace Shared {\n"
" public partial class Widget {\n"
" public void OnlyInAssemblyTwo() {}\n"
" }\n"
"}\n"
),
}
def _widget_methods_by_assembly(r):
"""Map each Widget class node -> set of member-method labels hanging off it."""
widgets = _nodes_labeled(r, "Widget")
label_of = {n["id"]: n["label"] for n in r["nodes"]}
return widgets, {
w["id"]: {
label_of[e["target"]] for e in r["edges"]
if e["relation"] == "method" and e["source"] == w["id"]
}
for w in widgets
}
def test_same_namespace_partials_in_different_assemblies_not_merged(tmp_path):
"""#2411: same fully-qualified name under TWO .csproj projects is two
genuinely distinct types — never one node with phantom cross-assembly edges."""
calls, r = _extract(tmp_path, _TWO_ASSEMBLIES)
widgets, methods = _widget_methods_by_assembly(r)
assert len(widgets) == 2, \
f"partials in different assemblies must stay distinct: {widgets}"
asm_one = next(w["id"] for w in widgets if "asmone" in w["id"].lower())
asm_two = next(w["id"] for w in widgets if "asmtwo" in w["id"].lower())
assert methods[asm_one] == {".OnlyInAssemblyOne()", ".AlsoInAssemblyOne()"}, \
f"AsmOne's Widget owns exactly its own two members: {methods[asm_one]}"
assert methods[asm_two] == {".OnlyInAssemblyTwo()"}, \
f"AsmTwo's Widget owns exactly its own member: {methods[asm_two]}"
phantom = [
e for e in r["edges"]
if e["relation"] in ("contains", "method")
and e["target"] == asm_one and "asmtwo" in str(e["source"]).lower()
]
assert not phantom, f"no AsmTwo-derived edge may reach AsmOne's Widget: {phantom}"
def test_partial_halves_within_one_csproj_still_merge(tmp_path):
"""Adding a .csproj must not break the #2332 merge within one project."""
calls, r = _extract(tmp_path, {
"src/AsmOne/AsmOne.csproj": "<Project Sdk=\"Microsoft.NET.Sdk\"></Project>\n",
"src/AsmOne/Widget.cs": (
"namespace Shared {\n"
" public partial class Widget {\n"
" public void Alpha() {}\n"
" }\n"
"}\n"
),
"src/AsmOne/Widget.Part2.cs": (
"namespace Shared {\n"
" public partial class Widget {\n"
" public void Beta() { Alpha(); }\n"
" }\n"
"}\n"
),
})
widgets, methods = _widget_methods_by_assembly(r)
assert len(widgets) == 1, \
f"same-project halves must still collapse to ONE node: {widgets}"
assert methods[widgets[0]["id"]] == {".Alpha()", ".Beta()"}, \
f"canonical Widget must own members from BOTH halves: {methods}"
alpha = _find(r, ".Alpha()", "widget")
beta = _find(r, ".Beta()", "widget")
assert (beta, alpha) in calls, "cross-half in-class call must still resolve"
def test_assembly_probe_without_scanned_csproj(tmp_path):
"""The .csproj files exist on disk but are NOT in the scanned paths — the
assembly key comes from the ancestor-dir walk, not the paths seed."""
for name, body in _TWO_ASSEMBLIES.items():
p = tmp_path / name
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(body)
cs_only = {n: b for n, b in _TWO_ASSEMBLIES.items() if n.endswith(".cs")}
calls, r = _extract(tmp_path, cs_only)
widgets = _nodes_labeled(r, "Widget")
assert len(widgets) == 2, \
f"on-disk (unscanned) project files must still split assemblies: {widgets}"
tests/test_extract.py
+249
View File
@@ -1650,6 +1650,255 @@ def test_extract_parallel_still_spawns_pool_for_multiple_workers(tmp_path, monke
assert spawned["count"] == 1, "multi-worker runs must still use the pool"
def test_extract_falls_back_when_worker_future_breaks_pool(
tmp_path, monkeypatch, capsys
):
"""#2444: a BrokenProcessPool raised from future.result() (pool died while
results were being consumed) must trigger the sequential fallback, not be
swallowed per-future leaving empty per_file slots."""
from concurrent.futures.process import BrokenProcessPool
import concurrent.futures
from graphify import extract as extract_mod
class BrokenFuture:
def result(self):
raise BrokenProcessPool("simulated worker termination")
class FakePool:
def __init__(self, *a, **kw): pass
def __enter__(self): return self
def __exit__(self, *a): return False
def submit(self, *a, **kw):
return BrokenFuture()
monkeypatch.setattr(concurrent.futures, "ProcessPoolExecutor", FakePool)
monkeypatch.setattr(
concurrent.futures, "as_completed", lambda futures: iter(futures)
)
# A 1-CPU runner resolves max_workers to 1 and never enters the pool (#2173).
monkeypatch.setenv("GRAPHIFY_MAX_WORKERS", "2")
sequential_calls = 0
real_sequential = extract_mod._extract_sequential
def wrapped_sequential(*args, **kwargs):
nonlocal sequential_calls
sequential_calls += 1
return real_sequential(*args, **kwargs)
monkeypatch.setattr(extract_mod, "_extract_sequential", wrapped_sequential)
files = [FIXTURES / "sample.py"] * 25 # >= _PARALLEL_THRESHOLD
result = extract_mod.extract(files, cache_root=tmp_path / "cache")
assert sequential_calls == 1, "sequential fallback should have run exactly once"
assert result["nodes"], "sequential fallback must recover AST nodes"
assert "BrokenProcessPool" in capsys.readouterr().out
def test_extract_bpp_fallback_skips_already_completed_files(tmp_path, monkeypatch):
"""#2444: when the pool breaks mid-run, the sequential fallback must
re-extract only the files whose futures never completed."""
from concurrent.futures.process import BrokenProcessPool
import concurrent.futures
from graphify import extract as extract_mod
completed_before_break = 5
class GoodFuture:
def __init__(self, value): self._value = value
def result(self): return self._value
class BrokenFuture:
def result(self):
raise BrokenProcessPool("simulated worker termination")
class FakePool:
def __init__(self, *a, **kw):
self._submitted = 0
def __enter__(self): return self
def __exit__(self, *a): return False
def submit(self, fn, item):
self._submitted += 1
if self._submitted <= completed_before_break:
return GoodFuture(fn(item)) # extract in-process, eagerly
return BrokenFuture()
monkeypatch.setattr(concurrent.futures, "ProcessPoolExecutor", FakePool)
monkeypatch.setattr(
concurrent.futures, "as_completed", lambda futures: iter(futures)
)
monkeypatch.setenv("GRAPHIFY_MAX_WORKERS", "2")
retried: list[list[int]] = []
real_sequential = extract_mod._extract_sequential
def wrapped_sequential(uncached_work, *args, **kwargs):
retried.append([idx for idx, _ in uncached_work])
return real_sequential(uncached_work, *args, **kwargs)
monkeypatch.setattr(extract_mod, "_extract_sequential", wrapped_sequential)
files = [FIXTURES / "sample.py"] * 25
result = extract_mod.extract(files, cache_root=tmp_path / "cache")
assert len(retried) == 1, "sequential fallback should have run exactly once"
assert sorted(retried[0]) == list(range(completed_before_break, 25)), (
"files whose futures completed before the pool broke must not be re-extracted"
)
assert result["nodes"]
def test_extract_parallel_retries_failed_future_sequentially(
tmp_path, monkeypatch, capsys
):
"""#2445: a non-BPP per-future failure must be surfaced and retried
in-process, not silently replaced by a well-formed empty result."""
import concurrent.futures
from graphify import extract as extract_mod
class GoodFuture:
def __init__(self, value): self._value = value
def result(self): return self._value
class FailingFuture:
def result(self):
raise RuntimeError("simulated worker crash")
class FakePool:
def __init__(self, *a, **kw):
self._submitted = 0
def __enter__(self): return self
def __exit__(self, *a): return False
def submit(self, fn, item):
self._submitted += 1
if self._submitted == 1:
return FailingFuture()
return GoodFuture(fn(item))
monkeypatch.setattr(concurrent.futures, "ProcessPoolExecutor", FakePool)
monkeypatch.setattr(
concurrent.futures, "as_completed", lambda futures: iter(futures)
)
monkeypatch.setenv("GRAPHIFY_MAX_WORKERS", "2")
retried: list[list[int]] = []
real_sequential = extract_mod._extract_sequential
def wrapped_sequential(uncached_work, *args, **kwargs):
retried.append([idx for idx, _ in uncached_work])
return real_sequential(uncached_work, *args, **kwargs)
monkeypatch.setattr(extract_mod, "_extract_sequential", wrapped_sequential)
files = [FIXTURES / "sample.py"] * 25
result = extract_mod.extract(files, cache_root=tmp_path / "cache")
assert retried == [[0]], "only the failed file may be retried, exactly once"
assert result["nodes"]
err = capsys.readouterr().err
assert "worker failed" in err
assert "zero nodes" not in err, (
"a retried-and-recovered file must not trip the #1666 empty warning"
)
def test_extract_twice_failing_file_carries_error_marker(tmp_path, monkeypatch):
"""#2445: a file that fails in the pool AND on the sequential retry must
end up with an error-carrying result (via _safe_extract), not loop and not
masquerade as legitimately empty. Other files still complete."""
import concurrent.futures
from graphify import extract as extract_mod
bad_file = tmp_path / "boom.go"
bad_file.write_text("package main\n")
def _boom_extractor(path):
raise RuntimeError("extractor always crashes")
monkeypatch.setitem(extract_mod._DISPATCH, ".go", _boom_extractor)
class GoodFuture:
def __init__(self, value): self._value = value
def result(self): return self._value
class FailingFuture:
def result(self):
raise RuntimeError("simulated worker crash")
class FakePool:
def __init__(self, *a, **kw):
self._submitted = 0
def __enter__(self): return self
def __exit__(self, *a): return False
def submit(self, fn, item):
self._submitted += 1
if self._submitted == 1: # boom.go is first in the batch
return FailingFuture()
return GoodFuture(fn(item))
monkeypatch.setattr(concurrent.futures, "ProcessPoolExecutor", FakePool)
monkeypatch.setattr(
concurrent.futures, "as_completed", lambda futures: iter(futures)
)
monkeypatch.setenv("GRAPHIFY_MAX_WORKERS", "2")
captured: dict = {"calls": 0}
real_sequential = extract_mod._extract_sequential
def wrapped_sequential(uncached_work, per_file, *args, **kwargs):
captured["calls"] += 1
captured["retry_indices"] = [idx for idx, _ in uncached_work]
real_sequential(uncached_work, per_file, *args, **kwargs)
captured["per_file"] = list(per_file)
monkeypatch.setattr(extract_mod, "_extract_sequential", wrapped_sequential)
files = [bad_file] + [FIXTURES / "sample.py"] * 24
result = extract_mod.extract(files, cache_root=tmp_path / "cache")
assert captured["calls"] == 1, "the retry must be bounded: one pass, no loop"
assert captured["retry_indices"] == [0]
assert "error" in captured["per_file"][0], (
"a twice-failing file must carry an error marker, not a clean empty"
)
assert result["nodes"], "the other files must still complete"
def test_extract_legitimately_empty_result_keeps_no_error_marker(
tmp_path, monkeypatch, capsys
):
"""Guard for the #2445 error-marked None-fill: a file whose extractor
genuinely returns zero nodes gets a real (marker-free) result and still
trips the #1666 zero-nodes warning — behavior unchanged."""
from graphify import extract as extract_mod
empty_file = tmp_path / "empty.go"
empty_file.write_text("package main\n")
monkeypatch.setitem(
extract_mod._DISPATCH, ".go", lambda path: {"nodes": [], "edges": []}
)
captured: dict = {}
real_sequential = extract_mod._extract_sequential
def wrapped_sequential(uncached_work, per_file, *args, **kwargs):
real_sequential(uncached_work, per_file, *args, **kwargs)
captured["per_file"] = list(per_file)
monkeypatch.setattr(extract_mod, "_extract_sequential", wrapped_sequential)
extract_mod.extract([empty_file], cache_root=tmp_path / "cache")
assert "error" not in captured["per_file"][0], (
"a legitimately-empty extraction must not be error-marked"
)
assert "zero nodes" in capsys.readouterr().err, (
"the #1666 zero-nodes warning must still fire for a genuine empty"
)
# ---------------------------------------------------------------------------
# Bash extractor tests (#866)
# ---------------------------------------------------------------------------
tests/test_extract_cli.py
+90
View File
@@ -19,6 +19,96 @@ def _make_corpus(tmp_path):
return tmp_path
def test_extract_exits_nonzero_when_ast_extraction_raises(
monkeypatch, tmp_path, capsys
):
"""#2445: an AST-pass failure on a fresh build must not be presented as a
successful empty corpus (exit 0 + 0-node graph.json)."""
corpus = tmp_path / "corpus"
corpus.mkdir()
(corpus / "main.go").write_text("package main\nfunc main() {}\n")
out_dir = tmp_path / "out"
import graphify.extract as extractmod
def _ast_failed(paths, **kwargs):
raise RuntimeError("worker pool failed")
monkeypatch.setattr(extractmod, "extract", _ast_failed)
monkeypatch.setattr(mainmod, "_check_skill_version", lambda _: None)
monkeypatch.setattr(
mainmod.sys,
"argv",
["graphify", "extract", str(corpus), "--code-only",
"--out", str(out_dir)],
)
with pytest.raises(SystemExit) as exc_info:
mainmod.main()
assert exc_info.value.code == 1
assert (
"[graphify extract] AST extraction failed: worker pool failed"
in capsys.readouterr().err
)
assert not (out_dir / "graphify-out" / "graph.json").exists(), (
"graph.json must not be written when the whole AST pass is lost"
)
def test_extract_allow_partial_continues_past_ast_failure(
monkeypatch, tmp_path, capsys
):
"""#2445 complement: --allow-partial opts back into the best-effort path —
the run continues, and a graph built from the surviving (semantic) pass is
written with exit 0."""
corpus = _make_corpus(tmp_path) # main.go + README.md
out_dir = tmp_path / "out"
monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-test-fake-key")
import graphify.extract as extractmod
def _ast_failed(paths, **kwargs):
raise RuntimeError("worker pool failed")
monkeypatch.setattr(extractmod, "extract", _ast_failed)
def _one_chunk_succeeded(paths, **kwargs):
chunk = {
"nodes": [
{"id": "concept_main_entry", "label": "main entry point",
"type": "concept", "source_file": "README.md"},
],
"edges": [],
"hyperedges": [],
}
on_chunk = kwargs.get("on_chunk_done")
if on_chunk:
on_chunk(0, 1, chunk)
return {**chunk, "input_tokens": 100, "output_tokens": 50}
monkeypatch.setattr(
"graphify.llm.extract_corpus_parallel", _one_chunk_succeeded
)
monkeypatch.setattr(mainmod, "_check_skill_version", lambda _: None)
monkeypatch.setattr(
mainmod.sys,
"argv",
["graphify", "extract", str(corpus), "--backend", "claude",
"--allow-partial", "--out", str(out_dir)],
)
try:
mainmod.main()
except SystemExit as exc:
assert exc.code in (None, 0), f"unexpected exit code {exc.code}"
assert "AST extraction failed" in capsys.readouterr().err
assert (out_dir / "graphify-out" / "graph.json").exists(), (
"--allow-partial must still write the best-effort graph"
)
def test_extract_exits_nonzero_when_all_semantic_chunks_fail(
monkeypatch, tmp_path, capsys
):
tests/test_indirect_dispatch.py
+3 -2
View File
@@ -224,8 +224,9 @@ def test_cross_file_indirect_survives_id_relativization(tmp_path):
os.chdir(old)
nid = {n["label"].rstrip("()"): n["id"] for n in r["nodes"]}
assert (nid["schedule"], nid["on_event"]) in _rels(r, "indirect_call")
# the internal callable marker must never ship to graph.json
assert not any("_callable" in n for n in r["nodes"])
# the internal callable marker now persists to graph.json (#2438), so an
# incremental rebuild can resolve indirect_call edges into unchanged targets.
assert any(n.get("_callable") for n in r["nodes"])
def test_cross_file_imported_callback_emits_indirect_call(tmp_path):
tests/test_watch.py
+551
View File
@@ -2602,3 +2602,554 @@ def test_rebuild_code_fresh_build_defaults_undirected(tmp_path):
graph_path = corpus / "graphify-out" / "graph.json"
data = json.loads(graph_path.read_text(encoding="utf-8"))
assert data.get("directed", False) is False
def test_incremental_rebuild_preserves_call_to_unchanged_typescript_target(tmp_path):
"""#2406: rebuilding a changed caller retains its SHARED DIRECT calls into unchanged files.
Scope here is the shared direct-call pass (a plain `shared()`); member calls
(#2437) and indirect_call (#2438) are covered by the tests at the end of
this file.
A later edit that removes the call must still remove the edge, preventing a
stale-edge-preservation workaround.
"""
import json
from graphify.watch import _rebuild_code
corpus = tmp_path / "corpus"
corpus.mkdir()
target = corpus / "B.ts"
caller = corpus / "A.ts"
target.write_text(
"""
export function shared(): number {
return 1;
}
""".lstrip(),
encoding="utf-8",
)
caller.write_text(
"""
import { shared } from "./B";
export function run(): number {
return shared();
}
""".lstrip(),
encoding="utf-8",
)
graph_path = corpus / "graphify-out" / "graph.json"
def load_graph():
return json.loads(graph_path.read_text(encoding="utf-8"))
def node_id(graph, label, source_file):
return next(
node["id"]
for node in graph.get("nodes", [])
if node.get("label") == label
and node.get("source_file") == source_file
)
def has_call(graph):
run_id = node_id(graph, "run()", "A.ts")
shared_id = node_id(graph, "shared()", "B.ts")
return any(
edge.get("relation") == "calls"
and edge.get("source") == run_id
and edge.get("target") == shared_id
for edge in graph.get("links", graph.get("edges", []))
)
# Full-corpus baseline resolves A.run() -> B.shared().
assert _rebuild_code(
corpus,
no_cluster=True,
acquire_lock=False,
) is True
assert has_call(load_graph()), "full rebuild must create the cross-file call edge"
# Change only the caller while retaining the same call. The unchanged target
# must remain available to the cross-file resolver.
caller.write_text(
"""
import { shared } from "./B";
export function run(): number {
return shared() + 1;
}
""".lstrip(),
encoding="utf-8",
)
assert _rebuild_code(
corpus,
changed_paths=[caller],
no_cluster=True,
acquire_lock=False,
) is True
assert has_call(
load_graph()
), "incremental rebuild dropped the call edge to an unchanged target"
# Removing the call must remove the edge; do not merely preserve old outgoing
# edges from changed files.
caller.write_text(
"""
export function run(): number {
return 1;
}
""".lstrip(),
encoding="utf-8",
)
assert _rebuild_code(
corpus,
changed_paths=[caller],
no_cluster=True,
acquire_lock=False,
) is True
final_graph = load_graph()
run_id = node_id(final_graph, "run()", "A.ts")
assert not any(
edge.get("relation") == "calls"
and edge.get("source") == run_id
for edge in final_graph.get("links", final_graph.get("edges", []))
), "removed call must not survive as a stale edge"
# --- #2406 incremental shared-direct-call resolution helpers -----------------
def _2406_graph(corpus):
import json
return json.loads(
(corpus / "graphify-out" / "graph.json").read_text(encoding="utf-8")
)
def _2406_nid(graph, label, source_file):
return next(
(
node["id"]
for node in graph.get("nodes", [])
if node.get("label") == label and node.get("source_file") == source_file
),
None,
)
def _2406_calls(graph):
"""(source_id, target_id) of every `calls` edge."""
return [
(edge.get("source"), edge.get("target"))
for edge in graph.get("links", graph.get("edges", []))
if edge.get("relation") == "calls"
]
def _2406_seed(tmp_path, caller_src, target_src="export function shared(): number {\n return 1;\n}\n"):
"""Build a two-file TS corpus and do the initial full rebuild."""
from graphify.watch import _rebuild_code
corpus = tmp_path / "corpus"
corpus.mkdir(parents=True)
(corpus / "B.ts").write_text(target_src, encoding="utf-8")
(corpus / "A.ts").write_text(caller_src, encoding="utf-8")
assert _rebuild_code(corpus, no_cluster=True, acquire_lock=False) is True
return corpus
_2406_CALLER = (
'import { shared } from "./B";\n'
"\n"
"export function run(): number {\n"
" return shared();\n"
"}\n"
)
def test_incremental_rebuild_drops_call_when_import_is_removed(tmp_path):
"""#2406: no import evidence => the persisted target must not be resolved."""
from graphify.watch import _rebuild_code
corpus = _2406_seed(tmp_path, _2406_CALLER)
caller = corpus / "A.ts"
# `shared` is now a local no-op call with no import backing it.
caller.write_text(
"declare function shared(): number;\n"
"\n"
"export function run(): number {\n"
" return shared();\n"
"}\n",
encoding="utf-8",
)
assert _rebuild_code(
corpus, changed_paths=[caller], no_cluster=True, acquire_lock=False
) is True
graph = _2406_graph(corpus)
shared_id = _2406_nid(graph, "shared()", "B.ts")
run_id = _2406_nid(graph, "run()", "A.ts")
assert (run_id, shared_id) not in _2406_calls(graph)
def test_incremental_rebuild_uses_fresh_nodes_when_target_also_changed(tmp_path):
"""#2406: a changed target's persisted symbols must never win over fresh ones."""
from graphify.watch import _rebuild_code
corpus = _2406_seed(tmp_path, _2406_CALLER)
caller, target = corpus / "A.ts", corpus / "B.ts"
target.write_text(
"export function renamed(): number {\n return 2;\n}\n", encoding="utf-8"
)
caller.write_text(
'import { renamed } from "./B";\n'
"\n"
"export function run(): number {\n"
" return renamed();\n"
"}\n",
encoding="utf-8",
)
assert _rebuild_code(
corpus,
changed_paths=[caller, target],
no_cluster=True,
acquire_lock=False,
) is True
graph = _2406_graph(corpus)
calls = _2406_calls(graph)
run_id = _2406_nid(graph, "run()", "A.ts")
assert (run_id, _2406_nid(graph, "renamed()", "B.ts")) in calls
# The stale `shared()` node is gone entirely, so nothing can point at it.
assert _2406_nid(graph, "shared()", "B.ts") is None
def test_incremental_rebuild_context_excludes_deleted_target(tmp_path):
"""#2406: a deleted file cannot remain a resolver target."""
from graphify.watch import _rebuild_code
corpus = _2406_seed(tmp_path, _2406_CALLER)
caller, target = corpus / "A.ts", corpus / "B.ts"
target.unlink()
caller.write_text(
"export function run(): number {\n return shared();\n}\n", encoding="utf-8"
)
assert _rebuild_code(
corpus,
changed_paths=[caller, target],
no_cluster=True,
acquire_lock=False,
) is True
graph = _2406_graph(corpus)
assert _2406_nid(graph, "shared()", "B.ts") is None
assert _2406_calls(graph) == []
def test_incremental_rebuild_does_not_reparse_unchanged_targets(tmp_path, monkeypatch):
"""#2406 keeps the incremental contract: only changed files are extracted."""
import graphify.extract as extract_mod
from graphify.watch import _rebuild_code
corpus = _2406_seed(tmp_path, _2406_CALLER)
caller = corpus / "A.ts"
seen: list[list[str]] = []
real_extract = extract_mod.extract
def spy(paths, *args, **kwargs):
seen.append([Path(p).name for p in paths])
return real_extract(paths, *args, **kwargs)
monkeypatch.setattr(extract_mod, "extract", spy)
caller.write_text(_2406_CALLER.replace("shared();", "shared() + 1;"), encoding="utf-8")
assert _rebuild_code(
corpus, changed_paths=[caller], no_cluster=True, acquire_lock=False
) is True
assert seen == [["A.ts"]]
def test_incremental_rebuild_matches_full_rebuild_and_does_not_duplicate(tmp_path):
"""#2406: full/incremental parity for edges sourced by the changed file."""
from graphify.watch import _rebuild_code
corpus = _2406_seed(tmp_path, _2406_CALLER)
caller = corpus / "A.ts"
edited = _2406_CALLER.replace("shared();", "shared() + 1;")
caller.write_text(edited, encoding="utf-8")
# Two incremental rebuilds in a row must be idempotent (no duplicate edges).
for _ in range(2):
assert _rebuild_code(
corpus, changed_paths=[caller], no_cluster=True, acquire_lock=False
) is True
incremental = _2406_calls(_2406_graph(corpus))
assert len(incremental) == len(set(incremental))
# Same final corpus, built from scratch.
fresh = _2406_seed(tmp_path / "fresh", edited)
assert sorted(_2406_calls(_2406_graph(fresh))) == sorted(incremental)
def test_incremental_rebuild_preserves_python_call_to_unchanged_target(tmp_path):
"""#2406 is language-agnostic: the shared DIRECT cross-file call pass carries it."""
from graphify.watch import _rebuild_code
corpus = tmp_path / "corpus"
corpus.mkdir()
(corpus / "b.py").write_text("def shared():\n return 1\n", encoding="utf-8")
caller = corpus / "a.py"
caller.write_text(
"from b import shared\n\n\ndef run():\n return shared()\n", encoding="utf-8"
)
assert _rebuild_code(corpus, no_cluster=True, acquire_lock=False) is True
full = _2406_calls(_2406_graph(corpus))
assert full, "full rebuild must resolve the cross-file python call"
caller.write_text(
"from b import shared\n\n\ndef run():\n return shared() + 1\n",
encoding="utf-8",
)
assert _rebuild_code(
corpus, changed_paths=[caller], no_cluster=True, acquire_lock=False
) is True
assert sorted(_2406_calls(_2406_graph(corpus))) == sorted(full)
# --- #2437 / #2438: member + indirect calls into unchanged files -------------
# The #2406 resolution context now also carries the unchanged corpus's
# contains/method edges (member-call resolvers, #2437) and the persisted
# `_callable`/`_callable_class` markers (indirect_call guard, #2438), so both
# edge families survive an incremental rebuild exactly like shared direct calls.
_2437_TARGET = "export class Service {\n ping(): number { return 1; }\n}\n"
_2437_CALLER = (
'import { Service } from "./B";\n\n'
"export function run(): number {\n"
" const service = new Service();\n"
" return service.ping()%s;\n}\n"
)
def _2437_seed(tmp_path, caller_suffix=""):
"""Build the member-call corpus (TS receiver-typed call) and full-rebuild it."""
from graphify.watch import _rebuild_code
corpus = tmp_path / "corpus"
corpus.mkdir(parents=True)
(corpus / "B.ts").write_text(_2437_TARGET, encoding="utf-8")
(corpus / "A.ts").write_text(_2437_CALLER % caller_suffix, encoding="utf-8")
assert _rebuild_code(corpus, no_cluster=True, acquire_lock=False) is True
return corpus
_2438_CALLER = (
"from b import handler\n\n\ndef run(pool):\n%s return pool.submit(handler)\n"
)
def _2438_seed(tmp_path, caller_prefix="", target_src="def handler():\n return 1\n"):
"""Build the indirect-call corpus (py callback into b.py) and full-rebuild it."""
from graphify.watch import _rebuild_code
corpus = tmp_path / "corpus"
corpus.mkdir(parents=True)
(corpus / "b.py").write_text(target_src, encoding="utf-8")
(corpus / "a.py").write_text(_2438_CALLER % caller_prefix, encoding="utf-8")
assert _rebuild_code(corpus, no_cluster=True, acquire_lock=False) is True
return corpus
def _2438_indirects(graph):
"""(source_id, target_id) of every `indirect_call` edge."""
return [
(edge.get("source"), edge.get("target"))
for edge in graph.get("links", graph.get("edges", []))
if edge.get("relation") == "indirect_call"
]
def test_incremental_rebuild_preserves_member_call_to_unchanged_target(tmp_path):
"""#2437: a changed caller keeps its `service.ping()` edge into an unchanged file."""
from graphify.watch import _rebuild_code
corpus = _2437_seed(tmp_path)
full = _2406_calls(_2406_graph(corpus))
assert full, "full rebuild must resolve the cross-file member call"
caller = corpus / "A.ts"
caller.write_text(_2437_CALLER % " + 1", encoding="utf-8")
assert _rebuild_code(
corpus, changed_paths=[caller], no_cluster=True, acquire_lock=False
) is True
assert sorted(_2406_calls(_2406_graph(corpus))) == sorted(full)
def test_incremental_rebuild_preserves_indirect_call_to_unchanged_target(tmp_path):
"""#2438: the persisted `_callable` marker keeps `pool.submit(handler)` resolving."""
from graphify.watch import _rebuild_code
corpus = _2438_seed(tmp_path)
full = _2438_indirects(_2406_graph(corpus))
assert full, "full rebuild must resolve the cross-file indirect call"
caller = corpus / "a.py"
caller.write_text(_2438_CALLER % " x = 1\n", encoding="utf-8")
assert _rebuild_code(
corpus, changed_paths=[caller], no_cluster=True, acquire_lock=False
) is True
assert sorted(_2438_indirects(_2406_graph(corpus))) == sorted(full)
def test_incremental_rebuild_evicts_removed_member_call(tmp_path):
"""#2437: removing the member call from the caller must remove the edge —
the fix regenerates edges from fresh raw_calls, it never preserves stale ones."""
from graphify.watch import _rebuild_code
corpus = _2437_seed(tmp_path)
assert _2406_calls(_2406_graph(corpus)), "member-call baseline missing"
caller = corpus / "A.ts"
caller.write_text(
"export function run(): number {\n return 1;\n}\n", encoding="utf-8"
)
assert _rebuild_code(
corpus, changed_paths=[caller], no_cluster=True, acquire_lock=False
) is True
assert _2406_calls(_2406_graph(corpus)) == []
def test_incremental_rebuild_evicts_member_call_when_target_deleted(tmp_path):
"""#2437: a deleted callee file must not resurrect through the context edges."""
from graphify.watch import _rebuild_code
corpus = _2437_seed(tmp_path)
caller, target = corpus / "A.ts", corpus / "B.ts"
target.unlink()
assert _rebuild_code(
corpus,
changed_paths=[caller, target],
no_cluster=True,
acquire_lock=False,
) is True
graph = _2406_graph(corpus)
assert _2406_calls(graph) == []
assert not any(
node.get("source_file") == "B.ts" for node in graph.get("nodes", [])
), "deleted target's nodes must be evicted, not resurrected as context"
def test_incremental_rebuild_evicts_indirect_call_when_target_deleted(tmp_path):
"""#2438: a deleted callback target must not resurrect through the context nodes."""
from graphify.watch import _rebuild_code
corpus = _2438_seed(tmp_path)
caller, target = corpus / "a.py", corpus / "b.py"
target.unlink()
assert _rebuild_code(
corpus,
changed_paths=[caller, target],
no_cluster=True,
acquire_lock=False,
) is True
graph = _2406_graph(corpus)
assert _2438_indirects(graph) == []
assert not any(
node.get("source_file") == "b.py" for node in graph.get("nodes", [])
), "deleted target's nodes must be evicted, not resurrected as context"
def test_incremental_rebuild_callable_guard_excludes_unchanged_data_symbol(tmp_path):
"""#2438 keeps the #1566/#2137 guard: a same-named DATA symbol in an unchanged
file (`handler = 1`) is not `_callable`, so `pool.submit(handler)` must emit no
indirect_call on the full build or the incremental one."""
from graphify.watch import _rebuild_code
corpus = _2438_seed(tmp_path, target_src="handler = 1\n")
assert _2438_indirects(_2406_graph(corpus)) == []
caller = corpus / "a.py"
caller.write_text(_2438_CALLER % " x = 1\n", encoding="utf-8")
assert _rebuild_code(
corpus, changed_paths=[caller], no_cluster=True, acquire_lock=False
) is True
assert _2438_indirects(_2406_graph(corpus)) == []
def test_incremental_rebuild_legacy_graph_without_callable_markers(tmp_path):
"""#2438 degradation contract: a graph written before the `_callable` markers
persisted must not crash the incremental rebuild — the guard fails closed (no
indirect_call, pre-fix behavior) and the next full rebuild self-heals."""
import json
from graphify.watch import _rebuild_code
corpus = _2438_seed(tmp_path)
graph_path = corpus / "graphify-out" / "graph.json"
assert _2438_indirects(_2406_graph(corpus)), "indirect-call baseline missing"
# Simulate a pre-#2438 graph: strip the persisted callability markers.
legacy = json.loads(graph_path.read_text(encoding="utf-8"))
for node in legacy.get("nodes", []):
node.pop("_callable", None)
node.pop("_callable_class", None)
graph_path.write_text(json.dumps(legacy), encoding="utf-8")
caller = corpus / "a.py"
caller.write_text(_2438_CALLER % " x = 1\n", encoding="utf-8")
assert _rebuild_code(
corpus, changed_paths=[caller], no_cluster=True, acquire_lock=False
) is True
assert _2438_indirects(_2406_graph(corpus)) == []
# A full rebuild re-extracts the target, restores the markers, and the edge.
assert _rebuild_code(corpus, no_cluster=True, acquire_lock=False) is True
assert _2438_indirects(_2406_graph(corpus)), "full rebuild must self-heal"
def test_incremental_member_call_parity_and_idempotency(tmp_path):
"""#2437: repeated incremental rebuilds neither duplicate the member-call edge
nor diverge from a from-scratch build of the same corpus."""
from graphify.watch import _rebuild_code
corpus = _2437_seed(tmp_path)
caller = corpus / "A.ts"
caller.write_text(_2437_CALLER % " + 1", encoding="utf-8")
for _ in range(2):
assert _rebuild_code(
corpus, changed_paths=[caller], no_cluster=True, acquire_lock=False
) is True
incremental = _2406_calls(_2406_graph(corpus))
assert incremental, "member call lost across repeated incremental rebuilds"
assert len(incremental) == len(set(incremental))
fresh = _2437_seed(tmp_path / "fresh", caller_suffix=" + 1")
assert sorted(_2406_calls(_2406_graph(fresh))) == sorted(incremental)
def test_incremental_indirect_call_parity_and_idempotency(tmp_path):
"""#2438: repeated incremental rebuilds neither duplicate the indirect_call edge
nor diverge from a from-scratch build of the same corpus."""
from graphify.watch import _rebuild_code
corpus = _2438_seed(tmp_path)
caller = corpus / "a.py"
caller.write_text(_2438_CALLER % " x = 1\n", encoding="utf-8")
for _ in range(2):
assert _rebuild_code(
corpus, changed_paths=[caller], no_cluster=True, acquire_lock=False
) is True
incremental = _2438_indirects(_2406_graph(corpus))
assert incremental, "indirect call lost across repeated incremental rebuilds"
assert len(incremental) == len(set(incremental))
fresh = _2438_seed(tmp_path / "fresh", caller_prefix=" x = 1\n")
assert sorted(_2438_indirects(_2406_graph(fresh))) == sorted(incremental)