merge graphs offsets each input's own communities into a shared id
space so they do not fuse (number 3014), but that per source
partition has no relation to the previous MERGED clustering a labels
sidecar was built against. A later cluster only run's label reuse
remap read merge graphs' output as though its community field were
that previous clustering, and silently mismatched almost every
community since it was actually per source ids.
Add a previous flag that reads a prior merged graph.json and restores
its real community value onto each node that survives the merge, by
node id, before the per source offset is ever written out. A node
with no match (new since the previous merge) gets no community rather
than keeping an unrelated per source offset, since an invented value
is worse than none here. The per source partition stays available
regardless, in local community.
Fixes#3858.
(cherry picked from commit 60e153176c)
Publish the community-label and signature sidecars via the existing
atomic write helper and write labels before the .sig, so an interruption
between the two leaves the .sig trailing (which the staleness guard reads
as a mismatch and safely re-labels) rather than silently keeping stale
labels for a clustering that no longer exists.
Cherry-picked from #3853 by @shashank-100; the unrelated CI-matrix change
(dropping Python 3.13/3.14) in that PR was omitted.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
extract with an out destination outside the scan root stayed clean on
a fresh scan, since that branch already passes cache_root to detect,
but leaked the word count stat index cache file into the scan root on
the very next incremental run against the same destination.
detect_incremental had no cache_root parameter at all, so there was
nowhere for the caller to pass it even if it wanted to, and the word
count cache fell back to anchoring at the scan root instead of the
requested out directory.
detect_incremental now accepts cache_root and forwards it into its
own internal call to detect, and the incremental branch in cli.py
passes cache_root equal to out_root, matching what the fresh scan
branch a few lines below it already does.
Fixes issue 3847.
(cherry picked from commit 9553cae7d9)
FileNotFoundError is a subclass of OSError, so a missing Cargo.toml at
the scan root was caught by the same handler meant for ImportError
and connection failures and exited the whole process. That fires
after the AST pass has already completed, discarding every file of
completed structural extraction work just because an optional
enrichment source found nothing to read, an ordinary condition for
any repo whose manifest lives in a subdirectory, such as a Tauri app
that keeps it under its own separate frontend adjacent crate folder.
cargo_result already starts as the empty, handled shape the merge
step downstream expects, so catch FileNotFoundError on its own, print
a note, and let extraction continue with an empty cargo result instead
of exiting.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
(cherry picked from commit 94cc22be3b)
- #3716 (Solidity): non-callable members (struct/enum/event/error/state-var) had
no signature and fell back to the LINE NUMBER for their node id, churning ids on
incremental rebuilds. Use the member name (unique per kind in a contract scope)
as the discriminator instead.
- #3719 (method-ghost dedup): gate the candidate index on a method-shaped label
(leading '.' AND '()' suffix) so a dotfile label like '.env' is never treated as
a method; normalize the index/lookup key via make_id for parity with the alias
path. Adds a negative test proving two same-named methods in one file are left
unmerged (the len==1 ambiguity guard).
- #3714 (Erlang): the remote-call resolver reads module/name/arity off node
metadata, which the incremental context builders stripped for non-Ruby langs,
so foo:bar() silently failed to resolve on 'graphify update'. Forward the Erlang
resolution metadata keys through the watch.py/cli.py allow-lists (same fix shape
as the elixir _elixir_module marker).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
god_nodes ranks purely by graph degree, independent of the community
partition, so clearing it to an empty list on the stale path and
relying on the wiki subcommand's own recompute-if-empty fallback
further down worked today only because wiki happens to be the only
current reader of this value. Recompute it directly here instead, so
a future reader that does not carry the same fallback cannot silently
lose real data.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
(cherry picked from commit b071d6e788)
The node id set comparison missed a whole class of staleness: a merge,
a split, or a single node moving from one community to another can
leave the overall node set exactly unchanged while still describing a
different clustering, so a stale sidecar that happened to still cover
the same nodes kept winning. Compare each side's set of community
blocks instead, which catches a partition change even when the flat
node set stays identical, while still ignoring a pure id renumbering
of the same partition per the existing #1667 guard.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
(cherry picked from commit 30a5955931)
Fixes issue 2386.
The existing fallback only reconstructed communities from graph.json
when the sidecar was missing entirely; a sidecar that exists but was
written by an earlier clustering pass looked identical from the
outside and kept winning, since update and watch never regenerate
.graphify_analysis.json.
This compares the node id set each side covers, cheap and already in
memory, rather than the community ids themselves, since those can
renumber run to run even for the same partition. A mismatch means the
sidecar is stale: prefer the fresh reconstruction, recompute cohesion
with score_all so wiki articles and the HTML export do not swap one
silent degradation for a smaller one, reset gods_data so the existing
self heal at the wiki export site recomputes it, and print a warning
naming the exact command that refreshes the sidecar, since the whole
point of this issue is that nothing currently signals the divergence.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
(cherry picked from commit d4eb7af599)
Toward issue 2386: this hoists the reconstruction from graph.json's
per node community attribute out of the "sidecar is missing" branch
so it runs every time, with no behavior change yet since it is still
only used when the sidecar produced nothing. The next commit compares
it against the sidecar to also catch a sidecar that exists but is
stale.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
(cherry picked from commit 0932abc4e0)
#3603 resolves cross-file alias/import/require/use targets onto the real
defmodule node. Two guards ported from the competing #3566 so the resolver
does not over-reach:
- Only top-level modules are indexed as resolution targets. extract_elixir
marks them with _elixir_module; a nested defmodule (labeled with its bare
inner name) is left unmarked so an unrelated `use <Name>` from another file
cannot latch onto it. The marker rides through incremental rebuilds via the
resolution-context allow-list in watch.py and cli.py, so the fix does not
decay on the `graphify update` / watch path.
- A same-file alias is left unresolved: the import edge's source is the file
node, which already `contains` that module, so retargeting would duplicate
the pair as a weaker `imports` and clobber `contains` in the non-multi graph.
Adds regression tests for both guards plus an incremental-rebuild test that
routes through the real watch._rebuild_code builder (the path #3566's own
test bypassed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The prior retry suggestion, a bare repo relative path, resolves to the
file node rather than the symbol, so following it never actually
disambiguates anything. Now that path::symbol exists, point both the
explain CLI and the shared MCP resolver at it, using a generic
placeholder rather than one of the rival paths so the message stays
the same regardless of node iteration order.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
(cherry picked from commit c18f3d0a2c)
`tag or source.parent.parent.name` is empty whenever the graph is not two levels
below a named directory: `graphify global add /tmp/graph.json`, or any relative
path. Every node is then prefixed `::`, the prune key is `""`, and the manifest
entry cannot be removed because the `remove` subcommand reads an empty tag as a
missing argument.
Infer through `distinct_repo_tags`, the helper `merge-graphs` already uses, so the
two entry points cannot diverge again and the degrade is `"repo"`; absolutize
first so a relative path names the repo dir rather than the cwd placeholder. An
omitted `remove` tag stays a usage error, while an explicitly empty one is now
addressable so stores written by earlier revisions can be cleaned up.
(cherry picked from commit 6a317e6ef2)
When a member call's receiver type resolves to zero in-corpus declarations, park
{callee, receiver_type, lang, line} (names only, #3150-safe) on the caller; a merge-time pass
emits one calls/cross_repo/INFERRED edge only when the receiver type resolves to exactly one
declaration in another repo owning exactly one matching member (language-suffix matched,
non-sourceless). Strong-signal, single-definition guarded, edges-only — composes with the
#3007 cross-repo type link and the #3068 community offset. Wired for Java/C++/C#/Swift.
god_nodes had no exclusion parameter, so --exclude-hubs only reached cluster() and god-node
ranking ignored it. Apply the same percentile threshold cluster() uses (byte-identical
formula), thread it through the build/analyze/MCP paths; the default (no exclusion) output is
unchanged and the node dict shape is preserved for wiki/report consumers.
The raw and clustered write paths carried a byte-identical ~27-line block differing only in
whether the flag is read from merged or G.graph. Extract _handle_unverified_semantic_shrink
so the two paths cannot drift; behavior-identical (the #3203 and prune-sweep suites pass).
On the default incremental path a semantic chunk that returned successfully but under-produced
(prior semantic count > 1, fresh < prior) bypassed every guard, silently shrinking the graph.
Flag that per-source shrink (computed only from fresh nodes, so a genuinely deleted file is
never flagged) and route it through the existing shrink-guard refusal + manifest un-stamp so
the source re-extracts next run; --allow-partial overrides. Does not touch the prune path.
The hook-guard matched a search command appearing anywhere in the intercepted Bash string
(a comment, a quoted -m message, an echo), firing false nudges. Match only an executed
command position (start-of-segment after a shell operator, wrapper-aware), so prose lookalikes
stay quiet while genuine pipes/&&/;/$() invocations still fire.
--force --code-only over an existing graph set incremental_mode, so unchanged code files
were skipped and stale alias/import resolution persisted. Separate scan-mode from
graph-merge: --force --code-only now full-scans all code (via a new merge_existing_graph
gate) while still carrying the existing semantic tier forward (preserving the #2923 fix) and
still pruning deleted/excluded files. Other flag combinations are unchanged.
The #479 shrink guard is disarmed when a build is known-incomplete, but a run that silently
lost content to a hollow, unparseable, or file-omitting chunk was treated as complete, so it
could overwrite the existing graph with a smaller one. Mark those cases incomplete (uncovered
files or partial-semantic files) so the guard stays armed and refuses the shrink; a complete
run is untouched, a #2880-retry-recovered chunk does not count, and --allow-partial still
overrides.
merge-graphs kept each input's community ids as-is, so community 0 from repo A and community
0 from repo B fused into one meta-community spanning unrelated repos. Offset each input's
community ids past the running maximum so every input's communities stay globally distinct
after merge, preserving within-input structure and recording the original id in
local_community.
Complete the read-side BOM fix: the graphify update (no-args) recovery path at cli.py read
.graphify_root with plain utf-8, so a PowerShell-written BOM would fail the exists() check
and abort with path not found. Switch it to utf-8-sig like the other five reader sites.
Also clean up two invalid-escape f-strings in the windows skillgen test.
When merging graphs from multiple repos, a type declared in more than one repo under the
same fully-qualified namespace and name (a shared contract type) now gets a same_type_as
edge linking the declarations, so a cross-repo contract is navigable. Matching requires a
non-empty namespace plus label, a real sourced type declaration (not a method/field or
sourceless stub), and at least two distinct repos, so two unrelated types that merely share
a short name are not linked; the edge is INFERRED/0.9.
The #1757 guard scoped the semantic-cache WRITE to dispatched files, but the unfiltered
fresh result still fed build_merge, whose replace-set logic swapped a non-dispatched
file's entire prior contribution for a stray misattributed fragment (and logged the
'skipped out-of-scope source_file' warning). Apply the same allowlist to the result dict
before it reaches the merge, via a shared _semantic_source_matcher so the write guard and
the graph filter cannot drift.
A semantic result with no nodes and no hyperedges (only edges, or nothing) was cached and
stamped into the manifest, so an empty/degenerate LLM reply for a file froze that file:
detect_incremental saw it unchanged and never re-dispatched it. Reject zero-node results
from the cache read and write, drop edges from the manifest stamp tuple, and heal an
existing manifest by re-queueing files that were already stamped with a zero-node result.
A `--code-only --force` rebuild over an existing graph dropped the doc/paper/image
semantic tier, because force took the full-rebuild path and code-only never re-dispatched
those files. When an existing graph is present, keep incremental mode so build_merge
carries the semantic layer forward; files deleted from disk are still pruned.
Adds --no-dedup (default off, so dedup stays on) to skip the fuzzy near-duplicate merge
pass on build and incremental merge, for operators who would rather keep distinct symbols
that fuzzy-matched than pay the merge. Exact-id uniqueness is unaffected (it is a graph
structural invariant, not a dedup responsibility), and the flag arms the #479 shrink
guard so a surprising node drop is refused loudly. Mutually exclusive with --dedup-llm.
The label and cluster-only commands did not pass the viz node limit to to_html, so a
graph over the node limit raised and the except branch silently unlinked the graph.html
that update had produced. Always pass the limit (the aggregated community meta-graph
renders instead of raising), preserve the existing file on a failed render via an atomic
publish plus a stale marker, and regenerate a missing graph.html on the no-topology-change
fast path without reclustering.
The out-of-project read guard treated any non-absolute path as cwd-relative, but on
Windows a rooted-but-driveless path (`\foo\bar`) is not absolute yet resolves against
the current drive root, outside the project. Classify with a platform-correct predicate
(`not root and not drive` under the host path flavour) so the containment check is
reached. On POSIX the predicate reduces to `not is_absolute()`, so no behaviour changes
there.
The query header now leads with the graph file it opened and its node count,
shown relative to the CWD when the graph sits underneath it and absolute when it
does not — surfacing the case where a query run from a parent project silently
answers from the wrong corpus. graph_path is optional, so the header is
byte-identical for callers that do not pass it; the CLI query command and the MCP
query tool, which already resolve the path for querylog, now pass it.
affected anchored an absolute-path seed to Path.cwd(), so running it from
anywhere but the repo root made relative_to(cwd) raise and the query fell
through unmatched, silently returning nothing. It now derives the repo root
from the graph's own location (<root>/graphify-out/graph.json) and anchors the
seed there, so an absolute seed resolves regardless of cwd. Composes with the
#2707 relative-seed fix (root defaults to cwd for other callers).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2543 (adopts PR #2546, thanks @michaelxer): a failed extraction is no
longer stamped in the incremental manifest as up-to-date, so graphify
update retries it instead of skipping it forever; a manifest already
poisoned by the old behavior is healed on the next run; genuinely
unchanged files are not re-processed. Extended to the watch save_manifest
paths too.
#2554 (adopts PR #2555, thanks @annieyii): the claude-cli backend now
inspects the stdout envelope for an is_error result (e.g. a rate limit
returned with exit code 0) and raises it on both the zero and non-zero
exit paths, instead of parsing it as an empty success and bisecting
against a live rate limit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
cluster-only warns when --backend/--model/--batch-size are ignored on the
label-reuse path; the community-label prompt key no longer collides with
the discard sentinel (an echoed key was silently dropped); tree --root
exits non-zero when it matches no source file instead of flattening the
tree; and cluster-only stamps built_at_commit from the analysed graph, not
the shell cwd. Also folds in the cluster-only refused-write guard from
PR #2522 (thanks @aniJani).
Thanks @elecnix for the report.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2486 (thanks @adminwat): normalize dict-shaped hyperedge members to ids
(or drop with a warning) so a malformed hyperedge can't abort a completed
merge with a TypeError.
#2484 (thanks @sortakool; approach from @oleksii-tumanov's #1691):
merge-graphs relabels hyperedge member ids and ids with the repo prefix,
unions both inputs' hyperedges instead of clobbering, and writes both
persistence slots.
#2485 (thanks @sortakool): build_from_json reads hyperedges from the
top-level and nested slots; a full validation wipeout is reported loudly.
#2490 (thanks @PapiScholz): the skill Step-5 flow passes curated
community_labels to to_json, so graph.json ships community_name.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2411 (thanks @JensD-git): key the C# partial-class merge on assembly
(nearest ancestor .csproj/.fsproj/.vbproj) in addition to namespace and
name, so same-name partial classes in different assemblies stay distinct
while genuine partial halves within one project still merge. Fixes a
0.9.32 regression from #2332.
#2437/#2438 (thanks @aryanbonigala, builds on PR #2439): incremental
rebuilds no longer drop member-call and indirect_call edges from a
changed file into an unchanged target. Re-resolution now sees the
unchanged corpus (nodes, contains/method edges, and _callable markers,
which now persist to graph.json like _origin); edges to a genuinely
removed target are still evicted.
#2444/#2445 (thanks @Baziar, builds on PRs #2461/#2458): a
BrokenProcessPool triggers the sequential fallback instead of being
swallowed per future, a failed worker file is retried sequentially
rather than merged as empty, and a whole-pass AST failure on a fresh
build exits non-zero instead of writing a zero-node graph
(--allow-partial opts into a best-effort partial graph).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
graphify path decided hop direction from the persisted source/target order,
so a link stored in flipped order (pre-#563 graphs, raw dumps, merge-driver
output) printed backwards; explain had the same defect, and the query/merge
load shims clobbered in-file _src/_tgt markers. All now honor _src/_tgt (the
build-side direction truth), falling back to arc order for markerless files.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`_find_node` ranks matches but never reports that a tie was broken, so callers
taking `matches[0]` present one arbitrary file as the answer. Two workspaces
that each define `MetricsPort` put both nodes in the same `exact` tier,
separated only by `G.nodes()` iteration order — reorder the graph and the same
query answers with a different file, equally confidently.
`affected` already declines this via `resolve_seed()` returning None, and
`path` warns when the top two scores are within 10%. Only `explain` (and the
MCP `get_neighbors` tool, which shares the matcher) was silent.
Split `_find_node` into `_find_node_tiers` (logic unchanged, tiers exposed) plus
a flattening wrapper, and add `find_node_ambiguity()`, which returns rivals only
when the winning tier spans multiple source files. Matches within one file (a
file node plus its members) stay ordinary precedence and resolve as before.
`_disambiguate_file_node_labels` (#2032) already relabels colliding *file*
nodes; this covers the symbol case it does not reach.
#2212: benchmark, the merge-driver graph load, and callflow_html crashed
or silently failed on a --no-cluster graph.json (edges stored under
'edges', not 'links'). A shared load_node_link_graph helper normalizes
links/edges before node_link_graph and is used at all three sites.
#2210: _stale_graph_sources compared graph source_file spellings to the
scan with a raw string test (no NFC), and pruned any non-match with no
liveness check, so alive files (macOS NFD paths, legacy basenames) were
pruned as 'deleted'. It now compares NFC-on-both-sides and is fail-closed:
a corpus-missing source whose file still exists is pruned only when the
exclusion is provable, else kept with a warning. Prune message corrected
to 'deleted or excluded'.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An incremental `extract --no-cluster` wrote only the changed files over
graph.json with no merge, dropping every node/edge owned by an unchanged
file; and the id-canonicalization pass only learned batch files, so the
changed file's cross-file edges kept absolute-path target ids and
dangled. The raw path now merges the existing graph forward with the same
replace/prune semantics as the clustered path (new merge_raw_extraction
helper in build.py, shared loader), refuses to overwrite a corrupt
existing graph, and the remap pass now also learns in-root edge
target_file paths (existence-gated) so cross-file targets canonicalize.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The heuristic over-matched and silently dropped legitimate files:
- prose `.md`/`.rst` whose topic slug ends in a keyword (privacy-tokens.md,
token-economics.md) — only code was exempt, not prose;
- the unbounded Stage-2 `service.account` substring (regex `.` wildcard) matched
real source (google/oauth2/service_account.py) and prose slugs.
It also MISSED real secrets (.npmrc, .pypirc, secring, .git-credentials, and
case variants on case-insensitive filesystems), which were being indexed.
Fix: move service_account/aws_credentials to the boundary-checked keyword path
(so real source is spared, downloaded key files still drop), add a prose-note
carve-out (multi-word slugs indexed, bare `secrets.md`/`token.md` still dropped),
tighten id_rsa with a left boundary, add the missed secret dotfiles + secring,
lowercase the dir/segment comparisons, and count multi-dot slugs as multi-word.
Net effect is stricter on real secrets and stops the false-positive data loss.
Traceability: `graphify extract` now names the files skipped as sensitive (not
just a count), so a wrongly-flagged file is visible.
`graphify explain "<node>"` sorts a node's connections by neighbor
degree and shows only the top 20, appending a bare "... and N more"
for the rest. On a high-degree node (a logging/error helper called
from dozens of places is typical) that leaves the exact question
explain is meant to answer - "who calls this, what's the impact?" -
unanswered for 97% of the callers, with nothing pointing at where
they live (#2009).
The top-20 list and its ordering are unchanged (no behavior change
for nodes at or under the cutoff). Past it, the cut connections are
now grouped by (direction, source_file) with counts and printed
under a "Grouped by file:" section, sorted by count descending, so
the caller/callee distribution is visible without falling back to a
repo-wide grep. The aggregation itself is capped at 20 files with
its own "... and N more files" line for the pathological case where
the remainder is spread across more files than that.
Full per-caller detail behind a flag (--callers --all / --group-by=dir
as proposed in the issue) is left for a follow-up: it's a larger,
more opinionated surface (flag naming, pagination semantics) than
the literal complaint needs, and the default output no longer hides
the answer either way.
Four regression tests in tests/test_explain_cli.py: the pre-existing
truncation notice on a 30-connection node is unchanged, the grouped-
by-file output carries real counts (3 files, one at 4 and two at 3)
that sum back to exactly the cut total (no silent loss), a
byte-for-byte no-op check for nodes at/under the cutoff, and a
boundary check pinning the cutoff itself at exactly 20 connections
(no section) vs exactly 21 (one grouped entry) — the earlier tests
sit well clear of the edge and wouldn't catch a future off-by-one.
`graphify query` builds an undirected nx.Graph (so BFS/DFS can explore
both callers and callees of the seed node), but its text renderer
assumed the BFS/DFS visit order (u, v) was always the edge's
(source, target). On an undirected graph that assumption only holds
when the seed happens to be the caller: seeding on the callee makes
BFS/DFS visit the callee first, so a `caller --calls--> callee` edge
was rendered backwards as `callee --calls--> caller`. graph.json's
own source/target fields stay correct on disk; only the query
rendering was wrong.
`graphify path` and `graphify explain` don't have this problem
because they force directed=True on load (#849, #853), and the MCP
query_graph tool's _load_graph() does the same. Doing that for CLI
`query` too was tried and reverted: forcing a DiGraph makes
G.neighbors() return successors only, so a query seeded on a
leaf/sink node (no outgoing edges) found zero neighbors instead of
its callers — a recall regression, not just a display fix, and it
would make the CLI and MCP query tools diverge in what they discover
even though they'd render direction identically.
Fix instead mirrors the _src/_tgt pattern graphify/build.py already
uses for the same underlying problem (undirected storage loses
direction): the CLI now stashes each link's true source/target on
its edge data as _src/_tgt before constructing the (still undirected)
graph, and _subgraph_to_text renders EDGE lines from _src/_tgt when
present, falling back to (u, v) otherwise. Traversal itself is
unchanged, so recall is unaffected — verified against the unpatched
CLI, the node counts returned for the same seeds are identical before
and after this fix, only the printed edge direction changes.
Adds two regression tests in tests/test_query_cli.py seeding the same
`calls` edge from both endpoints; the callee-seeded case fails on the
prior code with the exact backwards-edge symptom above.
ayushcodes10