Completes #1774. The prior fix redirected the AST cache dir to CWD but
file_hash still called _ensure_stat_index(root) without the cache
location, so the hash fastpath's stat-index.json kept anchoring on the
key-root (the analyzed corpus) — leaving a stray graphify-out/cache/
stat-index.json inside a writable foreign corpus even though the AST
cache itself had moved to CWD.
Thread cache_root through file_hash -> _ensure_stat_index (which already
accepts it, #1747). Surfaced by an out-of-CWD parallel-extract edge case:
the leak was masked in the in-process test suite because _stat_index_root
is a set-once module global that an earlier test had already pinned. The
regression test resets that global to simulate a fresh process and
asserts the corpus stays clean while the stat index lands under CWD.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
With no explicit cache_root, extract() wrote graphify-out/cache/ under the
inferred common parent of the inputs — the analyzed source tree — so
scanning a read-only/foreign corpus silently polluted it.
The naive fix (point the root at CWD) breaks two other things that shared
the same parameter: file_hash keys become absolute/non-portable for an
out-of-CWD corpus, and the XAML/C# project-scan boundary would scan CWD
instead of the corpus. So split cache LOCATION from key/id ANCHOR:
load_cached/save_cached gain a cache_root arg for where the dir lives,
while `root` (inferred common parent) still anchors file_hash keys,
source_file relativization, node ids, and the XAML boundary. extract()
now locates the cache at CWD (or cache_root) but anchors on `root`; the
parallel worker tuple carries both. Existing callers passing cache_root
(CLI, watcher) are unchanged.
Adopts @SimiSips's #1802 (the CWD default + the two location tests) and
adds the decoupling plus a regression test that keys stay relative for a
corpus outside CWD — the property the one-line version would have lost.
Co-Authored-By: SimiSips <SimiSips@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Many nodes sharing one generic label (framework route handlers all
labelled GET/POST, a repeated handler) consumed every BFS seed slot, so
query traversal explored near-identical neighborhoods and buried the
actual target. Seed selection now dedups by normalized label (GET/Get/get
collapse together), keeping one representative per label, and the per-term
guarantee loop honors the same cap so it can't reintroduce a dupe.
Adopts @devcool20's seed-dedup from #1832 but drops that PR's second
mechanism — a per-label multiplicity penalty applied inside the shared
_score_nodes. That scorer also resolves shortest_path/explain endpoints,
so dividing scores there silently reweighted path/explain (out of scope
for #1766 and able to flip endpoint selection); the dedup alone bounds
the flood. Also normalizes the dedup key (the PR keyed on the raw label).
Adds the tests the PR was missing: dedup of homonymous labels,
case/diacritic normalization, per-term-guarantee cap, and a guard that
identical-label nodes still score equally in _score_nodes.
Co-Authored-By: devcool20 <devcool20@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The #1835 fix scoped save_semantic_cache's final CLI write to an
allowed_source_files allowlist, but the per-chunk incremental checkpoint
in llm.py `_checkpoint_chunk` — the write that actually runs on every
`graphify extract`/`update` via extract_corpus_parallel — still called
save_semantic_cache with no allowlist. A chunk whose model result
mis-attributes a node's source_file to another corpus file would merge
that stray fragment into the victim's cache entry (merge_existing=True).
Scope the checkpoint write to the chunk's own dispatched files (FileSlice
-> .rel, bare Path -> the relative source_file). Also hoist the
`import warnings` in cache.py to module level.
Adds an extract_corpus_parallel integration test: a chunk dispatching
only A.py that returns a node attributed to already-cached B.py must
leave B.py's cache entry untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#1831 — `graphify export graphml` crashed on any dict/list-valued
attribute (per-node metadata dict, graph-level hyperedges list) because
nx.write_graphml only accepts scalars; a real ~2,300-node graph failed
every export and left a 0-byte .graphml behind. to_graphml now coerces
None->"" and JSON-serializes non-scalars across graph/node/edge scopes
(int/float/bool/str pass through), and writes atomically via a temp file
so a failed export can't leave a partial file. Closes#1830.
#1807 followup — adopt @varuntej07's explicit in-guard sys.stdout.flush()
from #1811: piped stdout is block-buffered, so a small fully-buffered
output would only flush at interpreter shutdown (outside the guard),
where a closed-pipe reader escapes as a noisy shutdown error and nonzero
exit. Flushing inside the try closes that gap. Closes#1811.
Reported by @hofmockel (#1831) and @varuntej07 (#1807/#1811).
Co-Authored-By: hofmockel <hofmockel@users.noreply.github.com>
Co-Authored-By: varuntej07 <varuntej07@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#1810 — detection read only .gitignore/.graphifyignore, never
.git/info/exclude, which is where git records local-only excludes and
where `git worktree add` writes nested worktree paths. graphify walked
into those worktree copies and the graph exploded (one 5-worktree repo:
9.4k nodes/10MB -> 210k nodes/311MB, ~77% duplicate). detect now loads
info/exclude at lowest precedence (below every per-dir .gitignore, per
git, so a nearer `!` still wins) and resolves the linked-worktree /
submodule case where `.git` is a file to the shared common git dir.
#1809 — two git-hook gaps: (a) post-checkout never honored
GRAPHIFY_SKIP_HOOK, so the var stopped commit rebuilds but not
branch-switch ones; now checked in both. (b) with core.hooksPath shared
across worktrees, a commit in any linked worktree fired post-commit,
which wrote a rogue delta-only graph.json into it and raced deploy/CI
`git clean` against the detached rebuild. Both hooks now short-circuit
in a linked worktree (git-dir != git-common-dir), comparing ABSOLUTE
paths so the primary checkout (where --git-common-dir is the relative
".git") is never false-positived and skipped.
Adds regression tests: info/exclude honored + negation precedence;
both hooks honor the skip env and carry the worktree guard; and an
end-to-end guard check against a real `git worktree`.
Reported by @cdahl86-cyber (#1810, #1809); the worktree guard was
co-developed with @Claude-Madera's PR #1806.
Co-Authored-By: cdahl86-cyber <cdahl86-cyber@users.noreply.github.com>
Co-Authored-By: Claude-Madera <Claude-Madera@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#1807 — piping graphify into a reader that stops early (head,
Select-Object -First N, sed q) disconnected stdout mid-write, raising an
unhandled BrokenPipeError (OSError(EINVAL) on Windows) and exiting 255,
so CI wrappers and agent harnesses read a successful query as a failure.
The console entry point now wraps the CLI body: a closed-pipe reader is
treated as success — stdout is redirected to devnull so shutdown flush
can't raise again, and the process exits 0. Adds a subprocess regression
test.
#1804 — .nox/ (nox virtualenvs, tox's successor, same .nox/ tree shape)
was missing from _SKIP_DIRS while .tox was present, so nox site-packages
got fully indexed (one repo came out 91% venv noise). Added next to .tox
with a regression test.
Reported by @varuntej07 (#1807) and @igorregoir-lgtm (#1804).
Co-Authored-By: varuntej07 <varuntej07@users.noreply.github.com>
Co-Authored-By: igorregoir-lgtm <igorregoir-lgtm@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A solution folder is a virtual grouping, not a file: VS writes its name
as both the display name and the "path" (name == path, no real file).
extract_sln resolved it to an absolute filesystem path anyway and keyed
the node id off that. The CLI id-relativization pass only remaps ids of
real files in the scan set, so a virtual folder never matched and its
absolute id (with the local username) survived into a committed
graph.json.
Detect solution folders (name == path) and key their id/source_file off
the folder name only; real project files still resolve as before. Adds a
regression test asserting the folder node id is relative.
The earlier fix (0.9.13) covered .csproj/.sln file nodes but missed the
virtual folders, so #1789 was closed prematurely; this completes it.
Reported and diagnosed by @fremat79.
Co-Authored-By: fremat79 <fremat79@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
querylog wrote every query/path/explain question + corpus path (and full
responses under GRAPHIFY_QUERY_LOG_RESPONSES) to a default-on, unbounded,
fail-silent plaintext file at ~/.cache/graphify-queries.log — outside any repo's
.gitignore/retention, and undocumented. A default-on plaintext record of
proprietary queries contradicts graphify's on-device / no-telemetry posture.
Flip to opt-in: _log_path() returns None unless GRAPHIFY_QUERY_LOG_ENABLE=1
(default path) or GRAPHIFY_QUERY_LOG=<path> is set; GRAPHIFY_QUERY_LOG_DISABLE=1
still forces it off (back-compat, wins). Document all four env vars in the
README (the old entries implied default-on). Regression tests cover
default-off, both enable paths, disable-wins, and that log_query writes nothing
without opt-in.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The semantic pass mints a document node <slug>_doc; the markdown quick-scan
(extract_markdown) mints the bare <slug>. After a semantic build, a `graphify
update` (AST path) re-runs the quick-scan and the graph ends up with BOTH — one
document as two disconnected nodes, the file's edges split between them (semantic
`references`/hyperedges on the _doc twin, quick-scan cross-links on the bare
one). path/query traversals dead-end on the wrong twin; degree and communities
split.
build_from_json now reconciles the pair: when <slug> and <slug>_doc both exist
with the same source_file and both are file_type=document, remap the bare node
into the semantic _doc node (canonical, richer edges) and repoint its edges and
hyperedges. Remap-induced self-loops are dropped; pre-existing ones are left
alone. Gated to document twins for the same file, so a code symbol `foo` and an
unrelated `foo_doc` never merge. Regression tests for both.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_reconcile_existing_graph treated "source identity absent from the collected
corpus" as deletion and evicted its nodes/edges/hyperedges. But corpus absence
is ambiguous: it's also what you see when a file still exists and merely stopped
being collected (ignore rules or filters changed). Upgrading into the merged-
.gitignore scan semantics (#1363) mass-evicted 655 nodes from a deliberately-
built, .gitignore'd docs dir whose files were present the whole time — reported
as a successful rebuild.
Fail-closed: before evicting a corpus-absent identity, require Path(identity)
.exists() is False (identity is an absolute path). Alive-but-excluded sources
are preserved (nodes, edges, hyperedges) and a loud line reports how many were
kept and why. True deletions and renames still evict (old path gone from disk);
a full extract --force still purges deliberate exclusions via the AST ownership
rule. Existence is memoized (one stat per file that left the corpus).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
build_merge already drops a re-extracted file's stale base nodes before merging
(replace-per-source), so on current code an EDITED file passed only in
new_chunks is handled correctly. But the prune step still removed every node
whose source_file was in prune_sources, with no guard for re-extracted files —
so a caller following the old edit-workflow (pass the changed file in BOTH
new_chunks and prune_sources) had its freshly-built nodes deleted after the
merge, silently losing a concept whose label survived the edit.
Exclude new_sources (files present in new_chunks) from prune_set: a re-extracted
file is being replaced, never deleted, so "replace" wins over a contradictory
"delete" of the same source. Genuine deletions (in prune_sources but not
new_chunks) still prune. Regression tests for both.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The absolute-path-in-node-ids leak reported on 0.8.19 is already fixed on v8:
detect() returns paths relative to the scan root, so the CLI-produced graph.json
uses relative structural node ids (portable, no username/home leak). Lock it with
a regression test that extracts the same corpus from two different absolute
checkout dirs and asserts identical, leak-free node ids.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`graphify path` committed each endpoint to _score_nodes()[0]. The full-query
bonus tier only fires when the query equals/prefixes a label, so a query that is
a token subset of the intended label ("Reject-everything judge" vs "Degenerate
Reject-Everything Judge") got no bonus and a node prefix-matching one rare token
("Rejection Summary") could out-score it on IDF alone — anchoring the path on an
unrelated, often disconnected node and returning a false "No path found".
_pick_scored_endpoint() scans the score-ordered list and takes the first
candidate whose label contains EVERY query token, falling back to scored[0] when
none does — so when the head already full-matches (the common case) resolution
is unchanged. Wired into both the `path` CLI and the MCP _tool_shortest_path.
The close-runner-up ambiguity warning now fires only when the picked endpoint is
the raw score head (a full-token override was chosen on coverage, not score, so
the head's margin is irrelevant).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
suggest_questions()'s "isolated/weakly-connected nodes" filter was missing the
`file_type != "rationale"` exclusion that report.py's Knowledge Gaps section
already applies, so the same GRAPH_REPORT.md reported two different counts for
the same concept (757 vs 245 on a real graph) — an internal inconsistency that
made a healthy graph look like a documentation problem. Add the same filter so
both computations agree.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
extract_bash only created a cross-file edge for `source x.sh` / `. x.sh`. The
two most common ways one script runs another — `bash x.sh` and `./x.sh` —
produced no edge, so in any repo where scripts invoke each other by execution
the call topology was missing (each script left an isolated file+entry pair).
Emit a `calls` edge (context `script_invocation`) from the caller's entry (or
enclosing function) to the invoked script's entry node, for script-runner
commands (bash/sh/zsh/ksh/dash <path>) and bare `./x.sh`, but only when the
target resolves to a real .sh file on disk — so no phantom edges to missing or
function-shadowed names. Verified end-to-end: the edges land on real target
nodes (no dangling drop at build).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
.rake files are plain Ruby (Rake's task DSL is ordinary method calls), but the
extension was gated out everywhere, so rake tasks were classified as
unsupported, skipped, and their calls invisible. Add `.rake` to all seven `.rb`
gates the reporter mapped:
- detect.CODE_EXTENSIONS (classification)
- extract._DISPATCH (extractor dispatch)
- extract._LANG_FAMILY_BY_EXT-adjacent language-name map (.rake -> ruby)
- the ruby_member_calls LanguageResolver suffix set
- both `.rb`-suffix filters in ruby_resolution.py (raw-call gather + class-def index)
- analyze language-stats map
- build repo-tag map
The extractor already parsed the content; this is purely extension routing.
Regression test: a `.rake` task's `Widget.tally` resolves cross-file to the
`.rb` definition.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The reported bug — a method chained directly onto a `new X(...)` expression
(no intermediate variable) producing no calls edge — is already fixed on v8:
`new Merger(ctx).Combine(...)` emits calls -> Merger.Combine. Add a regression
test so the fluent new-expression receiver stays covered.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_rewire_unique_stub_nodes gated merge targets through _is_type_like_definition,
which rejects any label ending in `)`. So a function referenced from another
module (passed by name, e.g. FastAPI's Depends(get_db)) left its reference edge
dangling on a sourceless name-only stub while the real def had zero incoming
edges — "who references this function" returned nothing. Class/type symbols were
fine; only functions/methods suffered.
Top-level function defs (label `name()`, not `.name()` methods or `Class.m()`
qualifiers) are now eligible rewire targets, but only when:
- the label key matches exactly one such function corpus-wide (existing
unique-candidate guard — two same-named functions stay unresolved), AND
- the candidate shares a language family with the stub's referrers, so a
Python `get_db` reference can't bind to a unique Go `get_db()` (#1718/#1749
interop guard), AND
- the stub is not used as a supertype (inherits/implements/extends) — you
don't inherit from a function.
Types are unchanged. Regression tests: cross-module function ref binds to def;
cross-language, ambiguous, and supertype cases all correctly left unresolved.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The TypeError reported on 0.1.14 is already fixed on v8: sanitize_label coerces
None ('if text is None: return ""') and the source_file call site guards with
str(data.get("source_file") or ""). Add regression tests (unit + to_html
integration with null label/source_file) so it can't silently regress.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The live-introspection FK query joined information_schema.referential_
constraints, which Postgres only exposes for constraints where the current user
has WRITE access to the referencing table. A read-only introspection role
therefore got zero FK rows — while tables/views/routines still appeared (SELECT
is enough for those views) — so the graph silently lost every `references`
edge, contradicting the documented FK-mapping behavior.
Switch to pg_catalog.pg_constraint (world-readable, not privilege-filtered),
keyed by constraint oid rather than name — which also fixes a latent bug where
same-named constraints on sibling tables could cross-match in the old
name-based key_column_usage joins. Composite-FK column order is preserved with
UNNEST(conkey/confkey) WITH ORDINALITY. Mock test asserts the query targets
pg_constraint and not the privilege-filtered view, plus composite-FK ordering.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
extract_json emitted `imports` edges for package.json dependencies and
`extends`/`$ref` edges for tsconfig.json to target ids (`_make_id("ref", ...)`
/ `_make_id(key)`) that it never created as nodes. build_from_json drops edges
to unknown node ids silently (that case is filtered out of real_errors), so
dependency and extends structure vanished from the graph on two of the most
common files in any JS/TS repo, surfaced only by diagnose_extraction after the
fact.
The extractor now adds the referenced target as a `concept` node (external ref,
not a corpus file) before emitting each edge, so the edges survive build.
Regression test asserts no dangling endpoints, the concept nodes exist, and the
import/extends edges land on real targets with no self-loops.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`graphify update` (the watch._rebuild_code / _reconcile_existing_graph path)
evicted every hyperedge whose source_file is in the corpus, because on a full
update every corpus file counts as "rebuilt" and hyperedge eviction reused the
node/edge eviction set. But the AST pass never emits hyperedges, so nothing
replaced them — doc-sourced hyperedges (what semantic extraction produces) were
permanently lost on the first update after a full build, even on a no-op run.
Split out a hyperedge_evicted_source_identities set scoped to genuinely deleted
(and symlink-target-outside) sources only, not merely-rebuilt ones. Replacement-
by-id (new_hyperedge_ids) and dangling-member cleanup are unchanged, so a real
semantic re-extraction still replaces its own hyperedges and orphaned ones are
still dropped. Parametrized regression test (full + incremental doc update).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Java member calls (`gw.charge()`) resolved by bare method name, so a call bound
to any same-named method in the corpus — e.g. `PaymentGateway.charge` and
`AuditLog.charge` were indistinguishable, producing phantom cross-class edges
and a false god node.
The extractor now preserves the receiver and its static type, and
_resolve_java_member_calls binds the call against the receiver's declared type:
explicit-type receivers and `this` are exact; current-class fields, method
parameters, and explicitly-typed locals resolve via a method-scoped type table;
a missing/ambiguous/inherited/chained receiver is skipped rather than falling
back to a bare name match (same single-owner god-node guard as the C#/Swift/Ruby
resolvers). Fully-qualified and nested-type receivers are deferred since they
need package- and nesting-aware type identity.
Verified: `gw.ping()/gw.charge()` (gw: PaymentGateway) bind to PaymentGateway,
the three charge() calls dedup to one edge, and no edge targets the same-named
AuditLog methods. Applies cleanly to the post-#1737 layout (extract.py +
extractors/engine.py). 13 new tests; full suite 3135 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Case 1 — `extract <corpus> --out <dir>`: the graph went to <dir> (cache_root is
already passed to the AST extractor), but detect()'s word-count/stat-index cache
uses the scan root, so a stray graphify-out/cache/ was created inside the corpus
(and left behind even when the run aborted at the no-LLM-key gate). Thread an
optional cache_root through detect() -> cached_word_count() -> _ensure_stat_index()
and pass out_root from the extract CLI, so the stat index lives under --out. Entry
keys are absolute paths, so relocating the index file is safe.
Case 2 — `cluster-only --graph <elsewhere>/graphify-out/graph.json`: outputs
(GRAPH_REPORT.md, re-clustered graph.json, labels, analysis, html) were written to
the CWD's graphify-out/, ignoring where --graph lives. They now write beside the
input graph when it sits in a graphify-out/ dir (another project/tenant's output),
while still falling back to the CWD for an arbitrary archived backup/graph.json —
the restore-into-place workflow #934 pins.
Regression tests for both cases; #934 still passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The extraction spec forbids cross-language `calls` edges, and build already
dropped cross-language INFERRED `calls`. But `imports`/`references` had no such
guard: an unresolved Python `import time` resolved by bare stem (the #1504
old-stem alias) onto a `src/time.ts` file node, welding a polyglot repo's two
language halves together. In the reporter's repo three such edges were the only
bridge between 2409 Python and 1403 TS nodes, so every backend<->frontend
shortest path routed through time.ts, inflating its betweenness ~90x and making
it the #1 reported god node.
Hoist the interop-family map to a module constant and extend the edge-loop
guard to `imports`/`imports_from`/`references`. For these relations the edge is
dropped only when BOTH endpoints are known code languages of different families,
so a config/manifest -> code reference (unknown ext) is never mistaken for a
phantom. `calls` behavior is unchanged (still INFERRED-only, still drops when
either family differs). Regression tests: py->ts import dropped, ts->ts import
kept, config->code reference kept.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When the [sql] extra is absent, .sql files are counted as code and scanned but
extract_sql returns an error result and zero nodes — and the graph builds
"successfully" with the entire SQL corpus missing. Neither existing warning
catches it: #1666's zero-node warning skips results carrying an "error", and
#1689 only covers files with NO extractor at all (.sql HAS a dispatch entry).
extract() now scans per-file results for a "not installed" error, groups the
affected files by extension, and prints a warning naming the extra that
restores the language (pip install "graphifyy[sql]"), via a small
_EXTRA_FOR_EXTENSION map (sql, terraform, dm). The map is only consulted after
an extractor actually reports the dependency missing, so it can't mislabel a
language that has a working fallback.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
build_from_json's ghost-node merge iterated set(G.nodes()), so when two nodes
shared a (basename, label) key the "canonical" survivor was chosen by CPython's
per-process string-hash order — rebuilding the same extraction JSON in a fresh
process could pick a different survivor, silently changing which node id
represents a concept. That breaks any workflow persisting ids across a rebuild;
concretely it broke the cluster->relabel step (community membership referenced
an id that the second build merged away -> KeyError in report generation).
Two changes:
- Pass 1 and Pass 2 now iterate sorted(node_set), not set(node_set), the same
deterministic-order fix the edge loop below already uses on purpose.
- The #1257 ambiguity guard is extended to the case it did not cover: two
NON-AST nodes sharing a key but from DIFFERENT source files are distinct
concepts, not an AST ghost/canonical twin, so the key is marked ambiguous and
both survive rather than one arbitrarily merging away (data loss). A genuine
same-file duplicate (identical source_file) is not flagged and still
collapses to one node.
Reported with a precise root-cause, minimal repro, and real-world impact by
@erasmust-dotcom.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The extract_terraform move #1721 proposed already landed on v8 via the #1737
decomposition (extractors/terraform.py exists, extract.py re-exports it, and
extractors/LANGUAGE_EXTRACTORS registers it), so the code move is a no-op now.
But the regression test @Cekaru added with it had no equivalent on v8. Salvage
and generalize it: sweep every LANGUAGE_EXTRACTORS entry and assert graphify.
extract re-exports the SAME object (facade identity) and the registry maps to
it (registry identity), plus the concrete terraform anchor from the PR. This
institutionalizes the re-export-identity guarantee the split relies on, so a
future move that forgets a facade re-export fails loudly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two classes with the same simple name in different Maven modules
(FinancialEntryValidator in payment/ and core/) already survive as distinct
path-scoped nodes on v8 -- the "node silently disappears" report from 0.9.9 is
fixed. But a cross-module field/type `references` edge was still left dangling
on a sourceless phantom stub: _resolve_java_type_references (#1318) re-pointed
implements/inherits/extends/imports edges to the real definition using the
importing file's `import` statement, but its REPOINT_RELATIONS omitted
`references`, so bare-name resolution's shadow stub survived for field types.
A query about the referenced class could then miss it.
Add `references` to the Java resolver's REPOINT_RELATIONS. The C# sibling
already covers references; this brings Java to parity. The reference now
resolves to the imported package's class (falling back to same-package), and
the orphaned phantom is dropped. Regression test covers the ambiguous
two-module case: both reals present, no phantom, reference lands on the
imported class.
Reported with a precise root-cause and repro by @aviciot.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two correctness fixes found while analysing the reported 'graphify update
occasionally writes a partial graph.json' bug.
Enumeration (P0): detect()'s os.walk had no onerror handler, so any os.scandir
failure -- a transient PermissionError, or a directory created/deleted mid-walk
by concurrent writes (e.g. benchmarking racing the scan) -- was silently
swallowed and that entire subtree dropped out of the file list with no log, no
error. Downstream that becomes a silently partial graph.json. The walk now
records each skipped directory (surfaced as walk_errors in detect()'s result)
and warns to stderr, while still enumerating the rest of the tree. This stays
visible even when a --force/GRAPHIFY_FORCE rebuild bypasses the shrink guards.
Relatedly, to_json's #479 anti-shrink guard was fail-OPEN: a non-empty but
unreadable existing graph.json (corrupt or mid-write) proceeded with the
overwrite. It now fails SAFE -- refuse and point at force=True -- while an
empty/whitespace existing file (no nodes to lose) still proceeds. The size-cap
check keeps running before any read, so an oversized existing file is not
loaded into memory.
Pascal edges (P1): a class method declared in the interface section and defined
in the implementation section each emitted a "method" edge to the same node id,
and the edge helpers (unlike the node helpers) did not dedup, so ~half of a
Pascal/Delphi graph's method edges were doubled -- inflating degree/centrality
and tripping the #1739 cross-file resolver's single-owner god-node guard. Both
extractors now dedup edges on (source, target, relation).
Adds regression tests for all three behaviours.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both Pascal extractors resolved every call via a single file-wide
{method_name: node_id} dict, so two unrelated classes declaring a same-named
method (property accessors, generated COM/TLB wrapper classes) collapsed onto
whichever declaration was inserted last, producing wrong cross-class `calls`
edges. Resolution is now scoped: own class -> ancestor chain (inherits) ->
file-level free function -> unambiguous file-wide match; ambiguous at every
level emits no edge rather than guessing (same god-node guard as the Ruby
resolver).
Adds graphify/pascal_resolution.py, a corpus-wide post-extraction resolver
(registered via resolver_registry) that walks the inherits chain across file
boundaries, so a call from a manual descendant to a method it inherits from a
base class in a separate unit (the generated-base/manual-descendant split)
resolves. Also stops both extractors from emitting a duplicate base-class stub
carrying the referencing file's source_file, which collided with the real node
under cross-file id disambiguation. cache.py gives the new raw_calls bucket the
same portable-path treatment as nodes/edges so it round-trips.
Re-applied to the post-#1737 module layout (extractor hunks land in
graphify/extractors/pascal.py; registration stays in extract.py). Added one
adaptation the original PR predated: the cross-file resolver's god-node guard
now counts DISTINCT method nids, because the tree-sitter extractor emits a
method edge for both the interface declaration and the implementation, so the
same method_nid arrives twice -- without deduping, every inherited call looked
ambiguous and resolved to nothing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In a multi-term query, a lone generic term that exactly equals a short leaf
label (`home` vs a `home()` action, `list` vs a `list()` function) received
the full exact-tier bonus and outscored nodes matching several of the query's
terms by ~10x, so `_pick_seeds`' gap cutoff discarded the genuinely relevant
candidates and `path`-style consumers of `scored[0]` had no backstop.
`_score_nodes` now scales the per-term exact/prefix tier contributions by the
squared fraction of query terms the node's label matches
(`tiered * (matched / n_terms) ** 2`). Squared because the exact tier is 10x
the prefix tier; label hits only (source-path hits score but don't count as
coverage, so a colliding leaf in the target's own directory can't win its
exact tier back via path fragments); query tokens deduped order-preserving so
a repeated word can't quadratically restore the collision. Single-term and
full-coverage queries are unchanged (coverage == 1), keeping identifier
exact-match dominance. Guarded against zero-term division.
Dropped the unrelated README badge changes from the PR; kept the scoring fix
and its two regression tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Kotlin enum entries weren't extracted: the walker never descended into the enum
body (`enum_class_body` wasn't in _KOTLIN_CONFIG.body_fallback_child_types, so
_find_body returned None). Add `enum_class_body` to the fallback body types and a
`_kotlin_extra_walk` (dispatched for tree_sitter_kotlin, mirroring the Java/Swift
handling) that emits each enum_entry as a node with a `case_of` edge to the enum.
Re-applied from PR #1738 (@ivanzhl) onto the post-#1737 module layout: the walk
engine now lives in graphify/extractors/engine.py while _KOTLIN_CONFIG stays in
extract.py. Closes the Kotlin half of #1700 (Java was #1719).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`graphify extract` on a repo containing docs/papers/images hard-failed when no
LLM backend was configured — even for a user who only wants the code graph. The
only workaround was hand-building a .graphifyignore of everything non-code, which
is onerous (the "not code" set is far larger than the code set).
`--code-only` skips the semantic (doc/paper/image) pass entirely: it indexes the
code via pure local AST (no key required) and reports what it skipped ("skipping
N non-code file(s) ...") rather than silently dropping it. The no-key error on a
mixed repo now also points users at the flag. Code-only was always keyless; this
just makes a *mixed* repo usable without a key instead of failing outright.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`graphify uninstall` (and `graphify claude uninstall`) only looked at
`.claude/settings.json` and root `CLAUDE.md`. A user who relocates the
PreToolUse hook into `.claude/settings.local.json` and the `## graphify`
instructions into `CLAUDE.local.md` / `.claude/CLAUDE.local.md` - so they
are not committed to a shared repo - was left with both behind after
uninstall, which reported "nothing to do".
- `_uninstall_claude_hook` now strips the hook from both `settings.json`
and `settings.local.json` (factored into `_strip_graphify_hook`).
- `claude_uninstall` now strips the `## graphify` section from `CLAUDE.md`,
root `CLAUDE.local.md`, and `.claude/CLAUDE.local.md` (factored into
`_strip_graphify_md_section`), cleaning every present file rather than the
first, and always runs the hook cleanup.
- `_strip_graphify_md_section` reads the two newly-covered files defensively
so a non-UTF-8 or otherwise unreadable file can't abort uninstall.
Unrelated local-only files (no graphify content) are left byte-for-byte
untouched. The `--local` install flag suggested in the issue is left out of
scope; this change makes uninstall symmetric with wherever the config lives.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
merge-graphs prefixed each graph's node ids with `<repo>::` where repo was
`gp.parent.parent.name` — the graphify-out parent dir name. That tag is not
unique across inputs: `src/graphify-out` and `frontend/src/graphify-out` both
yield `src`, so a bare `app` node from a backend `src/app.js` and a frontend
`App.jsx` both became `src::app` and nx.compose silently merged them into one
node (one graph's label/source_file, both graphs' edges) — inventing false
cross-runtime `path` results with no warning.
New `distinct_repo_tags` guarantees a unique prefix per graph: colliding tags
are widened with their own parent dir (`frontend_src`), then an index suffix
backstops any residual duplicate. The handler prints a note when it disambiguates.
Verified end to end: the two `app` nodes now survive as `..._src::app` and
`frontend_src::app` instead of collapsing. Regression tests cover the merge case
and the tag uniquifier (pass-through, widening, and triple-collision fallback).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a regression test for the Date.now()/static-call shape (credit PR #1727 /
@2loch-ness6, who independently found the same fix): a capitalized builtin
receiver must not bind cross-file to a same-spelled user const/class. The same
guard added in 67f4f83 already covers it (verified); this locks the static-call
shape in, while allowing the legitimate same-file const reference.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`_resolve_typescript_member_calls` resolves a member call's receiver to a type
definition by casefolded label. For a builtin-typed receiver (`x: Date;
x.getTime()`), `_key("Date")` == "date" matched a same-named user `class DATE` /
`const DATE` in another file, binding the caller to it as a phantom
`references[call]` edge. In a real 3,368-file repo one module-local `const DATE`
accumulated 469 false cross-file edges (degree 472, betweenness 0.435) — a false
god node distorting path/god-node results.
Skip the resolution when the receiver type is an ECMAScript/Python builtin
global (Date, Promise, Map, ...), mirroring the guard the cross-file CALL
resolver already applies (#726). The guard is a strict no-op for genuine user
types, so legitimate constructor-injection member-call resolution (#1316) is
unaffected. Verified across new Date().method(), return-type, and var-decl-type
shapes: zero phantom edges, user DATE degree back to 1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to the previous commit on this branch. That fix's ambiguity check
missed a case: it detects "is this node the file itself" by checking
whether the node's id starts with the file's plain new_stem, but a
same-directory .h/.cpp pair that collides on their shared pre-extension id
gets salted apart by _disambiguate_colliding_node_ids into ids like
"tools_aolserver_utility_h_tools_aolserver_utility" -- no longer a clean
new_stem prefix.
That salted header silently failed to compute an empty suffix, so it never
entered the bare "utility" alias race at all, leaving an unrelated
wwwapi.masque.com/pages/utility.php as the lone (wrong) "unambiguous"
winner -- reproduced exactly against the real depot's
Tools/aolserver/utility.h and .cpp.
Detect "this node IS the file" by label instead: every file node's label is
its own basename regardless of what its id looks like after salting. That
keeps a salted file node in the alias competition, so the real collision
between the C header and the PHP file is correctly caught as ambiguous.
build_from_json's "pre-migration alias index" (#1504) registers each real
file's OLD-style bare-stem id (extension dropped) as an alias so a stale
cached fragment referencing that old form still resolves after an id-scheme
migration. It never checked for collisions: two unrelated real files easily
compute the same bare alias (e.g. "ping.h" and "ping.php" in different
directories both alias to "ping"), and dict.setdefault let whichever file
happened to iterate first in a Python set win, arbitrarily.
A dangling edge with a bare, deliberately-unscoped fallback target (e.g. the
C/C++ extractor's last-resort id for an #include it couldn't resolve to a
real path) would ride that alias onto whatever unrelated file won the
collision -- silently wiring, say, a C++ server file to an unrelated PHP
script, purely because both files happen to share a common basename
somewhere in the corpus.
Now every candidate for an alias is collected before any of them are
committed to norm_to_id, and the alias is only trusted when exactly one real
file claims it. Ambiguous aliases are dropped entirely, so the dangling edge
correctly stays dangling (and gets discarded) instead of merging two
unrelated files -- same "don't guess through ambiguity" principle already
applied to stub-node disambiguation and cross-file call resolution
elsewhere in this codebase.
Found via graphify-practice round 2: a `path` query between two unrelated
symbols routed through exactly this kind of bogus edge, traced back to
Dev/Poker/TDServer/server.cpp's unresolved `#include "ping.h"` /
`#include "utility.h"` landing on unrelated www.masque.com PHP scripts of
the same bare name.
ensure_named_node() tags the sourceless stub it creates for an
unresolved reference with origin_file, so _disambiguate_colliding_node_ids
can tell one file's unresolved reference apart from another's instead of
merging every file's same-named reference onto one shared bare id
(which can then collide with an unrelated same-named real definition
anywhere else in the corpus, since ids are case-normalized and global).
Five call sites duplicated that stub-creation logic inline instead of
calling ensure_named_node() -- Ruby's `Class.new(Super)` and
`class Foo < Base` inheritance, Python inheritance, Kotlin delegation
-specifier inheritance/conformance, and C++ base_class_clause
inheritance -- and none of them were updated when origin_file was added,
so all five still produce the un-disambiguated bare-id stub the fix was
meant to eliminate.
Four of the five sit directly inside _extract_generic, where
ensure_named_node() is already in scope as a closure, so they're
switched to call it directly. The fifth (Ruby's `Class.new(Super)`) is
handled by a separate helper, _ruby_extra_walk(), which doesn't have
that closure in scope; that one gets the same origin_file tag added
directly to its own inline stub dict, matching what ensure_named_node()
already does, without changing the helper's signature.
(A sixth occurrence of the same inline pattern exists in extract_apex(),
a fully separate regex-based extractor with no ensure_named_node()
equivalent of its own -- left out of this fix, which is scoped to the
shared _extract_generic path and its one directly-affiliated helper.)
Added a regression test: two different C++ files each inheriting from
the same undefined base class must produce two distinct stub nodes, not
one shared one. Fails on main (one shared 'base' id for both files),
passes with this fix.
The cross-file call resolver matches raw-call callees against a repo-wide
label index with no language check. In a repo that mixes a web app with a
native Android app, a TSX callback passed by name (register(refreshHeading))
resolved to a same-named Kotlin method and shipped as an INFERRED
indirect_call edge — a phantom the extraction spec itself forbids ('calls
edges MUST stay within one language'). Direct calls from non-JS/TS callers
had the same hole with no gate at all: a bare Python call bound to the lone
same-named Kotlin fun. Found on a production Next.js + Android codebase,
where the phantom edge also inflated the Kotlin node's betweenness enough
to surface it as a top suggested question in GRAPH_REPORT.md.
Resolution candidates are now filtered by language interop family before
the single-candidate/import-evidence logic runs. Families are grouped by
REAL interop so legitimate cross-language resolution keeps working:
Kotlin/Java/Scala/Groovy share the JVM, C/C++/Objective-C/CUDA share
headers (Swift bridges to ObjC), and JS/TS variants plus Vue/Svelte/Astro
SFCs compile into one module graph. Candidates whose family is unknown
(no source_file, non-code nodes) are never filtered, preserving the
previous permissive behavior, and callers with an unmapped extension skip
the guard entirely.
The per-chunk checkpoint feature added merge_existing without test coverage.
Add tests asserting merge_existing=True unions a file's slices across chunks and
the default still overwrites (the authoritative final write in extract).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
What changed
- Stabilize relative rebuild execution before graphify-out queue/lock setup.
- Recover from deleted transient hook CWDs when GRAPHIFY_REPO_ROOT points at the repository root.
- Fail cleanly when the current working directory is gone and no repo root fallback is available.
- Add regression coverage for both fallback and clean-failure paths.
Why
- Detached post-commit/post-checkout rebuilds can inherit a transient CWD that is deleted before the background process starts.
- _rebuild_code previously used relative graphify-out paths before Path.cwd()/watch_path resolution could be handled, causing FileNotFoundError: [Errno 2] No such file or directory.
Validation
- .venv/bin/pytest tests/test_watch.py -q: 54 passed, 2 skipped.
- .venv/bin/ruff check graphify/watch.py tests/test_watch.py: passed.
- .venv/bin/python -m py_compile graphify/watch.py tests/test_watch.py: passed.
- env -u PYTHONPATH -u PYTHONHOME PYTHONHASHSEED=0 .venv/bin/pytest -q: 2904 passed, 30 skipped.
Notes
- Full suite without PYTHONHASHSEED=0 exposed an unrelated ordering-sensitive labeling test; with the deterministic hash seed used by graphify hooks, the suite passes.
_max_graph_file_bytes() resolves the graph-load memory-bomb cap and parses the
GRAPHIFY_MAX_GRAPH_BYTES env var (plain bytes, MB/GB suffix, fallbacks), but had
no behavioral tests — only the default constant was asserted.
- Add tests for: unset/blank -> default, plain integer, MB/GB suffix (binary),
case-insensitive and space-tolerant suffixes, unparseable -> default, and
non-positive -> default.
- Fix the docstring, which described the suffix as "decimal multipliers of 1024"
(self-contradictory). MB/GB are treated as binary (MiB/GiB); the tests lock
that behavior in.
No behavior change.
`.m` is shared by Objective-C implementation files and MATLAB, but the extractor
dispatch routed every `.m` to extract_objc unconditionally. Feeding real MATLAB
to the Objective-C tree-sitter grammar yields root.has_error and garbage
nodes/edges — worse than skipping, because it pollutes the graph with wrong data.
_get_extractor now content-sniffs `.m` the same way `.h` already is: a genuine
Objective-C `.m` carries an ObjC directive (@implementation/@interface/@import/
#import) and still routes to extract_objc; a `.m` without one (MATLAB, Octave)
gets no extractor, so it is surfaced by the no-AST-extractor warning (#1689)
instead of mis-parsed. `.mm` is unambiguously Objective-C++ and is left untouched.
This stops the wrong-by-omission garbage; wiring a real tree-sitter-matlab
extractor (the issue's primary ask) remains a follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When classify_file() returned None — an extensionless, non-shebang file
(Dockerfile, Gemfile, Makefile, Rakefile, LICENSE, ...) or an unsupported
extension — the file left no trace at all: not counted, not listed, nothing.
A user had no way to tell from graphify's output that those files were even
considered.
detect() now collects these into an "unclassified" list in its result, and
`graphify extract` prints a one-line summary after the scan counts:
"N file(s) not classified (no supported extension or shebang), skipped:
Dockerfile, Makefile, ...". Real code/docs are unaffected. This is the
visibility half of the issue; wiring up extractors/manifest handling for
Dockerfile/Makefile-style files remains a separate feature.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
safishamsi