The #2475 fix recomputed the match path against the explicit scan root but
skipped the _nfc() normalization rel_anchor gets, so an NFD-named path
could miss a parent ignore pattern under the #2544 NFC-matching regime.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The #2600 title helper preferred a cwd-relative label, which still leaks
host/user path segments when the graph is built from a directory above the
project. Keep from the graphify-out / GRAPHIFY_OUT marker onward first
(portable in every case); fall back to cwd-relative only for a fully custom
output path with no marker.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The <title> previously embedded str(output_path), so Windows absolute
host paths leaked into a tracked artifact (regression of #433, #2598).
Prefer a cwd-relative label, else keep from the graphify-out segment
onward, else the filename only.
tree-sitter gives an arrow with one unparenthesised parameter a `parameter`
field (singular) and no `parameters` list node, so `_js_local_bound_names`
never saw it: `x => sink(x)` bound nothing, and `x` read as a by-name
reference to any same-named callable in the corpus, fabricating an
indirect_call edge (INFERRED, 0.8).
The parenthesised form was always handled, so `(x) => …` and `x => …`
behaved differently. Same singular/plural trap as `catch_clause.parameter`.
Strictly subtractive on real code: +0/-88 indirect_call edges over 5,402
files of node_modules, +0/-98 over a 6,000-file mixed corpus.
An ignore rule naming a directory with an accent silently does nothing on
macOS, and the files it was meant to exclude get scanned anyway.
macOS (APFS/HFS+) returns filenames in NFD — "ç" comes back as "c" + U+0327
COMBINING CEDILLA — while editors write ignore files in NFC, where the same
"ç" is the single codepoint U+00E7. The two render identically and compare
unequal, so `fnmatch` never matches and the pattern is a no-op.
Found in a real repo: a `.graphifyignore` containing `Orçamento/` failed to
exclude that directory, and 9 client contract PDFs were picked up for semantic
extraction — i.e. queued to be sent to an LLM — despite an explicit rule
against it. The failure is silent: there is no warning, and the only symptom
is a file count that does not match what you expect. A user who does not
count would never know. That is what makes this worth fixing rather than
documenting: the rule appears to work.
Both sides are now normalized to NFC before matching, at three boundaries:
the pattern (in `_parse_gitignore_line`, so it covers .graphifyignore,
.gitignore and $GIT_DIR/info/exclude alike) and the two path forms used in
`_is_ignored` (`target.name` and the anchor-relative path).
NFC is already the form Linux and Windows produce, so this is a no-op there
and only repairs the macOS mismatch.
Tests: two regression tests cover both directions (NFC pattern vs NFD path on
disk, and the reverse); both fail before this change and pass after. A third
asserts ASCII patterns are unaffected, so the normalization cannot regress
existing behavior.
Full suite: 3833 passed. The 13 failures in tests/test_terraform.py are
pre-existing on a clean upstream checkout (optional tree_sitter_hcl not
installed) and unrelated to this change.
`_find_node_tiers` builds two normalizations of the query: `term`, which
tokenizes on \w+ so punctuation becomes a space, and `norm_query`, which
keeps it. The exact tier compared the node id against `term` only, so
`term == nid_lower` was false for every id carrying punctuation, and
`norm_query` — which already held the right form, and is even one of the
two trigram needles — was never compared against the id at all. Comparing
`norm_query` to the folded id closes that half.
It does not reach ids carrying non-ASCII text. `_node_search_text`
indexed the id raw while every query path folds through
`_strip_diacritics`, which NFKD-decomposes. Hangul syllables decompose
into conjoining jamo, and jamo have combining class 0, so they survive
the combining-character filter: the needle's trigrams and the posting's
trigrams were disjoint, `_trigram_candidates` returned a candidate list
without the node, and it was dropped before any predicate ran. The
folded id is now part of the indexed text.
Both halves are additive. An id that resolved before resolves to the
same node; only ids that previously resolved to nothing can now resolve.
The folded field is appended, and only when the fold actually differs,
so field positions do not move and an all-ASCII graph indexes byte for
byte what it indexed before. Index build, median of 7 runs:
graph trigrams postings build
5k all-ASCII before 1723 238876 96ms
5k all-ASCII after 1723 238876 97ms
17k real before 33442 2311784 998ms
17k real after 33490 2312462 990ms
5k half-Hangul before 1735 256381 116ms
5k half-Hangul after 1739 266381 124ms
The real graph is the 17269-node one measured below; 354 of its ids are
non-ASCII, so the index grows 0.03% and the build stays inside run-to-run
noise. The half-Hangul row is a deliberate worst case — every other node
id Korean — and even there the cost is paid once per graph load, on a
graph where id lookup previously returned nothing at all.
On a real 17269-node graph with Korean source filenames, every node id
fed back to itself, full population:
id class total before after
contains punctuation 2326 0 2326
contains Hangul 354 0 354
ASCII, no punctuation 14589 14589 14589
And every query that graph can produce — all 17269 ids plus all 16537
distinct labels — through `_find_node_tiers` on both variants in one
process: 31126 identical, 2680 that returned nothing before and resolve
now, 0 with a changed first result, 0 lost, 0 with a widened exact tier.
Every difference is a query that previously returned nothing.
Left alone deliberately: `_score_query` compares the id raw in the same
way, so `path` and query seeding still cannot take a punctuated id, and
the prefix tier also matches ids against `term` only. Both are behaviour
changes beyond this defect rather than part of it.
One note for the regression tests: the non-ASCII case needs a graph of
at least ~10 nodes. `_trigram_candidates` bails out to a full scan when
`min(present) > int(n * 0.10)`, so on a small synthetic graph the index
path is never taken and the test passes with the defect still present.
The 0.9.37 #2551 warning fired on valid TypeScript/TSX: tree-sitter-typescript
sets root.has_error on tiny fully-recovered errors (a `&` in a JSX string
attribute; a semicolon-less `in_*` interface member) that still extract every
symbol. The warning now fires only when recovery plausibly cost symbols — the
file yielded <=1 node, or a materialized ERROR region spans more than one line —
so the genuine Kotlin one-line-body (#2551) and Luau (#2520) cases still warn
while valid TS goes silent. Warning also prints a root-relative path.
Thanks @Sid-AutoWisdom and @atlasplatformu-ai.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Prevent package-qualified calls and types from collapsing onto unrelated bare-name symbols. Preserve Go import evidence, resolve internal packages exactly, canonicalize external type stubs, and support incremental resolution context.
Investigation, implementation, and regression fixtures prepared with OpenAI Codex.
These land together because they are interleaved in extract.py/engine.py.
#2584 (PR #2588, thanks @phudayyy): the 0.9.38 dynamic-import dedupe keyed
only on target, so an in-function import() suppressed the file-level edge
affected follows. Dedupe now keys on the importing file, emitting one
file-level dynamic_import edge per file/target while keeping the call-site
edge.
#2586 / #2417 (PR #2586, thanks @EZZEASY): a Python member call on an
untyped receiver (x.get(...)) no longer binds by name to a same-named
module function. walk_calls now defers non-self/cls/super Python member
calls to the evidence-gated resolver; super().method() still resolves.
Known trade: same-file x = Thing(); x.method() loses its evidence-free
edge (precision over recall, per #2553).
#2589/#2591 resolver arm (in _resolve_objc_member_calls): the @protocol
exclusion and the self.field/_ivar receiver resolution (paired with the
objc.py extractor changes committed separately).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2589 (PR #2500): a @protocol declaration (labeled <Name>) is excluded from
the receiver-type index, so it no longer collides with a same-named class.
#2590 (PR #2501): a category/class-extension interface is keyed off the
base stem and folds into the base class instead of minting a duplicate
node.
#2591 (fresh): @property and ivar declarations are captured into a
per-class field-type table, and a message send to a self.field / _ivar
receiver resolves through it (bare field name only, so Foo.shared cannot
fabricate to a FooShared class). All hold the single-definition guard and
emit INFERRED.
Adapts PRs #2500/#2501 (thanks @xiongjianxu); #2591 fresh.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
graphify watch only rebuilt on a code-file event, so deleting only doc
files while watching flagged needs_update but evicted nothing until the
next code change. A batch containing any vanished path now triggers a full
reconcile rebuild (which evicts the deleted source); a surviving modified
doc still writes the needs-LLM flag. The general deleted-file leak was
already fixed in 0.9.10; this closes the live-watcher residual.
Thanks @angmeng.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fuzzy dedup compared same-file labels with prefix-weighted Jaro-Winkler, so
two distinct entities differing by one content word (asset contribution
flow vs asset consumption flow) cleared the threshold and one was lost. A
one-token difference is now judged on the differing tokens (any distinct
content word blocks; stopword/typo variants still merge), with a
same-length Damerau-Levenshtein typo escape. Genuine typo and
whitespace/case/punct variants still collapse. The #2532 collision path is
untouched.
Adapts PR #2587 (thanks @wilyan09007) with two hardening deltas.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
These changes are interleaved across engine.py/extract.py by function, so
they land as one batch.
#2568 (thanks @imagineers-tyler): the 0.9.37 #2552 callback-body fix
unioned sibling closures' local names under the shared declaration, so a
local in one callback suppressed a real indirect_call in a sibling. Locals
are now scoped per body (keyed by body id, via walk_calls' extra_locals).
Restore-only, never fabricates; #2552 capture preserved.
#2565 (thanks @kskchaitanya1993): Kotlin property initializers — class,
top-level, companion, and `by lazy {}` — now seed call extraction, so
`val repo = createRepo()` produces a calls edge; literal initializers
produce none; FQ calls compose with the #2550 resolver.
#2561 (thanks @fakewaffle): Swift receiver typing now handles
`@Environment(Store.self)` (whitelisted; @Query/keypath/dotted skipped to
avoid a wrong edge) and in-corpus factory bindings via a marked concrete
return type; opaque/array/out-of-corpus returns stay unresolved.
#2577 (thanks @wilyan09007, PR #2579): the SQL extractor no longer emits a
reads_from edge to a CTE name. WITH names are scoped per query (a subquery
CTE no longer suppresses an outer real table of the same name), so a CTE
no longer mints a bare stub that binds to an unrelated same-named symbol.
#2575 (thanks @phudayyy, PR #2574): a dynamic `import('…')` inside a nested
function or at module scope now produces an edge, dynamic_import is
included in affected, and calls inside nested named functions are
collected; a dynamic import already captured as a deferred imports_from is
not double-counted.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2543 (adopts PR #2546, thanks @michaelxer): a failed extraction is no
longer stamped in the incremental manifest as up-to-date, so graphify
update retries it instead of skipping it forever; a manifest already
poisoned by the old behavior is healed on the next run; genuinely
unchanged files are not re-processed. Extended to the watch save_manifest
paths too.
#2554 (adopts PR #2555, thanks @annieyii): the claude-cli backend now
inspects the stdout envelope for an is_error result (e.g. a rate limit
returned with exit code 0) and raises it on both the zero and non-zero
exit paths, instead of parsing it as an empty success and bisecting
against a live rate limit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TypeScript (#2553, #2552, thanks @Earthfreedom):
- _resolve_typescript_member_calls matched a receiver type by name alone
and emitted EXTRACTED, so a third-party import could bind to an unrelated
local class of the same name. It now requires the matched type to be
same-file or imported by the caller's file, and tiers table-inferred
receivers to INFERRED.
- calls inside a callback passed to another call (const h = wrapper(arrow))
were never walked; the callback body is now walked and attributed to the
declaration, through the same import-gated resolution so it cannot
fabricate edges. The #2553 gate lands with #2552 by design.
Kotlin (#2526, #2550, #2551; adapts PR #2531, thanks @Mustaqeem66;
reports from @spaceBrownie and @thomasrengot-hub):
- match the bundled tree-sitter-kotlin 1.1.0 import node and resolve each
import to the real target node (imports were silently dropped), so
genuine calls promote to EXTRACTED via import evidence;
- a fully-qualified call com.example.Foo.bar() now produces a calls edge;
- a file the grammar cannot fully parse (e.g. one-line class C { val x })
now warns instead of silently extracting nothing, and declarations
recovered inside an error span keep their enclosing class.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Windows skill variant had a PowerShell Step 1 but its later steps came
from the shared core fragment as bash-only shell (cat-piped interpreter
invocations, rm -f, find -delete). The skillgen renderer now translates the
composed core to PowerShell for powershell-shell platforms (here-string
interpreter invocations, Remove-Item cleanup); POSIX skills are unchanged
and step / #2490 parity is enforced by a generator check. Not a regression
from 0.9.35 (skill files were unchanged between 0.9.34 and 0.9.35).
Thanks @tannermosher2015-debug.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_collision_rank no longer falls through to raw lexical source_file order.
It now ranks by a lifecycle penalty (active over archived) computed on the
root-relative path, then a reversed-segment tie-break, so the winner of a
collision (e.g. plans/_done/x.md vs plans/in-progress/x.md) no longer
depends on ASCII filename order, absolute-vs-relative path form, or the
checkout directory name.
Active/archived idea from PR #2540 (thanks @michaelxer); the root-relative
and reversed-segment determinism is implemented here.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
swift_extensions[].nid is now remapped in the in-process id/sym remap
passes (not just the cache path), so a cross-file extension merges onto
its base type before call resolution; static Foo.bar() and singleton
Foo.shared.bar() calls resolve again. The merge candidate set is gated by
language family + type-def predicate so it never absorbs a same-named
type from another language.
Folds in PR #2539 (thanks @pawelo446).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
cluster-only warns when --backend/--model/--batch-size are ignored on the
label-reuse path; the community-label prompt key no longer collides with
the discard sentinel (an echoed key was silently dropped); tree --root
exits non-zero when it matches no source file instead of flattening the
tree; and cluster-only stamps built_at_commit from the analysed graph, not
the shell cwd. Also folds in the cluster-only refused-write guard from
PR #2522 (thanks @aniJani).
Thanks @elecnix for the report.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2508 (thanks @Tomaskobel): callflow loads the graph directed and recovers
direction from _src/_tgt markers (consistent with the path fix), so
caller/callee columns are correct; indirect calls are now counted.
#2507 (thanks @filipechagas): relational-intent verbs (calls/uses/extends/
...) are excluded from the per-term seed guarantee, so a decoy matching
only the verb no longer becomes a traversal root, while a verb that is a
real symbol name can still be seeded on merit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2495 (thanks @alisson-acioli): graphify update now evicts a file newly
added to .graphifyignore/--exclude even though it still exists on disk
(positive ignore-rule evidence), with .gitignore-driven eviction gated on
an explicit full update; unchanged in-corpus files and fail-closed #1795
cases are preserved.
#2504 (thanks @te7ina-honey): the Java type resolver runs before the
unique-label stub rewire and parks imported-but-external types on their
FQN, and cross-file import resolution checks the package, so a local
class and a same-named external annotation (e.g. Spring @Component) no
longer collapse into one node. In-corpus annotation resolution unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2497 (thanks @sortakool): the shrink guard read the post-replace node
count and was skipped under prune_sources, so a broken partial re-extract
could silently destroy nodes. It now diffs the on-disk baseline by node
identity and refuses any loss from a source neither re-extracted nor
pruned this run, and reports replaced-node counts.
#2446 (thanks @AI-invest): prune_sources given absolute paths under a
non-standard layout now derives the scan root by suffix-matching stored
source paths, reports the accurate matched count, and warns instead of
saying 'already clean' when nothing matched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2486 (thanks @adminwat): normalize dict-shaped hyperedge members to ids
(or drop with a warning) so a malformed hyperedge can't abort a completed
merge with a TypeError.
#2484 (thanks @sortakool; approach from @oleksii-tumanov's #1691):
merge-graphs relabels hyperedge member ids and ids with the repo prefix,
unions both inputs' hyperedges instead of clobbering, and writes both
persistence slots.
#2485 (thanks @sortakool): build_from_json reads hyperedges from the
top-level and nested slots; a full validation wipeout is reported loudly.
#2490 (thanks @PapiScholz): the skill Step-5 flow passes curated
community_labels to to_json, so graph.json ships community_name.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
graphify path and the MCP shortest_path tool now build a digraph from the
true _src/_tgt directions and respect edge direction by default, so a
returned path no longer traverses edges backwards. --undirected (CLI) /
undirected=true (MCP) opt out; no directed path is reported plainly
instead of silently returning a reversed one.
Thanks @luliaz0601.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Track C# receiver types per lexical declaration scope (byte ranges) and
resolve by the call's position, instead of a method-wide flat table that
poisoned a name on any None-typed binding. A typed static local-function
parameter now keeps resolving even when an out var reuses the name in the
enclosing body. Fixes a regression from #2346; #2299 cross-method
independence and field-conflict poisoning are unchanged.
Thanks @JensD-git for the bisect and repro.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2411 (thanks @JensD-git): key the C# partial-class merge on assembly
(nearest ancestor .csproj/.fsproj/.vbproj) in addition to namespace and
name, so same-name partial classes in different assemblies stay distinct
while genuine partial halves within one project still merge. Fixes a
0.9.32 regression from #2332.
#2437/#2438 (thanks @aryanbonigala, builds on PR #2439): incremental
rebuilds no longer drop member-call and indirect_call edges from a
changed file into an unchanged target. Re-resolution now sees the
unchanged corpus (nodes, contains/method edges, and _callable markers,
which now persist to graph.json like _origin); edges to a genuinely
removed target are still evicted.
#2444/#2445 (thanks @Baziar, builds on PRs #2461/#2458): a
BrokenProcessPool triggers the sequential fallback instead of being
swallowed per future, a failed worker file is retried sequentially
rather than merged as empty, and a whole-pass AST failure on a fresh
build exits non-zero instead of writing a zero-node graph
(--allow-partial opts into a best-effort partial graph).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Print a one-time note about free early access to app.graphify.com at the
end of the install summary. Not shown on any other command.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
module Foo::Bar and nested module Foo; module Bar are canonicalized to
one fully-qualified label, and include/extend/prepend keep the full
constant path, so include Foo::Bar resolves. Mixin resolution is scoped
and lexical: a qualified external name (extend ActiveSupport::Concern)
no longer binds to a local module named Concern, while a genuine
in-corpus include Foo::Concern still resolves. Nested-declared classes
keep a last-segment index so typed-receiver calls (Processor.new) keep
resolving.
Adapts the approach from #1394 (thanks @FolatheDuckofDuckingburg) to the
post-refactor extractors/engine.py, with an added last-segment guard for
nested-class receiver resolution.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- C# receivers declared inline via out-var / is / case / switch-arm patterns
are now typed into the per-method table, so their member calls resolve (#2346).
- partial class halves across files now merge to one class node (new
_merge_csharp_partial_class_nodes pass, mirroring the Swift-extension merge),
so cross-half member calls resolve instead of splitting the class (#2332).
- Kotlin anonymous-object (object : Foo {}) members now get nodes, contains/
implements edges, and their calls resolve (#2347).
All in-corpus only, never a wrong edge.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Node/edge ownership was keyed on source_file alone, but each file has an AST
tier and a semantic tier. A semantic-only re-extract deleted a doc's AST
headings, and a full _rebuild_code deleted document AST nodes. Add an
_is_ast_tier() predicate (with a source_location legacy fallback for the
unreliable _origin marker, #2334), backfill _origin on load so graphs
self-heal, make build_merge / merge_raw_extraction replace by (source_file,
tier) instead of source_file (coexist policy: an AST re-extract replaces only
AST nodes and keeps the semantic layer, and vice versa), and scope the
full-rebuild AST drop to sources actually regenerated (#2336) so a
semantic-backed doc keeps its AST layer. Deletion prune stays tier-blind.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The build_merge fix only covers the incremental path reached by
`graphify <path>`. `graphify update` rebuilds through _rebuild_code in
watch.py and never calls build_merge at all, so it still downgraded a
directed graph — which is the symptom named in the issue.
Two sites: the clustered path called build_from_json(result) with no
directed=, taking its directed=False default, and the --no-cluster path
built candidate_graph_data straight from the raw merged extraction, which
never carries a directed key, so the written file lost it entirely. Both
now inherit from existing_graph_data, defaulting to False when there is no
existing graph, matching build_merge's semantics.
_canonical_graph_for_compare normalizes a missing directed key to False so
an older graph.json without the key does not register as changed against a
candidate that now writes it explicitly.
build_merge took graph direction from its caller only, defaulting to
directed=False, and rebuilt with that default at build.py:1327 — discarding
the direction of the graph it had just loaded from graph_path. Since the
only call site (cli.py:3547) passes no directed=, every incremental
`graphify update` through the merge path silently returned an undirected
graph for a directed one.
_load_existing_graph now also returns the on-disk `directed` flag, parsed
from the JSON it already reads, and build_merge's parameter becomes
bool | None: None inherits that flag when a graph exists and falls back to
False when there is none, while an explicit True/False still wins.
deduplicate_entities() rebuilt its remap table by re-scanning the whole
unique_nodes list once per merged union-find component, costing
O(nodes x components) -- roughly 137M membership checks on a 50k-node
corpus with 2.5k merged components, about a third of dedup wall-clock.
Build an id -> (position, node) index once and slice each component out of
it instead: O(members log members) per component.
The index carries the enumeration position, not just the node, because
_pick_winner() selects via min(), which returns the first minimum -- so
ties (equal chunk-suffix status and equal id length) are resolved by list
order. Sorting the group by id instead changed 179 of 2358 survivors on a
tie-heavy corpus. Sorting by position reproduces unique_nodes order
exactly, leaving survivors and edges byte-identical.
Benchmark, median of 5 runs:
nodes before after speedup
5000 2.37s 2.29s 1.03x
20000 10.58s 9.70s 1.09x
50000 31.72s 21.92s 1.45x
Isolated remap loop at 50k nodes / 2.5k components: 6.838s -> 0.019s (370x).
Merge counts and output node counts are identical at every scale.
This does not address the pre-existing arrival-order dependence of the
component structure itself, which originates in the fuzzy-merge loop's
uf.find() short-circuit and is out of scope here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
rajashidattapy