The #2475 fix recomputed the match path against the explicit scan root but
skipped the _nfc() normalization rel_anchor gets, so an NFD-named path
could miss a parent ignore pattern under the #2544 NFC-matching regime.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The #2600 title helper preferred a cwd-relative label, which still leaks
host/user path segments when the graph is built from a directory above the
project. Keep from the graphify-out / GRAPHIFY_OUT marker onward first
(portable in every case); fall back to cwd-relative only for a fully custom
output path with no marker.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The <title> previously embedded str(output_path), so Windows absolute
host paths leaked into a tracked artifact (regression of #433, #2598).
Prefer a cwd-relative label, else keep from the graphify-out segment
onward, else the filename only.
tree-sitter gives an arrow with one unparenthesised parameter a `parameter`
field (singular) and no `parameters` list node, so `_js_local_bound_names`
never saw it: `x => sink(x)` bound nothing, and `x` read as a by-name
reference to any same-named callable in the corpus, fabricating an
indirect_call edge (INFERRED, 0.8).
The parenthesised form was always handled, so `(x) => …` and `x => …`
behaved differently. Same singular/plural trap as `catch_clause.parameter`.
Strictly subtractive on real code: +0/-88 indirect_call edges over 5,402
files of node_modules, +0/-98 over a 6,000-file mixed corpus.
An ignore rule naming a directory with an accent silently does nothing on
macOS, and the files it was meant to exclude get scanned anyway.
macOS (APFS/HFS+) returns filenames in NFD — "ç" comes back as "c" + U+0327
COMBINING CEDILLA — while editors write ignore files in NFC, where the same
"ç" is the single codepoint U+00E7. The two render identically and compare
unequal, so `fnmatch` never matches and the pattern is a no-op.
Found in a real repo: a `.graphifyignore` containing `Orçamento/` failed to
exclude that directory, and 9 client contract PDFs were picked up for semantic
extraction — i.e. queued to be sent to an LLM — despite an explicit rule
against it. The failure is silent: there is no warning, and the only symptom
is a file count that does not match what you expect. A user who does not
count would never know. That is what makes this worth fixing rather than
documenting: the rule appears to work.
Both sides are now normalized to NFC before matching, at three boundaries:
the pattern (in `_parse_gitignore_line`, so it covers .graphifyignore,
.gitignore and $GIT_DIR/info/exclude alike) and the two path forms used in
`_is_ignored` (`target.name` and the anchor-relative path).
NFC is already the form Linux and Windows produce, so this is a no-op there
and only repairs the macOS mismatch.
Tests: two regression tests cover both directions (NFC pattern vs NFD path on
disk, and the reverse); both fail before this change and pass after. A third
asserts ASCII patterns are unaffected, so the normalization cannot regress
existing behavior.
Full suite: 3833 passed. The 13 failures in tests/test_terraform.py are
pre-existing on a clean upstream checkout (optional tree_sitter_hcl not
installed) and unrelated to this change.
`_find_node_tiers` builds two normalizations of the query: `term`, which
tokenizes on \w+ so punctuation becomes a space, and `norm_query`, which
keeps it. The exact tier compared the node id against `term` only, so
`term == nid_lower` was false for every id carrying punctuation, and
`norm_query` — which already held the right form, and is even one of the
two trigram needles — was never compared against the id at all. Comparing
`norm_query` to the folded id closes that half.
It does not reach ids carrying non-ASCII text. `_node_search_text`
indexed the id raw while every query path folds through
`_strip_diacritics`, which NFKD-decomposes. Hangul syllables decompose
into conjoining jamo, and jamo have combining class 0, so they survive
the combining-character filter: the needle's trigrams and the posting's
trigrams were disjoint, `_trigram_candidates` returned a candidate list
without the node, and it was dropped before any predicate ran. The
folded id is now part of the indexed text.
Both halves are additive. An id that resolved before resolves to the
same node; only ids that previously resolved to nothing can now resolve.
The folded field is appended, and only when the fold actually differs,
so field positions do not move and an all-ASCII graph indexes byte for
byte what it indexed before. Index build, median of 7 runs:
graph trigrams postings build
5k all-ASCII before 1723 238876 96ms
5k all-ASCII after 1723 238876 97ms
17k real before 33442 2311784 998ms
17k real after 33490 2312462 990ms
5k half-Hangul before 1735 256381 116ms
5k half-Hangul after 1739 266381 124ms
The real graph is the 17269-node one measured below; 354 of its ids are
non-ASCII, so the index grows 0.03% and the build stays inside run-to-run
noise. The half-Hangul row is a deliberate worst case — every other node
id Korean — and even there the cost is paid once per graph load, on a
graph where id lookup previously returned nothing at all.
On a real 17269-node graph with Korean source filenames, every node id
fed back to itself, full population:
id class total before after
contains punctuation 2326 0 2326
contains Hangul 354 0 354
ASCII, no punctuation 14589 14589 14589
And every query that graph can produce — all 17269 ids plus all 16537
distinct labels — through `_find_node_tiers` on both variants in one
process: 31126 identical, 2680 that returned nothing before and resolve
now, 0 with a changed first result, 0 lost, 0 with a widened exact tier.
Every difference is a query that previously returned nothing.
Left alone deliberately: `_score_query` compares the id raw in the same
way, so `path` and query seeding still cannot take a punctuated id, and
the prefix tier also matches ids against `term` only. Both are behaviour
changes beyond this defect rather than part of it.
One note for the regression tests: the non-ASCII case needs a graph of
at least ~10 nodes. `_trigram_candidates` bails out to a full scan when
`min(present) > int(n * 0.10)`, so on a small synthetic graph the index
path is never taken and the test passes with the defect still present.
The 0.9.37 #2551 warning fired on valid TypeScript/TSX: tree-sitter-typescript
sets root.has_error on tiny fully-recovered errors (a `&` in a JSX string
attribute; a semicolon-less `in_*` interface member) that still extract every
symbol. The warning now fires only when recovery plausibly cost symbols — the
file yielded <=1 node, or a materialized ERROR region spans more than one line —
so the genuine Kotlin one-line-body (#2551) and Luau (#2520) cases still warn
while valid TS goes silent. Warning also prints a root-relative path.
Thanks @Sid-AutoWisdom and @atlasplatformu-ai.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Prevent package-qualified calls and types from collapsing onto unrelated bare-name symbols. Preserve Go import evidence, resolve internal packages exactly, canonicalize external type stubs, and support incremental resolution context.
Investigation, implementation, and regression fixtures prepared with OpenAI Codex.
These land together because they are interleaved in extract.py/engine.py.
#2584 (PR #2588, thanks @phudayyy): the 0.9.38 dynamic-import dedupe keyed
only on target, so an in-function import() suppressed the file-level edge
affected follows. Dedupe now keys on the importing file, emitting one
file-level dynamic_import edge per file/target while keeping the call-site
edge.
#2586 / #2417 (PR #2586, thanks @EZZEASY): a Python member call on an
untyped receiver (x.get(...)) no longer binds by name to a same-named
module function. walk_calls now defers non-self/cls/super Python member
calls to the evidence-gated resolver; super().method() still resolves.
Known trade: same-file x = Thing(); x.method() loses its evidence-free
edge (precision over recall, per #2553).
#2589/#2591 resolver arm (in _resolve_objc_member_calls): the @protocol
exclusion and the self.field/_ivar receiver resolution (paired with the
objc.py extractor changes committed separately).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2589 (PR #2500): a @protocol declaration (labeled <Name>) is excluded from
the receiver-type index, so it no longer collides with a same-named class.
#2590 (PR #2501): a category/class-extension interface is keyed off the
base stem and folds into the base class instead of minting a duplicate
node.
#2591 (fresh): @property and ivar declarations are captured into a
per-class field-type table, and a message send to a self.field / _ivar
receiver resolves through it (bare field name only, so Foo.shared cannot
fabricate to a FooShared class). All hold the single-definition guard and
emit INFERRED.
Adapts PRs #2500/#2501 (thanks @xiongjianxu); #2591 fresh.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
graphify watch only rebuilt on a code-file event, so deleting only doc
files while watching flagged needs_update but evicted nothing until the
next code change. A batch containing any vanished path now triggers a full
reconcile rebuild (which evicts the deleted source); a surviving modified
doc still writes the needs-LLM flag. The general deleted-file leak was
already fixed in 0.9.10; this closes the live-watcher residual.
Thanks @angmeng.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fuzzy dedup compared same-file labels with prefix-weighted Jaro-Winkler, so
two distinct entities differing by one content word (asset contribution
flow vs asset consumption flow) cleared the threshold and one was lost. A
one-token difference is now judged on the differing tokens (any distinct
content word blocks; stopword/typo variants still merge), with a
same-length Damerau-Levenshtein typo escape. Genuine typo and
whitespace/case/punct variants still collapse. The #2532 collision path is
untouched.
Adapts PR #2587 (thanks @wilyan09007) with two hardening deltas.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
These changes are interleaved across engine.py/extract.py by function, so
they land as one batch.
#2568 (thanks @imagineers-tyler): the 0.9.37 #2552 callback-body fix
unioned sibling closures' local names under the shared declaration, so a
local in one callback suppressed a real indirect_call in a sibling. Locals
are now scoped per body (keyed by body id, via walk_calls' extra_locals).
Restore-only, never fabricates; #2552 capture preserved.
#2565 (thanks @kskchaitanya1993): Kotlin property initializers — class,
top-level, companion, and `by lazy {}` — now seed call extraction, so
`val repo = createRepo()` produces a calls edge; literal initializers
produce none; FQ calls compose with the #2550 resolver.
#2561 (thanks @fakewaffle): Swift receiver typing now handles
`@Environment(Store.self)` (whitelisted; @Query/keypath/dotted skipped to
avoid a wrong edge) and in-corpus factory bindings via a marked concrete
return type; opaque/array/out-of-corpus returns stay unresolved.
#2577 (thanks @wilyan09007, PR #2579): the SQL extractor no longer emits a
reads_from edge to a CTE name. WITH names are scoped per query (a subquery
CTE no longer suppresses an outer real table of the same name), so a CTE
no longer mints a bare stub that binds to an unrelated same-named symbol.
#2575 (thanks @phudayyy, PR #2574): a dynamic `import('…')` inside a nested
function or at module scope now produces an edge, dynamic_import is
included in affected, and calls inside nested named functions are
collected; a dynamic import already captured as a deferred imports_from is
not double-counted.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2543 (adopts PR #2546, thanks @michaelxer): a failed extraction is no
longer stamped in the incremental manifest as up-to-date, so graphify
update retries it instead of skipping it forever; a manifest already
poisoned by the old behavior is healed on the next run; genuinely
unchanged files are not re-processed. Extended to the watch save_manifest
paths too.
#2554 (adopts PR #2555, thanks @annieyii): the claude-cli backend now
inspects the stdout envelope for an is_error result (e.g. a rate limit
returned with exit code 0) and raises it on both the zero and non-zero
exit paths, instead of parsing it as an empty success and bisecting
against a live rate limit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TypeScript (#2553, #2552, thanks @Earthfreedom):
- _resolve_typescript_member_calls matched a receiver type by name alone
and emitted EXTRACTED, so a third-party import could bind to an unrelated
local class of the same name. It now requires the matched type to be
same-file or imported by the caller's file, and tiers table-inferred
receivers to INFERRED.
- calls inside a callback passed to another call (const h = wrapper(arrow))
were never walked; the callback body is now walked and attributed to the
declaration, through the same import-gated resolution so it cannot
fabricate edges. The #2553 gate lands with #2552 by design.
Kotlin (#2526, #2550, #2551; adapts PR #2531, thanks @Mustaqeem66;
reports from @spaceBrownie and @thomasrengot-hub):
- match the bundled tree-sitter-kotlin 1.1.0 import node and resolve each
import to the real target node (imports were silently dropped), so
genuine calls promote to EXTRACTED via import evidence;
- a fully-qualified call com.example.Foo.bar() now produces a calls edge;
- a file the grammar cannot fully parse (e.g. one-line class C { val x })
now warns instead of silently extracting nothing, and declarations
recovered inside an error span keep their enclosing class.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
C0KERNEL