Two core Julia constructs were dropped by the extractor:
- macro name(...) ... end produced no node, even though macros are first-class,
heavily used definitions. The grammar gives a macro the same signature shape
as a function, so it now mints a definition (labelled @name) and its body is
walked for calls -- with the signature skipped so the macro does not call
itself.
- @enum Name a b c (and the @enum Name begin ... end block form) is a
macrocall_expression that was ignored entirely. The first identifier in the
argument list is the enum type; the rest, including members inside a begin/end
block, are contained by it. Other macro calls still fall through so
definitions nested in e.g. an @testset block are walked.
Added tests covering macro extraction with body-call attribution (no self-loop)
and both @enum forms with members contained by their type.
(cherry picked from commit 87edb5a278)
Polish on #3848: remove the unused mid-file tree_sitter imports (F401)
and add a negative test asserting a bare constructor param without
val/var is not emitted as a field property.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A derived type's contains section binds procedures as the type's methods
(procedure :: area => circle_area, or plain procedure :: scale), but the
derived_type_definition branch only minted the type node and ignored the
binding list entirely. The type ended up in the graph with no connection to
its own methods, and calls dispatched through the type were unrepresented.
Collect the bindings while walking the type and, after the whole tree is
walked (so the module's internal procedures already have nodes), add a method
edge from the type to each implementing procedure. A binding to a procedure
from another module resolves to a sourceless stub via ensure_named_node, the
same treatment parameter and return types already get, so the corpus rewire
can collapse it and no dangling edge is produced.
(cherry picked from commit 75f8fff533)
defprotocol and defimpl were not recognized as definition keywords, so the
walker fell through to generic recursion with no parent module. The protocol
and implementation nodes were never created, and their functions were attached
to the file node instead of their container (two orphaned size() nodes hanging
off the file rather than off the protocol and its List implementation).
Handle both like module-like containers: mint the protocol/impl node, contain
it in its enclosing scope, and walk the do-block so each def becomes a method
of the protocol or implementation. The implementation also gets an implements
edge to its protocol; a cross-file protocol target is filtered by the existing
dangling-edge guard rather than left hanging.
(cherry picked from commit 5f4e979693)
A module instantiation minted its target id with a bare, unscoped
_make_id(name), while a module definition uses _make_id(stem, name). The
two never matched, so instantiating a module defined in the same file:
module counter(...); endmodule
module top;
counter u1(...);
endmodule
split `counter` into a real definition node and a second, sourced phantom
that carried the `instantiates` edge. `top`'s dependency on the module you
actually defined was invisible, and because the phantom was sourced it also
baked this file's path into the id and blocked the corpus-level rewire that
would otherwise collapse it (the #1402 duplicate shape the other
extractors guard against).
Pre-scan module names so a same-file instantiation (Verilog allows
forward/any-order instantiation) resolves to the definition id. A module
defined in another file becomes a SOURCELESS stub instead, so the
corpus-level rewire collapses it onto the real definition rather than a
per-file duplicate — mirroring the Go extractor's ensure_named_node.
Adds regression coverage for both the local-definition link and the
cross-file sourceless stub.
(cherry picked from commit bca8bbac72)
Covers a same name PHP/JS function pair in one file: exactly one
surviving node, the JS call to the colliding name absent as a calls
edge entirely rather than misattached, and the unrelated JS symbol's
own contains edge from the shared file node still present.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
(cherry picked from commit f59ce06c8c)
Covers the issue's exact reproduction (two functions plus the call
edge between them), that line numbers survive the masking pass, that
a script tag with only a src attribute contributes nothing without
crashing, and that a plain PHP file with no script tag is unaffected.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
(cherry picked from commit 2d408de529)
Companion to #3243: extend definition_file portability to the paths build_from_json does not
cover — the watch/incremental relativize + cache-root rebase and the direct extract() return
normalization — via a shared _PORTABLE_PATH_KEYS list so source_file and definition_file stay
in lockstep. Together with #3243 this closes all four persistence paths (graph.json, AST
cache, watch, direct extract).
A qualified Kotlin supertype (`class Foo : com.example.Base()`) resolved to the first
identifier (`com`), collapsing distinct bases onto a bogus node. Take the last segment of the
user type (handling generics like `Container<String>` -> `Container`), so the edge targets
the real base; simple and delegated supertypes are unaffected.
A qualified Scala base (`extends pkg.Base with other.Trait`) parses as a
stable_type_identifier (or generic_type wrapping one), which the heritage handler did not
match, so the edge was dropped or targeted the full qualified string. Add a helper that
takes the tail type name across all three shapes, emitting inherits for the first parent and
mixes_in for the rest, with no package/generic-arg false edges.
Only class_declaration was treated as a class type, so PHP interfaces, enums, and traits
produced no node and no heritage edges. Add them as class-like containers (and walk the
enum body) so their extends/implements/use heritage is captured as inherits/implements/
mixes_in edges to the right targets.
Add a Robot Framework extractor using the official robot.api parser (pure-Python, optional
[robot] extra), emitting suite/test/keyword nodes plus keyword-call, resource-import, and
library edges. Keyword resolution is case/space/underscore-insensitive (RF's real semantics)
and handles BDD prefixes and ${CURDIR}/${EXECDIR} path resolution; degrades to an actionable
error when robotframework is absent.
A guarded function head (def foo(x) when is_integer(x), do: ...) nests the head call under a
binary_operator, so a def whose only clause carries a guard was not extracted. Unwrap the
guard to the head call (the head is always the first child, so the guard predicate is never
selected) and emit the real function name; multi-clause, multi-condition guards, and defp
are handled.
The Common Lisp extractor minted its id prefix from the bare path.stem instead of the
canonical _file_stem used by every other extractor, so two same-basename .lisp files in
different directories collided (one file's nodes vanished on merge) and nested .lisp files
tripped a spurious legacy-id nudge. Use _make_id(_file_stem(path)) to match the shared
convention; cross-file rewire is unaffected (it matches on label, not id).
normalize_id strips !/?/= from Ruby method names, so save and save! (or foo and foo=) minted
the same node id and the second was dropped as a seen_ids collision. Add a Ruby symbol
sanitizer (!->_bang, ?->_pred, =->_eq) applied to the id only (the label keeps the raw
spelling), and rekey method_index on the raw name so member-call resolution still matches.
Ids stay stable across incremental re-extraction.
When a C/C++/ObjC declaration (header prototype) and its definition (implementation body)
merge into one node, the survivor kept the declaration's site, so the node pointed at the
prototype instead of the body. Record the definition's file/location on the survivor and
surface it as a Defined in: line; the pick is deterministic regardless of file order, and a
declaration with no definition keeps its own site.
A PowerShell enum definition and its members were not extracted, so a [Color] type
reference resolved to a sourceless phantom stub instead of the real enum. Emit the enum as
a real sourced node with its members as contained children (mirroring class_statement), so
references resolve to it; no builtins are fabricated.
A defclass whose superclass is defined in another file lost its inherits edge. Mint a
sourceless stub for the cross-file base (source_file empty, origin_file set) like the
import-stub path, so the corpus rewire collapses it onto the real definition and the
inherits edge survives; a same-file superclass still binds locally.
An abstract type declared with a supertype (abstract type Dog <: Animal end) matched only
the bare-identifier form, so the subtyping declaration dropped the node entirely. Parse the
type_head shape shared with structs (factored into a helper) and emit the inherits edge; a
bare abstract type still works.
Zig lets enums and unions carry methods like structs, but only the struct branch recursed
into the container body, so enum/union methods and the calls in their bodies were dropped.
Walk enum/union children the same way, parenting methods to the container with the same id
scheme; variants and fields are not turned into nodes.
An Obsidian-style [[wikilink]] resolves to a note anywhere in the vault by basename, but
resolution only tried sibling files, so cross-folder links were silently lost. Add a
vault-wide fallback that fires only when relative/sibling resolution misses, with a
deterministic tiebreak on ambiguous basenames (shallowest path, then lexicographic
root-relative path) and a once-per-scan index so it stays O(N).
The no-dup test asserted on _edge_labels (a set), so count()==1 was trivially
true whether or not the dedup ran. Count raw edge occurrences (normalized
labels) so a regressed dedup — @Uses({X, X}) emitting 2 edges — actually fails
the test. Adds a guard that string/enum annotation arguments (@RequestMapping(
"/x"), @Retention(RetentionPolicy.RUNTIME)) do not fabricate type refs. Adds
the CHANGELOG entry.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Java annotation extraction read only the annotation's name field, dropping
class-literal arguments (@Repeatable(RubricsFor.class), @Uses({A.class,B.class}))
and annotation-member return types (RubricFor[] value()), so a container
annotation became a disconnected island. Emit references edges for both, with
qualified-identity preservation; string/enum annotation arguments are not
treated as type refs, and JDK annotations resolve to sourceless stubs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the two gaps the deep-dive flagged: a direct test of
_parse_frontmatter_fallback (the PyYAML-absent path, previously unexercised)
and an assertion that heading ids stay _make_id(stem, title) regardless of
frontmatter/node_kind (pins no id-churn). Also adds the CHANGELOG entry.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds an additive node_kind ("page"|"heading") attribute so a docs corpus can
distinguish the page node from its heading nodes, and parses leading YAML
frontmatter onto the page node via the bounded sanitize_metadata (values become
capped attributes, not graph nodes). Also fixes a leading YAML `#` comment in
frontmatter being extracted as an H1. Node ids/labels/file_type are unchanged,
so existing markdown graphs are not re-keyed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The extractor emitted `imports` edges to _cl_id(mod_name) with no node created,
so a :use/require edge dangled (verified: sample.lisp left 'cl' and 'alexandria'
as edgeless targets) and never resolved to an in-corpus defpackage. Mint a
sourceless stub for each import target (the established cross-file pattern):
edges now have real targets, the corpus rewire collapses a stub onto a unique
in-corpus defpackage of the same name (:use :mylib -> the real package), and an
external one (cl) persists as a clean leaf. origin_file is stripped before
persist. Adds a no-dangling-edges regression test. Also adds the CHANGELOG entry.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tree-sitter-commonlisp-backed extractor for packages, classes, functions,
methods, generics, macros, variable definers, and same-file calls. Handles the
grammar's dedicated defun node (defun/defmacro/defmethod/defgeneric via
defun_header) and the generic list_lit + leading-symbol forms (defvar/defclass/
defpackage), recurses into wrapper macros (eval-when/progn) and reader
conditionals, and maps CL operator chars (= < > ? ! + *) to readable id
suffixes. Wired into detect/extract dispatch, the extractor registry, a
[commonlisp] optional extra, and the README; ships a fixture and a 23-test suite
behind an importorskip guard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The #2751 recovery minted node ids via _make_id(stem, name); a punctuation-only
name (TEST_CASE("***")) normalizes to empty, collapsing onto the bare file-stem
id — colliding with the file namespace and swallowing every later such test
under one id via seen_ids (#1899). Detect the collapse and fall back to a stable
line-positional id so each stays distinct. Adds a regression test (also covers
SCENARIO) and the CHANGELOG entry.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
doctest/Catch2 register tests with a string-literal name (TEST_CASE("...")),
which tree-sitter-cpp parses as an ERROR node, so every test function was
silently dropped. Adds a line-anchored regex fallback that appends one code
node per top-level TEST_CASE/TEST_CASE_TEMPLATE/SCENARIO plus a contains edge
from the file node, mirroring the Groovy Spock fallback.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In _extract_generic's class branch the `contains` edge was hard-coded to source
from the file node, so a nested class/object/trait attached to the file instead
of its enclosing type across ~19 languages (only C# even flagged the node with
is_nested_type, and still emitted no edge to the parent). The edge now sources
from parent_class_nid when set, else the file node — keeping the containment
tree connected (file -> Outer -> Inner). A `!= class_nid` guard avoids a
self-loop when same-name nesting collides ids (class ids omit the enclosing
name). The C# is_nested_type flag is retained (load-bearing for cross-file
resolution). Methods were already parent-sourced and are unaffected.
_kotlin_collect_type_refs had no equivalent of _JAVA_BUILTIN_TYPES /
_PYTHON_ANNOTATION_NOISE / _GO_PREDECLARED_TYPES, so every Boolean/String/
Int/List/... type annotation in a .kt file became a real graph node/edge.
Unrelated files sharing only these language-level types were getting
merged into the same community by cluster-only.
Add _KOTLIN_BUILTIN_TYPES (kotlin.* scalars, collections, throwables -
deliberately not Android/JVM framework types like Context/View/Bundle,
matching how _JAVA_BUILTIN_TYPES doesn't filter framework types either)
and check it plus _JAVA_BUILTIN_TYPES (Kotlin freely references
java.util.Date/UUID/etc) at the three call sites that previously appended
unconditionally.
Deliberately excludes "Result" from the filter list even though
kotlin.Result exists - it collides with the very common user-defined
sealed-class name (confirmed against this repo's own sample.kt fixture,
which defines its own Result<T>).
Verified on a ~400-file Android/Kotlin project: 3693->3239 nodes (-12%),
6481->5221 edges (-19%), with a previously merged 5-file grab-bag
community cleanly splitting out an unrelated screen-dimension utility.
Kotlin enum entries weren't extracted: the walker never descended into the enum
body (`enum_class_body` wasn't in _KOTLIN_CONFIG.body_fallback_child_types, so
_find_body returned None). Add `enum_class_body` to the fallback body types and a
`_kotlin_extra_walk` (dispatched for tree_sitter_kotlin, mirroring the Java/Swift
handling) that emits each enum_entry as a node with a `case_of` edge to the enum.
Re-applied from PR #1738 (@ivanzhl) onto the post-#1737 module layout: the walk
engine now lives in graphify/extractors/engine.py while _KOTLIN_CONFIG stays in
extract.py. Closes the Kotlin half of #1700 (Java was #1719).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a _JAVA_BUILTIN_TYPES skip list so ubiquitous java.lang/util/io/time/etc.
type names (String, List, Map, Optional, ...) are not emitted as references
edges (they never resolve to a project node). Mirrors _GO_PREDECLARED_TYPES /
_PYTHON_ANNOTATION_NOISE. Nested user-type generic args still resolve.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
class Foo : Bar by baz produced no edge because the delegation_specifier loop only handled constructor_invocation and bare user_type children; the by form wraps user_type in an explicit_delegation node. Add that branch so the implements edge (and generic-arg recovery) fires.
interface X extends A, B captured the parent list in iface_re group 2 but the handler only read group 1, so no inheritance edge was emitted. Split the parent list and emit one extends edge per parent (mirroring the class branch).
The Swift `enum_entry` handler in `_swift_extra_walk` iterated the entry's
children only for the `simple_identifier` case name (emitting a `case_of`
edge) and never descended into the sibling `enum_type_parameters` node, where
associated-value types live (`enum_type_parameters -> user_type ->
type_identifier`). As a result `case started(Session)` silently dropped the
`Event -> Session` type reference.
Descend into each `enum_type_parameters` child after emitting `case_of`, run
`_swift_collect_type_refs` over its named children, and emit a `references`
edge from the enum node to each collected type (context `type`, or
`generic_arg` for generic roles), guarding target != enum node. Mirrors the
existing Swift property/parameter/return-type emit style.
Fixture: add `case failed(Config)` to `NetworkError` in sample.swift.
Test: assert (`NetworkError`, `Config`) in references(context=type).
The C++ base_class_clause handler's `template_type` branch read the base
name (`sub.child_by_field_name("name")`) and emitted the `inherits` edge,
but never descended into the base's `template_argument_list`. As a result
`class Car : public Base<Dep>` emitted `Car -> Base` (inherits) yet dropped
the `Car -> Dep` generic_arg reference entirely.
The Java handler `_emit_java_parent_type` already emits these generic_arg
references for base-class type arguments; C++ was the asymmetric gap.
Fix: after emitting the `inherits` edge, grab the base's `arguments` field
(the `template_argument_list`) and run `_cpp_collect_type_refs` over each
named argument with the generic flag set, emitting a `references` edge
(context "generic_arg") per collected type, guarding target != class node.
`_cpp_collect_type_refs` already handles nested/qualified args, so
`Base<std::vector<Dep>>` is covered too.
Adds a templated base (`Connection<T>`) + derived class
(`PooledClient : public Connection<HttpClient>`) to tests/fixtures/sample.cpp
and a test mirroring the Java generic-parents test.
The C# class-body walker only handled field_declaration, so a
property's type produced no references(field) edge. In idiomatic C#,
auto-properties (`public Widget Main { get; set; }`) — not bare fields
— are the standard way to declare state, so this silently dropped most
of a class's type relationships.
Add a property_declaration branch alongside the field_declaration
handler, guarded the same way (ts_module == tree_sitter_c_sharp,
parent_class_nid set). A property exposes its type on the node directly
(no variable_declaration wrapper), so read it via
child_by_field_name("type") and collect refs with
_csharp_collect_type_refs, mirroring the Java/PHP/Kotlin siblings so
List<Widget> yields both the List field ref and the Widget generic_arg
ref. Only emit when target != parent_class_nid.
PHP 8 constructor property promotion (`__construct(private Repo $repo)`)
parses the promoted parameter as `property_promotion_parameter`, not
`simple_parameter`. The PHP parameter loop filtered on `simple_parameter`
only, so promoted params were skipped entirely: their type emitted no
`parameter_type` edge on the constructor, and — because a promoted param
is also a real class field — no `field` edge on the class either. A
non-promoted param in the same signature still emitted `parameter_type`,
so the type reference was silently dropped for exactly the promoted case.
The promoted param's type sits in the same direct named-child shape the
loop already reads for `simple_parameter`, so widening the filter to
accept `property_promotion_parameter` makes the existing type extraction
emit the `parameter_type` edge. Additionally, for a promoted param, emit
a `field`-context references edge on the class (mirroring the
`property_declaration` handler), guarded so it only fires when a parent
class is in scope and the target is not the class node itself. Normal
`simple_parameter` behaviour is unchanged.
Adds a promoted-property constructor to tests/fixtures/sample.php and
test_php_constructor_property_promotion_contexts asserting the promoted
type appears as both `field` and `parameter_type`, and that a
non-promoted param does not leak a field edge.
`@protocol Derived <Base>` dropped the protocol-adoption (inheritance)
edge. The protocol_declaration handler in extract_objc walked children
for method declarations but ignored the protocol_reference_list child
that holds the adopted protocols, so no implements edge was ever emitted
for protocol-on-protocol adoption.
The extractor already handled `@interface Foo <Proto>` adoption, but that
nests the protocol name under a parameterized_arguments node; protocol-on-
protocol adoption uses a different grammar node (protocol_reference_list)
whose adopted-name is a direct `identifier` child, so it was never
matched. Walk protocol_reference_list and emit an implements edge for each
adopted protocol, mirroring the @interface handling.
Adds a defined Base/Derived protocol pair to the ObjC fixture and a
regression test asserting the Derived->Base implements edge.
The `class_statement` handler read only the first `simple_name` child —
the class name — and never inspected the base type(s) after the `:`
token. As a result `class Dog : Animal` dropped the Dog->Animal
inheritance edge entirely; derived classes appeared as isolated nodes.
Walk the class_statement children, and once the `:` token is seen treat
each following `simple_name` as a base type. Matching the C# convention
(PowerShell has no syntactic base-vs-interface split), the first base is
emitted as `inherits` and the rest as `implements`, resolved via
ensure_named_node.
Adds a Shape/Circle inheritance pair to tests/fixtures/sample.ps1 and a
regression test asserting ("Circle","Shape") in the inherits edges.
The Scala field handler matched only `val_definition`, so a mutable field
(`var b: Repo`), which parses as `var_definition`, had its type reference
silently dropped from the graph. val and var nodes are structurally identical
(both expose a `type` field), so the existing type-collection logic works
unchanged. Widen the guard to accept var_definition.
Adds a var field to the Scala fixture and a regression test.
The SystemVerilog class-body field regex in _augment_systemverilog_semantics
matched only unqualified `<type> <name>;` declarations. Its `^\s*` prefix
consumes leading whitespace but not leading class-property qualifiers, so a
qualified field such as `rand Config m_cfg;` (three tokens) failed the
two-token shape and its type reference was silently dropped from the graph.
Consume optional leading qualifiers (rand/randc/local/protected/static/const/
automatic/var) before the type token. Zero qualifiers preserves the existing
behavior; the type and name capture are unchanged.
Adds test_systemverilog_qualified_field_references plus rand- and
protected-qualified fields (and a Config class) to the shared .sv fixture.
Only bare-identifier imports (`using Foo`) emitted edges. tree-sitter-julia
wraps qualified paths in `scoped_identifier` (`using Base.Threads`), relative
paths in `import_path` (`using ..Sibling`), and the package of a
`selected_import` may itself be a `scoped_identifier`
(`import Base.Threads: nthreads`). None of those were matched, so qualified and
relative imports were silently dropped, and scoped selected-imports pointed at
the selected symbol instead of the module.
Resolve the module name from identifier / scoped_identifier / import_path in
all three positions. Adds fixture lines + a regression test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Function calls (`y = f(x)`) were silently dropped — only `subroutine_call`
(`call sub(...)`) was handled in walk_calls. tree-sitter-fortran represents a
function invocation as a `call_expression`, which had no branch, so every
function-to-function call produced no edge.
Handle `call_expression`. Because Fortran uses the same `name(...)` syntax for
array indexing, the callee is resolved against procedures defined in the file
(`target_nid in seen_ids`) before emitting — so array accesses like `arr(i)`
cannot fabricate spurious `calls` edges. Adds a function + caller to the
fixture and a regression test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
rajatnagda45