An ambiguous extension (`.inc`, `.h`, TS-family, ...) is hard-mapped to one
global default; when that is wrong the file does not error — it parses as the
wrong language and yields a few incidental nodes, so the graph looks populated
while the real symbols are missing. A project can now declare what an extension
means via `language.<ext>=<language|.ext>` lines in `.graphifyrc` (which already
holds `viz_node_limit`). The declaration flows through file classification,
extractor selection, grammar/resolution, imports, watch, and the AST cache
(salt + manifest language field + incremental invalidation), is thread-local
per scan root, and is forwarded to the worker pool. Default detection is
byte-for-byte unchanged when no declaration is present.
Landed from PR #3789 (squashed), which carried @abhay-codes07's original #3075
forward with reviewed follow-ups. Authorship credited to both contributors.
Co-Authored-By: Andre Brait <andrebrait@gmail.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An unqualified call fell through to a file-wide last-writer-wins label map, so
it bound to the last-declared same-named method. Walk the caller's class chain
(own class + in-file bases nearest-first, then enclosing classes) and bind a
single hit; fail-closed on a tie; free functions/constructors still resolve.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Refuse a read-only destination in _atomic_replace on Windows (nt-only guard;
os.replace there raises PermissionError and the fallback would silently clobber
it), plus test-only guards: utf-8 encoding kwargs, posix-shape path assertions,
FIFO/AF_UNIX/symlink skips, and a replace-retry for AV/indexer locks. No POSIX
behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
_spawn_cannot_reimport_main (#3669) skips the process pool when spawn
workers cannot re-import __main__, judged by os.path.isfile on
__main__.__file__. The pip/uv console script on Windows is a launcher
exe with a zipped __main__.py, so __file__ is
...\Scripts\graphify.exe\__main__.py, not a file on disk, and every
`graphify update` / `graphify extract` from the normal Windows command
extracted on one core.
The pool works there: __main__ has a module spec, so
multiprocessing.spawn passes the module NAME to the worker, and a spec
named __main__ is skipped in the worker altogether ("zip archives ...
we don't even try"). Return False whenever __main__ has a spec,
mirroring get_preparation_data. stdin, -c and the REPL have no spec, so
#3669 is unchanged for them. The two stdin/REPL tests now also set
__spec__ = None, as a real stdin caller has.
Cold build of graphify's own repo with graphify.exe, alternated:
39.6 s -> 26.5 s median (12 workers), identical graph.json.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit f64594d7e5)
A CommonJS require() of a relative module with no file behind it resolved
to the attempted absolute path, and the target id plus every destructured
symbol under it were minted from that path. The checkout location and the
OS username ended up in node ids, so the same repo gave a different
graph.json in every clone.
#2457 fixed this for static imports only; require() goes through
_require_imports_js. Apply the same rule there: a resolved path that is
not a file is unresolved and gets the stable ref id.
The two destructured/member require tests asserted symbol edges into
cjs_require.js's missing ./foundation and ./helpers, i.e. those
absolute-path ids. They now run the fixture next to real modules, so
they still check one imports edge per binder.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8gwEpjajiWHVW3c9smuKf
(cherry picked from commit 5759ab1331)
A 0-byte file has no line 1, so the file node's source_location=L1 was an
anchor that can never resolve. Leave it None (how other location-less file
nodes already look) in the shared tree-sitter path, and keep the node: an
empty __init__.py is still a module other files import.
Closes#4041
Co-Authored-By: Grok Bot <noreply@x.ai>
(cherry picked from commit 29597adec8)
Use existing plain-import provenance to distinguish separate and same-line from-imports. Reject type aliases and dynamic namespace references, and protect changed and unchanged external callers through incremental CLI extraction.
Fixes#3793
AI-Assisted-By: OpenAI Codex
(cherry picked from commit 5e240c8bd3)
The unique-stub rewire copied a same-label class onto every edge, so a
Python base could inherit a TypeScript class. Refuse that write when the
language families differ. A same-language match still rewires.
Fixes#2207
(cherry picked from commit 48cdc8f484)
Follow-up on #4007: is_interpreter_variable fired for any bare-$VAR
command word, so `"$DIR/run.sh" stage.py` could mint a spurious invokes
edge to the argument on top of the real .sh calls edge. A bare
interpreter variable ($PYTHON) has no path separator; gate on
`"/" not in cmd_word`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Covers python3/python plus a bare interpreter variable, a non shell
interpreter running a dot sh target landing on its entrypoint node,
and two negative cases: a non interpreter command with a script
looking argument, and a missing target. Confirmed to fail against
pre fix code except for the two negative cases, which hold either way.
(cherry picked from commit 97d4c9207c)
Two small follow-ups on the 0.9.75 batch:
- merge-graphs --previous now reports a clear error and exits 1 on a
malformed/unreadable JSON file instead of crashing with an uncaught
traceback (follow-up to #4006).
- JS/TS super.m() fails closed (no edge) instead of falling back to the
file-wide name map, which bound it to the overriding method as a wrong
self-loop. The extends chain is unknown at this pass, so super can
never be resolved here; inherited this.m() still keeps the plain
lookup (follow-up to #4012).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
self.save, a paren-less save and save(x) with no receiver all send to
the current object, but they went through the file-wide name map,
which keeps only the last definition of each name, so they landed on
whichever class the file declares last, as EXTRACTED. In sinatra's
base.rb, Helpers#not_found was wired to Base#error and
AcceptEntry#to_s to MimeTypeEntry#to_str.
Route them through _self_call_target with the in-file inherits walk.
A top-level def and a mixed-in module method are not methods of an
unrelated class, so implicit-self sends to them resolve as before.
(cherry picked from commit 0175b64783)
self.save(), a bare save() (implicit self) and super.ping() in a Swift
method went through the file-wide name map, which keeps only the last
definition of each name, so they landed on whichever class the file
declares last, as EXTRACTED. In Alamofire, DataRequest.init calling
super.init() was recorded as DataRequest.init calling itself, and
URLEncodedFormEncoder's keyed container calls went to the unkeyed one.
Route them through _self_call_target with the in-file inherits walk,
which Swift supports because its inherits edges exist at this point.
Free functions, constructors and calls from an extension of the same
type resolve exactly as before.
(cherry picked from commit a8333ea8a1)
this.save() in one JS/TS class was resolved through the file-wide name
map, which keeps only the last definition of each name, so a class that
calls this.save() was wired to the save() of whichever class the file
declared last, as EXTRACTED. In zod's v3 types.ts every ZodString,
ZodNumber and ZodBigInt check method landed on ZodDate._addCheck.
Route this.m() and super.m() through the self-call lookup the Python
fix added, renamed _self_call_target. It binds to the caller's own
class when that class defines m(). JS/TS extends edges are only emitted
by the later symbol pass, so the base chain is unknown at this point;
anything the own class does not define keeps the plain lookup, which
leaves inherited this.m() edges exactly as they were.
(cherry picked from commit 33184e7b06)
self.m(), cls.m() and super().m() are exempt from the receiver-typed
defer because the receiver is the caller's own instance, but the lookup
they fell through to was the file-wide name map, which keeps only the
last definition of each name. Two classes in one file that both define
m() sent every self-call to whichever was declared last, as EXTRACTED.
On httpx that wires the sync Client's request/send chain into
AsyncClient, and every super().__init__() in _exceptions.py into
RequestNotRead.__init__.
Resolve these against the enclosing class, then its in-file bases
nearest first (super skips the class itself), the same order the ObjC
resolver uses for self. A tie between two bases on one level binds
nothing, and a method of a class outside the chain is never the target.
A module-level callable stored on the instance still binds as before.
(cherry picked from commit 02e1ed4e10)
In _resolve_python_member_calls, candidate imported modules are gathered from imported_by_filenode by checking in contains_children. Following #3410, functions enclosing a nested def also have contains edges to their nested functions. When a caller imports both a module and a function from that module (where the function contains a nested def), both the module and the function matched the candidate filter because the function's _module_stem_key evaluates to the stem of its source file. The resulting len(mods) == 2 triggered the ambiguity guard and dropped all module.func() calls from that file.
Exclude contained symbols by verifying not in file_of_node, which is already populated for all containment targets and excludes top-level file/module nodes.
(cherry picked from commit 542e4869ba)
Under the spawn start method (the Windows default) each worker re-imports the
parent's __main__ by its __file__ path. A stdin (`… | python -`), `python -c`,
or REPL caller has no such file — the path is missing or `<stdin>` — so every
worker dies during bootstrap and the pool raises BrokenProcessPool before any
work is done. The shipped SKILL.md pipes a heredoc into the interpreter, so on
Windows this is the path most users take, not an edge case: every skill-driven
run prints one full worker traceback per core, then a fallback warning that
blames a missing `if __name__ == "__main__":` guard — a fix the caller cannot
apply, because there is no file to put a guard in.
Detect the unusable-__main__ case before creating the pool: when the start
method is spawn (or the platform is Windows) and __main__ has no importable
file, run sequentially and print one note that names the real cause (stdin/-c/
REPL) and the remedy (write the step to a .py file, or pass parallel=False). A
script that DOES have a __main__ file but omits the guard is a different failure
this cannot catch here; it still surfaces via the existing BrokenProcessPool
fallback, whose __main__-guard message is now accurate for that remaining case.
Tests cover the detector (stdin, REPL-without-__file__, a real script, and the
fork start method) and that extract() with >= _PARALLEL_THRESHOLD uncached files
and an unusable __main__ never attempts the pool yet still produces correct
output. All five fail without the guard.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JJbLfztSxm2tH5cJwBbe9q
(cherry picked from commit b475196ed1)
_extract_parallel falls back to sequential on BrokenProcessPool, but an
OSError raised while constructing the ProcessPoolExecutor escaped and
killed the whole extraction (and `graphify update`).
Seen on macOS: ProcessPoolExecutor -> multiprocessing SemLock -> sem_open
fails with OSError(ENOSPC), "No space left on device", once leaked POSIX
named semaphores exhaust kern.posix.sem.max — with the disk nowhere near
full. Sequential extraction needs no semaphore, so hand back False the
same way the BrokenProcessPool branch does, with a warning that names the
OS error rather than the Windows __main__ hint.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit c154c01d50)
`script_invocation` edges (#1756) only ever resolved a path that is a
bare literal -- `./helpers.sh`, or `bash ./helpers.sh`. Both paths into
that branch go through `literal()`, which rejects by design any token
containing `$`, so the single most common way a shell script calls a
sibling is invisible:
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
"$script_dir/helpers.sh" --flag
`cmd` is None for that command node, the `elif cmd and ...` guard short-
circuits, and no edge is emitted. On a 206-file shell tree that uses the
`script_dir` idiom throughout, `script_invocation` produced *zero* edges
-- the feature was effectively dead for that whole style of codebase.
The `source` branch already solved the same problem for
`source "$DIR/lib/x.sh"` (#2079): strip the leading `${VAR}/` segments
with `_bash_source_suffix` and resolve the literal remainder against the
sourcing file's own directory, because the canonical `script_dir` idiom
makes that variable equal to exactly that. This applies the identical
treatment to the exec position -- read the command-name node's raw text
rather than its `literal()`, strip the same way, and fall through to the
existing `resolved.is_file()` gate.
Nothing new is fabricated. `_bash_source_suffix` still returns None when
the remainder holds another expansion (`"$dir/$name.sh"`) or a `..`
segment, and the on-disk check still means a wrong script-dir guess
emits nothing at all. The three existing negative tests -- missing,
shadowed, and dynamic invocations -- pass unchanged.
Measured on the same 206-file tree: 0 -> 16 edges across 10 distinct
file pairs, every one hand-checked to a real invocation of a real file,
no false positives. Full suite: no change in outcomes (the 25 failures
here are pre-existing and unrelated -- terraform, ollama, skillgen).
Fixes#3416.
(cherry picked from commit 2e112c39fc)
`from . import brain, ledger` in a directory with no __init__.py — a
namespace package, which `python -m pkg.mod` runs without complaint —
emitted nothing: _resolve_python_module_path returns None when there is
no module file to probe, and the #1146 submodule branch sits behind that
None. So every `brain.think()` / `ledger.write()` call in such a repo was
invisible to the #1883 module arm, and the most-called functions carried
in-degree 0 in the graph.
_resolve_python_namespace_dir mirrors the module-path walk (relative
base, scan root, then sys.path-root ancestors per #2072) and returns only
a directory that exists inside the root and has no __init__.py. The fact
collector uses it when the module path resolves to no file; the existing
submodule probe then emits the same imports_from edges a regular package
gets. A name that is not a module file on disk still emits nothing, and
a namespace package binds no symbols of its own.
Measured on a 275-node Python repo laid out this way, same graphify
version with and without the change: calls 201 -> 307, nodes unchanged;
think() 0 -> 15 callers, remember() 0 -> 10, ledger write() 0 -> 20.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KTNEBgonu6ASph6w8R2Wcp
(cherry picked from commit e2ec796bfd)
_LANGUAGE_BUILTIN_GLOBALS is one union across every language, right
for a BARE call (String(x) really would become a god node) but wrong
for a MEMBER call. open is a Python builtin and Set a JavaScript one,
so session.open() in Swift or _server.Set() in C# named after another
language's builtin was silently discarded outright: no same file
edge, but also no raw_calls entry, so cross file resolution never got
a chance to try it.
A member call carries a receiver, so it is not the ambiguous case the
union guards against. Lets a builtin named member call reach the
existing tgt_nid = None deferral every receiver typed resolver
already relies on for other cases (a capitalized Python receiver, any
Java member call, and so on), so it can only ever reach an edge
through a guarded, receiver typed resolver, never the unguarded
bare name path a real god node would need.
Verified against real cross file setups for Python (class qualified
call) and Swift (field typed receiver, reproducing the reporter's own
AppleReadyStreamFinder.find -> RemuxPlayback.open example), plus that
a same file member call still cannot bind to an unrelated bare
function sharing the builtin name.
Scoped to the shared engine only. go.py and rust.py carry the same
outer filter, and the reported issue asks for the same change there
too, but tracing where their raw_calls entries actually go shows the
shared cross file resolver skips every is_member_call entry
unconditionally (extract.py, the "obj.log() has no import evidence"
guard) regardless of builtin status, and neither language has a
dedicated receiver typed resolver the way Python, Java, C#, Swift,
TypeScript, C++, and Objective-C do. Making the same change there
would be safe but currently inert, so left out rather than shipping a
change with no verifiable effect; worth revisiting once either
language gets its own receiver typed cross file resolver.
Fixes#3381.
(cherry picked from commit c0f9ff57e0)
A Python type reference to a name imported from another module dangled or mis-resolved.
Run Python cross-file import resolution before the generic stub rewire and repoint sourceless
AST reference/inherits/implements edges onto the exact imported definition using the
file-scoped (module, symbol) -> node_id binding (not a bare-name match), so two same-named
types in different modules bind to their own; orphaned stubs are pruned only when no edge
still touches them.
This reverts #3155 (b568488): the source_file backfill also fired on a shared external stub
(e.g. a cross-file `Path` reference with no source_file), giving the stub a file identity so
the orphan prune swept it when that one file was deleted even though another file still
referenced it — regressing test_prune_sweeps_orphans. The fix needs to distinguish a
legacy node that genuinely lost its provenance from an external stub that must stay
sourceless; sending #3155 back as needs-work.
A sourceless legacy node now backfills its source_file from incident-edge provenance (only
when real edge evidence exists, deterministic min, never fabricated), and a declined data
JSON now emits a single file-identity node carrying source_file instead of nothing, so
freshness/affected checks can see the file. No node is resurrected and config JSON does not
double-add its file node.
A CommonJS export wrapped in a higher-order function (exports.x = wrap(fn),
module.exports.y = onCall({...}, handler)) was not captured, so the exported symbol was
missing. Reach through the wrapping call to the topmost function closure argument (mirroring
the const-export HOF path #2552/#2568); the wrapper stays a call edge, not the export's
identity, and a non-function arg fabricates nothing.
A command substitution on the right of an assignment (x=$(foo)) was not recorded as a
call, unlike a bare $(foo). Gate call emission on the ancestor chain: allow a call inside a
command_substitution when it sits under a variable_assignment, while still suppressing bare
and arg-position substitutions and process substitutions.
A class extending a built-in base (PHP `extends \Exception`) emits a sourceless supertype
stub, and the corpus rewire bound it to the unique same-labelled real definition anywhere
in the corpus — so in a PHP+TS monorepo a PHP class inherited from the TypeScript
Exception, a phantom cross-language edge / god node. Gate the supertype-target rewire with
a per-language curated builtin-base set: refuse the rewire only when the referring file's
language names the base as a builtin and the resolved target is in a different language
family. Same-language inheritance and user classes named like a builtin still link.
A data-shaped JSON that extract_json intentionally declines (returning a `skipped`
marker, not an `error`) was counted as a failed extraction, so it was kept out of the
incremental manifest and re-processed on every run. Treat the `skipped` decline as a
valid empty outcome, disjoint from genuine failures (which still return `error` and are
still reported).
The factory-object owner path emitted the `contains` edge inside the per-member
loop; add_node dedups on id but add_edge does not, so N methods assigned to one
object flooded the graph with N identical contains edges. Emit it once per owner.
Tests: assert a single contains edge across four methods, cover arrow-function
assignment (the dominant modern factory shape), and lock the negative case that a
non-object-literal receiver (`external.handler = fn`) is not captured.
Preserve callable members assigned to a local object-literal factory API
(`const api = {}; api.foo = fn`), modeling the API object beneath its factory
and attaching the assigned functions as methods. The lazy owner-node minting
(only for identifiers proven to be object-literal bindings in the enclosing
function) avoids the #1077 config-object flood.
Partially addresses #2524 (the object-literal-method shorthand form
`return { foo() {} }` remains out of scope).
Adds the two invariants the deep-dive flagged: require(variable) must not
fabricate an edge, and a module-scope require still yields exactly one
imports_from edge (guards the module-vs-body pass partition against future
double-counting). Also adds the CHANGELOG entry.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A require(...) written inside a function body (the canonical lazy-require idiom
for breaking circular deps) produced no edge at all, while the identical require
at module scope resolved as EXTRACTED imports_from/imports edges. walk_calls now
feeds nested require declarations to the same _require_imports_js routine,
attributing the edge to the enclosing callable; dynamic require(var) is still
skipped and no bare local node is minted (#1077 scope guard).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2705 fixed `function` nested in a `function`, but the React idiom that
motivated the issue -- a function declared inside an arrow-defined component
(`const Panel = () => { function handleClick(){} }`) or an arrow callback
(`useEffect(() => { function h(){} })`) -- was still missed: the main walk
never recurses into arrow bodies and the scan bailed at the arrow boundary.
Refactors the inline scan into a module-level _scan_js_nested_function_
declarations that also descends through arrow / function-expression bodies
(attributing nested declarations to the nearest enclosing named scope), and
invokes it from the const-arrow branch of _js_extra_walk. Nested bodies join
function_bodies, so the central _tracked_body_ids guard prevents double-walk.
Adds tests for both arrow idioms.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A named `function`/`generator_function` declaration nested inside another
function body now gets its own node, a `contains` edge from the enclosing
function, and its own call-attribution scope, so calls made from inside such a
function are no longer dropped as dangling.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The #2690 form-3 (`$(dirname …)`) branch had no `..` guard and the tracked
form-4 branch let `..` walk past the base to an arbitrary host path, so a
hostile corpus could make the extractor stat and record an out-of-tree file
(source "$(dirname "$VAR")/../../../../etc/passwd") — a corpus-side info leak,
since resolve_bash_source_edges only filters *resolved* cross-file edges, not
the extractor's own edge/probe. Adds a lexical _within_tree gate: form 3
rejects `..` outright (its base is a guess), and a tracked form-4 base may
reach a sibling via $VAR/../lib but cannot escape past base.parent. Adds two
traversal-rejection tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extends variable-built `source` resolution to two path-construction forms
that were silently dropped: `source "$(dirname "$VAR")/lib/x.sh"` (command
substitution in the source argument) and `source "$VAR/../lib/x.sh"` (a
`..` suffix when the leading var is a tracked var_bases entry).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds two regression tests on top of #2684: each referencing symbol gets its
own INFERRED uses edge (guarding against the old every-class fan-out and
against source collapse), and a true module-top-level reference emits no edge
(the deliberate drop). Also adds the CHANGELOG entry.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rewrites Pass 2 of the Python cross-file import resolver so an INFERRED
`uses` edge anchors on the symbol whose body actually references the
imported name (a class as a unit, or a module-level function) at the real
reference line, instead of fanning out from the import statement line to
every class in the importing file.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SrijanSriv