Follow-up on #4007: is_interpreter_variable fired for any bare-$VAR
command word, so `"$DIR/run.sh" stage.py` could mint a spurious invokes
edge to the argument on top of the real .sh calls edge. A bare
interpreter variable ($PYTHON) has no path separator; gate on
`"/" not in cmd_word`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Covers python3/python plus a bare interpreter variable, a non shell
interpreter running a dot sh target landing on its entrypoint node,
and two negative cases: a non interpreter command with a script
looking argument, and a missing target. Confirmed to fail against
pre fix code except for the two negative cases, which hold either way.
(cherry picked from commit 97d4c9207c)
Two small follow-ups on the 0.9.75 batch:
- merge-graphs --previous now reports a clear error and exits 1 on a
malformed/unreadable JSON file instead of crashing with an uncaught
traceback (follow-up to #4006).
- JS/TS super.m() fails closed (no edge) instead of falling back to the
file-wide name map, which bound it to the overriding method as a wrong
self-loop. The extends chain is unknown at this pass, so super can
never be resolved here; inherited this.m() still keeps the plain
lookup (follow-up to #4012).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
self.save, a paren-less save and save(x) with no receiver all send to
the current object, but they went through the file-wide name map,
which keeps only the last definition of each name, so they landed on
whichever class the file declares last, as EXTRACTED. In sinatra's
base.rb, Helpers#not_found was wired to Base#error and
AcceptEntry#to_s to MimeTypeEntry#to_str.
Route them through _self_call_target with the in-file inherits walk.
A top-level def and a mixed-in module method are not methods of an
unrelated class, so implicit-self sends to them resolve as before.
(cherry picked from commit 0175b64783)
self.save(), a bare save() (implicit self) and super.ping() in a Swift
method went through the file-wide name map, which keeps only the last
definition of each name, so they landed on whichever class the file
declares last, as EXTRACTED. In Alamofire, DataRequest.init calling
super.init() was recorded as DataRequest.init calling itself, and
URLEncodedFormEncoder's keyed container calls went to the unkeyed one.
Route them through _self_call_target with the in-file inherits walk,
which Swift supports because its inherits edges exist at this point.
Free functions, constructors and calls from an extension of the same
type resolve exactly as before.
(cherry picked from commit a8333ea8a1)
this.save() in one JS/TS class was resolved through the file-wide name
map, which keeps only the last definition of each name, so a class that
calls this.save() was wired to the save() of whichever class the file
declared last, as EXTRACTED. In zod's v3 types.ts every ZodString,
ZodNumber and ZodBigInt check method landed on ZodDate._addCheck.
Route this.m() and super.m() through the self-call lookup the Python
fix added, renamed _self_call_target. It binds to the caller's own
class when that class defines m(). JS/TS extends edges are only emitted
by the later symbol pass, so the base chain is unknown at this point;
anything the own class does not define keeps the plain lookup, which
leaves inherited this.m() edges exactly as they were.
(cherry picked from commit 33184e7b06)
self.m(), cls.m() and super().m() are exempt from the receiver-typed
defer because the receiver is the caller's own instance, but the lookup
they fell through to was the file-wide name map, which keeps only the
last definition of each name. Two classes in one file that both define
m() sent every self-call to whichever was declared last, as EXTRACTED.
On httpx that wires the sync Client's request/send chain into
AsyncClient, and every super().__init__() in _exceptions.py into
RequestNotRead.__init__.
Resolve these against the enclosing class, then its in-file bases
nearest first (super skips the class itself), the same order the ObjC
resolver uses for self. A tie between two bases on one level binds
nothing, and a method of a class outside the chain is never the target.
A module-level callable stored on the instance still binds as before.
(cherry picked from commit 02e1ed4e10)
In _resolve_python_member_calls, candidate imported modules are gathered from imported_by_filenode by checking in contains_children. Following #3410, functions enclosing a nested def also have contains edges to their nested functions. When a caller imports both a module and a function from that module (where the function contains a nested def), both the module and the function matched the candidate filter because the function's _module_stem_key evaluates to the stem of its source file. The resulting len(mods) == 2 triggered the ambiguity guard and dropped all module.func() calls from that file.
Exclude contained symbols by verifying not in file_of_node, which is already populated for all containment targets and excludes top-level file/module nodes.
(cherry picked from commit 542e4869ba)
Under the spawn start method (the Windows default) each worker re-imports the
parent's __main__ by its __file__ path. A stdin (`… | python -`), `python -c`,
or REPL caller has no such file — the path is missing or `<stdin>` — so every
worker dies during bootstrap and the pool raises BrokenProcessPool before any
work is done. The shipped SKILL.md pipes a heredoc into the interpreter, so on
Windows this is the path most users take, not an edge case: every skill-driven
run prints one full worker traceback per core, then a fallback warning that
blames a missing `if __name__ == "__main__":` guard — a fix the caller cannot
apply, because there is no file to put a guard in.
Detect the unusable-__main__ case before creating the pool: when the start
method is spawn (or the platform is Windows) and __main__ has no importable
file, run sequentially and print one note that names the real cause (stdin/-c/
REPL) and the remedy (write the step to a .py file, or pass parallel=False). A
script that DOES have a __main__ file but omits the guard is a different failure
this cannot catch here; it still surfaces via the existing BrokenProcessPool
fallback, whose __main__-guard message is now accurate for that remaining case.
Tests cover the detector (stdin, REPL-without-__file__, a real script, and the
fork start method) and that extract() with >= _PARALLEL_THRESHOLD uncached files
and an unusable __main__ never attempts the pool yet still produces correct
output. All five fail without the guard.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JJbLfztSxm2tH5cJwBbe9q
(cherry picked from commit b475196ed1)
_extract_parallel falls back to sequential on BrokenProcessPool, but an
OSError raised while constructing the ProcessPoolExecutor escaped and
killed the whole extraction (and `graphify update`).
Seen on macOS: ProcessPoolExecutor -> multiprocessing SemLock -> sem_open
fails with OSError(ENOSPC), "No space left on device", once leaked POSIX
named semaphores exhaust kern.posix.sem.max — with the disk nowhere near
full. Sequential extraction needs no semaphore, so hand back False the
same way the BrokenProcessPool branch does, with a warning that names the
OS error rather than the Windows __main__ hint.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit c154c01d50)
`script_invocation` edges (#1756) only ever resolved a path that is a
bare literal -- `./helpers.sh`, or `bash ./helpers.sh`. Both paths into
that branch go through `literal()`, which rejects by design any token
containing `$`, so the single most common way a shell script calls a
sibling is invisible:
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
"$script_dir/helpers.sh" --flag
`cmd` is None for that command node, the `elif cmd and ...` guard short-
circuits, and no edge is emitted. On a 206-file shell tree that uses the
`script_dir` idiom throughout, `script_invocation` produced *zero* edges
-- the feature was effectively dead for that whole style of codebase.
The `source` branch already solved the same problem for
`source "$DIR/lib/x.sh"` (#2079): strip the leading `${VAR}/` segments
with `_bash_source_suffix` and resolve the literal remainder against the
sourcing file's own directory, because the canonical `script_dir` idiom
makes that variable equal to exactly that. This applies the identical
treatment to the exec position -- read the command-name node's raw text
rather than its `literal()`, strip the same way, and fall through to the
existing `resolved.is_file()` gate.
Nothing new is fabricated. `_bash_source_suffix` still returns None when
the remainder holds another expansion (`"$dir/$name.sh"`) or a `..`
segment, and the on-disk check still means a wrong script-dir guess
emits nothing at all. The three existing negative tests -- missing,
shadowed, and dynamic invocations -- pass unchanged.
Measured on the same 206-file tree: 0 -> 16 edges across 10 distinct
file pairs, every one hand-checked to a real invocation of a real file,
no false positives. Full suite: no change in outcomes (the 25 failures
here are pre-existing and unrelated -- terraform, ollama, skillgen).
Fixes#3416.
(cherry picked from commit 2e112c39fc)
`from . import brain, ledger` in a directory with no __init__.py — a
namespace package, which `python -m pkg.mod` runs without complaint —
emitted nothing: _resolve_python_module_path returns None when there is
no module file to probe, and the #1146 submodule branch sits behind that
None. So every `brain.think()` / `ledger.write()` call in such a repo was
invisible to the #1883 module arm, and the most-called functions carried
in-degree 0 in the graph.
_resolve_python_namespace_dir mirrors the module-path walk (relative
base, scan root, then sys.path-root ancestors per #2072) and returns only
a directory that exists inside the root and has no __init__.py. The fact
collector uses it when the module path resolves to no file; the existing
submodule probe then emits the same imports_from edges a regular package
gets. A name that is not a module file on disk still emits nothing, and
a namespace package binds no symbols of its own.
Measured on a 275-node Python repo laid out this way, same graphify
version with and without the change: calls 201 -> 307, nodes unchanged;
think() 0 -> 15 callers, remember() 0 -> 10, ledger write() 0 -> 20.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KTNEBgonu6ASph6w8R2Wcp
(cherry picked from commit e2ec796bfd)
_LANGUAGE_BUILTIN_GLOBALS is one union across every language, right
for a BARE call (String(x) really would become a god node) but wrong
for a MEMBER call. open is a Python builtin and Set a JavaScript one,
so session.open() in Swift or _server.Set() in C# named after another
language's builtin was silently discarded outright: no same file
edge, but also no raw_calls entry, so cross file resolution never got
a chance to try it.
A member call carries a receiver, so it is not the ambiguous case the
union guards against. Lets a builtin named member call reach the
existing tgt_nid = None deferral every receiver typed resolver
already relies on for other cases (a capitalized Python receiver, any
Java member call, and so on), so it can only ever reach an edge
through a guarded, receiver typed resolver, never the unguarded
bare name path a real god node would need.
Verified against real cross file setups for Python (class qualified
call) and Swift (field typed receiver, reproducing the reporter's own
AppleReadyStreamFinder.find -> RemuxPlayback.open example), plus that
a same file member call still cannot bind to an unrelated bare
function sharing the builtin name.
Scoped to the shared engine only. go.py and rust.py carry the same
outer filter, and the reported issue asks for the same change there
too, but tracing where their raw_calls entries actually go shows the
shared cross file resolver skips every is_member_call entry
unconditionally (extract.py, the "obj.log() has no import evidence"
guard) regardless of builtin status, and neither language has a
dedicated receiver typed resolver the way Python, Java, C#, Swift,
TypeScript, C++, and Objective-C do. Making the same change there
would be safe but currently inert, so left out rather than shipping a
change with no verifiable effect; worth revisiting once either
language gets its own receiver typed cross file resolver.
Fixes#3381.
(cherry picked from commit c0f9ff57e0)
A Python type reference to a name imported from another module dangled or mis-resolved.
Run Python cross-file import resolution before the generic stub rewire and repoint sourceless
AST reference/inherits/implements edges onto the exact imported definition using the
file-scoped (module, symbol) -> node_id binding (not a bare-name match), so two same-named
types in different modules bind to their own; orphaned stubs are pruned only when no edge
still touches them.
This reverts #3155 (b568488): the source_file backfill also fired on a shared external stub
(e.g. a cross-file `Path` reference with no source_file), giving the stub a file identity so
the orphan prune swept it when that one file was deleted even though another file still
referenced it — regressing test_prune_sweeps_orphans. The fix needs to distinguish a
legacy node that genuinely lost its provenance from an external stub that must stay
sourceless; sending #3155 back as needs-work.
A sourceless legacy node now backfills its source_file from incident-edge provenance (only
when real edge evidence exists, deterministic min, never fabricated), and a declined data
JSON now emits a single file-identity node carrying source_file instead of nothing, so
freshness/affected checks can see the file. No node is resurrected and config JSON does not
double-add its file node.
A CommonJS export wrapped in a higher-order function (exports.x = wrap(fn),
module.exports.y = onCall({...}, handler)) was not captured, so the exported symbol was
missing. Reach through the wrapping call to the topmost function closure argument (mirroring
the const-export HOF path #2552/#2568); the wrapper stays a call edge, not the export's
identity, and a non-function arg fabricates nothing.
A command substitution on the right of an assignment (x=$(foo)) was not recorded as a
call, unlike a bare $(foo). Gate call emission on the ancestor chain: allow a call inside a
command_substitution when it sits under a variable_assignment, while still suppressing bare
and arg-position substitutions and process substitutions.
A class extending a built-in base (PHP `extends \Exception`) emits a sourceless supertype
stub, and the corpus rewire bound it to the unique same-labelled real definition anywhere
in the corpus — so in a PHP+TS monorepo a PHP class inherited from the TypeScript
Exception, a phantom cross-language edge / god node. Gate the supertype-target rewire with
a per-language curated builtin-base set: refuse the rewire only when the referring file's
language names the base as a builtin and the resolved target is in a different language
family. Same-language inheritance and user classes named like a builtin still link.
A data-shaped JSON that extract_json intentionally declines (returning a `skipped`
marker, not an `error`) was counted as a failed extraction, so it was kept out of the
incremental manifest and re-processed on every run. Treat the `skipped` decline as a
valid empty outcome, disjoint from genuine failures (which still return `error` and are
still reported).
The factory-object owner path emitted the `contains` edge inside the per-member
loop; add_node dedups on id but add_edge does not, so N methods assigned to one
object flooded the graph with N identical contains edges. Emit it once per owner.
Tests: assert a single contains edge across four methods, cover arrow-function
assignment (the dominant modern factory shape), and lock the negative case that a
non-object-literal receiver (`external.handler = fn`) is not captured.
Preserve callable members assigned to a local object-literal factory API
(`const api = {}; api.foo = fn`), modeling the API object beneath its factory
and attaching the assigned functions as methods. The lazy owner-node minting
(only for identifiers proven to be object-literal bindings in the enclosing
function) avoids the #1077 config-object flood.
Partially addresses #2524 (the object-literal-method shorthand form
`return { foo() {} }` remains out of scope).
Adds the two invariants the deep-dive flagged: require(variable) must not
fabricate an edge, and a module-scope require still yields exactly one
imports_from edge (guards the module-vs-body pass partition against future
double-counting). Also adds the CHANGELOG entry.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A require(...) written inside a function body (the canonical lazy-require idiom
for breaking circular deps) produced no edge at all, while the identical require
at module scope resolved as EXTRACTED imports_from/imports edges. walk_calls now
feeds nested require declarations to the same _require_imports_js routine,
attributing the edge to the enclosing callable; dynamic require(var) is still
skipped and no bare local node is minted (#1077 scope guard).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2705 fixed `function` nested in a `function`, but the React idiom that
motivated the issue -- a function declared inside an arrow-defined component
(`const Panel = () => { function handleClick(){} }`) or an arrow callback
(`useEffect(() => { function h(){} })`) -- was still missed: the main walk
never recurses into arrow bodies and the scan bailed at the arrow boundary.
Refactors the inline scan into a module-level _scan_js_nested_function_
declarations that also descends through arrow / function-expression bodies
(attributing nested declarations to the nearest enclosing named scope), and
invokes it from the const-arrow branch of _js_extra_walk. Nested bodies join
function_bodies, so the central _tracked_body_ids guard prevents double-walk.
Adds tests for both arrow idioms.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A named `function`/`generator_function` declaration nested inside another
function body now gets its own node, a `contains` edge from the enclosing
function, and its own call-attribution scope, so calls made from inside such a
function are no longer dropped as dangling.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The #2690 form-3 (`$(dirname …)`) branch had no `..` guard and the tracked
form-4 branch let `..` walk past the base to an arbitrary host path, so a
hostile corpus could make the extractor stat and record an out-of-tree file
(source "$(dirname "$VAR")/../../../../etc/passwd") — a corpus-side info leak,
since resolve_bash_source_edges only filters *resolved* cross-file edges, not
the extractor's own edge/probe. Adds a lexical _within_tree gate: form 3
rejects `..` outright (its base is a guess), and a tracked form-4 base may
reach a sibling via $VAR/../lib but cannot escape past base.parent. Adds two
traversal-rejection tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extends variable-built `source` resolution to two path-construction forms
that were silently dropped: `source "$(dirname "$VAR")/lib/x.sh"` (command
substitution in the source argument) and `source "$VAR/../lib/x.sh"` (a
`..` suffix when the leading var is a tracked var_bases entry).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds two regression tests on top of #2684: each referencing symbol gets its
own INFERRED uses edge (guarding against the old every-class fan-out and
against source collapse), and a true module-top-level reference emits no edge
(the deliberate drop). Also adds the CHANGELOG entry.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rewrites Pass 2 of the Python cross-file import resolver so an INFERRED
`uses` edge anchors on the symbol whose body actually references the
imported name (a class as a unit, or a module-level function) at the real
reference line, instead of fanning out from the import statement line to
every class in the importing file.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Prevent package-qualified calls and types from collapsing onto unrelated bare-name symbols. Preserve Go import evidence, resolve internal packages exactly, canonicalize external type stubs, and support incremental resolution context.
Investigation, implementation, and regression fixtures prepared with OpenAI Codex.
These land together because they are interleaved in extract.py/engine.py.
#2584 (PR #2588, thanks @phudayyy): the 0.9.38 dynamic-import dedupe keyed
only on target, so an in-function import() suppressed the file-level edge
affected follows. Dedupe now keys on the importing file, emitting one
file-level dynamic_import edge per file/target while keeping the call-site
edge.
#2586 / #2417 (PR #2586, thanks @EZZEASY): a Python member call on an
untyped receiver (x.get(...)) no longer binds by name to a same-named
module function. walk_calls now defers non-self/cls/super Python member
calls to the evidence-gated resolver; super().method() still resolves.
Known trade: same-file x = Thing(); x.method() loses its evidence-free
edge (precision over recall, per #2553).
#2589/#2591 resolver arm (in _resolve_objc_member_calls): the @protocol
exclusion and the self.field/_ivar receiver resolution (paired with the
objc.py extractor changes committed separately).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2543 (adopts PR #2546, thanks @michaelxer): a failed extraction is no
longer stamped in the incremental manifest as up-to-date, so graphify
update retries it instead of skipping it forever; a manifest already
poisoned by the old behavior is healed on the next run; genuinely
unchanged files are not re-processed. Extended to the watch save_manifest
paths too.
#2554 (adopts PR #2555, thanks @annieyii): the claude-cli backend now
inspects the stdout envelope for an is_error result (e.g. a rate limit
returned with exit code 0) and raises it on both the zero and non-zero
exit paths, instead of parsing it as an empty success and bisecting
against a live rate limit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2411 (thanks @JensD-git): key the C# partial-class merge on assembly
(nearest ancestor .csproj/.fsproj/.vbproj) in addition to namespace and
name, so same-name partial classes in different assemblies stay distinct
while genuine partial halves within one project still merge. Fixes a
0.9.32 regression from #2332.
#2437/#2438 (thanks @aryanbonigala, builds on PR #2439): incremental
rebuilds no longer drop member-call and indirect_call edges from a
changed file into an unchanged target. Re-resolution now sees the
unchanged corpus (nodes, contains/method edges, and _callable markers,
which now persist to graph.json like _origin); edges to a genuinely
removed target are still evicted.
#2444/#2445 (thanks @Baziar, builds on PRs #2461/#2458): a
BrokenProcessPool triggers the sequential fallback instead of being
swallowed per future, a failed worker file is retried sequentially
rather than merged as empty, and a whole-pass AST failure on a fresh
build exits non-zero instead of writing a zero-node graph
(--allow-partial opts into a best-effort partial graph).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Absolute/machine-slug ids still leaked into edge endpoints from producers
the target_file-stamp loop didn't reach. Three fixes: apply id_remap to
raw_calls caller_nid so module-top-level indirect_call sources canonicalize
(#2231); a general backstop in the final relativization pass that learns
_make_id(abs source_file) -> canonical id for every node and rewrites all
node ids and edge endpoints (in-root -> _file_node_id, out-of-root -> ext_),
suffix-aware for __entry; and target_file stamps on bash source/entry edges
so they ride the same canonicalization. No node id or edge endpoint now
carries the scan-root slug for any file in the batch. Builds on #2250.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to #1899. That fix taught the relativization pass to catch a NODE
whose id was minted from an absolute out-of-root path and give it a portable
"ext_"-namespaced id, by matching the node's own id against
_make_id(str(its source_file)). But several cross-file resolvers (Python
relative imports, C/C++/ObjC quoted #include) only ever emit an EDGE for an
import target, no node -- so when that target lives outside the scan root,
the belt-and-braces pass has nothing to learn the old->new id from, and the
edge keeps the raw _make_id(str(absolute_path)) slug forever. The scan path,
including the OS username, ends up in links[].source/target, and differs
between machines/checkouts even though the node id sets are identical.
_import_c also never stamped the transient `target_file` hint (#1814/#2169)
its Python/JS siblings already use for exactly this kind of cross-file
target canonicalization, so it could not benefit from that machinery either.
Fix, in the two places this root cause actually lives:
- _import_c now stamps target_file on a resolved #include, mirroring
_import_python/_import_js.
- The id_remap pass that already walks target_file-stamped edges to
canonicalize in-root-but-unscanned targets now also handles the
out-of-root branch it previously skipped ("leave its ids alone"): an
existing out-of-root target gets the same portable ext_-namespaced id an
out-of-root NODE already gets, so an edge with no node of its own is
covered too. A target that does not exist on disk still stays dangling,
unchanged from before.
_portable_out_of_root_sf moved next to id_remap so both the new edge-target
branch and the existing node-level pass share one implementation.
Four tests in tests/test_extract.py: the out-of-root #include gets a
portable id instead of the raw slug, and its transient target_file hint
never leaks into the returned edge; the same corpus built from two
differently-nested checkout paths produces a byte-identical target (the
reported non-determinism, made explicit); an in-root, same-batch include
still resolves to the real node's id (negative/regression guard); and the
equivalent out-of-root Python relative import is fixed too, since the gap
was in the shared remap path, not language-specific.
Known limitation: this covers every current target_file-stamping resolver
(Python relative imports, C/C++/ObjC #include, JS/TS/Svelte/Astro/Vue
rescued imports). A resolver that mints a path-derived edge target WITHOUT
stamping target_file at all -- none do today -- would still leak; the fix
closes the gap in the shared mechanism, not a per-language allowlist.
#2079 resolved `source "${VAR}/lib/x.sh"` by stripping the leading expansion and
resolving the literal suffix against the sourcing script's own directory. That is
correct for the canonical
`DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"` idiom, but wrong whenever
the variable points somewhere else. With
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
source "${ROOT}/lib/utils.sh"
the real target is <root>/lib/utils.sh, yet a same-named decoy under the script
dir (<root>/scripts/lib/utils.sh) won: a wrong imports_from edge to a real node.
Track top-level variable assignments and use the assigned base for the leading
variable:
- the script-dir idiom, including any number of trailing `/..` hops (and the
`$0` spelling), resolves to the script dir walked up that many times
- a literal value resolves as-is when absolute, or against the script dir when
relative
- anything else -- a value built from other variables, or command substitution we
do not model -- stays untracked, so the previous script-dir guess is kept
Only the base changes. The suffix guards are untouched: an expansion left in the
suffix, an empty suffix, and a `..` component in the suffix are all still
rejected, the edge is still gated on is_file() and still emitted as INFERRED.
Two coverage gaps left by #2141 / #2157, both misses rather than fabrications.
1. Extensionless shebang scripts. _SHEBANG_DISPATCH already routes a
`#!/usr/bin/env bash` file with no extension to extract_bash, so its functions
are indexed, but the cross-file source-resolution pass picked participants by
filename suffix (`p.suffix in (".sh", ".bash")`). A sourced extensionless lib
was therefore excluded and calls into it never bound. Select by shape as well:
the bash extractor tags every node it emits with metadata.language == "bash".
The suffix check stays so an empty .sh file, which has no nodes to inspect,
still participates.
2. Bare `source lib.sh` (no ./ prefix). Only the `raw.startswith((".", "/"))`
branch recorded a bash_sources entry; a bare name fell through to the opaque
`imports` fallback, so neither the source edge nor calls into the lib resolved
even though the file usually sits beside the script. The else branch now binds
a sibling of that name when one exists.
The existence gate from the ./-prefixed branch carries over: a bare name that
resolves to no sibling keeps the old `imports` edge and records no bash_sources
entry, so nothing is invented. is_file() is wrapped against OSError so a name
that is invalid for the platform degrades instead of raising.
Transitive sources (a->b->c) remain unresolved, as the issue notes.
`_extract_parallel` spawned a ProcessPoolExecutor whenever there were at least
_PARALLEL_THRESHOLD (20) uncached files, even when the resolved worker count was
1. A one-worker pool buys no parallelism: it still pays a process spawn plus an
IPC round trip per file, and it is the one residual case where the parent's
rebuild watchdog (os._exit) can orphan a worker that is mid-task.
The Windows post-commit hook exports GRAPHIFY_MAX_WORKERS=1, so this was the
default there for any rebuild touching 20+ uncached files.
Gate the pool on the resolved worker count -- after the GRAPHIFY_MAX_WORKERS
override and the win32/floor clamps -- and return False when it is 1. That reuses
the existing contract: the caller already falls back to `_extract_sequential`
in-process when `_extract_parallel` returns False.
Tests: no pool is constructed with GRAPHIFY_MAX_WORKERS=1 and 25 uncached files,
and a multi-worker run still takes the pool path.
Only item 2 of #2173 is addressed here. Item 1 (the `graphify watch` rebuild
timeout) needs a maintainer decision first: `watch()` currently arms no timeout
at all on any platform -- there is no signal.SIGALRM branch in graphify/watch.py
to add an `else` to -- so applying the hook's shape means adding a watchdog that
os._exit(1)s a long-running foreground watcher on a slow-but-healthy rebuild.
That is a behaviour change rather than a Windows-compat fix, so it is left out of
this PR.
The #2139 ${VAR} source handler now also records bash_sources so
resolve_bash_source_edges binds calls into the sourced lib's functions,
not just the source edge. Add the end-to-end oracle plus the previously
untested _bash_source_suffix guards (mid-path $, whole-var, .. traversal
fabricate nothing).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`source "${BENCH_DIR}/lib/x.sh"` (the `dirname "${BASH_SOURCE[0]}"` idiom)
took the bare-name branch, which baked the unexpanded `${BENCH_DIR}` text
into the target id via `_make_id`. That id matches no node, so the edge was
flagged dangling and dropped at export — shared shell libraries looked
orphaned and were split into separate communities.
Detect a `$`-expansion in the source argument, strip the leading
expansion segment(s), and resolve the literal suffix against the script's
own directory (which is what the canonical idiom makes the variable). Emit
`imports_from` as INFERRED only when it resolves to a real file on disk;
skip otherwise instead of emitting a dead id. Bare-name sources keep their
existing behavior.
extract_bash only linked calls whose callee was defined in the same file, so a
call to a function from a `source`d library was silently dropped and shell
scripts looked disconnected from the libraries they use. The resolver for exactly
this, resolve_bash_source_edges, already existed with tests but had no call site
and no raw_calls/bash_sources to work from.
extract_bash now emits `bash_sources` (the files it `source`s) and `raw_calls`
(calls whose callee isn't defined locally), and the pipeline runs them through
resolve_bash_source_edges after the id-remap passes, so caller and function node
ids are final and the already-emitted source edge is de-duped. Resolution is
scoped to the source relationship: a call to an external command never binds to a
same-named function in an unsourced file, and bash raw_calls are excluded from the
generic global-name resolver for the same reason.
`from pkg import mod as alias` correctly emits the file-level
`imports_from` edge, but every downstream `alias.func()` call was
dropped: the module arm of the cross-file member-call resolver
(#1883, _resolve_python_member_calls in extract.py) matches a call
receiver against the imported module's own file stem, with no
awareness that the local binding in the importing file can be a
different name. `from pkg import mod` / `mod.func()` resolves because
the receiver ("mod") equals the stem; aliasing breaks the match
because the receiver ("alias") never does, and the calls edge
silently disappears while imports_from stays present -- the graph
looks connected, only the symbol-level reverse query comes back
empty. `import pkg.mod as alias` regresses the same way through the
same resolver.
Root cause is a missing propagation, not a missing feature: the local
alias is already parsed correctly in two places (_python_imported_names
in extractors/resolution.py, and the aliased_import branch of
_import_python in extract.py) but discarded before it reaches the
edges the module arm reads.
Fix threads the alias through as a `local_alias` field on the
`imports`/`imports_from` edge (mirroring the existing `target_file`
transient-hint pattern, #1814, including its pop-once-consumed
hygiene so the hint never reaches graph.json):
- _import_python now splits the alias off `import pkg.mod as alias`
and stamps it on the edge instead of only using it to compute the
bare module_name.
- _SymbolResolutionFacts.module_imports gains a 4th `local_name` slot
so the `from pkg import submod [as alias]` submodule path (#1146)
carries the binding through to _apply_symbol_resolution_facts,
which now stamps `local_alias` on the edge whenever it differs from
the submodule's own stem.
- The module arm's receiver match now accepts the tracked alias in
addition to the module's real stem, keyed per (importing file,
target module) so two files aliasing the same module differently
each match their own binding.
- extract() pops `local_alias` off every edge right after
run_language_resolvers runs, and build_from_json drops it from edge
attrs too -- the same two spots target_file is dropped at (#1814),
except the extract()-side pop has to happen AFTER the resolver
reads the field, not at the earlier point _disambiguate_colliding_
node_ids already pops target_file: that function runs before
run_language_resolvers, so popping local_alias there would strip it
before the resolver ever sees it and silently undo the fix above.
Known limitation, left out of scope: two different aliases bound to
the same module in the same file only resolve the last one
registered, since the match is one alias slot keyed per (importing
file, target module) -- not a regression, since the parent resolved
neither.
Adds five regression tests in tests/test_extract.py: the issue's own
shape (`from pkg import gate as m_gate`), its try/except-guarded
variant (the issue's literal repro, confirming the drop is
independent of try: nesting), the adjacent `import mathlib as m` and
dotted `import pkg.gate as g_alias` forms, and the relative `from .
import gate as r_gate` form -- plus an assertion that `local_alias`
never survives into the returned edges. All fail on the parent commit
with the exact missing-edge symptom and pass after the fix. Full
suite: 3554 passed vs. 3549 on the unfixed parent, a delta of exactly
these five new tests; ruff clean. #2080's calls-edge-direction
regression tests (test_serve.py) still pass unchanged.
