Windows hash-seed re-exec wait (#3816/#3799), Terraform secret-named
variable-default/output redaction (#3817), COBOL fixed-format
sequence-number detection (#3813) and PERFORM...THRU both-endpoint
linking (#3814), and gate the GRAPH_REPORT undocumented-components
reason on a real semantic layer (#3828/#3801).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Covers issue 3801 directly: a code plus concept only graph (no
document, paper or image nodes anywhere) must not offer undocumented
components as a reason for an isolated node, only possible missing
edges. Adding a single document node elsewhere in the same graph
shape brings the fuller reason back, confirming the gate tracks the
graph's actual content rather than being tied to a particular test
fixture shape.
Confirmed against pre fix code via git stash: the first test fails
there and passes only with the fix applied.
(cherry picked from commit 02e1b4d6c2)
GRAPH_REPORT.md's Knowledge Gaps section always offered undocumented
components as a possible explanation for an isolated node, regardless
of whether the graph could ever actually have one. That phrase only
means anything when a semantic layer exists, document, paper or image
nodes an LLM extracted meaning from, where an isolated node genuinely
could be mentioned but never explained further. A code only graph has
no such nodes at all, so offering it as a live possibility on every
report is misleading.
The reason now checks whether any node in the graph carries a
document, paper or image file_type before including that clause,
using the same VALID_FILE_TYPES distinction validate.py already
draws between the semantic layer and code/rationale/concept nodes, so
no new build time flag is needed.
Fixes issue 3801.
(cherry picked from commit 8a49e1f38a)
A variable or output names its secret in the block label, so the literal
sits under the generic key default/value and the key-name check never
flags it: variable "db_password" { default = "..." } reached graph.json
verbatim. Redact default/value when the label matches the sensitive-key
pattern or the block sets Terraform's own sensitive = true.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 01e417166b)
On Windows os.exec* does not replace the process: CPython spawns a new one
and the caller exits at once, so update/extract/cluster-only/label returned
before graph.json was written, and the exiting parent intermittently died
with an access violation (exit 139). Run the child with subprocess, wait for
it and propagate its exit status there; POSIX keeps the real os.execvpe.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 94a9657e93)
`PERFORM A THRU Z` (and `THROUGH`) runs the whole paragraph range A..Z, but the
PERFORM pattern captured only the first name, so a `calls` edge was emitted to A
and the range-end Z got none. "What performs Z?" then answered nothing even
though this paragraph does, and any reachability walk stopped short of the
range's tail.
Capture the optional `THRU`/`THROUGH` target and enqueue both endpoints for
resolution against the paragraph table. Intermediate paragraphs in the range are
not individually linked (that would need paragraph source ordering); the two
endpoints are the reliable, common signal and both now resolve. A plain
`PERFORM A` is unchanged.
Regression test: `PERFORM A-PARA THRU Z-PARA` now links MAIN to both A-PARA and
Z-PARA (fails on the old single-capture pattern).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JJbLfztSxm2tH5cJwBbe9q
(cherry picked from commit abcdde1a65)
The fixed/free format detector counted a line as fixed-format only when its
first six columns were blank (`^ {6}[ *\-/ ]`). Legacy mainframe COBOL — the
main reason to have a COBOL extractor — puts a sequence NUMBER in columns 1-6,
so every such file scored zero fixed markers and was classified free-format.
In free mode the code is taken verbatim (no column-8 slice), so the sequence
number stays glued to the statement: `000400 MAIN-PARA.` no longer matches the
paragraph anchor `^NAME .? .$`, and the whole procedure layer disappears. A
seven-line program with two paragraphs and a PERFORM collapsed to just the file
and program nodes — zero paragraphs, zero `calls` edges — silently.
Match a sequence area that is six spaces OR six digits, followed by a valid
indicator column (space = code, `*`/`/` = comment, `-` = continuation, `D` =
debug). Once the file is correctly read as fixed-format the existing
column-8..72 slice strips the sequence number and paragraphs/PERFORMs resolve
as they should. Blank-sequence fixed-format and genuinely free-format source are
unaffected (a 6-digit prefix is not something free-format code begins a line
with).
Regression test: a sequence-numbered program now yields its program, both
paragraphs, and the PERFORM `calls` edge — and fails on the old detector.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JJbLfztSxm2tH5cJwBbe9q
(cherry picked from commit 96b9522bd5)
Point the README and he-IL translation at the single current invite (the old 2DDrEgvZb4/598Ad9zQZ links are retired). All Discord references now resolve to one canonical invite.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#3461fixes#3409 (PHP closures produce no nodes / inner calls lost). Adds the
coverage its tests omitted:
- the reporter's exact repro — a FILE-SCOPE argument-position closure — asserting
the closure node exists AND its inner call resolves to the closure (not the file);
- a no-duplicate-node assertion guarding the walk partition against double-walk;
- a method-scope closure asserting its inner call attributes to the closure, not
the enclosing method (pins the function_boundary_types behavior).
Verified end-to-end: closure->handler() edges attribute to {closure#N} nodes with
no duplicates.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address maintainer nits by moving _php_get_route_name and the
_PHP_ROUTING_VERBS frozenset out of the _extract_generic body
to the module level, preventing them from being repeatedly
re-initialized per file on large codebases.
(cherry picked from commit e845f15b68)
Address edge cases identified in review:
- Nested route closures (e.g., inside group() or prefix()) now correctly compose their full path by walking up the AST across closure boundaries.
- Non-routing closures passed to methods that share verbs (like ->get('user:42')) are no longer misidentified as routes. A route path must begin with '/' to qualify, otherwise it correctly falls back to an ordinal name.
(cherry picked from commit 38861aa1c4)
Address reviewer feedback:
- Replace line-based {closure@N} names with stable ordinal {closure#N}
names scoped per enclosing class or file. Ordinals are unaffected by
line insertions/deletions above a closure; only reordering closures
or inserting one earlier in the same scope shifts later ordinals,
which is a much rarer edit.
- Route closures passed to routing verbs (get/post/put/patch/delete/
options/any/match/map) receive a semantic 'VERB /path' label that
is both stable and directly meaningful to graph queries (the primary
use-case from the issue).
Note: adding anonymous_function_creation_expression and arrow_function
to function_boundary_types (introduced in the first commit) means calls
inside a closure now attribute to the closure rather than the enclosing
named function. This is the correct behaviour; existing codebases with
closures inside named functions will see those edges move on next
incremental update.
(cherry picked from commit 5400b9cbd7)
The bug:
PHP closures (`anonymous_function_creation_expression` and `arrow_function`)
were invisible to the graph, disconnecting API routes and callbacks from
the backend code.
The fix:
- Added closure syntax node types to _PHP_CONFIG's `function_types`.
- Updated engine.py to synthesize line-bound names (e.g., `{closure@42}`)
for unnamed closures matching these AST node types, enabling them to be
extracted as linkable nodes in the graph without ID collisions.
(cherry picked from commit c8c3af1960)
#3729 (absolute package import resolution) and #3784 (retract phantom
package/module-collision import edges) landed in the same cut and interacted: a
legitimate `from pkg import sub` (pure package, no colliding pkg.py) now resolves
its provisional edge to the real `pkg/__init__.py` node, but #3784 retracted it —
via the __init__.py candidate id and the len==1 blind fallback — even though there
is no name collision (no phantom cycle to fix). This dropped the correct
package-dependency edge (caught by test_absolute_package_import_targets_package_init).
Tighten #3784 to retract ONLY genuine module-collision phantoms: drop the
__init__.py candidate id forms (keep the module-file `pkg`/`pkg.py` collision ids
that the #3777 fix actually targets) and remove the len==1 blind fallback (a real
phantom always matches by its module-file id, so no fallback is needed). Both the
#3729 package-import tests and all 26 #3784 phantom-cycle tests now pass together.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`to_wiki()` and `to_obsidian()` regenerate the full page set on every call and
write every page unconditionally — `to_wiki` even unlinks every `*.md` up front,
guaranteeing a full rewrite. An export re-runs on any graph.json change, so on
an always-on `watch`/export setup this rewrites tens of thousands of identical
pages per run (measured: 31,417 pages / 137 MB per export, ~7 GB/day), and each
rewrite is also a modification event that re-fires Obsidian re-indexing, sync
clients, and downstream inotify pipelines for content that did not change.
Compare before writing:
- New `paths.write_text_atomic_if_changed(path, text) -> bool` writes (and
reports True) only when the content differs from what is on disk, comparing
DECODED text — text-mode writes translate `\n`↔`\r\n` on Windows, so a byte
compare would report every file changed.
- `export._owned_write` uses it and still records the note in `_written`
(the ownership + prune-exclusion set, #1896), so an unchanged note stays owned
and is neither pruned nor dropped from the manifest.
- `to_wiki` drops the up-front unlink-all — which forced the rewrite and opened
a window where a reader saw an empty `wiki/` mid-export — for an end-of-run
sweep that removes only the `*.md` this run did not produce, keeping the same
orphan guarantee for relabelled/removed communities.
Tests: the helper skips an identical write (mtime untouched) and writes on a
diff or a missing file; an Obsidian re-run leaves an unchanged note's mtime
untouched yet still owned (not pruned) and still rewrites a note whose body
changed; a wiki re-run leaves unchanged articles untouched and still sweeps a
relabelled community's stale page. The wiki/export/obsidian suites stay green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JJbLfztSxm2tH5cJwBbe9q
(cherry picked from commit 91546de3b4)
`_redact_value` (the #3644 guard that keeps hardcoded credentials in .tf files
out of graph.json and the MCP query/get_node surface) recursed into map values
but not list values. HCL routinely nests objects inside tuples —
`list(object(...))` variables, `dynamic` blocks, tuple defaults — and
`_parse_attr_value` turns those into Python lists of dicts, so:
config = { password = "x" } -> redacted (map recursion)
configs = [{ password = "x" }] -> LEAKED (list not recursed)
The list form left the secret in the serialized node verbatim, defeating the
guard for a very common Terraform shape (a `connections`/`containers`/`secrets`
tuple of objects).
Recurse into lists and tuples as well, under the same key — the list branch is
only reached when the key is not itself sensitive (a sensitive key redacts the
whole value above), so a scalar element carries no key signal and is returned
as-is while a dict element is checked against its own inner keys. A non-secret
list of scalars is untouched.
Tests: a unit check that a secret inside a list-of-objects and a nested list is
redacted (while a sensitive key still redacts the whole list and a plain scalar
list is left intact), and an end-to-end check that a `password`/`token` inside a
tuple-of-objects attribute never reaches the serialized node. Both fail without
the fix. The terraform suites otherwise stay green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JJbLfztSxm2tH5cJwBbe9q
(cherry picked from commit 2998cb2121)
The existing probe assertions now expect the module flag form. A new
test drives the probe with a launcher stub path that is not even a
real file in place of argv zero, confirming the fix no longer depends
on it being anything runnable, the exact shape of the Windows uv
launcher regression this closes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
(cherry picked from commit e2fd1c4f30)
The previous form replayed sys.argv[0] as the script path to run,
which happens to work for a POSIX console script wrapper or a python
dash m invocation, both of which are real, runnable Python content.
It does not work for a uv, pip, or pipx console script launcher on
Windows, which installs as a native executable stub with no such
content, so the interpreter has nothing to open and every affected
command failed outright the moment it tried to pin the seed.
Re execing through the module flag instead never depends on argv[0]
being anything runnable at all, so a launcher stub that is not even a
real file no longer matters, and this now matches the exact
invocation shape the end to end test for this function already
exercised and proved working.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
(cherry picked from commit 6509f609dc)
One test sets a shared, absolute GRAPHIFY_OUT and confirms the marker
written by an update from a relative path is resolved to an absolute
form. A companion test confirms the ordinary, relative GRAPHIFY_OUT
case still preserves the caller supplied relative path exactly as the
existing 777 test already expects, so this fix only changes the
shared output case.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
(cherry picked from commit 181a46dca9)
Update writes a raw, caller supplied watch path into the graphify_root
marker, which #777 deliberately chose over a resolved absolute so a
committed marker file stays portable across clones and CI runners.
That is fine as long as every reader of the marker runs with the same
CWD it was written from, which normally holds since the generated git
hooks and an unqualified graphify watch both assume it.
It stops holding once GRAPHIFY_OUT is set to an absolute, shared
location, the multi worktree setup from #686. The same marker file is
then reachable from any worktree's CWD, so a relative value like the
dot from graphify update dot resolves against whichever worktree
happens to read it later instead of the one that was actually
scanned, silently falling back to the wrong scan root.
Add a small helper that only resolves in that shared, absolute case
and otherwise keeps the existing relative behaviour untouched, so the
777 portability guarantee for the common, single clone case is
preserved rather than reverted.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
(cherry picked from commit e36edf8462)
Cover an unrelated mention of the word graphify no longer suppressing
the real registration block, a stale or hand edited block correctly
getting refreshed on re install instead of being left alone, and both
the CLAUDE.md and GEMINI.md writers now passing newline equals empty
string so an existing file's own line endings are never translated on
write.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
(cherry picked from commit ddf2497f68)
_register_always_on_block had three problems, all only visible on the
claude and codebuddy paths that call it. Path.write_text opens in
text mode, so on Windows appending a few lines to an existing
CLAUDE.md rewrote the WHOLE file to CRLF, turning a one line
registration into an eighty five line diff for anyone who keeps that
file in a dotfiles repo or syncs it across platforms. Separately, the
re install guard was a bare check for the word graphify anywhere in
the file, so a user whose file mentioned the project for any other
reason got silently treated as already registered and never received
the real block, and an edited or stale block could never be repaired
by re running install since the word was still present either way.
Switch the write to pass newline equals empty string so existing line
endings are never translated, and switch the guard to
_replace_or_append_section, the same marker anchored helper
claude_install and gemini_install already use, so a stale block is
correctly refreshed instead of silently skipped. That helper's
boundary detection was hardcoded to the next H2 heading, which does
not fit this block's own heading level, so it gains a boundary_prefix
parameter mirroring the one _remove_marker_section already has, kept
in sync with a new shared marker constant so the paired removal
function matches on exactly the same heading. Deliberately kept at H1
rather than switched to the H2 markers the issue itself suggested,
since the existing removal function anchors on H1 specifically to
never collide with a genuine user authored H2 heading elsewhere in
the same file, a design choice from a separate earlier issue that a
plain heading level change would have quietly broken.
gemini_install writes GEMINI.md through the exact same text mode
write, so it gets the same newline fix.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
(cherry picked from commit c325aad3b0)
The two node-lookup MCP tools read arguments["label"] directly, so a client
that passes the node identifier under "node_id" (or "id") gets a
KeyError('label') instead of a result. Several of the agent frameworks graphify
advertises support for send the argument that way, so the tools are unusable
from those clients while query_graph/god_nodes work fine against the same graph.
Extract a small module-level _node_arg() that accepts label/node_id/id, return
a short guidance string when none is given (instead of raising), and declare
node_id in both inputSchemas so the parameter is discoverable. Add a unit test.
(cherry picked from commit cf84c59735)
Cover a Tauri style layout where the manifest lives one level down and
must still be found, a discovered workspace manifest whose own
members resolve relative to its own directory rather than the outer
scan root, a decoy manifest inside a dependency install folder that
must never be searched into, and a manifest nested past the discovery
depth cap that is correctly left unfound.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
(cherry picked from commit 6d2ef5455f)
introspect_cargo only ever looked at root divided by Cargo dot toml,
with no search, so any repository whose crate lives in a subdirectory
got no crate nodes or dependency edges at all, even though the AST
pass had already parsed every rs file there. The common case is a
Tauri app, whose default scaffold keeps its manifest under its own
frontend adjacent crate folder rather than the project root.
Add a bounded depth search that only runs when the root manifest is
missing, reusing the same noise directory pruning the AST scanner
already applies so a vendored or example manifest nested inside a
dependency folder never wins over the project's own. A workspace
manifest found this way now resolves its own members relative to its
own directory rather than the outer scan root, since those two can
now differ.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
(cherry picked from commit 577e51749a)
One test runs extract with the cargo flag against a repo that has no
Cargo.toml at all and confirms it still succeeds and still writes the
graph built from the code already indexed, with a note explaining the
crate edges were skipped. A companion test confirms a repo that does
have a valid manifest still gets its crate node, so the fix only
changes the missing manifest path.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
(cherry picked from commit a9b14fef06)
FileNotFoundError is a subclass of OSError, so a missing Cargo.toml at
the scan root was caught by the same handler meant for ImportError
and connection failures and exited the whole process. That fires
after the AST pass has already completed, discarding every file of
completed structural extraction work just because an optional
enrichment source found nothing to read, an ordinary condition for
any repo whose manifest lives in a subdirectory, such as a Tauri app
that keeps it under its own separate frontend adjacent crate folder.
cargo_result already starts as the empty, handled shape the merge
step downstream expects, so catch FileNotFoundError on its own, print
a note, and let extraction continue with an empty cargo result instead
of exiting.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
(cherry picked from commit 94cc22be3b)
Since a real re exec replaces the process it runs in, every test that
needs to observe whether it fired runs a small probe in its own
subprocess with a deliberately constructed environment, rather than
mocking inside the pytest process itself, which pytest's own handling
of its current test marker would defeat anyway. Cover each of the
four affected commands re executing with the seed pinned when unset,
an explicit seed never being overridden, unrelated commands never re
executing, the guard against firing while pytest itself is mid test,
a failed re exec being survived rather than raised, and a full real
invocation of update still completing successfully end to end through
the whole re exec.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
(cherry picked from commit 2f5f2d705e)
The generated git hooks already export PYTHONHASHSEED=0 because
networkx's Louvain implementation iterates string keyed sets whose
order is randomized per process, so community assignments otherwise
churn between runs with no code change. A bare graphify update, which
the CLAUDE.md template tells agents to run after every code change,
skipped this, clustering differently every time and producing a
large, spurious diff in the output directory.
PYTHONHASHSEED is read once at interpreter startup, so setting it on
os.environ from inside an already running process has no effect on
that process's own hash randomization. The only way to pin it for a
command already in flight is to restart the interpreter with it set
from the start, so main() now re execs itself with the seed set
before doing anything else, for exactly the commands whose output
depends on clustering, only when the caller has not already chosen a
seed themselves, degrading instead of raising if the re exec itself
fails so an unusual host that disallows it can still run graphify.
Skips this entirely while running under pytest, detected via the
PYTEST_CURRENT_TEST variable pytest itself sets for the duration of
a running test. Dozens of existing tests call main() directly with a
monkeypatched argv to simulate a full CLI run in process, which only
worked because main() was previously free of this kind of side
effect; without the guard, every one of those calls replaced the
test process running them the moment PYTHONHASHSEED happened to be
unset, which it normally is in a dev or CI environment, with no
traceback or error, just an empty, silently green test run.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
(cherry picked from commit b70382ceda)
extractors/go.py mints every import target as `go_pkg_<raw import string>` with
no module lookup, so an intra-module import never reaches a file: the sink has
no source_file and nothing links it onward. `affected` and any reachability
walk miss every consumer that arrives through an import, while qualified calls
(`b.F()`) resolve, because the call pass already filters callees by
_go_import_path_for_file. This inverts that mapping once per run and repoints
the intra-module `imports_from` edges onto the imported package's file nodes,
the way the JS/TS resolver repoints module specifiers. Stdlib and external
imports have no file in the corpus and stay sinks by design.
Measured on a 744-file single-go.mod module, same commit, with and without
this change: a 111-file handler package goes from 0/21 direct intra-module
imports resolved to 21/21 (24/24 transitive); the 57 intra-module go_pkg_
nodes disappear; +16,616 imports_from edges; no other relation changes.
Closes#3746
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit fb40961f4a)
- #3716 (Solidity): non-callable members (struct/enum/event/error/state-var) had
no signature and fell back to the LINE NUMBER for their node id, churning ids on
incremental rebuilds. Use the member name (unique per kind in a contract scope)
as the discriminator instead.
- #3719 (method-ghost dedup): gate the candidate index on a method-shaped label
(leading '.' AND '()' suffix) so a dotfile label like '.env' is never treated as
a method; normalize the index/lookup key via make_id for parity with the alias
path. Adds a negative test proving two same-named methods in one file are left
unmerged (the len==1 ambiguity guard).
- #3714 (Erlang): the remote-call resolver reads module/name/arity off node
metadata, which the incremental context builders stripped for non-Ruby langs,
so foo:bar() silently failed to resolve on 'graphify update'. Forward the Erlang
resolution metadata keys through the watch.py/cli.py allow-lists (same fix shape
as the elixir _elixir_module marker).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
