Replaced O(n) linear scan `next((n for n in candidates if n["id"] == neighbor_id))` with O(1) dict lookup via pre-built `candidates_by_id`. Also pre-caches `_norm()` results in `norm_cache` to avoid recomputing per inner iteration.
For a 36k-file codebase (~100k high-entropy candidates) this reduces the pass-2 loop from O(n^2*B) (~30–100B iterations) to O(n*B), eliminating the multi-minute CPU hang after AST extraction completes.
Adds complete Persian translation (README.fa-IR.md) with RTL formatting and updates main README language selector to include Persian between Arabic and Italian.
JS/TS symbol resolution only handled named imports. A default-exported
symbol (`export default class Foo`) imported as `import Foo from './foo'`
produced only a file→file `imports_from` edge; the class node received no
incoming symbol edge. On codebases that default-export most classes
(NestJS services/helpers/models, etc.) those symbols looked like isolated
leaf nodes, and `graphify affected "<Class>"` / `explain` reported no
callers.
Record default imports with imported_name="default", register a "default"
export for `export default <class|function|identifier>`, and let the
existing exported-origin resolver wire the `imports` edge — which also
resolves calls made through the local binding, even when it is renamed
(`import Bar from './foo'; new Bar()`). `export { X as default }` was
already handled via the export-clause path; anonymous defaults
(`export default class {}`) have no resolvable symbol and stay file-level.
Adds regression tests for default-export class/function/identifier import
resolution and renamed-binding call resolution.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A single `!` rule in .graphifyignore set a blanket `has_negation` flag that
disabled directory-level pruning for EVERY ignored directory during the
os.walk in detect(). One unrelated `!docs/**` therefore made the walk descend
bin/, obj/, wwwroot/, generated/, … on large repos — a pathological slowdown.
Output stayed correct (the per-file `_is_ignored` filter still excluded those
files), but the walk visited the entire tree.
The bypass was unnecessary: `_is_ignored` already honours negations correctly —
last-match-wins lets `!dir/` un-ignore a directory (so it is not pruned), and
the gitignore parent-exclusion rule means a `!` cannot rescue a file beneath an
excluded directory, so descending an ignored dir to find a re-included file is
never needed. Prune purely on `_is_noise_dir` + `_is_ignored`.
Adds a regression test that tracks os.walk and asserts the ignored dir is never
descended while the negation still re-includes its target.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes#1264
Also fixes a narrow watch-mode path-filtering bug found while running the required local
full-suite gate for this PR. Hidden directories inside the watched corpus are still ignored;
hidden ancestors outside the watched root no longer suppress valid file events.
_read_tsconfig_aliases joined alias targets onto the tsconfig's own
directory and ignored compilerOptions.baseUrl. In the common monorepo /
NestJS layout (baseUrl "./src" with "@services/*": ["services/*"]), the
alias resolved to <dir>/services instead of <dir>/src/services, so every
aliased import failed to resolve and the import edge was silently dropped
— leaving cross-file caller graphs nearly empty on alias-heavy TS repos.
Resolve `paths` relative to `baseUrl` (TypeScript's actual semantics),
defaulting to "." so configs without baseUrl keep their current behavior.
Add a regression test covering a subdirectory baseUrl; the existing alias
test only exercised baseUrl ".", which is why this slipped through.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two Windows fixes for the claude-cli backend:
1. _call_llm (the label/community-naming path) spawned a bare ["claude", ...],
which CreateProcess cannot resolve to the npm claude.cmd shim (PATHEXT does
not apply) — every labeling batch failed with WinError 2. Mirror the
extraction path's resolution: prefer shutil.which("claude.cmd") and pass
the resolved path. Regression test included.
2. Both claude -p spawn sites now pass CREATE_NO_WINDOW on Windows. Without
it, each per-batch claude.cmd spawn allocates a console — with Windows
Terminal as the default terminal, a labeling run pops one visible window
per batch on the user's desktop for the duration of the model call.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Callable nodes are labeled with a trailing "()" (e.g. classifyProperty()),
so a bare-name query like "classifyProperty" falls through the exact-label
pass and then ties with any prefix sibling (classifyPropertySafe()) in the
contains pass, returning "No unique node match" even though exactly one
callable with that name exists.
Add a bare-name pass between exact-label and exact-source matching that
compares query and label with the trailing "()" stripped from each. Bare
queries now resolve decorated labels and vice versa; genuine duplicates
still return None.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
llm.py: add explicit edge direction rule to extraction system prompt —
source = actor (caller/importer/subclass), target = acted-upon (callee/
imported/base). LLM was systematically emitting callee->caller for calls
edges because the schema never stated direction semantics.
build.py: extend ghost-node merge to catch LLM nodes that populate
source_location (bypassing the old None check). Now uses _origin=="ast"
as the canonical signal — AST nodes always win; any non-AST node sharing
(basename, label) with an AST node is collapsed into the AST canonical.
Fixes LLM bare-stem IDs (bpe_get_pairs) surviving alongside AST
parent-qualified IDs (mingpt_bpe_get_pairs) and carrying reversed edges.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
collect_files ran one full recursive rglob pass per supported extension
(~85 walks), descending into node_modules/.git/venv and filtering those
paths only after enumerating them, and re-evaluated gitignore patterns
per file without the shared cache.
Replace the loop with a single os.walk that prunes noise dirs (and, when
no negation patterns exist, ignored dirs) in place -- the same pattern
the follow_symlinks=True branch and detect.py's scan walk already use --
and pass the shared _is_ignored cache. Extension matching switches to
p.suffix in _EXTENSIONS, matching the follow_symlinks branch and
preserving the .f/.F Fortran case distinction.
Synthetic benchmark (200 source files + 5,000-file node_modules):
0.295s -> 0.007s, identical result set.
Tests: parity oracle against the old implementation on the fixtures and
on a synthetic tree (noise dirs, hidden dirs, gitignore negation), plus
a scandir-counting test asserting each directory is read at most once
and noise dirs are never entered.
Co-authored-by: Cursor <cursoragent@cursor.com>
_body_content used substring checks (startswith("---") / find("\n---")),
so `----` thematic breaks and `--- text` prose lines were mistaken for
frontmatter delimiters and everything above them was silently excluded
from the hash -- edits there never invalidated the cache.
Both delimiters must now be a whole line of exactly three dashes
(optional trailing whitespace). For well-formed frontmatter the stripped
body stays byte-identical to the previous implementation, so existing
semantic-cache hashes do not churn.
Co-authored-by: Cursor <cursoragent@cursor.com>
global_add deduplicates external-library nodes (no source_file) by label
against externals already in the global graph, but dropped every edge
incident to a skipped node. Each repo after the first lost its edges to
shared externals, so cross-repo "what uses library X" queries only saw
the first-added repo.
Build a remap from each deduplicated external to the surviving global
node and rewrite edge endpoints through it before add_edge, skipping
self-loops introduced by the remapping.
Co-authored-by: Cursor <cursoragent@cursor.com>
Pass 2 selected the merge winner from the union of both normalized-label
groups, so never-compared same-label/cross-file nodes could be pulled
into the union, bypassing the #1046/#1178 guards. Pick the winner from
[node, neighbor] only; group members that belong together still merge
via pass 1 (same file) or their own verified comparison.
Co-authored-by: Cursor <cursoragent@cursor.com>
graphify extract <path> --out R is documented to send all output to
R/graphify-out/, but the AST and semantic extraction caches were still
anchored at the scanned project (cache_root=target / root=target). That
re-created a graphify-out/ directory inside the project the user
explicitly asked to keep clean.
Anchor both caches at out_root instead. With --out unset, out_root
equals target, so existing in-project behavior is unchanged.
Adds test_extract_out_keeps_project_root_clean: runs extract from a
project root with --out pointing elsewhere and asserts the artifacts
land under --out while the project directory stays byte-identical.
`claude -p --output-format json` now emits a JSON array of streamed event
objects instead of a single envelope dict, so `_call_claude_cli` crashed
with `'list' object has no attribute 'get'` on every chunk. Add a shared
`_claude_cli_envelope` helper that normalizes both shapes (extracts the
terminal {"type":"result"} object from an array) and use it at both CLI
call sites.
extract intentionally stops at graph.json; GRAPH_REPORT.md requires cluster-only.
Use --no-label to skip LLM community naming (no API key needed in CI).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
graphify/skills/ contains 126 markdown files that trigger semantic extraction.
Add a temporary .graphifyignore entry during CI to keep the build pure AST.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- extract graphify/ (code only) instead of . to avoid LLM API key requirement;
the repo root contains docs/skills/.md files that trigger semantic extraction
- use --out . so output writes to ./graphify-out/ not ./graphify/graphify-out/
- remove --out from export html (flag does not exist; HTML auto-written next to graph)
- drop nonexistent --code-only flag from extract command
- add comments explaining each flag's behavior
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Runs graphify against its own source on every GitHub release (AST-only,
no API cost) and attaches graph.json + graph.html + GRAPH_REPORT.md as
graphify-self-graph.tar.gz to the release. Also supports manual runs via
workflow_dispatch, uploading the bundle as a 7-day workflow artifact.
Closes#1238
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- export.py: guard to_obsidian/to_canvas against dangling community member IDs
(KeyError crash when a node in communities dict is absent from graph, #1236)
- detect.py: NFC-normalize path before hashing Office sidecar filename to fix
macOS NFC/NFD mismatch causing --update to re-extract all Office files (#1226)
- extract.py: add _is_config_json() to skip data JSON files (only extract
package.json, tsconfig.json, eslint, deno, JSON Schema etc.) eliminating
561 orphan key-nodes on large repos (#1224)
- llm.py: add GRAPHIFY_LLM_TEMPERATURE env var + _resolve_temperature() helper;
auto-omit temperature for o1/o3/o4/gpt-5 reasoning models that reject temp=0;
mirrors GRAPHIFY_MAX_OUTPUT_TOKENS precedence pattern (#1191)
- tests: 20 new regression tests across obsidian, detect, extract, llm_backends
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- graphify/_minhash.py: self-contained MinHash/MinHashLSH using pure numpy,
byte-identical hash math to datasketch (sha1_hash32, Mersenne-prime permutation).
Drops datasketch + scipy transitive dep — eliminates EDR hang on Windows where
numpy.testing platform.machine() subprocess spawn was intercepted at import time
- dedup.py: import from graphify._minhash instead of datasketch
- pyproject.toml: replace datasketch>=1.6 with numpy>=1.21
- detect.py: memoize _is_ignored/_eval results in a dict[Path,bool] cache per
detect() call; each unique ancestor dir evaluated once across all sibling files,
eliminating ~42M redundant fnmatch calls on large repos (~34% whole-run speedup)
- tests/test_minhash.py: 11 tests including import-isolation guard asserting scipy
and numpy.testing are not loaded after import graphify.dedup
- tests/test_detect.py: 2 cache tests — correctness (cached==uncached with negation
patterns) and hit-count (each dir evaluated exactly once across siblings)
Closes#1234, closes#1235
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- security.py: replace global socket.getaddrinfo monkey-patch with per-connection
_SSRFGuardedHTTPConnection/HTTPSConnection subclasses (thread-safe, closes TOCTOU)
- security.py: add GRAPHIFY_MAX_GRAPH_BYTES env var override for 512MB cap (MB/GB suffix
supported); improve cap error message to cite the env var
- llm.py: wrap untrusted source files in XML delimiters with sha256 fingerprint;
neutralise jailbreak sentinel tokens to mitigate prompt injection
- dedup.py: skip code nodes in label-based dedup passes; code symbols now deduplicated
by ID only, preventing distinct same-named symbols from merging
- extract.py: cross-file calls resolution now consults import evidence before bailing
on ambiguous callee names; emits EXTRACTED edges when named import is unambiguous
- analyze.py: extend _BUILTIN_NOISE_LABELS with stdlib types and modules
- __main__.py: CLAUDE.md template uses MANDATORY language for graphify-first rule;
PreToolUse hook message hardened to imperative; graphify export html auto-falls
back to community-aggregation view when graph.json exceeds size cap
- tests/test_pg_introspect.py: add importorskip guard for tree_sitter_sql
Closes#1211, #1210, #1205, #1219, #1227; resolves discussion #1019
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Safi