- graphify/_minhash.py: self-contained MinHash/MinHashLSH using pure numpy,
byte-identical hash math to datasketch (sha1_hash32, Mersenne-prime permutation).
Drops datasketch + scipy transitive dep — eliminates EDR hang on Windows where
numpy.testing platform.machine() subprocess spawn was intercepted at import time
- dedup.py: import from graphify._minhash instead of datasketch
- pyproject.toml: replace datasketch>=1.6 with numpy>=1.21
- detect.py: memoize _is_ignored/_eval results in a dict[Path,bool] cache per
detect() call; each unique ancestor dir evaluated once across all sibling files,
eliminating ~42M redundant fnmatch calls on large repos (~34% whole-run speedup)
- tests/test_minhash.py: 11 tests including import-isolation guard asserting scipy
and numpy.testing are not loaded after import graphify.dedup
- tests/test_detect.py: 2 cache tests — correctness (cached==uncached with negation
patterns) and hit-count (each dir evaluated exactly once across siblings)
Closes#1234, closes#1235
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- security.py: replace global socket.getaddrinfo monkey-patch with per-connection
_SSRFGuardedHTTPConnection/HTTPSConnection subclasses (thread-safe, closes TOCTOU)
- security.py: add GRAPHIFY_MAX_GRAPH_BYTES env var override for 512MB cap (MB/GB suffix
supported); improve cap error message to cite the env var
- llm.py: wrap untrusted source files in XML delimiters with sha256 fingerprint;
neutralise jailbreak sentinel tokens to mitigate prompt injection
- dedup.py: skip code nodes in label-based dedup passes; code symbols now deduplicated
by ID only, preventing distinct same-named symbols from merging
- extract.py: cross-file calls resolution now consults import evidence before bailing
on ambiguous callee names; emits EXTRACTED edges when named import is unambiguous
- analyze.py: extend _BUILTIN_NOISE_LABELS with stdlib types and modules
- __main__.py: CLAUDE.md template uses MANDATORY language for graphify-first rule;
PreToolUse hook message hardened to imperative; graphify export html auto-falls
back to community-aggregation view when graph.json exceeds size cap
- tests/test_pg_introspect.py: add importorskip guard for tree_sitter_sql
Closes#1211, #1210, #1205, #1219, #1227; resolves discussion #1019
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bandit, pip-audit, and safety are already declared in the dev dependency
group but nothing in CI invokes them, so a new HIGH-severity finding or
a newly-disclosed CVE in a pinned dep can land without anyone noticing
until the next manual audit.
Add a security-scan job that runs bandit (-ll, HIGH-severity only) and
pip-audit (--strict) on every push and PR. Marked continue-on-error so
this doesn't block PRs on pre-existing findings -- a follow-up should
do the cleanup pass and flip the flag.
safety intentionally omitted: it requires a free-tier API key for the
new commercial backend, which is a setup burden for forks. pip-audit
covers the same ground using the PyPI JSON advisory feed and OSV.
1. graphify merge-chunks dumped the entire node list to the terminal
instead of the node count. __main__.py concatenated merged['nodes']
(a list of dicts) into an f-string where it clearly meant
len(merged['nodes']) -- the other two values in the same line use
len() correctly.
2. global_graph._load_manifest silently returned a fresh empty manifest
on any JSON parse error. That is reachable through normal interrupted
writes (the manifest is rewritten in full on every global_add /
global_remove, with no fsync or atomic rename), and the failure mode
is total data loss: every tracked repo disappears from
~/.graphify/global-manifest.json on the next read.
Back the corrupt file up to <path>.corrupt.<unix_ts> and print to
stderr before returning the empty default. Users can then recover
manually and the failure is visible rather than silent.
All 27 tree-sitter-* deps were unversioned in pyproject.toml. Users
installing via 'pip install graphifyy' (the README's primary install
path) bypass uv.lock entirely and resolve whatever tree-sitter-*
versions PyPI happens to serve. A breaking minor bump in any grammar
package can land in user installs without notice.
Add explicit lower bounds (matching uv.lock) and upper bounds one
minor above (or one major above for 0.x packages with frequent breaks).
Ranges chosen to allow patch updates without re-pinning while blocking
incompatible major/minor jumps.
- analyze.py: pass length_bound=max_cycle_length to nx.simple_cycles() so
networkx prunes during enumeration instead of post-filtering; drops report
generation from never-returns to ~0.1s on dense graphs (#1196)
- llm.py: replace hardcoded min(40+16*n,4096) label_communities token budget
with _resolve_max_tokens(min(64+24*n,8192)) — 24 tok/community covers 5-word
JSON entries; 8192 cap fits 16k-context models; env var now honoured (#1200)
- dedup.py: add prefix-extension guard in Pass 2 and _llm_tiebreak — skip merge
when one normalised label is a strict prefix of the other (getActiveSession /
getActiveSessions, parseConfig / parseConfigFile). Option (a) rejected: dropping
the >=12 early-out from _short_label_blocked breaks test_typo_merged (#1201)
- tests/test_dedup.py: two new regression tests verifying prefix guard fires for
extension pairs and does not fire for same-length typo pairs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Guards _norm, _norm_label, and _strip_diacritics against None node labels that cause TypeError in unicodedata.normalize(). Fixes#1194. Consistent with existing security.py:270 precedent.
Co-authored-by: freiit <freiit@users.noreply.github.com>
Adds extra_body parameter support for custom/OpenAI-compat providers so users can pass provider-specific params (e.g. thinking budget for Claude via Bedrock compat). Adds multi-batch label_communities for 16k-context models — batches multiple community descriptions into a single LLM call instead of one per community. Partial batch failures are handled gracefully.
Co-authored-by: EirikWolf <EirikWolf@users.noreply.github.com>
The --push/--user/--password export flags feed both the neo4j and falkordb
dispatch branches, so the neo4j_ prefix was misleading - a neo4j_password that
reads FALKORDB_PASSWORD made no sense. Renamed to push_uri/push_user/
push_password, and the password env lookup now reads the backend-specific var
(FALKORDB_PASSWORD for falkordb, NEO4J_PASSWORD otherwise) instead of OR-ing both.
The no-push 'graphify export falkordb' path advertised
'redis-cli -x GRAPH.QUERY graphify < cypher.txt', but FalkorDB rejects that
with 'query with more than one statement is not supported' - cypher.txt is a
multi-statement Neo4j script. The individual statements ARE valid OpenCypher
(verified by loading them one at a time), only bulk script import is unsupported.
Message + skill docs now say so and point to --push (the verified load path).
Makes the FalkorDB option a first-class sibling of Neo4j in the agent skill,
not just the export CLI:
- --falkordb / --falkordb-push shorthands documented in core.md + the shared
exports.md reference, so they render into all modular platform skills and
read exactly like --neo4j / --neo4j-push. (The aider/devin monoliths are
diff-frozen vs v8 by skillgen's roundtrip guard, so they are left untouched.)
- README command reference switched to the /graphify ./raw --falkordb-push form.
- Documented URI scheme is now falkordb://localhost:6379; the scheme is only
informational (host/port are parsed out), so redis:// or a bare host:port
remain equivalent. Regenerated skill artifacts + expected/ snapshots.
Adds support for the XML-based `.slnx` solution format (VS 2022 17.13+ replacement for `.sln`). Extracts project references as `contains` edges and build dependencies as `imports` edges. XXE-protected XML parsing with size cap. Wired into `_DISPATCH` and `CODE_EXTENSIONS`. 6 new tests passing.
Co-authored-by: bakgaard <bakgaard@users.noreply.github.com>
Adds `graphify-mcp` as a named console script pointing to `graphify.serve:_main`, making the MCP stdio server directly invocable as a first-class CLI command from uv tool / pipx installs. MCP client configs can now use `"command": "graphify-mcp"` instead of `python -m graphify.serve`.
Co-authored-by: jr2804 <jr2804@users.noreply.github.com>
The Agent Skills spec only defines name, description, license,
compatibility, metadata, and allowed-tools as valid frontmatter fields.
The trigger: /graphify line was non-spec, silently ignored by spec-
following hosts, and flagged by agentskills validate CI checks.
- gen.py: removed trigger emission from _render_frontmatter; added
_is_trigger_line() helper for roundtrip allow-list
- fragments/core/aider.md: removed hardcoded trigger: /graphify
- platforms.toml: removed trigger doc comment and trigger="" entries
- test_skillgen.py: replaced trigger-assertion tests with a single
test asserting no host has trigger: in frontmatter
- Regenerated all 125 skill artifacts
Routing intent is preserved: the description field already contains
"treated as a graphify query first" and "graphify-out/ exists".
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
graphify codebuddy install was writing CODEBUDDY.md and settings.json
but not copying the SKILL.md. Added _copy_skill_file("codebuddy") call
to match the --platform codebuddy path. README hook description updated
from "Glob and Grep" to "Bash search and file reads" to match actual
hook matchers (Bash + Read|Glob).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three-part fix:
dedup.py: Pass 1 exact-merge now skips nodes with an empty source_file.
Previously all no-source_file nodes with the same label landed in one
bucket and were merged, destroying distinct symbols (third-party deps,
standalone functions) that happened to share a short name.
update.md (skillgen + all 13 host variants): the --update merge now
passes both deleted AND changed files to prune_sources, mirroring what
watch._rebuild_code already does correctly. Old nodes for re-extracted
files are pruned before fresh AST is inserted — no fuzzy reconciliation
needed, no cross-file collapse possible.
export.py: anti-shrink guard message now names fuzzy dedup as a
possible cause (not only "missing chunk files"), and advises a full
rebuild as the safe recovery path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds FalkorDB as a sibling option to the existing Neo4j sink, selected via
`graphify export falkordb [--push redis://localhost:6379]`.
- New push_to_falkordb() in graphify/export.py mirrors push_to_neo4j; FalkorDB
is OpenCypher-compatible so the MERGE/SET upsert queries are identical.
- export falkordb subcommand wired in graphify/__main__.py (cypher.txt when no
--push, direct push otherwise). Auth is optional; target graph defaults to
"graphify".
- falkordb optional extra in pyproject.toml (and in the all extra).
- Tests: CLI cypher generation (CI-safe) + real-FalkorDB integration tests that
skip when no instance is reachable.
- README extras table + command reference and CHANGELOG updated.
#1174: affected.py load_graph now forces directed=True before
node_link_graph, matching the identical fix in serve.py and __main__.py.
Undirected graphs (directed:false in graph.json) were causing in_edges
to fall back to a direction-blind scan, missing true callers and
reporting false positives. Regression test added.
#1173: post-commit and post-checkout hook bodies now read
graphify-out/.graphify_root before calling _rebuild_code, falling back
to Path('.') if absent. A scoped build (graphify src/) no longer gets
silently expanded to the full repo on the next commit. Tests added.
#1172: Step 9 cleanup split into rm -f for fixed files and
find -maxdepth 1 -delete for the chunk glob. Under fish/zsh an
unmatched glob aborts the entire rm -f line, leaving temp files on disk.
Fixed in the three skillgen source fragments and regenerated.
#1163: detect_incremental type guard on stored mtime — if the manifest
contains a dict-valued mtime (schema drift from older versions), coerce
to None rather than propagating a non-numeric into comparisons.
Regression test added.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
#1170 — replace nohup with cross-platform Python detach in git hooks.
Git for Windows MSYS has no nohup so post-commit/post-checkout hooks
silently failed. Now uses subprocess.Popen with DETACHED_PROCESS |
CREATE_NEW_PROCESS_GROUP on Windows, start_new_session=True on POSIX.
Quoting-safe (argv list). Fixes#1161.
#1169 — fix _is_sensitive false positives on topic-mentioning filenames.
token-economics-of-recall.md and password-policy-discussion.md were
silently dropped as secrets. Generic keywords (token/secret/password)
now only fire when the keyword ends the filename stem or the stem is
≤2 words. Specific patterns (.env/.pem/id_rsa etc.) remain unconditional.
#1165 — fix multi-word endpoint resolution in _score_nodes.
graphify path "AuthService" "UserRepo" never fired the exact-match bonus
because per-token comparison never equalled the full label. Now joins
normalized tokens and compares against the full label and its tokenized
form. O(1) per node, affects query_graph and shortest_path uniformly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
#1154: scope numpy>=2.0 constraint to python_version>='3.13' only.
numpy 1.26.4 ships no cp313 wheel so uv sync falls back to a source
build requiring a C compiler. The marker avoids forcing numpy 2.x on
3.10-3.12 users who have working 1.x environments.
#1160: codex platform skill now installs to .codex/skills/graphify/
instead of .agents/skills/graphify/. The hook already wrote to .codex/
so the skill destination was inconsistent. Propagates automatically
through install/uninstall (both read _PLATFORM_CONFIG dynamically).
Updated all codex-specific test assertions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
#1118 — prune stale AST nodes on full re-extraction (#1116)
Stamps every AST-extracted node with _origin="ast" in extract(). On a
full rebuild _rebuild_code drops any AST-marked node absent from the
fresh output even when its source file survives, fixing stale symbols.
Backward-compat: marker-less nodes from pre-1118 graphs survive one
cycle then self-heal.
#1110 — stop reading images and PDFs as garbage in headless extract
Images route through per-backend vision payloads (base64/data-URI/bytes
for claude/openai/bedrock); non-vision backends get _strip_pixels for
graceful degradation. PDFs reuse pypdf. 5MB cap, 20-image chunk limit.
#1159 — Salesforce Apex extractor (.cls, .trigger)
Regex-based extractor: classes, interfaces, enums, methods, triggers,
SOQL/DML edges. No new dependency. Dispatched as .cls and .trigger.
#1107 — Azure OpenAI Service backend (--backend azure)
Uses AzureOpenAI SDK client (from existing openai package). Auto-detects
when AZURE_OPENAI_API_KEY + AZURE_OPENAI_ENDPOINT both set. Uses
max_completion_tokens (not deprecated max_tokens).
#1103 — live PostgreSQL introspection (--postgres DSN)
graphify extract --postgres "postgresql://..." introspects tables, views,
routines, and FK relations via information_schema (SERIALIZABLE READ ONLY).
Credentials sanitized on error. New graphify[postgres] extra (psycopg3).
Union-resolved llm.py conflict: Azure functions + bedrock images= param.
Fixed test_image_vision.py mock to accept timeout= kwarg (our #1112).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
python -m graphify.serve graph.json --transport http --port 8080 serves
the same MCP tools over the Streamable HTTP transport (spec 2025-03-26)
so a single shared process can serve the graph for a whole team.
- _build_server() refactors server registration into a shared factory
(stdio behavior is byte-for-byte unchanged — all 52 existing tests pass)
- _ApiKeyMiddleware: raw ASGI (not BaseHTTPMiddleware) preserves SSE
streaming; constant-time compare; RFC-6750 case-insensitive Bearer;
blank-key normalized to no-auth
- DNS-rebinding protection via TransportSecuritySettings; wildcard binds
disable it and print an exposure warning when no api-key is set
- session_idle_timeout reaps idle stateful sessions (default 3600s) so a
long-running shared server does not leak memory on client disconnect
- Dockerfile + .dockerignore for containerized team deployment
- 16 new tests via in-process ASGI test client (importorskip-guarded)
- stdio remains the default; no change for existing setups
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
#1147 — builtin annotation nodes inflate god-node rankings:
Add _PYTHON_ANNOTATION_NOISE frozenset (str/int/bool/float/bytes/
MagicMock/Mock/AsyncMock/...) and apply it alongside
_PYTHON_TYPE_CONTAINERS in the annotation walker so scalar builtins
and test mocks are never created as nodes or emitted as edges.
Defense-in-depth guard in analyze.god_nodes filters _BUILTIN_NOISE_LABELS
so pre-existing graphs are also protected.
#1146 — package-form imports create disconnected islands:
from pkg import submod is now resolved to a file-level imports_from
edge when submod.py or submod/__init__.py exists on disk. Fix lives
in _collect_python_symbol_resolution_facts: when target resolves to
a __init__.py, each imported name is checked as a potential submodule
file and stored in _SymbolResolutionFacts.module_imports. Applied in
_apply_symbol_resolution_facts using stem-based canonical IDs.
#1145 — AST vs semantic node ID ghost duplicates:
build_from_json now runs a two-pass merge after adding all nodes:
collect AST nodes (source_location set) then find semantic ghosts
(same basename+label, no source_location). Ghosts are removed and
their IDs added to norm_to_id so all edges re-point to the AST node.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
TTY-only (suppressed in CI/pipes). Windows ANSI enabled via ctypes.
Wrapped in try/except so it can never crash the install.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Backend resolution now defers until after file detection. A code-only
corpus (pure AST, zero LLM calls) runs without any API key.
Key validation only fires when needs_llm=True (semantic_files non-empty
or --dedup-llm passed). Error message now names why a key is needed and
notes that code-only corpora need none.
Applied from PR #1123 with one fix: _clear_backend_keys in tests now
also clears AWS_PROFILE/REGION, OLLAMA_BASE_URL to prevent CI flakes
on machines with ambient credentials.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_kiro_install was using a bare write_text that bypassed _copy_skill_file,
so the references/ sidecar and .graphify_version stamp were never written
despite kiro declaring skill_refs: "kiro". This left SKILL.md with 8 dead
references/*.md pointers on every install.
Fix: call _copy_skill_file("kiro", project=True) for the skill+sidecar+stamp,
keep the steering-file block inline. Uninstall now calls _remove_skill_file
which also cleans .graphify_version and references/.
Adds regression test asserting SKILL.md + references/ + .graphify_version +
no references.tmp + steering file, and that uninstall removes all of them.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
pytest.importorskip at module level matches the pattern used for other
optional extras (sql, dm) so CI passes without the optional dep.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds lower-bound version constraints matching tested versions in uv.lock. networkx>=3.4 is the binding constraint (edges= kwarg on node_link_data introduced in 3.4). rapidfuzz>=3.0 guards against the 2.x→3.x API break in rapidfuzz.distance. datasketch>=1.6 is conservative but correct. Also fixes stale uv.lock version metadata (0.8.30→0.8.31).
