Commit Graph

251 Commits

Author SHA1 Message Date
Giovane Cesar da Silva b56a415388 feat(extract): add opt-in --cargo crate dependency extractor
Closes #1264

Also fixes a narrow watch-mode path-filtering bug found while running the required local
full-suite gate for this PR. Hidden directories inside the watched corpus are still ignored;
hidden ancestors outside the watched root no longer suppress valid file events.
2026-06-11 15:08:36 -06:00
Safi a37672f25f fix: obsidian crash, NFC/NFD dedup, JSON data nodes, OpenAI temperature, JSON config detection
- export.py: guard to_obsidian/to_canvas against dangling community member IDs
  (KeyError crash when a node in communities dict is absent from graph, #1236)
- detect.py: NFC-normalize path before hashing Office sidecar filename to fix
  macOS NFC/NFD mismatch causing --update to re-extract all Office files (#1226)
- extract.py: add _is_config_json() to skip data JSON files (only extract
  package.json, tsconfig.json, eslint, deno, JSON Schema etc.) eliminating
  561 orphan key-nodes on large repos (#1224)
- llm.py: add GRAPHIFY_LLM_TEMPERATURE env var + _resolve_temperature() helper;
  auto-omit temperature for o1/o3/o4/gpt-5 reasoning models that reject temp=0;
  mirrors GRAPHIFY_MAX_OUTPUT_TOKENS precedence pattern (#1191)
- tests: 20 new regression tests across obsidian, detect, extract, llm_backends

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 19:31:47 +01:00
Safi 5504c84324 perf/fix: replace datasketch with pure-numpy MinHash; memoize detect ignore checks
- graphify/_minhash.py: self-contained MinHash/MinHashLSH using pure numpy,
  byte-identical hash math to datasketch (sha1_hash32, Mersenne-prime permutation).
  Drops datasketch + scipy transitive dep — eliminates EDR hang on Windows where
  numpy.testing platform.machine() subprocess spawn was intercepted at import time
- dedup.py: import from graphify._minhash instead of datasketch
- pyproject.toml: replace datasketch>=1.6 with numpy>=1.21
- detect.py: memoize _is_ignored/_eval results in a dict[Path,bool] cache per
  detect() call; each unique ancestor dir evaluated once across all sibling files,
  eliminating ~42M redundant fnmatch calls on large repos (~34% whole-run speedup)
- tests/test_minhash.py: 11 tests including import-isolation guard asserting scipy
  and numpy.testing are not loaded after import graphify.dedup
- tests/test_detect.py: 2 cache tests — correctness (cached==uncached with negation
  patterns) and hit-count (each dir evaluated exactly once across siblings)

Closes #1234, closes #1235

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 15:50:38 +01:00
Safi 6695f0aefd fix: security hardening, dedup correctness, and large-graph support
- security.py: replace global socket.getaddrinfo monkey-patch with per-connection
  _SSRFGuardedHTTPConnection/HTTPSConnection subclasses (thread-safe, closes TOCTOU)
- security.py: add GRAPHIFY_MAX_GRAPH_BYTES env var override for 512MB cap (MB/GB suffix
  supported); improve cap error message to cite the env var
- llm.py: wrap untrusted source files in XML delimiters with sha256 fingerprint;
  neutralise jailbreak sentinel tokens to mitigate prompt injection
- dedup.py: skip code nodes in label-based dedup passes; code symbols now deduplicated
  by ID only, preventing distinct same-named symbols from merging
- extract.py: cross-file calls resolution now consults import evidence before bailing
  on ambiguous callee names; emits EXTRACTED edges when named import is unambiguous
- analyze.py: extend _BUILTIN_NOISE_LABELS with stdlib types and modules
- __main__.py: CLAUDE.md template uses MANDATORY language for graphify-first rule;
  PreToolUse hook message hardened to imperative; graphify export html auto-falls
  back to community-aggregation view when graph.json exceeds size cap
- tests/test_pg_introspect.py: add importorskip guard for tree_sitter_sql

Closes #1211, #1210, #1205, #1219, #1227; resolves discussion #1019

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 12:45:20 +01:00
Safi e477825a97 fix: three correctness bugs — cycle hang, label token budget, fuzzy dedup prefix merge
- analyze.py: pass length_bound=max_cycle_length to nx.simple_cycles() so
  networkx prunes during enumeration instead of post-filtering; drops report
  generation from never-returns to ~0.1s on dense graphs (#1196)

- llm.py: replace hardcoded min(40+16*n,4096) label_communities token budget
  with _resolve_max_tokens(min(64+24*n,8192)) — 24 tok/community covers 5-word
  JSON entries; 8192 cap fits 16k-context models; env var now honoured (#1200)

- dedup.py: add prefix-extension guard in Pass 2 and _llm_tiebreak — skip merge
  when one normalised label is a strict prefix of the other (getActiveSession /
  getActiveSessions, parseConfig / parseConfigFile). Option (a) rejected: dropping
  the >=12 early-out from _short_label_blocked breaks test_typo_merged (#1201)

- tests/test_dedup.py: two new regression tests verifying prefix guard fires for
  extension pairs and does not fire for same-length typo pairs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 23:49:16 +01:00
EirikWolf 7477b469ee feat: extra_body for custom providers + multi-batch label_communities (#1197)
Adds extra_body parameter support for custom/OpenAI-compat providers so users can pass provider-specific params (e.g. thinking budget for Claude via Bedrock compat). Adds multi-batch label_communities for 16k-context models — batches multiple community descriptions into a single LLM call instead of one per community. Partial batch failures are handled gracefully.

Co-authored-by: EirikWolf <EirikWolf@users.noreply.github.com>
2026-06-08 23:22:00 +01:00
Rasmus Bækgaard 29e57cd295 feat: add .slnx solution file support (#1189)
Adds support for the XML-based `.slnx` solution format (VS 2022 17.13+ replacement for `.sln`). Extracts project references as `contains` edges and build dependencies as `imports` edges. XXE-protected XML parsing with size cap. Wired into `_DISPATCH` and `CODE_EXTENSIONS`. 6 new tests passing.

Co-authored-by: bakgaard <bakgaard@users.noreply.github.com>
2026-06-08 10:35:32 +01:00
Safi 8a04560bf5 fix: remove non-spec trigger: field from skill frontmatter (#1180)
The Agent Skills spec only defines name, description, license,
compatibility, metadata, and allowed-tools as valid frontmatter fields.
The trigger: /graphify line was non-spec, silently ignored by spec-
following hosts, and flagged by agentskills validate CI checks.

- gen.py: removed trigger emission from _render_frontmatter; added
  _is_trigger_line() helper for roundtrip allow-list
- fragments/core/aider.md: removed hardcoded trigger: /graphify
- platforms.toml: removed trigger doc comment and trigger="" entries
- test_skillgen.py: replaced trigger-assertion tests with a single
  test asserting no host has trigger: in frontmatter
- Regenerated all 125 skill artifacts

Routing intent is preserved: the description field already contains
"treated as a graphify query first" and "graphify-out/ exists".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 15:49:52 +01:00
Safi 9e1ad425a9 feat: add CodeBuddy platform support (#1136)
Adds graphify skill installation for CodeBuddy (https://www.codebuddy.ai/).
CodeBuddy uses the same agent+hook mechanism as Claude Code.

- graphify codebuddy install — writes ~/.codebuddy/skills/graphify/SKILL.md
  and a CODEBUDDY.md always-on section
- graphify codebuddy uninstall — removes both cleanly
- graphify install --platform codebuddy — same as above
- Registers Bash + Read|Glob PreToolUse hooks in .codebuddy/settings.json
- Full install/uninstall roundtrip tests (35 tests)

Co-authored-by: studyzy <studyzy@gmail.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 14:34:13 +01:00
Safi 6a549e42d5 fix: four bugs — affected direction, hook root, glob fish/zsh, manifest drift (#1174 #1173 #1172 #1163)
#1174: affected.py load_graph now forces directed=True before
node_link_graph, matching the identical fix in serve.py and __main__.py.
Undirected graphs (directed:false in graph.json) were causing in_edges
to fall back to a direction-blind scan, missing true callers and
reporting false positives. Regression test added.

#1173: post-commit and post-checkout hook bodies now read
graphify-out/.graphify_root before calling _rebuild_code, falling back
to Path('.') if absent. A scoped build (graphify src/) no longer gets
silently expanded to the full repo on the next commit. Tests added.

#1172: Step 9 cleanup split into rm -f for fixed files and
find -maxdepth 1 -delete for the chunk glob. Under fish/zsh an
unmatched glob aborts the entire rm -f line, leaving temp files on disk.
Fixed in the three skillgen source fragments and regenerated.

#1163: detect_incremental type guard on stored mtime — if the manifest
contains a dict-valued mtime (schema drift from older versions), coerce
to None rather than propagating a non-numeric into comparisons.
Regression test added.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 12:40:44 +01:00
Safi a8dbbe59cf fix: land PRs #1170 #1169 #1165 (hooks, sensitive filter, score_nodes)
#1170 — replace nohup with cross-platform Python detach in git hooks.
Git for Windows MSYS has no nohup so post-commit/post-checkout hooks
silently failed. Now uses subprocess.Popen with DETACHED_PROCESS |
CREATE_NEW_PROCESS_GROUP on Windows, start_new_session=True on POSIX.
Quoting-safe (argv list). Fixes #1161.

#1169 — fix _is_sensitive false positives on topic-mentioning filenames.
token-economics-of-recall.md and password-policy-discussion.md were
silently dropped as secrets. Generic keywords (token/secret/password)
now only fire when the keyword ends the filename stem or the stem is
≤2 words. Specific patterns (.env/.pem/id_rsa etc.) remain unconditional.

#1165 — fix multi-word endpoint resolution in _score_nodes.
graphify path "AuthService" "UserRepo" never fired the exact-match bonus
because per-token comparison never equalled the full label. Now joins
normalized tokens and compares against the full label and its tokenized
form. O(1) per node, affects query_graph and shortest_path uniformly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 12:20:36 +01:00
Safi f146be3ea2 fix: numpy Python 3.13 pin + codex skill dir (#1154 #1160)
#1154: scope numpy>=2.0 constraint to python_version>='3.13' only.
numpy 1.26.4 ships no cp313 wheel so uv sync falls back to a source
build requiring a C compiler. The marker avoids forcing numpy 2.x on
3.10-3.12 users who have working 1.x environments.

#1160: codex platform skill now installs to .codex/skills/graphify/
instead of .agents/skills/graphify/. The hook already wrote to .codex/
so the skill destination was inconsistent. Propagates automatically
through install/uninstall (both read _PLATFORM_CONFIG dynamically).
Updated all codex-specific test assertions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 12:11:34 +01:00
Safi 7467c1b6a4 feat/fix: land PRs #1118 #1110 #1159 #1107 #1103 (graph quality + new features)
#1118 — prune stale AST nodes on full re-extraction (#1116)
Stamps every AST-extracted node with _origin="ast" in extract(). On a
full rebuild _rebuild_code drops any AST-marked node absent from the
fresh output even when its source file survives, fixing stale symbols.
Backward-compat: marker-less nodes from pre-1118 graphs survive one
cycle then self-heal.

#1110 — stop reading images and PDFs as garbage in headless extract
Images route through per-backend vision payloads (base64/data-URI/bytes
for claude/openai/bedrock); non-vision backends get _strip_pixels for
graceful degradation. PDFs reuse pypdf. 5MB cap, 20-image chunk limit.

#1159 — Salesforce Apex extractor (.cls, .trigger)
Regex-based extractor: classes, interfaces, enums, methods, triggers,
SOQL/DML edges. No new dependency. Dispatched as .cls and .trigger.

#1107 — Azure OpenAI Service backend (--backend azure)
Uses AzureOpenAI SDK client (from existing openai package). Auto-detects
when AZURE_OPENAI_API_KEY + AZURE_OPENAI_ENDPOINT both set. Uses
max_completion_tokens (not deprecated max_tokens).

#1103 — live PostgreSQL introspection (--postgres DSN)
graphify extract --postgres "postgresql://..." introspects tables, views,
routines, and FK relations via information_schema (SERIALIZABLE READ ONLY).
Credentials sanitized on error. New graphify[postgres] extra (psycopg3).

Union-resolved llm.py conflict: Azure functions + bedrock images= param.
Fixed test_image_vision.py mock to accept timeout= kwarg (our #1112).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 01:20:38 +01:00
Safi 2a683aac2e feat(serve): add Streamable HTTP transport to MCP server (#1143)
python -m graphify.serve graph.json --transport http --port 8080 serves
the same MCP tools over the Streamable HTTP transport (spec 2025-03-26)
so a single shared process can serve the graph for a whole team.

- _build_server() refactors server registration into a shared factory
  (stdio behavior is byte-for-byte unchanged — all 52 existing tests pass)
- _ApiKeyMiddleware: raw ASGI (not BaseHTTPMiddleware) preserves SSE
  streaming; constant-time compare; RFC-6750 case-insensitive Bearer;
  blank-key normalized to no-auth
- DNS-rebinding protection via TransportSecuritySettings; wildcard binds
  disable it and print an exposure warning when no api-key is set
- session_idle_timeout reaps idle stateful sessions (default 3600s) so a
  long-running shared server does not leak memory on client disconnect
- Dockerfile + .dockerignore for containerized team deployment
- 16 new tests via in-process ASGI test client (importorskip-guarded)
- stdio remains the default; no change for existing setups

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 00:43:40 +01:00
Safi f85339bcb1 fix(extract): don't require LLM API key for code-only corpus (#1122)
Backend resolution now defers until after file detection. A code-only
corpus (pure AST, zero LLM calls) runs without any API key.
Key validation only fires when needs_llm=True (semantic_files non-empty
or --dedup-llm passed). Error message now names why a key is needed and
notes that code-only corpora need none.

Applied from PR #1123 with one fix: _clear_backend_keys in tests now
also clears AWS_PROFILE/REGION, OLLAMA_BASE_URL to prevent CI flakes
on machines with ambient credentials.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 22:28:57 +01:00
Safi 3a4bdf54cc fix(install): route kiro install/uninstall through shared progressive helper (#1142)
_kiro_install was using a bare write_text that bypassed _copy_skill_file,
so the references/ sidecar and .graphify_version stamp were never written
despite kiro declaring skill_refs: "kiro". This left SKILL.md with 8 dead
references/*.md pointers on every install.

Fix: call _copy_skill_file("kiro", project=True) for the skill+sidecar+stamp,
keep the steering-file block inline. Uninstall now calls _remove_skill_file
which also cleans .graphify_version and references/.

Adds regression test asserting SKILL.md + references/ + .graphify_version +
no references.tmp + steering file, and that uninstall removes all of them.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 22:15:09 +01:00
Safi b368c0eb25 revert: remove importorskip now that uv.lock includes tree-sitter-hcl
CI installs --all-extras so all 8 Terraform tests run fully.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 21:46:25 +01:00
Safi 30f6f6fb85 fix(tests): skip Terraform tests when tree-sitter-hcl not installed
pytest.importorskip at module level matches the pattern used for other
optional extras (sql, dm) so CI passes without the optional dep.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 21:43:30 +01:00
Safi 200edecea5 feat(extract): add Terraform/HCL AST extraction via tree-sitter-hcl (#1129)
Adds extract_terraform() for .tf/.tfvars/.hcl files. Nodes: resources,
data sources, modules, variables, outputs, providers, locals. Edges:
contains, references (interpolation), depends_on. Node IDs are
directory-scoped for cross-file resolution. tree-sitter-hcl added as
optional extra (graphify[terraform]) matching sql/dm precedent.
8 tests including .tfvars documented no-op behaviour.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 21:38:56 +01:00
Safi 75c4de7759 honour GRAPHIFY_API_TIMEOUT in claude-cli and Anthropic SDK backends (#1112)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 10:45:01 +01:00
Safi 3a90ac2784 add query logging to ~/.cache/graphify-queries.log (#1128)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-03 21:04:01 +01:00
Dennis Cornwell 25df580061 fix: relativize manifest, .graphify_root, and cache source_file fields (#777)
Fixes #777. Relativize manifest keys, .graphify_root, and cache source_file fields on persist; re-anchor on load. In-memory callers still see absolute paths. Symlink round-trip fixed in follow-up commit 8f09326.
2026-06-03 21:03:06 +01:00
Safi 0fdfdedcaa harden hook interpreter detection against injection and unquoted exec
Per adversarial review of the #1127 fix:

- Apply the filesystem-path allowlist to sys.executable before embedding it in
  the generated hook script. Paths with metacharacters outside [a-zA-Z0-9/_.@:\-]
  (spaces, dollar signs, backticks, semicolons) are replaced with an empty string
  so the pinned probe is safely skipped rather than injecting shell commands that
  execute on every git commit.
- Change _PINNED='...' to single-quote assignment so no shell expansion can occur
  even if a character slipped through the allowlist (belt and suspenders).
- Quote $GRAPHIFY_PYTHON in both nohup exec lines so paths with spaces work
  (common in Windows C:\Program Files\... installs).
- Update test to assert the sanitized value, not raw sys.executable, so the test
  stays correct after any future sanitization changes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-03 15:40:06 +01:00
Safi 88bb1864aa fix hook install silent no-op for uv tool and pipx installs (#1127)
graphify hook install generated scripts that resolved the Python interpreter
purely at git-trigger time via 'command -v graphify'. GUI git clients (VS Code,
GitKraken), CI runners, and non-login shells often run with a minimal PATH that
omits ~/.local/bin -- the uv-tool / pipx launcher location. command -v graphify
returned empty, the python3/python fallbacks could not import graphify (it lives
only in the isolated venv), and the hook silently exited 0 with no output.

Fix: embed sys.executable of the currently-running install process as a pinned
first probe. Since 'graphify hook install' itself runs under the correct isolated
interpreter, sys.executable is always the right path. It is validated at
hook-runtime via 'import graphify' before use, so a stale pinned path safely
falls through to the existing dynamic detection rather than breaking the hook.

Also make the final fallback loud: print a diagnostic to stderr before exiting 0
so the failure is visible rather than invisible.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 15:40:06 +01:00
Safi cb5e701c67 make incremental extract tests deterministic under a real LLM env key
test_manifest_written_after_extract assumes a docs corpus fails with no LLM
backend, but _run inherited the developer's environment. With a real
ANTHROPIC_API_KEY exported and the anthropic package present (now pulled by
[all]), the claude backend was detected and the extract succeeded, breaking the
'no backend' assertion. Strip the backend-selecting env vars in _run so the
tests hold regardless of the developer's shell. CI has no keys set, so it was
green there already.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 00:20:39 +01:00
Safi 250834a0c4 add anthropic extra and fix backend-missing hints for uv tool installs
uv tool install graphifyy runs graphify in an isolated venv, so when a backend
needs anthropic/openai/boto3 the old 'Run: pip install anthropic' advice never
reached it. Worse, claude was the only backend with no [extra] at all, so a user
with ANTHROPIC_API_KEY set could not satisfy it without --with anthropic.

Add an anthropic optional extra (and include it in [all]), and replace the
package-missing ImportError messages with a shared _backend_pkg_hint that points
at 'uv tool install graphifyy[<extra>] --force' first, then the pip/venv path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 00:08:03 +01:00
Safi 5cc7ec8a23 close the Read-tool graph bypass with a Read/Glob PreToolUse hook
The Bash search hook only nudges grep/rg/find, so an agent that answers a
codebase question by Read-ing many source files one by one (the most common way
the graph gets skipped) slips right past it (#1114). Add _READ_SETTINGS_HOOK
matching Read|Glob: it fires only when graphify-out/graph.json exists, only for
a source/doc file outside graphify-out/, injects the same query-first
additionalContext, and never blocks (every branch fails open). Install and
uninstall now register and dedup both hooks idempotently.

Implemented independently rather than merging the community PR #1120; same idea,
our own hook (Read/Glob only, no fragile multi-file cat/head/tail heuristic).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 00:00:33 +01:00
Safi 9a298c5a32 write antigravity rules and workflows on project-scoped install
graphify install --project --platform antigravity went through the skill-only
branch (grouped with copilot/pi/kimi), so it copied SKILL.md but never wrote
.agents/rules/graphify.md or .agents/workflows/graphify.md - antigravity's
always-on layer - even though the project uninstall path removes them. Extract
the frontmatter + rules + workflows writes into _antigravity_finalize and call
it from both the global antigravity install and the project-scoped path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 23:48:54 +01:00
Safi d5d49f3764 add end-to-end test that gemini install leaves no dead reference pointer
Locks the body<->refs coupling: gemini ships claude's lean skill.md body but
resolves references through a separate path, so a real install with the real
claude bundle must leave every references/ pointer in SKILL.md resolvable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 22:50:33 +01:00
Safi 763b673350 tighten F2 and F3 from review: bounded decompression, ollama DNS + clean error
F2: replace the header-trust ratio check with an authoritative bounded
streaming-decompression pass. The zip central-directory sizes are
attacker-controlled, so a member that under-declares its size could dodge the
declared-size checks; now every member is stream-decompressed with a hard byte
ceiling, so actual expansion past the cap is caught regardless of the headers.
The cheap declared-size pre-filter stays as a fast reject for honest bombs.

F3: _validate_ollama_base_url now resolves the host, so an alias that points at
a link-local/metadata IP is blocked too, not just literal IPs. The extract
command gains an early gate that turns the metadata block into a clean
'error: ...' exit(2) instead of a deep traceback; a warn toggle keeps the
single user-facing LAN warning in the in-flow call.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 22:50:33 +01:00
Safi 46a1d4cabb harden ollama metadata egress and cpp path handling (F3, F5)
F3: OLLAMA_BASE_URL pointing at a link-local or cloud-metadata address
(169.254.x, metadata.google.*, 0.0.0.0) now fails closed instead of only
warning, since no real Ollama host lives there and it is a classic SSRF
target. General LAN hosts still warn-and-allow.

F5: the Fortran C-preprocessor call now passes an absolute path. cpp has no
'--' end-of-options terminator, so an attacker-named corpus file like
'-I/etc/x.F90' could otherwise be parsed as a cpp option; an absolute path
always begins with '/' and cannot be.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 22:23:15 +01:00
Safi c50ffc2285 cap untrusted office/PDF parsing to stop zip-bomb DoS (F2)
.docx/.xlsx are zip+XML containers parsed during a corpus scan with no size
guard, so a few-KB zip-bomb could decompress to gigabytes and OOM-kill the
process. Screen every office/PDF file before openpyxl/python-docx/pypdf touch
it: an on-disk size cap, a total-uncompressed cap, and a compression-ratio
check on the archive members. Files that exceed the caps are skipped (treated
as empty) rather than parsed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 22:19:17 +01:00
Safi e3993e4801 validate custom-provider base_url and gate project-local providers (F1)
A custom provider's base_url is where the full corpus and the user's API key
are sent. A project-local ./.graphify/providers.json travels with a cloned or
shared repo, so loading it silently was a corpus/key exfiltration channel.
Require GRAPHIFY_ALLOW_LOCAL_PROVIDERS=1 to load the project-local file (the
user's own ~/.graphify/providers.json stays trusted), reject non-http(s)
schemes on load and on 'provider add', and warn on plaintext-http egress. We
deliberately do not apply the ingest SSRF guard here so legitimate on-prem
https LLM gateways still work.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 22:16:58 +01:00
Safi 4a200159af ship the references sidecar for gemini installs
gemini has no _PLATFORM_CONFIG entry and installs claude's skill.md body
verbatim. Since #1121 that body is the lean progressive core that links to
references/, but _packaged_skill_refs_dir returned None for gemini, so the
install laid down a SKILL.md with 8 dead reference pointers. Resolve gemini
to the claude references bundle (the same body it already ships) and guard
it in test_claude_twins_ride_the_claude_bundle.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 22:10:08 +01:00
Safi 4806c1ee7e guard the last tomllib import in test_install_references for Python 3.10
Same 3.11+-stdlib issue as gen.py: this test imported tomllib directly, failing
the 3.10 job. Fall back to tomli on <3.11.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 21:36:53 +01:00
Safi 6137cdba43 harden _always_on against missing blocks and pin skillgen baselines
R1: the 6 always-on blocks were read into module-level constants at import,
so a missing/corrupt always_on/*.md crashed `import graphify.__main__` and
bricked every CLI command, not just install. Make _always_on lazy + lru_cached,
raising a clear "reinstall" error only on the install path that needs the block;
a module __getattr__ keeps the legacy constant names importable for the tests.
Verified: deleting a block no longer crashes `graphify --version`.

skillgen baselines: the self-check guards pinned to the moving `origin/v8` ref,
which stops pointing at the pre-split state once the split lands on v8 (the
always-on-roundtrip guard then fails in CI and monolith-roundtrip goes vacuous).
Pin _v8_baseline_ref, ALWAYS_ON_BASELINE_REF, and the two monolith roundtrip_refs
to the immutable pre-split commit SHA, matching the stated "does not track HEAD"
intent; update the 4 tests that asserted the old ref string.

R2: add tests/test_wheel_packaging.py - build the wheel and assert every
references bundle and always-on block ships in it, so a package-data glob miss
can't pass the repo-tree guards yet hard-exit `graphify install` for real users.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 21:08:11 +01:00
Soham Patankar fbe1e9977f feat(skills): progressive-disclosure split for all platforms (generator + drift fence) (#1121)
All-platform progressive-disclosure skill split + generator (addresses #1106).

Splits each platform's skill into a lean core (~615 lines, full default pipeline inline) + on-demand references/, generated from a single source via tools/skillgen with a CI/pre-commit drift gate. 13 hosts split, aider/devin stay monoliths. Also fixes the stale bare-path bugs across the previously hand-maintained variants and moves the always-on blocks into packaged markdown.

Verified: all 5 generator guards pass, byte-verbatim load-bearing slices, lean cores self-sufficient on the default path across all 13 split hosts, references gated to non-default branches, description preserves the graphify-out-query-first clause. Supersedes #1119 (Claude-first subset).

Known follow-up applied on top: harden _always_on() against a missing packaged file so a partial install can't brick the CLI.
2026-06-02 20:48:13 +01:00
Safi 1d4e11226f make tree-sitter-dm an optional extra to fix default install (fixes #1104)
tree-sitter-dm (BYOND DreamMaker) publishes only a Windows wheel, so on
Linux/macOS uv/pip compiled it from source and aborted the entire
`uv tool install graphifyy` when a C toolchain or python3-dev was missing.
It is the only core grammar lacking Linux/Mac wheels. Move it from core
dependencies to an optional `dm` extra (also in `all`); the default install
now needs no compiler. extract_dm already imports the grammar lazily with a
graceful fallback, and .dmi/.dmm/.dmf use no tree-sitter, so only .dm/.dme
AST parsing is gated behind the extra.

Also: guard the .dm grammar tests with pytest.importorskip so a plain
`uv sync` (no extras) doesn't hard-fail; bump to 0.8.28 with a changelog
upgrade note; switch the README extras table + type notes from pip to
`uv tool install` (uv is the recommended installer, and pip-into-a-uv-tool-env
is the exact ModuleNotFoundError the README already warns about). 0.8.28 also
ships Kilo Code (#512) and the Dart parser modernization (#1098).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 21:44:59 +01:00
Alexey Z ec3cb5eb3e feat(dart): modernize AST parser, support nested generics and part-of redirection (#1098)
Modernizes the Dart extractor: comment stripping, `part of` redirection, nested-generic-aware extends/with/implements parsing, generic type-argument mapping, and generic call detection.

Verified against current v8: merges cleanly, full suite 1530 passed, and inheritance edges connect to same-file class nodes with no ghost-node splits or dangling edges (consistent with the #1033/#1096 node-ID invariants).
2026-06-01 16:13:36 +01:00
alacasse a8005c218a feat: add Kilo Code support (#512)
Adds Kilo Code as a supported platform: native skill + /graphify command, install/uninstall, and a .kilo tool.execute.before plugin (mirroring the OpenCode integration).

JSONC config is handled non-destructively - existing .kilo/kilo.jsonc is read but never rewritten; automated plugin registration goes to kilo.json, preserving user comments (addresses the Qodo review flag).

Verified locally: full suite 1525 passed, install/uninstall smoke test works.
2026-06-01 16:05:07 +01:00
Safi 1dc5048b71 fold community labeling into llm.py instead of a standalone module
No behavior change. label_communities / generate_community_labels live next
to the backend dispatch (_call_llm, detect_backend) they use, so the lazy
import and separate file are no longer needed. Tests unchanged except the
import path; they still monkeypatch graphify.llm._call_llm.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 23:37:28 +01:00
Safi c8b329d8f2 auto-name communities with the configured backend in standalone CLI (fixes #1097)
Community labeling was an agent-only step (skill.md Step 5): inside Claude Code
or Gemini CLI the agent names communities itself. Run as a bare CLI
(python -m graphify extract . --backend X), no agent does Step 5, so labels
stayed Community 0/1/2 for every backend.

Add graphify/labeling.py: one batched _call_llm asking the backend for a
{cid: name} map from each community's top node labels (god nodes first),
with per-community placeholder fallback and graceful degradation (no backend,
API error, or malformed reply -> Community N, never crashes).

Wire it in:
- cluster-only auto-labels when no .graphify_labels.json exists (the reported
  repro path); --no-label opts out, --backend=<name> overrides auto-detect
- new `graphify label <path>` subcommand force-regenerates names
- extract prints a hint pointing standalone users at cluster-only

Tests mock the backend (no network): happy path, partial/fenced/malformed
replies, no-backend and error fallback, god_nodes dict shape.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-31 20:46:39 +01:00
Safi ad0c8c0a62 relativize symbol node IDs for root-level files to match spec (fixes #1096)
The #1033 remap relativized file node IDs but symbol IDs still embedded the
absolute parent-dir stem, so a root-level file's symbols became
<rootdir>_main_run while the file node was correctly main and the skill.md
spec (and semantic subagent) wants main_run -- splitting every top-level
file's symbols into AST/semantic ghost pairs.

Extend the remap chokepoint to also canonicalize the symbol stem prefix
(old _file_node_id(path) -> new _file_node_id(rel)), gated by source_file so
two files sharing a prefix cannot cross-contaminate. raw_calls.caller_nid is
rewritten too, since the cross-file call pass consumes it after the remap and
would otherwise dangle. No-op for nested files (immediate parent identical in
absolute and relative form).

The originally-filed root cause (facts-collector path form) was a stale-cache
red herring; the real bug is the symbol-level continuation of #1033.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-31 17:09:30 +01:00
Safi 88a8e3bc91 capture TS interface-extends and same-file class heritage (fixes #1095)
Two gaps in TypeScript inheritance:

1. interface heritage is an extends_type_clause node, not class_heritage,
   so the walker never saw it and interface extends produced no inherits
   edge. Add an extends_type_clause branch reusing _ts_heritage_clause_entries
   (handles multiple extends).

2. a same-file superclass has no import alias, so the use-fact resolver
   (which only consulted the import table) dropped it; only imported bases
   resolved. Add a same-file fallback against the file's own symbol_nodes,
   scoped to inherits/implements so it does not duplicate same-file calls
   that already resolve via the call-graph pass. Import resolution still
   takes precedence.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-31 16:47:52 +01:00
Safi 690b4e5d3e cap obsidian/canvas filenames to avoid ENAMETOOLONG on long labels (fixes #1094)
to_obsidian and to_canvas built note filenames from node labels with no
length cap, so a label >=255 bytes crashed write_text with OSError. Add a
shared _cap_filename helper that caps on UTF-8 bytes (not chars, so CJK
labels don't slip past) and appends an 8-char hash of the full label when
truncating, so two distinct labels sharing a long prefix stay distinct.
Both safe_name builders route node, community and canvas filenames through
it; wikilinks stay consistent because they read the same filename dict.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-31 16:41:18 +01:00
Safi c898dc62cd match AST file-level node IDs to the skill.md spec (fixes #1033)
AST file nodes were ID'd from the full relative path plus extension
(match_script_pipeline_step_py) while semantic subagents follow the
{parent_dir}_{stem} spec (script_pipeline_step), so every file split
into two disconnected ghost nodes.

Fix at the single remap chokepoint in extract(): file node IDs and all
edge endpoints already funnel through the #502 relative-path remap, so
changing that remap to emit _file_node_id (one parent dir, no extension)
converts the node and every referencing edge together - Python, TS, Lua,
C and bash import edges all stay connected. symbol_resolution pre-computes
the canonical form directly (bypassing the remap) so it is synced too.

Per-site conversion (as attempted in #1038/#1065) orphans edges because
it moves the node without the edge targets; the chokepoint approach
avoids that entirely.

Backward compat for existing graphs: graphify extract --force, as the
skill.md spec already documents.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-31 12:23:56 +01:00
Safi 0cf596aa8a fix test_no_incremental_without_manifest false positive
pytest derives tmp_path from the test name, so the path contained
"incremental" and the bare-word assertion always failed; check for
the actual incremental-mode phrases instead

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-31 00:10:19 +01:00
Manoj Mishra c066511bf2 feat: detect circular import dependencies at file level (#961)
* feat: detect circular import dependencies at file level

- Add find_import_cycles() to analyze.py
- Collapses symbol nodes to parent files, builds directed file graph
- Uses nx.simple_cycles() bounded by max_cycle_length (default 5)
- Deduplicates rotations, returns shortest cycles first
- Considers both imports_from and re_exports edges

Tested on a 976-file Next.js codebase: found 4 cycles including
a known utils↔barrel circular dependency and a 4-file API cycle.

* fix: resolve import-cycle merge blockers

- use source_file-only endpoint resolution (no label fallback)
- support Graph/DiGraph orientation via edge source_file
- return structured cycle records and include self-loops
- integrate Import Cycles section into GRAPH_REPORT.md
- expand cycle tests for real-schema IDs, undirected input,
  missing source_file nodes, and non-import relations
2026-05-30 21:37:47 +01:00
Safi cca13aa8fd fix anchored gitignore pattern leaking basename match into subtree (closes #1087)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-30 18:16:01 +01:00
Safi a9d6be6537 add custom LLM provider registry via providers.json (closes #1084)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-30 13:25:02 +01:00