The Bash search hook only nudges grep/rg/find, so an agent that answers a
codebase question by Read-ing many source files one by one (the most common way
the graph gets skipped) slips right past it (#1114). Add _READ_SETTINGS_HOOK
matching Read|Glob: it fires only when graphify-out/graph.json exists, only for
a source/doc file outside graphify-out/, injects the same query-first
additionalContext, and never blocks (every branch fails open). Install and
uninstall now register and dedup both hooks idempotently.
Implemented independently rather than merging the community PR #1120; same idea,
our own hook (Read/Glob only, no fragile multi-file cat/head/tail heuristic).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
graphify install --project --platform antigravity went through the skill-only
branch (grouped with copilot/pi/kimi), so it copied SKILL.md but never wrote
.agents/rules/graphify.md or .agents/workflows/graphify.md - antigravity's
always-on layer - even though the project uninstall path removes them. Extract
the frontmatter + rules + workflows writes into _antigravity_finalize and call
it from both the global antigravity install and the project-scoped path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Locks the body<->refs coupling: gemini ships claude's lean skill.md body but
resolves references through a separate path, so a real install with the real
claude bundle must leave every references/ pointer in SKILL.md resolvable.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
F2: replace the header-trust ratio check with an authoritative bounded
streaming-decompression pass. The zip central-directory sizes are
attacker-controlled, so a member that under-declares its size could dodge the
declared-size checks; now every member is stream-decompressed with a hard byte
ceiling, so actual expansion past the cap is caught regardless of the headers.
The cheap declared-size pre-filter stays as a fast reject for honest bombs.
F3: _validate_ollama_base_url now resolves the host, so an alias that points at
a link-local/metadata IP is blocked too, not just literal IPs. The extract
command gains an early gate that turns the metadata block into a clean
'error: ...' exit(2) instead of a deep traceback; a warn toggle keeps the
single user-facing LAN warning in the in-flow call.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
F3: OLLAMA_BASE_URL pointing at a link-local or cloud-metadata address
(169.254.x, metadata.google.*, 0.0.0.0) now fails closed instead of only
warning, since no real Ollama host lives there and it is a classic SSRF
target. General LAN hosts still warn-and-allow.
F5: the Fortran C-preprocessor call now passes an absolute path. cpp has no
'--' end-of-options terminator, so an attacker-named corpus file like
'-I/etc/x.F90' could otherwise be parsed as a cpp option; an absolute path
always begins with '/' and cannot be.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
.docx/.xlsx are zip+XML containers parsed during a corpus scan with no size
guard, so a few-KB zip-bomb could decompress to gigabytes and OOM-kill the
process. Screen every office/PDF file before openpyxl/python-docx/pypdf touch
it: an on-disk size cap, a total-uncompressed cap, and a compression-ratio
check on the archive members. Files that exceed the caps are skipped (treated
as empty) rather than parsed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A custom provider's base_url is where the full corpus and the user's API key
are sent. A project-local ./.graphify/providers.json travels with a cloned or
shared repo, so loading it silently was a corpus/key exfiltration channel.
Require GRAPHIFY_ALLOW_LOCAL_PROVIDERS=1 to load the project-local file (the
user's own ~/.graphify/providers.json stays trusted), reject non-http(s)
schemes on load and on 'provider add', and warn on plaintext-http egress. We
deliberately do not apply the ingest SSRF guard here so legitimate on-prem
https LLM gateways still work.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
gemini has no _PLATFORM_CONFIG entry and installs claude's skill.md body
verbatim. Since #1121 that body is the lean progressive core that links to
references/, but _packaged_skill_refs_dir returned None for gemini, so the
install laid down a SKILL.md with 8 dead reference pointers. Resolve gemini
to the claude references bundle (the same body it already ships) and guard
it in test_claude_twins_ride_the_claude_bundle.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Same 3.11+-stdlib issue as gen.py: this test imported tomllib directly, failing
the 3.10 job. Fall back to tomli on <3.11.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
R1: the 6 always-on blocks were read into module-level constants at import,
so a missing/corrupt always_on/*.md crashed `import graphify.__main__` and
bricked every CLI command, not just install. Make _always_on lazy + lru_cached,
raising a clear "reinstall" error only on the install path that needs the block;
a module __getattr__ keeps the legacy constant names importable for the tests.
Verified: deleting a block no longer crashes `graphify --version`.
skillgen baselines: the self-check guards pinned to the moving `origin/v8` ref,
which stops pointing at the pre-split state once the split lands on v8 (the
always-on-roundtrip guard then fails in CI and monolith-roundtrip goes vacuous).
Pin _v8_baseline_ref, ALWAYS_ON_BASELINE_REF, and the two monolith roundtrip_refs
to the immutable pre-split commit SHA, matching the stated "does not track HEAD"
intent; update the 4 tests that asserted the old ref string.
R2: add tests/test_wheel_packaging.py - build the wheel and assert every
references bundle and always-on block ships in it, so a package-data glob miss
can't pass the repo-tree guards yet hard-exit `graphify install` for real users.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All-platform progressive-disclosure skill split + generator (addresses #1106).
Splits each platform's skill into a lean core (~615 lines, full default pipeline inline) + on-demand references/, generated from a single source via tools/skillgen with a CI/pre-commit drift gate. 13 hosts split, aider/devin stay monoliths. Also fixes the stale bare-path bugs across the previously hand-maintained variants and moves the always-on blocks into packaged markdown.
Verified: all 5 generator guards pass, byte-verbatim load-bearing slices, lean cores self-sufficient on the default path across all 13 split hosts, references gated to non-default branches, description preserves the graphify-out-query-first clause. Supersedes #1119 (Claude-first subset).
Known follow-up applied on top: harden _always_on() against a missing packaged file so a partial install can't brick the CLI.
tree-sitter-dm (BYOND DreamMaker) publishes only a Windows wheel, so on
Linux/macOS uv/pip compiled it from source and aborted the entire
`uv tool install graphifyy` when a C toolchain or python3-dev was missing.
It is the only core grammar lacking Linux/Mac wheels. Move it from core
dependencies to an optional `dm` extra (also in `all`); the default install
now needs no compiler. extract_dm already imports the grammar lazily with a
graceful fallback, and .dmi/.dmm/.dmf use no tree-sitter, so only .dm/.dme
AST parsing is gated behind the extra.
Also: guard the .dm grammar tests with pytest.importorskip so a plain
`uv sync` (no extras) doesn't hard-fail; bump to 0.8.28 with a changelog
upgrade note; switch the README extras table + type notes from pip to
`uv tool install` (uv is the recommended installer, and pip-into-a-uv-tool-env
is the exact ModuleNotFoundError the README already warns about). 0.8.28 also
ships Kilo Code (#512) and the Dart parser modernization (#1098).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Modernizes the Dart extractor: comment stripping, `part of` redirection, nested-generic-aware extends/with/implements parsing, generic type-argument mapping, and generic call detection.
Verified against current v8: merges cleanly, full suite 1530 passed, and inheritance edges connect to same-file class nodes with no ghost-node splits or dangling edges (consistent with the #1033/#1096 node-ID invariants).
Adds Kilo Code as a supported platform: native skill + /graphify command, install/uninstall, and a .kilo tool.execute.before plugin (mirroring the OpenCode integration).
JSONC config is handled non-destructively - existing .kilo/kilo.jsonc is read but never rewritten; automated plugin registration goes to kilo.json, preserving user comments (addresses the Qodo review flag).
Verified locally: full suite 1525 passed, install/uninstall smoke test works.
No behavior change. label_communities / generate_community_labels live next
to the backend dispatch (_call_llm, detect_backend) they use, so the lazy
import and separate file are no longer needed. Tests unchanged except the
import path; they still monkeypatch graphify.llm._call_llm.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Community labeling was an agent-only step (skill.md Step 5): inside Claude Code
or Gemini CLI the agent names communities itself. Run as a bare CLI
(python -m graphify extract . --backend X), no agent does Step 5, so labels
stayed Community 0/1/2 for every backend.
Add graphify/labeling.py: one batched _call_llm asking the backend for a
{cid: name} map from each community's top node labels (god nodes first),
with per-community placeholder fallback and graceful degradation (no backend,
API error, or malformed reply -> Community N, never crashes).
Wire it in:
- cluster-only auto-labels when no .graphify_labels.json exists (the reported
repro path); --no-label opts out, --backend=<name> overrides auto-detect
- new `graphify label <path>` subcommand force-regenerates names
- extract prints a hint pointing standalone users at cluster-only
Tests mock the backend (no network): happy path, partial/fenced/malformed
replies, no-backend and error fallback, god_nodes dict shape.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The #1033 remap relativized file node IDs but symbol IDs still embedded the
absolute parent-dir stem, so a root-level file's symbols became
<rootdir>_main_run while the file node was correctly main and the skill.md
spec (and semantic subagent) wants main_run -- splitting every top-level
file's symbols into AST/semantic ghost pairs.
Extend the remap chokepoint to also canonicalize the symbol stem prefix
(old _file_node_id(path) -> new _file_node_id(rel)), gated by source_file so
two files sharing a prefix cannot cross-contaminate. raw_calls.caller_nid is
rewritten too, since the cross-file call pass consumes it after the remap and
would otherwise dangle. No-op for nested files (immediate parent identical in
absolute and relative form).
The originally-filed root cause (facts-collector path form) was a stale-cache
red herring; the real bug is the symbol-level continuation of #1033.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two gaps in TypeScript inheritance:
1. interface heritage is an extends_type_clause node, not class_heritage,
so the walker never saw it and interface extends produced no inherits
edge. Add an extends_type_clause branch reusing _ts_heritage_clause_entries
(handles multiple extends).
2. a same-file superclass has no import alias, so the use-fact resolver
(which only consulted the import table) dropped it; only imported bases
resolved. Add a same-file fallback against the file's own symbol_nodes,
scoped to inherits/implements so it does not duplicate same-file calls
that already resolve via the call-graph pass. Import resolution still
takes precedence.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
to_obsidian and to_canvas built note filenames from node labels with no
length cap, so a label >=255 bytes crashed write_text with OSError. Add a
shared _cap_filename helper that caps on UTF-8 bytes (not chars, so CJK
labels don't slip past) and appends an 8-char hash of the full label when
truncating, so two distinct labels sharing a long prefix stay distinct.
Both safe_name builders route node, community and canvas filenames through
it; wikilinks stay consistent because they read the same filename dict.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
AST file nodes were ID'd from the full relative path plus extension
(match_script_pipeline_step_py) while semantic subagents follow the
{parent_dir}_{stem} spec (script_pipeline_step), so every file split
into two disconnected ghost nodes.
Fix at the single remap chokepoint in extract(): file node IDs and all
edge endpoints already funnel through the #502 relative-path remap, so
changing that remap to emit _file_node_id (one parent dir, no extension)
converts the node and every referencing edge together - Python, TS, Lua,
C and bash import edges all stay connected. symbol_resolution pre-computes
the canonical form directly (bypassing the remap) so it is synced too.
Per-site conversion (as attempted in #1038/#1065) orphans edges because
it moves the node without the edge targets; the chokepoint approach
avoids that entirely.
Backward compat for existing graphs: graphify extract --force, as the
skill.md spec already documents.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
pytest derives tmp_path from the test name, so the path contained
"incremental" and the bare-word assertion always failed; check for
the actual incremental-mode phrases instead
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: detect circular import dependencies at file level
- Add find_import_cycles() to analyze.py
- Collapses symbol nodes to parent files, builds directed file graph
- Uses nx.simple_cycles() bounded by max_cycle_length (default 5)
- Deduplicates rotations, returns shortest cycles first
- Considers both imports_from and re_exports edges
Tested on a 976-file Next.js codebase: found 4 cycles including
a known utils↔barrel circular dependency and a 4-file API cycle.
* fix: resolve import-cycle merge blockers
- use source_file-only endpoint resolution (no label fallback)
- support Graph/DiGraph orientation via edge source_file
- return structured cycle records and include self-loops
- integrate Import Cycles section into GRAPH_REPORT.md
- expand cycle tests for real-schema IDs, undirected input,
missing source_file nodes, and non-import relations
Three compounding bugs caused ~30-50% of semantic chunks to come back
as 'hollow responses' on the claude-cli backend, triggering adaptive
bisection that doubled or tripled the number of subprocess calls.
Root causes
-----------
1. _parse_llm_json only stripped markdown fences when raw.startswith('```').
Claude frequently prepends a short preamble before the fence
('Here are the extracted entities:\n\n```json\n{...}```'), making
the check fail. json.loads then drops the chunk. Each bisected half
may exhibit the same failure, so cost compounds.
2. _call_claude_cli used --append-system-prompt, which layers graphify's
extraction prompt on top of Claude Code's default interactive-agent
prompt ('use markdown formatting', 'output text to communicate with
the user'). The conflicting instructions explain ~50% of the
preambles and fences from (1). Switching to --system-prompt (replace)
eliminates the conflict at the source.
3. claude-cli defaults to Opus, which is overkill for the structured
JSON extraction graphify performs. New GRAPHIFY_CLAUDE_CLI_MODEL env
var lets users opt into haiku / sonnet for big builds. Default
behaviour unchanged when the env var is unset.
Fix
---
- Robust _parse_llm_json: strips fences regardless of position, with a
balanced-brace fallback that scans for the first complete JSON object
in the response. Handles preambles, trailing prose, prose-wrapped
JSON without fences. Diagnostic log on terminal failure includes the
first 200 chars of the response.
- _call_claude_cli switches to --system-prompt.
- _call_claude_cli respects GRAPHIFY_CLAUDE_CLI_MODEL when set.
Tests (tests/test_llm_parser.py)
--------------------------------
- The four PR-body failure modes: preamble+fence, prose+JSON, raw JSON,
total refusal.
- Bonus: uppercase fence tag, unclosed fence, empty response.
- argv shape: --system-prompt present, --append-system-prompt absent.
- argv shape: --model added iff GRAPHIFY_CLAUDE_CLI_MODEL is set.
19/19 tests pass (9 pre-existing in test_claude_cli_backend.py +
10 new). Verified end-to-end on a 800-file repo: 0 hollow responses
after, vs ~30-50% before; output tokens -93%; wall time 44 min -> 4 min.
Adds graphify/mcp_ingest.py — extracts MCP server configurations into the
knowledge graph. Captures server nodes, NuGet/npm/pip package refs, commands,
env var requirements, and inter-server edges. Dispatched by filename before
the suffix lookup so generic .json extraction is unaffected. Env values are
discarded to prevent secret leakage. File size capped at 1 MiB. 29 tests.
Fixes: server_count budget now checked after validity guard so invalid entries
don't consume capacity; removed misleading uv run docstring example.
Co-Authored-By: adityachaudhary99 <adityachaudhary99@users.noreply.github.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
cluster-only re-runs Leiden clustering and then re-applies the existing
.graphify_labels.json by raw cid index, which causes labels to attach to
clusters whose members are unrelated to the label's original meaning
whenever the graph has changed between labeling and re-clustering.
Mirror the safety net already present in watch.py:_rebuild_code added in
#822 for the watch/update paths.
Adds a regression test that fails without the fix (label cids become
orphaned from graph.json community attributes after re-clustering).
Refs: #1027
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace inlined path normalisation with _norm_source_file (the same function
that builds node source_file keys) so prune_set and node attrs are normalised
identically. resolve() on root handles symlinked scan roots. Keep both raw and
normalised forms in prune_set so nodes with absolute source_file also match.
Add edge pruning and Windows backslash path tests per Opus review.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
prune_set in build_merge now includes relative-path variants of each deleted file
so manifest absolute paths (e.g. /home/user/corpus/module_b/utils.py) match graph
node source_file values (e.g. module_b/utils.py) regardless of OS or run context.
Fixes#1007.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Keep Graphify query segmentation focused on Chinese terms: rename the CJK helpers and extra to Chinese scope, cache the optional jieba import at module load, and keep a bigram fallback when jieba is unavailable.
Constraint: Reviewer asked either to broaden Hiragana/Katakana/Hangul support or rename CJK helpers; user chose Chinese-only because Japanese segmentation accuracy is uncertain.
Rejected: Broaden to Japanese and Korean segmentation | jieba is Chinese-oriented and the user explicitly limited scope to Chinese.
Confidence: high
Scope-risk: narrow
Directive: Do not label this path as CJK unless Hiragana/Katakana/Hangul segmentation is intentionally supported and tested.
Tested: uv run --with pytest pytest tests/test_serve.py tests/test_query_cli.py tests/test_benchmark.py
Tested: uv run --with pytest --with jieba pytest tests/test_serve.py -k "chinese or non_chinese"
Tested: graphify update .
Not-tested: Full test suite.
Co-authored-by: OmX <omx@oh-my-codex.dev>
TS 5.0 allows `"extends": ["./a", "./b"]`. _read_tsconfig_aliases called
`extends.startswith("@")` directly, so an array raised
`AttributeError: 'list' object has no attribute 'startswith'`. _safe_extract
caught it and skipped the WHOLE file — and since alias resolution fires on any
path-aliased import, every TS file using `@/`-style imports in a repo with
array-extends tsconfig was dropped, yielding an empty graph.
Normalize `extends` to a list (str -> [str], list kept, else []), process
entries in order (later overrides earlier, current config's paths override all
parents per TS semantics), and skip scoped npm configs (`@...`) per entry as
before.
Adds regression test test_tsconfig_array_extends_alias_resolves_existing_ts_file.
Safi