gemini has no _PLATFORM_CONFIG entry and installs claude's skill.md body
verbatim. Since #1121 that body is the lean progressive core that links to
references/, but _packaged_skill_refs_dir returned None for gemini, so the
install laid down a SKILL.md with 8 dead reference pointers. Resolve gemini
to the claude references bundle (the same body it already ships) and guard
it in test_claude_twins_ride_the_claude_bundle.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Same 3.11+-stdlib issue as gen.py: this test imported tomllib directly, failing
the 3.10 job. Fall back to tomli on <3.11.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
R1: the 6 always-on blocks were read into module-level constants at import,
so a missing/corrupt always_on/*.md crashed `import graphify.__main__` and
bricked every CLI command, not just install. Make _always_on lazy + lru_cached,
raising a clear "reinstall" error only on the install path that needs the block;
a module __getattr__ keeps the legacy constant names importable for the tests.
Verified: deleting a block no longer crashes `graphify --version`.
skillgen baselines: the self-check guards pinned to the moving `origin/v8` ref,
which stops pointing at the pre-split state once the split lands on v8 (the
always-on-roundtrip guard then fails in CI and monolith-roundtrip goes vacuous).
Pin _v8_baseline_ref, ALWAYS_ON_BASELINE_REF, and the two monolith roundtrip_refs
to the immutable pre-split commit SHA, matching the stated "does not track HEAD"
intent; update the 4 tests that asserted the old ref string.
R2: add tests/test_wheel_packaging.py - build the wheel and assert every
references bundle and always-on block ships in it, so a package-data glob miss
can't pass the repo-tree guards yet hard-exit `graphify install` for real users.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All-platform progressive-disclosure skill split + generator (addresses #1106).
Splits each platform's skill into a lean core (~615 lines, full default pipeline inline) + on-demand references/, generated from a single source via tools/skillgen with a CI/pre-commit drift gate. 13 hosts split, aider/devin stay monoliths. Also fixes the stale bare-path bugs across the previously hand-maintained variants and moves the always-on blocks into packaged markdown.
Verified: all 5 generator guards pass, byte-verbatim load-bearing slices, lean cores self-sufficient on the default path across all 13 split hosts, references gated to non-default branches, description preserves the graphify-out-query-first clause. Supersedes #1119 (Claude-first subset).
Known follow-up applied on top: harden _always_on() against a missing packaged file so a partial install can't brick the CLI.
tree-sitter-dm (BYOND DreamMaker) publishes only a Windows wheel, so on
Linux/macOS uv/pip compiled it from source and aborted the entire
`uv tool install graphifyy` when a C toolchain or python3-dev was missing.
It is the only core grammar lacking Linux/Mac wheels. Move it from core
dependencies to an optional `dm` extra (also in `all`); the default install
now needs no compiler. extract_dm already imports the grammar lazily with a
graceful fallback, and .dmi/.dmm/.dmf use no tree-sitter, so only .dm/.dme
AST parsing is gated behind the extra.
Also: guard the .dm grammar tests with pytest.importorskip so a plain
`uv sync` (no extras) doesn't hard-fail; bump to 0.8.28 with a changelog
upgrade note; switch the README extras table + type notes from pip to
`uv tool install` (uv is the recommended installer, and pip-into-a-uv-tool-env
is the exact ModuleNotFoundError the README already warns about). 0.8.28 also
ships Kilo Code (#512) and the Dart parser modernization (#1098).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Modernizes the Dart extractor: comment stripping, `part of` redirection, nested-generic-aware extends/with/implements parsing, generic type-argument mapping, and generic call detection.
Verified against current v8: merges cleanly, full suite 1530 passed, and inheritance edges connect to same-file class nodes with no ghost-node splits or dangling edges (consistent with the #1033/#1096 node-ID invariants).
Adds Kilo Code as a supported platform: native skill + /graphify command, install/uninstall, and a .kilo tool.execute.before plugin (mirroring the OpenCode integration).
JSONC config is handled non-destructively - existing .kilo/kilo.jsonc is read but never rewritten; automated plugin registration goes to kilo.json, preserving user comments (addresses the Qodo review flag).
Verified locally: full suite 1525 passed, install/uninstall smoke test works.
No behavior change. label_communities / generate_community_labels live next
to the backend dispatch (_call_llm, detect_backend) they use, so the lazy
import and separate file are no longer needed. Tests unchanged except the
import path; they still monkeypatch graphify.llm._call_llm.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Community labeling was an agent-only step (skill.md Step 5): inside Claude Code
or Gemini CLI the agent names communities itself. Run as a bare CLI
(python -m graphify extract . --backend X), no agent does Step 5, so labels
stayed Community 0/1/2 for every backend.
Add graphify/labeling.py: one batched _call_llm asking the backend for a
{cid: name} map from each community's top node labels (god nodes first),
with per-community placeholder fallback and graceful degradation (no backend,
API error, or malformed reply -> Community N, never crashes).
Wire it in:
- cluster-only auto-labels when no .graphify_labels.json exists (the reported
repro path); --no-label opts out, --backend=<name> overrides auto-detect
- new `graphify label <path>` subcommand force-regenerates names
- extract prints a hint pointing standalone users at cluster-only
Tests mock the backend (no network): happy path, partial/fenced/malformed
replies, no-backend and error fallback, god_nodes dict shape.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The #1033 remap relativized file node IDs but symbol IDs still embedded the
absolute parent-dir stem, so a root-level file's symbols became
<rootdir>_main_run while the file node was correctly main and the skill.md
spec (and semantic subagent) wants main_run -- splitting every top-level
file's symbols into AST/semantic ghost pairs.
Extend the remap chokepoint to also canonicalize the symbol stem prefix
(old _file_node_id(path) -> new _file_node_id(rel)), gated by source_file so
two files sharing a prefix cannot cross-contaminate. raw_calls.caller_nid is
rewritten too, since the cross-file call pass consumes it after the remap and
would otherwise dangle. No-op for nested files (immediate parent identical in
absolute and relative form).
The originally-filed root cause (facts-collector path form) was a stale-cache
red herring; the real bug is the symbol-level continuation of #1033.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two gaps in TypeScript inheritance:
1. interface heritage is an extends_type_clause node, not class_heritage,
so the walker never saw it and interface extends produced no inherits
edge. Add an extends_type_clause branch reusing _ts_heritage_clause_entries
(handles multiple extends).
2. a same-file superclass has no import alias, so the use-fact resolver
(which only consulted the import table) dropped it; only imported bases
resolved. Add a same-file fallback against the file's own symbol_nodes,
scoped to inherits/implements so it does not duplicate same-file calls
that already resolve via the call-graph pass. Import resolution still
takes precedence.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
to_obsidian and to_canvas built note filenames from node labels with no
length cap, so a label >=255 bytes crashed write_text with OSError. Add a
shared _cap_filename helper that caps on UTF-8 bytes (not chars, so CJK
labels don't slip past) and appends an 8-char hash of the full label when
truncating, so two distinct labels sharing a long prefix stay distinct.
Both safe_name builders route node, community and canvas filenames through
it; wikilinks stay consistent because they read the same filename dict.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
AST file nodes were ID'd from the full relative path plus extension
(match_script_pipeline_step_py) while semantic subagents follow the
{parent_dir}_{stem} spec (script_pipeline_step), so every file split
into two disconnected ghost nodes.
Fix at the single remap chokepoint in extract(): file node IDs and all
edge endpoints already funnel through the #502 relative-path remap, so
changing that remap to emit _file_node_id (one parent dir, no extension)
converts the node and every referencing edge together - Python, TS, Lua,
C and bash import edges all stay connected. symbol_resolution pre-computes
the canonical form directly (bypassing the remap) so it is synced too.
Per-site conversion (as attempted in #1038/#1065) orphans edges because
it moves the node without the edge targets; the chokepoint approach
avoids that entirely.
Backward compat for existing graphs: graphify extract --force, as the
skill.md spec already documents.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
pytest derives tmp_path from the test name, so the path contained
"incremental" and the bare-word assertion always failed; check for
the actual incremental-mode phrases instead
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: detect circular import dependencies at file level
- Add find_import_cycles() to analyze.py
- Collapses symbol nodes to parent files, builds directed file graph
- Uses nx.simple_cycles() bounded by max_cycle_length (default 5)
- Deduplicates rotations, returns shortest cycles first
- Considers both imports_from and re_exports edges
Tested on a 976-file Next.js codebase: found 4 cycles including
a known utils↔barrel circular dependency and a 4-file API cycle.
* fix: resolve import-cycle merge blockers
- use source_file-only endpoint resolution (no label fallback)
- support Graph/DiGraph orientation via edge source_file
- return structured cycle records and include self-loops
- integrate Import Cycles section into GRAPH_REPORT.md
- expand cycle tests for real-schema IDs, undirected input,
missing source_file nodes, and non-import relations
Three compounding bugs caused ~30-50% of semantic chunks to come back
as 'hollow responses' on the claude-cli backend, triggering adaptive
bisection that doubled or tripled the number of subprocess calls.
Root causes
-----------
1. _parse_llm_json only stripped markdown fences when raw.startswith('```').
Claude frequently prepends a short preamble before the fence
('Here are the extracted entities:\n\n```json\n{...}```'), making
the check fail. json.loads then drops the chunk. Each bisected half
may exhibit the same failure, so cost compounds.
2. _call_claude_cli used --append-system-prompt, which layers graphify's
extraction prompt on top of Claude Code's default interactive-agent
prompt ('use markdown formatting', 'output text to communicate with
the user'). The conflicting instructions explain ~50% of the
preambles and fences from (1). Switching to --system-prompt (replace)
eliminates the conflict at the source.
3. claude-cli defaults to Opus, which is overkill for the structured
JSON extraction graphify performs. New GRAPHIFY_CLAUDE_CLI_MODEL env
var lets users opt into haiku / sonnet for big builds. Default
behaviour unchanged when the env var is unset.
Fix
---
- Robust _parse_llm_json: strips fences regardless of position, with a
balanced-brace fallback that scans for the first complete JSON object
in the response. Handles preambles, trailing prose, prose-wrapped
JSON without fences. Diagnostic log on terminal failure includes the
first 200 chars of the response.
- _call_claude_cli switches to --system-prompt.
- _call_claude_cli respects GRAPHIFY_CLAUDE_CLI_MODEL when set.
Tests (tests/test_llm_parser.py)
--------------------------------
- The four PR-body failure modes: preamble+fence, prose+JSON, raw JSON,
total refusal.
- Bonus: uppercase fence tag, unclosed fence, empty response.
- argv shape: --system-prompt present, --append-system-prompt absent.
- argv shape: --model added iff GRAPHIFY_CLAUDE_CLI_MODEL is set.
19/19 tests pass (9 pre-existing in test_claude_cli_backend.py +
10 new). Verified end-to-end on a 800-file repo: 0 hollow responses
after, vs ~30-50% before; output tokens -93%; wall time 44 min -> 4 min.
Adds graphify/mcp_ingest.py — extracts MCP server configurations into the
knowledge graph. Captures server nodes, NuGet/npm/pip package refs, commands,
env var requirements, and inter-server edges. Dispatched by filename before
the suffix lookup so generic .json extraction is unaffected. Env values are
discarded to prevent secret leakage. File size capped at 1 MiB. 29 tests.
Fixes: server_count budget now checked after validity guard so invalid entries
don't consume capacity; removed misleading uv run docstring example.
Co-Authored-By: adityachaudhary99 <adityachaudhary99@users.noreply.github.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
cluster-only re-runs Leiden clustering and then re-applies the existing
.graphify_labels.json by raw cid index, which causes labels to attach to
clusters whose members are unrelated to the label's original meaning
whenever the graph has changed between labeling and re-clustering.
Mirror the safety net already present in watch.py:_rebuild_code added in
#822 for the watch/update paths.
Adds a regression test that fails without the fix (label cids become
orphaned from graph.json community attributes after re-clustering).
Refs: #1027
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace inlined path normalisation with _norm_source_file (the same function
that builds node source_file keys) so prune_set and node attrs are normalised
identically. resolve() on root handles symlinked scan roots. Keep both raw and
normalised forms in prune_set so nodes with absolute source_file also match.
Add edge pruning and Windows backslash path tests per Opus review.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
prune_set in build_merge now includes relative-path variants of each deleted file
so manifest absolute paths (e.g. /home/user/corpus/module_b/utils.py) match graph
node source_file values (e.g. module_b/utils.py) regardless of OS or run context.
Fixes#1007.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Keep Graphify query segmentation focused on Chinese terms: rename the CJK helpers and extra to Chinese scope, cache the optional jieba import at module load, and keep a bigram fallback when jieba is unavailable.
Constraint: Reviewer asked either to broaden Hiragana/Katakana/Hangul support or rename CJK helpers; user chose Chinese-only because Japanese segmentation accuracy is uncertain.
Rejected: Broaden to Japanese and Korean segmentation | jieba is Chinese-oriented and the user explicitly limited scope to Chinese.
Confidence: high
Scope-risk: narrow
Directive: Do not label this path as CJK unless Hiragana/Katakana/Hangul segmentation is intentionally supported and tested.
Tested: uv run --with pytest pytest tests/test_serve.py tests/test_query_cli.py tests/test_benchmark.py
Tested: uv run --with pytest --with jieba pytest tests/test_serve.py -k "chinese or non_chinese"
Tested: graphify update .
Not-tested: Full test suite.
Co-authored-by: OmX <omx@oh-my-codex.dev>
TS 5.0 allows `"extends": ["./a", "./b"]`. _read_tsconfig_aliases called
`extends.startswith("@")` directly, so an array raised
`AttributeError: 'list' object has no attribute 'startswith'`. _safe_extract
caught it and skipped the WHOLE file — and since alias resolution fires on any
path-aliased import, every TS file using `@/`-style imports in a repo with
array-extends tsconfig was dropped, yielding an empty graph.
Normalize `extends` to a list (str -> [str], list kept, else []), process
entries in order (later overrides earlier, current config's paths override all
parents per TS semantics), and skip scoped npm configs (`@...`) per entry as
before.
Adds regression test test_tsconfig_array_extends_alias_resolves_existing_ts_file.
`graphify export html|obsidian|wiki|svg|graphml|neo4j` reads `communities`
exclusively from `.graphify_analysis.json` (set to `{}` if missing). The
post-commit / watch rebuild path doesn't regenerate that sidecar — only
graph.json + GRAPH_REPORT.md. Several skill workflows also delete temp
files at the end of `graphify extract`. In both cases the per-node
`community` attribute (`to_json` writes it on every node) is intact, but
the CLI ignores it.
Observed failure: `graphify export html` on a graph that exceeds the
viz node limit prints
Graph has 64703 nodes (above 5000 limit). Building aggregated community view...
Single community - aggregated view not useful. Skipping graph.html.
even though the same graph.json has 2,026 distinct `community` values
on its nodes — `to_html` just received an empty `communities` dict and
the aggregator collapsed to a single meta-node.
Fix: when the analysis sidecar is absent (or its `communities` field is
empty), reconstruct the `cid -> [node_ids]` mapping from the per-node
attribute in graph.json. The sidecar remains the canonical source of
truth when present; the reconstruction is a strict fallback. Every
downstream subcommand (`html`, `obsidian`, `wiki`, `svg`, `graphml`,
`neo4j`) sees the same shape it always did, just populated from the
graph itself instead of an externally-cached sidecar.
Tests added (tests/test_cli_export.py):
- `test_export_html_falls_back_to_node_community_attribute` —
delete the sidecar, run export html, confirm `graph.html` exists
and the "Single community" bail-out path does NOT fire.
- `test_export_html_fallback_recovers_multiple_communities` —
stronger guarantee that the reconstructed community count equals
what the sidecar would have provided (no silent data loss).
- `test_export_html_no_community_data_at_all_still_succeeds` —
hand-build a graph.json with no per-node `community` attribute
(older `to_json` versions, manually-constructed graphs); the
command must still exit cleanly rather than crash.
All 26 tests in test_cli_export.py pass; ruff clean on both files.
The post-commit hook passes `git diff --name-only HEAD~1 HEAD` as
changed_paths to `_rebuild_code`. That list includes deletions, and
`_rebuild_code` correctly identifies them (lines 352-367 in watch.py)
and evicts the stale nodes from the preserved set. The rebuilt graph
is intentionally smaller.
`_check_shrink` then refuses to overwrite the existing graph.json
because it only sees the node-count delta, not the cause. The guard
fires with "Refusing to overwrite — you may be missing chunk files
from a previous session. Pass --force to override."
Result: every commit that deletes a tracked file silently leaves stale
nodes in graph.json. The user must either pass --force (which also
disables the guard for legitimate failure modes) or manually re-run
`graphify update . --force` after delete-heavy commits.
Fix: thread a `had_explicit_deletions` flag from `_rebuild_code` into
`_check_shrink`. When the caller has declared the deletions, the
smaller graph is the expected outcome and the guard is skipped. The
guard remains intact for SILENT shrinkage — its actual purpose —
from failed semantic chunks or corrupted runs.
The fix is opt-in by design: callers that don't pass `changed_paths`
(e.g. the post-checkout full rebuild path) keep the old conservative
behavior. Only paths that explicitly track deletions get the bypass.
Tests added (tests/test_watch.py):
- `test_check_shrink_blocks_silent_shrink` — pre-existing behavior intact
- `test_check_shrink_allows_force_override` — pre-existing behavior intact
- `test_check_shrink_allows_explicit_deletions` — new: deletion bypass
- `test_check_shrink_allows_no_existing_data` — first-run case
- `test_check_shrink_allows_growth` — sanity
- `test_check_shrink_unlinks_tmp_on_refuse` — cleanup on refusal
- `test_check_shrink_keeps_tmp_when_deletions_declared` — no spurious unlink
- `test_rebuild_code_prunes_deleted_file_nodes` — end-to-end probe of the
exact scenario the post-commit hook triggers (git init, build, delete
one file, re-run with the deleted path in changed_paths, verify the
graph shrinks and the surviving file's nodes are preserved)
All 8 new tests pass; full test_watch.py + test_build.py + test_export.py
(74 tests) pass with no regressions.
If `graphify extract --backend claude` runs without the `anthropic`
package installed (pip install graphifyy doesn't pull it in), every
semantic chunk fails inside extract_corpus_parallel. The per-chunk
errors print to stderr but the function returns the empty merged
accumulator anyway, so extract proceeds to write an AST-only graph.json
and exit 0. CI that checks exit status sees success even though the
requested semantic pass produced no nodes.
Track per-chunk success via the existing on_chunk_done callback, which
only fires after a chunk succeeds. If fresh extraction was requested
(uncached_paths non-empty) and zero chunks completed, abort before the
merge/cluster/write phase with exit 1 and a message naming the backend.
The same shape covers other backends with optional SDK deps (openai,
google-generativeai). Cached-only runs are unaffected: uncached_paths
is empty and the guard does not fire.
Tests in tests/test_extract_cli.py simulate the all-failed and
one-succeeded paths by patching extract_corpus_parallel directly.
- Add 'export_statement' to import_types for JS/TS/TSX configs
- Extend _import_js to detect 'export { X } from ./mod' re-exports
- Emit 're_exports' edges linking barrel files to source symbols
- Preserve walk-through for 'export function/const' declarations
- Add 're_exports' to clean_edges allowlist for cross-file edges
Tested on a 976-file Next.js codebase: detects 162 re_exports edges
and 5760 symbol-level imports (previously 0 for both).
