`graphify extract --backend <gemini|claude|claude-cli|openai|kimi|...>` produced
zero hyperedges for any corpus: llm._EXTRACTION_SYSTEM only showed
"hyperedges":[] in its output schema and never described what a hyperedge is, so
every model returned the empty array. Meanwhile the agent/skill path, whose
references/extraction-spec.md fully documents hyperedges ("3 or more nodes
participate together..."), produced them — the two prompts had drifted.
Bring the native prompt in line with the skill spec: add the hyperedge
instruction and a populated schema example. The parse/merge side already handled
hyperedges, so this is prompt-only. Verified with a real claude-cli run — a doc
that previously yielded 0 hyperedges now yields one, correctly relativized (#1418).
Adds two guard tests: the native prompt must request hyperedges with a populated
example, and it must share the skill spec's hyperedge wording so they can't drift
apart again.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The GRAPHIFY_OUT override (custom output-dir name / absolute path, #686) was only
respected by some readers. `graphify extract` and several commands hardcoded the
literal "graphify-out", so `GRAPHIFY_OUT=custom-out graphify extract` still wrote
to graphify-out/ and downstream query/serve/update looked in the wrong place.
Resolve the output-dir name through graphify.paths everywhere it matters:
- new graphify.paths.out_path()/default_graph_json() helpers
- __main__: extract write dir, cluster-only/label, query/affected/benchmark
defaults, save-result --memory-dir, uninstall --purge, cache-check
- detect: _MANIFEST_PATH, memory/ + converted/ dirs, and the scan-exclude (a
renamed output dir is no longer re-ingested as source input)
- transcribe._TRANSCRIPTS_DIR; build_merge/serve/benchmark/prs graph-path defaults
Default behaviour is unchanged: with no env var everything still uses graphify-out/.
Verified end-to-end (extract -> cluster-only -> query under GRAPHIFY_OUT=custom-out
writes/reads custom-out/, no stray graphify-out/) and added a CLI regression test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The split-skill runbook passed '.' as report.generate's root argument in
Steps 4 and 5, so `/graphify /some/path` produced a report titled
"# Graph Report - ." regardless of the scanned directory. It now passes
'INPUT_PATH' (the Aider/Devin monoliths were already correct). Display-only:
no path written to graph.json or manifest.json was affected. Regenerated +
blessed the split-skill artifacts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The skill runbooks called save_manifest(...) with no root=, so manifest keys
were stored as absolute paths (e.g. /Users/.../main.go). Cloning or moving the
repo then broke `graphify --update`: detect_incremental matched none of the
cached keys, so the entire corpus re-extracted (and the report showed ghost
nodes). The native `graphify extract`/`update` CLI already passed root=target;
the agent-executed runbooks did not.
All four runbook call sites now pass root='INPUT_PATH', relativizing manifest
keys to the scan root (portable forward-slash form, per save_manifest's #777
support): the lean-core skill.md Step 9, the shared --update reference, and the
Aider/Devin monoliths.
The monolith edit is registered as a new sanctioned change-class
(_is_manifest_root_fix_line) in the round-trip guard, mirroring how the #1392
runbook fixes were sanctioned. Regenerated + blessed all artifacts; added a
regression test asserting every shipped runbook threads root= into save_manifest.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#1418: build_from_json relativized source_file on nodes and edges but stored
graph.hyperedges[] verbatim, so a semantic subagent's absolute path leaked into
graph.json. Relativize hyperedges in build_from_json (to_json has no root to
relativize against), mirroring the existing node/edge handling.
#1423: consolidate the GRAPHIFY_OUT output-dir name into a single graphify.paths
module (was duplicated in __main__, cache, watch) and route the path guards
through it — security.validate_graph_path's base=None discovery + fallback,
callflow_html's project-root resolution, and the post-commit/post-checkout hook
bodies (which now read the env var at hook-run time). A renamed output dir is no
longer validated against the wrong base or missed by the hook.
Tests: hyperedge relativization (test_hypergraph), GRAPHIFY_OUT discovery
(test_security), updated the hook-body contract assertion (test_hooks).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The monoliths are hand-maintained single files frozen against a pinned pristine-v8 blob by the round-trip guard, so they were excluded from the 0.8.44 #1392 batch. Evolve the guard from a positional zip (line-count-exact, single-line-class allowlist) to a multiset diff that classifies every added/removed line against documented sanctioned change-classes, so the multi-line fixes can land while any unsanctioned drift still fails. Add predicates for the four fix classes and broaden the enum/chunk-cleanup predicates to match both the v8 and rewritten forms.
Both monoliths now: thread directed=IS_DIRECTED through every build_from_json call (a --directed run no longer collapses reciprocal edges), scope semantic extraction to document/paper/image, unlink a stale .graphify_cached.json on a cache miss, and run Step 4's zero-node guard before any write with the report/analysis gated on to_json persisting the graph (#1392).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes the non-crash tier of #1392 in the Claude-path skill fragments:
- #6/#7: build_from_json/build_merge now take directed=IS_DIRECTED in Step 4, Step 5 rebuild, and the --update merge/diff, with prose telling the agent to substitute IS_DIRECTED like INPUT_PATH (a --directed run no longer silently rebuilds undirected and collapses reciprocal edges)
- #10: semantic extraction only flattens document/paper/image, not code (AST already covers code) so subagents stop re-reading every source file
- #12: .graphify_cached.json is deleted on a cache miss so Part C never merges a stale cache from a prior run
- #11: --update now transcribes changed video files and moves transcripts to documents before the semantic pipeline
- #4/#5/#23: transcribe writes via write_text (no shell redirect), uses GRAPHIFY_WHISPER_MODEL/PROMPT env, status to stderr
- #2/#3: add-watch and exports use $(cat graphify-out/.graphify_python) explicitly; MCP Desktop config documents the absolute interpreter path
- #21: extraction-spec example id namespaced (auth_session_validatetoken)
- #22: query term split keeps tokens >= 3 chars
aider/devin monoliths are pinned by the roundtrip invariant and excluded; their own Step 1 already instructs replacing python3 with the resolved interpreter.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Five fixes in the skillgen fragments (re-rendered): chunk paths now derive from
cwd to match where Part C globs (a non-cwd scan produced "no nodes"); the
code-only fast path writes an empty .graphify_semantic.json so Part C doesn't
FileNotFoundError; --cluster-only relies on the self-contained CLI instead of
re-running Steps 5-9 against deleted intermediate files; and Step 4 runs the
zero-node guard before any write and only writes GRAPH_REPORT.md / analysis when
to_json actually persisted the graph (honoring the #479 shrink-guard).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 0.8.43 str-path coercion (#1386) ran Path(f) over every item in
extract_files_direct, but extract_corpus_parallel feeds it FileSlice units from
the oversized-doc slicing (#1369), and Path(FileSlice) raises TypeError -- so
semantic extraction of any Markdown file larger than _FILE_CHAR_CAP crashed.
Coerce only non-Path/non-FileSlice entries. The #1386 tests used small files, so
slicing never ran; added a regression test with a file that actually slices.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
apm.yml was a .yml document handled by the LLM, so the same package got a
different file-anchored node id from its own manifest than from each dependent's
dependency reference and split into duplicate nodes. New manifest_ingest module
parses apm.yml/pyproject.toml/go.mod/pom.xml deterministically into ONE package
node per package, keyed by name via ids.make_id, plus depends_on edges; routed
to the AST path (CODE) so the LLM never sees them. Package nodes are exempt from
the file-stem prefix remap so the canonical id is stable across manifests and
dedup collapses references to a single hub node.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On WSL/POSIX, Path("C:\\...").is_absolute() is False, so a Windows absolute
core.hooksPath (or rev-parse --git-path output) was joined under the repo root
and mkdir'd as a literal backslash-named junk directory while install reported
success and the real .git/hooks got nothing. Both branches of _hooks_dir now
reject drive-letter / backslash paths with a clear error so the failure is loud.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
extract_markdown only emitted heading nodes + contains edges and never parsed
link syntax, so a doc full of [text](./other.md) links (index.md,
table-of-contents.md) had no edges to the docs it links and never became a hub.
Add a deterministic link pass: inline, reference-style, and [[wikilinks]],
resolved relative to the source file, external URLs/anchors/images skipped, with
the target id built via the same _make_id recipe so the edge merges onto the
real doc node instead of an orphan.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
extract_corpus_parallel and extract_files_direct are typed list[Path] but
crashed with AttributeError on str paths (f.suffix in slicing/partition, f.parent
in packing). Coerce files = [Path(f) for f in files] at both public entry points;
the AST extract() already coerced.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
extract._make_id and build._normalize_id were copy-pasted forks of the same
NFKC/casefold recipe kept in sync by mirrored docstrings -- the root of the
recurring ID-drift ghost-node bug class (#811/#550/#1033/#1104). Move the recipe
to graphify/ids.py and have all four producers delegate to it: extract, build,
and (completing the migration) mcp_ingest and symbol_resolution, whose
"avoid an import cycle" copies are moot now that ids.py is dependency-free. The
contract test asserts all four resolve to the shared recipe, with hypothesis
property tests for make_id == normalize_id and idempotency.
Co-Authored-By: danielnguyenfinhub <danielnguyenfinhub@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
label_communities logged-and-skipped any batch whose LLM response was malformed
JSON, silently losing ~100 names per failed batch on large graphs. Split the
batch at the midpoint and retry each half (smaller prompt -> smaller output),
mirroring _extract_with_adaptive_retry; the base case re-raises so the caller
skips just that batch. Removed leftover scaffolding and corrected the docstring.
Co-Authored-By: CJdev232 <CJdev232@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pypdf 6.11.0->6.13.3 (CVE-2026-48155/48156), yt-dlp 2026.3.17->2026.6.9,
pyjwt 2.12.1->2.13.0, cryptography 48.0.0->49.0.0, python-multipart
0.0.28->0.0.32, with lower-bound floors for the direct deps (pypdf, yt-dlp) so
installs get the patched versions. Lockfile regenerated for 0.8.42.
Co-Authored-By: hypnwtykvmpr <hypnwtykvmpr@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_read_files capped every file at 20,000 chars, so a Markdown/text/rST document
longer than that lost everything past the cap with no recovery. Oversized
splittable-text files are now split at heading/paragraph boundaries into units
that each fit the cap and together cover the whole file; every slice reports its
parent file as source_file so the graph isn't fragmented per-slice, and a slice
that still overflows output is bisected and retried. Code and PDFs are never
sliced.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
After #1361 added root= to the --update build_merge call, build_merge's prune
(which runs after the merge) began matching freshly re-extracted changed-file
nodes and deleting them — a regression in 0.8.41 where --update on a changed
file wiped its nodes. This drops `changed` from prune_sources (replace-on-
re-extract from #1344 already reconciles changed files), passes root= to the
full build's build_from_json so its node-key base matches the update side, and
pins the extraction-spec source_file to the verbatim path so the two runs never
drift. Re-blessed skillgen expected/ snapshots.
Co-Authored-By: RelywOo <RelywOo@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add record_declaration to the Java class types so a record becomes a first-class
type node instead of an isolated file node, and add object_creation_expression
to call types with a dedicated branch that reads the callee from the `type`
field (new Foo() has no `name` field), so `new Foo(...)` emits a calls edge to
the constructed type.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A .graphifyignore made graphify skip that directory's .gitignore entirely, so a
file excluded only by .gitignore (including neutrally-named secrets the
sensitive-file heuristic misses) got indexed into the graph and could leak into
committed graph artifacts. Read .gitignore first and .graphifyignore last so
their patterns merge and graphifyignore negations still win.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add changelog entries for the Swift cross-file (#1356), update prune (#1361),
obsidian canvas (#1324), and edge source_file backfill (#1279) fixes that
shipped without changelog notes, and a README troubleshooting entry for the
LLM JSON-truncation warnings and how to reduce them.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ollama/openai/deepseek/kimi set max_tokens in their backend config, but the
openai-compat dispatch read only max_completion_tokens (which only gemini
defines), so their output silently capped at the 8192 fallback and truncated
deep-mode JSON. Read either key and give the openai config an explicit cap;
GRAPHIFY_MAX_OUTPUT_TOKENS still overrides.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three pass-2 guards (mirrored in the --dedup-llm pair collection): block merges
when labels' embedded numbers differ as zero-padding-insensitive multisets;
block cross-file merges of file-anchored rationale/document nodes (same-file
still merges); and score cross-file long labels on plain Jaro instead of
Jaro-Winkler so the prefix bonus can't fabricate merges of shared-prefix but
token-divergent entities (jest-native vs react-native), while genuine cross-file
duplicates still clear Jaro and same-file near-duplicates keep Jaro-Winkler.
Co-Authored-By: van4oza <van4oza@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Completes the source_file convention fix begun in #1344 (build_merge
replace-on-re-extract) and #1361 (pass root= to build_merge in the --update
runbook). Two gaps still let the full build and incremental --update emit
different source_file bases for the same file, so the source_file-keyed replace
missed and duplicates accumulated:
1. extraction-spec(.md/-compact.md): the subagent's source_file slot was an
unpinned "relative/path", so it invented a base per run (and the node id,
derived from the same path, drifted too). Pin it to the verbatim FILE_LIST
path so _norm_source_file(root) canonicalizes every run identically.
2. core.md: the full build called build_from_json WITHOUT root=, so #1361's
update-side root= had no matching base on the full-build side. Pass
root='INPUT_PATH' at both sites (Step 4 export, Step 5 report) so the full
build and --update relativize to the same base.
update.md prune_sources = deleted only. Changed files are replaced by build_merge
(#1344); once root= aligns the bases, leaving `changed` in prune_sources would
delete the freshly re-extracted nodes.
Engine (build.py) unchanged. Regenerated all skill artifacts via
tools/skillgen/gen.py. Adds test_build_merge_root_collapses_convention_drift.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Semantic/LLM edges occasionally omit source_file, which build only normalized
when already present, so the field reached graph.json empty and downstream
validation flagged it. Backfill from the source (then target) node in
build_from_json and in the --no-cluster raw-write path, which bypasses it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
to_canvas built cards solely by iterating communities, so a graph with no
community data (--no-cluster builds, or a missing analysis sidecar) wrote the
empty 32-byte {"nodes":[],"edges":[]} shell while notes rendered fine. Fall back
to one synthetic community covering every node so the canvas reflects the graph.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The /graphify --update runbook called build_merge with absolute prune_sources
but no root=, so _norm_source_file never relativized them to match the graph's
relative source_file values. Nothing was pruned and changed/deleted files left
ghost nodes that compounded on every incremental update. Fix the shared skillgen
fragment to pass root='INPUT_PATH' and re-render all platform artifacts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Capture property/field initializer constructor calls, build a per-file Swift
type table from property/parameter declarations, and add a member-call
resolution pass that types the receiver and emits an edge only when the type
name resolves to exactly one definition. Additive and INFERRED-only; the
is_member_call drop and the #543/#1219 god-node guards stay intact.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Harden HTML export against U+2028/U+2029 script-breakout XSS + two crash-on-adversarial-input fixes (non-dict LLM JSON, _extract_parallel IndexError). Validated: full suite 2107 passed, HTML export smoke clean. Thanks @mistic96.
build_merge: prune a re-extracted file's stale nodes/edges before merge instead of accumulating (fixes#1283, #1285). Validated: full suite 2107 passed. Thanks @RelywOo.
Index PowerShell .psd1 manifests + emit Import-Module/dot-source edges (closes#1331). Builds on the shipped .psm1 support. Validated: full suite 2107 passed, 18 new tests. Thanks @geektan123.
The query skill was split across two fragments so no platform got both
capabilities: Claude had the vocab/IDF query-expansion step but no fallback if
the CLI was unavailable; every other platform had the inline NetworkX fallback
but the weaker raw-question matcher. Merge into one unified query reference +
stub (Step 0 expansion -> CLI traversal -> inline NetworkX fallback, plus
path/explain inline) shipped to all hosts. Remove the query_variant enum, its
toml field, and the _CLI_ONLY_QUERY_HEADINGS coverage-audit exemption. Re-render
all skill artifacts and re-bless expected/. skillgen check/audit-coverage/
monolith-roundtrip/schema-singleton all pass. Refs #1325.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Same-named types in different packages left implements/inherits edges stuck on
bare shadow stubs, isolating the real interface (_rewire_unique_stub_nodes only
fixes the globally-unique case). New _resolve_java_type_references pass uses each
referencing file's import statements (+ package decl) to build an FQN->def index,
re-points dangling implements/inherits/imports edges to the exact definition, and
drops the orphaned stub. External/stdlib imports stay unresolved (correct). Runs
after id-disambiguation so target ids are final. Java-scoped; other _extract_generic
languages share the same bare-name fallback and remain a follow-up.
Adds tests/test_java_type_resolution.py (simple, ambiguous-by-import, build-survival).
Refs #1318.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adopts the approach from #1330 (thanks @duncan-daydream) on top of the v0.8.40
Swift import fix: _import_swift returns (id,label) module pairs, the extractor
materializes a type=module anchor node per import, and _disambiguate_colliding_node_ids
exempts type=module nodes so the same module imported from N files collapses to
one shared node (enables reverse traversal "what imports CoreKit"). The --no-cluster
writer now dedupes nodes by id and edges to match the clustered build_from_json path.
Replaces the interim _import_label/synthesize_import_module_nodes mechanism.
Adds tests/test_swift_import_resolution.py (cross-file collapse, build survival)
and dedupe_nodes coverage. Refs #1327, #1330.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bump version to 0.8.40; document this release in CHANGELOG (PowerShell .psm1
indexing, Swift import survival, no-cluster edge dedup, custom OpenAI/Anthropic
endpoints, JS/TS assignment-form extraction, community-name + --graph fixes,
four production bug fixes, perf, security CI); add .psm1 to the README
supported-extensions table.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- #1315: add .psm1 to CODE_EXTENSIONS + _DISPATCH so PowerShell modules are indexed
- #1327: synthesize a module node for Swift import targets (new LanguageConfig
flag synthesize_import_module_nodes) so imports edges survive build.py pruning;
strengthen the Swift dangling-edge test to also assert edge targets
- #1317: dedupe parallel edges by (source,target,relation) in the --no-cluster
and incremental update write paths so edge counts are deterministic and
`update` is idempotent
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extract JS/TS this.X=, exports.X=, prototype, class arrow fields, and function expressions (closes#1322). Validated locally against v8: full suite 2069 passed.
export.py: to_json now accepts community_labels and writes community_name onto
each node. Previously cluster-only wrote labels only to GRAPH_REPORT.md,
graph.html, and .graphify_labels.json — graph.json stored only the numeric cid,
so query/MCP showed blank or numeric community values (#1305).
__main__.py: pass community_labels=labels to to_json in cluster-only path.
explain command now prefers community_name over raw numeric community field.
serve.py: query and get_node read paths prefer community_name over community,
with fallback so old graphs without the field still work. Adds --graph flag as
an alias for the positional argument in graphify-mcp/_main(), fixing
"unrecognized arguments: --graph" for users following the documented pattern
shared by every other graphify subcommand (#1304).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
safishamsi