Two integrity improvements to the core skill runbook (rendered via skillgen):
1. Graph-health gate (new Step 4.5): runs diagnostics.diagnose_extraction
read-only after build, before labeling, and surfaces dangling/missing/
self-loop and same-endpoint-collapsed edges — the silent-corruption modes of
incremental updates and AST/LLM id mismatches. Never aborts.
2. Anchor the AST and semantic caches on the scan root (root/cache_root=
'INPUT_PATH') instead of the cwd, matching the CLI extract path
(ast cache_root=out_root, check/save_semantic_cache(root=out_root)) and
build_from_json/save_manifest (#1361 parity). Without this, running from a
cwd != scan root cold-misses the cache and can split AST vs semantic caches.
Source edited in tools/skillgen/fragments/core/core.md; artifacts + expected/
regenerated via 'python -m tools.skillgen' (+ --bless). Full suite green
(2229 passed); skillgen + cache suites green (80 passed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
_score_nodes and _find_node scan every node per query (O(nodes x terms)),
so query latency scales with total graph size regardless of where the
answer lives. Add a lazily-built character-trigram index (cached on the
graph object, auto-invalidated on hot-reload like _idf_cache) that narrows
each query to a small candidate superset before the unchanged scoring loop.
Results are byte-identical: the index is a pure candidate generator over
the exact fields the scorer reads (norm_label, label_tokens, nid,
source_file); a non-candidate node always scores 0, and IDF stays a
whole-graph statistic. _find_node candidates are returned in graph
iteration order so its exact/prefix/substring ordering, and matches[0],
stay unchanged.
A selectivity guard falls back to the full scan when a query term is too
short to trigram or its rarest trigram is still common (broad terms like
model/client), preserving a never-worse contract.
The index builds eagerly at load and before a reloaded graph is swapped
in, so neither the first query nor the first post-reload query pays the
one-time build cost. Storage format is unchanged (in-memory index only).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
graphify install --platform hermes always wrote the skill to ~/.hermes/skills,
the POSIX path. On Windows, Hermes scans %LOCALAPPDATA%\hermes\skills, so the
installed skill was never discovered. _platform_skill_destination now has a
hermes branch: Windows -> %LOCALAPPDATA%\hermes\skills, other OSes unchanged
(~/.hermes/skills). Pure path logic — no skillgen regeneration.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A class defined once but referenced via type annotations in N other files appeared
as 1+N nodes — the extras carrying the referencing file's path (with extension)
baked into the id (e.g. pkg_a_py_thing). ensure_named_node's cross-file fallback
called add_node, which stamps the referencing file as source_file; that sourced
stub then collided in _disambiguate_colliding_node_ids (baking the .py path into
the id) and _rewire_unique_stub_nodes skipped it (a node with a source_file is
treated as a real definition, not a stub).
The fallback now emits a SOURCELESS stub (mirroring the inheritance-base path), so
disambiguation ignores it and the rewire collapses it onto the canonical
definition. The helper is duplicated across all six language extractors, so the
fix is applied to all six. Genuinely-defined duplicates (same name, different
files) still stay separate — only cross-file references collapse.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cover the #1410 fix: to_obsidian and to_canvas must never emit a punctuation-only
filename (e.g. `@.md` from a `@/*` tsconfig paths key) — valid on disk but empty
once a downstream tool re-slugs on word chars (crashes `qmd update`). Both tests
exercise the public exporters and fail against the pre-fix safe_name().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An all-punctuation node label (e.g. `@/*` from a tsconfig paths entry) survived
the unsafe-char strip in `to_obsidian`'s `safe_name()` as a bare `@`, producing
`@.md`. That filename is valid on disk but empty once a downstream tool re-slugs
on word chars — qmd's handelize() reduces "@" -> "" and raises, aborting the
entire `qmd update` (every collection on the machine stops reindexing).
Require at least one word char in the stem; otherwise fall back to "unnamed"
(the existing dedup handles collisions). Applied to both safe_name occurrences.
Fixes#1409
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CUDA is a C++ superset, so .cu/.cuh files parse cleanly with
tree-sitter-cpp (already a dependency). Two registrations wire it up:
- detect.py: add .cu/.cuh to CODE_EXTENSIONS so they're detected and
watched (watch.py's _WATCHED_EXTENSIONS derives from CODE_EXTENSIONS).
- extract.py: route .cu/.cuh through extract_cpp in _DISPATCH, which
also makes collect_files() pick them up (_EXTENSIONS = _DISPATCH.keys()).
Adds tests/fixtures/sample.cu (kernel + __device__/host functions +
struct + includes) and CUDA cases in test_languages.py covering kernel/
device function extraction, structs, includes, and host call edges.
Documents the new extensions in the README extension table and CHANGELOG.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The opencode plugin template embedded the reminder string with backticks
around `graphify query "<question>"`. Because the plugin prepends
`echo "<reminder>" && <cmd>` to the user's bash command, those backticks
triggered bash command substitution: every grep/rg/find invocation silently
ran `graphify query "<question>"` and substituted its output
("No matching nodes found.") into the reminder text shown to the agent.
This both corrupted tool output with graphify noise, and actually spawned
a graphify process + loaded graph.json + ran a BFS traversal with the
literal token <question> on every search.
Fix: remove the backticks. Adds a guard comment in the template so future
editors don't reintroduce the bug, and a regression test that asserts the
reminder string contains no backticks and no $() constructs.
`graphify extract --backend <gemini|claude|claude-cli|openai|kimi|...>` produced
zero hyperedges for any corpus: llm._EXTRACTION_SYSTEM only showed
"hyperedges":[] in its output schema and never described what a hyperedge is, so
every model returned the empty array. Meanwhile the agent/skill path, whose
references/extraction-spec.md fully documents hyperedges ("3 or more nodes
participate together..."), produced them — the two prompts had drifted.
Bring the native prompt in line with the skill spec: add the hyperedge
instruction and a populated schema example. The parse/merge side already handled
hyperedges, so this is prompt-only. Verified with a real claude-cli run — a doc
that previously yielded 0 hyperedges now yields one, correctly relativized (#1418).
Adds two guard tests: the native prompt must request hyperedges with a populated
example, and it must share the skill spec's hyperedge wording so they can't drift
apart again.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The GRAPHIFY_OUT override (custom output-dir name / absolute path, #686) was only
respected by some readers. `graphify extract` and several commands hardcoded the
literal "graphify-out", so `GRAPHIFY_OUT=custom-out graphify extract` still wrote
to graphify-out/ and downstream query/serve/update looked in the wrong place.
Resolve the output-dir name through graphify.paths everywhere it matters:
- new graphify.paths.out_path()/default_graph_json() helpers
- __main__: extract write dir, cluster-only/label, query/affected/benchmark
defaults, save-result --memory-dir, uninstall --purge, cache-check
- detect: _MANIFEST_PATH, memory/ + converted/ dirs, and the scan-exclude (a
renamed output dir is no longer re-ingested as source input)
- transcribe._TRANSCRIPTS_DIR; build_merge/serve/benchmark/prs graph-path defaults
Default behaviour is unchanged: with no env var everything still uses graphify-out/.
Verified end-to-end (extract -> cluster-only -> query under GRAPHIFY_OUT=custom-out
writes/reads custom-out/, no stray graphify-out/) and added a CLI regression test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The split-skill runbook passed '.' as report.generate's root argument in
Steps 4 and 5, so `/graphify /some/path` produced a report titled
"# Graph Report - ." regardless of the scanned directory. It now passes
'INPUT_PATH' (the Aider/Devin monoliths were already correct). Display-only:
no path written to graph.json or manifest.json was affected. Regenerated +
blessed the split-skill artifacts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The skill runbooks called save_manifest(...) with no root=, so manifest keys
were stored as absolute paths (e.g. /Users/.../main.go). Cloning or moving the
repo then broke `graphify --update`: detect_incremental matched none of the
cached keys, so the entire corpus re-extracted (and the report showed ghost
nodes). The native `graphify extract`/`update` CLI already passed root=target;
the agent-executed runbooks did not.
All four runbook call sites now pass root='INPUT_PATH', relativizing manifest
keys to the scan root (portable forward-slash form, per save_manifest's #777
support): the lean-core skill.md Step 9, the shared --update reference, and the
Aider/Devin monoliths.
The monolith edit is registered as a new sanctioned change-class
(_is_manifest_root_fix_line) in the round-trip guard, mirroring how the #1392
runbook fixes were sanctioned. Regenerated + blessed all artifacts; added a
regression test asserting every shipped runbook threads root= into save_manifest.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#1418: build_from_json relativized source_file on nodes and edges but stored
graph.hyperedges[] verbatim, so a semantic subagent's absolute path leaked into
graph.json. Relativize hyperedges in build_from_json (to_json has no root to
relativize against), mirroring the existing node/edge handling.
#1423: consolidate the GRAPHIFY_OUT output-dir name into a single graphify.paths
module (was duplicated in __main__, cache, watch) and route the path guards
through it — security.validate_graph_path's base=None discovery + fallback,
callflow_html's project-root resolution, and the post-commit/post-checkout hook
bodies (which now read the env var at hook-run time). A renamed output dir is no
longer validated against the wrong base or missed by the hook.
Tests: hyperedge relativization (test_hypergraph), GRAPHIFY_OUT discovery
(test_security), updated the hook-body contract assertion (test_hooks).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The monoliths are hand-maintained single files frozen against a pinned pristine-v8 blob by the round-trip guard, so they were excluded from the 0.8.44 #1392 batch. Evolve the guard from a positional zip (line-count-exact, single-line-class allowlist) to a multiset diff that classifies every added/removed line against documented sanctioned change-classes, so the multi-line fixes can land while any unsanctioned drift still fails. Add predicates for the four fix classes and broaden the enum/chunk-cleanup predicates to match both the v8 and rewritten forms.
Both monoliths now: thread directed=IS_DIRECTED through every build_from_json call (a --directed run no longer collapses reciprocal edges), scope semantic extraction to document/paper/image, unlink a stale .graphify_cached.json on a cache miss, and run Step 4's zero-node guard before any write with the report/analysis gated on to_json persisting the graph (#1392).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes the non-crash tier of #1392 in the Claude-path skill fragments:
- #6/#7: build_from_json/build_merge now take directed=IS_DIRECTED in Step 4, Step 5 rebuild, and the --update merge/diff, with prose telling the agent to substitute IS_DIRECTED like INPUT_PATH (a --directed run no longer silently rebuilds undirected and collapses reciprocal edges)
- #10: semantic extraction only flattens document/paper/image, not code (AST already covers code) so subagents stop re-reading every source file
- #12: .graphify_cached.json is deleted on a cache miss so Part C never merges a stale cache from a prior run
- #11: --update now transcribes changed video files and moves transcripts to documents before the semantic pipeline
- #4/#5/#23: transcribe writes via write_text (no shell redirect), uses GRAPHIFY_WHISPER_MODEL/PROMPT env, status to stderr
- #2/#3: add-watch and exports use $(cat graphify-out/.graphify_python) explicitly; MCP Desktop config documents the absolute interpreter path
- #21: extraction-spec example id namespaced (auth_session_validatetoken)
- #22: query term split keeps tokens >= 3 chars
aider/devin monoliths are pinned by the roundtrip invariant and excluded; their own Step 1 already instructs replacing python3 with the resolved interpreter.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Five fixes in the skillgen fragments (re-rendered): chunk paths now derive from
cwd to match where Part C globs (a non-cwd scan produced "no nodes"); the
code-only fast path writes an empty .graphify_semantic.json so Part C doesn't
FileNotFoundError; --cluster-only relies on the self-contained CLI instead of
re-running Steps 5-9 against deleted intermediate files; and Step 4 runs the
zero-node guard before any write and only writes GRAPH_REPORT.md / analysis when
to_json actually persisted the graph (honoring the #479 shrink-guard).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 0.8.43 str-path coercion (#1386) ran Path(f) over every item in
extract_files_direct, but extract_corpus_parallel feeds it FileSlice units from
the oversized-doc slicing (#1369), and Path(FileSlice) raises TypeError -- so
semantic extraction of any Markdown file larger than _FILE_CHAR_CAP crashed.
Coerce only non-Path/non-FileSlice entries. The #1386 tests used small files, so
slicing never ran; added a regression test with a file that actually slices.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
apm.yml was a .yml document handled by the LLM, so the same package got a
different file-anchored node id from its own manifest than from each dependent's
dependency reference and split into duplicate nodes. New manifest_ingest module
parses apm.yml/pyproject.toml/go.mod/pom.xml deterministically into ONE package
node per package, keyed by name via ids.make_id, plus depends_on edges; routed
to the AST path (CODE) so the LLM never sees them. Package nodes are exempt from
the file-stem prefix remap so the canonical id is stable across manifests and
dedup collapses references to a single hub node.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On WSL/POSIX, Path("C:\\...").is_absolute() is False, so a Windows absolute
core.hooksPath (or rev-parse --git-path output) was joined under the repo root
and mkdir'd as a literal backslash-named junk directory while install reported
success and the real .git/hooks got nothing. Both branches of _hooks_dir now
reject drive-letter / backslash paths with a clear error so the failure is loud.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
extract_markdown only emitted heading nodes + contains edges and never parsed
link syntax, so a doc full of [text](./other.md) links (index.md,
table-of-contents.md) had no edges to the docs it links and never became a hub.
Add a deterministic link pass: inline, reference-style, and [[wikilinks]],
resolved relative to the source file, external URLs/anchors/images skipped, with
the target id built via the same _make_id recipe so the edge merges onto the
real doc node instead of an orphan.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
extract_corpus_parallel and extract_files_direct are typed list[Path] but
crashed with AttributeError on str paths (f.suffix in slicing/partition, f.parent
in packing). Coerce files = [Path(f) for f in files] at both public entry points;
the AST extract() already coerced.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
extract._make_id and build._normalize_id were copy-pasted forks of the same
NFKC/casefold recipe kept in sync by mirrored docstrings -- the root of the
recurring ID-drift ghost-node bug class (#811/#550/#1033/#1104). Move the recipe
to graphify/ids.py and have all four producers delegate to it: extract, build,
and (completing the migration) mcp_ingest and symbol_resolution, whose
"avoid an import cycle" copies are moot now that ids.py is dependency-free. The
contract test asserts all four resolve to the shared recipe, with hypothesis
property tests for make_id == normalize_id and idempotency.
Co-Authored-By: danielnguyenfinhub <danielnguyenfinhub@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
label_communities logged-and-skipped any batch whose LLM response was malformed
JSON, silently losing ~100 names per failed batch on large graphs. Split the
batch at the midpoint and retry each half (smaller prompt -> smaller output),
mirroring _extract_with_adaptive_retry; the base case re-raises so the caller
skips just that batch. Removed leftover scaffolding and corrected the docstring.
Co-Authored-By: CJdev232 <CJdev232@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pypdf 6.11.0->6.13.3 (CVE-2026-48155/48156), yt-dlp 2026.3.17->2026.6.9,
pyjwt 2.12.1->2.13.0, cryptography 48.0.0->49.0.0, python-multipart
0.0.28->0.0.32, with lower-bound floors for the direct deps (pypdf, yt-dlp) so
installs get the patched versions. Lockfile regenerated for 0.8.42.
Co-Authored-By: hypnwtykvmpr <hypnwtykvmpr@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_read_files capped every file at 20,000 chars, so a Markdown/text/rST document
longer than that lost everything past the cap with no recovery. Oversized
splittable-text files are now split at heading/paragraph boundaries into units
that each fit the cap and together cover the whole file; every slice reports its
parent file as source_file so the graph isn't fragmented per-slice, and a slice
that still overflows output is bisected and retried. Code and PDFs are never
sliced.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
After #1361 added root= to the --update build_merge call, build_merge's prune
(which runs after the merge) began matching freshly re-extracted changed-file
nodes and deleting them — a regression in 0.8.41 where --update on a changed
file wiped its nodes. This drops `changed` from prune_sources (replace-on-
re-extract from #1344 already reconciles changed files), passes root= to the
full build's build_from_json so its node-key base matches the update side, and
pins the extraction-spec source_file to the verbatim path so the two runs never
drift. Re-blessed skillgen expected/ snapshots.
Co-Authored-By: RelywOo <RelywOo@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add record_declaration to the Java class types so a record becomes a first-class
type node instead of an isolated file node, and add object_creation_expression
to call types with a dedicated branch that reads the callee from the `type`
field (new Foo() has no `name` field), so `new Foo(...)` emits a calls edge to
the constructed type.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A .graphifyignore made graphify skip that directory's .gitignore entirely, so a
file excluded only by .gitignore (including neutrally-named secrets the
sensitive-file heuristic misses) got indexed into the graph and could leak into
committed graph artifacts. Read .gitignore first and .graphifyignore last so
their patterns merge and graphifyignore negations still win.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add changelog entries for the Swift cross-file (#1356), update prune (#1361),
obsidian canvas (#1324), and edge source_file backfill (#1279) fixes that
shipped without changelog notes, and a README troubleshooting entry for the
LLM JSON-truncation warnings and how to reduce them.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ollama/openai/deepseek/kimi set max_tokens in their backend config, but the
openai-compat dispatch read only max_completion_tokens (which only gemini
defines), so their output silently capped at the 8192 fallback and truncated
deep-mode JSON. Read either key and give the openai config an explicit cap;
GRAPHIFY_MAX_OUTPUT_TOKENS still overrides.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three pass-2 guards (mirrored in the --dedup-llm pair collection): block merges
when labels' embedded numbers differ as zero-padding-insensitive multisets;
block cross-file merges of file-anchored rationale/document nodes (same-file
still merges); and score cross-file long labels on plain Jaro instead of
Jaro-Winkler so the prefix bonus can't fabricate merges of shared-prefix but
token-divergent entities (jest-native vs react-native), while genuine cross-file
duplicates still clear Jaro and same-file near-duplicates keep Jaro-Winkler.
Co-Authored-By: van4oza <van4oza@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Completes the source_file convention fix begun in #1344 (build_merge
replace-on-re-extract) and #1361 (pass root= to build_merge in the --update
runbook). Two gaps still let the full build and incremental --update emit
different source_file bases for the same file, so the source_file-keyed replace
missed and duplicates accumulated:
1. extraction-spec(.md/-compact.md): the subagent's source_file slot was an
unpinned "relative/path", so it invented a base per run (and the node id,
derived from the same path, drifted too). Pin it to the verbatim FILE_LIST
path so _norm_source_file(root) canonicalizes every run identically.
2. core.md: the full build called build_from_json WITHOUT root=, so #1361's
update-side root= had no matching base on the full-build side. Pass
root='INPUT_PATH' at both sites (Step 4 export, Step 5 report) so the full
build and --update relativize to the same base.
update.md prune_sources = deleted only. Changed files are replaced by build_merge
(#1344); once root= aligns the bases, leaving `changed` in prune_sources would
delete the freshly re-extracted nodes.
Engine (build.py) unchanged. Regenerated all skill artifacts via
tools/skillgen/gen.py. Adds test_build_merge_root_collapses_convention_drift.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Semantic/LLM edges occasionally omit source_file, which build only normalized
when already present, so the field reached graph.json empty and downstream
validation flagged it. Backfill from the source (then target) node in
build_from_json and in the --no-cluster raw-write path, which bypasses it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
to_canvas built cards solely by iterating communities, so a graph with no
community data (--no-cluster builds, or a missing analysis sidecar) wrote the
empty 32-byte {"nodes":[],"edges":[]} shell while notes rendered fine. Fall back
to one synthetic community covering every node so the canvas reflects the graph.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The /graphify --update runbook called build_merge with absolute prune_sources
but no root=, so _norm_source_file never relativized them to match the graph's
relative source_file values. Nothing was pruned and changed/deleted files left
ghost nodes that compounded on every incremental update. Fix the shared skillgen
fragment to pass root='INPUT_PATH' and re-render all platform artifacts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Capture property/field initializer constructor calls, build a per-file Swift
type table from property/parameter declarations, and add a member-call
resolution pass that types the receiver and emits an edge only when the type
name resolves to exactly one definition. Additive and INFERRED-only; the
is_member_call drop and the #543/#1219 god-node guards stay intact.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Harden HTML export against U+2028/U+2029 script-breakout XSS + two crash-on-adversarial-input fixes (non-dict LLM JSON, _extract_parallel IndexError). Validated: full suite 2107 passed, HTML export smoke clean. Thanks @mistic96.
build_merge: prune a re-extracted file's stale nodes/edges before merge instead of accumulating (fixes#1283, #1285). Validated: full suite 2107 passed. Thanks @RelywOo.
Index PowerShell .psd1 manifests + emit Import-Module/dot-source edges (closes#1331). Builds on the shipped .psm1 support. Validated: full suite 2107 passed, 18 new tests. Thanks @geektan123.
safishamsi