build._is_ast_tier reads _origin when present and otherwise guesses the tier
from the shape of source_location (L<line> means AST). extract() stamps its AST
items _origin:"ast", but nothing stamped the semantic side, so a semantic node
whose location happened to look like "L12" read as AST — and the next
`graphify update` deleted it along with the re-extracted code file it seemed to
belong to. Part C, extract_corpus_parallel, and the CLI merge now setdefault
_origin:"semantic" on every semantic node/edge before the merge, so a semantic
item is never mistaken for an AST one. AST items (stamped "ast") are untouched;
setdefault preserves any existing stamp.
Landed from PR #3110. Skill artifacts regenerated fresh from the updated
fragments (not the PR's stale committed blobs); all five skillgen validators
pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Persist validated Codex extraction responses as chunk files so the shared B3 merge preserves semantic nodes and edges.
Fail before cache and graph updates when extraction responses are missing or invalid.
Add regression coverage for chunk merging, cached reruns, and failure handling.
Fixes#4311
Co-authored-by: OpenAI Codex <codex@openai.com>
(cherry picked from commit da5850f0ac)
Step 9's report never mentioned that graphify-out/ was written into the
working tree, so a repository that does not ignore it was left with
untracked machine-local files in git status and no explanation. The report
now adds one sentence in a git repository, with the two ways to handle it.
The note goes in the three core fragments (core, aider, devin); the devin
monolith change is sanctioned by a new round-trip predicate.
(cherry picked from commit b4e6769371)
Read existing CLI build-config options in fresh and incremental skill
scan fragments, keeping corpus boundaries consistent without changing
standalone detection API defaults. Regenerate all host artifacts and
snapshots. Add rendered-snippet regressions and explicit-override tests.
Fixes#4240.
AI-Assisted-By: AI coding assistant
(cherry picked from commit 3f68952d80)
The split-host runbook told the agent to skip to Part C when every doc,
paper and image hit the semantic cache. Part C reads
.graphify_semantic.json unconditionally and only the Step B3 merge writes
it, so a second run on an unchanged mixed corpus failed with
FileNotFoundError; writing an empty file instead dropped every cached node.
The all-cached case now skips B1/B2 but still runs B3. Step B0 also clears
.graphify_chunk_*.json before dispatch: B3 merges every chunk on disk, and a
leftover from an interrupted run would otherwise be merged as fresh.
The aider and devin monoliths keep the old sentence; they are pinned to the
v8 baseline by --monolith-roundtrip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit bab1bb2775)
The Windows skill variant had a PowerShell Step 1 but its later steps came
from the shared core fragment as bash-only shell (cat-piped interpreter
invocations, rm -f, find -delete). The skillgen renderer now translates the
composed core to PowerShell for powershell-shell platforms (here-string
interpreter invocations, Remove-Item cleanup); POSIX skills are unchanged
and step / #2490 parity is enforced by a generator check. Not a regression
from 0.9.35 (skill files were unchanged between 0.9.34 and 0.9.35).
Thanks @tannermosher2015-debug.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2486 (thanks @adminwat): normalize dict-shaped hyperedge members to ids
(or drop with a warning) so a malformed hyperedge can't abort a completed
merge with a TypeError.
#2484 (thanks @sortakool; approach from @oleksii-tumanov's #1691):
merge-graphs relabels hyperedge member ids and ids with the repo prefix,
unions both inputs' hyperedges instead of clobbering, and writes both
persistence slots.
#2485 (thanks @sortakool): build_from_json reads hyperedges from the
top-level and nested slots; a full validation wipeout is reported loudly.
#2490 (thanks @PapiScholz): the skill Step-5 flow passes curated
community_labels to to_json, so graph.json ships community_name.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The --update runbook's Step 9 stamped the entire detected corpus into the
manifest, so a semantic file (doc/paper/image) whose chunk failed or was omitted
was recorded as done and never re-queued on the next update — losing its content
permanently. The runbook now builds the manifest the way the library extract
path does: cli._stamped_manifest_files stamps only files that actually produced
nodes/edges/hyperedges, dispatched-but-empty files have their stale hash cleared,
and scan_corpus drops newly-excluded in-root rows.
Applied to the Claude, Aider, and Devin skill bodies and the shared update
reference; all 134 artifacts regenerated. gen.py gains a sanctioned monolith-
diff predicate for the new stamping lines.
The semantic cache keyed entries on sha256(file content + path) alone, with
no component for the extraction prompt that produced them. After an upgrade
that changed the prompt, every unchanged file was a cache hit and replayed
the older prompt's extraction: the run exited 0, cost.json looked cheap, and
the graph silently carried two prompt generations side by side. The reporter
saw 506 of 512 docs replay an older vintage on a rebuild they expected to be
cold; deleting the whole cache was the only workaround.
output, and invalidating them on every release would re-bill extraction for
unchanged files. Fingerprinting the prompt itself keeps both properties:
entries survive releases that don't touch the prompt, and invalidate only
when it actually changed.
Semantic entries now live under cache/semantic/p{fingerprint}/, mirroring the
AST cache's v{version}/ layout. Both extraction paths pass their prompt: the
Python/CLI path from llm.py's _EXTRACTION_SYSTEM (shared by every backend),
and the skill path via a new prompt_file argument in Step B0/B3 naming the
references/extraction-spec.md the subagents were handed. The fingerprint
normalizes line endings so a CRLF checkout isn't mistaken for a new prompt.
Pre-existing entries predate fingerprinting and have unknowable vintage, so
they are still served rather than re-billing a whole corpus on upgrade — but
check_semantic_cache now warns with the count, turning "no signal at all"
into a visible one. merge_existing refuses to fuse such an entry into a
current-vintage write, which would mix two prompts inside one entry and then
attest the result to a prompt that produced half of it.
Old-fingerprint entries are pruned by liveness only, never swept wholesale
the way stale AST versions are: two hosts with different prompts (verbose vs
compact extraction-spec) can share one graphify-out/, and a wholesale sweep
would have each run delete the other's entries and re-bill on every
alternation. prune/clear/cached_files glob recursively so fingerprinted
entries can't become unprunable orphans (the #1527 failure mode).
The two monolith skills (aider, devin) inline their prompt instead of
shipping a spec sidecar and stay on the unfingerprinted path for now.
Step 1's POSIX interpreter probe ran `uv tool run graphifyy python -c ...`,
but the graphifyy package exposes its executable as `graphify`, so uv treated
`python` as a missing `graphifyy` command. The probe failed, and `2>/dev/null`
swallowed uv's "use --from" hint, leaving PYTHON on a system interpreter that
does not have graphify installed. The probe now runs
`uv tool run --from graphifyy python -c ...`.
Applied to the skillgen source fragments (shell/posix.md and the aider/devin
monoliths) and regenerated all skill*.md; added a sanctioned monolith-diff
predicate and re-blessed the expected/ snapshots. The PowerShell path was
already correct (it probes the venv python.exe directly).
Reported and originally patched by @mohammedMsgm in #1736.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Usage block's bare `/graphify` comment read "full pipeline on current
directory -> Obsidian vault", contradicting Step 6 (HTML viz always; Obsidian
vault only when --obsidian is explicitly given). The comment predated the
opt-in change and told agents a bare /graphify produces a vault. It now reads
"full pipeline on current directory (HTML viz; add --obsidian for a vault)".
Fixed at the skillgen source (fragments/core/core.md + the aider monolith
override), added a sanctioned-diff predicate so the monolith round-trip guard
allows the change, and re-blessed the expected/ snapshots. Every generated
skill-*.md now carries the corrected comment; the only content change across
all rendered files is this one line.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#1586: the skill's interpreter-detection allowlist rejected any shebath path
with a char outside [a-zA-Z0-9/_.-], so Homebrew's versioned python@3.13 path
was skipped and detection fell through to a bare python3 without graphify.
Allow @ in the skillgen source fragments (core/devin/aider/posix), regenerate
all skill artifacts, and register the change as a sanctioned monolith diff.
Injection chars (; $ ( etc.) are still rejected.
#1606: graphify merge-graphs crashed with an unhandled NetworkXError when
inputs disagreed on directed/multigraph. _to_simple only converted MultiGraph,
leaving a DiGraph to fail compose. Normalize every input to a plain undirected
Graph (the merged cross-repo view is undirected anyway).
Also confirmed on v8: `graphify kilo install` (#1605) works (reporter was on an
older version), and `pip3 install graphifyy` failing on macOS (#1585) is a
broken Homebrew Python 3.14 / pyexpat env, not a graphify issue — pip crashes
in its own vendored distlib before graphify is touched.
skillgen --check / --monolith-roundtrip / --schema-singleton all green; shell
smoke accepts python@3.13 and still rejects `; rm -rf /` and `$(evil)`. Full
suite 2789.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Hermes and the other AGENTS.md hosts (Codex, Aider, OpenClaw, Droid, Trae, ...)
run the graphify CLI directly and do not dispatch subagents. The Step 3 extraction
guidance only described the no-API-key path as "fall straight through to subagent
dispatch", so on `/graphify .` those agents had no path they could take, fixated on
the GEMINI/ANTHROPIC key language, and looped for minutes insisting on a missing key
before eventually proceeding (a pure-code corpus is AST-only and needs no key at all).
Step 3 now opens with a hoisted, host-agnostic statement -- graphify needs no API
key, never prompt for one, never block on one; code is AST-only; a code-only corpus
skips semantic extraction entirely -- and the no-key fallback now spells out a
terminal-only path (write the empty semantic file and continue, or extract content
inline) instead of assuming subagent dispatch.
Applied in the shared core fragment and in the aider/devin core variants, so all 16
skill bodies carry it; the aider/devin change is registered as a sanctioned
monolith-roundtrip diff. Regression test (test_extraction_states_no_api_key_required_
for_every_host) renders every host and pins the wording, ordering, and the
non-subagent fallback so this can't silently regress.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two integrity improvements to the core skill runbook (rendered via skillgen):
1. Graph-health gate (new Step 4.5): runs diagnostics.diagnose_extraction
read-only after build, before labeling, and surfaces dangling/missing/
self-loop and same-endpoint-collapsed edges — the silent-corruption modes of
incremental updates and AST/LLM id mismatches. Never aborts.
2. Anchor the AST and semantic caches on the scan root (root/cache_root=
'INPUT_PATH') instead of the cwd, matching the CLI extract path
(ast cache_root=out_root, check/save_semantic_cache(root=out_root)) and
build_from_json/save_manifest (#1361 parity). Without this, running from a
cwd != scan root cold-misses the cache and can split AST vs semantic caches.
Source edited in tools/skillgen/fragments/core/core.md; artifacts + expected/
regenerated via 'python -m tools.skillgen' (+ --bless). Full suite green
(2229 passed); skillgen + cache suites green (80 passed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The split-skill runbook passed '.' as report.generate's root argument in
Steps 4 and 5, so `/graphify /some/path` produced a report titled
"# Graph Report - ." regardless of the scanned directory. It now passes
'INPUT_PATH' (the Aider/Devin monoliths were already correct). Display-only:
no path written to graph.json or manifest.json was affected. Regenerated +
blessed the split-skill artifacts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The skill runbooks called save_manifest(...) with no root=, so manifest keys
were stored as absolute paths (e.g. /Users/.../main.go). Cloning or moving the
repo then broke `graphify --update`: detect_incremental matched none of the
cached keys, so the entire corpus re-extracted (and the report showed ghost
nodes). The native `graphify extract`/`update` CLI already passed root=target;
the agent-executed runbooks did not.
All four runbook call sites now pass root='INPUT_PATH', relativizing manifest
keys to the scan root (portable forward-slash form, per save_manifest's #777
support): the lean-core skill.md Step 9, the shared --update reference, and the
Aider/Devin monoliths.
The monolith edit is registered as a new sanctioned change-class
(_is_manifest_root_fix_line) in the round-trip guard, mirroring how the #1392
runbook fixes were sanctioned. Regenerated + blessed all artifacts; added a
regression test asserting every shipped runbook threads root= into save_manifest.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes the non-crash tier of #1392 in the Claude-path skill fragments:
- #6/#7: build_from_json/build_merge now take directed=IS_DIRECTED in Step 4, Step 5 rebuild, and the --update merge/diff, with prose telling the agent to substitute IS_DIRECTED like INPUT_PATH (a --directed run no longer silently rebuilds undirected and collapses reciprocal edges)
- #10: semantic extraction only flattens document/paper/image, not code (AST already covers code) so subagents stop re-reading every source file
- #12: .graphify_cached.json is deleted on a cache miss so Part C never merges a stale cache from a prior run
- #11: --update now transcribes changed video files and moves transcripts to documents before the semantic pipeline
- #4/#5/#23: transcribe writes via write_text (no shell redirect), uses GRAPHIFY_WHISPER_MODEL/PROMPT env, status to stderr
- #2/#3: add-watch and exports use $(cat graphify-out/.graphify_python) explicitly; MCP Desktop config documents the absolute interpreter path
- #21: extraction-spec example id namespaced (auth_session_validatetoken)
- #22: query term split keeps tokens >= 3 chars
aider/devin monoliths are pinned by the roundtrip invariant and excluded; their own Step 1 already instructs replacing python3 with the resolved interpreter.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Five fixes in the skillgen fragments (re-rendered): chunk paths now derive from
cwd to match where Part C globs (a non-cwd scan produced "no nodes"); the
code-only fast path writes an empty .graphify_semantic.json so Part C doesn't
FileNotFoundError; --cluster-only relies on the self-contained CLI instead of
re-running Steps 5-9 against deleted intermediate files; and Step 4 runs the
zero-node guard before any write and only writes GRAPH_REPORT.md / analysis when
to_json actually persisted the graph (honoring the #479 shrink-guard).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
After #1361 added root= to the --update build_merge call, build_merge's prune
(which runs after the merge) began matching freshly re-extracted changed-file
nodes and deleting them — a regression in 0.8.41 where --update on a changed
file wiped its nodes. This drops `changed` from prune_sources (replace-on-
re-extract from #1344 already reconciles changed files), passes root= to the
full build's build_from_json so its node-key base matches the update side, and
pins the extraction-spec source_file to the verbatim path so the two runs never
drift. Re-blessed skillgen expected/ snapshots.
Co-Authored-By: RelywOo <RelywOo@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The query skill was split across two fragments so no platform got both
capabilities: Claude had the vocab/IDF query-expansion step but no fallback if
the CLI was unavailable; every other platform had the inline NetworkX fallback
but the weaker raw-question matcher. Merge into one unified query reference +
stub (Step 0 expansion -> CLI traversal -> inline NetworkX fallback, plus
path/explain inline) shipped to all hosts. Remove the query_variant enum, its
toml field, and the _CLI_ONLY_QUERY_HEADINGS coverage-audit exemption. Re-render
all skill artifacts and re-bless expected/. skillgen check/audit-coverage/
monolith-roundtrip/schema-singleton all pass. Refs #1325.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Makes the FalkorDB option a first-class sibling of Neo4j in the agent skill,
not just the export CLI:
- --falkordb / --falkordb-push shorthands documented in core.md + the shared
exports.md reference, so they render into all modular platform skills and
read exactly like --neo4j / --neo4j-push. (The aider/devin monoliths are
diff-frozen vs v8 by skillgen's roundtrip guard, so they are left untouched.)
- README command reference switched to the /graphify ./raw --falkordb-push form.
- Documented URI scheme is now falkordb://localhost:6379; the scheme is only
informational (host/port are parsed out), so redis:// or a bare host:port
remain equivalent. Regenerated skill artifacts + expected/ snapshots.
The Agent Skills spec only defines name, description, license,
compatibility, metadata, and allowed-tools as valid frontmatter fields.
The trigger: /graphify line was non-spec, silently ignored by spec-
following hosts, and flagged by agentskills validate CI checks.
- gen.py: removed trigger emission from _render_frontmatter; added
_is_trigger_line() helper for roundtrip allow-list
- fragments/core/aider.md: removed hardcoded trigger: /graphify
- platforms.toml: removed trigger doc comment and trigger="" entries
- test_skillgen.py: replaced trigger-assertion tests with a single
test asserting no host has trigger: in frontmatter
- Regenerated all 125 skill artifacts
Routing intent is preserved: the description field already contains
"treated as a graphify query first" and "graphify-out/ exists".
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
#1174: affected.py load_graph now forces directed=True before
node_link_graph, matching the identical fix in serve.py and __main__.py.
Undirected graphs (directed:false in graph.json) were causing in_edges
to fall back to a direction-blind scan, missing true callers and
reporting false positives. Regression test added.
#1173: post-commit and post-checkout hook bodies now read
graphify-out/.graphify_root before calling _rebuild_code, falling back
to Path('.') if absent. A scoped build (graphify src/) no longer gets
silently expanded to the full repo on the next commit. Tests added.
#1172: Step 9 cleanup split into rm -f for fixed files and
find -maxdepth 1 -delete for the chunk glob. Under fish/zsh an
unmatched glob aborts the entire rm -f line, leaving temp files on disk.
Fixed in the three skillgen source fragments and regenerated.
#1163: detect_incremental type guard on stored mtime — if the manifest
contains a dict-valued mtime (schema drift from older versions), coerce
to None rather than propagating a non-numeric into comparisons.
Regression test added.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
All-platform progressive-disclosure skill split + generator (addresses #1106).
Splits each platform's skill into a lean core (~615 lines, full default pipeline inline) + on-demand references/, generated from a single source via tools/skillgen with a CI/pre-commit drift gate. 13 hosts split, aider/devin stay monoliths. Also fixes the stale bare-path bugs across the previously hand-maintained variants and moves the always-on blocks into packaged markdown.
Verified: all 5 generator guards pass, byte-verbatim load-bearing slices, lean cores self-sufficient on the default path across all 13 split hosts, references gated to non-default branches, description preserves the graphify-out-query-first clause. Supersedes #1119 (Claude-first subset).
Known follow-up applied on top: harden _always_on() against a missing packaged file so a partial install can't brick the CLI.
