Both fixes landed after the initial 0.9.11 section was written but before
0.9.11 was published, so they belong in that release's notes rather than a
new version.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Step 1's POSIX interpreter probe ran `uv tool run graphifyy python -c ...`,
but the graphifyy package exposes its executable as `graphify`, so uv treated
`python` as a missing `graphifyy` command. The probe failed, and `2>/dev/null`
swallowed uv's "use --from" hint, leaving PYTHON on a system interpreter that
does not have graphify installed. The probe now runs
`uv tool run --from graphifyy python -c ...`.
Applied to the skillgen source fragments (shell/posix.md and the aider/devin
monoliths) and regenerated all skill*.md; added a sanctioned monolith-diff
predicate and re-blessed the expected/ snapshots. The PowerShell path was
already correct (it probes the venv python.exe directly).
Reported and originally patched by @mohammedMsgm in #1736.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Kotlin enum entries weren't extracted: the walker never descended into the enum
body (`enum_class_body` wasn't in _KOTLIN_CONFIG.body_fallback_child_types, so
_find_body returned None). Add `enum_class_body` to the fallback body types and a
`_kotlin_extra_walk` (dispatched for tree_sitter_kotlin, mirroring the Java/Swift
handling) that emits each enum_entry as a node with a `case_of` edge to the enum.
Re-applied from PR #1738 (@ivanzhl) onto the post-#1737 module layout: the walk
engine now lives in graphify/extractors/engine.py while _KOTLIN_CONFIG stays in
extract.py. Closes the Kotlin half of #1700 (Java was #1719).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Decompose extract.py / __main__.py / export.py into focused modules (#1737,
verbatim, no behavior change). Plus fixes since 0.9.10: merge-graphs distinct
repo tags (#1729), uninstall cleans Claude .local files (#1731), and
extract --code-only for keyless code-only indexing of a mixed repo (#1734).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
main() was a 2,760-line function that was almost entirely a 2,538-line
if/elif chain dispatching every non-install subcommand (query, path, explain,
diagnose, affected, reflect, save-result, extract, update, cluster-only, label,
build, global, merge-graphs, merge-driver, watch, tree, export, benchmark,
clone, prs, provider, hook*, check-update, and the `graphify <path>` redirect).
Move that chain, plus its 5 chain-only helpers (_StageTimer, _clone_repo,
_default_graph_path, _enforce_graph_size_cap_or_exit, _run_hook_guard) and 4
chain-only constants (_SEARCH_NUDGE, _READ_NUDGE, _HOOK_SOURCE_EXTS,
_GEMINI_NUDGE_TEXT), into graphify/cli.py as dispatch_command(cmd). main() now
does its setup (encoding, stale-skill check, version/help) then
`if dispatch_install_cli(cmd): return` else `dispatch_command(cmd)`.
The chain ends in its own unknown-command exit, so it's called as a statement —
no return-value plumbing. The one cli->__main__ coupling, the path-redirect's
recursive main() call, is handled by a lazy import (_reenter_main), so import
direction stays __main__ -> cli with no cycle. __main__ re-exports every moved
symbol, so importers/tests are unchanged.
__main__.py 3,388 -> 662 LOC (5,368 at branch start, -88%). Verified end-to-end:
version/help/unknown-command, the `graphify <path>` redirect (rebuilds a graph),
and query/path/explain. Full suite unchanged: 3036 passed, 29 skipped; skillgen OK.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Now that the resolution passes and the tree-sitter engine live in their own
modules, these three bespoke extractors are self-contained and move cleanly
(verbatim), pulling only the specific engine/resolution/base helpers they need:
- extractors/pascal.py: extract_pascal + regex helpers + _PAS_* constants
- extractors/objc.py: extract_objc + Objective-C member-call resolution
- extractors/julia.py: extract_julia
Re-exported from extract.py and registered in extractors/__init__.py (27 langs).
The remaining in-file extractors (js/ts config family + vue/svelte/astro/xaml)
stay because they share _JS_CONFIG/_TS_CONFIG and the xaml dispatch caches with
extract_js/extract_csharp and the dispatcher.
extract.py 5,947 -> 4,740 LOC (17,054 at branch start, -72%). Full suite
unchanged: 3036 passed, 29 skipped; skillgen --check OK.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Moves the config-driven extraction engine — _extract_generic (~2.2k LOC) and its
67-function closure of per-language tree-sitter walkers/type-collectors
(_js_*, _ts_*, _python_*, _java_*, _csharp_*, _cpp_*, _swift_*, _kotlin_*,
_php_*, _ruby_*, _scala_*) plus 10 engine-private constants (language builtin/type
tables) — into graphify/extractors/engine.py, verbatim.
AST analysis proved the engine closure references neither `extract` nor
`_DISPATCH`, so the dispatcher facade (extract, _DISPATCH, _get_extractor,
collect_files, the thin config-driven extractor wrappers) stays in extract.py and
imports the engine. Import direction: extract.py -> engine -> {resolution, models,
base}; no shared constant crosses the boundary (verified), and engine pulls a
single name (_resolve_js_import_target) from resolution, so there is no cycle.
extract.py re-exports every moved symbol, so importers and tests reaching into
engine internals are unchanged.
extract.py 10,270 -> 5,947 LOC (17,054 at branch start, -65%). Full suite
unchanged: 3036 passed, 29 skipped; skillgen --check OK.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
setuptools uses an explicit package list, so the new graphify/exporters/ package
(base, html, graphdb — split out of export.py) was absent from the built wheel,
which would break `import graphify.exporters.*` for installed users. Add it
alongside graphify and graphify.extractors. No version change.
Verified: fresh-venv install of the rebuilt wheel imports every new module,
resolves all backward-compat re-exports, builds a multi-language graph, and runs
query/path/explain and install/uninstall.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
main() carried a 256-line if/elif chain routing every install-family command
(install, uninstall, and the 22 per-platform targets: claude, codebuddy, gemini,
cursor, vscode, copilot, kilo, kiro, devin, pi, amp, agents/skills,
aider/codex/opencode/claw/droid/trae/trae-cn/hermes, antigravity). That block
only ever reads sys.argv and calls functions that already live in install.py, so
move it there verbatim as a guarded dispatch_install_cli(cmd) -> bool; main() now
does `if dispatch_install_cli(cmd): return`.
The command set is derived from the block's own conditions (so none is missed),
and the branch's early `return` becomes `return True` so the help path still
short-circuits. Verified end-to-end: `graphify claude install/uninstall`, unknown
command, and the install test suite all behave identically.
__main__.py 3,641 -> 3,388 LOC (5,368 at branch start, -37%). Full suite
unchanged: 3036 passed, 29 skipped; skillgen --check OK.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Moves the cross-file symbol-resolution / import-resolution / decl-def-merge
passes — the largest remaining self-contained subsystem in extract.py — into
two new modules, verbatim:
- extractors/models.py: the shared data types (LanguageConfig, the
_Symbol*Fact / _SymbolResolutionFacts dataclasses) and two shared caches
(_WORKSPACE_PACKAGE_CACHE, _JS_CACHE_BYPASS_SUFFIXES). Homed here so both
extract.py and resolution.py import them without a cycle; the mutable
workspace cache keeps a single shared object identity (only ever .clear()'d
in place), verified in the smoke test.
- extractors/resolution.py: 60 resolution functions (_resolve_*, _collect_*,
_apply_symbol_resolution_facts, _disambiguate_colliding_node_ids,
_merge_decl_def_classes, the JS/TS/Python import walkers, tsconfig/workspace
resolution) and their 12 private constants.
extract.py re-exports every moved name, so importers (__main__, watch, tests
that reach into _JS_RESOLVE_EXTS / _resolve_cross_file_imports / the fact
dataclasses) are unchanged. Import direction is strictly extract.py ->
resolution -> {models, base}; AST analysis confirmed no moved non-entry symbol
is referenced from outside its new module.
extract.py 12,632 -> 10,270 LOC (17,054 at the start of this branch, -40%).
Full suite unchanged: 3036 passed, 29 skipped; skillgen --check OK.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
export.py mixed nine independent export targets in one 1,671-LOC module. Move
the two largest self-contained ones into a new graphify/exporters/ package,
verbatim, re-exported from export.py so every importer (from graphify.export
import to_html / push_to_neo4j) is unchanged:
- exporters/html.py: to_html + its private helpers (_html_script, _html_styles,
_hyperedge_script, _viz_node_limit) and MAX_NODES_FOR_VIZ.
- exporters/graphdb.py: push_to_neo4j, push_to_falkordb.
- exporters/base.py: COMMUNITY_COLORS (shared by the HTML/SVG/Obsidian
exporters), homed here so per-format modules and export.py both import it
without a cycle.
Verified: AST closure-privacy analysis, facade object identity, ruff clean.
export.py drops 1,671 -> 962 LOC. Full suite unchanged: 3036 passed, 29 skipped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Continues the graphify/extract.py -> graphify/extractors/ split (MIGRATION.md,
upstream #1212), which already moved blade/zig/elixir/razor. Moves the
independent bespoke extractors whose closures are fully private (no shared
_extract_generic core, no shared mutable caches), verbatim, following the
documented invariants:
dart, rust, go, powershell (+psd1 manifest), fortran, sql,
dm (dm/dmm/dmi/dmf), bash, apex, terraform, sln,
pascal_forms (delphi .dfm + lazarus .lfm), json_config
Each language's private helper funcs and constants move with it; only the
`extract_<lang>` entry points (plus fortran's _cpp_preprocess, which has a
direct unit test) are re-exported from extract.py's facade block, so every
existing importer (__main__.py, watch.py, tests) is unchanged and object
identity is preserved. Registry (extractors/__init__.py) grows 4 -> 22 langs.
Verified: AST closure-privacy analysis (no symbol referenced from outside its
moved set except via the facade); byte-identity of every moved span;
extract._DISPATCH still resolves every extension; ruff clean; skillgen --check
OK. extract.py drops 17,054 -> 13,121 LOC. Full suite unchanged: 3036 passed,
29 skipped (excluding env-only openai tests).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
__main__.py was 5,368 LOC, more than half of it per-platform install/uninstall
machinery interleaved with the CLI dispatcher. Move that subsystem — 68
functions (all *_install/*_uninstall, _copy_skill_file, _platform_skill_destination,
_always_on, etc.) plus 21 module constants (_PLATFORM_CONFIG and the platform
skill/hook payload constants) — into a new graphify/install.py, extracted verbatim.
Behavior-preserving:
- install.py lives in the same package dir as __main__ so packaged-asset lookups
via Path(__file__).parent ("always_on"/"skills") resolve unchanged.
- __main__ re-exports all 89 moved names, so `from graphify.__main__ import
claude_install` (and every other import, incl. private helpers used by tests)
keeps working, with object identity preserved.
- The install cluster was verified (AST) to have zero back-references into
__main__, so no circular import; __main__ imports _PLATFORM_CONFIG et al. one-way.
__main__.py drops 5,368 -> 3,642 LOC. Full suite unchanged: 3036 passed, 29
skipped (excluding the env-only openai tests). ruff check clean; skillgen --check OK.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`graphify extract` on a repo containing docs/papers/images hard-failed when no
LLM backend was configured — even for a user who only wants the code graph. The
only workaround was hand-building a .graphifyignore of everything non-code, which
is onerous (the "not code" set is far larger than the code set).
`--code-only` skips the semantic (doc/paper/image) pass entirely: it indexes the
code via pure local AST (no key required) and reports what it skipped ("skipping
N non-code file(s) ...") rather than silently dropping it. The no-key error on a
mixed repo now also points users at the flag. Code-only was always keyless; this
just makes a *mixed* repo usable without a key instead of failing outright.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`graphify uninstall` (and `graphify claude uninstall`) only looked at
`.claude/settings.json` and root `CLAUDE.md`. A user who relocates the
PreToolUse hook into `.claude/settings.local.json` and the `## graphify`
instructions into `CLAUDE.local.md` / `.claude/CLAUDE.local.md` - so they
are not committed to a shared repo - was left with both behind after
uninstall, which reported "nothing to do".
- `_uninstall_claude_hook` now strips the hook from both `settings.json`
and `settings.local.json` (factored into `_strip_graphify_hook`).
- `claude_uninstall` now strips the `## graphify` section from `CLAUDE.md`,
root `CLAUDE.local.md`, and `.claude/CLAUDE.local.md` (factored into
`_strip_graphify_md_section`), cleaning every present file rather than the
first, and always runs the hook cleanup.
- `_strip_graphify_md_section` reads the two newly-covered files defensively
so a non-UTF-8 or otherwise unreadable file can't abort uninstall.
Unrelated local-only files (no graphify content) are left byte-for-byte
untouched. The `--local` install flag suggested in the issue is left out of
scope; this change makes uninstall symmetric with wherever the config lives.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
merge-graphs prefixed each graph's node ids with `<repo>::` where repo was
`gp.parent.parent.name` — the graphify-out parent dir name. That tag is not
unique across inputs: `src/graphify-out` and `frontend/src/graphify-out` both
yield `src`, so a bare `app` node from a backend `src/app.js` and a frontend
`App.jsx` both became `src::app` and nx.compose silently merged them into one
node (one graph's label/source_file, both graphs' edges) — inventing false
cross-runtime `path` results with no warning.
New `distinct_repo_tags` guarantees a unique prefix per graph: colliding tags
are widened with their own parent dir (`frontend_src`), then an index suffix
backstops any residual duplicate. The handler prints a note when it disambiguates.
Verified end to end: the two `app` nodes now survive as `..._src::app` and
`frontend_src::app` instead of collapsing. Regression tests cover the merge case
and the tag uniquifier (pass-through, widening, and triple-collision fallback).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a regression test for the Date.now()/static-call shape (credit PR #1727 /
@2loch-ness6, who independently found the same fix): a capitalized builtin
receiver must not bind cross-file to a same-spelled user const/class. The same
guard added in 67f4f83 already covers it (verified); this locks the static-call
shape in, while allowing the legitimate same-file const reference.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Correctness batch since 0.9.9: TS/JS builtin-typed receiver no longer collapses
onto a same-named user symbol (#1726); no cross-language calls edges (#1718);
build_merge ambiguous-alias no longer merges unrelated files (#1713); base-class
stubs tagged with origin_file (#1707); Java enum constants as nodes (#1719);
rebuild recovers from a deleted hook cwd (#1703); per-chunk semantic-cache
checkpoint (#1715); SECURITY.md http-transport doc + GRAPHIFY_MAX_GRAPH_BYTES
tests (#1714, #1722). Nine merged PRs plus #1726.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`_resolve_typescript_member_calls` resolves a member call's receiver to a type
definition by casefolded label. For a builtin-typed receiver (`x: Date;
x.getTime()`), `_key("Date")` == "date" matched a same-named user `class DATE` /
`const DATE` in another file, binding the caller to it as a phantom
`references[call]` edge. In a real 3,368-file repo one module-local `const DATE`
accumulated 469 false cross-file edges (degree 472, betweenness 0.435) — a false
god node distorting path/god-node results.
Skip the resolution when the receiver type is an ECMAScript/Python builtin
global (Date, Promise, Map, ...), mirroring the guard the cross-file CALL
resolver already applies (#726). The guard is a strict no-op for genuine user
types, so legitimate constructor-injection member-call resolution (#1316) is
unaffected. Verified across new Date().method(), return-type, and var-decl-type
shapes: zero phantom edges, user DATE degree back to 1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to the previous commit on this branch. That fix's ambiguity check
missed a case: it detects "is this node the file itself" by checking
whether the node's id starts with the file's plain new_stem, but a
same-directory .h/.cpp pair that collides on their shared pre-extension id
gets salted apart by _disambiguate_colliding_node_ids into ids like
"tools_aolserver_utility_h_tools_aolserver_utility" -- no longer a clean
new_stem prefix.
That salted header silently failed to compute an empty suffix, so it never
entered the bare "utility" alias race at all, leaving an unrelated
wwwapi.masque.com/pages/utility.php as the lone (wrong) "unambiguous"
winner -- reproduced exactly against the real depot's
Tools/aolserver/utility.h and .cpp.
Detect "this node IS the file" by label instead: every file node's label is
its own basename regardless of what its id looks like after salting. That
keeps a salted file node in the alias competition, so the real collision
between the C header and the PHP file is correctly caught as ambiguous.
build_from_json's "pre-migration alias index" (#1504) registers each real
file's OLD-style bare-stem id (extension dropped) as an alias so a stale
cached fragment referencing that old form still resolves after an id-scheme
migration. It never checked for collisions: two unrelated real files easily
compute the same bare alias (e.g. "ping.h" and "ping.php" in different
directories both alias to "ping"), and dict.setdefault let whichever file
happened to iterate first in a Python set win, arbitrarily.
A dangling edge with a bare, deliberately-unscoped fallback target (e.g. the
C/C++ extractor's last-resort id for an #include it couldn't resolve to a
real path) would ride that alias onto whatever unrelated file won the
collision -- silently wiring, say, a C++ server file to an unrelated PHP
script, purely because both files happen to share a common basename
somewhere in the corpus.
Now every candidate for an alias is collected before any of them are
committed to norm_to_id, and the alias is only trusted when exactly one real
file claims it. Ambiguous aliases are dropped entirely, so the dangling edge
correctly stays dangling (and gets discarded) instead of merging two
unrelated files -- same "don't guess through ambiguity" principle already
applied to stub-node disambiguation and cross-file call resolution
elsewhere in this codebase.
Found via graphify-practice round 2: a `path` query between two unrelated
symbols routed through exactly this kind of bogus edge, traced back to
Dev/Poker/TDServer/server.cpp's unresolved `#include "ping.h"` /
`#include "utility.h"` landing on unrelated www.masque.com PHP scripts of
the same bare name.
ensure_named_node() tags the sourceless stub it creates for an
unresolved reference with origin_file, so _disambiguate_colliding_node_ids
can tell one file's unresolved reference apart from another's instead of
merging every file's same-named reference onto one shared bare id
(which can then collide with an unrelated same-named real definition
anywhere else in the corpus, since ids are case-normalized and global).
Five call sites duplicated that stub-creation logic inline instead of
calling ensure_named_node() -- Ruby's `Class.new(Super)` and
`class Foo < Base` inheritance, Python inheritance, Kotlin delegation
-specifier inheritance/conformance, and C++ base_class_clause
inheritance -- and none of them were updated when origin_file was added,
so all five still produce the un-disambiguated bare-id stub the fix was
meant to eliminate.
Four of the five sit directly inside _extract_generic, where
ensure_named_node() is already in scope as a closure, so they're
switched to call it directly. The fifth (Ruby's `Class.new(Super)`) is
handled by a separate helper, _ruby_extra_walk(), which doesn't have
that closure in scope; that one gets the same origin_file tag added
directly to its own inline stub dict, matching what ensure_named_node()
already does, without changing the helper's signature.
(A sixth occurrence of the same inline pattern exists in extract_apex(),
a fully separate regex-based extractor with no ensure_named_node()
equivalent of its own -- left out of this fix, which is scoped to the
shared _extract_generic path and its one directly-affiliated helper.)
Added a regression test: two different C++ files each inheriting from
the same undefined base class must produce two distinct stub nodes, not
one shared one. Fails on main (one shared 'base' id for both files),
passes with this fix.
The cross-file call resolver matches raw-call callees against a repo-wide
label index with no language check. In a repo that mixes a web app with a
native Android app, a TSX callback passed by name (register(refreshHeading))
resolved to a same-named Kotlin method and shipped as an INFERRED
indirect_call edge — a phantom the extraction spec itself forbids ('calls
edges MUST stay within one language'). Direct calls from non-JS/TS callers
had the same hole with no gate at all: a bare Python call bound to the lone
same-named Kotlin fun. Found on a production Next.js + Android codebase,
where the phantom edge also inflated the Kotlin node's betweenness enough
to surface it as a top suggested question in GRAPH_REPORT.md.
Resolution candidates are now filtered by language interop family before
the single-candidate/import-evidence logic runs. Families are grouped by
REAL interop so legitimate cross-language resolution keeps working:
Kotlin/Java/Scala/Groovy share the JVM, C/C++/Objective-C/CUDA share
headers (Swift bridges to ObjC), and JS/TS variants plus Vue/Svelte/Astro
SFCs compile into one module graph. Candidates whose family is unknown
(no source_file, non-code nodes) are never filtered, preserving the
previous permissive behavior, and callers with an unmapped extension skip
the guard entirely.
The per-chunk checkpoint feature added merge_existing without test coverage.
Add tests asserting merge_existing=True unions a file's slices across chunks and
the default still overwrites (the authoritative final write in extract).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Semantic extraction only wrote to the cache once, at the very end of the
run (save_semantic_cache in __main__ after extract_corpus_parallel returns).
A run interrupted partway — a crash, a kill, or a claude-cli/API run that
exits when it hits a rate limit — therefore lost every completed chunk and
restarted from scratch. On a large corpus with a slow local backend this can
throw away many hours of work.
Persist each chunk's results to the semantic cache as soon as it completes,
in both the serial and threaded paths of extract_corpus_parallel. Add a
merge_existing option to save_semantic_cache so a file split into slices
across several chunks accumulates its slices instead of the later chunk
overwriting the earlier one. The checkpoint is best-effort (a cache write
error never aborts extraction) and can be disabled with
GRAPHIFY_NO_INCREMENTAL_CACHE. Default behaviour of save_semantic_cache is
unchanged (merge_existing defaults to False).
What changed
- Stabilize relative rebuild execution before graphify-out queue/lock setup.
- Recover from deleted transient hook CWDs when GRAPHIFY_REPO_ROOT points at the repository root.
- Fail cleanly when the current working directory is gone and no repo root fallback is available.
- Add regression coverage for both fallback and clean-failure paths.
Why
- Detached post-commit/post-checkout rebuilds can inherit a transient CWD that is deleted before the background process starts.
- _rebuild_code previously used relative graphify-out paths before Path.cwd()/watch_path resolution could be handled, causing FileNotFoundError: [Errno 2] No such file or directory.
Validation
- .venv/bin/pytest tests/test_watch.py -q: 54 passed, 2 skipped.
- .venv/bin/ruff check graphify/watch.py tests/test_watch.py: passed.
- .venv/bin/python -m py_compile graphify/watch.py tests/test_watch.py: passed.
- env -u PYTHONPATH -u PYTHONHOME PYTHONHASHSEED=0 .venv/bin/pytest -q: 2904 passed, 30 skipped.
Notes
- Full suite without PYTHONHASHSEED=0 exposed an unrelated ordering-sensitive labeling test; with the deterministic hash seed used by graphify hooks, the suite passes.
_max_graph_file_bytes() resolves the graph-load memory-bomb cap and parses the
GRAPHIFY_MAX_GRAPH_BYTES env var (plain bytes, MB/GB suffix, fallbacks), but had
no behavioral tests — only the default constant was asserted.
- Add tests for: unset/blank -> default, plain integer, MB/GB suffix (binary),
case-insensitive and space-tolerant suffixes, unparseable -> default, and
non-positive -> default.
- Fix the docstring, which described the suffix as "decimal multipliers of 1024"
(self-contradictory). MB/GB are treated as binary (MiB/GiB); the tests lock
that behavior in.
No behavior change.
SECURITY.md said graphify never runs a network listener and only
communicates over stdio. README.md documents an opt-in --transport http
mode (with --host/--api-key flags) for sharing one server across a team,
which does open a network listener. Update the claim to describe the
default (stdio, no listener) and the documented opt-in exception.
Fixes since 0.9.8: explain punctuated-label matching (#1704); surface code files
with no AST extractor instead of dropping them silently (#1689); consistent
AST-extraction progress denominator (#1693); no dangling Obsidian wikilinks in
GRAPH_REPORT.md by default (#1712); MATLAB .m no longer force-parsed by the
Objective-C grammar (#1702); corrected the /graphify usage comment in the skill
files (#1681); surface unclassified files (Dockerfile/Makefile/...) instead of
vanishing (#1692).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`.m` is shared by Objective-C implementation files and MATLAB, but the extractor
dispatch routed every `.m` to extract_objc unconditionally. Feeding real MATLAB
to the Objective-C tree-sitter grammar yields root.has_error and garbage
nodes/edges — worse than skipping, because it pollutes the graph with wrong data.
_get_extractor now content-sniffs `.m` the same way `.h` already is: a genuine
Objective-C `.m` carries an ObjC directive (@implementation/@interface/@import/
#import) and still routes to extract_objc; a `.m` without one (MATLAB, Octave)
gets no extractor, so it is surfaced by the no-AST-extractor warning (#1689)
instead of mis-parsed. `.mm` is unambiguously Objective-C++ and is left untouched.
This stops the wrong-by-omission garbage; wiring a real tree-sitter-matlab
extractor (the issue's primary ask) remains a follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Usage block's bare `/graphify` comment read "full pipeline on current
directory -> Obsidian vault", contradicting Step 6 (HTML viz always; Obsidian
vault only when --obsidian is explicitly given). The comment predated the
opt-in change and told agents a bare /graphify produces a vault. It now reads
"full pipeline on current directory (HTML viz; add --obsidian for a vault)".
Fixed at the skillgen source (fragments/core/core.md + the aider monolith
override), added a sanctioned-diff predicate so the monolith round-trip guard
allows the change, and re-blessed the expected/ snapshots. Every generated
skill-*.md now carries the corrected comment; the only content change across
all rendered files is this one line.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When classify_file() returned None — an extensionless, non-shebang file
(Dockerfile, Gemfile, Makefile, Rakefile, LICENSE, ...) or an unsupported
extension — the file left no trace at all: not counted, not listed, nothing.
A user had no way to tell from graphify's output that those files were even
considered.
detect() now collects these into an "unclassified" list in its result, and
`graphify extract` prints a one-line summary after the scan counts:
"N file(s) not classified (no supported extension or shebang), skipped:
Dockerfile, Makefile, ...". Real code/docs are unaffected. This is the
visibility half of the issue; wiring up extractors/manifest handling for
Dockerfile/Makefile-style files remains a separate feature.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GRAPH_REPORT.md rendered the Community Hubs section as Obsidian wikilinks, but
the `_COMMUNITY_*.md` notes they target are only created by the opt-in
`--obsidian` export — and the report is written at build time, before any export
runs. So on a default run every link dangled: inside an Obsidian vault they
spawned phantom `_COMMUNITY_*` nodes in the graph view, and outside Obsidian they
rendered as literal brackets that navigate nowhere.
`generate()` now takes `obsidian: bool = False`; by default the hubs render as a
plain list, and the wikilink form is emitted only when a caller opts in. The
Obsidian export's own community notes already cross-link each other, so the vault
stays navigable without the report's links. Mirrors the #1444/#1465 portability
fix that was applied to `export wiki`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Intermediate progress lines count against len(uncached_work) ("X/Y uncached
files"), but the final line switched to total_files ("Y/Y files"), which includes
cached hits and files with no extractor that never entered uncached_work. On a
large corpus with unsupported-language files, the total jumped upward right after
99% with no explanation. Both the parallel and sequential final lines now report
the same uncached_work denominator, so the count no longer appears to change
mid-run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extensions like .r/.R (also .ejs, .ets) are in CODE_EXTENSIONS, so those files
are classified as code and counted in the scan, but there is no entry for them
in the extractor dispatch — so they produce zero nodes and are silently absent
from the graph while the CLI still reports success. The #1666 zero-node warning
deliberately skips them (it only fires when an extractor exists), so nothing
surfaced the gap.
extract() now emits a warning listing the offending extensions and counts
("N file(s) are classified as code but graphify has no AST extractor ...: .r (2)")
so a primarily-R (or .ejs/.ets) corpus no longer looks fully mapped when it is
not. Grouped by extension, fires only for files actually present with no
extractor (today: .ejs, .ets, .r). Adding real grammars for these remains the
follow-up; this removes the silent-data-loss now.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`_find_node` built its search term with `_search_tokens` (\w+ tokenization), so
"blockStream.ts" became "blockstream ts" (space where the '.' was) while the
node's stored `norm_label` keeps punctuation ("blockstream.ts"). The verbatim
case is already rescued by the `term == label_tokens` tier (the node label
tokenizes the same way), but that is a coincidence: if `label` and `norm_label`
diverge, an exactly-typed punctuated label fails to resolve through `explain`
even though `path`/`query` find it.
Add a punctuation-preserving `norm_query` (`_strip_diacritics(label).lower()`)
matched against `norm_label`/`bare_label` across the exact/prefix/substring tiers
(and fed to the trigram prefilter so candidates are not missed). Purely additive,
symmetric with how norm_label is stored. Regression tests cover the verbatim
file-label case and the label/norm_label divergence case that only norm_query
resolves.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Badges reranked for click-through and split: lean top strip (PyPI, Downloads,
Discord, LinkedIn, YC S26) with trust signals leading and social/company links
at the tail; a "Community and links" footer (Discord, X, Sponsor, Book); CI
badge dropped, book also linked from "Learn more".
- Hero leads with three differentiator bullets, corrected to match the codebase:
code is tree-sitter AST (deterministic, no LLM, local), while docs/PDFs/images/
video use your assistant's model or a configured API key for a semantic pass.
Removed the inaccurate "local embedder" claim (there is no embedder; dedup is
MinHash/LSH and there is no vector store) and qualified "zero LLM tokens" to the
code path. "Not a vector index" is accurate.
- Added a 30-second quickstart (install + graphify install + /graphify .) above
the fold, moved "See it in action" above "What it does" so differentiation is
shown before it is enumerated, and compressed the "works in" list to one line.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a wide matrix over _run_hook_guard covering the search/read detection
boundaries and the gemini BeforeTool contract:
- search: grep-family variants (grep/pgrep/egrep/fgrep/ripgrep), token matches
(rg/find/fd/ack/ag with trailing space), piped commands; and silence for
non-search commands, empty/missing/non-string command, 'find' without a
trailing space, 'ag' mid-word, top-level vs nested tool_input, non-dict
tool_input, and no-graph.
- read: source/framework extensions, uppercase and multi-dot names, Windows
backslash paths, glob patterns; and silence for .json (not .js), lockfiles,
images, extensionless files, an extension on a directory segment, targets
under the (default and custom-named) output dir, and no-graph.
- fail-open: malformed/empty/binary stdin and a throwing graph-existence check
never crash or block.
- gemini: always returns {"decision":"allow"}, nudges only with a graph, and
stays "allow" even if the check throws.
- dispatch/exit/encoding via real subprocess: missing/unknown mode exits 0
silently, every mode exits 0 (never blocks), and the read nudge's em dash
round-trips as valid UTF-8.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Gemini hook was a `python -c "..."` one-liner that (a) depended on a bare
`python` being on PATH (frequently `python`/`py` or absent on Windows) and
(b) embedded backticks + escaped quotes that Windows PowerShell mangles. Same
fix as the Claude/Codebuddy hooks: it now invokes `graphify hook-guard gemini`
via the absolute exe path.
The gemini mode always returns {"decision":"allow"} (never blocks a tool) and
appends the graph nudge as additionalContext only when graph.json exists — the
BeforeTool contract Gemini expects, byte-identical to the old payload. It takes
no stdin, honors GRAPHIFY_OUT, and the matcher stays "read_file|list_directory"
so install/uninstall find and replace old hooks unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The hooks were inline POSIX bash (case/esac, [ -f ], single-quoted echo),
which Windows cmd.exe/PowerShell cannot parse. On Windows the hook failed
silently, so the "run `graphify query` before grepping/reading raw files"
nudge was never injected and users fell back to manual /graphify.
The detection logic (grep-command match; source/doc extension match; skip if
the target is under the output dir; require graph.json to exist) moved into a
shell-agnostic `graphify hook-guard <search|read>` subcommand, invoked via the
absolute exe path resolved by _resolve_graphify_exe() — the exact pattern the
codex hook already uses. A single console-script invocation has no shell syntax,
so it parses identically under sh, cmd.exe and PowerShell.
Behavior on macOS/Linux is unchanged: the nudge payload is byte-identical
(compact JSON, same additionalContext text), matchers stay "Bash"/"Read|Glob"
so install/uninstall still find and replace old hooks, and the command still
contains "graphify". The graph-exists check now honors GRAPHIFY_OUT instead of
the hardcoded graphify-out/ path. Codex stays a no-op there (hook-check) because
Codex Desktop rejects additionalContext.
Detection is fully unit-tested (ported test_read_hook.py + new test_search_hook.py,
byte-identical output on POSIX); Windows execution itself is not testable in CI
here, but the mechanism is now shell-independent by construction.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@sub4biz verified against the live DeepSeek API that deepseek-v4-flash (and
v4-pro) have thinking ENABLED by default, contradicting the built-in config's
stale "non-thinking" comment (now corrected).
The naive fix (mirror the kimi branch and force thinking off) is the wrong
call: @sub4biz's production testing on real corpora found that disabling
thinking removes a rare reasoning-leak failure — which the adaptive
extraction/labeling retry already recovers from — but trades it for far more
frequent benign truncation AND measurably lower extraction quality and file
coverage, confirmed by a blind second reviewer.
So thinking stays ON by default (quality/coverage), with a documented opt-in
`GRAPHIFY_DISABLE_THINKING=1` for users who prefer run-to-run stability. Applies
to reasoning-capable OpenAI-compatible backends at both extra_body sites
(extraction + labeling). An explicit providers.json extra_body still wins, and
the moonshot/kimi branch is unchanged (it must disable thinking or content is empty).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@krishnateja7 root-caused this precisely: the files were never reaching
extraction, so the 0.9.7 no-cache-on-empty mitigation could not surface them.
Two discovery-layer filters were the cause:
(a) A bare `snapshots/` directory was pruned as a Jest/Vitest artifact, which
killed legitimate code namespaces like a Rails `app/services/snapshots/`. It is
now pruned only when it actually contains `.snap` files or sits directly under a
JS test root (`__tests__`/`__test__`). `__snapshots__` stays unconditionally pruned.
(b) `_is_sensitive` dropped files on a bare name-keyword hit (device_token.rb,
passwords_controller.rb) even when `classify_file` had already resolved them to
source code. A genuine programming-language source file is now exempt from the
weak keyword heuristic, while real secret stores in data/config formats
(credentials.json, secrets.yaml, .env, .pem, ...) are still caught — those route
through the CODE path for manifest parsing but are deliberately not exempted.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An animated SVG that mirrors the real `graphify path` output: a terminal
types the query on the left while the answer draws itself hop by hop across
the knowledge graph on the right, ending on "3 hops. Zero files opened."
Pure SMIL inside a self-contained dark card, so it renders inline on both
GitHub themes with no JS and no external fonts. Palette is brand-only:
muted emerald (sampled from the logo) as the single accent, no neon.
Generated by scripts/gen_demo_path.py (re-run to regenerate; STATIC=1 bakes
a still frame for QA).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the Hebrew translation under docs/translations/README.he-IL.md and
links it from the language row (30 -> 31 languages). Merged from PR #1639
by @AdirBuskila; the language-row link was adapted to the current README
structure by hand since the English README changed after the PR was opened.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two related fixes in the community-labeling path:
#1690 (thanks @vdgbcrypto): a truncated or slightly malformed reply no
longer discards the whole batch with "Expecting value: line 1 column 6".
`_parse_label_response` now salvages the complete `"id": "name"` pairs from
a reply that failed a strict `json.loads` (e.g. one truncated mid-object),
raising only when no pairs can be recovered. The per-batch token budget was
also raised (256 + 48*n, was 64 + 24*n) so models that prepend a short
preamble have headroom to finish the JSON. The exact provider truncation
could not be reproduced without a live key; the parser and budget address
the mechanism.
#1694 (thanks @sub4biz): cluster-only mode reported a hardcoded
`0 input * 0 output` token cost because the labeling LLM calls were never
accounted for. `_call_llm` now accumulates per-response usage into an
optional accumulator threaded through the labeling path and surfaced in
GRAPH_REPORT.md. Backends that do not return usage (the Claude Code CLI)
contribute nothing, which is honest rather than estimated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A stalled local model wedged for `timeout * (max_retries + 1)`, which with
the default 6 retries turned one long stall into a ~21-minute block with no
progress. `_call_openai_compat` now defaults the Ollama backend to zero
client-side retries (a local model that stalls will not un-stall on retry);
set `GRAPHIFY_MAX_RETRIES` to opt back in. Other backends are unchanged.
This bounds the wait; the underlying stall is driven by the model server
and is non-deterministic, so it is not eliminated. Thanks @Kyzcreig for the report.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`_TOKENIZER.encode(content)` raises ValueError by default when the text
contains a special token such as `<|endoftext|>`, so a doc or corpus that
merely mentions these strings crashed the entire semantic pass. Both
`encode` sites in `_estimate_file_tokens` now pass `disallowed_special=()`
so such text is tokenized as ordinary bytes. Thanks @Kyzcreig for the report.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
safishamsi