extract.py: build name -> all candidates map instead of last-write-wins dict.
Skip cross-file INFERRED calls where the callee name resolves to 2+ nodes
(common names like log/execute/find with no import evidence to pick the right
target) — prevents spurious edges from polluting god_nodes degree ranking.
__main__.py: wrap cluster-only to_html in try/except ValueError so large
graphs (>5000 nodes) don't crash the cluster command.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
watch.py: filter preserved nodes by ID membership in new AST output instead
of file_type — INFERRED/AMBIGUOUS nodes on code files also carry file_type=code
and were being wrongly dropped, triggering the to_json safety check refusal.
hooks.py: detach post-commit and post-checkout rebuilds with nohup + disown
so git commit returns immediately instead of blocking for the full rebuild
duration. Rebuild log written to ~/.cache/graphify-rebuild.log.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit adds full VB.NET language support to graphify, raising the
supported language count from 25 to 26. The implementation follows the
established LanguageConfig pattern used by all other tree-sitter-backed
extractors.
New dependency:
- Adds optional extra [vbnet] backed by tree-sitter-vbnet (published
to PyPI at https://pypi.org/project/tree-sitter-vbnet/0.1.0/).
Install with: pip install graphifyy[vbnet]
graphify/detect.py:
- Added .vb to CODE_EXTENSIONS so VB.NET files are discovered during
corpus ingestion and file-system watching.
graphify/extract.py:
- _import_vbnet(): import handler for imports_statement nodes; emits
imports edges using the namespace_name child text.
- _vbnet_extra_walk(): extra-walk hook that intercepts namespace_block
nodes, emits a namespace node, and recurses.
- _VBNET_CONFIG: full LanguageConfig covering class_block / module_block /
structure_block / interface_block as class types; method_declaration /
constructor_declaration / property_declaration as function types;
invocation call nodes with target/member_access fields.
- VB.NET-specific branches in _extract_generic:
* Class body: VB.NET has no wrapper body node; inherits and implements
are named fields directly on the class_block. Emits separate inherits
and implements edges for each base type, stripping generic arguments.
* Constructor name: constructor_declaration carries no name field in
the grammar; always resolves to New.
* Function body: uses the declaration node itself as body sentinel so
the call-graph pass can find invocations inside methods.
- extract_vbnet(path): public wrapper that delegates to _extract_generic.
- _DISPATCH['.vb']: routes .vb files to extract_vbnet.
pyproject.toml:
- Added vbnet = ['tree-sitter-vbnet'] optional dependency group.
- Added 'tree-sitter-vbnet' to the all extra.
tests/fixtures/sample.vb:
- New fixture file exercising: Imports statements, Namespace block,
Interface, Class with Inherits + Implements, Module, Structure,
Sub/Function/Property methods, and method calls.
tests/test_languages.py:
- Added 13 tests covering: no-error, class/interface/module/structure
detection, method detection, imports relation, inherits edge,
implements edge, and no-dangling-edges invariant.
README.md:
- Updated language count 25 to 26.
- Added VB.NET to language list and file-extension table.
- extract_sql(): deterministic tree-sitter extraction of tables, views,
functions, foreign key references, and FROM/JOIN reads_from edges
- .sql added to CODE_EXTENSIONS and dispatch table
- tree-sitter-sql added as optional dep under [sql] extra
- xlsx_extract_structure(): extracts sheet/table/column nodes from .xlsx
(utility — pipeline wiring in follow-up)
- 6 new SQL tests, 447 total passing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Token-budget chunking cuts the truncation rate but doesn't eliminate
it. Output token cost scales with extractable concept density rather
than input tokens — a chunk that lands on a directory of dense design
docs can pack under the input budget while needing more than
`max_completion_tokens=8192` to express every named concept, so the
response is truncated mid-string and `_parse_llm_json` returns an
empty fragment.
Pre-tuning chunk size to be conservative enough that this never
happens leaves throughput on the table for the common case. Adding a
hard `max_files_per_chunk` cap on top of `token_budget` reintroduces
the "tune a static constant" problem the previous commit set out to
fix.
The fix uses the API's own truncation signal:
1. `_call_openai_compat` and `_call_claude` now expose `finish_reason`
on the result dict (Anthropic's `stop_reason == "max_tokens"` is
normalised to `"length"`).
2. `_extract_with_adaptive_retry` checks it: when truncated, splits
the chunk in half and recurses on each half. Recursion is bounded
by `max_retry_depth` (default 3 → at most 8x fanout per top-level
chunk).
3. Single-file chunks that truncate can't recover and surface a
warning rather than infinite-loop.
4. `extract_corpus_parallel` routes every chunk through the retry
wrapper. The `on_chunk_done` callback fires once per top-level
chunk with the merged result — recursive splits are invisible to
callers.
Three independent improvements to extract_corpus_parallel:
1. Token-aware chunking. Replaces `chunk_size=20` static packing with
a greedy packer keyed on `token_budget` (default 60_000), grouped
by parent directory so related artefacts share a chunk. Pass
`token_budget=None` to fall back to fixed-count packing.
2. Optional tiktoken (added to the [kimi] extra). When available,
`_estimate_file_tokens` uses cl100k_base for accurate counts;
without it, the existing chars/4 heuristic kicks in. Kimi-K2 ships
a tiktoken-based tokenizer so estimates against Moonshot are very
close to truth.
3. True parallelism. The function name said "parallel" but the body
was a sequential for-loop. Now uses ThreadPoolExecutor capped at
`max_concurrency` (default 4 — conservative against provider rate
limits). `on_chunk_done(idx, total, result)` still fires once per
chunk with the original submission idx so progress UIs work
unchanged. `max_concurrency=1` skips the pool to preserve
sequential semantics.
Plus failure tolerance: a chunk raising is now caught, logged to
stderr, and the run continues. Other chunks' results merge as normal.
On a 162-file repo (~125k words), the same work that took ~36 min
sequential under the old code finishes in ~7 min.
Adds docs/docker-mcp-sqlite.md, a reproducible recipe for installing the
SQLite MCP server into Docker MCP Toolkit so any connected MCP client
(Claude Code, Cursor, VS Code, etc.) gains six SQLite tools alongside
graphify's knowledge-graph tools.
Notes the catalog has two SQLite images at time of writing: `mcp/sqlite`
(marked Archived but works) and `mcp/sqlite-mcp-server` (broken
entrypoint). Recommends the working one.
Linked from README.md under a new 'Optional integrations' section.
This is unrelated to the upstream graphify pipeline; it lives as an
optional companion runbook for users who want a lightweight persistent
SQL workspace exposed to their MCP-aware AI clients.
Co-Authored-By: Oz <oz-agent@warp.dev>
AST and semantic entries now write to cache/ast/ and cache/semantic/
respectively. Previously both used the flat cache/ dir causing semantic
results to overwrite AST entries for code files on mixed corpora, making
the shrink guard fire on every subsequent update run.
Migration: load_cached falls back to legacy flat cache/ for AST reads
so existing cache entries are not lost on upgrade.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Grep and Glob tools removed in CC v2.1.117; searches now go through Bash.
Hook now reads stdin tool_input and pattern-matches on search commands.
Uninstall/reinstall handles both old and new matcher for clean upgrades.
Closes#578
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
import(`./handlers/${name}`) previously produced a garbage edge to a
path containing the unresolved ${name} expression. Now detects
template_substitution child nodes and breaks without emitting an edge.
Static template literals (no interpolation) still resolve correctly.
Adds 2 new tests: one asserting dynamic templates produce no edge,
one asserting static templates resolve like plain strings.
Adds _dynamic_import_js() helper (65 lines) that detects import() call
expressions in JS/TS, resolves the module path (same logic as static
imports including .js→.ts mapping and tsconfig aliases), and emits
imports_from edges from the enclosing function. Hooked into walk_calls
for JS/TS configs.
Also adds tests/fixtures/dynamic_import.ts fixture and 5 new tests
in tests/test_languages.py (all passing alongside 110 existing tests).
- #550: _file_stem() includes parent dir to prevent node ID collisions for same-named files
- #555: extract() relativizes source_file paths before returning for cross-machine portability
- #562: to_json() returns bool; _rebuild_code() writes report/html only if json succeeded
- #563: skill prompts store rationale as node attribute, not separate node; enforce calls direction
- #566: Show All / Hide All buttons added to HTML community panel
- #575: _import_js() resolves tsconfig.json compilerOptions.paths aliases before external fallback
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
html.escape() the values that land in <title> and <h1>, and replace </
with <\/ in the JSON embedded inside <script> so crafted graph labels
or --label values cannot break out. Mirrors the _js_safe() pattern in
export.py.
Reported by Qodo on PR #557.
