#655 and #656 already fixed in cache.py and security.py.
#657: add rationale to file_type schema in all 12 skill variants; warn against inventing concept.
#658: add explicit chunk-merge step with token summation before save_semantic_cache.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_import_js only created a file→file imports_from edge, leaving named
imported symbols (classes, error types, constants) as degree-1 orphans
with no connection to their consumers.
Track resolved_path when the import target resolves to a local file
(relative path or tsconfig alias). After emitting the file-level edge,
walk import_clause → named_imports → import_specifier to emit a
file → symbol EXTRACTED edge for each named specifier, using the same
_make_id(target_stem, name) key that _extract_generic uses when defining
the symbol.
On a 1696-file TypeScript monorepo this raised edge count from 8530 to
11267 (+2737 edges) and reduced isolated communities from 878 to 604.
The python3 -c "..." approach still failed on Windows Conda (no python3
shim) and PowerShell (JSON curly brace/quote parsing). Replace the inline
command with 'graphify hook-check' — a new shell-agnostic subcommand that
prints the hookSpecificOutput JSON if graph.json exists and exits 0 silently
if not. Works on PowerShell, cmd.exe, macOS, and Linux with no quoting or
interpreter-name issues.
Users must re-run 'graphify codex install' to regenerate the hook.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Bypasses the node-count safety check in to_json for refactors that
legitimately shrink the graph (renames, package deletions). Also
honored by post-commit and post-checkout hooks via GRAPHIFY_FORCE=1.
Implements the approach from #639 (targeted at v5) adapted for v6.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
`extract_kotlin` previously emitted zero `calls` edges (and zero
`raw_calls` entries) on the current PyPI grammar. The Kotlin branch
of `walk_calls` only matched node type `simple_identifier`, but
PyPI's `tree_sitter_kotlin` produces `identifier` for the equivalent
plain-identifier node. The `simple_identifier` ↔ `identifier` rename
is a generation gap between tree-sitter-kotlin grammar versions —
older forks (and the JVM `io.github.bonede:tree-sitter-kotlin`
binding) still use `simple_identifier`.
Accept both names so the extractor works across grammar generations.
Also widens `_KOTLIN_CONFIG.name_fallback_child_types` for the same
reason (defensive — currently the `name` field path covers
class/function name resolution, but if that field is dropped in a
future grammar update the fallback would face the same rename).
Tested against `tests/fixtures/sample.kt`: edges go from 6
(file-contains + class-method only) to 10 (adds 4 in-file `calls`
edges resolved by the walker:
- .get() → .buildRequest() @ L8
- .post() → .buildRequest() @ L12
- createClient() → Config @ L21
- createClient() → HttpClient @ L22).
A new regression test `test_kotlin_emits_in_file_calls` asserts the
four edges so this exact bug can't recur.
Found via graphify-kmp (Kotlin Multiplatform port of graphify) —
its `PythonParityTest` flagged 4 KMP-only edges that Python missed.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace bash-only [ -f ] file check with a cross-platform Python one-liner
so the hook works on Windows where cmd.exe has no [ builtin.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
extract.py: build name -> all candidates map instead of last-write-wins dict.
Skip cross-file INFERRED calls where the callee name resolves to 2+ nodes
(common names like log/execute/find with no import evidence to pick the right
target) — prevents spurious edges from polluting god_nodes degree ranking.
__main__.py: wrap cluster-only to_html in try/except ValueError so large
graphs (>5000 nodes) don't crash the cluster command.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
watch.py: filter preserved nodes by ID membership in new AST output instead
of file_type — INFERRED/AMBIGUOUS nodes on code files also carry file_type=code
and were being wrongly dropped, triggering the to_json safety check refusal.
hooks.py: detach post-commit and post-checkout rebuilds with nohup + disown
so git commit returns immediately instead of blocking for the full rebuild
duration. Rebuild log written to ~/.cache/graphify-rebuild.log.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit adds full VB.NET language support to graphify, raising the
supported language count from 25 to 26. The implementation follows the
established LanguageConfig pattern used by all other tree-sitter-backed
extractors.
New dependency:
- Adds optional extra [vbnet] backed by tree-sitter-vbnet (published
to PyPI at https://pypi.org/project/tree-sitter-vbnet/0.1.0/).
Install with: pip install graphifyy[vbnet]
graphify/detect.py:
- Added .vb to CODE_EXTENSIONS so VB.NET files are discovered during
corpus ingestion and file-system watching.
graphify/extract.py:
- _import_vbnet(): import handler for imports_statement nodes; emits
imports edges using the namespace_name child text.
- _vbnet_extra_walk(): extra-walk hook that intercepts namespace_block
nodes, emits a namespace node, and recurses.
- _VBNET_CONFIG: full LanguageConfig covering class_block / module_block /
structure_block / interface_block as class types; method_declaration /
constructor_declaration / property_declaration as function types;
invocation call nodes with target/member_access fields.
- VB.NET-specific branches in _extract_generic:
* Class body: VB.NET has no wrapper body node; inherits and implements
are named fields directly on the class_block. Emits separate inherits
and implements edges for each base type, stripping generic arguments.
* Constructor name: constructor_declaration carries no name field in
the grammar; always resolves to New.
* Function body: uses the declaration node itself as body sentinel so
the call-graph pass can find invocations inside methods.
- extract_vbnet(path): public wrapper that delegates to _extract_generic.
- _DISPATCH['.vb']: routes .vb files to extract_vbnet.
pyproject.toml:
- Added vbnet = ['tree-sitter-vbnet'] optional dependency group.
- Added 'tree-sitter-vbnet' to the all extra.
tests/fixtures/sample.vb:
- New fixture file exercising: Imports statements, Namespace block,
Interface, Class with Inherits + Implements, Module, Structure,
Sub/Function/Property methods, and method calls.
tests/test_languages.py:
- Added 13 tests covering: no-error, class/interface/module/structure
detection, method detection, imports relation, inherits edge,
implements edge, and no-dangling-edges invariant.
README.md:
- Updated language count 25 to 26.
- Added VB.NET to language list and file-extension table.
- extract_sql(): deterministic tree-sitter extraction of tables, views,
functions, foreign key references, and FROM/JOIN reads_from edges
- .sql added to CODE_EXTENSIONS and dispatch table
- tree-sitter-sql added as optional dep under [sql] extra
- xlsx_extract_structure(): extracts sheet/table/column nodes from .xlsx
(utility — pipeline wiring in follow-up)
- 6 new SQL tests, 447 total passing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Token-budget chunking cuts the truncation rate but doesn't eliminate
it. Output token cost scales with extractable concept density rather
than input tokens — a chunk that lands on a directory of dense design
docs can pack under the input budget while needing more than
`max_completion_tokens=8192` to express every named concept, so the
response is truncated mid-string and `_parse_llm_json` returns an
empty fragment.
Pre-tuning chunk size to be conservative enough that this never
happens leaves throughput on the table for the common case. Adding a
hard `max_files_per_chunk` cap on top of `token_budget` reintroduces
the "tune a static constant" problem the previous commit set out to
fix.
The fix uses the API's own truncation signal:
1. `_call_openai_compat` and `_call_claude` now expose `finish_reason`
on the result dict (Anthropic's `stop_reason == "max_tokens"` is
normalised to `"length"`).
2. `_extract_with_adaptive_retry` checks it: when truncated, splits
the chunk in half and recurses on each half. Recursion is bounded
by `max_retry_depth` (default 3 → at most 8x fanout per top-level
chunk).
3. Single-file chunks that truncate can't recover and surface a
warning rather than infinite-loop.
4. `extract_corpus_parallel` routes every chunk through the retry
wrapper. The `on_chunk_done` callback fires once per top-level
chunk with the merged result — recursive splits are invisible to
callers.
Three independent improvements to extract_corpus_parallel:
1. Token-aware chunking. Replaces `chunk_size=20` static packing with
a greedy packer keyed on `token_budget` (default 60_000), grouped
by parent directory so related artefacts share a chunk. Pass
`token_budget=None` to fall back to fixed-count packing.
2. Optional tiktoken (added to the [kimi] extra). When available,
`_estimate_file_tokens` uses cl100k_base for accurate counts;
without it, the existing chars/4 heuristic kicks in. Kimi-K2 ships
a tiktoken-based tokenizer so estimates against Moonshot are very
close to truth.
3. True parallelism. The function name said "parallel" but the body
was a sequential for-loop. Now uses ThreadPoolExecutor capped at
`max_concurrency` (default 4 — conservative against provider rate
limits). `on_chunk_done(idx, total, result)` still fires once per
chunk with the original submission idx so progress UIs work
unchanged. `max_concurrency=1` skips the pool to preserve
sequential semantics.
Plus failure tolerance: a chunk raising is now caught, logged to
stderr, and the run continues. Other chunks' results merge as normal.
On a 162-file repo (~125k words), the same work that took ~36 min
sequential under the old code finishes in ~7 min.
Safi