Third call site that re-implemented the same .js→.ts rewrite in
isolation. Previously only handled the explicit .js→.ts case; bare
paths, multi-dot helper files, and alias-resolved dynamic imports
all dropped silently.
Now uses _resolve_js_module_path on both branches (relative and
alias) — same shape as the static-import and Svelte regex paths.
Real-world impact: TS files using `await import('./foo')` patterns
for code splitting (e.g. lazy-loading a profanity check) now produce
edges to the resolved target.
When both a file (foo.ts) and a directory (foo/) exist at the same path,
both TypeScript and Vite prefer the file. The previous ordering checked
directory first and fell through unchanged when the directory had no
index, silently dropping every import like 'from ./auth' when an
auth/ subdirectory existed alongside auth.ts.
The generalized resolver already handles .svelte.js because the append
loop iterates _JS_RESOLVE_EXTS = (.ts, .tsx, .svelte, .js, .jsx, .mjs).
Adds three explicit tests to pin the behaviour and document the priority
choice:
- test_resolve_svelte_to_svelte_js_for_javascript_rune_files
JS-only Svelte 5 project: .svelte → .svelte.js works the same
way as .svelte.ts in TS projects. No special-casing needed —
the generalized append loop covers both.
- test_resolve_svelte_prefers_svelte_ts_over_svelte_js
Hybrid case (both files exist, e.g. .svelte.ts source plus
.svelte.js build artifact): .ts wins. Documents the deliberate
source-first priority — graphify is a source-code tool, not a
runtime resolver, so we differ from Vite's default JS-first order.
- test_resolve_real_svelte_file_wins_over_svelte_ts_sibling
Existence check short-circuits before any extension append, so a
real .svelte file always wins over a .svelte.ts sibling.
Two changes that landed together because they share the same code path:
1. Generalize the bare-path append to handle multi-dot filenames
The previous resolver only appended extensions when path.suffix == ""
(truly bare paths). Real codebases use a lot of multi-dot patterns:
foo.shared.ts ← imported as './foo.shared'
foo.config.ts ← imported as './foo.config'
foo.compile.ts ← imported as './foo.compile'
foo.integration.ts ← imported as './foo.integration' (test helper)
foo.triggers.ts ← imported as './foo.triggers' (test helper)
foo.svelte.ts ← imported as './foo.svelte' (Svelte 5 rune)
foo.d.ts ← imported as './foo.d' (ambient types)
For all of these, .suffix is the meaningful middle segment (.shared,
.config, .integration, etc.) — not in the .js/.jsx/.svelte handled
list, so the resolver fell through and the import dropped to a phantom.
The fix unifies the bare-path and .svelte→.svelte.ts cases into a
single rule: append each candidate extension to the FULL filename, not
to the stripped stem. This subsumes:
bare path: foo → foo.ts
Svelte rune file: foo.svelte → foo.svelte.ts
multi-dot helper: foo.shared → foo.shared.ts
ambient declaration: foo.d → foo.d.ts
No behaviour change for paths that DO exist (.is_file() short-circuit)
or for the .js→.ts / .jsx→.tsx convention (handled before the append
loop so we don't accidentally match foo.js → foo.js.ts when foo.ts
is the real file).
2. Rename _resolve_with_extensions → _resolve_js_module_path
The function is JS/TS/Svelte-specific (Vite resolver order, mirrors the
convention used by _import_js, _JS_CONFIG, _TS_CONFIG). The original
name suggested it was a generic path utility. Renamed to make scope
explicit and align with the existing _import_js / _JS_CONFIG naming
pattern. Constants renamed to match: _JS_RESOLVE_EXTS, _JS_INDEX_FILES.
Tests
-----
4 new tests in tests/test_import_extension_resolution.py:
- test_resolve_multi_dot_helper_file: foo.shared → foo.shared.ts
- test_resolve_multi_dot_with_explicit_extension_still_works:
foo.shared.ts (explicit) still wins
- test_resolve_ambient_d_ts_via_bare_path: foo.d → foo.d.ts
- test_end_to_end_multi_dot_import_resolves: tree-sitter pipeline
sanity check via extract_js
Existing 28 tests updated for the rename. 32/32 pass; 7 pre-existing
unrelated failures elsewhere in the suite.
Validation
----------
On a 1,873-file SvelteKit codebase, applying both rules over the v0.7.5
baseline:
baseline: 12,096 edges
with the resolver fix: 20,151 edges (+8,055 = +67%)
The +2,652 over the previous version of this branch is attributable
entirely to multi-dot filename recovery, primarily test helper imports
('*.integration.ts', '*.triggers.ts'), domain-shared modules
('*.shared.ts'), and config files.
Adds 8 tests covering import shapes that came up during real-codebase
validation against a 1,873-file SvelteKit project:
- test_type_only_import_with_bare_path_resolves
`import type { X } from './foo'` — type-only imports must go
through the same resolver. Common pattern in TS codebases.
- test_named_imports_emit_symbol_edges_after_resolution
`import { foo, bar } from './module'` — verifies the per-symbol
`imports` edges (file → module.foo, file → module.bar) target the
correct stem after resolution. The symbol target_stem comes from
_file_stem(resolved), so resolution must happen first.
- test_alias_directory_import_resolves_to_index_ts
`from '$lib/queue'` — alias + directory composes correctly.
- test_resolve_does_not_match_partial_directory_name
Regression guard: `from './foo'` where only `foo-extra.ts` exists
must NOT accidentally resolve to it.
- test_resolve_directory_without_index_returns_unchanged
A directory with no index.* must fall through, not pick a random
.ts inside.
- test_resolve_handles_subpath_into_directory_with_index
`./foo/sub` where `./foo/sub/index.ts` exists.
- test_resolve_does_not_treat_dotfile_as_extension
Path('.env-types.ts').suffix is '.ts' (correct), but worth pinning.
- test_resolve_chain_alias_and_extension_compose
Two-layer resolution: alias → bare path → .svelte.ts. Verifies
the full chain works end-to-end for the Svelte 5 rune-file case.
Also expanded test_named_imports_emit_symbol_edges_after_resolution to
catch a subtle regression class: per-symbol import edges (line 319-340
in _import_js) build their target id from _file_stem(resolved). If
resolution fails or returns the wrong path, the symbol edges silently
target a different stem and downstream "where is X used?" queries miss
real callers.
_import_js previously only rewrote .js→.ts and .jsx→.tsx, leaving every
other common TypeScript / SvelteKit / Vite import shape unresolved. The
resulting node id wouldn't match the target file's own _make_id, so
build_from_json dropped the edge as external.
Three missed shapes:
1. Bare paths (no extension) — TS convention:
`import { foo } from './foo'` → real file is foo.ts
2. .svelte → .svelte.ts (Svelte 5 rune-only files):
`import { x } from './x.svelte'` → real file is x.svelte.ts
3. Directory imports / barrel index files:
`import { x } from './queue'` → real file is queue/index.ts
Fix
---
New helper _resolve_with_extensions(p: Path) -> Path mirrors Vite/TS
resolver order:
1. exact path (file)
2. .js→.ts, .jsx→.tsx (existing TS-ESM convention)
3. bare path → .ts/.tsx/.svelte/.js/.jsx/.mjs
4. bare path → directory's index.{ts,tsx,js,jsx}
5. .svelte → .svelte.ts (Svelte 5 rune file)
Falls back to the original path on no match — preserves pre-fix behaviour
for genuinely external modules (build_from_json drops them as phantoms).
Wired into _import_js (relative + alias branches) and extract_svelte's
regex pass for dynamic_import so static and dynamic imports both benefit.
Subtle: uses .is_file() / .is_dir() rather than .exists(). When the
import is a directory, .exists() returns True and would short-circuit
before the index.ts lookup ever ran.
Tests
-----
20 new tests in tests/test_import_extension_resolution.py:
Resolver unit tests (12):
- existing path returned unchanged
- bare path → .ts / .tsx / .svelte
- .ts wins over .svelte for ambiguous bare paths (Vite order)
- directory → index.ts
- directory prefers index.ts over index.js
- .svelte → .svelte.ts (Svelte 5 rune file)
- .js → .ts (TS ESM convention)
- .jsx → .tsx
- real .js stays .js when .ts doesn't exist
- unresolvable returns input unchanged
End-to-end (8):
- bare-path import resolves in TS file
- directory import resolves to index.ts
- .svelte import resolves to .svelte.ts rune file
- explicit .ts/.svelte imports still work (regression guard)
- external module specifiers unchanged
- alias + bare path resolves
- dynamic_import bare path resolves
Bring TypeScript AST extraction to parity with Java and C# by adding the
declaration types upstream graphify currently skips:
- interface_declaration (parity with Java/C# class_types)
- enum_declaration + members
- type_alias_declaration
- module-level const literals (object/array/string/call/new/template/number)
via _js_extra_walk extension
- new_expression as call type for both _JS_CONFIG and _TS_CONFIG
Tested against tests/fixtures/typescript_advanced.ts: 8 expected node
types extracted (IUserRepository, UserStatus, UserId, USER_REPOSITORY,
DEFAULT_ROLES, USER_CONFIG, UserService, UserModule).
Validated on a 3,800-file TypeScript monorepo (NestJS + Next.js): yields
~1,885 interfaces, ~147 enums, ~405 type aliases, ~2,236 const literal
nodes, and ~1,935 instantiates edges that were previously invisible.
1. NEW: extract_markdown() — structurally indexes .md/.mdx files into the
knowledge graph. Headings become nodes, code blocks become nodes with
language tags, and nesting produces 'contains' edges. Zero new deps
(pure regex/line-by-line parsing, no tree-sitter needed).
2. FIX: collect_files() _EXTENSIONS was hardcoded and missing 18 extensions
that _DISPATCH already supported (.jsx, .mjs, .ex, .exs, .jl, .vue,
.svelte, .dart, .v, .sv, .sql, .f, .F, .f90, etc). Now uses
set(_DISPATCH.keys()) to stay automatically in sync.
3. Added deploy_guide.md test fixture and 6 new test cases.
4. Updated test_collect_files_from_dir to use dynamic extension set.
- Add Fortran support (26th language): .f/.F/.f90/.F90/.f95/.F95/.f03/.F03/.f08/.F08
via tree-sitter-fortran; capital-F files preprocessed with cpp -w -P
- Add graphify export {html,obsidian,wiki,svg,graphml,neo4j} CLI subcommands
- Add graphify query/path/explain CLI subcommands
- Reduce skill.md from 63KB to 47KB by replacing Python heredocs with CLI calls
- Extend to_html() with node_limit param for auto-aggregation on large graphs
- Add integration tests for all export/query/path/explain subcommands
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Users running `graphify update .` against a graph that contains nodes
preserved from older graphify versions see the warning:
[graphify] Extraction warning (31 issues): Node 1661 (id='telepathy_app')
has invalid file_type 'None' - must be one of [...]
The cause: `_rebuild_code` in watch.py preserves nodes from the existing
graph.json that aren't in the new AST output. Some of those nodes
(stubs, semantic concepts created before file_type was always
populated) have file_type=null in the on-disk JSON, which becomes
Python None on load. The validator in validate.py:33 then flags every
such node.
Add a normalization step in `build_from_json` next to the existing
`source` -> `source_file` legacy canonicalization: if a node's
file_type is None or empty, default it to "concept" before validation.
This silences the false positive while keeping the validator strict for
genuinely invalid values like "weird_type".
When a user has an older tree-sitter installed against a newer language
binding (or vice versa), Language() raises TypeError with messages like
"missing 1 required positional argument: 'name'". The previous catch-all
Exception handler stored that bare message in the per-file error field,
giving users no actionable signal.
Add a dedicated TypeError branch in _extract_generic() that returns a
clearer error with the upgrade command:
tree-sitter version mismatch for tree_sitter_python: ... .
Try: pip install --upgrade tree-sitter tree-sitter-languages
Behavior is unchanged for all non-version-mismatch errors - the broader
Exception handler still runs as before.
Codex Desktop rejects hookSpecificOutput.additionalContext on PreToolUse.
hook-check is now a no-op — graph guidance reaches the agent via AGENTS.md/skill.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
`extract_kotlin` previously emitted zero `calls` edges (and zero
`raw_calls` entries) on the current PyPI grammar. The Kotlin branch
of `walk_calls` only matched node type `simple_identifier`, but
PyPI's `tree_sitter_kotlin` produces `identifier` for the equivalent
plain-identifier node. The `simple_identifier` ↔ `identifier` rename
is a generation gap between tree-sitter-kotlin grammar versions —
older forks (and the JVM `io.github.bonede:tree-sitter-kotlin`
binding) still use `simple_identifier`.
Accept both names so the extractor works across grammar generations.
Also widens `_KOTLIN_CONFIG.name_fallback_child_types` for the same
reason (defensive — currently the `name` field path covers
class/function name resolution, but if that field is dropped in a
future grammar update the fallback would face the same rename).
Tested against `tests/fixtures/sample.kt`: edges go from 6
(file-contains + class-method only) to 10 (adds 4 in-file `calls`
edges resolved by the walker:
- .get() → .buildRequest() @ L8
- .post() → .buildRequest() @ L12
- createClient() → Config @ L21
- createClient() → HttpClient @ L22).
A new regression test `test_kotlin_emits_in_file_calls` asserts the
four edges so this exact bug can't recur.
Found via graphify-kmp (Kotlin Multiplatform port of graphify) —
its `PythonParityTest` flagged 4 KMP-only edges that Python missed.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit adds full VB.NET language support to graphify, raising the
supported language count from 25 to 26. The implementation follows the
established LanguageConfig pattern used by all other tree-sitter-backed
extractors.
New dependency:
- Adds optional extra [vbnet] backed by tree-sitter-vbnet (published
to PyPI at https://pypi.org/project/tree-sitter-vbnet/0.1.0/).
Install with: pip install graphifyy[vbnet]
graphify/detect.py:
- Added .vb to CODE_EXTENSIONS so VB.NET files are discovered during
corpus ingestion and file-system watching.
graphify/extract.py:
- _import_vbnet(): import handler for imports_statement nodes; emits
imports edges using the namespace_name child text.
- _vbnet_extra_walk(): extra-walk hook that intercepts namespace_block
nodes, emits a namespace node, and recurses.
- _VBNET_CONFIG: full LanguageConfig covering class_block / module_block /
structure_block / interface_block as class types; method_declaration /
constructor_declaration / property_declaration as function types;
invocation call nodes with target/member_access fields.
- VB.NET-specific branches in _extract_generic:
* Class body: VB.NET has no wrapper body node; inherits and implements
are named fields directly on the class_block. Emits separate inherits
and implements edges for each base type, stripping generic arguments.
* Constructor name: constructor_declaration carries no name field in
the grammar; always resolves to New.
* Function body: uses the declaration node itself as body sentinel so
the call-graph pass can find invocations inside methods.
- extract_vbnet(path): public wrapper that delegates to _extract_generic.
- _DISPATCH['.vb']: routes .vb files to extract_vbnet.
pyproject.toml:
- Added vbnet = ['tree-sitter-vbnet'] optional dependency group.
- Added 'tree-sitter-vbnet' to the all extra.
tests/fixtures/sample.vb:
- New fixture file exercising: Imports statements, Namespace block,
Interface, Class with Inherits + Implements, Module, Structure,
Sub/Function/Property methods, and method calls.
tests/test_languages.py:
- Added 13 tests covering: no-error, class/interface/module/structure
detection, method detection, imports relation, inherits edge,
implements edge, and no-dangling-edges invariant.
README.md:
- Updated language count 25 to 26.
- Added VB.NET to language list and file-extension table.
- extract_sql(): deterministic tree-sitter extraction of tables, views,
functions, foreign key references, and FROM/JOIN reads_from edges
- .sql added to CODE_EXTENSIONS and dispatch table
- tree-sitter-sql added as optional dep under [sql] extra
- xlsx_extract_structure(): extracts sheet/table/column nodes from .xlsx
(utility — pipeline wiring in follow-up)
- 6 new SQL tests, 447 total passing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Token-budget chunking cuts the truncation rate but doesn't eliminate
it. Output token cost scales with extractable concept density rather
than input tokens — a chunk that lands on a directory of dense design
docs can pack under the input budget while needing more than
`max_completion_tokens=8192` to express every named concept, so the
response is truncated mid-string and `_parse_llm_json` returns an
empty fragment.
Pre-tuning chunk size to be conservative enough that this never
happens leaves throughput on the table for the common case. Adding a
hard `max_files_per_chunk` cap on top of `token_budget` reintroduces
the "tune a static constant" problem the previous commit set out to
fix.
The fix uses the API's own truncation signal:
1. `_call_openai_compat` and `_call_claude` now expose `finish_reason`
on the result dict (Anthropic's `stop_reason == "max_tokens"` is
normalised to `"length"`).
2. `_extract_with_adaptive_retry` checks it: when truncated, splits
the chunk in half and recurses on each half. Recursion is bounded
by `max_retry_depth` (default 3 → at most 8x fanout per top-level
chunk).
3. Single-file chunks that truncate can't recover and surface a
warning rather than infinite-loop.
4. `extract_corpus_parallel` routes every chunk through the retry
wrapper. The `on_chunk_done` callback fires once per top-level
chunk with the merged result — recursive splits are invisible to
callers.
Three independent improvements to extract_corpus_parallel:
1. Token-aware chunking. Replaces `chunk_size=20` static packing with
a greedy packer keyed on `token_budget` (default 60_000), grouped
by parent directory so related artefacts share a chunk. Pass
`token_budget=None` to fall back to fixed-count packing.
2. Optional tiktoken (added to the [kimi] extra). When available,
`_estimate_file_tokens` uses cl100k_base for accurate counts;
without it, the existing chars/4 heuristic kicks in. Kimi-K2 ships
a tiktoken-based tokenizer so estimates against Moonshot are very
close to truth.
3. True parallelism. The function name said "parallel" but the body
was a sequential for-loop. Now uses ThreadPoolExecutor capped at
`max_concurrency` (default 4 — conservative against provider rate
limits). `on_chunk_done(idx, total, result)` still fires once per
chunk with the original submission idx so progress UIs work
unchanged. `max_concurrency=1` skips the pool to preserve
sequential semantics.
Plus failure tolerance: a chunk raising is now caught, logged to
stderr, and the run continues. Other chunks' results merge as normal.
On a 162-file repo (~125k words), the same work that took ~36 min
sequential under the old code finishes in ~7 min.
Grep and Glob tools removed in CC v2.1.117; searches now go through Bash.
Hook now reads stdin tool_input and pattern-matches on search commands.
Uninstall/reinstall handles both old and new matcher for clean upgrades.
Closes#578
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
import(`./handlers/${name}`) previously produced a garbage edge to a
path containing the unresolved ${name} expression. Now detects
template_substitution child nodes and breaks without emitting an edge.
Static template literals (no interpolation) still resolve correctly.
Adds 2 new tests: one asserting dynamic templates produce no edge,
one asserting static templates resolve like plain strings.
Adds _dynamic_import_js() helper (65 lines) that detects import() call
expressions in JS/TS, resolves the module path (same logic as static
imports including .js→.ts mapping and tsconfig aliases), and emits
imports_from edges from the enclosing function. Hooked into walk_calls
for JS/TS configs.
Also adds tests/fixtures/dynamic_import.ts fixture and 5 new tests
in tests/test_languages.py (all passing alongside 110 existing tests).
- #352: add skill-kiro.md to pyproject.toml package-data
- #341: guard edge_betweenness at >5000 nodes; use approximate k=100 for suggest_questions on large graphs
- #354/#229: add Step 6b in skill.md to call to_wiki() when --wiki given (before Step 9 cleanup)
- #356: call _install_opencode_plugin() from install --platform opencode path
- #350: add cache_root param to extract() so subdirectory runs keep cache at ./graphify-out/cache/
- #230: PHP class_constant_access_expression emits references_constant edges
- #232: PHP scoped_call_expression (static method calls) emits calls edges
- #287: os.replace fallback for Windows WinError 5; graphify update exits 1 on failure; templates use graphify update . instead of python3 -c
- #348: cross-file call resolution for all languages via raw_calls + global label map pass in extract()
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Christian Winther