Commit Graph
325 Commits
Author SHA1 Message Date
Christian Winther b68ec63494 fix(extract): apply resolver fixups to JS/TS dynamic_import handler
Third call site that re-implemented the same .js→.ts rewrite in
isolation. Previously only handled the explicit .js→.ts case; bare
paths, multi-dot helper files, and alias-resolved dynamic imports
all dropped silently.

Now uses _resolve_js_module_path on both branches (relative and
alias) — same shape as the static-import and Svelte regex paths.

Real-world impact: TS files using `await import('./foo')` patterns
for code splitting (e.g. lazy-loading a profanity check) now produce
edges to the resolved target.
2026-05-05 00:07:50 +02:00
Christian Winther 0dfc26e57f fix: prefer file matches over directory matches in resolver
When both a file (foo.ts) and a directory (foo/) exist at the same path,
both TypeScript and Vite prefer the file. The previous ordering checked
directory first and fell through unchanged when the directory had no
index, silently dropping every import like 'from ./auth' when an
auth/ subdirectory existed alongside auth.ts.
2026-05-04 23:44:32 +02:00
Christian Winther 5f5b59309c test(extract): cover .svelte.js + hybrid TS/JS Svelte 5 rune files
The generalized resolver already handles .svelte.js because the append
loop iterates _JS_RESOLVE_EXTS = (.ts, .tsx, .svelte, .js, .jsx, .mjs).
Adds three explicit tests to pin the behaviour and document the priority
choice:

  - test_resolve_svelte_to_svelte_js_for_javascript_rune_files
      JS-only Svelte 5 project: .svelte → .svelte.js works the same
      way as .svelte.ts in TS projects. No special-casing needed —
      the generalized append loop covers both.

  - test_resolve_svelte_prefers_svelte_ts_over_svelte_js
      Hybrid case (both files exist, e.g. .svelte.ts source plus
      .svelte.js build artifact): .ts wins. Documents the deliberate
      source-first priority — graphify is a source-code tool, not a
      runtime resolver, so we differ from Vite's default JS-first order.

  - test_resolve_real_svelte_file_wins_over_svelte_ts_sibling
      Existence check short-circuits before any extension append, so a
      real .svelte file always wins over a .svelte.ts sibling.
2026-05-04 22:47:29 +02:00
Christian Winther 49c3b50b5f fix(extract): generalize resolver to multi-dot filenames + rename
Two changes that landed together because they share the same code path:

1. Generalize the bare-path append to handle multi-dot filenames

   The previous resolver only appended extensions when path.suffix == ""
   (truly bare paths). Real codebases use a lot of multi-dot patterns:

     foo.shared.ts        ← imported as './foo.shared'
     foo.config.ts        ← imported as './foo.config'
     foo.compile.ts       ← imported as './foo.compile'
     foo.integration.ts   ← imported as './foo.integration' (test helper)
     foo.triggers.ts      ← imported as './foo.triggers'  (test helper)
     foo.svelte.ts        ← imported as './foo.svelte'    (Svelte 5 rune)
     foo.d.ts             ← imported as './foo.d'         (ambient types)

   For all of these, .suffix is the meaningful middle segment (.shared,
   .config, .integration, etc.) — not in the .js/.jsx/.svelte handled
   list, so the resolver fell through and the import dropped to a phantom.

   The fix unifies the bare-path and .svelte→.svelte.ts cases into a
   single rule: append each candidate extension to the FULL filename, not
   to the stripped stem. This subsumes:

     bare path:           foo           → foo.ts
     Svelte rune file:    foo.svelte    → foo.svelte.ts
     multi-dot helper:    foo.shared    → foo.shared.ts
     ambient declaration: foo.d         → foo.d.ts

   No behaviour change for paths that DO exist (.is_file() short-circuit)
   or for the .js→.ts / .jsx→.tsx convention (handled before the append
   loop so we don't accidentally match foo.js → foo.js.ts when foo.ts
   is the real file).

2. Rename _resolve_with_extensions → _resolve_js_module_path

   The function is JS/TS/Svelte-specific (Vite resolver order, mirrors the
   convention used by _import_js, _JS_CONFIG, _TS_CONFIG). The original
   name suggested it was a generic path utility. Renamed to make scope
   explicit and align with the existing _import_js / _JS_CONFIG naming
   pattern. Constants renamed to match: _JS_RESOLVE_EXTS, _JS_INDEX_FILES.

Tests
-----
4 new tests in tests/test_import_extension_resolution.py:

  - test_resolve_multi_dot_helper_file: foo.shared → foo.shared.ts
  - test_resolve_multi_dot_with_explicit_extension_still_works:
    foo.shared.ts (explicit) still wins
  - test_resolve_ambient_d_ts_via_bare_path: foo.d → foo.d.ts
  - test_end_to_end_multi_dot_import_resolves: tree-sitter pipeline
    sanity check via extract_js

Existing 28 tests updated for the rename. 32/32 pass; 7 pre-existing
unrelated failures elsewhere in the suite.

Validation
----------
On a 1,873-file SvelteKit codebase, applying both rules over the v0.7.5
baseline:

  baseline:                 12,096 edges
  with the resolver fix:    20,151 edges  (+8,055 = +67%)

The +2,652 over the previous version of this branch is attributable
entirely to multi-dot filename recovery, primarily test helper imports
('*.integration.ts', '*.triggers.ts'), domain-shared modules
('*.shared.ts'), and config files.
2026-05-04 22:41:50 +02:00
Christian Winther 0267cd789f test(extract): exhaustive coverage for extension resolution edge cases
Adds 8 tests covering import shapes that came up during real-codebase
validation against a 1,873-file SvelteKit project:

  - test_type_only_import_with_bare_path_resolves
      `import type { X } from './foo'` — type-only imports must go
      through the same resolver. Common pattern in TS codebases.

  - test_named_imports_emit_symbol_edges_after_resolution
      `import { foo, bar } from './module'` — verifies the per-symbol
      `imports` edges (file → module.foo, file → module.bar) target the
      correct stem after resolution. The symbol target_stem comes from
      _file_stem(resolved), so resolution must happen first.

  - test_alias_directory_import_resolves_to_index_ts
      `from '$lib/queue'` — alias + directory composes correctly.

  - test_resolve_does_not_match_partial_directory_name
      Regression guard: `from './foo'` where only `foo-extra.ts` exists
      must NOT accidentally resolve to it.

  - test_resolve_directory_without_index_returns_unchanged
      A directory with no index.* must fall through, not pick a random
      .ts inside.

  - test_resolve_handles_subpath_into_directory_with_index
      `./foo/sub` where `./foo/sub/index.ts` exists.

  - test_resolve_does_not_treat_dotfile_as_extension
      Path('.env-types.ts').suffix is '.ts' (correct), but worth pinning.

  - test_resolve_chain_alias_and_extension_compose
      Two-layer resolution: alias → bare path → .svelte.ts. Verifies
      the full chain works end-to-end for the Svelte 5 rune-file case.

Also expanded test_named_imports_emit_symbol_edges_after_resolution to
catch a subtle regression class: per-symbol import edges (line 319-340
in _import_js) build their target id from _file_stem(resolved). If
resolution fails or returns the wrong path, the symbol edges silently
target a different stem and downstream "where is X used?" queries miss
real callers.
2026-05-04 22:33:29 +02:00
Christian Winther 2b1efe8f08 fix(extract): TS bare-path / .svelte.ts / index.ts import resolution
_import_js previously only rewrote .js→.ts and .jsx→.tsx, leaving every
other common TypeScript / SvelteKit / Vite import shape unresolved. The
resulting node id wouldn't match the target file's own _make_id, so
build_from_json dropped the edge as external.

Three missed shapes:

  1. Bare paths (no extension) — TS convention:
     `import { foo } from './foo'`            → real file is foo.ts
  2. .svelte → .svelte.ts (Svelte 5 rune-only files):
     `import { x } from './x.svelte'`         → real file is x.svelte.ts
  3. Directory imports / barrel index files:
     `import { x } from './queue'`            → real file is queue/index.ts

Fix
---
New helper _resolve_with_extensions(p: Path) -> Path mirrors Vite/TS
resolver order:

  1. exact path (file)
  2. .js→.ts, .jsx→.tsx (existing TS-ESM convention)
  3. bare path → .ts/.tsx/.svelte/.js/.jsx/.mjs
  4. bare path → directory's index.{ts,tsx,js,jsx}
  5. .svelte → .svelte.ts (Svelte 5 rune file)

Falls back to the original path on no match — preserves pre-fix behaviour
for genuinely external modules (build_from_json drops them as phantoms).

Wired into _import_js (relative + alias branches) and extract_svelte's
regex pass for dynamic_import so static and dynamic imports both benefit.

Subtle: uses .is_file() / .is_dir() rather than .exists(). When the
import is a directory, .exists() returns True and would short-circuit
before the index.ts lookup ever ran.

Tests
-----
20 new tests in tests/test_import_extension_resolution.py:

  Resolver unit tests (12):
    - existing path returned unchanged
    - bare path → .ts / .tsx / .svelte
    - .ts wins over .svelte for ambiguous bare paths (Vite order)
    - directory → index.ts
    - directory prefers index.ts over index.js
    - .svelte → .svelte.ts (Svelte 5 rune file)
    - .js → .ts (TS ESM convention)
    - .jsx → .tsx
    - real .js stays .js when .ts doesn't exist
    - unresolvable returns input unchanged

  End-to-end (8):
    - bare-path import resolves in TS file
    - directory import resolves to index.ts
    - .svelte import resolves to .svelte.ts rune file
    - explicit .ts/.svelte imports still work (regression guard)
    - external module specifiers unchanged
    - alias + bare path resolves
    - dynamic_import bare path resolves
2026-05-04 22:26:48 +02:00
Safi ee85bbfbfd fix antigravity install paths .agents → .agent 2026-05-04 19:00:35 +01:00
Safi 5b33e6d2e6 remove dedup architecture diagram 2026-05-04 18:51:59 +01:00
SafiandClaude Sonnet 4.6 579e1cc744 wire --dedup-llm through build pipeline and fix fresh-extract dedup bypass
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-04 18:51:14 +01:00
Safi 913feca6a6 CI: add v5-v7 branches, workflow_dispatch trigger, sql extra 2026-05-04 18:27:33 +01:00
Safi 4f08e23807 Update README: add graphify extract to command reference, fix CI badge branch 2026-05-04 18:26:17 +01:00
Safi b3c99ec0ed Fix hook rebuild losing community labels; fix gemini platform install (#705, #706) v0.7.5 2026-05-04 18:20:16 +01:00
Safi 682d124618 bump version to 0.7.5 2026-05-04 18:07:48 +01:00
Safi 239000d2c9 Add incremental updates to graphify extract: semantic cache + build_merge + manifest 2026-05-04 18:07:02 +01:00
Safi ec413d5233 Wire deduplicate_entities into build() and build_merge() 2026-05-04 18:02:38 +01:00
Safi 34380434c4 Add graphify/dedup.py: entropy gate + MinHash/LSH + Jaro-Winkler entity deduplication 2026-05-04 18:01:11 +01:00
Safi b26467a8eb Add datasketch and rapidfuzz as base dependencies 2026-05-04 17:51:46 +01:00
Safi b5e7cf3f55 Add implementation plan: incremental updates + entity deduplication 2026-05-04 16:30:36 +01:00
Safi 5ee4b01406 Add design spec: incremental updates + entity deduplication 2026-05-04 16:25:58 +01:00
Safi 6c3ccdc77b Fix #703: unify community counts in GRAPH_REPORT.md header and body 2026-05-04 13:59:27 +01:00
Safi 26a5a35200 bump version to 0.7.4 2026-05-04 12:40:43 +01:00
SafiandClaude Opus 4.7 741ac3655b Fix #700 #701: JSONC tsconfig parsing + Svelte dynamic import edge IDs
#700: _read_tsconfig_aliases() now handles // comments, /* */ block comments,
and trailing commas via a regex-based _strip_jsonc() helper. Tries plain
json.loads first, falls back to stripped parse, warns to stderr on failure
instead of silently returning {}.

#701 Bug A: removed startswith('.') filter from Svelte regex fallback so
aliased imports ($lib/, $partials/, @/) are no longer skipped.
#701 Bug B: fixed synthetic node IDs — source uses _make_id(str(path)),
target uses _make_id(str(normpath(parent/raw))) matching _extract_generic
and _import_js conventions so build_from_json no longer drops every edge.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-04 12:39:07 +01:00
Safi c1a7e250b6 bump version to 0.7.3 2026-05-04 11:46:39 +01:00
Safi c121fc4e42 README: clarify semantic extraction uses IDE model via skill, or Anthropic/Kimi for headless extract 2026-05-04 11:42:58 +01:00
Safi 6e84b42320 Clarify README: semantic extraction supports Anthropic and Kimi only (not OpenAI) 2026-05-04 11:41:40 +01:00
SafiandClaude Opus 4.7 d40e27463d Add graphify extract subcommand for headless semantic extraction (#698)
Enables docs-only corpora to run full LLM extraction in CI without Claude Code.
Handles code-only, docs-only, and mixed corpora; --no-cluster for raw dumps.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-04 11:31:54 +01:00
SafiandClaude Sonnet 4.6 b6ffdbb8dd v0.7.2: Fortran support + export CLI subcommands + skill.md size reduction
- Add Fortran support (26th language): .f/.F/.f90/.F90/.f95/.F95/.f03/.F03/.f08/.F08
  via tree-sitter-fortran; capital-F files preprocessed with cpp -w -P
- Add graphify export {html,obsidian,wiki,svg,graphml,neo4j} CLI subcommands
- Add graphify query/path/explain CLI subcommands
- Reduce skill.md from 63KB to 47KB by replacing Python heredocs with CLI calls
- Extend to_html() with node_limit param for auto-aggregation on large graphs
- Add integration tests for all export/query/path/explain subcommands

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-04 11:17:06 +01:00
Safi f81e3bc215 bump version to 0.6.9 v0.7.2 v0.7.3 v0.7.4 v0.7.1 v0.7.0 v0.6.9 2026-05-03 17:41:56 +01:00
SafiandClaude Sonnet 4.6 8e81720bc8 Fix #686 #652: GRAPHIFY_OUT env var for worktrees, Antigravity install auto-updates rules/workflow
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 17:36:17 +01:00
SafiandClaude Sonnet 4.6 6df69dce6c Fix #683: normalize source_file path separators + two-phase cohesion re-clustering
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 17:26:13 +01:00
SafiandClaude Sonnet 4.6 f61bb27d9c Fix #688: stricter VS Code Copilot instructions to enforce GRAPH_REPORT.md first
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 17:23:31 +01:00
Safi de268a067c fix Penpax waitlist link to graphifylabs.ai 2026-05-03 14:09:16 +01:00
SafiandClaude Sonnet 4.6 b11a8a7813 simplify README, move technical details to docs/how-it-works.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 13:31:28 +01:00
SafiandClaude Sonnet 4.6 d753413409 bump version to 0.6.8
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v0.6.8
2026-05-03 13:25:44 +01:00
SafiandClaude Sonnet 4.6 f065933391 Fix #676 #678 #681: graphifyignore negation, Antigravity slash command, Gemini hook Windows
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 13:22:17 +01:00
SafiandClaude Sonnet 4.6 893acb19df Fix #664: filter thin communities from GRAPH_REPORT.md by default
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 18:09:32 +01:00
SafiandClaude Sonnet 4.6 96267ad0a8 fix #651: resolve absolute graphify path at codex install time for VS Code extension on Windows
shutil.which() first; falls back to sys.executable sibling Scripts/graphify.exe
so the hook works even when venv/Scripts is not on Codex's PATH.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 17:38:24 +01:00
SafiandClaude Sonnet 4.6 cf7ce3a450 fix #651: hook-check exits silently instead of emitting unsupported additionalContext
Codex Desktop rejects hookSpecificOutput.additionalContext on PreToolUse.
hook-check is now a no-op — graph guidance reaches the agent via AGENTS.md/skill.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 17:29:40 +01:00
ba41258635 merge PR #663: parallel AST extraction with ProcessPoolExecutor (1.66x speedup on 84 files)
Co-Authored-By: hanzala-sohrab <hanzala-sohrab@users.noreply.github.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 17:24:57 +01:00
Safi e48428296c bump version to 0.6.7 v0.6.7 2026-05-02 17:13:08 +01:00
SafiandClaude Sonnet 4.6 d819827ea2 fix #655 #656 #657 #658: cache dir crash, sanitize_label None, rationale file_type, token counting
#655 and #656 already fixed in cache.py and security.py.
#657: add rationale to file_type schema in all 12 skill variants; warn against inventing concept.
#658: add explicit chunk-merge step with token summation before save_semantic_cache.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 17:10:20 +01:00
Safi bd92ab67d6 docs: remove VB.NET from README language count and file type table 2026-05-02 17:06:00 +01:00
Hanzala Sohrab 0a99a46bfc chore: update project dependencies and metadata in pyproject.toml 2026-05-02 21:34:20 +05:30
SafiandClaude Sonnet 4.6 1e848458f9 remove VB.NET support: tree-sitter-vbnet only has Windows wheels, no Linux/macOS
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 17:02:24 +01:00
SafiandClaude Sonnet 4.6 2ca3a5994b merge PR #579: dynamic import() extraction for JS/TS
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 16:52:19 +01:00
SafiandClaude Sonnet 4.6 bfe18f0c83 fix: add context=import to JS/TS named-import edges (caught by PR #573 test)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 16:50:33 +01:00
SafiandClaude Sonnet 4.6 28047b502f merge PR #573: cross-language edge context filters in MCP query tool
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 16:49:22 +01:00
SafiandClaude Sonnet 4.6 9881d7ca30 merge PR #557: graphify tree — D3 v7 collapsible-tree HTML view of graph.json
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 16:48:09 +01:00
24808ef8e6 merge PR #625: token-aware chunking with split-and-retry on truncation
Co-Authored-By: Jason Matthew <jasonm4130@gmail.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 16:46:37 +01:00
Safi 66e61c76b1 docs: add Docker MCP Toolkit + SQLite MCP runbook (#620) 2026-05-02 16:42:08 +01:00