125 Commits

Author SHA1 Message Date
Safi 8c4c67f3ad fix Ollama num_ctx: derive from actual chunk size instead of hardcoding 131072 (#798) 2026-05-09 23:42:44 +01:00
Porun db66b8727b feat: add callflow HTML export with Mermaid architecture diagrams 2026-05-09 23:26:01 +01:00
Safi 8c00287e84 fix Ollama context saturation: set num_ctx=131072, keep_alive=30m, serial by default (#798) 2026-05-09 22:51:36 +01:00
Safi 23f598f3a0 fix MultiGraph crash, hollow LLM response retry, and skill --help (#796, #795, #792)
#796: add edge_data()/edge_datas() helpers in build.py that tolerate
MultiGraph/MultiDiGraph; replace all G.edges[u,v] 2-tuple call sites in
__main__.py, serve.py, wiki.py, export.py, analyze.py, benchmark.py;
fix same pattern in 10 skill file inline heredocs

#795: all 12 skill files now short-circuit on /graphify --help or -h
and print the Usage block without running any pipeline steps

#792 (hollow response): add _response_is_hollow() predicate in llm.py;
when Ollama (or any backend) returns empty/null/whitespace content or a
parsed result with no nodes/edges, rewrite finish_reason="length" so
_extract_with_adaptive_retry bisects the chunk instead of silently
dropping it; applied to _call_openai_compat, _call_claude, _call_bedrock

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 21:17:25 +01:00
Safi 8e6483511d add Pascal regex fallback - extraction works without tree-sitter-pascal
tree-sitter-pascal is not on PyPI so extract_pascal() fell back to an
empty error result for all users. _extract_pascal_regex() now handles
unit/program/library headers, uses clauses, class/interface declarations
with inheritance, forward method decls, qualified impl headers, balanced
begin/end body extraction, and intra-file calls edges. All 15 previously
skipped pascal_required tests now run unconditionally and pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 15:52:17 +01:00
simeonbodurov 32bf8b4a37 feat(extract): add Pascal/Delphi and Lazarus IDE support (#781)
* feat(extract): add Pascal/Delphi language support

Adds full AST extraction for Pascal and Delphi source files using
tree-sitter-pascal (https://github.com/Isopod/tree-sitter-pascal).

Supported file extensions: .pas, .pp, .dpr, .dpk, .inc

Extracted nodes:
- File node (the .pas file itself)
- unit / program / library declarations
- class, interface, and helper type declarations
- procedure and function implementations

Extracted edges:
- file --contains--> module
- module --imports--> dependency (via uses clause, resolved to path-based IDs)
- class --inherits--> base class / interface
- class/module --contains/method--> procedure or function
- procedure --calls--> procedure (in-file call resolution)

Key design: uses clause targets are resolved to path-based node IDs by
scanning all Pascal files under the project root (_pascal_project_root +
_pascal_resolve_unit helpers). This avoids dangling import edges that
result from resolving bare unit names like "SysUtils" to IDs that never
match any file node.

Bare procedure calls (e.g. `Reset;` without parentheses) are detected
by inspecting statement nodes whose sole named child is an identifier,
in addition to the standard exprCall nodes used for calls with arguments.

Requires: pip install tree-sitter-pascal
(https://github.com/Isopod/tree-sitter-pascal)
If not installed, extract_pascal returns {"nodes":[], "edges":[], "error": ...}
so the rest of the pipeline is unaffected.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(extract): add Lazarus .lfm and .lpk file support

Adds two new extractors for Lazarus IDE-specific file formats:

extract_lazarus_form() — .lfm (Lazarus Form files)
  .lfm files are text-based UI component trees. The extractor parses
  `object Name: TClassName ... end` blocks to build a containment graph
  of form components, and captures `OnXxx = HandlerName` event bindings
  as `references` edges (context: "event") linking each component to
  its handler procedure.

extract_lazarus_package() — .lpk (Lazarus Package files)
  .lpk files are XML package definitions. The extractor reads the
  package name, required package dependencies (→ imports edges), and
  listed unit files (→ contains edges). Unit names are resolved to
  path-based node IDs via _pascal_resolve_unit so they connect to the
  same nodes produced by extract_pascal on .pas files.

Both extensions added to CODE_EXTENSIONS in detect.py and to _DISPATCH.
13 new tests in test_pascal.py cover both extractors.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(pascal): fix dangling inherits-edge targets and capture all base classes

The declType/typeref handler built inherits edge targets with
_make_id(_read(child)) — just the bare class name. But class nodes
use _make_id(stem, type_name), so targets never matched, making the
entire class hierarchy invisible in the graph.

Add _pascal_class_stem_cache and _pascal_resolve_class(): strips the
conventional T/I prefix, locates the defining file by stem lookup
(same cache mechanism as _pascal_resolve_unit), and returns the
correct _make_id(file_stem, class_name) ID. RTL/unresolvable bases
(e.g. TObject) fall back to _make_id(bare_name) with an explicit
stub node, following the same pattern as the Python extractor.

Also remove the `break` that stopped after the first typeref, so
all parents are captured (e.g. class(TBase, IInterface)).

Extend test_pascal_no_dangling_edges to also assert that within-file
edge targets (contains, method, inherits, calls) resolve to real nodes.

* feat(extract): add Delphi .dfm form file support

Adds extract_delphi_form() for Delphi Form files (.dfm), which use the
same `object Name: TClassName ... end` text syntax as Lazarus .lfm files.

Binary .dfm files (FF 0A magic header) are skipped gracefully with an
informative error message so the pipeline is unaffected.  Text .dfm files
are parsed identically to .lfm: component containment (`contains` edges)
and event handler references (`references`, context "event").

Adds .dfm to _DISPATCH and CODE_EXTENSIONS.
10 new tests in test_pascal.py, including a regression test for the
binary-format detection.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Add .lpr extension and fix removesuffix in Pascal extractor

Address review feedback from @safishamsi:
- Add .lpr (Lazarus program file, identical syntax to .dpr) to _DISPATCH
  in extract.py and CODE_EXTENSIONS in detect.py so Lazarus project entry
  points are indexed. Completes the promised Lazarus IDE support.
- Replace rstrip("()") with removesuffix("()") in the call-resolution
  dict comprehension for precise suffix removal (rstrip strips individual
  characters, not the literal string "()").
- Add .lpr assertions to test_pascal_dispatch_registered and
  test_pascal_detect_extensions_registered.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Simeon Bodurov <simeon.bodurov@speedy.bg>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 14:44:58 +01:00
nauman73 e5f263ba98 fix(windows): unblock pipeline on Windows consoles + missing __main__ guards (#788)
Three independent Windows compatibility fixes shipped together because they
all surface during the same first /graphify run on Windows.

graphify/benchmark.py
  print_benchmark() unconditionally printed U+2500 (box-drawing) and U+2192
  (rightwards arrow), which UnicodeEncodeError'd on stdouts that can't encode
  them — most notably the legacy Windows console at cp1252. New _safe()
  helper falls back to ASCII when the active stdout encoding can't carry the
  glyph; _hr() uses it. Two regression tests cover both paths and prove
  print_benchmark survives a cp1252-strict stream.

graphify/extract.py
  ProcessPoolExecutor on Windows uses spawn, so worker subprocesses
  re-import the calling __main__. When the caller is `python -c "..."` or a
  script without an `if __name__ == "__main__":` guard, the workers
  recursively spawn themselves and the pool dies. The user-visible failure
  was a 290-line traceback ending in BrokenProcessPool, hiding the actual
  cause. _extract_parallel now catches BrokenProcessPool, prints a one-line
  warning that names the __main__-guard idiom, and returns False so the
  public extract() routes to the existing _extract_sequential fallback. Two
  tests cover the parallel-returns-False contract and the sequential
  fallback wiring.

graphify/skill-windows.md
  Every `python -c "..."` block (30 in total) is replaced with a
  Write+run+delete pattern using PowerShell's literal here-string @'...'@.
  The old form was a quote-escaping minefield: any double-quote inside the
  Python source had to be backslash-escaped for the shell, and PowerShell's
  parser ate them inconsistently — failing on f-strings like
  `f'AST: {len(result["nodes"])} nodes'`. The new form passes Python source
  to disk literally, so what the model writes is what Python sees. The AST
  step's script template now includes an explicit `if __name__ == "__main__":`
  guard so multi-core extraction works even before the runtime fallback above
  kicks in. All 31 resulting heredoc blocks parse cleanly under
  `ast.parse`.

Co-authored-by: Nauman Hameed <Nauman.Hameed@enghouse.com>
2026-05-09 12:59:38 +01:00
tmaeder f88567b114 Recover from context-window-exceeded API errors in adaptive retry (#789)
Adaptive retry only recovered from `finish_reason="length"` (output
truncation). It did not handle the other shape of overflow: the API
rejecting the prompt outright with a 400 because the input plus
`max_completion_tokens` doesn't fit in the model's context window.

This shows up immediately on local OpenAI-compatible servers (LM
Studio, llama.cpp, vLLM) where the default context is small (4K-32K)
and a 60K-token chunk packed for cloud Kimi/Claude blows past it.
Without retry the whole chunk fails with no output, even though the
two halves would each fit cleanly.

Catch a heuristic set of context-overflow exception messages,
classify them as the same kind of recoverable failure as
`finish_reason="length"`, and split-recurse on the same path. Single-
file overflow returns an empty fragment so the rest of the corpus
keeps running. Unrelated errors (rate limit, auth, etc.) still
propagate.

Tested with qwen3.5-9b on LM Studio (32K ctx) against a 215-file
corpus where chunks 4-12 of 12 previously failed; with this change
the overflowing chunks self-heal by splitting in half.
2026-05-09 12:59:35 +01:00
Safi 0c29b2cb88 fix reversed call edges after --update and missing SKILL.md on install (#760, #725)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 22:00:57 +01:00
zuyeyang 5d0392524a fix(extract): add ALTER TABLE FK extraction + schema-qualified name support for SQL (#779)
The SQL parser (`extract_sql`) previously only extracted foreign key
relationships defined inline within CREATE TABLE column definitions.
FK constraints added via ALTER TABLE ... ADD CONSTRAINT ... FOREIGN KEY
... REFERENCES were silently ignored.

Additionally, `_obj_name()` only read the first identifier child of
object_reference nodes, so schema-qualified names like `Sales.Customer`
were truncated to just `Sales`.

Changes:
- Add `alter_table` handler to `walk()` that extracts FK edges from
  ALTER TABLE ... ADD CONSTRAINT ... FOREIGN KEY ... REFERENCES
- Fix `_obj_name()` to read the full object_reference text, preserving
  schema-qualified names (e.g. `Sales.Customer`)
- Fix inline FK resolution in create_table and _walk_from_refs to use
  full object_reference text instead of first identifier only
2026-05-08 20:31:25 +01:00
Chris S ca1b774dd9 Harden Google Workspace gws exports (#772) 2026-05-07 20:07:57 +01:00
SerkanGezici 8489b26d06 fix(extract): use language_tsx for .tsx files to enable JSX-aware parsing (#766)
tree-sitter-typescript ships two grammars:
- language_typescript: pure TypeScript, no JSX support
- language_tsx:        JSX-aware variant for .tsx files

Currently both .ts and .tsx are parsed with language_typescript, which
treats JSX syntax as parse errors. Every function declaration, arrow
function, and call_expression nested inside a JSX tree is silently
dropped from the extracted graph.

Repro on a representative React+TypeScript codebase (a 13-file Tauri app):
parsing each .tsx with language_typescript produces ~276 ERROR nodes per
file. Only declarations that happen to live before the first JSX block
survive.

Fix: add _TSX_CONFIG that mirrors _TS_CONFIG but selects language_tsx,
and route .tsx files to it in extract_js().

Effect on the same repo (graphify update --force):
  Nodes:        303 → 618  (+104%)
  Edges:        482 → 779  (+62%)
  Communities:   28 → 45   (+61%)
  Parse errors  276 → 0    per .tsx file

Tests added:
- tsx fixture with helpers + JSX-returning component
- helpers and component are captured
- JSX expression calls ({fmtDate(now)}) resolve to call edges
- wiring check: .tsx uses language_tsx, .ts uses language_typescript

Note: this fixes the parsing layer. Calls inside deeply nested arrow
function callbacks (e.g. items.map(x => <T>{f(x)}</T>)) are still
missed by the call extraction logic — separate enhancement.

Co-authored-by: Serkan Gezici <serkan@quadroaipilot.com>
2026-05-07 20:07:36 +01:00
Safi 0656ec62a4 security hardening: F-002/F-005/F-007/F-008/F-009/F-010/F-016/F-031/F-035/F-038/PR747-NEW-2 2026-05-07 16:32:26 +01:00
Safi fe76612d90 Merge PR #708: TypeScript interface/enum/type-alias/const/new_expression extraction 2026-05-07 11:07:24 +01:00
Safi 669bc32e0d Merge PR #752: Google Workspace shortcut export (.gdoc/.gsheet/.gslides)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-07 10:42:23 +01:00
Safi 9bbd14a7dd Merge PR #753: CommonJS require() imports as EXTRACTED edges
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-07 10:28:44 +01:00
Safi 09e69745bc Merge PR #759: Positional install platform names 2026-05-07 10:25:56 +01:00
Safi c986e28a70 Merge PR #758: Fix GRAPHIFY_OUT for query/path/explain 2026-05-07 10:25:50 +01:00
Safi 78a16c3733 Merge PR #672: Silence file_type None warning 2026-05-07 10:25:40 +01:00
Safi 935312c227 Merge PR #671: tree-sitter version mismatch hint 2026-05-07 10:25:35 +01:00
Chris Stephens 68804f7f70 Handle positional install platforms 2026-05-06 17:10:45 -04:00
Chris Stephens 81bc2f9298 Fix GRAPHIFY_OUT defaults for graph query commands 2026-05-06 17:04:57 -04:00
Mani Saint-Victor, MD 2dd6ee6a9c fix: promote cross-file call edges to EXTRACTED when import evidence exists
The cross-file call resolver in `extract()` unconditionally marked every
resolved call edge as INFERRED with confidence_score 0.8 — even when the
caller's file had an explicit `imports` (symbol) or `imports_from`
(module) edge to the callee. The new CJS require handler made this gap
visible: imports were correctly EXTRACTED but the call edges that those
imports backed remained INFERRED, so downstream consumers couldn't tell
high-evidence calls apart from name-match guesses.

This pass runs after the file-id remap (line 4736), so we relativize
node `source_file` paths before computing file_nids — otherwise the
caller's computed file_nid (absolute-path-derived) wouldn't match the
imports_from edge source (already remapped to relative form).

Promotion rule:
  - Symbol-level `imports` edge from caller's file -> callee node id
    => EXTRACTED, confidence_score 1.0
  - Module-level `imports_from` edge from caller's file -> callee's file
    => EXTRACTED, confidence_score 1.0
  - Otherwise => INFERRED, confidence_score 0.8 (existing behavior)

Validated on a 92-file CJS orchestrator: 5 previously-INFERRED edges
from runExecute() now resolve to EXTRACTED, and 88% of cross-file calls
in the corpus (104 of 118) promote, leaving INFERRED only for genuine
heuristic guesses with no import backing.

Adds two tests:
  - test_cross_file_call_promoted_to_extracted_with_import_evidence
  - test_cross_file_call_remains_inferred_without_import_evidence
2026-05-06 13:13:30 -04:00
Chris Stephens f704972b3e Add optional Google Workspace shortcut export 2026-05-06 11:37:11 -04:00
Mani Saint-Victor, MD c902ae952b fix: extract CommonJS require() imports as EXTRACTED edges
The JS/TS extractor only handled ES `import` statements; CommonJS
`require()` calls produced no import edges. Downstream, the call-graph
pass could not resolve which symbols belonged to which file, so every
cross-file call in CJS Node.js codebases was downgraded to INFERRED
even when the binding was a top-of-file destructured require.

Adds three patterns to `_js_extra_walk` via a new `_require_imports_js`
helper:

  const { foo, bar: alias } = require('./mod')   -> imports_from + per-symbol imports
  const mod = require('./mod')                   -> imports_from
  const x = require('./mod').y                   -> imports_from + symbol edge for y

Refactors path-resolution out of `_import_js` into
`_resolve_js_import_target` so ES imports and CJS requires share the
relative / tsconfig-alias / bare-module logic.

Tested in a 92-file CJS Node.js orchestrator codebase: confirmed all
five previously INFERRED `runExecute -> {loadFoundation,
validateDispatchConfig, fetchSymphonyIssues, listSymphonyWorktrees,
workspacePathForIssue}` edges resolve to real top-of-file destructured
requires, so downstream calls would now be EXTRACTED instead of
INFERRED.
2026-05-06 09:41:56 -04:00
Safi 25c1ea9212 Merge PR #735: Add Gemini and OpenAI semantic extraction backends (preserve Ollama priority) 2026-05-06 12:13:39 +01:00
Safi 0010d38337 Merge PR #717: fix(extract): TS bare-path / .svelte.ts / .svelte.js / multi-dot import resolution 2026-05-06 12:06:27 +01:00
Safi 06df2066a8 Merge PR #732: feat: add Groovy and Spock support 2026-05-06 12:04:51 +01:00
Safi 12a706b595 Merge PR #711: feat(extract): add Markdown structural extraction + sync collect_files extensions with _DISPATCH 2026-05-06 12:02:43 +01:00
Safi 519564b742 Merge PR #745: add .luau extension support (Roblox Luau) 2026-05-06 11:59:40 +01:00
Safi 660b605e18 Merge PR #736: fix(detect): forward follow_symlinks from detect_incremental to detect 2026-05-06 11:58:59 +01:00
daleardi 3457b9cd80 add .luau extension support (Roblox Luau)
Add `.luau` to CODE_EXTENSIONS and route it through extract_lua
(tree-sitter-lua). Roblox first-party code uses .luau; without this,
graphify silently skipped 379/479 files on a real Roblox codebase
and the resulting graph was dominated by vendored .lua dependencies.

tree-sitter-lua doesn't parse Luau type annotations, but it
successfully extracts function declarations and call edges from
Luau source — verified on a 379-file Roblox codebase (1265 nodes,
1471 edges, 236 communities).

A dedicated tree-sitter-luau grammar would be a richer long-term
fix; this is the minimal change to make Luau projects work today.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 23:56:51 -04:00
Safi 48888a7c26 add Ollama backend and cross-project global graph (#729)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 18:36:59 +01:00
Daniel Graham cc63a1711b Make Gemini extraction model configurable
The initial Gemini backend defaulted to 2.5 Flash, but large semantic extraction chunks can benefit from newer models and more output headroom. Move the default to Gemini 3 Flash Preview, add CLI and environment model overrides, and increase the Gemini completion budget while keeping low reasoning effort for cost control.

Constraint: Google exposes Gemini through an OpenAI-compatible chat-completions endpoint

Rejected: Hardcode Gemini 3.1 Pro as the default | higher cost for routine repository indexing

Confidence: medium

Scope-risk: narrow

Directive: Keep --model and GRAPHIFY_GEMINI_MODEL working before changing Gemini defaults again

Tested: uv run --directory vendor/graphify pytest tests/test_llm_backends.py tests/test_chunking.py -q

Not-tested: Live Gemini 3 extraction on the full cloud-edge repo before this commit
2026-05-05 10:11:12 -04:00
Alpha Nury 64585cf889 fix(detect): forward follow_symlinks from detect_incremental to detect
`detect_incremental(root)` always called `detect(root)` without forwarding
the `follow_symlinks` kwarg. As a result, corpora that include symlinked
sub-trees pointing to directories outside the scan root (e.g. a
`state_of_truth/` symlink pointing at `~/.hermes/state_of_truth/`) were
visible to a full `detect()` run with `follow_symlinks=True` but invisible
to any subsequent `--update` run. The incremental scan would then either
report no changes (silently dropping legitimate new files) or repeatedly
re-extract a phantom subset, depending on what was reachable without
crossing symlinks.

Add a keyword-only `follow_symlinks` parameter to `detect_incremental()`
and forward it. Default stays `False` for backwards compatibility — only
callers that already opt in to symlink following on `detect()` pick up
the new behaviour for incremental runs too.

Test: a corpus with a symlinked directory is invisible with
`follow_symlinks=False`, fully indexed with `follow_symlinks=True`, and
correctly reports zero new files on a second incremental scan after the
manifest is saved.
2026-05-05 15:25:08 +02:00
Daniel Graham a9cb692961 Prefer accessible semantic extraction backends
Gemini is often the cheaper available quota for low-stakes semantic graph extraction, while OpenAI is a useful fallback. Extend the direct extraction backend registry, CLI validation, docs, and tests so headless extraction can use GEMINI_API_KEY, GOOGLE_API_KEY, or OPENAI_API_KEY without changing the existing Claude and Kimi paths.

Constraint: Gemini supports OpenAI-compatible chat completions at the Google generative-language endpoint

Rejected: Native google-genai integration | higher dependency and response-shape churn for the same chat-completions path

Confidence: medium

Scope-risk: moderate

Directive: Keep backend detection explicit and test every accepted API-key environment variable before adding new providers

Tested: uv run --directory vendor/graphify pytest tests/test_llm_backends.py tests/test_chunking.py -q

Not-tested: Live Gemini/OpenAI API calls; no GEMINI_API_KEY or OPENAI_API_KEY present in this environment
2026-05-05 08:59:37 -04:00
Mikołaj Cekut 6aecfcf0fe chore: replace pl.allegro with com.nicklastrange in Groovy fixtures 2026-05-05 12:22:38 +02:00
Mikołaj Cekut dc69020a47 feat: add Groovy and Spock support
- Register .groovy and .gradle in CODE_EXTENSIONS, _DISPATCH, and collect_files
- Add _GROOVY_CONFIG (reuses Java import handler)
- Add regex-based _extract_spock_fallback for Spock spec files where
  tree-sitter-groovy wraps the body in ERROR nodes due to def-string methods
- _is_spock_file detects via regex scan (def "...") instead of node-label
  heuristic, avoiding false negatives on classes whose name differs from stem
- Fallback retains only file node + import edges from tree-sitter pass to
  prevent orphaned constructor/method nodes
- Add tree-sitter-groovy>=0.1.2 dependency
- Add 11 tests covering plain Groovy and Spock paths, including apostrophe
  in feature method names
2026-05-05 12:15:15 +02:00
Christian Winther b68ec63494 fix(extract): apply resolver fixups to JS/TS dynamic_import handler
Third call site that re-implemented the same .js→.ts rewrite in
isolation. Previously only handled the explicit .js→.ts case; bare
paths, multi-dot helper files, and alias-resolved dynamic imports
all dropped silently.

Now uses _resolve_js_module_path on both branches (relative and
alias) — same shape as the static-import and Svelte regex paths.

Real-world impact: TS files using `await import('./foo')` patterns
for code splitting (e.g. lazy-loading a profanity check) now produce
edges to the resolved target.
2026-05-05 00:07:50 +02:00
Christian Winther 0dfc26e57f fix: prefer file matches over directory matches in resolver
When both a file (foo.ts) and a directory (foo/) exist at the same path,
both TypeScript and Vite prefer the file. The previous ordering checked
directory first and fell through unchanged when the directory had no
index, silently dropping every import like 'from ./auth' when an
auth/ subdirectory existed alongside auth.ts.
2026-05-04 23:44:32 +02:00
Christian Winther 5f5b59309c test(extract): cover .svelte.js + hybrid TS/JS Svelte 5 rune files
The generalized resolver already handles .svelte.js because the append
loop iterates _JS_RESOLVE_EXTS = (.ts, .tsx, .svelte, .js, .jsx, .mjs).
Adds three explicit tests to pin the behaviour and document the priority
choice:

  - test_resolve_svelte_to_svelte_js_for_javascript_rune_files
      JS-only Svelte 5 project: .svelte → .svelte.js works the same
      way as .svelte.ts in TS projects. No special-casing needed —
      the generalized append loop covers both.

  - test_resolve_svelte_prefers_svelte_ts_over_svelte_js
      Hybrid case (both files exist, e.g. .svelte.ts source plus
      .svelte.js build artifact): .ts wins. Documents the deliberate
      source-first priority — graphify is a source-code tool, not a
      runtime resolver, so we differ from Vite's default JS-first order.

  - test_resolve_real_svelte_file_wins_over_svelte_ts_sibling
      Existence check short-circuits before any extension append, so a
      real .svelte file always wins over a .svelte.ts sibling.
2026-05-04 22:47:29 +02:00
Christian Winther 49c3b50b5f fix(extract): generalize resolver to multi-dot filenames + rename
Two changes that landed together because they share the same code path:

1. Generalize the bare-path append to handle multi-dot filenames

   The previous resolver only appended extensions when path.suffix == ""
   (truly bare paths). Real codebases use a lot of multi-dot patterns:

     foo.shared.ts        ← imported as './foo.shared'
     foo.config.ts        ← imported as './foo.config'
     foo.compile.ts       ← imported as './foo.compile'
     foo.integration.ts   ← imported as './foo.integration' (test helper)
     foo.triggers.ts      ← imported as './foo.triggers'  (test helper)
     foo.svelte.ts        ← imported as './foo.svelte'    (Svelte 5 rune)
     foo.d.ts             ← imported as './foo.d'         (ambient types)

   For all of these, .suffix is the meaningful middle segment (.shared,
   .config, .integration, etc.) — not in the .js/.jsx/.svelte handled
   list, so the resolver fell through and the import dropped to a phantom.

   The fix unifies the bare-path and .svelte→.svelte.ts cases into a
   single rule: append each candidate extension to the FULL filename, not
   to the stripped stem. This subsumes:

     bare path:           foo           → foo.ts
     Svelte rune file:    foo.svelte    → foo.svelte.ts
     multi-dot helper:    foo.shared    → foo.shared.ts
     ambient declaration: foo.d         → foo.d.ts

   No behaviour change for paths that DO exist (.is_file() short-circuit)
   or for the .js→.ts / .jsx→.tsx convention (handled before the append
   loop so we don't accidentally match foo.js → foo.js.ts when foo.ts
   is the real file).

2. Rename _resolve_with_extensions → _resolve_js_module_path

   The function is JS/TS/Svelte-specific (Vite resolver order, mirrors the
   convention used by _import_js, _JS_CONFIG, _TS_CONFIG). The original
   name suggested it was a generic path utility. Renamed to make scope
   explicit and align with the existing _import_js / _JS_CONFIG naming
   pattern. Constants renamed to match: _JS_RESOLVE_EXTS, _JS_INDEX_FILES.

Tests
-----
4 new tests in tests/test_import_extension_resolution.py:

  - test_resolve_multi_dot_helper_file: foo.shared → foo.shared.ts
  - test_resolve_multi_dot_with_explicit_extension_still_works:
    foo.shared.ts (explicit) still wins
  - test_resolve_ambient_d_ts_via_bare_path: foo.d → foo.d.ts
  - test_end_to_end_multi_dot_import_resolves: tree-sitter pipeline
    sanity check via extract_js

Existing 28 tests updated for the rename. 32/32 pass; 7 pre-existing
unrelated failures elsewhere in the suite.

Validation
----------
On a 1,873-file SvelteKit codebase, applying both rules over the v0.7.5
baseline:

  baseline:                 12,096 edges
  with the resolver fix:    20,151 edges  (+8,055 = +67%)

The +2,652 over the previous version of this branch is attributable
entirely to multi-dot filename recovery, primarily test helper imports
('*.integration.ts', '*.triggers.ts'), domain-shared modules
('*.shared.ts'), and config files.
2026-05-04 22:41:50 +02:00
Christian Winther 0267cd789f test(extract): exhaustive coverage for extension resolution edge cases
Adds 8 tests covering import shapes that came up during real-codebase
validation against a 1,873-file SvelteKit project:

  - test_type_only_import_with_bare_path_resolves
      `import type { X } from './foo'` — type-only imports must go
      through the same resolver. Common pattern in TS codebases.

  - test_named_imports_emit_symbol_edges_after_resolution
      `import { foo, bar } from './module'` — verifies the per-symbol
      `imports` edges (file → module.foo, file → module.bar) target the
      correct stem after resolution. The symbol target_stem comes from
      _file_stem(resolved), so resolution must happen first.

  - test_alias_directory_import_resolves_to_index_ts
      `from '$lib/queue'` — alias + directory composes correctly.

  - test_resolve_does_not_match_partial_directory_name
      Regression guard: `from './foo'` where only `foo-extra.ts` exists
      must NOT accidentally resolve to it.

  - test_resolve_directory_without_index_returns_unchanged
      A directory with no index.* must fall through, not pick a random
      .ts inside.

  - test_resolve_handles_subpath_into_directory_with_index
      `./foo/sub` where `./foo/sub/index.ts` exists.

  - test_resolve_does_not_treat_dotfile_as_extension
      Path('.env-types.ts').suffix is '.ts' (correct), but worth pinning.

  - test_resolve_chain_alias_and_extension_compose
      Two-layer resolution: alias → bare path → .svelte.ts. Verifies
      the full chain works end-to-end for the Svelte 5 rune-file case.

Also expanded test_named_imports_emit_symbol_edges_after_resolution to
catch a subtle regression class: per-symbol import edges (line 319-340
in _import_js) build their target id from _file_stem(resolved). If
resolution fails or returns the wrong path, the symbol edges silently
target a different stem and downstream "where is X used?" queries miss
real callers.
2026-05-04 22:33:29 +02:00
Christian Winther 2b1efe8f08 fix(extract): TS bare-path / .svelte.ts / index.ts import resolution
_import_js previously only rewrote .js→.ts and .jsx→.tsx, leaving every
other common TypeScript / SvelteKit / Vite import shape unresolved. The
resulting node id wouldn't match the target file's own _make_id, so
build_from_json dropped the edge as external.

Three missed shapes:

  1. Bare paths (no extension) — TS convention:
     `import { foo } from './foo'`            → real file is foo.ts
  2. .svelte → .svelte.ts (Svelte 5 rune-only files):
     `import { x } from './x.svelte'`         → real file is x.svelte.ts
  3. Directory imports / barrel index files:
     `import { x } from './queue'`            → real file is queue/index.ts

Fix
---
New helper _resolve_with_extensions(p: Path) -> Path mirrors Vite/TS
resolver order:

  1. exact path (file)
  2. .js→.ts, .jsx→.tsx (existing TS-ESM convention)
  3. bare path → .ts/.tsx/.svelte/.js/.jsx/.mjs
  4. bare path → directory's index.{ts,tsx,js,jsx}
  5. .svelte → .svelte.ts (Svelte 5 rune file)

Falls back to the original path on no match — preserves pre-fix behaviour
for genuinely external modules (build_from_json drops them as phantoms).

Wired into _import_js (relative + alias branches) and extract_svelte's
regex pass for dynamic_import so static and dynamic imports both benefit.

Subtle: uses .is_file() / .is_dir() rather than .exists(). When the
import is a directory, .exists() returns True and would short-circuit
before the index.ts lookup ever ran.

Tests
-----
20 new tests in tests/test_import_extension_resolution.py:

  Resolver unit tests (12):
    - existing path returned unchanged
    - bare path → .ts / .tsx / .svelte
    - .ts wins over .svelte for ambiguous bare paths (Vite order)
    - directory → index.ts
    - directory prefers index.ts over index.js
    - .svelte → .svelte.ts (Svelte 5 rune file)
    - .js → .ts (TS ESM convention)
    - .jsx → .tsx
    - real .js stays .js when .ts doesn't exist
    - unresolvable returns input unchanged

  End-to-end (8):
    - bare-path import resolves in TS file
    - directory import resolves to index.ts
    - .svelte import resolves to .svelte.ts rune file
    - explicit .ts/.svelte imports still work (regression guard)
    - external module specifiers unchanged
    - alias + bare path resolves
    - dynamic_import bare path resolves
2026-05-04 22:26:48 +02:00
ibrahimyuecel eca3277fb9 feat(ts): extract interface, enum, type_alias, const literal, new_expression
Bring TypeScript AST extraction to parity with Java and C# by adding the
declaration types upstream graphify currently skips:

- interface_declaration (parity with Java/C# class_types)
- enum_declaration + members
- type_alias_declaration
- module-level const literals (object/array/string/call/new/template/number)
  via _js_extra_walk extension
- new_expression as call type for both _JS_CONFIG and _TS_CONFIG

Tested against tests/fixtures/typescript_advanced.ts: 8 expected node
types extracted (IUserRepository, UserStatus, UserId, USER_REPOSITORY,
DEFAULT_ROLES, USER_CONFIG, UserService, UserModule).

Validated on a 3,800-file TypeScript monorepo (NestJS + Next.js): yields
~1,885 interfaces, ~147 enums, ~405 type aliases, ~2,236 const literal
nodes, and ~1,935 instantiates edges that were previously invisible.
2026-05-04 20:46:36 +03:00
Safi 239000d2c9 Add incremental updates to graphify extract: semantic cache + build_merge + manifest 2026-05-04 18:07:02 +01:00
Safi ec413d5233 Wire deduplicate_entities into build() and build_merge() 2026-05-04 18:02:38 +01:00
Safi 34380434c4 Add graphify/dedup.py: entropy gate + MinHash/LSH + Jaro-Winkler entity deduplication 2026-05-04 18:01:11 +01:00
Farhan M 68081c1c89 feat(extract): add Markdown structural extraction + sync collect_files extensions
1. NEW: extract_markdown() — structurally indexes .md/.mdx files into the
   knowledge graph. Headings become nodes, code blocks become nodes with
   language tags, and nesting produces 'contains' edges. Zero new deps
   (pure regex/line-by-line parsing, no tree-sitter needed).

2. FIX: collect_files() _EXTENSIONS was hardcoded and missing 18 extensions
   that _DISPATCH already supported (.jsx, .mjs, .ex, .exs, .jl, .vue,
   .svelte, .dart, .v, .sv, .sql, .f, .F, .f90, etc). Now uses
   set(_DISPATCH.keys()) to stay automatically in sync.

3. Added deploy_guide.md test fixture and 6 new test cases.
4. Updated test_collect_files_from_dir to use dynamic extension set.
2026-05-04 19:08:50 +05:30
Safi b6ffdbb8dd v0.7.2: Fortran support + export CLI subcommands + skill.md size reduction
- Add Fortran support (26th language): .f/.F/.f90/.F90/.f95/.F95/.f03/.F03/.f08/.F08
  via tree-sitter-fortran; capital-F files preprocessed with cpp -w -P
- Add graphify export {html,obsidian,wiki,svg,graphml,neo4j} CLI subcommands
- Add graphify query/path/explain CLI subcommands
- Reduce skill.md from 63KB to 47KB by replacing Python heredocs with CLI calls
- Extend to_html() with node_limit param for auto-aggregation on large graphs
- Add integration tests for all export/query/path/explain subcommands

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-04 11:17:06 +01:00