#1118 — prune stale AST nodes on full re-extraction (#1116)
Stamps every AST-extracted node with _origin="ast" in extract(). On a
full rebuild _rebuild_code drops any AST-marked node absent from the
fresh output even when its source file survives, fixing stale symbols.
Backward-compat: marker-less nodes from pre-1118 graphs survive one
cycle then self-heal.
#1110 — stop reading images and PDFs as garbage in headless extract
Images route through per-backend vision payloads (base64/data-URI/bytes
for claude/openai/bedrock); non-vision backends get _strip_pixels for
graceful degradation. PDFs reuse pypdf. 5MB cap, 20-image chunk limit.
#1159 — Salesforce Apex extractor (.cls, .trigger)
Regex-based extractor: classes, interfaces, enums, methods, triggers,
SOQL/DML edges. No new dependency. Dispatched as .cls and .trigger.
#1107 — Azure OpenAI Service backend (--backend azure)
Uses AzureOpenAI SDK client (from existing openai package). Auto-detects
when AZURE_OPENAI_API_KEY + AZURE_OPENAI_ENDPOINT both set. Uses
max_completion_tokens (not deprecated max_tokens).
#1103 — live PostgreSQL introspection (--postgres DSN)
graphify extract --postgres "postgresql://..." introspects tables, views,
routines, and FK relations via information_schema (SERIALIZABLE READ ONLY).
Credentials sanitized on error. New graphify[postgres] extra (psycopg3).
Union-resolved llm.py conflict: Azure functions + bedrock images= param.
Fixed test_image_vision.py mock to accept timeout= kwarg (our #1112).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds graphify/mcp_ingest.py — extracts MCP server configurations into the
knowledge graph. Captures server nodes, NuGet/npm/pip package refs, commands,
env var requirements, and inter-server edges. Dispatched by filename before
the suffix lookup so generic .json extraction is unaffected. Env values are
discarded to prevent secret leakage. File size capped at 1 MiB. 29 tests.
Fixes: server_count budget now checked after validity guard so invalid entries
don't consume capacity; removed misleading uv run docstring example.
Co-Authored-By: adityachaudhary99 <adityachaudhary99@users.noreply.github.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add 'export_statement' to import_types for JS/TS/TSX configs
- Extend _import_js to detect 'export { X } from ./mod' re-exports
- Emit 're_exports' edges linking barrel files to source symbols
- Preserve walk-through for 'export function/const' declarations
- Add 're_exports' to clean_edges allowlist for cross-file edges
Tested on a 976-file Next.js codebase: detects 162 re_exports edges
and 5760 symbol-level imports (previously 0 for both).
tree-sitter-swift parses both `class Foo` and `extension Foo` as
`class_declaration`, and node ids carry the file stem, so `extension Foo`
in a sibling file produced a second `Foo` node instead of attaching to
the original. Same-file extensions already dedupe via seen_ids; only the
cross-file case leaked.
Per-file extraction now tags `extension` class_declarations, and the
corpus-level `extract()` runs a merge pass: when exactly one
non-extension declaration shares the label, the extension nodes redirect
onto it and their edges are rewritten (self-loops dropped, duplicates
collapsed). Extensions of types outside the corpus and ambiguous label
matches stay untouched.
On a 25-file Swift project this collapses Parser from 6 split nodes
(top of the god-node list, four entries) to one canonical node, and
lets the generic cross-file call resolver attach previously ambiguous
call edges to the right target.
Security: _is_sensitive now flags underscore-prefixed names (api_token.txt, oauth_token.json) by replacing \b with lookarounds; adds _SENSITIVE_DIRS check on parent path components (parts[:-1]) so .ssh/, secrets/, .aws/ directories are always skipped; aligns both patterns to (?![a-zA-Z]) for consistent underscore-after-keyword behavior (#920)
Fix: --wiki Relationships section always empty because _cross_community_links read community from node attrs (always None) instead of the communities dict; _god_node_article had the same bug and never linked to the owning community; fixed by building a node->community map in to_wiki() and threading it through (#925)
Fix: --watch now respects .graphifyignore; patterns loaded once at startup, handler checks _is_ignored before extension filter so node_modules/, .venv/, build/ churn no longer triggers rebuilds (#928)
Fix: C++ struct inheritance edges via base_class_clause; initialize base="" per iteration to prevent stale carryover (#915)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(extract): add Pascal/Delphi language support
Adds full AST extraction for Pascal and Delphi source files using
tree-sitter-pascal (https://github.com/Isopod/tree-sitter-pascal).
Supported file extensions: .pas, .pp, .dpr, .dpk, .inc
Extracted nodes:
- File node (the .pas file itself)
- unit / program / library declarations
- class, interface, and helper type declarations
- procedure and function implementations
Extracted edges:
- file --contains--> module
- module --imports--> dependency (via uses clause, resolved to path-based IDs)
- class --inherits--> base class / interface
- class/module --contains/method--> procedure or function
- procedure --calls--> procedure (in-file call resolution)
Key design: uses clause targets are resolved to path-based node IDs by
scanning all Pascal files under the project root (_pascal_project_root +
_pascal_resolve_unit helpers). This avoids dangling import edges that
result from resolving bare unit names like "SysUtils" to IDs that never
match any file node.
Bare procedure calls (e.g. `Reset;` without parentheses) are detected
by inspecting statement nodes whose sole named child is an identifier,
in addition to the standard exprCall nodes used for calls with arguments.
Requires: pip install tree-sitter-pascal
(https://github.com/Isopod/tree-sitter-pascal)
If not installed, extract_pascal returns {"nodes":[], "edges":[], "error": ...}
so the rest of the pipeline is unaffected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(extract): add Lazarus .lfm and .lpk file support
Adds two new extractors for Lazarus IDE-specific file formats:
extract_lazarus_form() — .lfm (Lazarus Form files)
.lfm files are text-based UI component trees. The extractor parses
`object Name: TClassName ... end` blocks to build a containment graph
of form components, and captures `OnXxx = HandlerName` event bindings
as `references` edges (context: "event") linking each component to
its handler procedure.
extract_lazarus_package() — .lpk (Lazarus Package files)
.lpk files are XML package definitions. The extractor reads the
package name, required package dependencies (→ imports edges), and
listed unit files (→ contains edges). Unit names are resolved to
path-based node IDs via _pascal_resolve_unit so they connect to the
same nodes produced by extract_pascal on .pas files.
Both extensions added to CODE_EXTENSIONS in detect.py and to _DISPATCH.
13 new tests in test_pascal.py cover both extractors.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(pascal): fix dangling inherits-edge targets and capture all base classes
The declType/typeref handler built inherits edge targets with
_make_id(_read(child)) — just the bare class name. But class nodes
use _make_id(stem, type_name), so targets never matched, making the
entire class hierarchy invisible in the graph.
Add _pascal_class_stem_cache and _pascal_resolve_class(): strips the
conventional T/I prefix, locates the defining file by stem lookup
(same cache mechanism as _pascal_resolve_unit), and returns the
correct _make_id(file_stem, class_name) ID. RTL/unresolvable bases
(e.g. TObject) fall back to _make_id(bare_name) with an explicit
stub node, following the same pattern as the Python extractor.
Also remove the `break` that stopped after the first typeref, so
all parents are captured (e.g. class(TBase, IInterface)).
Extend test_pascal_no_dangling_edges to also assert that within-file
edge targets (contains, method, inherits, calls) resolve to real nodes.
* feat(extract): add Delphi .dfm form file support
Adds extract_delphi_form() for Delphi Form files (.dfm), which use the
same `object Name: TClassName ... end` text syntax as Lazarus .lfm files.
Binary .dfm files (FF 0A magic header) are skipped gracefully with an
informative error message so the pipeline is unaffected. Text .dfm files
are parsed identically to .lfm: component containment (`contains` edges)
and event handler references (`references`, context "event").
Adds .dfm to _DISPATCH and CODE_EXTENSIONS.
10 new tests in test_pascal.py, including a regression test for the
binary-format detection.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Add .lpr extension and fix removesuffix in Pascal extractor
Address review feedback from @safishamsi:
- Add .lpr (Lazarus program file, identical syntax to .dpr) to _DISPATCH
in extract.py and CODE_EXTENSIONS in detect.py so Lazarus project entry
points are indexed. Completes the promised Lazarus IDE support.
- Replace rstrip("()") with removesuffix("()") in the call-resolution
dict comprehension for precise suffix removal (rstrip strips individual
characters, not the literal string "()").
- Add .lpr assertions to test_pascal_dispatch_registered and
test_pascal_detect_extensions_registered.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Simeon Bodurov <simeon.bodurov@speedy.bg>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The SQL parser (`extract_sql`) previously only extracted foreign key
relationships defined inline within CREATE TABLE column definitions.
FK constraints added via ALTER TABLE ... ADD CONSTRAINT ... FOREIGN KEY
... REFERENCES were silently ignored.
Additionally, `_obj_name()` only read the first identifier child of
object_reference nodes, so schema-qualified names like `Sales.Customer`
were truncated to just `Sales`.
Changes:
- Add `alter_table` handler to `walk()` that extracts FK edges from
ALTER TABLE ... ADD CONSTRAINT ... FOREIGN KEY ... REFERENCES
- Fix `_obj_name()` to read the full object_reference text, preserving
schema-qualified names (e.g. `Sales.Customer`)
- Fix inline FK resolution in create_table and _walk_from_refs to use
full object_reference text instead of first identifier only
tree-sitter-typescript ships two grammars:
- language_typescript: pure TypeScript, no JSX support
- language_tsx: JSX-aware variant for .tsx files
Currently both .ts and .tsx are parsed with language_typescript, which
treats JSX syntax as parse errors. Every function declaration, arrow
function, and call_expression nested inside a JSX tree is silently
dropped from the extracted graph.
Repro on a representative React+TypeScript codebase (a 13-file Tauri app):
parsing each .tsx with language_typescript produces ~276 ERROR nodes per
file. Only declarations that happen to live before the first JSX block
survive.
Fix: add _TSX_CONFIG that mirrors _TS_CONFIG but selects language_tsx,
and route .tsx files to it in extract_js().
Effect on the same repo (graphify update --force):
Nodes: 303 → 618 (+104%)
Edges: 482 → 779 (+62%)
Communities: 28 → 45 (+61%)
Parse errors 276 → 0 per .tsx file
Tests added:
- tsx fixture with helpers + JSX-returning component
- helpers and component are captured
- JSX expression calls ({fmtDate(now)}) resolve to call edges
- wiring check: .tsx uses language_tsx, .ts uses language_typescript
Note: this fixes the parsing layer. Calls inside deeply nested arrow
function callbacks (e.g. items.map(x => <T>{f(x)}</T>)) are still
missed by the call extraction logic — separate enhancement.
Co-authored-by: Serkan Gezici <serkan@quadroaipilot.com>
The JS/TS extractor only handled ES `import` statements; CommonJS
`require()` calls produced no import edges. Downstream, the call-graph
pass could not resolve which symbols belonged to which file, so every
cross-file call in CJS Node.js codebases was downgraded to INFERRED
even when the binding was a top-of-file destructured require.
Adds three patterns to `_js_extra_walk` via a new `_require_imports_js`
helper:
const { foo, bar: alias } = require('./mod') -> imports_from + per-symbol imports
const mod = require('./mod') -> imports_from
const x = require('./mod').y -> imports_from + symbol edge for y
Refactors path-resolution out of `_import_js` into
`_resolve_js_import_target` so ES imports and CJS requires share the
relative / tsconfig-alias / bare-module logic.
Tested in a 92-file CJS Node.js orchestrator codebase: confirmed all
five previously INFERRED `runExecute -> {loadFoundation,
validateDispatchConfig, fetchSymphonyIssues, listSymphonyWorktrees,
workspacePathForIssue}` edges resolve to real top-of-file destructured
requires, so downstream calls would now be EXTRACTED instead of
INFERRED.
Add `.luau` to CODE_EXTENSIONS and route it through extract_lua
(tree-sitter-lua). Roblox first-party code uses .luau; without this,
graphify silently skipped 379/479 files on a real Roblox codebase
and the resulting graph was dominated by vendored .lua dependencies.
tree-sitter-lua doesn't parse Luau type annotations, but it
successfully extracts function declarations and call edges from
Luau source — verified on a 379-file Roblox codebase (1265 nodes,
1471 edges, 236 communities).
A dedicated tree-sitter-luau grammar would be a richer long-term
fix; this is the minimal change to make Luau projects work today.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Register .groovy and .gradle in CODE_EXTENSIONS, _DISPATCH, and collect_files
- Add _GROOVY_CONFIG (reuses Java import handler)
- Add regex-based _extract_spock_fallback for Spock spec files where
tree-sitter-groovy wraps the body in ERROR nodes due to def-string methods
- _is_spock_file detects via regex scan (def "...") instead of node-label
heuristic, avoiding false negatives on classes whose name differs from stem
- Fallback retains only file node + import edges from tree-sitter pass to
prevent orphaned constructor/method nodes
- Add tree-sitter-groovy>=0.1.2 dependency
- Add 11 tests covering plain Groovy and Spock paths, including apostrophe
in feature method names
Bring TypeScript AST extraction to parity with Java and C# by adding the
declaration types upstream graphify currently skips:
- interface_declaration (parity with Java/C# class_types)
- enum_declaration + members
- type_alias_declaration
- module-level const literals (object/array/string/call/new/template/number)
via _js_extra_walk extension
- new_expression as call type for both _JS_CONFIG and _TS_CONFIG
Tested against tests/fixtures/typescript_advanced.ts: 8 expected node
types extracted (IUserRepository, UserStatus, UserId, USER_REPOSITORY,
DEFAULT_ROLES, USER_CONFIG, UserService, UserModule).
Validated on a 3,800-file TypeScript monorepo (NestJS + Next.js): yields
~1,885 interfaces, ~147 enums, ~405 type aliases, ~2,236 const literal
nodes, and ~1,935 instantiates edges that were previously invisible.
1. NEW: extract_markdown() — structurally indexes .md/.mdx files into the
knowledge graph. Headings become nodes, code blocks become nodes with
language tags, and nesting produces 'contains' edges. Zero new deps
(pure regex/line-by-line parsing, no tree-sitter needed).
2. FIX: collect_files() _EXTENSIONS was hardcoded and missing 18 extensions
that _DISPATCH already supported (.jsx, .mjs, .ex, .exs, .jl, .vue,
.svelte, .dart, .v, .sv, .sql, .f, .F, .f90, etc). Now uses
set(_DISPATCH.keys()) to stay automatically in sync.
3. Added deploy_guide.md test fixture and 6 new test cases.
4. Updated test_collect_files_from_dir to use dynamic extension set.
- Add Fortran support (26th language): .f/.F/.f90/.F90/.f95/.F95/.f03/.F03/.f08/.F08
via tree-sitter-fortran; capital-F files preprocessed with cpp -w -P
- Add graphify export {html,obsidian,wiki,svg,graphml,neo4j} CLI subcommands
- Add graphify query/path/explain CLI subcommands
- Reduce skill.md from 63KB to 47KB by replacing Python heredocs with CLI calls
- Extend to_html() with node_limit param for auto-aggregation on large graphs
- Add integration tests for all export/query/path/explain subcommands
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit adds full VB.NET language support to graphify, raising the
supported language count from 25 to 26. The implementation follows the
established LanguageConfig pattern used by all other tree-sitter-backed
extractors.
New dependency:
- Adds optional extra [vbnet] backed by tree-sitter-vbnet (published
to PyPI at https://pypi.org/project/tree-sitter-vbnet/0.1.0/).
Install with: pip install graphifyy[vbnet]
graphify/detect.py:
- Added .vb to CODE_EXTENSIONS so VB.NET files are discovered during
corpus ingestion and file-system watching.
graphify/extract.py:
- _import_vbnet(): import handler for imports_statement nodes; emits
imports edges using the namespace_name child text.
- _vbnet_extra_walk(): extra-walk hook that intercepts namespace_block
nodes, emits a namespace node, and recurses.
- _VBNET_CONFIG: full LanguageConfig covering class_block / module_block /
structure_block / interface_block as class types; method_declaration /
constructor_declaration / property_declaration as function types;
invocation call nodes with target/member_access fields.
- VB.NET-specific branches in _extract_generic:
* Class body: VB.NET has no wrapper body node; inherits and implements
are named fields directly on the class_block. Emits separate inherits
and implements edges for each base type, stripping generic arguments.
* Constructor name: constructor_declaration carries no name field in
the grammar; always resolves to New.
* Function body: uses the declaration node itself as body sentinel so
the call-graph pass can find invocations inside methods.
- extract_vbnet(path): public wrapper that delegates to _extract_generic.
- _DISPATCH['.vb']: routes .vb files to extract_vbnet.
pyproject.toml:
- Added vbnet = ['tree-sitter-vbnet'] optional dependency group.
- Added 'tree-sitter-vbnet' to the all extra.
tests/fixtures/sample.vb:
- New fixture file exercising: Imports statements, Namespace block,
Interface, Class with Inherits + Implements, Module, Structure,
Sub/Function/Property methods, and method calls.
tests/test_languages.py:
- Added 13 tests covering: no-error, class/interface/module/structure
detection, method detection, imports relation, inherits edge,
implements edge, and no-dangling-edges invariant.
README.md:
- Updated language count 25 to 26.
- Added VB.NET to language list and file-extension table.
- extract_sql(): deterministic tree-sitter extraction of tables, views,
functions, foreign key references, and FROM/JOIN reads_from edges
- .sql added to CODE_EXTENSIONS and dispatch table
- tree-sitter-sql added as optional dep under [sql] extra
- xlsx_extract_structure(): extracts sheet/table/column nodes from .xlsx
(utility — pipeline wiring in follow-up)
- 6 new SQL tests, 447 total passing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
import(`./handlers/${name}`) previously produced a garbage edge to a
path containing the unresolved ${name} expression. Now detects
template_substitution child nodes and breaks without emitting an edge.
Static template literals (no interpolation) still resolve correctly.
Adds 2 new tests: one asserting dynamic templates produce no edge,
one asserting static templates resolve like plain strings.
Adds _dynamic_import_js() helper (65 lines) that detects import() call
expressions in JS/TS, resolves the module path (same logic as static
imports including .js→.ts mapping and tsconfig aliases), and emits
imports_from edges from the enclosing function. Hooked into walk_calls
for JS/TS configs.
Also adds tests/fixtures/dynamic_import.ts fixture and 5 new tests
in tests/test_languages.py (all passing alongside 110 existing tests).
* feat: add Swift language support
Add tree-sitter-swift extractor for classes, structs, protocols,
functions, imports, and call graph edges. Includes 8 passing tests.
* feat: full Swift AST support — enums, extensions, actors, conformance
- Enums: extract enum types, methods, and cases (case_of edges)
- Extensions: methods attach to the original type (no duplicate nodes)
- Actors: recognized via unified class_declaration node type
- Conformance/inheritance: inherits edges from : Protocol syntax
- deinit/subscript: name resolution for nameless declarations
- 12 new tests (110 total, all passing)
style: replace all em dashes with hyphens
fix: explain hidden .graphify/ folder in skill output and README
fix: rename .graphify/ to graphify-out/ so output is visible by default
Safi