Commit Graph

209 Commits

Author SHA1 Message Date
Safi 435da06c70 Release 0.8.43
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 10:31:01 +01:00
Safi 9e8fa2a28c Bump vulnerable dependencies to patched versions (#1375)
pypdf 6.11.0->6.13.3 (CVE-2026-48155/48156), yt-dlp 2026.3.17->2026.6.9,
pyjwt 2.12.1->2.13.0, cryptography 48.0.0->49.0.0, python-multipart
0.0.28->0.0.32, with lower-bound floors for the direct deps (pypdf, yt-dlp) so
installs get the patched versions. Lockfile regenerated for 0.8.42.

Co-Authored-By: hypnwtykvmpr <hypnwtykvmpr@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 22:48:50 +01:00
Safi baf2410f08 Release 0.8.42
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 20:05:25 +01:00
Safi 9e0b8766f4 Release 0.8.41
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 19:09:43 +01:00
Safi fd152add64 Release 0.8.40
Bump version to 0.8.40; document this release in CHANGELOG (PowerShell .psm1
indexing, Swift import survival, no-cluster edge dedup, custom OpenAI/Anthropic
endpoints, JS/TS assignment-form extraction, community-name + --graph fixes,
four production bug fixes, perf, security CI); add .psm1 to the README
supported-extensions table.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 02:38:28 +01:00
Safi c4689a2f47 bump version to 0.8.39 2026-06-12 20:36:47 +01:00
Safi 70a1ad7278 Merge pull request #1176 from galshubeli/falkordb-backend
feat: add FalkorDB export backend (#1175)
2026-06-12 20:15:30 +01:00
Safi 70380b8ea0 Merge pull request #1208 from nucleusjay/pin-tree-sitter-deps
Pin tree-sitter dependency version ranges
2026-06-12 20:15:27 +01:00
Safi 1bb30fcc56 bump version to 0.8.38 2026-06-11 23:46:54 +01:00
Safi e6d2dc4d6f release 0.8.37 2026-06-10 21:45:02 +01:00
Safi 5504c84324 perf/fix: replace datasketch with pure-numpy MinHash; memoize detect ignore checks
- graphify/_minhash.py: self-contained MinHash/MinHashLSH using pure numpy,
  byte-identical hash math to datasketch (sha1_hash32, Mersenne-prime permutation).
  Drops datasketch + scipy transitive dep — eliminates EDR hang on Windows where
  numpy.testing platform.machine() subprocess spawn was intercepted at import time
- dedup.py: import from graphify._minhash instead of datasketch
- pyproject.toml: replace datasketch>=1.6 with numpy>=1.21
- detect.py: memoize _is_ignored/_eval results in a dict[Path,bool] cache per
  detect() call; each unique ancestor dir evaluated once across all sibling files,
  eliminating ~42M redundant fnmatch calls on large repos (~34% whole-run speedup)
- tests/test_minhash.py: 11 tests including import-isolation guard asserting scipy
  and numpy.testing are not loaded after import graphify.dedup
- tests/test_detect.py: 2 cache tests — correctness (cached==uncached with negation
  patterns) and hit-count (each dir evaluated exactly once across siblings)

Closes #1234, closes #1235

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 15:50:38 +01:00
nucleusjay 6f6d74d42d Pin tree-sitter dependency version ranges
All 27 tree-sitter-* deps were unversioned in pyproject.toml. Users
installing via 'pip install graphifyy' (the README's primary install
path) bypass uv.lock entirely and resolve whatever tree-sitter-*
versions PyPI happens to serve. A breaking minor bump in any grammar
package can land in user installs without notice.

Add explicit lower bounds (matching uv.lock) and upper bounds one
minor above (or one major above for 0.x packages with frequent breaks).
Ranges chosen to allow patch updates without re-pinning while blocking
incompatible major/minor jumps.
2026-06-09 03:35:45 -04:00
Safi 137bc2d951 release 0.8.36
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 23:58:08 +01:00
Jan 42d1b8d02f feat: add graphify-mcp console script entry point (#1190)
Adds `graphify-mcp` as a named console script pointing to `graphify.serve:_main`, making the MCP stdio server directly invocable as a first-class CLI command from uv tool / pipx installs. MCP client configs can now use `"command": "graphify-mcp"` instead of `python -m graphify.serve`.

Co-authored-by: jr2804 <jr2804@users.noreply.github.com>
2026-06-08 10:35:21 +01:00
Safi 6dbcb89f3d release 0.8.35
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 14:42:30 +01:00
Gal Shubeli 7aa300709f feat: add FalkorDB export backend (#1175)
Adds FalkorDB as a sibling option to the existing Neo4j sink, selected via
`graphify export falkordb [--push redis://localhost:6379]`.

- New push_to_falkordb() in graphify/export.py mirrors push_to_neo4j; FalkorDB
  is OpenCypher-compatible so the MERGE/SET upsert queries are identical.
- export falkordb subcommand wired in graphify/__main__.py (cypher.txt when no
  --push, direct push otherwise). Auth is optional; target graph defaults to
  "graphify".
- falkordb optional extra in pyproject.toml (and in the all extra).
- Tests: CLI cypher generation (CI-safe) + real-FalkorDB integration tests that
  skip when no instance is reachable.
- README extras table + command reference and CHANGELOG updated.
2026-06-07 15:03:49 +03:00
Safi 3744e6e220 release 0.8.34
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 12:51:33 +01:00
Safi f146be3ea2 fix: numpy Python 3.13 pin + codex skill dir (#1154 #1160)
#1154: scope numpy>=2.0 constraint to python_version>='3.13' only.
numpy 1.26.4 ships no cp313 wheel so uv sync falls back to a source
build requiring a C compiler. The marker avoids forcing numpy 2.x on
3.10-3.12 users who have working 1.x environments.

#1160: codex platform skill now installs to .codex/skills/graphify/
instead of .agents/skills/graphify/. The hook already wrote to .codex/
so the skill destination was inconsistent. Propagates automatically
through install/uninstall (both read _PLATFORM_CONFIG dynamically).
Updated all codex-specific test assertions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 12:11:34 +01:00
Safi 7467c1b6a4 feat/fix: land PRs #1118 #1110 #1159 #1107 #1103 (graph quality + new features)
#1118 — prune stale AST nodes on full re-extraction (#1116)
Stamps every AST-extracted node with _origin="ast" in extract(). On a
full rebuild _rebuild_code drops any AST-marked node absent from the
fresh output even when its source file survives, fixing stale symbols.
Backward-compat: marker-less nodes from pre-1118 graphs survive one
cycle then self-heal.

#1110 — stop reading images and PDFs as garbage in headless extract
Images route through per-backend vision payloads (base64/data-URI/bytes
for claude/openai/bedrock); non-vision backends get _strip_pixels for
graceful degradation. PDFs reuse pypdf. 5MB cap, 20-image chunk limit.

#1159 — Salesforce Apex extractor (.cls, .trigger)
Regex-based extractor: classes, interfaces, enums, methods, triggers,
SOQL/DML edges. No new dependency. Dispatched as .cls and .trigger.

#1107 — Azure OpenAI Service backend (--backend azure)
Uses AzureOpenAI SDK client (from existing openai package). Auto-detects
when AZURE_OPENAI_API_KEY + AZURE_OPENAI_ENDPOINT both set. Uses
max_completion_tokens (not deprecated max_tokens).

#1103 — live PostgreSQL introspection (--postgres DSN)
graphify extract --postgres "postgresql://..." introspects tables, views,
routines, and FK relations via information_schema (SERIALIZABLE READ ONLY).
Credentials sanitized on error. New graphify[postgres] extra (psycopg3).

Union-resolved llm.py conflict: Azure functions + bedrock images= param.
Fixed test_image_vision.py mock to accept timeout= kwarg (our #1112).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 01:20:38 +01:00
Safi 4b06fd7bf7 release 0.8.33
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 09:40:14 +01:00
Safi ec1a3bc9e9 release 0.8.32
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 22:50:47 +01:00
Safi 200edecea5 feat(extract): add Terraform/HCL AST extraction via tree-sitter-hcl (#1129)
Adds extract_terraform() for .tf/.tfvars/.hcl files. Nodes: resources,
data sources, modules, variables, outputs, providers, locals. Edges:
contains, references (interpolation), depends_on. Node IDs are
directory-scoped for cross-file resolution. tree-sitter-hcl added as
optional extra (graphify[terraform]) matching sql/dm precedent.
8 tests including .tfvars documented no-op behaviour.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 21:38:56 +01:00
William Mcgrann 830f0396d8 build: add version floors to networkx, datasketch, rapidfuzz
Adds lower-bound version constraints matching tested versions in uv.lock. networkx>=3.4 is the binding constraint (edges= kwarg on node_link_data introduced in 3.4). rapidfuzz>=3.0 guards against the 2.x→3.x API break in rapidfuzz.distance. datasketch>=1.6 is conservative but correct. Also fixes stale uv.lock version metadata (0.8.30→0.8.31).
2026-06-05 10:14:46 +01:00
Safi 7a588fb1d2 release 0.8.31 2026-06-03 21:04:01 +01:00
Safi 724f1e3186 release 0.8.30
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 00:26:14 +01:00
Safi 250834a0c4 add anthropic extra and fix backend-missing hints for uv tool installs
uv tool install graphifyy runs graphify in an isolated venv, so when a backend
needs anthropic/openai/boto3 the old 'Run: pip install anthropic' advice never
reached it. Worse, claude was the only backend with no [extra] at all, so a user
with ANTHROPIC_API_KEY set could not satisfy it without --with anthropic.

Add an anthropic optional extra (and include it in [all]), and replace the
package-missing ImportError messages with a shared _backend_pkg_hint that points
at 'uv tool install graphifyy[<extra>] --force' first, then the pip/venv path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 00:08:03 +01:00
Safi 345e1122d3 release 0.8.29
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 23:19:48 +01:00
Safi adfe8f0794 make skillgen's toml import work on Python 3.10
gen.py imported tomllib unconditionally, but that is 3.11+ stdlib only and
graphify supports (and CI tests) 3.10 - so importing tools.skillgen.gen, and
thus collecting test_skillgen.py, crashed the 3.10 test job with
ModuleNotFoundError. Fall back to the tomli backport on <3.11 and add tomli to
the dev group under that marker. Verified the fallback loads and parses with
tomllib forced absent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 21:12:43 +01:00
Soham Patankar fbe1e9977f feat(skills): progressive-disclosure split for all platforms (generator + drift fence) (#1121)
All-platform progressive-disclosure skill split + generator (addresses #1106).

Splits each platform's skill into a lean core (~615 lines, full default pipeline inline) + on-demand references/, generated from a single source via tools/skillgen with a CI/pre-commit drift gate. 13 hosts split, aider/devin stay monoliths. Also fixes the stale bare-path bugs across the previously hand-maintained variants and moves the always-on blocks into packaged markdown.

Verified: all 5 generator guards pass, byte-verbatim load-bearing slices, lean cores self-sufficient on the default path across all 13 split hosts, references gated to non-default branches, description preserves the graphify-out-query-first clause. Supersedes #1119 (Claude-first subset).

Known follow-up applied on top: harden _always_on() against a missing packaged file so a partial install can't brick the CLI.
2026-06-02 20:48:13 +01:00
Safi 1d4e11226f make tree-sitter-dm an optional extra to fix default install (fixes #1104)
tree-sitter-dm (BYOND DreamMaker) publishes only a Windows wheel, so on
Linux/macOS uv/pip compiled it from source and aborted the entire
`uv tool install graphifyy` when a C toolchain or python3-dev was missing.
It is the only core grammar lacking Linux/Mac wheels. Move it from core
dependencies to an optional `dm` extra (also in `all`); the default install
now needs no compiler. extract_dm already imports the grammar lazily with a
graceful fallback, and .dmi/.dmm/.dmf use no tree-sitter, so only .dm/.dme
AST parsing is gated behind the extra.

Also: guard the .dm grammar tests with pytest.importorskip so a plain
`uv sync` (no extras) doesn't hard-fail; bump to 0.8.28 with a changelog
upgrade note; switch the README extras table + type notes from pip to
`uv tool install` (uv is the recommended installer, and pip-into-a-uv-tool-env
is the exact ModuleNotFoundError the README already warns about). 0.8.28 also
ships Kilo Code (#512) and the Dart parser modernization (#1098).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 21:44:59 +01:00
alacasse a8005c218a feat: add Kilo Code support (#512)
Adds Kilo Code as a supported platform: native skill + /graphify command, install/uninstall, and a .kilo tool.execute.before plugin (mirroring the OpenCode integration).

JSONC config is handled non-destructively - existing .kilo/kilo.jsonc is read but never rewritten; automated plugin registration goes to kilo.json, preserving user comments (addresses the Qodo review flag).

Verified locally: full suite 1525 passed, install/uninstall smoke test works.
2026-06-01 16:05:07 +01:00
Safi 3922235eba release 0.8.27
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 23:31:50 +01:00
Safi 55d7dadb63 bump to 0.8.26
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-30 21:40:52 +01:00
Safi a7a732290c release 0.8.25: JS/TS phantom nodes, Lua imports, Windows cli, hook lock, antigravity path
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-29 16:59:17 +01:00
Safi fd1aca480f release 0.8.24: cross-language semantics, claude-cli fix, edge direction fix, security audit
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-29 02:28:52 +01:00
Danil Tarasov 0080fbd13c feat: add objc, julia, c, c++, scala, fortran, powershell semantic contexts 2026-05-29 00:05:28 +03:00
Safi ffbe425c2f release 0.8.23
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-28 14:52:01 +01:00
Safi 43bf3c2371 release 0.8.22
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-28 00:26:31 +01:00
Safi dacbdb539a feat: BYOND DreamMaker support, --mode deep flag, changelog fixes (#884, #1030)
- Feat: extract_dm (tree-sitter-dm), extract_dmi (PNG icon states),
  extract_dmm (tile dict uses edges), extract_dmf (window/elem hierarchy)
  for .dm .dme .dmi .dmm .dmf; 26 tests, fixtures, pyproject.toml dep
- Feat: graphify extract --mode deep flag; deep_mode threaded through all
  four LLM backends via extract_corpus_parallel
- Fix: CHANGELOG 0.8.21 entries for #1050, #1046, #1047 that were missing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-27 23:55:12 +01:00
Safi 80301a06bf fix: OpenCode project path, hook loop guard, deterministic output, Amp platform, punctuation search, builtin god-node filter, .svh Verilog (#1040, #1018, #1037, #948, #994, #916, #1042)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-27 12:39:22 +01:00
Safi b07f0eb6a7 bump version to 0.8.20 2026-05-26 20:34:22 +01:00
Safi d9debdf39a bump version to 0.8.19 2026-05-26 12:52:04 +01:00
Candy c7a05d67a1 Clarify Chinese query segmentation scope (#1026)
Keep Graphify query segmentation focused on Chinese terms: rename the CJK helpers and extra to Chinese scope, cache the optional jieba import at module load, and keep a bigram fallback when jieba is unavailable.

Constraint: Reviewer asked either to broaden Hiragana/Katakana/Hangul support or rename CJK helpers; user chose Chinese-only because Japanese segmentation accuracy is uncertain.

Rejected: Broaden to Japanese and Korean segmentation | jieba is Chinese-oriented and the user explicitly limited scope to Chinese.

Confidence: high

Scope-risk: narrow

Directive: Do not label this path as CJK unless Hiragana/Katakana/Hangul segmentation is intentionally supported and tested.

Tested: uv run --with pytest pytest tests/test_serve.py tests/test_query_cli.py tests/test_benchmark.py

Tested: uv run --with pytest --with jieba pytest tests/test_serve.py -k "chinese or non_chinese"

Tested: graphify update .

Not-tested: Full test suite.

Co-authored-by: OmX <omx@oh-my-codex.dev>
2026-05-26 12:24:04 +01:00
vishal-gandhi 065a621fa6 feat: add Devin CLI support (graphify devin install/uninstall) (#1020)
* feat: add Devin CLI support

- Add 'devin' to _PLATFORM_CONFIG with skill-devin.md
- Override _platform_skill_destination for devin:
  user scope -> ~/.config/devin/skills/graphify/SKILL.md
  project scope -> .devin/skills/graphify/SKILL.md
- Add _DEVIN_RULES + _devin_rules_install/_uninstall:
  writes .windsurf/rules/graphify.md (always-on context)
  Devin reads .windsurf/rules/*.md the same way Windsurf IDE does
- Add graphify devin install [--project] / uninstall subcommand
- Create graphify/skill-devin.md with Devin YAML frontmatter
  (name, argument-hint, model, allowed-tools, triggers)
- Add 21 tests in tests/test_devin.py covering user/project scope
- Register _DEVIN_RULES in test_install_strings.py policy checks
- Update README.md and pyproject.toml to list Devin CLI

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* expand skill-devin.md to full feature parity with skill-pi.md

Rebuilds skill-devin.md from ~267 lines to 1176 lines, matching the
depth of other mature platform skills (pi, droid, codex).

Added:
- Parallel Part A + B dispatch guidance (AST alongside semantic subagents)
- Full subagent extraction prompt with rationale, semantic similarity,
  hyperedge, and confidence_score rules
- Extraction cache check (Step B0) and cache save/load flow
- --mode deep, --svg, --graphml, --neo4j/--neo4j-push, --mcp, --watch
- /graphify path, /graphify explain, /graphify add, /graphify query
- BFS vs DFS traversal with --dfs and --budget flags
- Token reduction benchmark step
- --update (incremental) and --cluster-only pipelines with graph diff
- Video/audio transcription (Step 2.5) with Whisper domain-hint strategy
- git commit hook section
- Always-on Devin rules section referencing `graphify devin install`

Updated test_devin.py: corrected python_c_syntax assertion to match
the interpreter-detection pattern used across all mature skills.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: vishal-gandhi <ivishalgandhi@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-05-26 09:17:18 +01:00
Safi 98100f3b35 Bump version to 0.8.18 2026-05-24 20:41:31 +01:00
Safi 73c3c33ac7 Bump version to 0.8.17 2026-05-23 23:36:56 +01:00
Safi 990ac706d8 bump version to 0.8.16
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 18:58:31 +01:00
Safi ff14ad5245 bump version to 0.8.15
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 14:04:54 +01:00
hypnwtyk b6127aa5a7 feat(multigraph): add runtime compatibility probe (#956)
* feat(bash): harden extractor — literal filtering, entrypoint nodes, AST-ancestry-aware command detection

Builds on tree-sitter-bash extractor from #866. Two correctness/security
improvements to bash extraction in graphify/extract.py:

1. Reject command/process substitutions at extraction time. Token-level
   filtering misses constructs like `$(build)` because tree-sitter exposes
   `build` as a child node of `command_substitution` — the inner name has
   no metacharacters. Added `is_inside_expansion(node)` that walks
   `node.parent` until it finds `command_substitution` or
   `process_substitution`. Used as a gate in both `walk` and `walk_calls`.
   Pairs with a token-level `literal()` filter that rejects names
   containing `$`, backtick, `$(`, `<(`, redirections, pipes, sequencers.

2. Entrypoint node. Every .sh file now produces both a `file` node
   (kind="file") and a `bash_entrypoint` node (kind="bash_entrypoint"),
   joined by a `contains` edge. A separate top-level `walk_calls(root,
   entry_nid, ...)` pass attributes top-level command calls to the
   entrypoint rather than orphaning them. Matches the entrypoint pattern
   other-language extractors use. Node metadata gains language+kind.

Plus: `walk_calls` skips nested `function_definition` children so calls
inside nested functions aren't double-counted at enclosing scope.

Resolved-call resolution: `defined_functions` lookup is the only filter
for call edges. User-defined functions named like external commands
(install, find, git, ...) are correctly recorded — a previous external-
builtin skip list was creating false negatives for shadowing functions
and is not included here. Skip list belongs with raw/unresolved call
recording (not in this PR).

Devtools (bundled): pyproject.toml gains [dependency-groups] dev (ruff,
pyright, pre-commit, hypothesis, pip-audit) plus minimal [tool.ruff],
[tool.ruff.lint], [tool.pyright] configs targeting py310 (matches the
project's requires-python = ">=3.10").

Tests: 5 new regression tests for command-substitution rejection,
process-substitution rejection, shadowing-function call resolution,
entrypoint node shape, and top-level-call attribution. 826/826 pass
(was 821); 15/15 bash-relevant tests pass (was 10).

* feat(detect): parse macOS/BSD and GNU env(1) shebang option forms

Upstream's _shebang_file_type parses shebangs via line[2:].split() and only
handles `#!/usr/bin/env <interp>`. Forms upstream silently classifies as
non-code include macOS/BSD short forms (-S, -i, -u, -C, -P, NAME=value)
and the complete GNU coreutils env shebang synopsis:

    #!/usr/bin/env -[v]S[option]... [name=value]... command [args]...

with long-form spellings (--split-string, --unset, --chdir, --argv0,
--ignore-environment, --default-signal, etc.), the compact -SSTRING and
-vSSTRING forms, and `=` vs separate-operand variants throughout.

Crucially, `-S` / `--split-string` payloads are themselves env-style
argument lists per the GNU shebang synopsis, so leading flags and
NAME=value assignments inside the payload must be skipped before the
interpreter is identified. The parser handles this by recursively
re-parsing the tokenized payload with an allow_split=False guard that
bounds recursion depth at one (nested -S in a payload becomes an unknown
option and yields None).

Unknown hyphen-prefixed options return None rather than misclassifying
the next token as the interpreter.

_shebang_file_type becomes a 4-line wrapper. Read buffer raised 128 -> 256
to accommodate longer env -S strings.

Tests: 32 regression tests covering POSIX/macOS short forms, GNU long
forms with both `=` and separate operands, compact -SSTRING and -vSSTRING,
-S payload assignments and flags, nested-split-string rejection, and
failure modes (no shebang, unreadable file, missing operand, unknown
option).

* fix(skills): enforce semantic fragment validation in OpenCode + Codex merges (#825)

Closes #825. Adds graphify.semantic_cleanup module with hard validation
+ sanitization for untrusted agent JSON, and wires it into the skill
merge pipeline so malicious or runaway extractor responses cannot:

- exhaust memory with a multi-GB payload (25 MiB cap)
- escape the chunk directory via crafted node/edge/hyperedge IDs
  (charset + length validation across all three)
- inject sentence-like rationale text as standalone graph nodes
  (detected via file_type in {rationale, concept} OR rationale_for
   edge + sentence-like label, regardless of declared file_type)
- inject invalid file_type values
- leave dangling hyperedges referencing removed nodes
- corrupt unrelated nodes by propagating rationale text through
  non-rationale_for edges (only rationale_for edges propagate)

Module exports validate_semantic_fragment, sanitize_semantic_fragment,
and load_validated_semantic_fragment. Wired into skill-opencode.md and
skill-codex.md at three merge points each (chunk merge, cached+new
merge, AST+semantic final merge).

Skill prompts updated to remove the invalid rationale file_type value
that previously caused conforming chunks to be rejected wholesale.
Valid set is now {code, document, paper, image}.

Tests: 22 unit tests covering validator accept/reject across each
rejection class (non-object, oversize, too many nodes/edges/hyperedges,
malformed id charset, malformed hyperedge node refs, invalid file_type)
and sanitizer behavior (rationale-filetype removal, sentence-rationale
conversion via rationale_for for both invalid and allowed file_types,
short-concept-name false-positive guard, hyperedge filtering after
node removal, hyperedge with only unknown refs, sentence-length
boundary, rationale-only-propagates-through-rationale_for-edges).

880/880 tests pass.

* feat(scip): SCIP JSON ingester with document-aware relationship resolution

Adds graphify.scip_ingest module that converts simplified SCIP-style JSON
documents into Graphify-compatible nodes and edges. Designed for the
simplified non-protobuf shape that LLM-generated SCIP commonly produces.

Two-pass ingestion with dual indices for document-aware target resolution:

  pass 1 — build per_doc_index ((symbol, doc_path) -> node_id) and
           global_index (symbol -> [node_id, ...]) across every valid
           symbol in every valid document. Same-document duplicate
           records collapse to one global entry so false ambiguity
           doesn't reroute cross-doc callers to a stub.
  pass 2 — emit nodes for indexed symbols, then walk relationships.
           Resolution order:
             1. same-doc match (per_doc_index)
             2. unique cross-doc match (global_index[symbol] len == 1)
             3. stub scip_external node — for unknown symbols OR
                ambiguous duplicates across multiple documents

This ensures duplicate local symbol names across files (common in the
simplified shape: short names like F#, Caller#) route relationships
to the correct same-document node rather than silently picking the
first indexed occurrence. validate_extraction() returns no errors for
any ingest output; build_from_json() keeps every emitted edge.

Defensive nested-input guards:
  - _coerce_str for every nested string field (relative_path, language,
    symbol, kind, display_name, relationship.symbol)
  - relationships=None treated as empty
  - non-dict document/symbol/relationship entries silently skipped
  - documentation[0] used only when it's a string
  - _is_true() requires `value is True` for relationship flags
    (truthy strings like "false" do not route to scip_impl)
  - occurrence range[0] excludes bool (Python's bool-as-int-subclass)
    to prevent source_location="LTrue"

Module is stdlib-only (hashlib, re, typing.Any). Not wired to the CLI
in this phase — importable as `from graphify.scip_ingest import
ingest_scip_json`.

Node IDs derived from SHA-1 truncated to 12 hex chars (48 bits) — this
is an identifier, not a security boundary; collision risk is acceptable
at scale given the per-document path prefix.

Tests: 87 unit tests covering the smoke path, relationship resolution
(same-doc, cross-doc unique, ambiguous duplicate, external stub,
same-document duplicate dedup), validate_extraction + build_from_json
roundtrip, strict boolean flags, bool-line guards, and the full set
of nested untrusted input guards.

1044/1044 tests pass.

* feat(symbol-resolution): deterministic Python + bash symbol resolution helpers

Adds graphify.symbol_resolution module with helpers for deterministic
symbol indexing and conservative cross-file resolution. Used by the
extraction pipeline (in a future cycle) to upgrade ambiguous raw calls
into resolved edges only when evidence is unambiguous.

Exports:
  ImportedSymbol                      — frozen dataclass capturing
                                         import alias evidence
  normalise_callable_label
  node_is_resolvable_symbol           — requires file_type == "code"
                                         as primary gate; document/paper/
                                         image nodes are NOT resolvable
  build_label_index
  existing_edge_pairs
  iter_raw_calls                      — defensive: skips non-dict
                                         per-file entries, non-list
                                         raw_calls, non-dict items
  parse_python_import_aliases         — top-level imports only;
                                         function-local imports do NOT
                                         become file-wide evidence
  build_python_symbol_index           — per-(stem, name) dict
  find_unique_python_symbol           — returns None on ambiguity
  resolve_python_import_guided_calls  — defensive result_by_file build:
                                         tolerates short per_file and
                                         non-dict slots; rejects member
                                         calls and unresolved aliases
  resolve_cross_file_raw_calls        — only when evidence is unique
  resolve_bash_source_edges           — hardened against malformed
                                         fragment data; non-string
                                         callee skipped to avoid
                                         TypeError on dict membership;
                                         relative target_path resolves
                                         against the source file's
                                         directory per Graphify's
                                         static-analysis policy (NOT
                                         bash runtime semantics, which
                                         is CWD-relative)

Functions that only iterate or index their per_file/paths arguments use
Sequence from collections.abc for proper covariance. Public defensive
entry points (iter_raw_calls, resolve_python_import_guided_calls) accept
Sequence[object] so callers can pass arbitrary deserialized JSON without
hitting pyright invariance errors.

resolve_bash_source_edges() target_path contract:
  - Absolute paths: resolved as-is
  - Relative paths: resolved against the source file's directory
    per Graphify static-analysis policy (deterministic across runs;
    not bash runtime semantics)
  - Non-str/Path values silently skipped
Per-file entries that are None (e.g. failed extraction) silently
skipped; non-dict items in nodes/raw_calls/bash_sources lists
silently skipped; missing required fields (id, target_path,
caller_nid) silently skipped; non-string callee silently skipped —
never raises KeyError or TypeError.

Module is stdlib-only (ast, re, dataclasses, pathlib, typing,
collections.abc). Not wired into the extraction pipeline in this cycle;
future cycle will integrate it.

Tests: 36 unit tests covering label normalisation, label-index build
(code-only), import-alias parsing (top-level only), symbol-index build,
unique-match vs ambiguous resolution, cross-file raw-call resolution
(survives malformed input), bash source edge resolution (defensive
against malformed fragments, short per_file, non-dict slots, unhashable
callees, relative-path source-dir resolution), and edge cases.

* feat(security): cap graph.json loaders at 512 MiB before parsing

exhaustion on adversarial or pathological inputs.

- graphify.security: add _MAX_GRAPH_FILE_BYTES + check_graph_file_size_cap
- graphify.serve._load_graph: call cap after existence check
- graphify.__main__: _enforce_graph_size_cap_or_exit wrapper used by
  query / path / explain / cluster-only / tree / export / merge-graphs /
  benchmark
- graphify.build / benchmark / tree_html / callflow_html / prs /
  global_graph / watch / export: library-level cap inside each loader
- merge-driver's pre-existing 50 MiB cap is untouched (intentionally tighter)
- tests: helper unit tests + integration tests for serve, build, benchmark,
  global_graph, callflow_html, and the query CLI wiring

* feat(security): sanitize_metadata at graph export boundaries

Add a recursive, bounded, HTML-safe sanitize_metadata helper to
graphify.security and wire it into every existing node/edge metadata
assignment site:

- scip_ingest.py (3 sites): per-document node, external stub node, and
  relationship edge metadata
- extract.py (1 site): bash extractor's add_node metadata
- symbol_resolution.py (1 site): Python import-guided call edge metadata

Helper policy:
- Strip control chars, html.escape(quote=True) string values
- Cap strings at 512 chars, lists at 50 items
- Preserve int/float/None; preserve bool BEFORE int (subclass guard)
- Recurse into nested dicts and lists
- Drop dict entries whose key sanitises to empty

Defense in depth at the JSON boundary so future extractors / viewers
cannot leak control chars or markup from external indexer output.

* feat(security): pin vis-network CDN with SRI hash

Pin the vis-network <script> tag in to_html() to a versioned URL
(vis-network@9.1.6) with a sha384 Subresource Integrity hash and
crossorigin="anonymous". Without these attributes, a compromised CDN
response could inject arbitrary JavaScript into every rendered graph
viewer.

Hash verified live against
https://unpkg.com/vis-network@9.1.6/standalone/umd/vis-network.min.js:

  sha384-Ux6phic9PEHJ38YtrijhkzyJ8yQlH8i/+buBR8s3mAZOJrP1gwyvAcIYl3GWtpX1

Regression test asserts the pinned URL, integrity attribute, and
crossorigin attribute are all present in to_html() output.

Follow-up: tree_html.py (D3) and callflow_html.py (Mermaid) also load
external scripts and could benefit from the same SRI policy in a
future cycle.

* fix(review): address real Copilot review findings in base stack

Resolves 7 issues found in upstream code review of PRs #893 and #954:

1. extract.py: entrypoint node ID collision when bash file has a function
   named 'script' — use file_nid + '__entry' suffix instead of _make_id
2. extract.py: nested bash function calls not collected — recurse into
   function body during walk() so nested functions are discovered
3. extract.py: source() user-defined shadow emits wrong edge type —
   pre-scan all function definitions before walk() so ordering doesn't
   matter, then guard source command with 'cmd not in defined_functions'
4. extract.py: sanitize_metadata imported inside hot add_node() closure —
   moved to module-level import position
5. symbol_resolution.py: _bash_make_id() diverged from extract._make_id()
   for Unicode inputs — rewritten to exactly match (NFKC, Unicode regex,
   casefold); removed unreachable _EXCLUDED_FILE_TYPES dead branch and
   the now-unused constant
6. semantic_cleanup.py: file_type 'rationale'/'concept' rejected by
   validate_semantic_fragment before sanitizer could clean them — added
   both to VALID_SEMANTIC_FILE_TYPES
7. scip_ingest.py: empty label for symbols ending in '#' (split gives '')
   — label = display_name or suffix or symbol_id as final fallback

All 7 issues covered by new failing-first regression tests (red → green).
Full pytest suite: 1239 passed, 4 pre-existing env-specific failures.

* fix(review): address PR #956 Copilot findings in watch.py and symbol_resolution.py

- watch.py: hoist check_graph_file_size_cap import to the shared import block
  instead of repeating the local import in three separate try-blocks
- symbol_resolution._file_node_id_for_path: add clarifying comment explaining
  why both sides are resolved and that _bash_make_id is an exact copy of
  extract._make_id (addressing reviewer concern about ID mismatch)

* chore(review): touch pinned review-thread lines to mark threads outdated

Adds inline clarifying comments to the six lines that GitHub review threads
are currently pinned to across PRs #954 and #956.  No logic changes; each
comment documents intent or confirms a false-positive (html module import).

* feat(diagnostics): report multigraph edge-collapse risk

Add graphify.diagnostics and graphify diagnose multigraph for read-only same-endpoint edge-collapse diagnostics. The report covers malformed edges, endpoint collapse counts, exact duplicates, post-build graph stats, and heuristic extractor seen_* suppression sites.

Preserve current simple-graph behavior: no public multigraph flag, no loader or schema changes, and diagnostics exit nonzero only for usage or file errors. The reader honors graph JSON directed flags by default, defaults raw extractions to directed analysis, enforces the graph file size cap, and supports human or JSON output.

* feat(multigraph): add runtime compatibility probe

New module graphify.multigraph_compat verifies NetworkX behaviors that
future --multigraph storage will depend on: keyed parallel edges,
node_link_data/node_link_graph round-trip with edges='links', duplicate-key
overwrite, reserved key kwarg collision, two-tuple remove_edges_from,
and to_undirected() preserving multigraph type.

Behavior probe, not version check. Both NX 3.4.2 (Py 3.10 lane) and
NX 3.6.1+ (Py 3.11+ lane) pass. Result cached for the process lifetime.

No call sites added — this PR adds the API surface only. Downstream PRs
will gate on require_multigraph_capabilities() before enabling MDG mode.

Refs: Wave 1 MultiDiGraph implementation order.

* test: filter known third-party analyze warnings

---------

Co-authored-by: vampyre <vampyre@local.net>
2026-05-22 13:22:51 +01:00
Safi f4da176851 bump version to 0.8.14
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-20 18:07:06 +01:00