42 Commits

Author SHA1 Message Date
Safi 91f4d120b6 fix worktree hook crash, prune isolated code nodes, restore rationale-node prohibition
- hooks.py: use git rev-parse --git-path hooks so install/status/uninstall
  work in linked worktrees where .git is a file not a directory (fixes #865)
- build.py: prune degree-0 code nodes after graph assembly to remove
  bundled/synthetic symbols with no connections (fixes #728)
- skill.md: restore explicit rule prohibiting file_type:"rationale" nodes
  and remind model to store rationale as an attribute instead (fixes #751)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-14 23:03:21 +01:00
Safi a01e0981ed fix #374 #401 #410 #413 #385: vscode install cmd, core.hooksPath, absolute cache root, Windows PATH docs, Kiro YAML colon 2026-04-17 08:33:02 +01:00
Safi 494f519bf4 README: highlight multimodal capability upfront 2026-04-05 23:43:59 +01:00
Safi d23ae1774d v2: hypergraph support - hyperedges in graph.json, shaded regions in HTML, report section 2026-04-05 23:18:25 +01:00
Safi e6fefa8a2c v2: semantic similarity edges, scored higher in surprising connections 2026-04-05 23:12:47 +01:00
Safi 0f11f6a32d v2: confidence scores on INFERRED edges, avg shown in report 2026-04-05 23:06:07 +01:00
Safi e8b8e2039d v2: fast --update for code-only changes, parallel AST+semantic, graphify claude install 2026-04-05 23:00:24 +01:00
Safi e4bc87a284 README: clarify any folder works, clean up usage section 2026-04-05 22:48:23 +01:00
Safi 0a31c0862b add git commit hook - auto-rebuilds graph after every commit v1.0.0 2026-04-05 22:40:34 +01:00
Safi 2a8b6c16ad fix test_watch: update import after _run_update renamed to _notify_only 2026-04-05 20:58:23 +01:00
Safi 014758f61d README: clarify --watch auto-sync behavior for code vs docs 2026-04-05 20:20:34 +01:00
Safi 5617a7d285 watch: auto-rebuild graph on code changes without LLM, notify on doc/image changes 2026-04-05 20:13:26 +01:00
Safi 154919bb64 correct benchmark numbers — token reduction scales with corpus size, small corpora ~1x 2026-04-05 19:44:34 +01:00
Safi 3baf316044 add real graphify output (GRAPH_REPORT.md + graph.json) for httpx and mixed-corpus worked examples 2026-04-05 19:39:11 +01:00
Safi 0fee55726d add reproducible worked examples for httpx, karpathy-repos, and mixed-corpus benchmarks 2026-04-05 19:28:05 +01:00
Safi c488808842 Add reproducible worked example with 7 input files and README 2026-04-05 19:19:00 +01:00
Safi 546fc0b346 README: add --watch to usage, describe --wiki in What you get 2026-04-05 17:15:35 +01:00
Safi 5ae2be8ca7 Update README: add --wiki to usage and output tree 2026-04-05 17:09:02 +01:00
Safi fd2cc8d8a7 Surface all options post-run; wiki-first follow-ups; --update regenerates wiki 2026-04-05 17:07:57 +01:00
Safi ef06d7d8d9 Add --wiki export: agent-crawlable knowledge wiki from graph 2026-04-05 16:54:50 +01:00
Safi 93b8ec3cde Use graph.json for follow-up questions; bump to 0.1.6 2026-04-05 14:46:06 +01:00
Safi e98c1b6297 fix: HTML always generated by default in Step 6, not flag-gated 2026-04-05 14:22:42 +01:00
Safi a4ae70892a release: 0.1.5 2026-04-05 14:18:47 +01:00
Safi a1be5abb24 refactor: dead imports, node_community helper, split to_html and call_tool
- Remove unused shutil (cache.py), sys (ingest.py), inline import os (detect.py)
- Extract _node_community_map() helper - was copy-pasted 8 times across analyze.py + export.py
- Split to_html() 285-line monolith into _html_styles() + _html_script() + thin orchestrator
- Split serve.py call_tool() if/elif chain into per-tool handler functions + dispatch table
- Extract _find_node() helper shared across get_node and get_neighbors handlers

All 231 tests pass.
2026-04-05 14:01:55 +01:00
Safi 0061b36c29 fix: 5 skill gaps - graphml usage, manifest timing, graph existence checks, no-viz clarity
- Add --graphml to Usage table (was implemented but undocumented there)
- Remove early manifest save from --update merge step (Step 9 owns it; saving early meant failed pipelines left manifest ahead of graph)
- query/path/explain now check graph.json exists before running, with clear "run /graphify first" message
- --no-viz: clarify it skips both Obsidian vault and HTML (was contradictory)
2026-04-05 13:50:07 +01:00
Safi c88528dfa3 perf: larger chunks + code-only fast path + timing estimates
- Semantic extraction chunks: 12-15 → 20-25 files (fewer subagent round trips)
- Code-only corpora skip semantic dispatch entirely (AST covers it)
- Print estimated time before extraction so the wait feels intentional
2026-04-05 13:42:11 +01:00
Safi a7f8cab1a9 test: add end-to-end pipeline integration test
Covers detect → extract → build → cluster → analyze → report → export
using existing fixtures. AST-only (no LLM calls), catches regressions
in how modules connect, not just individual module behaviour.
2026-04-05 00:42:30 +01:00
Safi fa702bd69f after pipeline, offer to guide user through the graph 2026-04-05 00:35:47 +01:00
Safi 1cd52f191e shorten README - cut design principles, files section, marketing padding 2026-04-05 00:31:42 +01:00
Safi d7a54d7e14 fix critical install bug, add --graphml to pipeline, update changelog 2026-04-05 00:29:42 +01:00
Safi 8708333484 feat: vis.js HTML graph, token reduction benchmark, repo cleanup
- Replace pyvis with custom vis.js renderer: node size by degree,
  click-to-inspect panel with clickable neighbors, search box,
  community filter, physics clustering by community
- HTML graph generated by default on every run (no --html flag needed)
- Token reduction benchmark auto-runs after every /graphify on corpora >5k words
- Fix 292 edge warnings: silently skip stdlib/external edges in build.py
- Fix build() to merge extractions before building (cross-extraction edges were dropped)
- Add 5 HTML renderer tests (223 total)
- Remove unnecessary files: lib/, tests/eval_attention.py, misplaced eval reports
- Add graphify-out/ and .graphify_*.json to .gitignore
- Bump version to 0.1.4, remove pyvis dependency
- README: token reduction as top-level selling point, vis.js in tech stack,
  graph.html in output listing, correct test count and install command
2026-04-05 00:20:56 +01:00
Safi a7d1969f02 docs: update surprising connections description, test count
style: replace all em dashes with hyphens

fix: explain hidden .graphify/ folder in skill output and README

fix: rename .graphify/ to graphify-out/ so output is visible by default
2026-04-05 00:20:56 +01:00
Safi 258eff3e9f feat: composite surprise score — cross-type, cross-repo, community distance, peripheral→hub 2026-04-05 00:20:56 +01:00
Safi a7f910667d feat: GraphML export (--graphml flag) for Gephi and yEd 2026-04-05 00:20:56 +01:00
Safi cf7f42d3e7 docs: lead with Karpathy problem → graphify answer framing
fix: add pytest to CI install step

docs: reframe README as Claude Code skill, fix worked/ description

fix: use graphifyy on PyPI until graphify name is reclaimed

fix: pyproject.toml structure, URLs, description; README clarifications

feat: keywords, CHANGELOG, requires note, CI end-to-end install check
2026-04-05 00:20:56 +01:00
Safi 24bcd6d448 docs: CI, architecture guide, worked examples, README fixes
- Add GitHub Actions CI workflow (Python 3.10 and 3.12)
- Add CI badge to README
- Add ARCHITECTURE.md: pipeline overview, module table, schema, how to
  add a language extractor, security summary
- Move eval reports from tests/ to worked/httpx/ and worked/mixed-corpus/
- Fix README: test count 163→212, language table (13 languages via
  tree-sitter), extract.py description, worked examples links

benchmark: 8.8x token reduction on nanoGPT + minGPT + micrograd

- Run AST extraction on 29 Python files across 3 Karpathy repos
- 177 nodes, 246 edges, 17 communities (Leiden)
- 8.8x avg token reduction vs naive full-corpus context stuffing
- Notable: micrograd cleanly splits into engine/nn communities;
  nanoGPT model vs training loop correctly separated
- Honest: stdlib import noise flagged, config isolates documented

benchmark: 71.5x token reduction on mixed corpus (code+papers+images)

Full run: nanoGPT+minGPT+micrograd + 5 research papers + 4 images
285 nodes, 340 edges, 53 communities
Average BFS query: 1,726 tokens vs 123,488 naive (71.5x)
Code-only (AST) sub-benchmark: 8.8x on 13k-word corpus
2026-04-05 00:20:56 +01:00
Safi 81a43f028f feat: 13-language AST support and token benchmark
Java, C, C++, Ruby, C#, Kotlin, Scala, PHP via tree-sitter (13 total)
benchmark.py measures BFS subgraph tokens vs corpus tokens
5 skill bug fixes (cohesion crash, dead step, missing MCP tool)
2026-04-04 18:56:38 +01:00
Safi 41e4e3576a security: SSRF protection, HTML escaping, path guards, encoding hardening
- graphify/security.py (new): centralised security module
    - validate_url(): blocks file://, ftp://, data:, any non-http/https scheme
    - _NoFileRedirectHandler: re-validates redirect targets, blocks file:// redirects
    - safe_fetch(): streams response, 50MB hard cap, non-2xx raises, timeout
    - safe_fetch_text(): safe_fetch + UTF-8 decode with errors=replace
    - validate_graph_path(): resolves path, requires inside .graphify/, base must exist
    - sanitize_label(): strip control chars, cap 256, html.escape() — mirrors
      code-review-graph's _sanitize_name pattern
- graphify/ingest.py: _fetch_html() and _download_binary() now use safe_fetch*;
  ingest() validates URL scheme and wraps network calls in try/except;
  YAML frontmatter: newlines stripped from question before embedding
- graphify/extract.py: all 33 bare .decode() → .decode("utf-8", errors="replace")
  — non-UTF-8 source files degrade gracefully instead of crashing extraction
- graphify/export.py: sanitize_label() on all node labels and edge titles
  before pyvis embeds them in HTML output
- graphify/serve.py: _load_graph() validates graph_path via validate_graph_path()
  and wraps JSONDecodeError with recovery message; sanitize_label() on MCP
  text output
- graphify/detect.py: os.walk(..., followlinks=False) made explicit
- SECURITY.md (new): threat model, mitigations table, reporting process
- tests/test_security.py (new): 20 tests covering all security.py functions
2026-04-04 18:56:38 +01:00
Safi 0b7460a9e3 perf+fix: parallel extraction, faster imports, bug fixes
45x faster cluster import, 135x faster detect, parallel subagent extraction,
auto-exclude venvs/caches, suggest_questions fix, manifest fix, install CLI
2026-04-04 18:56:38 +01:00
Safi e7a03a0539 feat: cache, multi-language extraction, MCP, memory feedback
call-graph INFERRED edges, multi-language semantic extraction, SHA256 cache,
MCP stdio server with shortest_path, Q&A memory feedback loop
2026-04-04 18:56:38 +01:00
Safi ce47198be1 feat: Claude Code skill, Obsidian vault, install, tests
skill.md with full pipeline steps, Obsidian as default output (canvas, tags,
dataview, graph colors), two-command install, 71 tests, .gitignore, deps
2026-04-04 18:53:43 +01:00
Safi 77966a7622 feat: core pipeline
detect, extract (AST + call-graph), build, cluster, analyze, report, export
PDF extraction, tree-sitter AST, pyvis HTML, schema validation
2026-04-04 18:52:44 +01:00