lee/graphify
1
0
Fork 0
mirror of https://github.com/safishamsi/graphify.git synced 2026-07-13 02:47:00 +00:00
Code Issues Packages Projects Releases Wiki Activity
51 Commits 12 Branches 161 Tags
v2
Commit Graph

51 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Safi 1184e7d4d5 fix: use correct Python interpreter for pipx installs 2026-04-06 14:07:42 +01:00
Safi d3c7f950d5 bump version to 0.2.2 2026-04-06 14:04:39 +01:00
Safi 5f02b3dfe1 fix graspologic optional, fall back to networkx louvain, add privacy section 2026-04-06 14:02:28 +01:00
Safi 5164d972ed ci: add v2 branch to CI triggers 2026-04-06 01:29:28 +01:00
Safi 6dc07e5f99 add PyPI badge, set homepage to PyPI 2026-04-06 01:27:48 +01:00
Safi ccdd465a08 bump version to 0.2.1 2026-04-06 01:26:53 +01:00
Safi 90fac738e2 README: explain PreToolUse hook behavior in detail 2026-04-06 01:25:32 +01:00
Safi 83c2f4245c fix obsidian always generating, uninstall early return bug, add missing tests 2026-04-06 01:23:34 +01:00
Safi b37f187767 README: fix CI badge to v2, update curl URL, add how it works, graphify claude install section 2026-04-06 01:16:09 +01:00
Safi 6fe4c01bb8 bump version to 0.2.0 2026-04-06 01:06:35 +01:00
Safi a7fc2493f0 v2: confidence scores, hyperedges, rationale extraction, git hooks, Claude Code hooks 
- confidence_score required on every edge (INFERRED: 0.4-0.9, EXTRACTED: 1.0, AMBIGUOUS: 0.1-0.3)
- semantically_similar_to edges for non-obvious cross-file conceptual links
- hyperedges for 3+ node group relationships - fixed cache and merge pipeline that was silently dropping them
- check_semantic_cache returns 4-tuple including cached_hyperedges
- extract.py: mine the "why" - module/class/function docstrings and rationale comments (# NOTE: # IMPORTANT: # HACK: # WHY: # RATIONALE: # TODO: # FIXME:) as rationale_for nodes
- skill.md: rationale_for in relation schema, doc files extract design rationale
- obsidian output opt-in (--obsidian flag) - default output is graph.html + graph.json + GRAPH_REPORT.md only
- hooks.py: post-checkout hook added alongside post-commit - graph rebuilds on branch switch
- claude install: writes .claude/settings.json PreToolUse hook on Glob/Grep - Claude checks graph before searching raw files
- README updated with all v2 features
 2026-04-06 00:58:55 +01:00
Safi 494f519bf4 README: highlight multimodal capability upfront 2026-04-05 23:43:59 +01:00
Safi d23ae1774d v2: hypergraph support - hyperedges in graph.json, shaded regions in HTML, report section 2026-04-05 23:18:25 +01:00
Safi e6fefa8a2c v2: semantic similarity edges, scored higher in surprising connections 2026-04-05 23:12:47 +01:00
Safi 0f11f6a32d v2: confidence scores on INFERRED edges, avg shown in report 2026-04-05 23:06:07 +01:00
Safi e8b8e2039d v2: fast --update for code-only changes, parallel AST+semantic, graphify claude install 2026-04-05 23:00:24 +01:00
Safi e4bc87a284 README: clarify any folder works, clean up usage section 2026-04-05 22:48:23 +01:00
Safi 0a31c0862b add git commit hook - auto-rebuilds graph after every commit v1.0.0 2026-04-05 22:40:34 +01:00
Safi 2a8b6c16ad fix test_watch: update import after _run_update renamed to _notify_only 2026-04-05 20:58:23 +01:00
Safi 014758f61d README: clarify --watch auto-sync behavior for code vs docs 2026-04-05 20:20:34 +01:00
Safi 5617a7d285 watch: auto-rebuild graph on code changes without LLM, notify on doc/image changes 2026-04-05 20:13:26 +01:00
Safi 154919bb64 correct benchmark numbers — token reduction scales with corpus size, small corpora ~1x 2026-04-05 19:44:34 +01:00
Safi 3baf316044 add real graphify output (GRAPH_REPORT.md + graph.json) for httpx and mixed-corpus worked examples 2026-04-05 19:39:11 +01:00
Safi 0fee55726d add reproducible worked examples for httpx, karpathy-repos, and mixed-corpus benchmarks 2026-04-05 19:28:05 +01:00
Safi c488808842 Add reproducible worked example with 7 input files and README 2026-04-05 19:19:00 +01:00
Safi 546fc0b346 README: add --watch to usage, describe --wiki in What you get 2026-04-05 17:15:35 +01:00
Safi 5ae2be8ca7 Update README: add --wiki to usage and output tree 2026-04-05 17:09:02 +01:00
Safi fd2cc8d8a7 Surface all options post-run; wiki-first follow-ups; --update regenerates wiki 2026-04-05 17:07:57 +01:00
Safi ef06d7d8d9 Add --wiki export: agent-crawlable knowledge wiki from graph 2026-04-05 16:54:50 +01:00
Safi 93b8ec3cde Use graph.json for follow-up questions; bump to 0.1.6 2026-04-05 14:46:06 +01:00
Safi e98c1b6297 fix: HTML always generated by default in Step 6, not flag-gated 2026-04-05 14:22:42 +01:00
Safi a4ae70892a release: 0.1.5 2026-04-05 14:18:47 +01:00
Safi a1be5abb24 refactor: dead imports, node_community helper, split to_html and call_tool 
- Remove unused shutil (cache.py), sys (ingest.py), inline import os (detect.py)
- Extract _node_community_map() helper - was copy-pasted 8 times across analyze.py + export.py
- Split to_html() 285-line monolith into _html_styles() + _html_script() + thin orchestrator
- Split serve.py call_tool() if/elif chain into per-tool handler functions + dispatch table
- Extract _find_node() helper shared across get_node and get_neighbors handlers

All 231 tests pass.
 2026-04-05 14:01:55 +01:00
Safi 0061b36c29 fix: 5 skill gaps - graphml usage, manifest timing, graph existence checks, no-viz clarity 
- Add --graphml to Usage table (was implemented but undocumented there)
- Remove early manifest save from --update merge step (Step 9 owns it; saving early meant failed pipelines left manifest ahead of graph)
- query/path/explain now check graph.json exists before running, with clear "run /graphify first" message
- --no-viz: clarify it skips both Obsidian vault and HTML (was contradictory)
 2026-04-05 13:50:07 +01:00
Safi c88528dfa3 perf: larger chunks + code-only fast path + timing estimates 
- Semantic extraction chunks: 12-15 → 20-25 files (fewer subagent round trips)
- Code-only corpora skip semantic dispatch entirely (AST covers it)
- Print estimated time before extraction so the wait feels intentional
 2026-04-05 13:42:11 +01:00
Safi a7f8cab1a9 test: add end-to-end pipeline integration test 
Covers detect → extract → build → cluster → analyze → report → export
using existing fixtures. AST-only (no LLM calls), catches regressions
in how modules connect, not just individual module behaviour.
 2026-04-05 00:42:30 +01:00
Safi fa702bd69f after pipeline, offer to guide user through the graph 2026-04-05 00:35:47 +01:00
Safi 1cd52f191e shorten README - cut design principles, files section, marketing padding 2026-04-05 00:31:42 +01:00
Safi d7a54d7e14 fix critical install bug, add --graphml to pipeline, update changelog 2026-04-05 00:29:42 +01:00
Safi 8708333484 feat: vis.js HTML graph, token reduction benchmark, repo cleanup 
- Replace pyvis with custom vis.js renderer: node size by degree,
  click-to-inspect panel with clickable neighbors, search box,
  community filter, physics clustering by community
- HTML graph generated by default on every run (no --html flag needed)
- Token reduction benchmark auto-runs after every /graphify on corpora >5k words
- Fix 292 edge warnings: silently skip stdlib/external edges in build.py
- Fix build() to merge extractions before building (cross-extraction edges were dropped)
- Add 5 HTML renderer tests (223 total)
- Remove unnecessary files: lib/, tests/eval_attention.py, misplaced eval reports
- Add graphify-out/ and .graphify_*.json to .gitignore
- Bump version to 0.1.4, remove pyvis dependency
- README: token reduction as top-level selling point, vis.js in tech stack,
  graph.html in output listing, correct test count and install command
 2026-04-05 00:20:56 +01:00
Safi a7d1969f02 docs: update surprising connections description, test count 
style: replace all em dashes with hyphens

fix: explain hidden .graphify/ folder in skill output and README

fix: rename .graphify/ to graphify-out/ so output is visible by default
 2026-04-05 00:20:56 +01:00
Safi 258eff3e9f feat: composite surprise score — cross-type, cross-repo, community distance, peripheral→hub 2026-04-05 00:20:56 +01:00
Safi a7f910667d feat: GraphML export (--graphml flag) for Gephi and yEd 2026-04-05 00:20:56 +01:00
Safi cf7f42d3e7 docs: lead with Karpathy problem → graphify answer framing 
fix: add pytest to CI install step

docs: reframe README as Claude Code skill, fix worked/ description

fix: use graphifyy on PyPI until graphify name is reclaimed

fix: pyproject.toml structure, URLs, description; README clarifications

feat: keywords, CHANGELOG, requires note, CI end-to-end install check
 2026-04-05 00:20:56 +01:00
Safi 24bcd6d448 docs: CI, architecture guide, worked examples, README fixes 
- Add GitHub Actions CI workflow (Python 3.10 and 3.12)
- Add CI badge to README
- Add ARCHITECTURE.md: pipeline overview, module table, schema, how to
  add a language extractor, security summary
- Move eval reports from tests/ to worked/httpx/ and worked/mixed-corpus/
- Fix README: test count 163→212, language table (13 languages via
  tree-sitter), extract.py description, worked examples links

benchmark: 8.8x token reduction on nanoGPT + minGPT + micrograd

- Run AST extraction on 29 Python files across 3 Karpathy repos
- 177 nodes, 246 edges, 17 communities (Leiden)
- 8.8x avg token reduction vs naive full-corpus context stuffing
- Notable: micrograd cleanly splits into engine/nn communities;
  nanoGPT model vs training loop correctly separated
- Honest: stdlib import noise flagged, config isolates documented

benchmark: 71.5x token reduction on mixed corpus (code+papers+images)

Full run: nanoGPT+minGPT+micrograd + 5 research papers + 4 images
285 nodes, 340 edges, 53 communities
Average BFS query: 1,726 tokens vs 123,488 naive (71.5x)
Code-only (AST) sub-benchmark: 8.8x on 13k-word corpus
 2026-04-05 00:20:56 +01:00
Safi 81a43f028f feat: 13-language AST support and token benchmark 
Java, C, C++, Ruby, C#, Kotlin, Scala, PHP via tree-sitter (13 total)
benchmark.py measures BFS subgraph tokens vs corpus tokens
5 skill bug fixes (cohesion crash, dead step, missing MCP tool)
 2026-04-04 18:56:38 +01:00
Safi 41e4e3576a security: SSRF protection, HTML escaping, path guards, encoding hardening 
- graphify/security.py (new): centralised security module
    - validate_url(): blocks file://, ftp://, data:, any non-http/https scheme
    - _NoFileRedirectHandler: re-validates redirect targets, blocks file:// redirects
    - safe_fetch(): streams response, 50MB hard cap, non-2xx raises, timeout
    - safe_fetch_text(): safe_fetch + UTF-8 decode with errors=replace
    - validate_graph_path(): resolves path, requires inside .graphify/, base must exist
    - sanitize_label(): strip control chars, cap 256, html.escape() — mirrors
      code-review-graph's _sanitize_name pattern
- graphify/ingest.py: _fetch_html() and _download_binary() now use safe_fetch*;
  ingest() validates URL scheme and wraps network calls in try/except;
  YAML frontmatter: newlines stripped from question before embedding
- graphify/extract.py: all 33 bare .decode() → .decode("utf-8", errors="replace")
  — non-UTF-8 source files degrade gracefully instead of crashing extraction
- graphify/export.py: sanitize_label() on all node labels and edge titles
  before pyvis embeds them in HTML output
- graphify/serve.py: _load_graph() validates graph_path via validate_graph_path()
  and wraps JSONDecodeError with recovery message; sanitize_label() on MCP
  text output
- graphify/detect.py: os.walk(..., followlinks=False) made explicit
- SECURITY.md (new): threat model, mitigations table, reporting process
- tests/test_security.py (new): 20 tests covering all security.py functions
 2026-04-04 18:56:38 +01:00
Safi 0b7460a9e3 perf+fix: parallel extraction, faster imports, bug fixes 
45x faster cluster import, 135x faster detect, parallel subagent extraction,
auto-exclude venvs/caches, suggest_questions fix, manifest fix, install CLI
 2026-04-04 18:56:38 +01:00
Safi e7a03a0539 feat: cache, multi-language extraction, MCP, memory feedback 
call-graph INFERRED edges, multi-language semantic extraction, SHA256 cache,
MCP stdio server with shortest_path, Q&A memory feedback loop
 2026-04-04 18:56:38 +01:00
Safi ce47198be1 feat: Claude Code skill, Obsidian vault, install, tests 
skill.md with full pipeline steps, Obsidian as default output (canvas, tags,
dataview, graph colors), two-command install, 71 tests, .gitignore, deps
 2026-04-04 18:53:43 +01:00