lee/graphify
1
0
Fork 0
mirror of https://github.com/safishamsi/graphify.git synced 2026-09-27 07:55:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
40 Commits 12 Branches 217 Tags
1a54f16b8028892300bfdaa63fcfe4c4a03af05d
Commit Graph
40 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Safi 1a54f16b80 feat: always-on hooks and README updates for all platforms 2026-04-06 16:06:31 +01:00
Safi aed4a981a1 v3: MIT license, multi-platform packaging and CI 2026-04-06 16:06:31 +01:00
Safi 3694822040 document node deduplication strategy in build.py 2026-04-06 16:06:31 +01:00
Safi 5461feb7e8 refactor extract.py: LanguageConfig + _extract_generic replaces 10 copy-pasted extractors (2527 → 1588 lines) 2026-04-06 16:06:31 +01:00
Safi 6570488658 fix build() dropping hyperedges when merging multiple extractions 2026-04-06 16:06:31 +01:00
Safi 38d967fc63 feat: multi-platform skill files and install routing (Codex, OpenCode, OpenClaw) 2026-04-06 16:06:31 +01:00
Safi 07de9d5c22 fix: use correct Python interpreter for pipx installs 2026-04-06 16:06:31 +01:00
Safi c1bf6c90dc fix graspologic optional, fall back to networkx louvain, add privacy section 2026-04-06 16:06:31 +01:00
Safi 4ba8198c7c v2.1: PyPI release and CI for v2 branch 2026-04-06 16:06:31 +01:00
Safi c46a4ebb68 README: fix CI badge to v2, update curl URL, add how it works, graphify claude install section 2026-04-06 16:06:31 +01:00
Safi 0ee77c0b6a bump version to 0.2.0 2026-04-06 16:06:31 +01:00
Safi 317d73f8f7 v2: confidence scores, hyperedges, rationale extraction, git hooks, Claude Code hooks 
- confidence_score required on every edge (INFERRED: 0.4-0.9, EXTRACTED: 1.0, AMBIGUOUS: 0.1-0.3)
- semantically_similar_to edges for non-obvious cross-file conceptual links
- hyperedges for 3+ node group relationships - fixed cache and merge pipeline that was silently dropping them
- check_semantic_cache returns 4-tuple including cached_hyperedges
- extract.py: mine the "why" - module/class/function docstrings and rationale comments (# NOTE: # IMPORTANT: # HACK: # WHY: # RATIONALE: # TODO: # FIXME:) as rationale_for nodes
- skill.md: rationale_for in relation schema, doc files extract design rationale
- obsidian output opt-in (--obsidian flag) - default output is graph.html + graph.json + GRAPH_REPORT.md only
- hooks.py: post-checkout hook added alongside post-commit - graph rebuilds on branch switch
- claude install: writes .claude/settings.json PreToolUse hook on Glob/Grep - Claude checks graph before searching raw files
- README updated with all v2 features
 2026-04-06 16:06:31 +01:00
Safi 3f8169c807 README: highlight multimodal capability upfront 2026-04-06 16:06:31 +01:00
Safi 3a117c93e8 v2: hypergraph support - hyperedges in graph.json, shaded regions in HTML, report section 2026-04-06 16:06:31 +01:00
Safi 6fa4c7e662 v2: semantic similarity edges, scored higher in surprising connections 2026-04-06 16:06:31 +01:00
Safi dafe6c9f03 v2: confidence scores on INFERRED edges, avg shown in report 2026-04-06 16:06:31 +01:00
Safi 7e3da961a9 v2: fast --update for code-only changes, parallel AST+semantic, graphify claude install 2026-04-06 16:06:31 +01:00
Safi 63d80200c5 README: clarify any folder works, clean up usage section 2026-04-06 16:06:31 +01:00
Safi 66f1f40de8 add git commit hook - auto-rebuilds graph after every commit 2026-04-06 16:06:31 +01:00
Safi e2fd4f944e watch: auto-rebuild graph on code changes without LLM, notify on doc/image changes 2026-04-06 16:06:31 +01:00
Safi 21e443e201 Add reproducible worked example with 7 input files and README 2026-04-06 16:06:31 +01:00
Safi d213c03adf Add --wiki export: agent-crawlable knowledge wiki from graph 2026-04-06 16:06:31 +01:00
Safi 92ab83ea38 Use graph.json for follow-up questions; bump to 0.1.6 2026-04-06 16:06:31 +01:00
Safi cb4ec5d39d fix: HTML always generated by default in Step 6, not flag-gated 2026-04-06 16:06:31 +01:00
Safi 010583674e release: 0.1.5 2026-04-06 16:06:31 +01:00
Safi d8b1e82079 fix: 5 skill gaps - graphml usage, manifest timing, graph existence checks, no-viz clarity 
- Add --graphml to Usage table (was implemented but undocumented there)
- Remove early manifest save from --update merge step (Step 9 owns it; saving early meant failed pipelines left manifest ahead of graph)
- query/path/explain now check graph.json exists before running, with clear "run /graphify first" message
- --no-viz: clarify it skips both Obsidian vault and HTML (was contradictory)
 2026-04-06 16:06:31 +01:00
Safi ef8c2fef50 perf: larger chunks + code-only fast path + timing estimates 
- Semantic extraction chunks: 12-15 → 20-25 files (fewer subagent round trips)
- Code-only corpora skip semantic dispatch entirely (AST covers it)
- Print estimated time before extraction so the wait feels intentional
 2026-04-06 16:06:31 +01:00
Safi 0f4de8e47f test: add end-to-end pipeline integration test 
Covers detect → extract → build → cluster → analyze → report → export
using existing fixtures. AST-only (no LLM calls), catches regressions
in how modules connect, not just individual module behaviour.
 2026-04-06 16:06:31 +01:00
Safi 5efcf114ee fix critical install bug, add --graphml to pipeline, update changelog 2026-04-06 16:06:31 +01:00
Safi d4b24d8609 feat: vis.js HTML graph, token reduction benchmark, repo cleanup 
- Replace pyvis with custom vis.js renderer: node size by degree,
  click-to-inspect panel with clickable neighbors, search box,
  community filter, physics clustering by community
- HTML graph generated by default on every run (no --html flag needed)
- Token reduction benchmark auto-runs after every /graphify on corpora >5k words
- Fix 292 edge warnings: silently skip stdlib/external edges in build.py
- Fix build() to merge extractions before building (cross-extraction edges were dropped)
- Add 5 HTML renderer tests (223 total)
- Remove unnecessary files: lib/, tests/eval_attention.py, misplaced eval reports
- Add graphify-out/ and .graphify_*.json to .gitignore
- Bump version to 0.1.4, remove pyvis dependency
- README: token reduction as top-level selling point, vis.js in tech stack,
  graph.html in output listing, correct test count and install command
 2026-04-06 16:06:31 +01:00
Safi 5db8f7ce39 docs: update surprising connections description, test count 
style: replace all em dashes with hyphens

fix: explain hidden .graphify/ folder in skill output and README

fix: rename .graphify/ to graphify-out/ so output is visible by default
 2026-04-06 16:06:31 +01:00
Safi 693d2ba991 feat: composite surprise score — cross-type, cross-repo, community distance, peripheral→hub 2026-04-06 16:06:31 +01:00
Safi 7e82212304 feat: GraphML export (--graphml flag) for Gephi and yEd 2026-04-06 16:06:31 +01:00
Safi 64e07abd98 docs: CI, architecture guide, worked examples, README fixes 
- Add GitHub Actions CI workflow (Python 3.10 and 3.12)
- Add CI badge to README
- Add ARCHITECTURE.md: pipeline overview, module table, schema, how to
  add a language extractor, security summary
- Move eval reports from tests/ to worked/httpx/ and worked/mixed-corpus/
- Fix README: test count 163→212, language table (13 languages via
  tree-sitter), extract.py description, worked examples links

benchmark: 8.8x token reduction on nanoGPT + minGPT + micrograd

- Run AST extraction on 29 Python files across 3 Karpathy repos
- 177 nodes, 246 edges, 17 communities (Leiden)
- 8.8x avg token reduction vs naive full-corpus context stuffing
- Notable: micrograd cleanly splits into engine/nn communities;
  nanoGPT model vs training loop correctly separated
- Honest: stdlib import noise flagged, config isolates documented

benchmark: 71.5x token reduction on mixed corpus (code+papers+images)

Full run: nanoGPT+minGPT+micrograd + 5 research papers + 4 images
285 nodes, 340 edges, 53 communities
Average BFS query: 1,726 tokens vs 123,488 naive (71.5x)
Code-only (AST) sub-benchmark: 8.8x on 13k-word corpus
 2026-04-06 16:06:31 +01:00
Safi 81a43f028f feat: 13-language AST support and token benchmark 
Java, C, C++, Ruby, C#, Kotlin, Scala, PHP via tree-sitter (13 total)
benchmark.py measures BFS subgraph tokens vs corpus tokens
5 skill bug fixes (cohesion crash, dead step, missing MCP tool)
 2026-04-04 18:56:38 +01:00
Safi 41e4e3576a security: SSRF protection, HTML escaping, path guards, encoding hardening 
- graphify/security.py (new): centralised security module
    - validate_url(): blocks file://, ftp://, data:, any non-http/https scheme
    - _NoFileRedirectHandler: re-validates redirect targets, blocks file:// redirects
    - safe_fetch(): streams response, 50MB hard cap, non-2xx raises, timeout
    - safe_fetch_text(): safe_fetch + UTF-8 decode with errors=replace
    - validate_graph_path(): resolves path, requires inside .graphify/, base must exist
    - sanitize_label(): strip control chars, cap 256, html.escape() — mirrors
      code-review-graph's _sanitize_name pattern
- graphify/ingest.py: _fetch_html() and _download_binary() now use safe_fetch*;
  ingest() validates URL scheme and wraps network calls in try/except;
  YAML frontmatter: newlines stripped from question before embedding
- graphify/extract.py: all 33 bare .decode() → .decode("utf-8", errors="replace")
  — non-UTF-8 source files degrade gracefully instead of crashing extraction
- graphify/export.py: sanitize_label() on all node labels and edge titles
  before pyvis embeds them in HTML output
- graphify/serve.py: _load_graph() validates graph_path via validate_graph_path()
  and wraps JSONDecodeError with recovery message; sanitize_label() on MCP
  text output
- graphify/detect.py: os.walk(..., followlinks=False) made explicit
- SECURITY.md (new): threat model, mitigations table, reporting process
- tests/test_security.py (new): 20 tests covering all security.py functions
 2026-04-04 18:56:38 +01:00
Safi 0b7460a9e3 perf+fix: parallel extraction, faster imports, bug fixes 
45x faster cluster import, 135x faster detect, parallel subagent extraction,
auto-exclude venvs/caches, suggest_questions fix, manifest fix, install CLI
 2026-04-04 18:56:38 +01:00
Safi e7a03a0539 feat: cache, multi-language extraction, MCP, memory feedback 
call-graph INFERRED edges, multi-language semantic extraction, SHA256 cache,
MCP stdio server with shortest_path, Q&A memory feedback loop
 2026-04-04 18:56:38 +01:00
Safi ce47198be1 feat: Claude Code skill, Obsidian vault, install, tests 
skill.md with full pipeline steps, Obsidian as default output (canvas, tags,
dataview, graph colors), two-command install, 71 tests, .gitignore, deps
 2026-04-04 18:53:43 +01:00
Safi 77966a7622 feat: core pipeline 
detect, extract (AST + call-graph), build, cluster, analyze, report, export
PDF extraction, tree-sitter AST, pyvis HTML, schema validation
 2026-04-04 18:52:44 +01:00