diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 644b79fd..6608230f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -24,7 +24,13 @@ jobs: - name: Install dependencies run: | pip install -e ".[mcp,pdf,watch]" + pip install pytest - name: Run tests run: | - pytest tests/ -q --tb=short + python -m pytest tests/ -q --tb=short + + - name: Verify install works end-to-end + run: | + graphify --help + graphify install diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 00000000..8c472da6 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,32 @@ +# Changelog + +## 0.1.3 (2026-04-04) + +- Fix: `pyproject.toml` structure — `requires-python` and `dependencies` were incorrectly placed under `[project.urls]` +- Add: GitHub repository and issues URLs to PyPI page +- Add: `keywords` for PyPI search discoverability +- Docs: README clarifies Claude Code requirement, temporary PyPI name, worked examples footnote + +## 0.1.1 (2026-04-04) + +- Add: CI badge to README (GitHub Actions, Python 3.10 + 3.12) +- Add: ARCHITECTURE.md — pipeline overview, module table, extraction schema, how to add a language +- Add: SECURITY.md — threat model, mitigations, vulnerability reporting +- Add: `worked/` directory with eval reports (karpathy-repos 71.5x benchmark, httpx, mixed-corpus) +- Fix: pytest not found in CI — added explicit `pip install pytest` step +- Fix: README test count (163 → 212), language table, worked examples links +- Docs: README reframed as Claude Code skill; Karpathy problem → graphify answer framing + +## 0.1.0 (2026-04-03) + +Initial release. + +- 13-language AST extraction via tree-sitter (Python, JS, TS, Go, Rust, Java, C, C++, Ruby, C#, Kotlin, Scala, PHP) +- Leiden community detection via graspologic with oversized community splitting +- SHA256 semantic cache — warm re-runs skip unchanged files +- MCP stdio server — `query_graph`, `get_node`, `get_neighbors`, `shortest_path`, `god_nodes` +- Memory feedback loop — Q&A results saved to `.graphify/memory/`, extracted on `--update` +- Obsidian vault export with wikilinks, community tags, Canvas layout +- Security module — URL validation, safe fetch with size cap, path guards, label sanitisation +- `graphify install` CLI — copies skill to `~/.claude/skills/` and registers in `CLAUDE.md` +- Parallel subagent extraction for docs, papers, and images diff --git a/README.md b/README.md index 865f4b8c..494e646e 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,9 @@ [![CI](https://github.com/safishamsi/graphify/actions/workflows/ci.yml/badge.svg?branch=v1)](https://github.com/safishamsi/graphify/actions/workflows/ci.yml) - any folder of files → persistent knowledge graph → Obsidian vault, graph.json, audit report +**A Claude Code skill.** Type `/graphify` in Claude Code — it reads your files, builds a knowledge graph, and gives you back structure you didn't know was there. + +> Andrej Karpathy keeps a `/raw` folder where he drops papers, tweets, screenshots, and notes. The problem: that folder becomes opaque. You forget what's in it. You can't see what connects. graphify is the answer to that problem. ``` /graphify ./raw @@ -17,30 +19,41 @@ └── memory/ Q&A results filed back in — what you ask grows the graph on next --update ``` -[placeholder: animated GIF showing the full pipeline — detect → extract → cluster → report → Obsidian vault] - ## Why this exists -**The problem:** Andrej Karpathy described it well: he keeps a `/raw` folder where he drops papers, tweets, screenshots, and notes. The problem is that folder becomes opaque. You forget what's in it. You can't see what connects. Ask Claude "what links paper A to the code in repo B?" and it will hallucinate — it hasn't read both, and even if it has, it has no memory of that connection next session. +graphify takes that observation and builds the missing infrastructure: -**What LLMs get wrong:** Naive summarization fills in every gap confidently. You get a summary that sounds complete but you can't tell what was actually in the files vs invented by the model. And next session, it's all gone — no memory of what it extracted. +| His problem | What graphify adds | +|---|---| +| Folder becomes opaque | Community detection surfaces structure automatically | +| Forget what's in it | Persistent `graph.json` — query weeks later without re-reading | +| Can't see connections | Cross-community surprising connections as a first-class output | +| Claude hallucinates missing links | `EXTRACTED` / `INFERRED` / `AMBIGUOUS` — honest about what was found vs guessed | +| Context resets every session | Memory feedback loop — what you ask grows the graph on `--update` | +| Only works on text | PDFs, images, screenshots, tweets, any language via vision | + +**What LLMs get wrong without it:** Naive summarization fills every gap confidently. You get output that sounds complete but you can't tell what was actually in the files vs invented. And next session, it's all gone. **What graphify does differently:** -- **Persistent graph** — relationships are stored in `.graphify/graph.json` and survive across sessions. Query weeks later without re-reading anything. -- **Honest audit trail** — every edge is tagged `EXTRACTED` (explicitly stated), `INFERRED` (call-graph or reasonable deduction), or `AMBIGUOUS` (flagged for review). You always know what was found vs invented. +- **Persistent graph** — relationships stored in `.graphify/graph.json`, survive across sessions. Query weeks later without re-reading anything. +- **Honest audit trail** — every edge tagged `EXTRACTED` (explicitly stated), `INFERRED` (call-graph or reasonable deduction), or `AMBIGUOUS` (flagged for review). You always know what was found vs invented. - **Cross-document surprise** — Leiden community detection finds clusters, then surfaces cross-community connections: the things you would never think to ask about directly. -- **Feedback loop** — every query answer is saved to `.graphify/memory/`. On next `--update`, that Q&A becomes a node. The graph grows from what you ask, not just what you add. +- **Feedback loop** — every query answer saved to `.graphify/memory/`. On next `--update`, that Q&A becomes a node. The graph grows from what you ask, not just what you add. The result: a navigable map of your corpus that is honest about what it knows and what it guessed. ## Install +**Requires:** [Claude Code](https://claude.ai/code) (the CLI or desktop app) and Python 3.10+ + ```bash -pip install graphify && graphify install +pip install graphifyy && graphify install ``` -That's it. This copies the skill file into `~/.claude/skills/graphify/` and registers it in `~/.claude/CLAUDE.md` automatically. The Python package and all dependencies install on first `/graphify` run — you never touch pip manually again. +> **Note:** The PyPI package is temporarily named `graphifyy` while the `graphify` name is being reclaimed. The CLI, skill command, and everything else is still called `graphify` — only `pip install` uses the extra `y`. + +This copies the skill file into `~/.claude/skills/graphify/` and registers it in `~/.claude/CLAUDE.md`. The Python package and all dependencies install automatically on first `/graphify` run — you never touch pip again. Then open Claude Code in any directory and type: @@ -72,7 +85,9 @@ When the user types `/graphify`, invoke the Skill tool with `skill: "graphify"` ## Usage -```bash +All commands are typed inside Claude Code: + +``` /graphify # run on current directory /graphify ./raw # run on a specific folder /graphify ./raw --mode deep # more aggressive INFERRED edge extraction @@ -174,8 +189,10 @@ If corpora in your domain consistently contain structures graphify doesn't extra | Corpus | Type | Reduction | Eval report | |--------|------|-----------|-------------| | Karpathy repos + 5 research papers + 4 images | Mixed (code + papers + images) | **71.5x** | [`worked/karpathy-repos/review.md`](worked/karpathy-repos/review.md) | -| httpx (Python HTTP client) | Codebase | — | [`worked/httpx/review.md`](worked/httpx/review.md) + [`GRAPH_REPORT.md`](worked/httpx/GRAPH_REPORT.md) | -| Mixed corpus (code + paper + Arabic image) | Multi-type | — | [`worked/mixed-corpus/review.md`](worked/mixed-corpus/review.md) | +| httpx (Python HTTP client) | Codebase (6 files) | small corpus¹ | [`worked/httpx/review.md`](worked/httpx/review.md) + [`GRAPH_REPORT.md`](worked/httpx/GRAPH_REPORT.md) | +| Mixed corpus (code + paper + Arabic image) | Multi-type (5 files) | small corpus¹ | [`worked/mixed-corpus/review.md`](worked/mixed-corpus/review.md) | + +¹ Small corpora fit in a single context window — graph value is structural clarity, not token reduction. Reduction ratios grow with corpus size. Each includes the full graph output and an honest evaluation of what the skill got right and wrong. @@ -214,7 +231,7 @@ skills/graphify/ ARCHITECTURE.md module responsibilities, extraction schema, how to add a language SECURITY.md threat model, mitigations, vulnerability reporting -worked/ eval reports from real corpora (httpx, mixed-corpus) +worked/ eval reports from real corpora (karpathy-repos, httpx, mixed-corpus) tests/ 212 tests, one file per module pyproject.toml pip install graphify | pip install graphify[mcp,neo4j,pdf,watch] ``` diff --git a/pyproject.toml b/pyproject.toml index a350a405..d68b1b77 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -3,11 +3,12 @@ requires = ["setuptools>=68"] build-backend = "setuptools.build_meta" [project] -name = "graphify" -version = "0.1.1" -description = "Turn any codebase, docs, or images into a queryable knowledge graph" +name = "graphifyy" +version = "0.1.3" +description = "Claude Code skill — turn any folder of code, docs, papers, images, or tweets into a queryable knowledge graph" readme = "README.md" license = { text = "MIT" } +keywords = ["claude", "claude-code", "knowledge-graph", "rag", "graphrag", "obsidian", "community-detection", "tree-sitter", "leiden", "llm"] requires-python = ">=3.10" dependencies = [ "networkx", @@ -29,6 +30,11 @@ dependencies = [ "tree-sitter-php", ] +[project.urls] +Homepage = "https://github.com/safishamsi/graphify" +Repository = "https://github.com/safishamsi/graphify" +Issues = "https://github.com/safishamsi/graphify/issues" + [project.optional-dependencies] mcp = ["mcp"] neo4j = ["neo4j"]