5 Commits

Author SHA1 Message Date
Christopher Beaulieu 3d99603e8d refactor(eval): restructure as worked/ case study for upstream alignment
Restructures the rsl-siege-manager evaluation to match upstream's
worked/karpathy-repos/ convention before opening against safishamsi/graphify:

- Move evaluations/rsl-siege-manager/ -> worked/rsl-siege-manager/
- Flatten graphify-out-with-tests/ artifacts to the case directory root
  (matches karpathy-repos layout: README.md, review.md, GRAPH_REPORT.md,
  graph.html, graph.json, manifest.json at top level).
- Drop the codebase-only artifact set; keep only the tests-included run as
  the more realistic default (most users won't pre-configure .graphifyignore).
- Rename runbook.md -> README.md (corpus description + reproduction steps).
- Rename results.md -> review.md (matches karpathy-repos), and reframe from
  "do not adopt" verdict to neutral findings + suggested follow-ups.
- Drop evaluations/README.md fork-only index.
- Revert .gitignore evaluations/ exception block — no longer needed now that
  artifacts live at worked/rsl-siege-manager/ (no graphify-out/ in path).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-15 13:22:04 -04:00
Safi 934957cbfd update worked example READMEs for v3 multi-platform 2026-04-06 16:20:13 +01:00
Safi 21e443e201 Add reproducible worked example with 7 input files and README 2026-04-06 16:06:31 +01:00
Safi 5db8f7ce39 docs: update surprising connections description, test count
style: replace all em dashes with hyphens

fix: explain hidden .graphify/ folder in skill output and README

fix: rename .graphify/ to graphify-out/ so output is visible by default
2026-04-06 16:06:31 +01:00
Safi 64e07abd98 docs: CI, architecture guide, worked examples, README fixes
- Add GitHub Actions CI workflow (Python 3.10 and 3.12)
- Add CI badge to README
- Add ARCHITECTURE.md: pipeline overview, module table, schema, how to
  add a language extractor, security summary
- Move eval reports from tests/ to worked/httpx/ and worked/mixed-corpus/
- Fix README: test count 163→212, language table (13 languages via
  tree-sitter), extract.py description, worked examples links

benchmark: 8.8x token reduction on nanoGPT + minGPT + micrograd

- Run AST extraction on 29 Python files across 3 Karpathy repos
- 177 nodes, 246 edges, 17 communities (Leiden)
- 8.8x avg token reduction vs naive full-corpus context stuffing
- Notable: micrograd cleanly splits into engine/nn communities;
  nanoGPT model vs training loop correctly separated
- Honest: stdlib import noise flagged, config isolates documented

benchmark: 71.5x token reduction on mixed corpus (code+papers+images)

Full run: nanoGPT+minGPT+micrograd + 5 research papers + 4 images
285 nodes, 340 edges, 53 communities
Average BFS query: 1,726 tokens vs 123,488 naive (71.5x)
Code-only (AST) sub-benchmark: 8.8x on 13k-word corpus
2026-04-06 16:06:31 +01:00