Commit Graph
238 Commits
Author SHA1 Message Date
cod3warrior fd0ebb5957 wiki: sanitize Windows-reserved characters in filenames (#594)
On Windows, to_wiki crashes with OSError [Errno 22] when a node label
contains any of < > : " / \ | ? * (Windows reserved chars in
filenames). Repro: an Obsidian note titled "Czy Wii jeszcze ok?.md"
becomes a community label, then write_text fails because Windows
rejects the question mark in the path.

The previous _safe_filename only handled three characters (/ space :)
which is enough on Linux/macOS but not on Windows. Extended the
substitution to cover all Windows-reserved chars plus trailing dots
and spaces (also reserved on Windows), and added a 200-char length
cap to stay well under MAX_PATH-segment limits.

Falls back to 'unnamed' if sanitization produces an empty string,
so we never try to create a file named "" or ".".

Tested on a 1773-file vault with mixed Polish/English content where
~12 community labels and several god node labels contained reserved
characters. Generates 763 wiki articles cleanly.
2026-05-02 14:15:09 +01:00
opnsrcntrbtr b9871d7019 Fix #563: prevent rationale-node leakage and preserve calls direction (#576)
* Fix #563: prevent rationale-node leakage and preserve calls direction

Two AST-extractor bugs were inflating god-node centrality:

1. Rationale leakage in cross-file resolvers
   _resolve_cross_file_imports indexed any node whose label didn't end in
   ')' or '.py' as an importable entity, so rationale nodes (whose labels
   are docstring text) ended up in both stem_to_entities and local_classes.
   Result: phantom '<file>_rationale_N --uses--> ImportedThing' edges for
   every imported entity in every file with docstrings.

   The same leak existed in cross-file call resolution via
   global_label_to_nid, where rationale labels could collide with callee
   names.

   Fix: skip nodes with file_type == 'rationale' in all three index loops.

2. Calls / rationale_for direction lost at export
   to_json() called nx.json_graph.node_link_data() on a NetworkX undirected
   Graph, which canonicalizes endpoint order. The build path already
   stashes _src and _tgt on every edge for exactly this case (with a
   comment to that effect at build.py:100), but the exporter never
   consulted them, so 'calls' and 'rationale_for' edges were emitted with
   arbitrary direction.

   Fix: in to_json(), restore link['source'] / link['target'] from
   _src / _tgt, then pop those internal fields before writing graph.json.

Repro on a minimal 3-file project (queue.py + contact_form.py + a test
file with multi-paragraph docstrings) showed:

  Before fix: 31 edges, 12 incident on SQLiteQueue, 7 phantom uses-edges
              from rationale nodes, 1 inverted calls edge.
  After fix:  24 edges, 5 incident on SQLiteQueue, 0 phantom edges,
              all calls edges go caller -> callee.

Adds tests/test_issue_563.py with 5 regression tests covering both bugs
plus a direction-pin on the existing sample_calls.py fixture. Full suite
passes (446/446).

* Address Copilot review on #576

- tests: identify rationale nodes by file_type metadata, not id substring
  (decouples regression from current `<stem>_rationale_<line>` naming)
- tests: assert contact_form -> SQLiteQueue.enqueue calls edge directly
  (previously only inversions were guarded; a dropped enqueue edge would
  have passed silently)
- extract.py: clarify cross-file import indexing comment — the filter
  intentionally indexes only class-level entities (function/method labels
  end in "()" and are excluded by design)

* Fix #563 (HTML export): restore arrow direction in graph.html

to_json was patched to consult _src/_tgt before serializing edge
endpoints, but to_html still read endpoints directly from G.edges(),
so calls and rationale_for arrows in the rendered graph.html still
pointed in canonicalized (often inverted) order.

Apply the same direction restore in to_html when building vis.js
edges. Use data.get (not pop) since G.edges(data=True) yields the
live attribute dict and other exporters may run after to_html.

Adds test_to_html_preserves_calls_and_rationale_for_direction:
parses RAW_EDGES out of the rendered HTML and pins both the
caller->callee assertions from #563 and the rationale->parent
direction. Without the fix, the test reports 11 flipped arrows
on the 3-file repro.
2026-05-02 14:15:06 +01:00
teamclaw ac72cd3aaa fix(wiki): clear stale articles before regenerating to prevent orphan accumulation (#558)
to_wiki() writes a fresh set of community + god-node articles each call but
never deletes old files from previous runs. Since community labels are
LLM-generated and non-deterministic across rebuilds (per skill.md Step 5),
the same conceptual community is often named differently each time, leaving
its previous file as an orphan. After N rebuilds, wiki/ contains roughly N
times the active article count, with index.md only referencing the most
recent run's labels.

Real-world: a knowledge corpus accumulated 822 wiki .md files over 5
rebuilds, of which only 111 were referenced by index.md (710 orphans).

Fix: clear *.md files in the output directory at the start of to_wiki().
This is consistent with its existing fully-regenerative behavior — it
always writes the full set of articles + index, never partial updates.
Subdirectories and non-.md files are preserved (only top-level .md is
touched), so any user-placed auxiliary assets survive.

Tests: two new regression tests cover (1) stale article cleanup across
runs with different labels, and (2) preservation of non-.md user files
and nested subdirectories.
2026-05-02 14:15:03 +01:00
saxster 48c9f024fb docs(skill): forced-rank confidence scores for INFERRED edges (#546)
Closes #540.

Production audit on a 10,129-edge graph showed the INFERRED
confidence_score distribution is bimodal, not graded:

  | Score bucket | Count | % of INFERRED |
  |--------------|-------|---------------|
  | <0.4         |     0 | 0%            |
  | 0.4-0.6      | 5,807 | 57%           |
  | 0.6-0.8      |    14 | 0.1%          |
  | 0.8+         | 4,308 | 42%           |

Subagents collapse the continuous "0.4-0.9" guidance to a binary:
0.5 for "uncertain", 0.85+ for "confident", almost nothing in between.
Downstream filtering by confidence is therefore an on/off switch, not
the gradient the prompt promises.

Replace continuous ranges with a forced-rank discrete set:

  0.95  direct structural evidence
  0.85  strong inference
  0.75  reasonable inference
  0.65  weak inference
  0.55  speculative but plausible

Models follow discrete rubrics far better than continuous ranges
(documented in calibration literature; same reason MCQ rubrics
outperform 0-100 scales). The set is anchored at non-round midpoints
to discourage 0.5 as a default.

Applied uniformly across all 10 skill-*.md files:
- 7 long-form (skill.md, skill-codex.md, skill-copilot.md,
  skill-droid.md, skill-opencode.md, skill-windows.md, skill-trae.md):
  full forced-rank table.
- 3 short-form (skill-claw.md, skill-aider.md, skill-kiro.md):
  inline set notation INFERRED ∈ {0.55, 0.65, 0.75, 0.85, 0.95}.

Pure prompt edit — no code changes, no test impact. Effect is
observable only via re-extraction and inspection of the new
confidence_score distribution.
2026-05-02 14:14:55 +01:00
saxster e1636a0e88 docs(skill): save_manifest at end of --update merge step (#545)
Closes #538.

The full-pipeline path's Step 9 already calls save_manifest, so the
NEXT --update can diff against the full-rebuild's baseline.

But the --update flow itself does NOT save the manifest after merging
the incremental result. Consequence: a file deleted between
--update #1 and --update #2 keeps reappearing in #2's deleted_files
list (since the manifest still records its old mtime), generating a
spurious ghost-node prune attempt every run.

Add save_manifest(incremental['files']) at the end of the --update
merge step in both skill.md and skill-copilot.md (the only two skill
files carrying the merge step verbatim). The 'incremental' dict is
already in scope from the prune block above; its 'files' key is the
full current file list, which is exactly what save_manifest needs.

Pure prompt edit — no code changes, no test impact.
2026-05-02 14:14:52 +01:00
saxster 8b41440ad1 docs(skill): clarify --update prune message — split drift vs no-drift (#544)
Closes #539.

The current prune message in the --update flow is ambiguous:

    Pruned 0 ghost nodes from 13 deleted file(s)

A reader can't tell whether (a) 13 files deleted, 0 of them had nodes
worth pruning (benign — graph already clean), or (b) 13 ghost nodes
still exist and only 0 got pruned (bug). The denominator is opaque.

Split into two messages so the semantics are explicit:

- Drift case: "Pruned N ghost node(s) from M deleted file(s) — drift
  detected and corrected."
- No-drift case: "M file(s) deleted since last run, but no ghost
  nodes were present in the graph — no drift."

Applied to both skill.md and skill-copilot.md (the only two skill
files carrying the --update merge step verbatim).
2026-05-02 14:14:49 +01:00
Koushik-Salammagari e1947826fc fix(hooks): expand ~ in core.hooksPath before resolving install target (#554)
When a user configures core.hooksPath = ~/gitconfig/hooks in .gitconfig,
_hooks_dir() was constructing Path("~/gitconfig/hooks") without calling
expanduser(), so hooks were installed into <repo>/~/gitconfig/hooks instead
of the intended absolute path.

Add Path.expanduser() call immediately after reading the raw string from
git config, before the is_absolute() / root-relative fallback logic.

Fixes #547
2026-05-02 14:14:46 +01:00
Arun96980 441e3b6574 fix: add 'concept' to VALID_FILE_TYPES (#604) 2026-05-02 14:14:42 +01:00
Safi 0999822364 update README: add yaml/yml to file type table and feature description 2026-04-30 22:36:24 +01:00
Safi 47a994ad5b bump to v0.5.7 v0.5.7 2026-04-30 22:34:08 +01:00
SafiandClaude Sonnet 4.6 5aaa268650 add yaml/yml to DOC_EXTENSIONS so k8s/kustomize corpora are indexed (fixes #633)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 22:32:22 +01:00
Safi 3755fdc885 swap downloads badge back to pepy.tech 2026-04-30 09:48:37 +01:00
Safi 4e871ad36c swap downloads badge to shields.io for cache refresh 2026-04-30 09:47:55 +01:00
SafiandClaude Sonnet 4.6 97099edc38 bump to v0.5.6
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v0.5.6
2026-04-30 09:01:35 +01:00
SafiandClaude Sonnet 4.6 4360f9644b move changelog out of README into CHANGELOG.md, add yt-dlp legal notice
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 22:44:12 +01:00
SafiandClaude Sonnet 4.6 f755aca58f fix kimi temperature 400 error and community label deletion on cleanup (fixes #610, #608)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 17:03:44 +01:00
SafiandClaude Sonnet 4.6 28b17d37f1 remove Python upper bound to support 3.14+ (fixes #607)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 13:33:41 +01:00
SafiandClaude Sonnet 4.6 326c03e132 remove product-site from graphify repo
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 12:28:30 +01:00
SafiandClaude Sonnet 4.6 44fc32e3e2 add product-site as part of graphify repo (was separate submodule)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 12:27:56 +01:00
SafiandClaude Sonnet 4.6 744827166e update product site: fix pip install package name to graphifyy, update stats to 38k stars / 400k+ downloads / 1900+ waitlist
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 12:26:00 +01:00
SafiandClaude Sonnet 4.6 c750582db4 fix SyntaxWarning: use raw string for shell glob pattern with backslash escapes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 10:02:56 +01:00
SafiandClaude Sonnet 4.6 71d1b394e9 fix NameError in Kimi tip: use module-level os import instead of _os
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 09:58:54 +01:00
Safi a4ad901284 Update v0.5.5 release notes in README v0.5.5 2026-04-29 09:48:55 +01:00
SafiandClaude Sonnet 4.6 f9c344b546 Remember scan root so graphify update works without a path arg
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 09:43:51 +01:00
SafiandClaude Sonnet 4.6 59cbad3937 Fix Go package-call false-negative and llm.py robustness
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 08:57:16 +01:00
SafiandClaude Sonnet 4.6 5904081d7a Add Kimi K2.6 backend, fix phantom god nodes (#598), fix concept file_type (#601)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 08:51:53 +01:00
Safi eceaaad61e Add v0.5.4 release notes to README 2026-04-28 15:06:26 +01:00
SafiandClaude Sonnet 4.6 dd86271312 Fix SSRF DNS rebinding TOCTOU and yt-dlp URL bypass (#591, #592)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v0.5.4
2026-04-28 15:04:52 +01:00
SafiandClaude Sonnet 4.6 7359cdace9 Fix AST/semantic cache namespace collision breaking graphify update (#582)
AST and semantic entries now write to cache/ast/ and cache/semantic/
respectively. Previously both used the flat cache/ dir causing semantic
results to overwrite AST entries for code files on mixed corpora, making
the shrink guard fire on every subsequent update run.

Migration: load_cached falls back to legacy flat cache/ for AST reads
so existing cache entries are not lost on upgrade.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v0.5.3
2026-04-28 11:22:39 +01:00
SafiandClaude Sonnet 4.6 ee1df22b25 Fix PreToolUse hook for Claude Code v2.1.117+ (Glob|Grep → Bash matcher)
Grep and Glob tools removed in CC v2.1.117; searches now go through Bash.
Hook now reads stdin tool_input and pattern-matches on search commands.
Uninstall/reinstall handles both old and new matcher for clean upgrades.

Closes #578

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v0.5.2
2026-04-28 00:56:54 +01:00
SafiandClaude Sonnet 4.6 a566bfb566 Bump to v0.5.1
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v0.5.1
2026-04-27 21:45:57 +01:00
SafiandClaude Sonnet 4.6 4563b043f2 Fix 6 bugs: ID collisions, path portability, alias resolution, HTML controls, desync guard, rationale prompt
- #550: _file_stem() includes parent dir to prevent node ID collisions for same-named files
- #555: extract() relativizes source_file paths before returning for cross-machine portability
- #562: to_json() returns bool; _rebuild_code() writes report/html only if json succeeded
- #563: skill prompts store rationale as node attribute, not separate node; enforce calls direction
- #566: Show All / Hide All buttons added to HTML community panel
- #575: _import_js() resolves tsconfig.json compilerOptions.paths aliases before external fallback

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 21:44:22 +01:00
Safi 6175e0a8ea fix #547 #559 #570: expand ~ in hooksPath, fix gitignore inline comments, nosec on write sinks 2026-04-27 21:27:26 +01:00
Safi 770d7f54c4 Add The Memory Layer book badge to README 2026-04-25 15:56:59 +01:00
Safi 8bed332ff4 release: bump to v0.5.0 — clone, merge-graphs, shrink guard, dedup, bug fixes v0.5.0 2026-04-23 20:09:03 +01:00
Safi df9b7ec54f fix #479 #451: build_merge(), pre-write shrink guard, label dedup, chunk-suffix prompt block 2026-04-23 20:04:43 +01:00
Safi 2faeed99a2 feat: cross-repo merge-graphs; fix #527 CLAUDE_CONFIG_DIR; fix #524 graphify-out excluded from source scan 2026-04-23 19:59:14 +01:00
Safi 2c49da24f0 feat: graphify clone <github-url> — clone any repo and run full pipeline on it 2026-04-23 19:49:56 +01:00
Safi 5843ffc277 gitignore: exclude local benchmark scripts from repo 2026-04-23 19:38:54 +01:00
Safi b326aa83c4 update README: add check-update command, Java inheritance note 2026-04-22 23:32:25 +01:00
Safi e915a877e2 bump to v0.4.31 2026-04-22 23:28:22 +01:00
Safi 64f38acf3e implement #488 #482 #472 #490: legacy schema canonicalization, Java inheritance, aggregated HTML viz, check-update subcommand 2026-04-22 23:28:09 +01:00
Safi e4bdcc2487 add tests for canvas vault-relative paths and hook .exe skip 2026-04-22 23:04:40 +01:00
Safi 8981f7c8ef bump to v0.4.30 2026-04-22 23:02:41 +01:00
Safi 4bc20527c3 fix #513 #514: skip head -1 on .exe hook binary, fix to_canvas() hardcoded obsidian path 2026-04-22 23:01:53 +01:00
Safi 85e713644e bump to v0.4.29 2026-04-22 09:28:41 +01:00
Safi 52ad45bd27 fix #504: export PYTHONUTF8=1 in Step 1 to prevent garbled CJK output on Windows 2026-04-22 09:28:26 +01:00
Safi 86d6d9317f fix #498: opencode config to .opencode/opencode.json, fix skill temp file paths, add --wiki step 2026-04-22 09:19:53 +01:00
SafiandClaude Sonnet 4.6 d9b2928da1 fix: deterministic GRAPH_REPORT on large graphs, stable edge node IDs, correct common-root inference
- analyze.py: add seed=42 to betweenness_centrality() — eliminates non-deterministic GRAPH_REPORT.md diffs on graphs >1000 nodes (#499)
- extract.py: fix common-root inference to stop at first diverging segment not sum of all matches (#502)
- extract.py: resolve root to absolute path; post-process file node IDs to project-relative after extraction so graph.json edge endpoints are stable across machines (#502)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v0.4.27
2026-04-22 09:04:38 +01:00
SafiandClaude Sonnet 4.6 f8fd8f8479 fix: wiki encoding+collisions, hook rebase guard, detect path resolve, readme gitignore docs
- wiki.py: add encoding="utf-8" to all write_text() calls (fixes Windows cp1252 crash #496)
- wiki.py: deduplicate filenames with _unique_slug() to prevent silent article overwrites (#497)
- hooks.py: skip post-commit/post-checkout during rebase/merge/cherry-pick (#485)
- detect.py: resolve root path at detect() entry so .graphifyignore patterns match consistently (#495)
- README.md: document manifest.json, cost.json gitignore and .graphifyignore platform file examples (#369)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v0.4.26
2026-04-22 06:26:15 +01:00