mirror of
https://github.com/safishamsi/graphify.git
synced 2026-09-14 01:26:07 +00:00
v0.7.2: Fortran support + export CLI subcommands + skill.md size reduction
- Add Fortran support (26th language): .f/.F/.f90/.F90/.f95/.F95/.f03/.F03/.f08/.F08
via tree-sitter-fortran; capital-F files preprocessed with cpp -w -P
- Add graphify export {html,obsidian,wiki,svg,graphml,neo4j} CLI subcommands
- Add graphify query/path/explain CLI subcommands
- Reduce skill.md from 63KB to 47KB by replacing Python heredocs with CLI calls
- Extend to_html() with node_limit param for auto-aggregation on large graphs
- Add integration tests for all export/query/path/explain subcommands
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Safi
co-authored by
Claude Sonnet 4.6
Claude Sonnet 4.6
parent
f81e3bc215
commit
b6ffdbb8dd
@@ -2,6 +2,28 @@
|
||||
|
||||
Full release notes with details on each version: [GitHub Releases](https://github.com/safishamsi/graphify/releases)
|
||||
|
||||
## 0.7.2 (2026-05-04)
|
||||
|
||||
- Feat: Fortran support - extracts modules, subroutines, functions, programs, `use` imports, and `call` edges from `.f`, `.F`, `.f90`, `.F90`, `.f95`, `.F95`, `.f03`, `.F03`, `.f08`, `.F08` files; names are lowercased for case-insensitive matching (#694)
|
||||
|
||||
## 0.7.1 (2026-05-04)
|
||||
|
||||
- Fix: Obsidian export - community labels with `.`, `&`, `(`, `)` now produce valid Obsidian tags; only `[a-zA-Z0-9_\-/]` characters survive, preventing broken Dataview queries (#690)
|
||||
- Fix: `_load_tsconfig_aliases()` now follows tsconfig `extends` chains - SvelteKit, Nuxt, and NestJS path aliases defined in extended configs are no longer silently dropped (#691)
|
||||
- Fix: `.svelte` files now get a regex pass over the template layer after JS AST extraction - `{#await import('./X.svelte')}` markup-level dynamic imports are captured as edges (#692)
|
||||
- Fix: recursion limit raised to 10,000 at extract entry points (main process + each worker) with a `_safe_extract` wrapper that skips pathological files with a clear warning instead of crashing the whole run (#695)
|
||||
|
||||
## 0.7.0 (2026-05-03)
|
||||
|
||||
Multi-dev busy-repo support: four gaps that caused merge conflicts, stale graphs, and silent cache misses in team workflows.
|
||||
|
||||
- Feat: `graphify hook install` now also configures a git merge driver for `graphify-out/graph.json` — union-merges two graph.json files so git never produces conflict markers in the knowledge graph; writes `.gitattributes` and registers `graphify merge-driver` in `.git/config`
|
||||
- Feat: `graphify merge-driver <base> <current> <other>` subcommand — takes two graph.json variants and writes their node/edge union back to `<current>`; always exits 0 so merge never blocks
|
||||
- Feat: Leiden community detection now seeded (`seed=42` when supported) for deterministic community IDs across parallel rebuilds — reduces JSON diff churn in multi-dev repos
|
||||
- Feat: `graph.json` now embeds `built_at_commit` (git HEAD) at write time; `GRAPH_REPORT.md` surfaces the commit hash and a freshness check hint
|
||||
- Fix: `file_hash` is now content-only (path removed from hash) — renamed files reuse their cache entry instead of re-extracting; cached `source_file` fields are updated to the new path on load
|
||||
- Fix: watch mode mixed-batch handling — commits with both code and non-code files now rebuild code immediately AND write `needs_update` flag; previously code changes were silently dropped in mixed batches
|
||||
|
||||
## 0.6.9 (2026-05-03)
|
||||
|
||||
- Fix: `source_file` path separators normalized to forward slashes at graph ingestion — same physical file emitted with backslashes (Windows AST extractor) and forward slashes (semantic subagents) now merges into one node instead of splitting into two disconnected components (#683)
|
||||
|
||||
@@ -124,7 +124,7 @@ Uninstall with the matching command (e.g. `graphify claude uninstall`).
|
||||
|
||||
| Type | Extensions |
|
||||
|------|-----------|
|
||||
| Code (25 languages) | `.py .ts .js .jsx .tsx .go .rs .java .c .cpp .rb .cs .kt .scala .php .swift .lua .zig .ps1 .ex .exs .m .jl .vue .svelte .sql` |
|
||||
| Code (26 languages) | `.py .ts .js .jsx .tsx .go .rs .java .c .cpp .rb .cs .kt .scala .php .swift .lua .zig .ps1 .ex .exs .m .jl .vue .svelte .sql .f .F .f90 .F90 .f95 .F95 .f03 .F03 .f08 .F08` |
|
||||
| Docs | `.md .mdx .html .txt .rst .yaml .yml` |
|
||||
| Office | `.docx .xlsx` (requires `pip install graphifyy[office]`) |
|
||||
| PDFs | `.pdf` |
|
||||
@@ -192,7 +192,7 @@ graphify-out/cost.json # local only
|
||||
**Workflow:**
|
||||
1. One person runs `/graphify .` and commits `graphify-out/`.
|
||||
2. Everyone pulls — their assistant reads the graph immediately.
|
||||
3. Run `graphify hook install` to auto-rebuild after each commit (AST only, no API cost).
|
||||
3. Run `graphify hook install` to auto-rebuild after each commit (AST only, no API cost). This also sets up a git merge driver so `graph.json` is never left with conflict markers — two devs committing in parallel get their graphs union-merged automatically.
|
||||
4. When docs or papers change, run `/graphify --update` to refresh those nodes.
|
||||
|
||||
---
|
||||
|
||||
+169
-1
@@ -1060,6 +1060,7 @@ def main() -> None:
|
||||
print(" explain \"X\" plain-language explanation of a node and its neighbors")
|
||||
print(" --graph <path> path to graph.json (default graphify-out/graph.json)")
|
||||
print(" clone <github-url> clone a GitHub repo locally and print its path for /graphify")
|
||||
print(" merge-driver <base> <current> <other> git merge driver: union-merge two graph.json files (set up via hook install)")
|
||||
print(" merge-graphs <g1> <g2> merge two or more graph.json files into one cross-repo graph")
|
||||
print(" --out <path> output path (default: graphify-out/merged-graph.json)")
|
||||
print(" --branch <branch> checkout a specific branch (default: repo default)")
|
||||
@@ -1516,10 +1517,12 @@ def main() -> None:
|
||||
labels = {cid: f"Community {cid}" for cid in communities}
|
||||
questions = suggest_questions(G, communities, labels)
|
||||
tokens = {"input": 0, "output": 0}
|
||||
from graphify.export import _git_head as _gh
|
||||
_commit = _gh()
|
||||
report = generate(G, communities, cohesion, labels, gods, surprises,
|
||||
{"warning": "cluster-only mode — file stats not available"},
|
||||
tokens, str(watch_path), suggested_questions=questions,
|
||||
min_community_size=min_community_size)
|
||||
min_community_size=min_community_size, built_at_commit=_commit)
|
||||
out = watch_path / "graphify-out"
|
||||
(out / "GRAPH_REPORT.md").write_text(report, encoding="utf-8")
|
||||
to_json(G, communities, str(out / "graph.json"))
|
||||
@@ -1640,6 +1643,37 @@ def main() -> None:
|
||||
print(f"open with: xdg-open {out} (or file://{out.resolve()})")
|
||||
sys.exit(0)
|
||||
|
||||
elif cmd == "merge-driver":
|
||||
# git merge driver for graph.json — takes (base, current, other) and writes
|
||||
# the union of current+other nodes/edges back to current. Always exits 0
|
||||
# so git never marks graph.json as conflicted.
|
||||
# Usage: graphify merge-driver %O %A %B (set in .git/config merge driver)
|
||||
if len(sys.argv) < 5:
|
||||
print("Usage: graphify merge-driver <base> <current> <other>", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
_base_path, _current_path, _other_path = sys.argv[2], sys.argv[3], sys.argv[4]
|
||||
import networkx as _nx
|
||||
from networkx.readwrite import json_graph as _jg
|
||||
def _load_graph(p: str):
|
||||
data = json.loads(Path(p).read_text(encoding="utf-8"))
|
||||
try:
|
||||
return _jg.node_link_graph(data, edges="links"), data
|
||||
except TypeError:
|
||||
return _jg.node_link_graph(data), data
|
||||
try:
|
||||
G_cur, _ = _load_graph(_current_path)
|
||||
G_oth, _ = _load_graph(_other_path)
|
||||
except Exception as exc:
|
||||
print(f"[graphify merge-driver] error loading graphs: {exc}", file=sys.stderr)
|
||||
sys.exit(0) # exit 0 so git doesn't block the merge
|
||||
merged = _nx.compose(G_cur, G_oth)
|
||||
try:
|
||||
out_data = _jg.node_link_data(merged, edges="links")
|
||||
except TypeError:
|
||||
out_data = _jg.node_link_data(merged)
|
||||
Path(_current_path).write_text(json.dumps(out_data, indent=2), encoding="utf-8")
|
||||
sys.exit(0)
|
||||
|
||||
elif cmd == "merge-graphs":
|
||||
# graphify merge-graphs graph1.json graph2.json ... --out merged.json
|
||||
args = sys.argv[2:]
|
||||
@@ -1700,6 +1734,140 @@ def main() -> None:
|
||||
local_path = _clone_repo(url, branch=branch, out_dir=out_dir)
|
||||
print(local_path)
|
||||
|
||||
elif cmd == "export":
|
||||
subcmd = sys.argv[2] if len(sys.argv) > 2 else ""
|
||||
if subcmd not in ("html", "obsidian", "wiki", "svg", "graphml", "neo4j"):
|
||||
print("Usage: graphify export <format>", file=sys.stderr)
|
||||
print(" html [--graph PATH] [--labels PATH] [--node-limit N] [--no-viz]", file=sys.stderr)
|
||||
print(" obsidian [--graph PATH] [--labels PATH] [--dir PATH]", file=sys.stderr)
|
||||
print(" wiki [--graph PATH] [--labels PATH]", file=sys.stderr)
|
||||
print(" svg [--graph PATH] [--labels PATH]", file=sys.stderr)
|
||||
print(" graphml [--graph PATH]", file=sys.stderr)
|
||||
print(" neo4j [--graph PATH] [--push URI] [--user U] [--password P]", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Parse shared args
|
||||
args = sys.argv[3:]
|
||||
graph_path = Path(_GRAPHIFY_OUT) / "graph.json"
|
||||
labels_path = Path(_GRAPHIFY_OUT) / ".graphify_labels.json"
|
||||
analysis_path = Path(_GRAPHIFY_OUT) / ".graphify_analysis.json"
|
||||
node_limit = 5000
|
||||
no_viz = False
|
||||
obsidian_dir = Path(_GRAPHIFY_OUT) / "obsidian"
|
||||
neo4j_uri: str | None = None
|
||||
neo4j_user = "neo4j"
|
||||
neo4j_password: str | None = None
|
||||
i = 0
|
||||
while i < len(args):
|
||||
a = args[i]
|
||||
if a == "--graph" and i + 1 < len(args):
|
||||
graph_path = Path(args[i + 1]); i += 2
|
||||
elif a == "--labels" and i + 1 < len(args):
|
||||
labels_path = Path(args[i + 1]); i += 2
|
||||
elif a == "--node-limit" and i + 1 < len(args):
|
||||
node_limit = int(args[i + 1]); i += 2
|
||||
elif a == "--no-viz":
|
||||
no_viz = True; i += 1
|
||||
elif a == "--dir" and i + 1 < len(args):
|
||||
obsidian_dir = Path(args[i + 1]); i += 2
|
||||
elif a == "--push" and i + 1 < len(args):
|
||||
neo4j_uri = args[i + 1]; i += 2
|
||||
elif a == "--user" and i + 1 < len(args):
|
||||
neo4j_user = args[i + 1]; i += 2
|
||||
elif a == "--password" and i + 1 < len(args):
|
||||
neo4j_password = args[i + 1]; i += 2
|
||||
else:
|
||||
i += 1
|
||||
|
||||
if not graph_path.exists():
|
||||
print(f"error: graph not found: {graph_path}. Run /graphify <path> first.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
from networkx.readwrite import json_graph as _jg
|
||||
from graphify.build import build_from_json as _bfj
|
||||
|
||||
_raw = json.loads(graph_path.read_text(encoding="utf-8"))
|
||||
try:
|
||||
G = _jg.node_link_graph(_raw, edges="links")
|
||||
except TypeError:
|
||||
G = _jg.node_link_graph(_raw)
|
||||
|
||||
# Load optional analysis/labels
|
||||
communities: dict[int, list[str]] = {}
|
||||
if analysis_path.exists():
|
||||
_an = json.loads(analysis_path.read_text(encoding="utf-8"))
|
||||
communities = {int(k): v for k, v in _an.get("communities", {}).items()}
|
||||
cohesion: dict[int, float] = {int(k): v for k, v in _an.get("cohesion", {}).items()}
|
||||
gods_data = _an.get("gods", [])
|
||||
else:
|
||||
cohesion = {}
|
||||
gods_data = []
|
||||
|
||||
labels: dict[int, str] = {}
|
||||
if labels_path.exists():
|
||||
labels = {int(k): v for k, v in json.loads(labels_path.read_text(encoding="utf-8")).items()}
|
||||
|
||||
out_dir = graph_path.parent
|
||||
|
||||
if subcmd == "html":
|
||||
from graphify.export import to_html as _to_html
|
||||
if no_viz:
|
||||
html_target = out_dir / "graph.html"
|
||||
if html_target.exists():
|
||||
html_target.unlink()
|
||||
print("--no-viz: skipped graph.html")
|
||||
else:
|
||||
_to_html(G, communities, str(out_dir / "graph.html"),
|
||||
community_labels=labels or None, node_limit=node_limit)
|
||||
if G.number_of_nodes() <= node_limit:
|
||||
print(f"graph.html written - open in any browser, no server needed")
|
||||
|
||||
elif subcmd == "obsidian":
|
||||
from graphify.export import to_obsidian as _to_obsidian, to_canvas as _to_canvas
|
||||
n = _to_obsidian(G, communities, str(obsidian_dir),
|
||||
community_labels=labels or None, cohesion=cohesion or None)
|
||||
print(f"Obsidian vault: {n} notes in {obsidian_dir}/")
|
||||
_to_canvas(G, communities, str(obsidian_dir / "graph.canvas"),
|
||||
community_labels=labels or None)
|
||||
print(f"Canvas: {obsidian_dir}/graph.canvas")
|
||||
print(f"Open {obsidian_dir}/ as a vault in Obsidian.")
|
||||
|
||||
elif subcmd == "wiki":
|
||||
from graphify.wiki import to_wiki as _to_wiki
|
||||
from graphify.analyze import god_nodes as _god_nodes
|
||||
if not gods_data:
|
||||
gods_data = _god_nodes(G)
|
||||
n = _to_wiki(G, communities, str(out_dir / "wiki"),
|
||||
community_labels=labels or None, cohesion=cohesion or None,
|
||||
god_nodes_data=gods_data)
|
||||
print(f"Wiki: {n} articles written to {out_dir}/wiki/")
|
||||
print(f" {out_dir}/wiki/index.md -> agent entry point")
|
||||
|
||||
elif subcmd == "svg":
|
||||
from graphify.export import to_svg as _to_svg
|
||||
_to_svg(G, communities, str(out_dir / "graph.svg"),
|
||||
community_labels=labels or None)
|
||||
print(f"graph.svg written - embeds in Obsidian, Notion, GitHub READMEs")
|
||||
|
||||
elif subcmd == "graphml":
|
||||
from graphify.export import to_graphml as _to_graphml
|
||||
_to_graphml(G, communities, str(out_dir / "graph.graphml"))
|
||||
print(f"graph.graphml written - open in Gephi, yEd, or any GraphML tool")
|
||||
|
||||
elif subcmd == "neo4j":
|
||||
if neo4j_uri:
|
||||
from graphify.export import push_to_neo4j as _push
|
||||
if neo4j_password is None:
|
||||
print("error: --password required for --push", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
result = _push(G, uri=neo4j_uri, user=neo4j_user,
|
||||
password=neo4j_password, communities=communities)
|
||||
print(f"Pushed to Neo4j: {result['nodes']} nodes, {result['edges']} edges")
|
||||
else:
|
||||
from graphify.export import to_cypher as _to_cypher
|
||||
_to_cypher(G, str(out_dir / "cypher.txt"))
|
||||
print(f"cypher.txt written - import with: cypher-shell < {out_dir}/cypher.txt")
|
||||
|
||||
elif cmd == "benchmark":
|
||||
from graphify.benchmark import run_benchmark, print_benchmark
|
||||
graph_path = sys.argv[2] if len(sys.argv) > 2 else "graphify-out/graph.json"
|
||||
|
||||
+1
-1
@@ -18,7 +18,7 @@ class FileType(str, Enum):
|
||||
|
||||
_MANIFEST_PATH = "graphify-out/manifest.json"
|
||||
|
||||
CODE_EXTENSIONS = {'.py', '.ts', '.js', '.jsx', '.tsx', '.mjs', '.ejs', '.go', '.rs', '.java', '.cpp', '.cc', '.cxx', '.c', '.h', '.hpp', '.rb', '.swift', '.kt', '.kts', '.cs', '.scala', '.php', '.lua', '.toc', '.zig', '.ps1', '.ex', '.exs', '.m', '.mm', '.jl', '.vue', '.svelte', '.dart', '.v', '.sv', '.sql', '.r'}
|
||||
CODE_EXTENSIONS = {'.py', '.ts', '.js', '.jsx', '.tsx', '.mjs', '.ejs', '.go', '.rs', '.java', '.cpp', '.cc', '.cxx', '.c', '.h', '.hpp', '.rb', '.swift', '.kt', '.kts', '.cs', '.scala', '.php', '.lua', '.toc', '.zig', '.ps1', '.ex', '.exs', '.m', '.mm', '.jl', '.vue', '.svelte', '.dart', '.v', '.sv', '.sql', '.r', '.f', '.F', '.f90', '.F90', '.f95', '.F95', '.f03', '.F03', '.f08', '.F08'}
|
||||
DOC_EXTENSIONS = {'.md', '.mdx', '.txt', '.rst', '.html', '.yaml', '.yml'}
|
||||
PAPER_EXTENSIONS = {'.pdf'}
|
||||
IMAGE_EXTENSIONS = {'.png', '.jpg', '.jpeg', '.gif', '.webp', '.svg'}
|
||||
|
||||
+57
-4
@@ -11,6 +11,15 @@ from networkx.readwrite import json_graph
|
||||
from graphify.security import sanitize_label
|
||||
from graphify.analyze import _node_community_map
|
||||
|
||||
def _obsidian_tag(name: str) -> str:
|
||||
"""Sanitize a community name for use as an Obsidian tag.
|
||||
|
||||
Obsidian tags only allow alphanumerics, hyphens, underscores, and slashes.
|
||||
Spaces become underscores; everything else is stripped.
|
||||
"""
|
||||
return re.sub(r"[^a-zA-Z0-9_\-/]", "", name.replace(" ", "_"))
|
||||
|
||||
|
||||
def _strip_diacritics(text: str) -> str:
|
||||
import unicodedata
|
||||
nfkd = unicodedata.normalize("NFKD", text)
|
||||
@@ -340,7 +349,17 @@ def attach_hyperedges(G: nx.Graph, hyperedges: list) -> None:
|
||||
G.graph["hyperedges"] = existing
|
||||
|
||||
|
||||
def to_json(G: nx.Graph, communities: dict[int, list[str]], output_path: str, *, force: bool = False) -> bool:
|
||||
def _git_head() -> str | None:
|
||||
"""Return the current git HEAD commit hash, or None if not in a git repo."""
|
||||
import subprocess as _sp
|
||||
try:
|
||||
r = _sp.run(["git", "rev-parse", "HEAD"], capture_output=True, text=True, timeout=3)
|
||||
return r.stdout.strip() if r.returncode == 0 else None
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def to_json(G: nx.Graph, communities: dict[int, list[str]], output_path: str, *, force: bool = False, built_at_commit: str | None = None) -> bool:
|
||||
# Safety check: refuse to silently shrink an existing graph (#479)
|
||||
existing_path = Path(output_path)
|
||||
if not force and existing_path.exists():
|
||||
@@ -383,6 +402,9 @@ def to_json(G: nx.Graph, communities: dict[int, list[str]], output_path: str, *,
|
||||
link["source"] = true_src
|
||||
link["target"] = true_tgt
|
||||
data["hyperedges"] = getattr(G, "graph", {}).get("hyperedges", [])
|
||||
commit = built_at_commit if built_at_commit is not None else _git_head()
|
||||
if commit:
|
||||
data["built_at_commit"] = commit
|
||||
with open(output_path, "w", encoding="utf-8") as f: # nosec
|
||||
json.dump(data, f, indent=2)
|
||||
return True
|
||||
@@ -436,6 +458,7 @@ def to_html(
|
||||
output_path: str,
|
||||
community_labels: dict[int, str] | None = None,
|
||||
member_counts: dict[int, int] | None = None,
|
||||
node_limit: int | None = None,
|
||||
) -> None:
|
||||
"""Generate an interactive vis.js HTML visualization of the graph.
|
||||
|
||||
@@ -445,9 +468,39 @@ def to_html(
|
||||
|
||||
If member_counts is provided (aggregated community view), node sizes are
|
||||
based on community member counts rather than graph degree.
|
||||
|
||||
If node_limit is set and the graph exceeds it, automatically builds an
|
||||
aggregated community-level meta-graph instead of raising ValueError.
|
||||
"""
|
||||
limit = _viz_node_limit()
|
||||
limit = node_limit if node_limit is not None else _viz_node_limit()
|
||||
if G.number_of_nodes() > limit:
|
||||
if node_limit is not None:
|
||||
# Build aggregated community meta-graph
|
||||
from collections import Counter as _Counter
|
||||
import networkx as _nx
|
||||
print(f"Graph has {G.number_of_nodes()} nodes (above {limit} limit). Building aggregated community view...")
|
||||
node_to_community = {nid: cid for cid, members in communities.items() for nid in members}
|
||||
meta = _nx.Graph()
|
||||
for cid, members in communities.items():
|
||||
meta.add_node(str(cid), label=(community_labels or {}).get(cid, f"Community {cid}"))
|
||||
edge_counts = _Counter()
|
||||
for u, v in G.edges():
|
||||
cu, cv = node_to_community.get(u), node_to_community.get(v)
|
||||
if cu is not None and cv is not None and cu != cv:
|
||||
edge_counts[(min(cu, cv), max(cu, cv))] += 1
|
||||
for (cu, cv), w in edge_counts.items():
|
||||
meta.add_edge(str(cu), str(cv), weight=w,
|
||||
relation=f"{w} cross-community edges", confidence="AGGREGATED")
|
||||
if meta.number_of_nodes() <= 1:
|
||||
print("Single community - aggregated view not useful. Skipping graph.html.")
|
||||
return
|
||||
meta_communities = {cid: [str(cid)] for cid in communities}
|
||||
mc = {cid: len(members) for cid, members in communities.items()}
|
||||
to_html(meta, meta_communities, output_path,
|
||||
community_labels=community_labels, member_counts=mc)
|
||||
print(f"graph.html written (aggregated: {meta.number_of_nodes()} community nodes, {meta.number_of_edges()} cross-community edges)")
|
||||
print("Tip: run with --obsidian for full node-level detail.")
|
||||
return
|
||||
raise ValueError(
|
||||
f"Graph has {G.number_of_nodes()} nodes - too large for HTML viz "
|
||||
f"(limit: {limit}). Use --no-viz, raise GRAPHIFY_VIZ_NODE_LIMIT, "
|
||||
@@ -639,7 +692,7 @@ def to_obsidian(
|
||||
ftype_tag = _FTYPE_TAG.get(ftype, f"graphify/{ftype}" if ftype else "graphify/document")
|
||||
dom_conf = _dominant_confidence(node_id)
|
||||
conf_tag = f"graphify/{dom_conf}"
|
||||
comm_tag = f"community/{community_name.replace(' ', '_')}"
|
||||
comm_tag = f"community/{_obsidian_tag(community_name)}"
|
||||
node_tags = [ftype_tag, conf_tag, comm_tag]
|
||||
|
||||
lines: list[str] = []
|
||||
@@ -751,7 +804,7 @@ def to_obsidian(
|
||||
lines.append("")
|
||||
|
||||
# Dataview live query (improvement 2)
|
||||
comm_tag_name = community_name.replace(" ", "_")
|
||||
comm_tag_name = _obsidian_tag(community_name)
|
||||
lines.append("## Live Query (requires Dataview plugin)")
|
||||
lines.append("")
|
||||
lines.append("```dataview")
|
||||
|
||||
+302
-17
@@ -10,6 +10,24 @@ from pathlib import Path
|
||||
from typing import Callable, Any
|
||||
from .cache import load_cached, save_cached
|
||||
|
||||
_RECURSION_LIMIT = 10_000
|
||||
|
||||
|
||||
def _raise_recursion_limit() -> None:
|
||||
if sys.getrecursionlimit() < _RECURSION_LIMIT:
|
||||
sys.setrecursionlimit(_RECURSION_LIMIT)
|
||||
|
||||
|
||||
def _safe_extract(extractor: Callable, path: Path) -> dict:
|
||||
try:
|
||||
return extractor(path)
|
||||
except RecursionError:
|
||||
print(f" warning: skipped {path} (recursion limit exceeded)", file=sys.stderr, flush=True)
|
||||
return {"nodes": [], "edges": [], "error": "recursion_limit_exceeded"}
|
||||
except Exception as e:
|
||||
print(f" warning: skipped {path} ({type(e).__name__}: {e})", file=sys.stderr, flush=True)
|
||||
return {"nodes": [], "edges": [], "error": f"{type(e).__name__}: {e}"}
|
||||
|
||||
|
||||
def _make_id(*parts: str) -> str:
|
||||
"""Build a stable node ID from one or more name parts."""
|
||||
@@ -30,9 +48,44 @@ def _file_stem(path: Path) -> str:
|
||||
_TSCONFIG_ALIAS_CACHE: dict[str, dict[str, str]] = {}
|
||||
|
||||
|
||||
def _read_tsconfig_aliases(tsconfig: Path, base_dir: Path, seen: set) -> dict[str, str]:
|
||||
"""Recursively read path aliases from a tsconfig, following extends chains.
|
||||
|
||||
Child config paths override parent. Circular extends are detected via seen set.
|
||||
npm package configs (e.g. @tsconfig/svelte) are skipped since they're not on disk.
|
||||
"""
|
||||
if str(tsconfig) in seen:
|
||||
return {}
|
||||
seen.add(str(tsconfig))
|
||||
try:
|
||||
data = json.loads(tsconfig.read_text(encoding="utf-8"))
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
aliases: dict[str, str] = {}
|
||||
extends = data.get("extends")
|
||||
if extends and not extends.startswith("@"):
|
||||
extended_path = (base_dir / extends).resolve()
|
||||
if not extended_path.suffix:
|
||||
extended_path = extended_path.with_suffix(".json")
|
||||
if extended_path.exists():
|
||||
aliases.update(_read_tsconfig_aliases(extended_path, extended_path.parent, seen))
|
||||
|
||||
paths = data.get("compilerOptions", {}).get("paths", {})
|
||||
for alias, targets in paths.items():
|
||||
if not targets:
|
||||
continue
|
||||
alias_prefix = alias.rstrip("/*")
|
||||
target_base = targets[0].rstrip("/*")
|
||||
aliases[alias_prefix] = str(base_dir / target_base)
|
||||
|
||||
return aliases
|
||||
|
||||
|
||||
def _load_tsconfig_aliases(start_dir: Path) -> dict[str, str]:
|
||||
"""Walk up from start_dir to find tsconfig.json and return compilerOptions.paths aliases.
|
||||
|
||||
Follows extends chains so SvelteKit/Nuxt/NestJS inherited aliases are included.
|
||||
Returns a dict mapping alias prefix (e.g. "@/") to resolved base dir (e.g. "src/").
|
||||
Result is cached by tsconfig path string.
|
||||
"""
|
||||
@@ -42,20 +95,7 @@ def _load_tsconfig_aliases(start_dir: Path) -> dict[str, str]:
|
||||
if tsconfig.exists():
|
||||
key = str(tsconfig)
|
||||
if key not in _TSCONFIG_ALIAS_CACHE:
|
||||
try:
|
||||
data = json.loads(tsconfig.read_text(encoding="utf-8"))
|
||||
paths = data.get("compilerOptions", {}).get("paths", {})
|
||||
aliases: dict[str, str] = {}
|
||||
for alias, targets in paths.items():
|
||||
if not targets:
|
||||
continue
|
||||
# Strip trailing /* from alias and target
|
||||
alias_prefix = alias.rstrip("/*")
|
||||
target_base = targets[0].rstrip("/*")
|
||||
aliases[alias_prefix] = str(candidate / target_base)
|
||||
_TSCONFIG_ALIAS_CACHE[key] = aliases
|
||||
except Exception:
|
||||
_TSCONFIG_ALIAS_CACHE[key] = {}
|
||||
_TSCONFIG_ALIAS_CACHE[key] = _read_tsconfig_aliases(tsconfig, candidate, seen=set())
|
||||
return _TSCONFIG_ALIAS_CACHE[key]
|
||||
return {}
|
||||
|
||||
@@ -1656,6 +1696,42 @@ def extract_js(path: Path) -> dict:
|
||||
return _extract_generic(path, config)
|
||||
|
||||
|
||||
def extract_svelte(path: Path) -> dict:
|
||||
"""Extract imports from .svelte files: script-block via JS AST + template regex fallback.
|
||||
|
||||
Tree-sitter only sees the <script> block. Svelte template syntax like
|
||||
{#await import('./X.svelte')} lives in the markup layer and is invisible
|
||||
to the JS parser, so a regex pass covers those dynamic imports.
|
||||
"""
|
||||
result = _extract_generic(path, _JS_CONFIG)
|
||||
try:
|
||||
import re as _re
|
||||
src = path.read_text(encoding="utf-8", errors="replace")
|
||||
existing_ids = {n["id"] for n in result.get("nodes", [])}
|
||||
file_node_id = _make_id(path.stem, str(path))
|
||||
for m in _re.finditer(r"""import\(\s*['"]([^'"]+)['"]\s*\)""", src):
|
||||
raw = m.group(1)
|
||||
if not raw.startswith("."):
|
||||
continue
|
||||
node_id = _make_id(raw, str(path))
|
||||
if node_id in existing_ids:
|
||||
continue
|
||||
result.setdefault("nodes", []).append({
|
||||
"id": node_id, "label": raw,
|
||||
"file_type": "code", "source_file": str(path),
|
||||
"confidence": "EXTRACTED",
|
||||
})
|
||||
result.setdefault("edges", []).append({
|
||||
"source": file_node_id, "target": node_id,
|
||||
"relation": "dynamic_import", "confidence": "EXTRACTED",
|
||||
"source_file": str(path),
|
||||
})
|
||||
existing_ids.add(node_id)
|
||||
except Exception:
|
||||
pass
|
||||
return result
|
||||
|
||||
|
||||
def extract_java(path: Path) -> dict:
|
||||
"""Extract classes, interfaces, methods, constructors, and imports from a .java file."""
|
||||
return _extract_generic(path, _JAVA_CONFIG)
|
||||
@@ -2257,6 +2333,203 @@ def extract_julia(path: Path) -> dict:
|
||||
return {"nodes": nodes, "edges": edges}
|
||||
|
||||
|
||||
_FORTRAN_CPP_EXTS = {".F", ".F90", ".F95", ".F03", ".F08"}
|
||||
|
||||
|
||||
def _cpp_preprocess(path: Path) -> bytes:
|
||||
"""Run cpp -w -P on a capital-F Fortran file and return preprocessed bytes.
|
||||
|
||||
Falls back to raw file bytes if cpp is not available. Capital-F extensions
|
||||
conventionally require C preprocessor expansion (#ifdef MPI, #define REAL8, etc.)
|
||||
before parsing.
|
||||
"""
|
||||
import shutil
|
||||
import subprocess
|
||||
if not shutil.which("cpp"):
|
||||
return path.read_bytes()
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["cpp", "-w", "-P", str(path)],
|
||||
capture_output=True,
|
||||
timeout=30,
|
||||
)
|
||||
if result.returncode == 0 and result.stdout:
|
||||
return result.stdout
|
||||
except Exception:
|
||||
pass
|
||||
return path.read_bytes()
|
||||
|
||||
|
||||
def extract_fortran(path: Path) -> dict:
|
||||
"""Extract programs, modules, subroutines, functions, use statements, and calls from Fortran files.
|
||||
|
||||
Capital-F extensions (.F, .F90, etc.) are run through the C preprocessor before
|
||||
parsing so #ifdef/#define macros are resolved.
|
||||
"""
|
||||
try:
|
||||
import tree_sitter_fortran as tsfortran
|
||||
from tree_sitter import Language, Parser
|
||||
except ImportError:
|
||||
return {"nodes": [], "edges": [], "error": "tree-sitter-fortran not installed"}
|
||||
|
||||
try:
|
||||
language = Language(tsfortran.language())
|
||||
parser = Parser(language)
|
||||
source = _cpp_preprocess(path) if path.suffix in _FORTRAN_CPP_EXTS else path.read_bytes()
|
||||
tree = parser.parse(source)
|
||||
root = tree.root_node
|
||||
except Exception as e:
|
||||
return {"nodes": [], "edges": [], "error": str(e)}
|
||||
|
||||
stem = _file_stem(path)
|
||||
str_path = str(path)
|
||||
nodes: list[dict] = []
|
||||
edges: list[dict] = []
|
||||
seen_ids: set[str] = set()
|
||||
scope_bodies: list[tuple[str, object]] = []
|
||||
|
||||
def add_node(nid: str, label: str, line: int) -> None:
|
||||
if nid not in seen_ids:
|
||||
seen_ids.add(nid)
|
||||
nodes.append({
|
||||
"id": nid,
|
||||
"label": label,
|
||||
"file_type": "code",
|
||||
"source_file": str_path,
|
||||
"source_location": f"L{line}",
|
||||
})
|
||||
|
||||
def add_edge(src: str, tgt: str, relation: str, line: int,
|
||||
confidence: str = "EXTRACTED", weight: float = 1.0,
|
||||
context: str | None = None) -> None:
|
||||
edge = {
|
||||
"source": src,
|
||||
"target": tgt,
|
||||
"relation": relation,
|
||||
"confidence": confidence,
|
||||
"source_file": str_path,
|
||||
"source_location": f"L{line}",
|
||||
"weight": weight,
|
||||
}
|
||||
if context:
|
||||
edge["context"] = context
|
||||
edges.append(edge)
|
||||
|
||||
file_nid = _make_id(str(path))
|
||||
add_node(file_nid, path.name, 1)
|
||||
|
||||
def _fortran_name(stmt_node) -> str | None:
|
||||
"""Extract name from a *_statement node. Fortran is case-insensitive; lowercase."""
|
||||
for child in stmt_node.children:
|
||||
if child.type in ("name", "identifier"):
|
||||
return _read_text(child, source).lower()
|
||||
return None
|
||||
|
||||
def walk_calls(node, scope_nid: str) -> None:
|
||||
if node is None:
|
||||
return
|
||||
t = node.type
|
||||
if t in ("subroutine", "function", "module", "program", "internal_procedures"):
|
||||
return
|
||||
# call FOO(args) — tree-sitter-fortran uses subroutine_call
|
||||
if t == "subroutine_call":
|
||||
name_node = next((c for c in node.children if c.type == "identifier"), None)
|
||||
if name_node:
|
||||
callee = _read_text(name_node, source).lower()
|
||||
target_nid = _make_id(stem, callee)
|
||||
add_edge(scope_nid, target_nid, "calls", node.start_point[0] + 1,
|
||||
confidence="EXTRACTED", context="call")
|
||||
for child in node.children:
|
||||
walk_calls(child, scope_nid)
|
||||
|
||||
def walk(node, scope_nid: str) -> None:
|
||||
t = node.type
|
||||
|
||||
if t == "program":
|
||||
stmt = next((c for c in node.children if c.type == "program_statement"), None)
|
||||
name = _fortran_name(stmt) if stmt else None
|
||||
if name:
|
||||
nid = _make_id(stem, name)
|
||||
line = node.start_point[0] + 1
|
||||
add_node(nid, name, line)
|
||||
add_edge(file_nid, nid, "defines", line)
|
||||
scope_bodies.append((nid, node))
|
||||
for child in node.children:
|
||||
walk(child, nid)
|
||||
return
|
||||
|
||||
if t == "module":
|
||||
stmt = next((c for c in node.children if c.type == "module_statement"), None)
|
||||
name = _fortran_name(stmt) if stmt else None
|
||||
if name:
|
||||
nid = _make_id(stem, name)
|
||||
line = node.start_point[0] + 1
|
||||
add_node(nid, name, line)
|
||||
add_edge(file_nid, nid, "defines", line)
|
||||
for child in node.children:
|
||||
walk(child, nid)
|
||||
return
|
||||
|
||||
# subroutines/functions inside a module live under internal_procedures
|
||||
if t == "internal_procedures":
|
||||
for child in node.children:
|
||||
walk(child, scope_nid)
|
||||
return
|
||||
|
||||
if t == "subroutine":
|
||||
stmt = next((c for c in node.children if c.type == "subroutine_statement"), None)
|
||||
name = _fortran_name(stmt) if stmt else None
|
||||
if name:
|
||||
nid = _make_id(stem, name)
|
||||
line = node.start_point[0] + 1
|
||||
add_node(nid, f"{name}()", line)
|
||||
add_edge(scope_nid, nid, "defines", line)
|
||||
scope_bodies.append((nid, node))
|
||||
for child in node.children:
|
||||
walk(child, nid)
|
||||
return
|
||||
|
||||
if t == "function":
|
||||
stmt = next((c for c in node.children if c.type == "function_statement"), None)
|
||||
name = _fortran_name(stmt) if stmt else None
|
||||
if name:
|
||||
nid = _make_id(stem, name)
|
||||
line = node.start_point[0] + 1
|
||||
add_node(nid, f"{name}()", line)
|
||||
add_edge(scope_nid, nid, "defines", line)
|
||||
scope_bodies.append((nid, node))
|
||||
for child in node.children:
|
||||
walk(child, nid)
|
||||
return
|
||||
|
||||
if t == "use_statement":
|
||||
line = node.start_point[0] + 1
|
||||
# tree-sitter-fortran uses module_name node for the used module
|
||||
name_node = next((c for c in node.children if c.type in ("module_name", "name", "identifier")), None)
|
||||
if name_node:
|
||||
mod_name = _read_text(name_node, source).lower()
|
||||
imp_nid = _make_id(mod_name)
|
||||
add_node(imp_nid, mod_name, line)
|
||||
add_edge(scope_nid, imp_nid, "imports", line, context="use")
|
||||
return
|
||||
|
||||
for child in node.children:
|
||||
walk(child, scope_nid)
|
||||
|
||||
walk(root, file_nid)
|
||||
|
||||
_stmt_headers = {
|
||||
"subroutine_statement", "function_statement",
|
||||
"program_statement", "module_statement",
|
||||
}
|
||||
for scope_nid, body_node in scope_bodies:
|
||||
for child in body_node.children:
|
||||
if child.type not in _stmt_headers:
|
||||
walk_calls(child, scope_nid)
|
||||
|
||||
return {"nodes": nodes, "edges": edges}
|
||||
|
||||
|
||||
# ── Go extractor (custom walk) ────────────────────────────────────────────────
|
||||
|
||||
def extract_go(path: Path) -> dict:
|
||||
@@ -3682,8 +3955,18 @@ _DISPATCH: dict[str, Any] = {
|
||||
".m": extract_objc,
|
||||
".mm": extract_objc,
|
||||
".jl": extract_julia,
|
||||
".f": extract_fortran,
|
||||
".F": extract_fortran,
|
||||
".f90": extract_fortran,
|
||||
".F90": extract_fortran,
|
||||
".f95": extract_fortran,
|
||||
".F95": extract_fortran,
|
||||
".f03": extract_fortran,
|
||||
".F03": extract_fortran,
|
||||
".f08": extract_fortran,
|
||||
".F08": extract_fortran,
|
||||
".vue": extract_js,
|
||||
".svelte": extract_js,
|
||||
".svelte": extract_svelte,
|
||||
".dart": extract_dart,
|
||||
".v": extract_verilog,
|
||||
".sv": extract_verilog,
|
||||
@@ -3713,6 +3996,7 @@ def _extract_single_file(args: tuple) -> tuple[int, dict]:
|
||||
idx, path_str, cache_root_str = args
|
||||
path = Path(path_str)
|
||||
cache_root = Path(cache_root_str)
|
||||
_raise_recursion_limit()
|
||||
|
||||
# Check cache first (avoid re-extraction)
|
||||
cached = load_cached(path, cache_root)
|
||||
@@ -3723,7 +4007,7 @@ def _extract_single_file(args: tuple) -> tuple[int, dict]:
|
||||
if extractor is None:
|
||||
return idx, {"nodes": [], "edges": []}
|
||||
|
||||
result = extractor(path)
|
||||
result = _safe_extract(extractor, path)
|
||||
if "error" not in result:
|
||||
save_cached(path, result, cache_root)
|
||||
return idx, result
|
||||
@@ -3793,7 +4077,7 @@ def _extract_sequential(
|
||||
if extractor is None:
|
||||
per_file[idx] = {"nodes": [], "edges": []}
|
||||
continue
|
||||
result = extractor(path)
|
||||
result = _safe_extract(extractor, path)
|
||||
if "error" not in result:
|
||||
save_cached(path, result, effective_root)
|
||||
per_file[idx] = result
|
||||
@@ -3828,6 +4112,7 @@ def extract(
|
||||
max_workers: max subprocess count. Defaults to min(cpu_count, 8).
|
||||
"""
|
||||
_check_tree_sitter_version()
|
||||
_raise_recursion_limit()
|
||||
|
||||
# Infer a common root for cache keys (use first diverging segment, not sum of all matches)
|
||||
try:
|
||||
|
||||
+22
-419
@@ -43,18 +43,7 @@ Turn any folder of files into a navigable knowledge graph with community detecti
|
||||
|
||||
## What graphify is for
|
||||
|
||||
graphify is built around Andrej Karpathy's /raw folder workflow: drop anything into a folder - papers, tweets, screenshots, code, notes - and get a structured knowledge graph that shows you what you didn't know was connected.
|
||||
|
||||
Three things it does that Claude alone cannot:
|
||||
1. **Persistent graph** - relationships are stored in `graphify-out/graph.json` and survive across sessions. Ask questions weeks later without re-reading everything.
|
||||
2. **Honest audit trail** - every edge is tagged EXTRACTED, INFERRED, or AMBIGUOUS. You know what was found vs invented.
|
||||
3. **Cross-document surprise** - community detection finds connections between concepts in different files that you would never think to ask about directly.
|
||||
|
||||
Use it for:
|
||||
- A codebase you're new to (understand architecture before touching anything)
|
||||
- A reading list (papers + tweets + notes → one navigable graph)
|
||||
- A research corpus (citation graph + concept graph in one)
|
||||
- Your personal /raw folder (drop everything in, let it grow, query it)
|
||||
Drop any folder of code, docs, papers, images, or video into graphify and get a queryable knowledge graph. Persistent across sessions, honest audit trail (EXTRACTED/INFERRED/AMBIGUOUS), community detection surfaces cross-document connections you wouldn't think to ask about.
|
||||
|
||||
## What You Must Do When Invoked
|
||||
|
||||
@@ -566,84 +555,18 @@ Replace INPUT_PATH with the actual path.
|
||||
|
||||
If `--obsidian` was given:
|
||||
|
||||
- If `--obsidian-dir <path>` was also given, use that path as the vault directory. Otherwise default to `graphify-out/obsidian`.
|
||||
- If `--obsidian-dir <path>` was also given, pass it via `--dir`. Otherwise defaults to `graphify-out/obsidian`.
|
||||
|
||||
```bash
|
||||
$(cat graphify-out/.graphify_python) -c "
|
||||
import sys, json
|
||||
from graphify.build import build_from_json
|
||||
from graphify.export import to_obsidian, to_canvas
|
||||
from pathlib import Path
|
||||
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text())
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text())
|
||||
labels_raw = json.loads(Path('graphify-out/.graphify_labels.json').read_text()) if Path('graphify-out/.graphify_labels.json').exists() else {}
|
||||
|
||||
G = build_from_json(extraction)
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
labels = {int(k): v for k, v in labels_raw.items()}
|
||||
|
||||
obsidian_dir = 'OBSIDIAN_DIR' # replace with --obsidian-dir value, or 'graphify-out/obsidian' if not given
|
||||
|
||||
n = to_obsidian(G, communities, obsidian_dir, community_labels=labels or None, cohesion=cohesion)
|
||||
print(f'Obsidian vault: {n} notes in {obsidian_dir}/')
|
||||
|
||||
to_canvas(G, communities, f'{obsidian_dir}/graph.canvas', community_labels=labels or None)
|
||||
print(f'Canvas: {obsidian_dir}/graph.canvas - open in Obsidian for structured community layout')
|
||||
print()
|
||||
print(f'Open {obsidian_dir}/ as a vault in Obsidian.')
|
||||
print(' Graph view - nodes colored by community (set automatically)')
|
||||
print(' graph.canvas - structured layout with communities as groups')
|
||||
print(' _COMMUNITY_* - overview notes with cohesion scores and dataview queries')
|
||||
"
|
||||
graphify export obsidian
|
||||
# or with custom dir: graphify export obsidian --dir ~/vaults/my-project
|
||||
```
|
||||
|
||||
Generate the HTML graph (always, unless `--no-viz`):
|
||||
|
||||
```bash
|
||||
$(cat graphify-out/.graphify_python) -c "
|
||||
import sys, json
|
||||
from graphify.build import build_from_json
|
||||
from graphify.export import to_html
|
||||
from pathlib import Path
|
||||
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text())
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text())
|
||||
labels_raw = json.loads(Path('graphify-out/.graphify_labels.json').read_text()) if Path('graphify-out/.graphify_labels.json').exists() else {}
|
||||
|
||||
G = build_from_json(extraction)
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
labels = {int(k): v for k, v in labels_raw.items()}
|
||||
|
||||
NODE_LIMIT = 5000
|
||||
if G.number_of_nodes() > NODE_LIMIT:
|
||||
from collections import Counter
|
||||
print(f'Graph has {G.number_of_nodes()} nodes (above {NODE_LIMIT} limit). Building aggregated community view...')
|
||||
node_to_community = {nid: cid for cid, members in communities.items() for nid in members}
|
||||
import networkx as nx_meta
|
||||
meta = nx_meta.Graph()
|
||||
for cid, members in communities.items():
|
||||
meta.add_node(str(cid), label=labels.get(cid, f'Community {cid}'))
|
||||
edge_counts = Counter()
|
||||
for u, v in G.edges():
|
||||
cu, cv = node_to_community.get(u), node_to_community.get(v)
|
||||
if cu is not None and cv is not None and cu != cv:
|
||||
edge_counts[(min(cu, cv), max(cu, cv))] += 1
|
||||
for (cu, cv), w in edge_counts.items():
|
||||
meta.add_edge(str(cu), str(cv), weight=w, relation=f'{w} cross-community edges', confidence='AGGREGATED')
|
||||
if meta.number_of_nodes() > 1:
|
||||
meta_communities = {cid: [str(cid)] for cid in communities}
|
||||
member_counts = {cid: len(members) for cid, members in communities.items()}
|
||||
to_html(meta, meta_communities, 'graphify-out/graph.html', community_labels=labels or None, member_counts=member_counts)
|
||||
print(f'graph.html written (aggregated: {meta.number_of_nodes()} community nodes, {meta.number_of_edges()} cross-community edges)')
|
||||
print('Tip: run with --obsidian for full node-level detail.')
|
||||
else:
|
||||
print('Single community — aggregated view not useful. Skipping graph.html.')
|
||||
else:
|
||||
to_html(G, communities, 'graphify-out/graph.html', community_labels=labels or None)
|
||||
print('graph.html written - open in any browser, no server needed')
|
||||
"
|
||||
graphify export html # auto-aggregates to community view if graph > 5000 nodes
|
||||
# or: graphify export html --no-viz
|
||||
```
|
||||
|
||||
### Step 6b - Wiki (only if --wiki flag)
|
||||
@@ -653,27 +576,7 @@ else:
|
||||
Run this before Step 9 (cleanup) so `.graphify_labels.json` is still available.
|
||||
|
||||
```bash
|
||||
$(cat graphify-out/.graphify_python) -c "
|
||||
import json
|
||||
from graphify.build import build_from_json
|
||||
from graphify.wiki import to_wiki
|
||||
from graphify.analyze import god_nodes
|
||||
from pathlib import Path
|
||||
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text())
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text())
|
||||
labels_raw = json.loads(Path('graphify-out/.graphify_labels.json').read_text()) if Path('graphify-out/.graphify_labels.json').exists() else {}
|
||||
|
||||
G = build_from_json(extraction)
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
labels = {int(k): v for k, v in labels_raw.items()}
|
||||
gods = god_nodes(G)
|
||||
|
||||
n = to_wiki(G, communities, 'graphify-out/wiki', community_labels=labels or None, cohesion=cohesion, god_nodes_data=gods)
|
||||
print(f'Wiki: {n} articles written to graphify-out/wiki/')
|
||||
print(' graphify-out/wiki/index.md -> agent entry point')
|
||||
"
|
||||
graphify export wiki
|
||||
```
|
||||
|
||||
### Step 7 - Neo4j export (only if --neo4j or --neo4j-push flag)
|
||||
@@ -681,80 +584,27 @@ print(' graphify-out/wiki/index.md -> agent entry point')
|
||||
**If `--neo4j`** - generate a Cypher file for manual import:
|
||||
|
||||
```bash
|
||||
$(cat graphify-out/.graphify_python) -c "
|
||||
import sys, json
|
||||
from graphify.build import build_from_json
|
||||
from graphify.export import to_cypher
|
||||
from pathlib import Path
|
||||
|
||||
G = build_from_json(json.loads(Path('graphify-out/.graphify_extract.json').read_text()))
|
||||
to_cypher(G, 'graphify-out/cypher.txt')
|
||||
print('cypher.txt written - import with: cypher-shell < graphify-out/cypher.txt')
|
||||
"
|
||||
graphify export neo4j
|
||||
```
|
||||
|
||||
**If `--neo4j-push <uri>`** - push directly to a running Neo4j instance. Ask the user for credentials if not provided:
|
||||
|
||||
```bash
|
||||
$(cat graphify-out/.graphify_python) -c "
|
||||
import sys, json
|
||||
from graphify.build import build_from_json
|
||||
from graphify.cluster import cluster
|
||||
from graphify.export import push_to_neo4j
|
||||
from pathlib import Path
|
||||
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text())
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text())
|
||||
G = build_from_json(extraction)
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
|
||||
result = push_to_neo4j(G, uri='NEO4J_URI', user='NEO4J_USER', password='NEO4J_PASSWORD', communities=communities)
|
||||
print(f'Pushed to Neo4j: {result[\"nodes\"]} nodes, {result[\"edges\"]} edges')
|
||||
"
|
||||
graphify export neo4j --push bolt://localhost:7687 --user neo4j --password PASSWORD
|
||||
```
|
||||
|
||||
Replace `NEO4J_URI`, `NEO4J_USER`, `NEO4J_PASSWORD` with actual values. Default URI is `bolt://localhost:7687`, default user is `neo4j`. Uses MERGE - safe to re-run without creating duplicates.
|
||||
Default URI is `bolt://localhost:7687`, default user is `neo4j`. Uses MERGE - safe to re-run without creating duplicates.
|
||||
|
||||
### Step 7b - SVG export (only if --svg flag)
|
||||
|
||||
```bash
|
||||
$(cat graphify-out/.graphify_python) -c "
|
||||
import sys, json
|
||||
from graphify.build import build_from_json
|
||||
from graphify.export import to_svg
|
||||
from pathlib import Path
|
||||
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text())
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text())
|
||||
labels_raw = json.loads(Path('graphify-out/.graphify_labels.json').read_text()) if Path('graphify-out/.graphify_labels.json').exists() else {}
|
||||
|
||||
G = build_from_json(extraction)
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
labels = {int(k): v for k, v in labels_raw.items()}
|
||||
|
||||
to_svg(G, communities, 'graphify-out/graph.svg', community_labels=labels or None)
|
||||
print('graph.svg written - embeds in Obsidian, Notion, GitHub READMEs')
|
||||
"
|
||||
graphify export svg
|
||||
```
|
||||
|
||||
### Step 7c - GraphML export (only if --graphml flag)
|
||||
|
||||
```bash
|
||||
$(cat graphify-out/.graphify_python) -c "
|
||||
import json
|
||||
from graphify.build import build_from_json
|
||||
from graphify.export import to_graphml
|
||||
from pathlib import Path
|
||||
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text())
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text())
|
||||
|
||||
G = build_from_json(extraction)
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
|
||||
to_graphml(G, communities, 'graphify-out/graph.graphml')
|
||||
print('graph.graphml written - open in Gephi, yEd, or any GraphML tool')
|
||||
"
|
||||
graphify export graphml
|
||||
```
|
||||
|
||||
### Step 7d - MCP server (only if --mcp flag)
|
||||
@@ -782,15 +632,7 @@ To configure in Claude Desktop, add to `claude_desktop_config.json`:
|
||||
If `total_words` from `graphify-out/.graphify_detect.json` is greater than 5,000, run:
|
||||
|
||||
```bash
|
||||
$(cat graphify-out/.graphify_python) -c "
|
||||
import json
|
||||
from graphify.benchmark import run_benchmark, print_benchmark
|
||||
from pathlib import Path
|
||||
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text())
|
||||
result = run_benchmark('graphify-out/graph.json', corpus_words=detection['total_words'])
|
||||
print_benchmark(result)
|
||||
"
|
||||
graphify benchmark
|
||||
```
|
||||
|
||||
Print the output directly in chat. If `total_words <= 5000`, skip silently - the graph value is structural clarity, not token compression, for small corpora.
|
||||
@@ -916,7 +758,7 @@ import json
|
||||
from pathlib import Path
|
||||
|
||||
result = json.loads(open('graphify-out/.graphify_incremental.json').read()) if Path('graphify-out/.graphify_incremental.json').exists() else {}
|
||||
code_exts = {'.py','.ts','.js','.go','.rs','.java','.cpp','.c','.rb','.swift','.kt','.cs','.scala','.php','.cc','.cxx','.hpp','.h','.kts','.lua','.toc'}
|
||||
code_exts = {'.py','.ts','.js','.go','.rs','.java','.cpp','.c','.rb','.swift','.kt','.cs','.scala','.php','.cc','.cxx','.hpp','.h','.kts','.lua','.toc','.f','.F','.f90','.F90','.f95','.F95','.f03','.F03','.f08','.F08'}
|
||||
new_files = result.get('new_files', {})
|
||||
all_changed = [f for files in new_files.values() for f in files]
|
||||
code_only = all(Path(f).suffix.lower() in code_exts for f in all_changed)
|
||||
@@ -1019,45 +861,10 @@ Clean up after: `rm -f graphify-out/.graphify_old.json`
|
||||
|
||||
## For --cluster-only
|
||||
|
||||
Skip Steps 1–3. Load the existing graph from `graphify-out/graph.json` and re-run clustering:
|
||||
Skip Steps 1–3. Re-run clustering on the existing graph:
|
||||
|
||||
```bash
|
||||
$(cat graphify-out/.graphify_python) -c "
|
||||
import sys, json
|
||||
from graphify.cluster import cluster, score_all
|
||||
from graphify.analyze import god_nodes, surprising_connections
|
||||
from graphify.report import generate
|
||||
from graphify.export import to_json
|
||||
from networkx.readwrite import json_graph
|
||||
import networkx as nx
|
||||
from pathlib import Path
|
||||
|
||||
data = json.loads(Path('graphify-out/graph.json').read_text())
|
||||
G = json_graph.node_link_graph(data, edges='links')
|
||||
|
||||
detection = {'total_files': 0, 'total_words': 99999, 'needs_graph': True, 'warning': None,
|
||||
'files': {'code': [], 'document': [], 'paper': []}}
|
||||
tokens = {'input': 0, 'output': 0}
|
||||
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
gods = god_nodes(G)
|
||||
surprises = surprising_connections(G, communities)
|
||||
labels = {cid: 'Community ' + str(cid) for cid in communities}
|
||||
|
||||
report = generate(G, communities, cohesion, labels, gods, surprises, detection, tokens, '.')
|
||||
Path('graphify-out/GRAPH_REPORT.md').write_text(report)
|
||||
to_json(G, communities, 'graphify-out/graph.json')
|
||||
|
||||
analysis = {
|
||||
'communities': {str(k): v for k, v in communities.items()},
|
||||
'cohesion': {str(k): v for k, v in cohesion.items()},
|
||||
'gods': gods,
|
||||
'surprises': surprises,
|
||||
}
|
||||
Path('graphify-out/.graphify_analysis.json').write_text(json.dumps(analysis, indent=2))
|
||||
print(f'Re-clustered: {len(communities)} communities')
|
||||
"
|
||||
graphify cluster-only .
|
||||
```
|
||||
|
||||
Then run Steps 5–9 as normal (label communities, generate viz, benchmark, clean up, report).
|
||||
@@ -1073,113 +880,12 @@ Two traversal modes - choose based on the question:
|
||||
| BFS (default) | _(none)_ | "What is X connected to?" - broad context, nearest neighbors first |
|
||||
| DFS | `--dfs` | "How does X reach Y?" - trace a specific chain or dependency path |
|
||||
|
||||
First check the graph exists:
|
||||
```bash
|
||||
$(cat graphify-out/.graphify_python) -c "
|
||||
from pathlib import Path
|
||||
if not Path('graphify-out/graph.json').exists():
|
||||
print('ERROR: No graph found. Run /graphify <path> first to build the graph.')
|
||||
raise SystemExit(1)
|
||||
"
|
||||
```
|
||||
If it fails, stop and tell the user to run `/graphify <path>` first.
|
||||
|
||||
Load `graphify-out/graph.json`, then:
|
||||
|
||||
1. Find the 1-3 nodes whose label best matches key terms in the question.
|
||||
2. Run the appropriate traversal from each starting node.
|
||||
3. Read the subgraph - node labels, edge relations, confidence tags, source locations.
|
||||
4. Answer using **only** what the graph contains. Quote `source_location` when citing a specific fact.
|
||||
5. If the graph lacks enough information, say so - do not hallucinate edges.
|
||||
|
||||
```bash
|
||||
$(cat graphify-out/.graphify_python) -c "
|
||||
import sys, json
|
||||
from networkx.readwrite import json_graph
|
||||
import networkx as nx
|
||||
from pathlib import Path
|
||||
|
||||
data = json.loads(Path('graphify-out/graph.json').read_text())
|
||||
G = json_graph.node_link_graph(data, edges='links')
|
||||
|
||||
question = 'QUESTION'
|
||||
mode = 'MODE' # 'bfs' or 'dfs'
|
||||
terms = [t.lower() for t in question.split() if len(t) > 3]
|
||||
|
||||
# Find best-matching start nodes
|
||||
scored = []
|
||||
for nid, ndata in G.nodes(data=True):
|
||||
label = ndata.get('label', '').lower()
|
||||
score = sum(1 for t in terms if t in label)
|
||||
if score > 0:
|
||||
scored.append((score, nid))
|
||||
scored.sort(reverse=True)
|
||||
start_nodes = [nid for _, nid in scored[:3]]
|
||||
|
||||
if not start_nodes:
|
||||
print('No matching nodes found for query terms:', terms)
|
||||
sys.exit(0)
|
||||
|
||||
subgraph_nodes = set()
|
||||
subgraph_edges = []
|
||||
|
||||
if mode == 'dfs':
|
||||
# DFS: follow one path as deep as possible before backtracking.
|
||||
# Depth-limited to 6 to avoid traversing the whole graph.
|
||||
visited = set()
|
||||
stack = [(n, 0) for n in reversed(start_nodes)]
|
||||
while stack:
|
||||
node, depth = stack.pop()
|
||||
if node in visited or depth > 6:
|
||||
continue
|
||||
visited.add(node)
|
||||
subgraph_nodes.add(node)
|
||||
for neighbor in G.neighbors(node):
|
||||
if neighbor not in visited:
|
||||
stack.append((neighbor, depth + 1))
|
||||
subgraph_edges.append((node, neighbor))
|
||||
else:
|
||||
# BFS: explore all neighbors layer by layer up to depth 3.
|
||||
frontier = set(start_nodes)
|
||||
subgraph_nodes = set(start_nodes)
|
||||
for _ in range(3):
|
||||
next_frontier = set()
|
||||
for n in frontier:
|
||||
for neighbor in G.neighbors(n):
|
||||
if neighbor not in subgraph_nodes:
|
||||
next_frontier.add(neighbor)
|
||||
subgraph_edges.append((n, neighbor))
|
||||
subgraph_nodes.update(next_frontier)
|
||||
frontier = next_frontier
|
||||
|
||||
# Token-budget aware output: rank by relevance, cut at budget (~4 chars/token)
|
||||
token_budget = BUDGET # default 2000
|
||||
char_budget = token_budget * 4
|
||||
|
||||
# Score each node by term overlap for ranked output
|
||||
def relevance(nid):
|
||||
label = G.nodes[nid].get('label', '').lower()
|
||||
return sum(1 for t in terms if t in label)
|
||||
|
||||
ranked_nodes = sorted(subgraph_nodes, key=relevance, reverse=True)
|
||||
|
||||
lines = [f'Traversal: {mode.upper()} | Start: {[G.nodes[n].get(\"label\",n) for n in start_nodes]} | {len(subgraph_nodes)} nodes']
|
||||
for nid in ranked_nodes:
|
||||
d = G.nodes[nid]
|
||||
lines.append(f' NODE {d.get(\"label\", nid)} [src={d.get(\"source_file\",\"\")} loc={d.get(\"source_location\",\"\")}]')
|
||||
for u, v in subgraph_edges:
|
||||
if u in subgraph_nodes and v in subgraph_nodes:
|
||||
d = G.edges[u, v]
|
||||
lines.append(f' EDGE {G.nodes[u].get(\"label\",u)} --{d.get(\"relation\",\"\")} [{d.get(\"confidence\",\"\")}]--> {G.nodes[v].get(\"label\",v)}')
|
||||
|
||||
output = '\n'.join(lines)
|
||||
if len(output) > char_budget:
|
||||
output = output[:char_budget] + f'\n... (truncated at ~{token_budget} token budget - use --budget N for more)'
|
||||
print(output)
|
||||
"
|
||||
graphify query "QUESTION"
|
||||
# or: graphify query "QUESTION" --dfs --budget 3000
|
||||
```
|
||||
|
||||
Replace `QUESTION` with the user's actual question, `MODE` with `bfs` or `dfs`, and `BUDGET` with the token budget (default `2000`, or whatever `--budget N` specifies). Then answer based on the subgraph output above.
|
||||
Replace `QUESTION` with the user's actual question. Answer using **only** what the graph output contains. Quote `source_location` when citing a specific fact. If the graph lacks enough information, say so - do not hallucinate edges.
|
||||
|
||||
After writing the answer, save it back into the graph so it improves future queries:
|
||||
|
||||
@@ -1195,66 +901,11 @@ Replace `QUESTION` with the question, `ANSWER` with your full answer text, `SOUR
|
||||
|
||||
Find the shortest path between two named concepts in the graph.
|
||||
|
||||
First check the graph exists:
|
||||
```bash
|
||||
$(cat graphify-out/.graphify_python) -c "
|
||||
from pathlib import Path
|
||||
if not Path('graphify-out/graph.json').exists():
|
||||
print('ERROR: No graph found. Run /graphify <path> first to build the graph.')
|
||||
raise SystemExit(1)
|
||||
"
|
||||
```
|
||||
If it fails, stop and tell the user to run `/graphify <path>` first.
|
||||
|
||||
```bash
|
||||
$(cat graphify-out/.graphify_python) -c "
|
||||
import json, sys
|
||||
import networkx as nx
|
||||
from networkx.readwrite import json_graph
|
||||
from pathlib import Path
|
||||
|
||||
data = json.loads(Path('graphify-out/graph.json').read_text())
|
||||
G = json_graph.node_link_graph(data, edges='links')
|
||||
|
||||
a_term = 'NODE_A'
|
||||
b_term = 'NODE_B'
|
||||
|
||||
def find_node(term):
|
||||
term = term.lower()
|
||||
scored = sorted(
|
||||
[(sum(1 for w in term.split() if w in G.nodes[n].get('label','').lower()), n)
|
||||
for n in G.nodes()],
|
||||
reverse=True
|
||||
)
|
||||
return scored[0][1] if scored and scored[0][0] > 0 else None
|
||||
|
||||
src = find_node(a_term)
|
||||
tgt = find_node(b_term)
|
||||
|
||||
if not src or not tgt:
|
||||
print(f'Could not find nodes matching: {a_term!r} or {b_term!r}')
|
||||
sys.exit(0)
|
||||
|
||||
try:
|
||||
path = nx.shortest_path(G, src, tgt)
|
||||
print(f'Shortest path ({len(path)-1} hops):')
|
||||
for i, nid in enumerate(path):
|
||||
label = G.nodes[nid].get('label', nid)
|
||||
if i < len(path) - 1:
|
||||
edge = G.edges[nid, path[i+1]]
|
||||
rel = edge.get('relation', '')
|
||||
conf = edge.get('confidence', '')
|
||||
print(f' {label} --{rel}--> [{conf}]')
|
||||
else:
|
||||
print(f' {label}')
|
||||
except nx.NetworkXNoPath:
|
||||
print(f'No path found between {a_term!r} and {b_term!r}')
|
||||
except nx.NodeNotFound as e:
|
||||
print(f'Node not found: {e}')
|
||||
"
|
||||
graphify path "NODE_A" "NODE_B"
|
||||
```
|
||||
|
||||
Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then explain the path in plain language - what each hop means, why it's significant.
|
||||
Replace `NODE_A` and `NODE_B` with the actual concept names. Then explain the path in plain language - what each hop means, why it's significant.
|
||||
|
||||
After writing the explanation, save it back:
|
||||
|
||||
@@ -1268,56 +919,8 @@ $(cat graphify-out/.graphify_python) -m graphify save-result --question "Path fr
|
||||
|
||||
Give a plain-language explanation of a single node - everything connected to it.
|
||||
|
||||
First check the graph exists:
|
||||
```bash
|
||||
$(cat graphify-out/.graphify_python) -c "
|
||||
from pathlib import Path
|
||||
if not Path('graphify-out/graph.json').exists():
|
||||
print('ERROR: No graph found. Run /graphify <path> first to build the graph.')
|
||||
raise SystemExit(1)
|
||||
"
|
||||
```
|
||||
If it fails, stop and tell the user to run `/graphify <path>` first.
|
||||
|
||||
```bash
|
||||
$(cat graphify-out/.graphify_python) -c "
|
||||
import json, sys
|
||||
import networkx as nx
|
||||
from networkx.readwrite import json_graph
|
||||
from pathlib import Path
|
||||
|
||||
data = json.loads(Path('graphify-out/graph.json').read_text())
|
||||
G = json_graph.node_link_graph(data, edges='links')
|
||||
|
||||
term = 'NODE_NAME'
|
||||
term_lower = term.lower()
|
||||
|
||||
# Find best matching node
|
||||
scored = sorted(
|
||||
[(sum(1 for w in term_lower.split() if w in G.nodes[n].get('label','').lower()), n)
|
||||
for n in G.nodes()],
|
||||
reverse=True
|
||||
)
|
||||
if not scored or scored[0][0] == 0:
|
||||
print(f'No node matching {term!r}')
|
||||
sys.exit(0)
|
||||
|
||||
nid = scored[0][1]
|
||||
data_n = G.nodes[nid]
|
||||
print(f'NODE: {data_n.get(\"label\", nid)}')
|
||||
print(f' source: {data_n.get(\"source_file\",\"unknown\")}')
|
||||
print(f' type: {data_n.get(\"file_type\",\"unknown\")}')
|
||||
print(f' degree: {G.degree(nid)}')
|
||||
print()
|
||||
print('CONNECTIONS:')
|
||||
for neighbor in G.neighbors(nid):
|
||||
edge = G.edges[nid, neighbor]
|
||||
nlabel = G.nodes[neighbor].get('label', neighbor)
|
||||
rel = edge.get('relation', '')
|
||||
conf = edge.get('confidence', '')
|
||||
src_file = G.nodes[neighbor].get('source_file', '')
|
||||
print(f' --{rel}--> {nlabel} [{conf}] ({src_file})')
|
||||
"
|
||||
graphify explain "NODE_NAME"
|
||||
```
|
||||
|
||||
Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sentence explanation of what this node is, what it connects to, and why those connections are significant. Use the source locations as citations.
|
||||
|
||||
+2
-1
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "graphifyy"
|
||||
version = "0.6.9"
|
||||
version = "0.7.2"
|
||||
description = "AI coding assistant skill (Claude Code, Codex, OpenCode, Cursor, Gemini CLI, Aider, OpenClaw, Factory Droid, Trae, Hermes, Kiro, Pi, Google Antigravity) - turn any folder of code, docs, papers, images, or videos into a queryable knowledge graph"
|
||||
readme = "README.md"
|
||||
license = { file = "LICENSE" }
|
||||
@@ -34,6 +34,7 @@ dependencies = [
|
||||
"tree-sitter-objc",
|
||||
"tree-sitter-julia",
|
||||
"tree-sitter-verilog",
|
||||
"tree-sitter-fortran",
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
|
||||
Vendored
+17
@@ -0,0 +1,17 @@
|
||||
#define NDIM 3
|
||||
|
||||
module shapes
|
||||
#ifdef MPI
|
||||
use mpi
|
||||
#endif
|
||||
implicit none
|
||||
|
||||
contains
|
||||
|
||||
subroutine compute_volume(side, vol)
|
||||
real, intent(in) :: side
|
||||
real, intent(out) :: vol
|
||||
vol = side ** NDIM
|
||||
end subroutine compute_volume
|
||||
|
||||
end module shapes
|
||||
Vendored
+39
@@ -0,0 +1,39 @@
|
||||
module geometry
|
||||
use constants
|
||||
implicit none
|
||||
|
||||
real, parameter :: PI = 3.14159
|
||||
|
||||
contains
|
||||
|
||||
subroutine circle_area(radius, area)
|
||||
real, intent(in) :: radius
|
||||
real, intent(out) :: area
|
||||
area = PI * radius * radius
|
||||
end subroutine circle_area
|
||||
|
||||
function distance(x1, y1, x2, y2) result(d)
|
||||
real, intent(in) :: x1, y1, x2, y2
|
||||
real :: d
|
||||
d = sqrt((x2 - x1)**2 + (y2 - y1)**2)
|
||||
end function distance
|
||||
|
||||
subroutine print_area(radius)
|
||||
real, intent(in) :: radius
|
||||
real :: area
|
||||
call circle_area(radius, area)
|
||||
print *, "Area =", area
|
||||
end subroutine print_area
|
||||
|
||||
end module geometry
|
||||
|
||||
|
||||
program main
|
||||
use geometry
|
||||
implicit none
|
||||
|
||||
real :: r, a
|
||||
r = 5.0
|
||||
call circle_area(r, a)
|
||||
print *, "Circle area:", a
|
||||
end program main
|
||||
@@ -0,0 +1,199 @@
|
||||
"""Integration tests for graphify export subcommands and CLI commands.
|
||||
|
||||
Each test builds a minimal graph in a temp dir, runs the CLI command as a subprocess,
|
||||
and asserts the expected output file exists and is non-empty / valid.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
PYTHON = sys.executable
|
||||
FIXTURES = Path(__file__).parent / "fixtures"
|
||||
|
||||
|
||||
def _run(args: list[str], cwd: Path) -> subprocess.CompletedProcess:
|
||||
return subprocess.run(
|
||||
[PYTHON, "-m", "graphify"] + args,
|
||||
cwd=cwd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
|
||||
|
||||
def _make_graph(tmp_path: Path) -> Path:
|
||||
"""Build a minimal graph.json + analysis/labels files in tmp_path/graphify-out/."""
|
||||
out = tmp_path / "graphify-out"
|
||||
out.mkdir()
|
||||
|
||||
extraction = json.loads((FIXTURES / "extraction.json").read_text())
|
||||
from graphify.build import build_from_json
|
||||
from graphify.cluster import cluster, score_all
|
||||
from graphify.analyze import god_nodes, surprising_connections
|
||||
from graphify.export import to_json
|
||||
|
||||
G = build_from_json(extraction)
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
gods = god_nodes(G)
|
||||
surprises = surprising_connections(G, communities)
|
||||
labels = {cid: f"Community {cid}" for cid in communities}
|
||||
|
||||
to_json(G, communities, str(out / "graph.json"))
|
||||
|
||||
analysis = {
|
||||
"communities": {str(k): v for k, v in communities.items()},
|
||||
"cohesion": {str(k): v for k, v in cohesion.items()},
|
||||
"gods": gods,
|
||||
"surprises": surprises,
|
||||
}
|
||||
(out / ".graphify_analysis.json").write_text(json.dumps(analysis))
|
||||
(out / ".graphify_labels.json").write_text(
|
||||
json.dumps({str(k): v for k, v in labels.items()})
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
# ── graphify export html ─────────────────────────────────────────────────────
|
||||
|
||||
def test_export_html_creates_file(tmp_path):
|
||||
_make_graph(tmp_path)
|
||||
r = _run(["export", "html"], tmp_path)
|
||||
assert r.returncode == 0, r.stderr
|
||||
html = tmp_path / "graphify-out" / "graph.html"
|
||||
assert html.exists()
|
||||
assert html.stat().st_size > 0
|
||||
|
||||
|
||||
def test_export_html_no_viz_removes_file(tmp_path):
|
||||
out = _make_graph(tmp_path)
|
||||
(out / "graph.html").write_text("<html/>")
|
||||
r = _run(["export", "html", "--no-viz"], tmp_path)
|
||||
assert r.returncode == 0, r.stderr
|
||||
assert not (out / "graph.html").exists()
|
||||
|
||||
|
||||
def test_export_html_error_without_graph(tmp_path):
|
||||
r = _run(["export", "html"], tmp_path)
|
||||
assert r.returncode != 0
|
||||
|
||||
|
||||
# ── graphify export obsidian ─────────────────────────────────────────────────
|
||||
|
||||
def test_export_obsidian_creates_vault(tmp_path):
|
||||
_make_graph(tmp_path)
|
||||
r = _run(["export", "obsidian"], tmp_path)
|
||||
assert r.returncode == 0, r.stderr
|
||||
vault = tmp_path / "graphify-out" / "obsidian"
|
||||
assert vault.exists()
|
||||
md_files = list(vault.glob("*.md"))
|
||||
assert len(md_files) > 0
|
||||
|
||||
|
||||
def test_export_obsidian_custom_dir(tmp_path):
|
||||
_make_graph(tmp_path)
|
||||
custom = tmp_path / "my-vault"
|
||||
r = _run(["export", "obsidian", "--dir", str(custom)], tmp_path)
|
||||
assert r.returncode == 0, r.stderr
|
||||
assert custom.exists()
|
||||
assert len(list(custom.glob("*.md"))) > 0
|
||||
|
||||
|
||||
# ── graphify export wiki ─────────────────────────────────────────────────────
|
||||
|
||||
def test_export_wiki_creates_articles(tmp_path):
|
||||
_make_graph(tmp_path)
|
||||
r = _run(["export", "wiki"], tmp_path)
|
||||
assert r.returncode == 0, r.stderr
|
||||
wiki = tmp_path / "graphify-out" / "wiki"
|
||||
assert wiki.exists()
|
||||
assert (wiki / "index.md").exists()
|
||||
|
||||
|
||||
# ── graphify export graphml ──────────────────────────────────────────────────
|
||||
|
||||
def test_export_graphml_creates_file(tmp_path):
|
||||
_make_graph(tmp_path)
|
||||
r = _run(["export", "graphml"], tmp_path)
|
||||
assert r.returncode == 0, r.stderr
|
||||
gml = tmp_path / "graphify-out" / "graph.graphml"
|
||||
assert gml.exists()
|
||||
assert gml.stat().st_size > 0
|
||||
content = gml.read_text()
|
||||
assert "<graphml" in content
|
||||
|
||||
|
||||
# ── graphify export neo4j (cypher) ───────────────────────────────────────────
|
||||
|
||||
def test_export_neo4j_creates_cypher(tmp_path):
|
||||
_make_graph(tmp_path)
|
||||
r = _run(["export", "neo4j"], tmp_path)
|
||||
assert r.returncode == 0, r.stderr
|
||||
cypher = tmp_path / "graphify-out" / "cypher.txt"
|
||||
assert cypher.exists()
|
||||
assert cypher.stat().st_size > 0
|
||||
content = cypher.read_text()
|
||||
assert "MERGE" in content or "CREATE" in content
|
||||
|
||||
|
||||
# ── graphify query ───────────────────────────────────────────────────────────
|
||||
|
||||
def test_query_returns_output(tmp_path):
|
||||
_make_graph(tmp_path)
|
||||
r = _run(["query", "test"], tmp_path)
|
||||
assert r.returncode == 0, r.stderr
|
||||
assert len(r.stdout) > 0
|
||||
|
||||
|
||||
def test_query_dfs_flag(tmp_path):
|
||||
_make_graph(tmp_path)
|
||||
r = _run(["query", "test", "--dfs"], tmp_path)
|
||||
assert r.returncode == 0, r.stderr
|
||||
|
||||
|
||||
def test_query_budget_flag(tmp_path):
|
||||
_make_graph(tmp_path)
|
||||
r = _run(["query", "test", "--budget", "500"], tmp_path)
|
||||
assert r.returncode == 0, r.stderr
|
||||
|
||||
|
||||
def test_query_missing_graph_fails(tmp_path):
|
||||
r = _run(["query", "anything"], tmp_path)
|
||||
assert r.returncode != 0
|
||||
|
||||
|
||||
# ── graphify path ────────────────────────────────────────────────────────────
|
||||
|
||||
def test_path_runs_without_error(tmp_path):
|
||||
_make_graph(tmp_path)
|
||||
r = _run(["path", "Transformer", "LayerNorm"], tmp_path)
|
||||
# May find or not find a path — either is valid, should not crash
|
||||
assert r.returncode == 0, r.stderr
|
||||
|
||||
|
||||
def test_path_missing_graph_fails(tmp_path):
|
||||
r = _run(["path", "a", "b"], tmp_path)
|
||||
assert r.returncode != 0
|
||||
|
||||
|
||||
# ── graphify explain ─────────────────────────────────────────────────────────
|
||||
|
||||
def test_explain_runs_without_error(tmp_path):
|
||||
_make_graph(tmp_path)
|
||||
r = _run(["explain", "test"], tmp_path)
|
||||
assert r.returncode == 0, r.stderr
|
||||
|
||||
|
||||
def test_explain_missing_graph_fails(tmp_path):
|
||||
r = _run(["explain", "anything"], tmp_path)
|
||||
assert r.returncode != 0
|
||||
|
||||
|
||||
# ── graphify export unknown format ───────────────────────────────────────────
|
||||
|
||||
def test_export_unknown_format_fails(tmp_path):
|
||||
r = _run(["export", "pdf"], tmp_path)
|
||||
assert r.returncode != 0
|
||||
+71
-2
@@ -1,11 +1,11 @@
|
||||
"""Tests for language extractors: Java, C, C++, Ruby, C#, Kotlin, Scala, PHP, Swift, Go, Julia, JS/TS."""
|
||||
"""Tests for language extractors: Java, C, C++, Ruby, C#, Kotlin, Scala, PHP, Swift, Go, Julia, Fortran, JS/TS."""
|
||||
from __future__ import annotations
|
||||
from pathlib import Path
|
||||
import pytest
|
||||
from graphify.extract import (
|
||||
extract_java, extract_c, extract_cpp, extract_ruby,
|
||||
extract_csharp, extract_kotlin, extract_scala, extract_php,
|
||||
extract_swift, extract_go, extract_julia, extract_js,
|
||||
extract_swift, extract_go, extract_julia, extract_js, extract_fortran,
|
||||
)
|
||||
|
||||
FIXTURES = Path(__file__).parent / "fixtures"
|
||||
@@ -721,6 +721,75 @@ def test_julia_no_dangling_edges():
|
||||
assert e["source"] in node_ids, f"Dangling source: {e}"
|
||||
|
||||
|
||||
# ── Fortran extractor ────────────────────────────────────────────────────────
|
||||
|
||||
def test_fortran_finds_module():
|
||||
r = extract_fortran(FIXTURES / "sample.f90")
|
||||
assert "error" not in r
|
||||
labels = [n["label"] for n in r["nodes"]]
|
||||
assert "geometry" in labels
|
||||
|
||||
|
||||
def test_fortran_finds_subroutines():
|
||||
r = extract_fortran(FIXTURES / "sample.f90")
|
||||
labels = [n["label"] for n in r["nodes"]]
|
||||
assert any("circle_area" in l for l in labels)
|
||||
assert any("print_area" in l for l in labels)
|
||||
|
||||
|
||||
def test_fortran_finds_function():
|
||||
r = extract_fortran(FIXTURES / "sample.f90")
|
||||
labels = [n["label"] for n in r["nodes"]]
|
||||
assert any("distance" in l for l in labels)
|
||||
|
||||
|
||||
def test_fortran_finds_program():
|
||||
r = extract_fortran(FIXTURES / "sample.f90")
|
||||
labels = [n["label"] for n in r["nodes"]]
|
||||
assert "main" in labels
|
||||
|
||||
|
||||
def test_fortran_finds_use_imports():
|
||||
r = extract_fortran(FIXTURES / "sample.f90")
|
||||
import_edges = [e for e in r["edges"] if e["relation"] == "imports"]
|
||||
assert len(import_edges) >= 2
|
||||
|
||||
|
||||
def test_fortran_use_edges_have_use_context():
|
||||
r = extract_fortran(FIXTURES / "sample.f90")
|
||||
import_edges = [e for e in r["edges"] if e["relation"] == "imports"]
|
||||
assert all(e.get("context") == "use" for e in import_edges)
|
||||
|
||||
|
||||
def test_fortran_finds_calls():
|
||||
r = extract_fortran(FIXTURES / "sample.f90")
|
||||
call_edges = [e for e in r["edges"] if e["relation"] == "calls"]
|
||||
assert len(call_edges) >= 1
|
||||
|
||||
|
||||
def test_fortran_case_insensitive_names():
|
||||
r = extract_fortran(FIXTURES / "sample.f90")
|
||||
labels = [n["label"] for n in r["nodes"]]
|
||||
assert all(l == l.lower() or "(" in l for l in labels if l.endswith(("()", "")) and not "." in l)
|
||||
assert "geometry" in labels
|
||||
assert "main" in labels
|
||||
|
||||
|
||||
def test_fortran_no_dangling_edges():
|
||||
r = extract_fortran(FIXTURES / "sample.f90")
|
||||
node_ids = {n["id"] for n in r["nodes"]}
|
||||
for e in r["edges"]:
|
||||
assert e["source"] in node_ids, f"Dangling source: {e}"
|
||||
|
||||
|
||||
def test_fortran_capital_F_parses_preprocessed():
|
||||
r = extract_fortran(FIXTURES / "sample.F90")
|
||||
assert "error" not in r
|
||||
labels = [n["label"] for n in r["nodes"]]
|
||||
assert "shapes" in labels
|
||||
assert any("compute_volume" in l for l in labels)
|
||||
|
||||
|
||||
# ── TypeScript dynamic imports ───────────────────────────────────────────────
|
||||
|
||||
def test_ts_dynamic_import_no_error():
|
||||
|
||||
Reference in New Issue
Block a user