docs: refresh ARCHITECTURE.md module table + add a doc-parity test (keeps the #2558 'an' fix)

2026-08-26 16:26:42 +00:00 · 2026-08-12 20:57:39 +01:00
parent 39beeb9b4b
commit c1e41ea90e
2 changed files with 124 additions and 20 deletions
@@ -5,31 +5,48 @@ graphify is a Claude Code skill backed by a Python library. The skill orchestrat
 ## Pipeline

 ```
-detect()  →  extract()  →  build_graph()  →  cluster()  →  analyze()  →  report()  →  export()
+detect()  →  extract()  →  build()  →  cluster()  →  analyze helpers  →  report.generate()  →  export.to_*()
 ```

-Each stage is a single function in its own module. They communicate through plain Python dicts and NetworkX graphs - no shared state, no side effects outside `graphify-out/`.
+Each stage lives in its own module and they communicate through plain Python dicts and NetworkX graphs - no shared state, no side effects outside `graphify-out/`. Most stages are a single function; `analyze.py` and `export.py` are sets of sibling functions rather than one entry point.

 ## Module responsibilities

-| Module | Function | Input → Output |
-|--------|----------|----------------|
-| `detect.py` | `collect_files(root)` | directory → `[Path]` filtered list |
-| `extract.py` | `extract(path)` | file path → `{nodes, edges}` dict |
-| `build.py` | `build_graph(extractions)` | list of extraction dicts → `nx.Graph` |
-| `cluster.py` | `cluster(G)` | graph → graph with `community` attr on each node |
-| `analyze.py` | `analyze(G)` | graph → analysis dict (god nodes, surprises, questions) |
-| `report.py` | `render_report(G, analysis)` | graph + analysis → GRAPH_REPORT.md string |
-| `export.py` | `export(G, out_dir, ...)` | graph → Obsidian vault, graph.json, graph.html, graph.svg |
+Signatures below are the real ones - `tests/test_architecture_doc.py` imports every symbol named here, so this table cannot drift from the code.
+
+| Module | Entry point(s) | Input → Output |
+|--------|----------------|----------------|
+| `detect.py` | `detect(root)` | directory → scan summary dict: `files` grouped by category, plus `total_files`, `total_words`, `warning`, `scan_root`, … |
+| `extract.py` | `extract(paths, *, root=None, ...)`, `collect_files(target)` | **list** of file paths → `{nodes, edges}` dict. `collect_files` expands a directory into that list, and lives here, not in `detect.py` |
+| `build.py` | `build(extractions)`, `build_from_json(extraction)` | extraction dict(s) → `nx.Graph` |
+| `cluster.py` | `cluster(G)` | graph → `{community_id: [node_id, ...]}` (the graph is not mutated) |
+| `analyze.py` | `god_nodes(G)`, `surprising_connections(G)`, `suggest_questions(G, communities, community_labels)`, `find_import_cycles(G)`, `graph_diff(G_old, G_new)` | graph → one list/dict per analysis. There is no single `analyze()` entry point |
+| `report.py` | `generate(G, communities, cohesion_scores, community_labels, ...)` | graph + analysis → GRAPH_REPORT.md string |
+| `export.py` | `to_json`, `to_html`, `to_obsidian`, `to_svg`, `to_graphml`, `to_canvas`, `to_cypher` | graph → graph.json, graph.html, Obsidian vault, graph.svg, … one function per format |
+| `wiki.py` | `to_wiki(G, communities, output_dir, ...)` | graph → one markdown article per community + `index.md` |
 | `callflow_html.py` | `write_callflow_html(...)` | graphify-out files → Mermaid architecture/call-flow HTML |
-| `ingest.py` | `ingest(url, ...)` | URL → file saved to corpus dir |
-| `cache.py` | `check_semantic_cache / save_semantic_cache` | files → (cached, uncached) split |
-| `security.py` | validation helpers | URL / path / label → validated or raises |
-| `validate.py` | `validate_extraction(data)` | extraction dict → raises on schema errors |
-| `serve.py` | `start_server(graph_path)` | graph file path → MCP stdio server |
-| `watch.py` | `watch(root, flag_path)` | directory → writes flag file on change |
+| `ingest.py` | `ingest(url, target_dir, ...)` | URL → file saved to corpus dir |
+| `cache.py` | `check_semantic_cache(files, root)`, `save_semantic_cache(nodes, edges, ...)` | files → cached nodes / edges / hyperedges + the list of files still needing extraction |
+| `security.py` | `validate_url`, `safe_fetch`, `validate_graph_path`, `sanitize_label` | URL / path / label → validated value, or raises |
+| `validate.py` | `validate_extraction(data)`, `assert_valid(data)` | extraction dict → **list of schema error strings** (`validate_extraction` returns them; `assert_valid` raises) |
+| `serve.py` | `serve(graph_path)`, `serve_http(graph_path, *, host, port, ...)` | graph file path → MCP stdio server / HTTP server |
+| `watch.py` | `watch(watch_path, debounce=3.0)`, `check_update(watch_path)` | directory → rebuild on change; `check_update` reports whether a re-extraction is pending |
 | `benchmark.py` | `run_benchmark(graph_path)` | graph file → corpus vs subgraph token comparison |

+### Calling `extract()` from your own code
+
+`extract()` takes a **list** of paths, and `root` is keyword-only and optional:
+
+```python
+from pathlib import Path
+from graphify.extract import extract
+
+paths = [Path("src/lib/content.ts"), Path("src/pages/index.astro")]
+result = extract(paths, root=Path(".").resolve())   # pass root explicitly
+```
+
+Always pass `root`. Node ids and `source_file` values are derived relative to it; when it is omitted, `extract()` infers one from the paths you passed, which is the common parent of *that list* rather than your project root. A single-file call therefore anchors to that file's own directory, and ids can end up carrying path segments from the machine they were extracted on.
+
 ## Extraction output schema

 Every extractor returns:
@@ -45,7 +62,7 @@ Every extractor returns:
 }
 ```

-`validate.py` enforces this schema before `build_graph()` consumes it.
+`validate.py` enforces this schema before `build()` consumes it.

 ## Confidence labels

@@ -57,8 +74,8 @@ Every extractor returns:

 ## Adding a new language extractor

-1. Add an `extract_<lang>(path: Path) -> dict` function in `extract.py` following the existing pattern (tree-sitter parse → walk nodes → collect `nodes` and `edges` → call-graph second pass for INFERRED `calls` edges).
-2. Register the file suffix in `extract()` dispatch and `collect_files()`.
+1. Add an `extract_<lang>(path: Path) -> dict` function following the existing pattern (tree-sitter parse → walk nodes → collect `nodes` and `edges` → call-graph second pass for INFERRED `calls` edges). New languages go in their own module under `graphify/extractors/` - see `graphify/extractors/MIGRATION.md`; `extract.py` re-exports them while the existing ones are ported out of it.
+2. Register the file suffix in `extract()`'s dispatch table and in `collect_files()` (both in `extract.py`).
 3. Add the suffix to `CODE_EXTENSIONS` in `detect.py` and `_WATCHED_EXTENSIONS` in `watch.py`.
 4. Add the tree-sitter package to `pyproject.toml` dependencies.
 5. Add a fixture file to `tests/fixtures/` and tests to `tests/test_languages.py`.
@@ -0,0 +1,87 @@
+"""ARCHITECTURE.md's module table must name symbols that actually exist (#2640).
+
+The "Module responsibilities" table is the entry point for library users, and
+AGENTS.md points agents at the docs before the code. It had drifted to document
+six functions that do not exist (`detect.collect_files`, `build.build_graph`,
+`analyze.analyze`, `report.render_report`, `export.export`,
+`serve.start_server`) and to give `extract()` a single-path signature when it
+takes a list. Anyone following it wrote code that raised, or -- worse for
+`extract()` -- code that ran and silently produced non-canonical ids.
+
+These tests parse the table itself rather than restating it, so adding a row
+extends the coverage automatically and renaming a function in the code fails
+here until the doc is updated too.
+"""
+from __future__ import annotations
+
+import importlib
+import inspect
+import re
+from pathlib import Path
+
+import pytest
+
+_ARCHITECTURE = Path(__file__).parent.parent / "ARCHITECTURE.md"
+
+# A table row: | `<module>.py` | <entry points> | <input → output> |
+_ROW = re.compile(r"^\|\s*`(\w+)\.py`\s*\|(.*?)\|", re.MULTILINE)
+# A function reference inside a cell: `name(...)` or a bare `name`.
+_FUNC = re.compile(r"`([a-z_][a-z0-9_]*)(?:\(|`)")
+
+
+def _documented_symbols() -> list[tuple[str, str]]:
+    """(module, function) for every function named in the module table."""
+    text = _ARCHITECTURE.read_text(encoding="utf-8")
+    start = text.index("## Module responsibilities")
+    end = text.index("##", start + 3)
+    pairs: list[tuple[str, str]] = []
+    for module, cell in _ROW.findall(text[start:end]):
+        for func in _FUNC.findall(cell):
+            pairs.append((f"graphify.{module}", func))
+    return pairs
+
+
+def test_the_table_was_actually_parsed():
+    """Guard the parser itself: a regex that silently matches nothing would make
+    every parametrized test below vacuous."""
+    pairs = _documented_symbols()
+    assert len(pairs) >= 10, f"parsed too few symbols, regex likely broken: {pairs}"
+    modules = {m for m, _ in pairs}
+    assert {"graphify.extract", "graphify.build", "graphify.serve"} <= modules, modules
+
+
+@pytest.mark.parametrize("module,func", _documented_symbols())
+def test_architecture_table_symbols_exist(module, func):
+    mod = importlib.import_module(module)
+    assert hasattr(mod, func), (
+        f"ARCHITECTURE.md documents {module}.{func}, which does not exist. "
+        f"Update the table (or re-export the symbol)."
+    )
+
+
+def test_architecture_documents_extract_as_taking_a_list():
+    """`extract(path)` was documented for a function whose first parameter is a
+    list; a caller passing one Path gets TypeError: not iterable."""
+    from graphify.extract import extract
+
+    params = list(inspect.signature(extract).parameters.values())
+    assert params[0].name == "paths", params
+    assert "list" in str(params[0].annotation), params[0].annotation
+
+    text = _ARCHITECTURE.read_text(encoding="utf-8")
+    assert "`extract(path)`" not in text, "the single-path signature is back"
+    assert "extract(paths" in text
+
+
+def test_architecture_tells_library_callers_to_pass_root():
+    """The omitted `root=` is the parameter whose absence yields non-canonical
+    ids and source_file values, so the doc must not just name it -- it has to
+    say to pass it."""
+    from graphify.extract import extract
+
+    root = inspect.signature(extract).parameters["root"]
+    assert root.kind is inspect.Parameter.KEYWORD_ONLY, root.kind
+
+    text = _ARCHITECTURE.read_text(encoding="utf-8")
+    assert "root=Path" in text, "no worked example passing root"
+    assert "Always pass `root`" in text