* chore: declare pytest as a uv dev dependency
The contributing guide currently tells contributors `pip install pytest`
as a separate step, and CI does the same. Move pytest into PEP 735
`[dependency-groups]` so it's declared in pyproject.toml and `uv sync`
installs it by default (no `--with` workaround, no separate install
line). Update CI to use astral-sh/setup-uv + `uv sync` + `uv run pytest`,
and refresh the Contributing section of the README to match.
`[dependency-groups]` is the right home (vs `[project.optional-dependencies]`)
because pytest is dev-only and shouldn't appear in the published wheel's
optional features list alongside things like `pdf` or `mcp`.
* remove uv.lock from gitignore
When a graph exceeds the viz node limit, to_html() builds a
community-aggregated meta-graph and recursively calls itself.
The recursive call never carried hyperedges onto the meta-graph,
so graph.html always emitted const hyperedges = [] even when
graph.json contained plenty.
This fix remaps hyperedge node references from semantic node IDs
to community IDs before the recursive call, so hyperedge regions
render correctly in the aggregated view. Hyperedges that collapse
to fewer than 2 distinct communities are dropped (they wouldn't
render as a polygon anyway).
Fixes#1005
graphify-out regenerates differently on every `graphify update` even
when no source changed, so the committed graph is perpetually dirty and
the post-commit/post-checkout hooks fight every commit. Two independent
nondeterminism sources, each fixed here:
1. Edge direction flips. build.py builds an undirected graph and stores
direction in _src/_tgt; collapsing two edges onto the same node pair
is last-write-wins, and unstable edge iteration order flips them
run-to-run. Fixed by sorting edges by (source, target, relation)
before the add loop.
2. Clustering churn. The networkx Louvain fallback iterates string-keyed
sets whose order is randomized per-process by PYTHONHASHSEED, so
community assignments differ run-to-run even with seed=42. Fixed by
exporting PYTHONHASHSEED=0 in the generated post-commit and
post-checkout hook scripts.
With both fixes, `graphify update` is idempotent: rebuilding an
already-converged graphify-out reproduces graph.json and GRAPH_REPORT.md
byte-for-byte.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
stdlib ET does not cap entity expansion — a crafted .csproj or .lpk with nested
internal entities can exhaust memory. Pre-screen input bytes for <!DOCTYPE and
<!ENTITY before parsing (legitimate MSBuild/Lazarus files never contain these).
Also adds the missing 2 MiB size cap to extract_lpk (csproj already had one).
No new dependencies required.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds graphify/mcp_ingest.py — extracts MCP server configurations into the
knowledge graph. Captures server nodes, NuGet/npm/pip package refs, commands,
env var requirements, and inter-server edges. Dispatched by filename before
the suffix lookup so generic .json extraction is unaffected. Env values are
discarded to prevent secret leakage. File size capped at 1 MiB. 29 tests.
Fixes: server_count budget now checked after validity guard so invalid entries
don't consume capacity; removed misleading uv run docstring example.
Co-Authored-By: adityachaudhary99 <adityachaudhary99@users.noreply.github.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
cluster-only re-runs Leiden clustering and then re-applies the existing
.graphify_labels.json by raw cid index, which causes labels to attach to
clusters whose members are unrelated to the label's original meaning
whenever the graph has changed between labeling and re-clustering.
Mirror the safety net already present in watch.py:_rebuild_code added in
#822 for the watch/update paths.
Adds a regression test that fails without the fix (label cids become
orphaned from graph.json community attributes after re-clustering).
Refs: #1027
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three issues in _rebuild_code (watch.py):
1. _relativize_source_files was called on result after eviction list was built,
so existing nodes with absolute source_file were never normalized before comparison
2. deleted_paths and evict_sources used str() (backslashes on Windows) while
graph.json stores forward-slash paths via _norm_source_file
3. _relativize_source_files itself used str() instead of as_posix()
Also fix extract.py source_file relativization to use as_posix(). Closes#1007.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace inlined path normalisation with _norm_source_file (the same function
that builds node source_file keys) so prune_set and node attrs are normalised
identically. resolve() on root handles symlinked scan roots. Keep both raw and
normalised forms in prune_set so nodes with absolute source_file also match.
Add edge pruning and Windows backslash path tests per Opus review.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
prune_set in build_merge now includes relative-path variants of each deleted file
so manifest absolute paths (e.g. /home/user/corpus/module_b/utils.py) match graph
node source_file values (e.g. module_b/utils.py) regardless of OS or run context.
Fixes#1007.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Keep Graphify query segmentation focused on Chinese terms: rename the CJK helpers and extra to Chinese scope, cache the optional jieba import at module load, and keep a bigram fallback when jieba is unavailable.
Constraint: Reviewer asked either to broaden Hiragana/Katakana/Hangul support or rename CJK helpers; user chose Chinese-only because Japanese segmentation accuracy is uncertain.
Rejected: Broaden to Japanese and Korean segmentation | jieba is Chinese-oriented and the user explicitly limited scope to Chinese.
Confidence: high
Scope-risk: narrow
Directive: Do not label this path as CJK unless Hiragana/Katakana/Hangul segmentation is intentionally supported and tested.
Tested: uv run --with pytest pytest tests/test_serve.py tests/test_query_cli.py tests/test_benchmark.py
Tested: uv run --with pytest --with jieba pytest tests/test_serve.py -k "chinese or non_chinese"
Tested: graphify update .
Not-tested: Full test suite.
Co-authored-by: OmX <omx@oh-my-codex.dev>
TS 5.0 allows `"extends": ["./a", "./b"]`. _read_tsconfig_aliases called
`extends.startswith("@")` directly, so an array raised
`AttributeError: 'list' object has no attribute 'startswith'`. _safe_extract
caught it and skipped the WHOLE file — and since alias resolution fires on any
path-aliased import, every TS file using `@/`-style imports in a repo with
array-extends tsconfig was dropped, yielding an empty graph.
Normalize `extends` to a list (str -> [str], list kept, else []), process
entries in order (later overrides earlier, current config's paths override all
parents per TS semantics), and skip scoped npm configs (`@...`) per entry as
before.
Adds regression test test_tsconfig_array_extends_alias_resolves_existing_ts_file.
Add constrained query expansion step to /graphify query skill
## Problem
`graphify query` matches via case-folded substring + IDF — no stemming, no synonyms, no cross-language match. When the user's question uses different vocabulary than the graph labels (Slavic → English, "handlers" → "handler", "обработчик" → "handler"), the literal matcher returns 0
hits and the LLM consumer either gets empty subgraph or improvises an ungrounded keyword list from training memory (e.g. expanding "auth" to `{passport, sso, saml, oauth, jwt, scim, …}` regardless of whether those tokens exist in the corpus).
## Fix
Adds a `Step 0 — Constrained query expansion` block to the skill's `/graphify query` section. The LLM consumer extracts vocabulary from graph labels (CamelCase/snake_case split, length-filtered) and is instructed to pick **only** tokens present in that vocabulary, explicitly forbidden from inventing terms.
Effects:
- Bounded improvisation — fantom tokens (terms not in corpus) cannot be expanded, even when LLM "knows" they're related to the intent.
- Honest negative signal — if vocab is poor on a query's topic, expansion returns [] and the LLM tells the user, instead of fabricating a search.
- Auditability — selected tokens are printed to the user, and saved into `save-result` for the next --update to graph as Q&A nodes.
## Scope
Patches the canonical `graphify/skill.md`. The 11 host-variant skills (skill-codex.md, skill-aider.md, …) follow the same query-section contract but inline Python rather than calling `graphify query` CLI; those need a parallel patch with the inline form. Happy to follow up in a separate PR after review on the canonical patch.
## Test
On a graph built from the graphify repo itself (1284 nodes, 1454 vocab tokens), an unconstrained expansion of "укрупненная архитектура аутентификации" yields {auth, oauth, jwt, saml, sso, ldap, scim, mfa, 2fa, pin, passport, session, login, token} — of which 11/15 are absent
from the corpus. Constrained expansion against the actual vocab yields {credential, security, token, signature, user, architecture, component, module, overview} — 9 tokens, 0 fantom. Same retrieval, dramatically higher precision.
`graphify export html|obsidian|wiki|svg|graphml|neo4j` reads `communities`
exclusively from `.graphify_analysis.json` (set to `{}` if missing). The
post-commit / watch rebuild path doesn't regenerate that sidecar — only
graph.json + GRAPH_REPORT.md. Several skill workflows also delete temp
files at the end of `graphify extract`. In both cases the per-node
`community` attribute (`to_json` writes it on every node) is intact, but
the CLI ignores it.
Observed failure: `graphify export html` on a graph that exceeds the
viz node limit prints
Graph has 64703 nodes (above 5000 limit). Building aggregated community view...
Single community - aggregated view not useful. Skipping graph.html.
even though the same graph.json has 2,026 distinct `community` values
on its nodes — `to_html` just received an empty `communities` dict and
the aggregator collapsed to a single meta-node.
Fix: when the analysis sidecar is absent (or its `communities` field is
empty), reconstruct the `cid -> [node_ids]` mapping from the per-node
attribute in graph.json. The sidecar remains the canonical source of
truth when present; the reconstruction is a strict fallback. Every
downstream subcommand (`html`, `obsidian`, `wiki`, `svg`, `graphml`,
`neo4j`) sees the same shape it always did, just populated from the
graph itself instead of an externally-cached sidecar.
Tests added (tests/test_cli_export.py):
- `test_export_html_falls_back_to_node_community_attribute` —
delete the sidecar, run export html, confirm `graph.html` exists
and the "Single community" bail-out path does NOT fire.
- `test_export_html_fallback_recovers_multiple_communities` —
stronger guarantee that the reconstructed community count equals
what the sidecar would have provided (no silent data loss).
- `test_export_html_no_community_data_at_all_still_succeeds` —
hand-build a graph.json with no per-node `community` attribute
(older `to_json` versions, manually-constructed graphs); the
command must still exit cleanly rather than crash.
All 26 tests in test_cli_export.py pass; ruff clean on both files.
The post-commit hook passes `git diff --name-only HEAD~1 HEAD` as
changed_paths to `_rebuild_code`. That list includes deletions, and
`_rebuild_code` correctly identifies them (lines 352-367 in watch.py)
and evicts the stale nodes from the preserved set. The rebuilt graph
is intentionally smaller.
`_check_shrink` then refuses to overwrite the existing graph.json
because it only sees the node-count delta, not the cause. The guard
fires with "Refusing to overwrite — you may be missing chunk files
from a previous session. Pass --force to override."
Result: every commit that deletes a tracked file silently leaves stale
nodes in graph.json. The user must either pass --force (which also
disables the guard for legitimate failure modes) or manually re-run
`graphify update . --force` after delete-heavy commits.
Fix: thread a `had_explicit_deletions` flag from `_rebuild_code` into
`_check_shrink`. When the caller has declared the deletions, the
smaller graph is the expected outcome and the guard is skipped. The
guard remains intact for SILENT shrinkage — its actual purpose —
from failed semantic chunks or corrupted runs.
The fix is opt-in by design: callers that don't pass `changed_paths`
(e.g. the post-checkout full rebuild path) keep the old conservative
behavior. Only paths that explicitly track deletions get the bypass.
Tests added (tests/test_watch.py):
- `test_check_shrink_blocks_silent_shrink` — pre-existing behavior intact
- `test_check_shrink_allows_force_override` — pre-existing behavior intact
- `test_check_shrink_allows_explicit_deletions` — new: deletion bypass
- `test_check_shrink_allows_no_existing_data` — first-run case
- `test_check_shrink_allows_growth` — sanity
- `test_check_shrink_unlinks_tmp_on_refuse` — cleanup on refusal
- `test_check_shrink_keeps_tmp_when_deletions_declared` — no spurious unlink
- `test_rebuild_code_prunes_deleted_file_nodes` — end-to-end probe of the
exact scenario the post-commit hook triggers (git init, build, delete
one file, re-run with the deleted path in changed_paths, verify the
graph shrinks and the surviving file's nodes are preserved)
All 8 new tests pass; full test_watch.py + test_build.py + test_export.py
(74 tests) pass with no regressions.
- hooks.py: add _user_hooks_dir() to target .husky/ instead of .husky/_ on Husky 9 repos (#987)
- skill.md: replace unsubstituted 'INPUT_PATH' literal with '.' in generate() calls (#986)
- extract.py: wrap future.result() per-future so a single worker failure prints a warning instead of falling back to full sequential re-extraction (#943)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Replace non-ASCII → with -> in __main__.py print statements (#992)
- Coerce paths to Path objects at extract() entry to fix str.parent crash (#988)
- Fix serve.py ImportError to recommend graphifyy[mcp] extra (#967)
- Change relative import in _tool_god_nodes to absolute to fix script invocation (#966)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
If `graphify extract --backend claude` runs without the `anthropic`
package installed (pip install graphifyy doesn't pull it in), every
semantic chunk fails inside extract_corpus_parallel. The per-chunk
errors print to stderr but the function returns the empty merged
accumulator anyway, so extract proceeds to write an AST-only graph.json
and exit 0. CI that checks exit status sees success even though the
requested semantic pass produced no nodes.
Track per-chunk success via the existing on_chunk_done callback, which
only fires after a chunk succeeds. If fresh extraction was requested
(uncached_paths non-empty) and zero chunks completed, abort before the
merge/cluster/write phase with exit 1 and a message naming the backend.
The same shape covers other backends with optional SDK deps (openai,
google-generativeai). Cached-only runs are unaffected: uncached_paths
is empty and the guard does not fire.
Tests in tests/test_extract_cli.py simulate the all-failed and
one-succeeded paths by patching extract_corpus_parallel directly.
ArkTS (.ets) is the primary language for HarmonyOS/OpenHarmony
application development, used by projects like SceneBoard. The
tree-sitter TypeScript parser already handles .ets files for AST
extraction — the detect module just wasn't recognizing them.
Without this, `detect()` silently skips all .ets source files,
missing ~90% of code in OpenHarmony codebases.
Co-authored-by: Autumn <autumn@AutumndeMacBook-Pro.local>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Adds docs/translations/README.uz-UZ.md and inserts an entry for
🇺🇿 Oʻzbekcha into the language navigation bar of README.md and all
27 existing translations.
Co-authored-by: Javokhir Sherbaev <javokhir.sherbaev@noveogroup.com>
Safi