If `graphify extract --backend claude` runs without the `anthropic`
package installed (pip install graphifyy doesn't pull it in), every
semantic chunk fails inside extract_corpus_parallel. The per-chunk
errors print to stderr but the function returns the empty merged
accumulator anyway, so extract proceeds to write an AST-only graph.json
and exit 0. CI that checks exit status sees success even though the
requested semantic pass produced no nodes.
Track per-chunk success via the existing on_chunk_done callback, which
only fires after a chunk succeeds. If fresh extraction was requested
(uncached_paths non-empty) and zero chunks completed, abort before the
merge/cluster/write phase with exit 1 and a message naming the backend.
The same shape covers other backends with optional SDK deps (openai,
google-generativeai). Cached-only runs are unaffected: uncached_paths
is empty and the guard does not fire.
Tests in tests/test_extract_cli.py simulate the all-failed and
one-succeeded paths by patching extract_corpus_parallel directly.
ArkTS (.ets) is the primary language for HarmonyOS/OpenHarmony
application development, used by projects like SceneBoard. The
tree-sitter TypeScript parser already handles .ets files for AST
extraction — the detect module just wasn't recognizing them.
Without this, `detect()` silently skips all .ets source files,
missing ~90% of code in OpenHarmony codebases.
Co-authored-by: Autumn <autumn@AutumndeMacBook-Pro.local>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Adds docs/translations/README.uz-UZ.md and inserts an entry for
🇺🇿 Oʻzbekcha into the language navigation bar of README.md and all
27 existing translations.
Co-authored-by: Javokhir Sherbaev <javokhir.sherbaev@noveogroup.com>
- Add 'export_statement' to import_types for JS/TS/TSX configs
- Extend _import_js to detect 'export { X } from ./mod' re-exports
- Emit 're_exports' edges linking barrel files to source symbols
- Preserve walk-through for 'export function/const' declarations
- Add 're_exports' to clean_edges allowlist for cross-file edges
Tested on a 976-file Next.js codebase: detects 162 re_exports edges
and 5760 symbol-level imports (previously 0 for both).
* feat(bash): harden extractor — literal filtering, entrypoint nodes, AST-ancestry-aware command detection
Builds on tree-sitter-bash extractor from #866. Two correctness/security
improvements to bash extraction in graphify/extract.py:
1. Reject command/process substitutions at extraction time. Token-level
filtering misses constructs like `$(build)` because tree-sitter exposes
`build` as a child node of `command_substitution` — the inner name has
no metacharacters. Added `is_inside_expansion(node)` that walks
`node.parent` until it finds `command_substitution` or
`process_substitution`. Used as a gate in both `walk` and `walk_calls`.
Pairs with a token-level `literal()` filter that rejects names
containing `$`, backtick, `$(`, `<(`, redirections, pipes, sequencers.
2. Entrypoint node. Every .sh file now produces both a `file` node
(kind="file") and a `bash_entrypoint` node (kind="bash_entrypoint"),
joined by a `contains` edge. A separate top-level `walk_calls(root,
entry_nid, ...)` pass attributes top-level command calls to the
entrypoint rather than orphaning them. Matches the entrypoint pattern
other-language extractors use. Node metadata gains language+kind.
Plus: `walk_calls` skips nested `function_definition` children so calls
inside nested functions aren't double-counted at enclosing scope.
Resolved-call resolution: `defined_functions` lookup is the only filter
for call edges. User-defined functions named like external commands
(install, find, git, ...) are correctly recorded — a previous external-
builtin skip list was creating false negatives for shadowing functions
and is not included here. Skip list belongs with raw/unresolved call
recording (not in this PR).
Devtools (bundled): pyproject.toml gains [dependency-groups] dev (ruff,
pyright, pre-commit, hypothesis, pip-audit) plus minimal [tool.ruff],
[tool.ruff.lint], [tool.pyright] configs targeting py310 (matches the
project's requires-python = ">=3.10").
Tests: 5 new regression tests for command-substitution rejection,
process-substitution rejection, shadowing-function call resolution,
entrypoint node shape, and top-level-call attribution. 826/826 pass
(was 821); 15/15 bash-relevant tests pass (was 10).
* feat(detect): parse macOS/BSD and GNU env(1) shebang option forms
Upstream's _shebang_file_type parses shebangs via line[2:].split() and only
handles `#!/usr/bin/env <interp>`. Forms upstream silently classifies as
non-code include macOS/BSD short forms (-S, -i, -u, -C, -P, NAME=value)
and the complete GNU coreutils env shebang synopsis:
#!/usr/bin/env -[v]S[option]... [name=value]... command [args]...
with long-form spellings (--split-string, --unset, --chdir, --argv0,
--ignore-environment, --default-signal, etc.), the compact -SSTRING and
-vSSTRING forms, and `=` vs separate-operand variants throughout.
Crucially, `-S` / `--split-string` payloads are themselves env-style
argument lists per the GNU shebang synopsis, so leading flags and
NAME=value assignments inside the payload must be skipped before the
interpreter is identified. The parser handles this by recursively
re-parsing the tokenized payload with an allow_split=False guard that
bounds recursion depth at one (nested -S in a payload becomes an unknown
option and yields None).
Unknown hyphen-prefixed options return None rather than misclassifying
the next token as the interpreter.
_shebang_file_type becomes a 4-line wrapper. Read buffer raised 128 -> 256
to accommodate longer env -S strings.
Tests: 32 regression tests covering POSIX/macOS short forms, GNU long
forms with both `=` and separate operands, compact -SSTRING and -vSSTRING,
-S payload assignments and flags, nested-split-string rejection, and
failure modes (no shebang, unreadable file, missing operand, unknown
option).
* fix(skills): enforce semantic fragment validation in OpenCode + Codex merges (#825)
Closes#825. Adds graphify.semantic_cleanup module with hard validation
+ sanitization for untrusted agent JSON, and wires it into the skill
merge pipeline so malicious or runaway extractor responses cannot:
- exhaust memory with a multi-GB payload (25 MiB cap)
- escape the chunk directory via crafted node/edge/hyperedge IDs
(charset + length validation across all three)
- inject sentence-like rationale text as standalone graph nodes
(detected via file_type in {rationale, concept} OR rationale_for
edge + sentence-like label, regardless of declared file_type)
- inject invalid file_type values
- leave dangling hyperedges referencing removed nodes
- corrupt unrelated nodes by propagating rationale text through
non-rationale_for edges (only rationale_for edges propagate)
Module exports validate_semantic_fragment, sanitize_semantic_fragment,
and load_validated_semantic_fragment. Wired into skill-opencode.md and
skill-codex.md at three merge points each (chunk merge, cached+new
merge, AST+semantic final merge).
Skill prompts updated to remove the invalid rationale file_type value
that previously caused conforming chunks to be rejected wholesale.
Valid set is now {code, document, paper, image}.
Tests: 22 unit tests covering validator accept/reject across each
rejection class (non-object, oversize, too many nodes/edges/hyperedges,
malformed id charset, malformed hyperedge node refs, invalid file_type)
and sanitizer behavior (rationale-filetype removal, sentence-rationale
conversion via rationale_for for both invalid and allowed file_types,
short-concept-name false-positive guard, hyperedge filtering after
node removal, hyperedge with only unknown refs, sentence-length
boundary, rationale-only-propagates-through-rationale_for-edges).
880/880 tests pass.
* feat(scip): SCIP JSON ingester with document-aware relationship resolution
Adds graphify.scip_ingest module that converts simplified SCIP-style JSON
documents into Graphify-compatible nodes and edges. Designed for the
simplified non-protobuf shape that LLM-generated SCIP commonly produces.
Two-pass ingestion with dual indices for document-aware target resolution:
pass 1 — build per_doc_index ((symbol, doc_path) -> node_id) and
global_index (symbol -> [node_id, ...]) across every valid
symbol in every valid document. Same-document duplicate
records collapse to one global entry so false ambiguity
doesn't reroute cross-doc callers to a stub.
pass 2 — emit nodes for indexed symbols, then walk relationships.
Resolution order:
1. same-doc match (per_doc_index)
2. unique cross-doc match (global_index[symbol] len == 1)
3. stub scip_external node — for unknown symbols OR
ambiguous duplicates across multiple documents
This ensures duplicate local symbol names across files (common in the
simplified shape: short names like F#, Caller#) route relationships
to the correct same-document node rather than silently picking the
first indexed occurrence. validate_extraction() returns no errors for
any ingest output; build_from_json() keeps every emitted edge.
Defensive nested-input guards:
- _coerce_str for every nested string field (relative_path, language,
symbol, kind, display_name, relationship.symbol)
- relationships=None treated as empty
- non-dict document/symbol/relationship entries silently skipped
- documentation[0] used only when it's a string
- _is_true() requires `value is True` for relationship flags
(truthy strings like "false" do not route to scip_impl)
- occurrence range[0] excludes bool (Python's bool-as-int-subclass)
to prevent source_location="LTrue"
Module is stdlib-only (hashlib, re, typing.Any). Not wired to the CLI
in this phase — importable as `from graphify.scip_ingest import
ingest_scip_json`.
Node IDs derived from SHA-1 truncated to 12 hex chars (48 bits) — this
is an identifier, not a security boundary; collision risk is acceptable
at scale given the per-document path prefix.
Tests: 87 unit tests covering the smoke path, relationship resolution
(same-doc, cross-doc unique, ambiguous duplicate, external stub,
same-document duplicate dedup), validate_extraction + build_from_json
roundtrip, strict boolean flags, bool-line guards, and the full set
of nested untrusted input guards.
1044/1044 tests pass.
* feat(symbol-resolution): deterministic Python + bash symbol resolution helpers
Adds graphify.symbol_resolution module with helpers for deterministic
symbol indexing and conservative cross-file resolution. Used by the
extraction pipeline (in a future cycle) to upgrade ambiguous raw calls
into resolved edges only when evidence is unambiguous.
Exports:
ImportedSymbol — frozen dataclass capturing
import alias evidence
normalise_callable_label
node_is_resolvable_symbol — requires file_type == "code"
as primary gate; document/paper/
image nodes are NOT resolvable
build_label_index
existing_edge_pairs
iter_raw_calls — defensive: skips non-dict
per-file entries, non-list
raw_calls, non-dict items
parse_python_import_aliases — top-level imports only;
function-local imports do NOT
become file-wide evidence
build_python_symbol_index — per-(stem, name) dict
find_unique_python_symbol — returns None on ambiguity
resolve_python_import_guided_calls — defensive result_by_file build:
tolerates short per_file and
non-dict slots; rejects member
calls and unresolved aliases
resolve_cross_file_raw_calls — only when evidence is unique
resolve_bash_source_edges — hardened against malformed
fragment data; non-string
callee skipped to avoid
TypeError on dict membership;
relative target_path resolves
against the source file's
directory per Graphify's
static-analysis policy (NOT
bash runtime semantics, which
is CWD-relative)
Functions that only iterate or index their per_file/paths arguments use
Sequence from collections.abc for proper covariance. Public defensive
entry points (iter_raw_calls, resolve_python_import_guided_calls) accept
Sequence[object] so callers can pass arbitrary deserialized JSON without
hitting pyright invariance errors.
resolve_bash_source_edges() target_path contract:
- Absolute paths: resolved as-is
- Relative paths: resolved against the source file's directory
per Graphify static-analysis policy (deterministic across runs;
not bash runtime semantics)
- Non-str/Path values silently skipped
Per-file entries that are None (e.g. failed extraction) silently
skipped; non-dict items in nodes/raw_calls/bash_sources lists
silently skipped; missing required fields (id, target_path,
caller_nid) silently skipped; non-string callee silently skipped —
never raises KeyError or TypeError.
Module is stdlib-only (ast, re, dataclasses, pathlib, typing,
collections.abc). Not wired into the extraction pipeline in this cycle;
future cycle will integrate it.
Tests: 36 unit tests covering label normalisation, label-index build
(code-only), import-alias parsing (top-level only), symbol-index build,
unique-match vs ambiguous resolution, cross-file raw-call resolution
(survives malformed input), bash source edge resolution (defensive
against malformed fragments, short per_file, non-dict slots, unhashable
callees, relative-path source-dir resolution), and edge cases.
* feat(security): cap graph.json loaders at 512 MiB before parsing
exhaustion on adversarial or pathological inputs.
- graphify.security: add _MAX_GRAPH_FILE_BYTES + check_graph_file_size_cap
- graphify.serve._load_graph: call cap after existence check
- graphify.__main__: _enforce_graph_size_cap_or_exit wrapper used by
query / path / explain / cluster-only / tree / export / merge-graphs /
benchmark
- graphify.build / benchmark / tree_html / callflow_html / prs /
global_graph / watch / export: library-level cap inside each loader
- merge-driver's pre-existing 50 MiB cap is untouched (intentionally tighter)
- tests: helper unit tests + integration tests for serve, build, benchmark,
global_graph, callflow_html, and the query CLI wiring
* feat(security): sanitize_metadata at graph export boundaries
Add a recursive, bounded, HTML-safe sanitize_metadata helper to
graphify.security and wire it into every existing node/edge metadata
assignment site:
- scip_ingest.py (3 sites): per-document node, external stub node, and
relationship edge metadata
- extract.py (1 site): bash extractor's add_node metadata
- symbol_resolution.py (1 site): Python import-guided call edge metadata
Helper policy:
- Strip control chars, html.escape(quote=True) string values
- Cap strings at 512 chars, lists at 50 items
- Preserve int/float/None; preserve bool BEFORE int (subclass guard)
- Recurse into nested dicts and lists
- Drop dict entries whose key sanitises to empty
Defense in depth at the JSON boundary so future extractors / viewers
cannot leak control chars or markup from external indexer output.
* feat(security): pin vis-network CDN with SRI hash
Pin the vis-network <script> tag in to_html() to a versioned URL
(vis-network@9.1.6) with a sha384 Subresource Integrity hash and
crossorigin="anonymous". Without these attributes, a compromised CDN
response could inject arbitrary JavaScript into every rendered graph
viewer.
Hash verified live against
https://unpkg.com/vis-network@9.1.6/standalone/umd/vis-network.min.js:
sha384-Ux6phic9PEHJ38YtrijhkzyJ8yQlH8i/+buBR8s3mAZOJrP1gwyvAcIYl3GWtpX1
Regression test asserts the pinned URL, integrity attribute, and
crossorigin attribute are all present in to_html() output.
Follow-up: tree_html.py (D3) and callflow_html.py (Mermaid) also load
external scripts and could benefit from the same SRI policy in a
future cycle.
* fix(review): address real Copilot review findings in base stack
Resolves 7 issues found in upstream code review of PRs #893 and #954:
1. extract.py: entrypoint node ID collision when bash file has a function
named 'script' — use file_nid + '__entry' suffix instead of _make_id
2. extract.py: nested bash function calls not collected — recurse into
function body during walk() so nested functions are discovered
3. extract.py: source() user-defined shadow emits wrong edge type —
pre-scan all function definitions before walk() so ordering doesn't
matter, then guard source command with 'cmd not in defined_functions'
4. extract.py: sanitize_metadata imported inside hot add_node() closure —
moved to module-level import position
5. symbol_resolution.py: _bash_make_id() diverged from extract._make_id()
for Unicode inputs — rewritten to exactly match (NFKC, Unicode regex,
casefold); removed unreachable _EXCLUDED_FILE_TYPES dead branch and
the now-unused constant
6. semantic_cleanup.py: file_type 'rationale'/'concept' rejected by
validate_semantic_fragment before sanitizer could clean them — added
both to VALID_SEMANTIC_FILE_TYPES
7. scip_ingest.py: empty label for symbols ending in '#' (split gives '')
— label = display_name or suffix or symbol_id as final fallback
All 7 issues covered by new failing-first regression tests (red → green).
Full pytest suite: 1239 passed, 4 pre-existing env-specific failures.
* fix(review): address PR #956 Copilot findings in watch.py and symbol_resolution.py
- watch.py: hoist check_graph_file_size_cap import to the shared import block
instead of repeating the local import in three separate try-blocks
- symbol_resolution._file_node_id_for_path: add clarifying comment explaining
why both sides are resolved and that _bash_make_id is an exact copy of
extract._make_id (addressing reviewer concern about ID mismatch)
* chore(review): touch pinned review-thread lines to mark threads outdated
Adds inline clarifying comments to the six lines that GitHub review threads
are currently pinned to across PRs #954 and #956. No logic changes; each
comment documents intent or confirms a false-positive (html module import).
* feat(diagnostics): report multigraph edge-collapse risk
Add graphify.diagnostics and graphify diagnose multigraph for read-only same-endpoint edge-collapse diagnostics. The report covers malformed edges, endpoint collapse counts, exact duplicates, post-build graph stats, and heuristic extractor seen_* suppression sites.
Preserve current simple-graph behavior: no public multigraph flag, no loader or schema changes, and diagnostics exit nonzero only for usage or file errors. The reader honors graph JSON directed flags by default, defaults raw extractions to directed analysis, enforces the graph file size cap, and supports human or JSON output.
* feat(multigraph): add runtime compatibility probe
New module graphify.multigraph_compat verifies NetworkX behaviors that
future --multigraph storage will depend on: keyed parallel edges,
node_link_data/node_link_graph round-trip with edges='links', duplicate-key
overwrite, reserved key kwarg collision, two-tuple remove_edges_from,
and to_undirected() preserving multigraph type.
Behavior probe, not version check. Both NX 3.4.2 (Py 3.10 lane) and
NX 3.6.1+ (Py 3.11+ lane) pass. Result cached for the process lifetime.
No call sites added — this PR adds the API surface only. Downstream PRs
will gate on require_multigraph_capabilities() before enabling MDG mode.
Refs: Wave 1 MultiDiGraph implementation order.
* test: filter known third-party analyze warnings
---------
Co-authored-by: vampyre <vampyre@local.net>
Graph queries filtered every token with len > 2, which dropped common two-character Chinese search terms while trying to suppress short English noise. Centralize query token selection and apply the length gate only to pure-English tokens so mixed or non-English terms remain searchable.
Constraint: Issue #962 reports space-separated Chinese query terms such as 前端, 依赖, and 安装 are lost by graphify query.
Rejected: Add Chinese segmentation now | the reported failure is fixed by preserving existing space-separated non-English tokens without expanding query behavior.
Confidence: high
Scope-risk: narrow
Directive: Keep CLI, MCP query, and benchmark query tokenization on one helper when changing query-term rules.
Tested: uv run --with pytest pytest tests/test_serve.py tests/test_query_cli.py tests/test_benchmark.py
Tested: graphify update .
Not-tested: Full test suite.
Co-authored-by: OmX <omx@oh-my-codex.dev>
tree-sitter-swift parses both `class Foo` and `extension Foo` as
`class_declaration`, and node ids carry the file stem, so `extension Foo`
in a sibling file produced a second `Foo` node instead of attaching to
the original. Same-file extensions already dedupe via seen_ids; only the
cross-file case leaked.
Per-file extraction now tags `extension` class_declarations, and the
corpus-level `extract()` runs a merge pass: when exactly one
non-extension declaration shares the label, the extension nodes redirect
onto it and their edges are rewritten (self-loops dropped, duplicates
collapsed). Extensions of types outside the corpus and ambiguous label
matches stay untouched.
On a 25-file Swift project this collapses Parser from 6 split nodes
(top of the god-node list, four entries) to one canonical node, and
lets the generic cross-file call resolver attach previously ambiguous
call edges to the right target.
Backends routed through _call_openai_compat (gemini, openai, kimi,
deepseek, ollama) silently ignored the documented env override when
their backend config dict carried a hardcoded max_completion_tokens.
The dispatcher used:
cfg.get("max_completion_tokens", max_out)
which always returned the config-dict value when the key was present,
shadowing the env-var-resolved max_out.
For gemini specifically, the hardcoded cap of 16384 truncated
extracted-graph JSON mid-response on multi-document chunks (~17 specs
of 100-1500 lines each pushing the output past 16k tokens). Symptom:
cascading 'LLM returned invalid JSON, skipping chunk: Unterminated
string at column 4XXXX' followed by bisect-retry storms that bill
input tokens without producing graph nodes.
Fix: route the same _resolve_max_tokens(...) call that the Claude and
Bedrock paths already use, so the override applies uniformly across
backends.
Verified with gemini-2.5-pro over a 20-doc / 76k-input-token chunk:
output of 36008 tokens emitted without truncation, producing 193
nodes / 223 edges / 23 communities in a single chunk.
- fix SQL extractor using bare path.stem as node ID prefix — collides across same-named files in different dirs; use _file_stem() (directory-qualified) instead
- fix Python import resolver keying stem_to_entities by bare stem; add bare_to_qualified secondary index for absolute imports so cross-file edges survive duplicate filenames
- add stat-based mtime fastpath to file_hash: skip full SHA256 when size+mtime_ns unchanged, flush index atomically via atexit (same trade-off as make)
- add cache-check, merge-chunks, merge-semantic CLI subcommands so the skill pipeline can use library functions instead of inline Python
- fix absolute source_file paths from semantic subagents not being relativized before graph storage (#932): add root param to build_from_json/build/build_merge, pass scan target at both call sites
- fix failed semantic chunks permanently freezing their files in the manifest (#933): filter _manifest_files to only include doc/paper/image files that appear in sem_result nodes/edges before calling save_manifest
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
detect: add scan_root to return dict so skill can strip absolute prefix
when computing relative subdirectory breakdown; remove stale --no-semantic
flag reference from large-corpus warning (flag does not exist)
skill: clarify fast path checks CWD graphify-out/graph.json (project root);
remove hardcoded --backend gemini from multi-subfolder example — users
should pass whichever backend key they have; expand large-corpus gate
instruction to use scan_root for relative paths, filter graphify-out/
converted sidecars, and handle flat repos with no subdirectories
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fast path: if graphify-out/graph.json exists and user is asking a question
(not an explicit rebuild), skip detect entirely and run graphify query —
prevents the skill from refusing large already-built corpora (#930)
- Raise FILE_COUNT_UPPER 200 → 500 so typical 200-500 file codebases no longer
hit the large-corpus size gate on fresh extraction (#930)
- Subdirectory breakdown now strips the scan-root prefix so agent shows
relative names (core/, service/) not absolute paths rooted at / (#930)
- Document multi-subfolder CLI pattern: graphify extract ./sub/ places
graphify-out/ inside each subfolder; skill clobbers single root graphify-out
when run on subfolders separately (#930)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Security: _is_sensitive now flags underscore-prefixed names (api_token.txt, oauth_token.json) by replacing \b with lookarounds; adds _SENSITIVE_DIRS check on parent path components (parts[:-1]) so .ssh/, secrets/, .aws/ directories are always skipped; aligns both patterns to (?![a-zA-Z]) for consistent underscore-after-keyword behavior (#920)
Fix: --wiki Relationships section always empty because _cross_community_links read community from node attrs (always None) instead of the communities dict; _god_node_article had the same bug and never linked to the owning community; fixed by building a node->community map in to_wiki() and threading it through (#925)
Fix: --watch now respects .graphifyignore; patterns loaded once at startup, handler checks _is_ignored before extension filter so node_modules/, .venv/, build/ churn no longer triggers rebuilds (#928)
Fix: C++ struct inheritance edges via base_class_clause; initialize base="" per iteration to prevent stale carryover (#915)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The OpenAI-compatible API can return HTTP 200 with an empty `choices`
list or with `choices[0].message = None` (e.g. content-filtered
responses on Gemini, overwhelmed Ollama instances). Without a guard,
both sites raise an unhandled IndexError or AttributeError.
`_call_openai_compat` already documents this hazard ("Ollama can return
HTTP 200 with empty/null content") and has `_response_is_hollow` logic
downstream, but `_response_is_hollow` is unreachable when the choices
list itself is empty. The new guard closes that gap.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
- hooks.py: drop --path-format=absolute (added git 2.31), validate no newlines in path, anchor relative paths on repo root (#907)
- detect.py: seed save_manifest from existing manifest before loop so incremental callers don't erase untouched file entries (#917)
- cluster.py: drop round(..., 2) from cohesion_score so split threshold 0.05 fires correctly; add resolution param to _partition and cluster; add exclude_hubs_percentile to cluster with majority-vote reattachment (#919)
- report.py: format cohesion with :.2f for display
- __main__.py: wire --resolution and --exclude-hubs into extract and cluster-only commands (#919)
- C++ inheritance already written to disk by analysis agent (#915)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(analyze): exclude npm dep-block keys from god-node selection
Extends _JSON_NOISE_LABELS in graphify/analyze.py with the six npm
package.json dependency-block keys (dependencies, devDependencies,
peerDependencies, optionalDependencies, bundledDependencies,
bundleDependencies — lowercased to match the existing .strip().lower()
comparison in _is_json_key_node).
On JS/TS corpora with non-trivial dependency counts, the dep-block key
node accumulates contains+imports edges to every package entry and was
surfacing as the top god-node in the report. The fix is a one-line
extension of the existing frozenset; no new helpers or code paths.
Includes a parametrized regression test in tests/test_analyze.py
covering all five npm keys: dependencies, devDependencies,
peerDependencies, optionalDependencies, bundledDependencies.
Validated live against rsl-siege-manager @ 6085fd66 — zero npm dep-block
keys in the top-10 god-nodes after the fix. Pre-existing failures
(Windows symlink + SQL tree-sitter): 17, unchanged.
Closes#2
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style: drop fork-local issue refs from analyze comments
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(extract): force UTF-8 encoding on subprocess + loud failure on chunk errors
On Windows cp1252, subprocess.run(..., text=True) without encoding= raises
UnicodeEncodeError for chars like → ✅ ≥ in chunk content. Both
_call_claude_cli (llm.py:426) and the _call_llm claude-cli branch (llm.py:959)
lacked encoding=.
- Add encoding="utf-8" to both subprocess.run sites.
- Track failed_chunks in extract_corpus_parallel merged result dict.
- Print [graphify] WARNING: N/M semantic chunk(s) failed summary to stderr at
end of run when any chunk failed, so silent partial failures are visible.
- Add tests/test_charmap_encoding.py: 10 regression tests covering subprocess
encoding kwarg, loud-failure summary, and substitution validation.
Closes#3
* style: drop fork-local issue refs from llm/test comments
- Revert .h -> extract_c (C++ grammar rejects C++ keywords used as identifiers
in Linux-kernel-style headers; .hpp/.hxx/.hh already route to extract_cpp)
- Fix field_declaration block: use children_by_field_name("declarator") instead
of iterating all children with wrong type guard; replace ensure_node (undefined)
with add_node
- Fix _import_c include resolution: use _make_id(str(resolved)) to match the
file_nid scheme _extract_generic uses, not _make_id(_file_stem(resolved))
- Fix exact_merges counter in dedup Pass 1 to count only within-file merges
actually performed, not the raw unpartitioned group sizes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Route .h files through extract_cpp (was extract_c), fixing missing method nodes in C++ headers
- Extend _get_cpp_func_name to handle field_identifier, destructor_name, operator_name
- Add CPP-specific field_declaration branch in _extract_generic to emit class method/field nodes
- Partition dedup Pass 1 by source_file: only union same-label nodes within the same file;
cross-file matches fall through to Pass 2 fuzzy, preventing generic-label god nodes
- Add _resolve_c_include_path: resolve quoted #include paths to real files on disk so
target node IDs match what _extract_generic creates, fixing dangling include edges
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Common terms like 'error'/'exception' were stealing BFS seed slots from
rare identifiers like 'FooBarService', burning the token budget on noise.
- _compute_idf: weights query terms by inverse document frequency, cached
on G.graph so cost is paid once per graph load not per query
- _score_nodes: multiplies each tier bonus by IDF weight
- _pick_seeds: replaces fixed top-3 with gap-ratio selection — stops adding
seeds when score drops below 20% of the top match
- _subgraph_to_text: truncation hint now tells Claude to narrow with
context_filter or use get_node instead of just saying 'truncated'
Fixes#897
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
