Two tweaks on top of @bchan84x's #1852:
1. The definer heuristic decided the survivor pairwise, so when several
same-file nodes co-defined one ID the surviving label still depended on
arrival order (the exact 3-node case #1851 reports). Choose the survivor
by a total order (definer first, then shorter/canonical label, then
lexically) via a min over _collision_rank, so it is order-independent.
Direction preserves #1504 (lexically-first source path wins).
2. _defines_id now also recognizes a bare file-level node whose id is
exactly the slugified path (nid == prefix), not only <path>_<entity>.
Adds an order-independence test over all 6 permutations of definer +
same-file relabel + cross-file reference, and a bare-file-node test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Node IDs are <source-path>_<entity>, so a doc that merely references an entity
mints the entity's own ID and collides with the entity's node by construction.
The pre-dedup pass kept whichever arrived first, so chunk order decided whether
an entity kept its own attributes or a passing cross-reference's, and the
warning that fired (#1504) told the user a same-name-different-directory clash
had lost their data — while the one drop that really is lossy, two labels for
one ID from the same file, was silent because the warning was gated on
source_file differing.
The survivor is now the node whose source_file is the file its ID encodes (any
trailing slice of the path, so absolute, repo-relative, and pre-#1504
bare-stem IDs all resolve), falling back to first-seen. Reporting follows what
the drop actually costs: a cross-reference folding into the node it references
loses nothing (edges are keyed by ID and rewire to the survivor) and is silent;
a same-file relabel notes the discarded label; two files that both encode the
ID are distinct entities and still WARNING, now stating the ID-scheme cause
rather than a filename clash.
Refs #1851
When two LLM extraction chunks each process a file with the same name in
different directories, they independently generate the same node IDs and
deduplicate_entities() silently drops one node (first-writer-wins). The
data loss had no indication in any log, counter, or output.
Adds a stderr WARNING when a duplicate ID comes from a different
source_file, telling the user which files collided and recommending the
per-subfolder extract + merge-graphs workflow to avoid it.
Three pass-2 guards (mirrored in the --dedup-llm pair collection): block merges
when labels' embedded numbers differ as zero-padding-insensitive multisets;
block cross-file merges of file-anchored rationale/document nodes (same-file
still merges); and score cross-file long labels on plain Jaro instead of
Jaro-Winkler so the prefix bonus can't fabricate merges of shared-prefix but
token-divergent entities (jest-native vs react-native), while genuine cross-file
duplicates still clear Jaro and same-file near-duplicates keep Jaro-Winkler.
Co-Authored-By: van4oza <van4oza@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pass 2 selected the merge winner from the union of both normalized-label
groups, so never-compared same-label/cross-file nodes could be pulled
into the union, bypassing the #1046/#1178 guards. Pick the winner from
[node, neighbor] only; group members that belong together still merge
via pass 1 (same file) or their own verified comparison.
Co-authored-by: Cursor <cursoragent@cursor.com>
- analyze.py: pass length_bound=max_cycle_length to nx.simple_cycles() so
networkx prunes during enumeration instead of post-filtering; drops report
generation from never-returns to ~0.1s on dense graphs (#1196)
- llm.py: replace hardcoded min(40+16*n,4096) label_communities token budget
with _resolve_max_tokens(min(64+24*n,8192)) — 24 tok/community covers 5-word
JSON entries; 8192 cap fits 16k-context models; env var now honoured (#1200)
- dedup.py: add prefix-extension guard in Pass 2 and _llm_tiebreak — skip merge
when one normalised label is a strict prefix of the other (getActiveSession /
getActiveSessions, parseConfig / parseConfigFile). Option (a) rejected: dropping
the >=12 early-out from _short_label_blocked breaks test_typo_merged (#1201)
- tests/test_dedup.py: two new regression tests verifying prefix guard fires for
extension pairs and does not fire for same-length typo pairs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
