test(serve): add over-match guard for underscore token splitting (#2473)

The fix broadens seeding (splitting user_service into user + service), so add
the negative test the deep-dive flagged: an unrelated single-token node must not
out-rank the node matching the full multi-token query. (Note: _search_tokens
does not split camelCase, so the fix covers underscore/hyphen, not camelCase.)
Adds the CHANGELOG entry.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
safishamsi
2026-08-17 18:30:50 +01:00
co-authored by Claude Opus 4.8
parent 236ed9e548
commit 4be19990aa
2 changed files with 15 additions and 0 deletions
+1
View File
@@ -4,6 +4,7 @@ Full release notes with details on each version: [GitHub Releases](https://githu
## 0.9.46 (unreleased)
- Fix: `graphify query` treats `_` as a token separator (like `-`), so an underscore-spelled query (`user_service`) matches a hyphenated label (`user-service`); coverage-scaling keeps the broader tokenization from surfacing unrelated single-token noise (#2473, thanks @nadiadatepe-eng).
- Fix: the `post-checkout` hook skips its rebuild when HEAD is unchanged (e.g. `git checkout -b` with no start point), so creating a branch no longer triggers a full graph rebuild (#2421, thanks @nothariharan).
- Feature: Markdown nodes now carry a `node_kind` (`page` vs `heading`) attribute so a docs corpus can be filtered by kind, and leading YAML frontmatter is parsed onto the page node as bounded, sanitized attributes; a `#` comment inside frontmatter is no longer mis-extracted as a heading (thanks @evanthomasgelders). Node ids are unchanged, so existing markdown graphs are not re-keyed.
+14
View File
@@ -1697,3 +1697,17 @@ def test_snake_case_identifier_still_matches_itself():
scored = _score_nodes(G, _query_terms("_query_terms"))
assert scored and scored[0][1] == "n1"
def test_underscore_query_does_not_let_a_single_token_outrank_the_real_match():
"""Splitting on `_` broadens seeding, so an unrelated single-token node can now
be scored — but coverage-scaling/IDF must keep it from out-ranking the node
that matches the full multi-token query (the over-match guard for this fix)."""
G = nx.Graph()
G.add_node("real", label="user-service-client",
source_file="a.py", source_location="L1", community=0)
G.add_node("noise", label="user",
source_file="b.py", source_location="L1", community=1)
scored = _score_nodes(G, _query_terms("user_service_client"))
assert scored, "the multi-token query must match the full-label node"
assert scored[0][1] == "real", f"a single-token node out-ranked the real match: {scored}"