The extraction spec stores the WHY of a concept as a `rationale` attribute
on the node — deliberately not as a node of its own. `_score_query` matched
a question against `norm_label`, `label_tokens`, `source_file` and the node
id only, and `_node_search_text` indexed the same fields, so for a "why does
X …" question the node holding the answer was neither a trigram candidate
nor a seed unless the asker already knew its label (#2293: 0/5 answers
reached on a 5.5k-node corpus although every answer node existed and stated
the answer in its rationale).
Add a rationale tier, following the fix the issue proposes:
- `_RATIONALE_MATCH_BONUS = 0.75`: below the label substring tier (1.0),
above the source-path tier (0.5). A term found in the rationale adds to
the score and to the per-term singleton used for seed seating, but — like
the source tier — never to term coverage, so a long rationale adds recall
without winning back an exact-label tier it did not earn.
- `_node_rationale_text`: one normalizer (diacritics folded, lower-cased),
tolerant of the list an extractor may emit; "" when absent.
- `_node_search_text` appends the same text as a trailing field, only when
present, so the trigram prefilter stays a complete candidate generator and
every existing field position (#2467) is unchanged.
Nodes without a rationale score exactly as before.
Design credit: @nuboxworld-byte (issue #2293 and #2294). #2294 implements
the same idea but its diff carries the whole repository (+252k lines, 766
files) from a base mismatch and cannot be reviewed or merged as-is; this is
a fresh minimal implementation against v8.
Tests: rationale-only match ranks; tier order label-substring > rationale >
source; no coverage credit; list-valued attribute; search text carries the
field (and is unchanged without it); end to end a "why" question phrased
from the rationale seats the node as a seed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 95ff41101a)
Andy Tsai
6377f1609d