Constrained query expansion (#998)

Add constrained query expansion step to /graphify query skill

## Problem

`graphify query` matches via case-folded substring + IDF — no stemming, no synonyms, no cross-language match. When the user's question uses different vocabulary than the graph labels (Slavic → English, "handlers" → "handler", "обработчик" → "handler"), the literal matcher returns 0
hits and the LLM consumer either gets empty subgraph or improvises an ungrounded keyword list from training memory (e.g. expanding "auth" to `{passport, sso, saml, oauth, jwt, scim, …}` regardless of whether those tokens exist in the corpus).

## Fix

Adds a `Step 0 — Constrained query expansion` block to the skill's `/graphify query` section. The LLM consumer extracts vocabulary from graph labels (CamelCase/snake_case split, length-filtered) and is instructed to pick **only** tokens present in that vocabulary, explicitly forbidden from inventing terms.

Effects:
- Bounded improvisation — fantom tokens (terms not in corpus) cannot be expanded, even when LLM "knows" they're related to the intent.
- Honest negative signal — if vocab is poor on a query's topic,  expansion returns [] and the LLM tells the user, instead of  fabricating a search.
- Auditability — selected tokens are printed to the user, and saved into `save-result` for the next --update to graph as Q&A nodes.

## Scope

Patches the canonical `graphify/skill.md`. The 11 host-variant skills (skill-codex.md, skill-aider.md, …) follow the same query-section contract but inline Python rather than calling `graphify query` CLI; those need a parallel patch with the inline form. Happy to follow up in a separate PR after review on the canonical patch.

## Test

On a graph built from the graphify repo itself (1284 nodes, 1454 vocab tokens), an unconstrained expansion of "укрупненная архитектура аутентификации" yields {auth, oauth, jwt, saml, sso, ldap, scim, mfa, 2fa, pin, passport, session, login, token} — of which 11/15 are absent
from the corpus. Constrained expansion against the actual vocab yields {credential, security, token, signature, user, architecture, component, module, overview} — 9 tokens, 0 fantom. Same retrieval, dramatically higher precision.
This commit is contained in:
eugene-krivosheyev
2026-05-24 21:38:13 +02:00
committed by GitHub
parent ab4e5424ca
commit 238702b697
+45 -4
View File
@@ -965,20 +965,61 @@ Two traversal modes - choose based on the question:
| BFS (default) | _(none)_ | "What is X connected to?" - broad context, nearest neighbors first |
| DFS | `--dfs` | "How does X reach Y?" - trace a specific chain or dependency path |
### Step 0 — Constrained query expansion (REQUIRED before traversal)
graphify's `query` CLI matches nodes via case-folded substring + IDF — there is **no stemming, no synonyms, no cross-language match** inside the binary. If the user's question uses different language or different domain vocabulary than the graph's labels (user says "обработчик" / graph says "handler"; user says "authentication" / graph says "Guardian"), the literal matcher returns 0 hits and the answer collapses to noise.
Fix this **without inventing tokens** by expanding the query against the actual graph vocabulary first:
1. Extract the token vocabulary from node labels:
```bash
$(cat graphify-out/.graphify_python) -c "
import json, re
from pathlib import Path
data = json.loads(Path('graphify-out/graph.json').read_text())
vocab = set()
for n in data['nodes']:
for c in re.findall(r'[A-Za-z]+', n.get('label','') or ''):
for p in re.findall(r'[A-Z]+(?=[A-Z][a-z])|[A-Z]?[a-z]+|[A-Z]+', c):
t = p.lower()
if 3 <= len(t) <= 30:
vocab.add(t)
Path('graphify-out/.vocab.txt').write_text('\n'.join(sorted(vocab)))
print(f'vocab: {len(vocab)} tokens')
"
```
2. Read `graphify-out/.vocab.txt`. Then for the user's question, select **up to 12 tokens from this exact list** that semantically match the query intent. Hard constraints:
- You MUST pick only tokens present in the vocabulary file. Do NOT invent tokens.
- If a query concept has no plausible token in the vocab, skip it — do not substitute a near-synonym from training memory.
- If **no** vocab tokens match the query at all, output an empty list and tell the user the corpus has no relevant vocabulary for this question. Do not fabricate a search.
- Translate cross-language: Russian "аутентификация" → look for `auth`, `credential`, `token`, `security` IFF present in vocab.
- Morphology: "handlers" maps to `handler` IFF present; "todos" maps to `todo` IFF present.
3. Print the selection explicitly to the user before running the query, so the expansion is auditable:
```
Query expanded to (from graph vocab, N tokens): [token1, token2, ...]
```
If the list is empty, say so plainly and stop — do not proceed to traversal.
### Step 1 — Traversal
Build the **expanded query string** by joining the selected tokens with spaces. Use this string as `QUESTION` below — NOT the original user question. (The original question is preserved only for `save-result` at the end.)
```bash
graphify query "QUESTION"
# or: graphify query "QUESTION" --dfs --budget 3000
```
Replace `QUESTION` with the user's actual question. Answer using **only** what the graph output contains. Quote `source_location` when citing a specific fact. If the graph lacks enough information, say so - do not hallucinate edges.
Answer using **only** what the graph output contains. Quote `source_location` when citing a specific fact. If the graph lacks enough information, say so - do not hallucinate edges.
After writing the answer, save it back into the graph so it improves future queries:
After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node:
```bash
$(cat graphify-out/.graphify_python) -m graphify save-result --question "QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2
$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2
```
Replace `QUESTION` with the question, `ANSWER` with your full answer text, `SOURCE_NODES` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph.
Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph.
---