Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
129 lines
11 KiB
Markdown
129 lines
11 KiB
Markdown
# Capability and comparison scope
|
|
|
|
The experiment has nineteen fact methods behind a shared SDK/CLI owner, exposed
|
|
as twenty-two direct MCP tools or four Code Mode tools. The structured IR/call/
|
|
projection/switch facts require the explicitly pinned experimental Kuna patch;
|
|
they are not capabilities added to stock Kuna or integrated REA by registration
|
|
alone. Other providers are not wired to this experimental service.
|
|
|
|
A fair completed-task comparison first needs sufficient facts on both paths,
|
|
with a common correctness oracle. A complete universal RE interface is not
|
|
established here. The selected-case question requires native row/edge identities,
|
|
so it compares the two atomic adapters; no traditional-provider result for that
|
|
question is invented. Earlier traditional comparisons retain their own questions,
|
|
engine profiles and limitations in [the trial record](trials.md).
|
|
|
|
| Analyst question | Composable facts/relations | Executed evidence and limits |
|
|
| ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| Which implementation owns an observed message? | Callable entries/aliases, encoded strings, exact-address xrefs with occurrence identity and function attribution | Both recent SQLite tasks select the owner before capture. Name equality alone is insufficient; discovery/unattributed coverage remains explicit. |
|
|
| Which local condition controls a selected value/call? | Snapshot-local operations, SSA values, indexed operands, definitions/uses with PHI predecessor edges, recovered CFG edges | [Four real control-task runs](savepoint-control-study.md) verify the selected guards and machine instructions. [Full graph transport checks](native-graph-transport-study.md) establish producer fidelity, not global machine equivalence. |
|
|
| What crosses a call boundary? | Native call specifications, argument values and separate passing storage; selected callee inputs and bounded definition projections | Earlier pinned [call/boundary trials](trials.md) exercise these relations. Storage matches are candidates, not proof of runtime binding or complete ABI/memory effects; a new exporter hash alone is not a repeated cross-function model trial. |
|
|
| Which cases map to a recovered edge? | Original case identity/label/address, independent native default, flattened case/out-edge relations | [All 182 SQLite original targets](switch-study.md) match independent ELF table bytes. Four real models answer selected cases and distinguish the native default from the machine range guard. No complete case inventory is needed in model context. |
|
|
| What machine bytes support the interpretation? | Batched loaded-image bytes and selected linear instruction windows, each with source/coverage | The case models independently derive signed relative targets. These windows do not establish runtime execution/alignment, a complete machine CFG, live memory or arbitrary file-offset mappings. |
|
|
| Can the answer be audited without returning the report? | Snapshot/source identity, selected fields, batch reads, paging and retained raw/decoded Evidence | Real CLI/MCP checks verify source access and capacity fallbacks. Scope/partial/unknown survive composition; a guest-derived answer keeps its own authority. |
|
|
|
|
The [string-span experiment](string-span-study.md) extends exact xrefs with
|
|
batched incoming target-coordinate spans. It preserves all eleven selected
|
|
SQLite data occurrences, including nine pooled suffix pointers, with individual
|
|
endpoints/owners and raw source rows. Kuna already had aggregate string-span
|
|
counts; the new occurrence query shares its sparse index. Range membership alone
|
|
does not establish whole-string use, access overlap or runtime behavior.
|
|
Three real models complete the pooled-string task. The traditional original
|
|
inventory has a partial source audit after client truncation; atomic source
|
|
captures are fully checked. Code Mode's cumulative input/time improve in this
|
|
sample, while uncached input and memory exceed Direct. This remains task evidence,
|
|
not provider parity or a stable adapter advantage.
|
|
|
|
The useful grain so far is an identified object plus a relation with its exact
|
|
qualifiers: an operand needs its slot and PHI binding; a case/out-edge pair needs
|
|
both identities; an address needs its artifact/address space; an optimized
|
|
operation needs its snapshot and representation. One small JSON row is not
|
|
automatically one independently meaningful fact. Machine instructions and
|
|
normalized operations also cannot substitute for each other.
|
|
|
|
Selection, projection, batching, paging and bounded traversal compose those
|
|
facts. The analysis owner derives/validates relations; Code Mode selects and
|
|
combines them. A query can be fine-grained even when its fresh native capture
|
|
still computes a whole function. Same-lease reuse is measured separately from
|
|
that work and does not establish a cross-session cache or native acceleration.
|
|
|
|
The [control-query study](control-query-study.md) independently verifies all
|
|
2,845 control-dependence triples in the real SQLite VM through both MCP adapters.
|
|
Two real models also complete a two-level controller/SSA/path-witness task and
|
|
distinguish alternative predicates from conjunction. Unknown CFG seeds are
|
|
explicit, and the snapshot owner reuses a bounded lazy postdominator index.
|
|
Code Mode has lower cumulative input but higher uncached input and runtime costs
|
|
in that pair; precise derived facts do not imply a preferred calling adapter.
|
|
|
|
The [OpenSSH value-query study](value-query-study.md) adds exact location lookup
|
|
and inverse operand occurrences for value identities, including inputs/constants
|
|
without definitions. It checks every retained selected version/use against the
|
|
complete source graph; this is separate from machine-byte evidence and does not
|
|
turn SSA occurrences into a memory-alias proof.
|
|
|
|
The [stripped OpenSSH model comparison](ssh-warning-study.md) also verifies a
|
|
common warning/caller/stack/guard question through the two atomic adapters and
|
|
the traditional Kuna workflow. Exact machine facts suffice for the direct
|
|
answer; extra SSA capture is not required by every RE question. The corrected
|
|
traditional baseline has its complete declared native toolset. This expands
|
|
the completed-task evidence while retaining the local scope and cost confounds.
|
|
|
|
Memory alias dependencies, full machine-flow recovery, indirect runtime binding,
|
|
feasible paths and nontermination remain unknown. The current evidence does
|
|
not establish provider parity, broad complex-software superiority or a stable
|
|
Code Mode cost advantage. The [wait repeat](client-wait-study.md) shows why fewer
|
|
MCP requests or polls cannot replace correct answers and whole-task costs.
|
|
|
|
The [path query](path-query-study.md) composes selected recovered edges into one
|
|
paged witness under explicit exclusions, with zero-edge/not-found/unknown
|
|
outcomes. Both real SQLite MCP lanes preserve every original edge field; complete
|
|
native function fidelity is checked. This removes an observed graph-reading
|
|
burden without establishing feasible runtime paths or an adapter cost advantage.
|
|
|
|
The completed path-query pair answers the unchanged two-level task correctly in
|
|
both actual models; each uses four path reads and three selected block reads,
|
|
without a complete graph inventory in model context. Direct and Code Mode have
|
|
similar wall time with mixed token/resource costs. Stock 1.833 explicitly rejects
|
|
the unsupported path query while preserving readable Evidence/code in a real SDK
|
|
check. These are scoped observations, not stock-provider parity or complete
|
|
runtime reachability coverage.
|
|
|
|
The [function-boundary study](function-boundary-study.md) adds independent native
|
|
prototype entries/return descriptors. On real stripped SSH, prototype counts
|
|
4/10 differ from raw input counts 3/44; both actual models preserve that distinction,
|
|
all exact matches/missing tuples and the selected caller-to-callee/helper transfer.
|
|
Call-passing storage is separate from the caller value's own coordinate. Native
|
|
model labels and recovered entries do not prove original source arity/ABI.
|
|
Complete old function fidelity is preserved except for the new facet. Actual
|
|
old-native/stock checks return explicit unavailability with readable Evidence.
|
|
The two completed model tasks have mixed token/time/resource costs; no stable
|
|
Code Mode advantage or provider parity follows.
|
|
|
|
The [storage study](storage-study.md) exposes native address-space descriptors
|
|
and exact varnode_xref names without function decompilation. Width-qualified
|
|
coordinates compose with prototype/value/passing-storage facts. Real x86-64
|
|
and AArch64 OpenSSH catalog checks do not establish big-endian, alias, runtime
|
|
or full provider parity coverage.
|
|
|
|
Both real models complete the extended native storage/prototype/call task.
|
|
Code Mode reduces cumulative/uncached input and wall time in that pair but uses
|
|
more local CPU and more inner fact reads. The shared metadata capability is
|
|
verified separately from this adapter observation; earlier mixed pairs remain.
|
|
|
|
The [AArch64 reference study](aarch64-xref-study.md) corrects an observed native
|
|
ADRP+ADD/LOAD/STORE gap through the shared PC-relative value owner. Real arm64
|
|
SSH checks verify both warning owners, callers, selected machine words and
|
|
77 GOT-slot reads. Coverage stays bounded and does not propagate live GOT
|
|
contents. Traditional C/assembly preflight can answer the same new behavior-level
|
|
question, avoiding a baseline missing necessary facts. All three real models
|
|
complete it, while token/time/resource tradeoffs remain mixed. That pinned run
|
|
preserves the recovered no-return disagreement and uses machine facts for the
|
|
post-call path.
|
|
|
|
The [return-coverage study](aarch64-return-study.md) fixes the BTI entry split
|
|
and a separate false no-return promotion through the native owners. The fresh
|
|
three-way task passes with the strict call restored in both atomic captures;
|
|
their complete selected function documents agree. Direct has lower input,
|
|
time and CPU than Code Mode in this sample. Memory, analysis-profile differences,
|
|
the common client executor and n=1 sampling prevent a universal adapter claim.
|