Files
rea/experiments/atomic-codemode/capability-map.md
T

129 lines
11 KiB
Markdown

# Capability and comparison scope
The experiment has nineteen fact methods behind a shared SDK/CLI owner, exposed
as twenty-two direct MCP tools or four Code Mode tools. The structured IR/call/
projection/switch facts require the explicitly pinned experimental Kuna patch;
they are not capabilities added to stock Kuna or integrated REA by registration
alone. Other providers are not wired to this experimental service.
A fair completed-task comparison first needs sufficient facts on both paths,
with a common correctness oracle. A complete universal RE interface is not
established here. The selected-case question requires native row/edge identities,
so it compares the two atomic adapters; no traditional-provider result for that
question is invented. Earlier traditional comparisons retain their own questions,
engine profiles and limitations in [the trial record](trials.md).
| Analyst question | Composable facts/relations | Executed evidence and limits |
| ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Which implementation owns an observed message? | Callable entries/aliases, encoded strings, exact-address xrefs with occurrence identity and function attribution | Both recent SQLite tasks select the owner before capture. Name equality alone is insufficient; discovery/unattributed coverage remains explicit. |
| Which local condition controls a selected value/call? | Snapshot-local operations, SSA values, indexed operands, definitions/uses with PHI predecessor edges, recovered CFG edges | [Four real control-task runs](savepoint-control-study.md) verify the selected guards and machine instructions. [Full graph transport checks](native-graph-transport-study.md) establish producer fidelity, not global machine equivalence. |
| What crosses a call boundary? | Native call specifications, argument values and separate passing storage; selected callee inputs and bounded definition projections | Earlier pinned [call/boundary trials](trials.md) exercise these relations. Storage matches are candidates, not proof of runtime binding or complete ABI/memory effects; a new exporter hash alone is not a repeated cross-function model trial. |
| Which cases map to a recovered edge? | Original case identity/label/address, independent native default, flattened case/out-edge relations | [All 182 SQLite original targets](switch-study.md) match independent ELF table bytes. Four real models answer selected cases and distinguish the native default from the machine range guard. No complete case inventory is needed in model context. |
| What machine bytes support the interpretation? | Batched loaded-image bytes and selected linear instruction windows, each with source/coverage | The case models independently derive signed relative targets. These windows do not establish runtime execution/alignment, a complete machine CFG, live memory or arbitrary file-offset mappings. |
| Can the answer be audited without returning the report? | Snapshot/source identity, selected fields, batch reads, paging and retained raw/decoded Evidence | Real CLI/MCP checks verify source access and capacity fallbacks. Scope/partial/unknown survive composition; a guest-derived answer keeps its own authority. |
The [string-span experiment](string-span-study.md) extends exact xrefs with
batched incoming target-coordinate spans. It preserves all eleven selected
SQLite data occurrences, including nine pooled suffix pointers, with individual
endpoints/owners and raw source rows. Kuna already had aggregate string-span
counts; the new occurrence query shares its sparse index. Range membership alone
does not establish whole-string use, access overlap or runtime behavior.
Three real models complete the pooled-string task. The traditional original
inventory has a partial source audit after client truncation; atomic source
captures are fully checked. Code Mode's cumulative input/time improve in this
sample, while uncached input and memory exceed Direct. This remains task evidence,
not provider parity or a stable adapter advantage.
The useful grain so far is an identified object plus a relation with its exact
qualifiers: an operand needs its slot and PHI binding; a case/out-edge pair needs
both identities; an address needs its artifact/address space; an optimized
operation needs its snapshot and representation. One small JSON row is not
automatically one independently meaningful fact. Machine instructions and
normalized operations also cannot substitute for each other.
Selection, projection, batching, paging and bounded traversal compose those
facts. The analysis owner derives/validates relations; Code Mode selects and
combines them. A query can be fine-grained even when its fresh native capture
still computes a whole function. Same-lease reuse is measured separately from
that work and does not establish a cross-session cache or native acceleration.
The [control-query study](control-query-study.md) independently verifies all
2,845 control-dependence triples in the real SQLite VM through both MCP adapters.
Two real models also complete a two-level controller/SSA/path-witness task and
distinguish alternative predicates from conjunction. Unknown CFG seeds are
explicit, and the snapshot owner reuses a bounded lazy postdominator index.
Code Mode has lower cumulative input but higher uncached input and runtime costs
in that pair; precise derived facts do not imply a preferred calling adapter.
The [OpenSSH value-query study](value-query-study.md) adds exact location lookup
and inverse operand occurrences for value identities, including inputs/constants
without definitions. It checks every retained selected version/use against the
complete source graph; this is separate from machine-byte evidence and does not
turn SSA occurrences into a memory-alias proof.
The [stripped OpenSSH model comparison](ssh-warning-study.md) also verifies a
common warning/caller/stack/guard question through the two atomic adapters and
the traditional Kuna workflow. Exact machine facts suffice for the direct
answer; extra SSA capture is not required by every RE question. The corrected
traditional baseline has its complete declared native toolset. This expands
the completed-task evidence while retaining the local scope and cost confounds.
Memory alias dependencies, full machine-flow recovery, indirect runtime binding,
feasible paths and nontermination remain unknown. The current evidence does
not establish provider parity, broad complex-software superiority or a stable
Code Mode cost advantage. The [wait repeat](client-wait-study.md) shows why fewer
MCP requests or polls cannot replace correct answers and whole-task costs.
The [path query](path-query-study.md) composes selected recovered edges into one
paged witness under explicit exclusions, with zero-edge/not-found/unknown
outcomes. Both real SQLite MCP lanes preserve every original edge field; complete
native function fidelity is checked. This removes an observed graph-reading
burden without establishing feasible runtime paths or an adapter cost advantage.
The completed path-query pair answers the unchanged two-level task correctly in
both actual models; each uses four path reads and three selected block reads,
without a complete graph inventory in model context. Direct and Code Mode have
similar wall time with mixed token/resource costs. Stock 1.833 explicitly rejects
the unsupported path query while preserving readable Evidence/code in a real SDK
check. These are scoped observations, not stock-provider parity or complete
runtime reachability coverage.
The [function-boundary study](function-boundary-study.md) adds independent native
prototype entries/return descriptors. On real stripped SSH, prototype counts
4/10 differ from raw input counts 3/44; both actual models preserve that distinction,
all exact matches/missing tuples and the selected caller-to-callee/helper transfer.
Call-passing storage is separate from the caller value's own coordinate. Native
model labels and recovered entries do not prove original source arity/ABI.
Complete old function fidelity is preserved except for the new facet. Actual
old-native/stock checks return explicit unavailability with readable Evidence.
The two completed model tasks have mixed token/time/resource costs; no stable
Code Mode advantage or provider parity follows.
The [storage study](storage-study.md) exposes native address-space descriptors
and exact varnode_xref names without function decompilation. Width-qualified
coordinates compose with prototype/value/passing-storage facts. Real x86-64
and AArch64 OpenSSH catalog checks do not establish big-endian, alias, runtime
or full provider parity coverage.
Both real models complete the extended native storage/prototype/call task.
Code Mode reduces cumulative/uncached input and wall time in that pair but uses
more local CPU and more inner fact reads. The shared metadata capability is
verified separately from this adapter observation; earlier mixed pairs remain.
The [AArch64 reference study](aarch64-xref-study.md) corrects an observed native
ADRP+ADD/LOAD/STORE gap through the shared PC-relative value owner. Real arm64
SSH checks verify both warning owners, callers, selected machine words and
77 GOT-slot reads. Coverage stays bounded and does not propagate live GOT
contents. Traditional C/assembly preflight can answer the same new behavior-level
question, avoiding a baseline missing necessary facts. All three real models
complete it, while token/time/resource tradeoffs remain mixed. That pinned run
preserves the recovered no-return disagreement and uses machine facts for the
post-call path.
The [return-coverage study](aarch64-return-study.md) fixes the BTI entry split
and a separate false no-return promotion through the native owners. The fresh
three-way task passes with the strict call restored in both atomic captures;
their complete selected function documents agree. Direct has lower input,
time and CPU than Code Mode in this sample. Memory, analysis-profile differences,
the common client executor and n=1 sampling prevent a universal adapter claim.