rea/experiments/atomic-codemode

Atomic RE through Code Mode

This is an in-progress experiment, not an integrated provider release. Branch experiment/atomic-codemode-20261016 starts at main 42fc6c53ae188978deb3bedfe88287b6b316ba22 and carries the previous atomic-facts prototype. Historical measurements in ../atomic-re-tools/ retain their original baselines. No PR or merge is intended.

The branch was subsequently refreshed from main 60bf10245. Shared installation/file cleanup and string parsing now follow that main's owners. Historical model measurements retain their original frozen runtimes; the refresh has separate real-provider verification and resource receipts.

The goal is a real model completing reverse-engineering questions through composable facts, with Code Mode as the calling adapter. A configured feature flag, a scripted verifier or a small response does not prove that goal.

The capability map distinguishes verified task coverage, experimental exporter requirements and remaining unknowns before interpreting adapter comparisons.

The string-span study adds paged incoming occurrence queries for encoded object spans and independently verifies pooled suffix references on real SQLite. The same native sparse range index serves Kuna's existing string aggregates. Equivalent cold occurrence retrieval falls from 48.1 to 1.8 seconds; this is shared analysis reuse, not a Code Mode result. The subsequent frozen real-model runs all answer the selected task correctly: traditional takes 315.6 seconds, Direct 102.3 and Code Mode 93.6. Code Mode uses less cumulative input but more uncached input and peak memory than Direct. Traditional's full inventory is truncated by the client; one original cited payload remains only partially auditable. The study preserves that limitation.

The string-fact precision study preserves native NUL termination as true/false/unknown through the shared domain and public output contract. Real canonical MCP, SDK, Direct, Code Mode and CLI checks retain both NUL-terminated strings and a nonterminated printable run without length-based agent inference. The old coarse descriptor remains compatible. Both fresh real models complete the same frozen task after this repair: Direct takes 95.2 seconds, Code Mode 90.9. Code Mode lowers cumulative input and peak memory but increases uncached input and CPU. The results do not isolate the metadata change's performance effect; all source and usage checks are recorded.

The large-result study verifies that paging alone does not bound a large field. String metadata can now be selected without text, while SDK full reads remain available. Shared pre-allocation JSON checks preserve context and Evidence on transport refusal, including in-program Code Mode recovery. The synthetic fixture tests capacity; it does not establish real-app analysis quality. Real SQLite/SSH regressions are reported separately. The subsequent real Code Mode model passes the frozen capacity task. Direct retrieves correct data but labels two 17-byte prefixes as 16, failing the exact reporting criterion. Both use metadata selection without sending full blobs. Code Mode uses less input but more wall time; the differing correctness and byte window choices prevent a matched-quality efficiency ranking. All runs remain. The streamed inventory study then restores real FFmpeg queries whose complete native source exceeds 8 MiB, using the production streaming parser and bounded retained files. Real SDK/MCP lanes pass; LLVM and Node still exceed the existing 64 MiB budget. Those failures remain explicit. Both real models now complete the FFmpeg task with all cited raw captures verified. Code Mode uses less cumulative input but more uncached input and time. The study also retains two evaluator namespace failures and their full costs; the corrected pair uses actual registered tool discovery. The refusal-cache study then avoids repeated native work for proven immutable source-budget failures. Real SDK and both MCP adapters verify reuse and readable partial Evidence; cancellation/cleanup uncertainty remain uncached. The first Code Mode precondition failure and its cost remain in the record. Larger-target inventory coverage is still unresolved.

The later native query study makes selected LLVM and Node string queries answerable without retaining a whole oversized inventory. Both frozen LLVM model answers pass; Code Mode uses fewer input tokens in that pair, with only a modest wall-time difference. The shared byte checkpoint then batches missing byte windows through one native program while retaining per-window facts and Evidence. A same-engine SDK toggle reduces ten-window cold retrieval from about 94.3 to 8.5 seconds. Real provider preflight passes; new model trials for this byte change were not run because the user cleared the goal.

The AArch64 reference study finds and fixes missing ADRP-based string and memory references before the next model comparison. Real SDK and traditional MCP preflights now agree with independent machine evidence; the shared bounded producer preserves space/width and uncertainty. All three real models now complete the behavior-level task. Code Mode takes 151.3 seconds / 388,365 cumulative input tokens, Direct 164.9 / 515,929 and traditional 345.7 / 681,613. Code has more uncached input than traditional and more memory than Direct. These are the earlier pinned measurements.

The return-coverage study subsequently fixes two independent producer defects: BTI import-entry identity and false no-return promotion that erased the post-warning path. All three fresh model answers pass the same selected task. Direct uses 351,157 cumulative input tokens / 139.0 seconds, Code Mode 371,571 / 166.8, and traditional 662,683 / 263.1. Code Mode reduces MCP requests but adds model responses in this sample; correct primitives and fewer RPCs do not establish a stable adapter advantage.

The architecture storage study adds exact native register-name and address-space queries, sharing one bounded immutable capture per lease. It separates coordinates, widths and native map misses from hardware absence, SSA equality or alias claims. Both real SSH models complete the extended task; Code Mode uses 307,888 versus 415,450 cumulative input tokens and 142.2 versus 158.1 seconds, but more local CPU. This single pair does not establish a stable advantage; the full typecheck gate remains resource-limited.

The function-boundary study adds independent native prototype entries and return descriptors, separated from raw input varnodes. Both real SSH models correctly distinguish these inventories and compose the selected caller/parameter/helper transfer. Direct uses 331,674 cumulative input tokens / 118.4 seconds; Code Mode uses 359,789 / 125.7, with lower uncached input but mixed overall costs. Native and real-provider checks are recorded there.

The path-query study adds a shared, bounded recovered-CFG witness query with exclusions and paging. Both actual MCP adapters verify the same four real SQLite witnesses without requiring the model to fetch the entire graph. This establishes graph-relative facts; predicate feasibility stays unknown. Both real models now use the query and answer correctly on the same frozen controller task: direct takes 200.8 seconds / 495,895 cumulative input tokens; Code Mode takes 203.3 seconds / 554,477. Code Mode has lower uncached input, but no stable overall advantage is established. All prior pairs stay reported.

The control-query study verifies every derived control triple on the real SQLite VM, fixes unknown CFG seeds and reuses a lazy postdominator index. Its two-level investigation is frozen before model runs and separates alternative path predicates, SSA identity and finite recovered-edge witnesses from runtime claims. Both real-model answers now pass: Code Mode reduces cumulative input but increases uncached input, time, CPU and memory in this pair. Correct facts support both calling strategies; the query-cache improvement is separate from those mixed whole-task costs.

The discovery-summary study addresses repeated metadata from that pair. The complete method index now uses owned intent summaries; all full contracts and advertised tools remain available. Contracts and session admission have separate owners, with native loading deferred to open. Its repeat uses the same frozen control question and retains the earlier results.

That completed repeat again answers correctly in both lanes. The smaller index does not establish an overall task improvement: actual help use adds a Code Mode call, and cumulative/uncached tokens, local resources and end-to-end time have mixed changes. A large observed inter-call gap remains charged and unexplained.

The exact-value query study adds location-to-value and value-to-operand-use queries, including inputs/constants without defining operations. Real stripped OpenSSH checks preserve all 250 SSA versions / 369 occurrences at one selected policy location while reducing an equivalent core read task from 244 queries to six. A subsequent real-agent OpenSSH comparison completes the same machine-supported task through all three interfaces, after correcting an incomplete traditional evaluator toolset. Atomic direct has the lowest measured costs in this sample, using machine facts without decompilation; Code Mode uses the new queries but takes extra analysis steps. General efficiency claims still require broader evidence.

The reference is Hex-Rays IDA MCP: an on-demand reference lookup and an execution entrypoint over a richer analysis API. This experiment uses JavaScript and a QuickJS WASM environment rather than Python in the host process. It exposes only declared static analysis capabilities. It does not add target execution, patching or GUI control.

Three boundaries own the implementation:

• provider.mjs, native-ir.mjs: existing REA native target/process owners and parsed Kuna string/reference/SSA facts, retained sources, freshness and cleanup.
• api.mjs: named, validated fact/relationship contracts and their intent summaries. session.mjs owns shared admission and lease/file cleanup, loading the native provider only on open. SDK callers, CLI scripts and both MCP adapters use these same contracts and application owner.
• executor.mjs, server.mjs, cli.mjs: bounded JavaScript composition and presentation. No analysis is reimplemented in the execution environment.

The complete Code Mode MCP catalog is open_binary, reference, execute_javascript, close_binary. Setting REA_ATOMIC_ADAPTER=direct exposes the same fact API as facts_* MCP operations for comparison. This is a distinct experimental server; the production REA catalog is not filtered or replaced.

Use the pinned Node, an existing REA build and a caller-supplied Kuna 1.833:

npm ci --prefix experiments/atomic-codemode --ignore-scripts
node --test experiments/atomic-codemode/*.test.mjs
REA_KUNA_COMMAND=/absolute/path/to/kuna node experiments/atomic-codemode/server.mjs

The local MCP registration for real model trials is named rea-codemode. agent-trial.py --mcp-entrypoint /absolute/path/to/server.mjs --server-name rea-codemode --protocol-profile atomic --profile-parent /tmp creates an isolated configuration, home and empty working directory for the model. Shell/file tools, web search and delegation are disabled. No repository or oracle is placed in that workspace. The copied authentication file is removed with the isolated profile. Model output, actual MCP calls, original errors and usage remain in the trial directory. REA_ATOMIC_TRACE and REA_ATOMIC_EVIDENCE_DIR retain actual API calls and complete sources outside the model workspace when explicitly selected by the evaluator. The ambient environment and credentials are not logged.

Current bounds: 64 MiB guest heap, 32 KiB code and script value/error payloads, 64 host calls per execution, 120 seconds wall deadline and 2 seconds of guest synchronous execution. Native requests use existing supervision and a 1 GiB function-memory limit. Buffered formats and diagnostics keep an 8 MiB bound; string inventories stream through the production reader into the existing 64 MiB per-target source budget, without an aggregate stdout string. Two resident function indexes each derive from at most 2 MiB diagnostic or 8 MiB native wire source. Compressed native graphs admit at most 32 MiB decoded JSON plus independent row/text/relationship bounds. One decoded Evidence text view up to 32 MiB is also retained per lease; these source bounds are not parsed-heap size guarantees. Sources are content-digested and retained until close; explicit evaluator copies survive close. Capacity failures are explicit, not clipped facts. Leases and snapshot handles prevent cross-session joins.

The current experimental MCP profile also caps encoded tool-result bodies at 1 MiB, including text escaping and metadata; JSON-RPC/client framing is separate. reference.transport_limits advertises this policy in both modes. The QuickJS bridge remains 2 MiB. CLI/SDK transports have their own bounds; SDK full selected string values remain available. Prefer strings({query,fields:[]}) for metadata.

Invocation context and observed source IDs are additional bounded receipt metadata; the script payload bound is not a 32 KiB whole-MCP-wire limit. A real MCP probe found that a 40,000-byte guest exception bypassed that payload policy and produced a 40,183-byte response. Oversized errors now return explicit output_budget with omission metadata. Escaping expansion is checked too; ordinary smaller errors remain intact. verify-executor.mjs confirms real MCP delivery and continued access to source Evidence collected before the exception. It measures response bytes, not model tokens or an analysis speedup. Targeted JS tests at this stage totaled 42, alongside the separate real-provider and native-writer checks.

The diagnostic SSA format still has incomplete CFG and phi predecessor coverage. The opt-in native recovered IR experiment adds full recovered CFG edges, exact value/operand identities, phi incoming-edge bindings, selected values/blocks reads and derived control dependence. It requires an explicitly pinned patched engine; stock Kuna does not gain those capabilities from a wrapper. Runtime successors, memory aliases and machine-semantic equivalence remain unknown. Other providers are not yet wired to this experimental service.

The initial actual model comparison favors the focused traditional workflow. The seeded value-origin task has lower cumulative input and wall time through Code Mode, but initially exposes an IR-to-machine-path error. The revised trial correctly explains that machine path with lower observed task costs; broader claims still require more than this single seeded task.

The large SQLite VM diagnostic task also completes when its native graph exceeds capacity. Atomic direct beats Code Mode on cumulative input and wall time in that sample, and a traditional reused-project run is fastest. The actual CLI client already exposes JavaScript dispatch in every lane, regardless of requested feature flags. MCP request counts therefore do not establish fewer model rounds; the ledger now records actual model responses and client dispatch/wait calls.

instructions reads selected machine byte/decode windows through the same owner. Recovered operations use origin_address and blocks origin_start; these locations cannot be substituted for machine branch destinations. The value-origin trial demonstrates why this boundary matters. The experiment now has nineteen fact methods, still behind the same four-tool Code Mode catalog.

calls connects recovered call operations to native target specifications and separates argument SSA values from passing storage. values(is_input:true) selects retained callee inputs for exact storage comparisons. These candidate links preserve snapshot identities, missing specifications, unknown storage and runtime-binding limits. The cross-function task and oracle are fixed before its real-model trials; the graph wire format is version 3, with independent version-1 call facts.

references now selects incoming exact-address references or outgoing instruction/function references, with native kind filtering, field selection and paging. Source Evidence plus source_row identifies an occurrence. Function applicability, excluded/unattributed rows, indirect targets and discovery stay explicit. fields:[] requests only the always-retained identity columns across projection contracts. The reference question and independent oracle cover real SQLite VM aggregates and a machine call omitted from optimized IR.

bytes reads explicit virtual-address ranges as native loaded-image hex without decoding them as instructions. Batched reads preserve short reads and individual failures, with 64 KiB aggregate selected-byte capacity to bound hex/bridge growth. Complete hexdumps remain in Evidence. The original-file-offset mapping and live runtime memory remain unknown. This closes a direct byte-reading contract gap for data tables; it does not establish arbitrary memory-alias analysis.

functions provides selected name/alias/entry reads. Native snapshots report code and recovered-IR facets separately, preserving readable code when graph extraction is unavailable. Real capacity and cold/warm MCP checks are documented in native-engine.md. Whole-model costs and unequal-depth comparisons remain separate in the trial report.

The corrected SQLite comparison reaches the formatter implementation in all three interfaces. Code Mode uses fewer calls than atomic direct in that sample; a traditional run reusing its persistent whole-project cache is faster than Code Mode's fresh lease. Cache state, completed-answer coverage and actual uncached tokens must therefore accompany any cost comparison.

SDK views share immutable parsed inventories with their owner. A real SDK probe initially changed a returned string and made the next query disappear under the same Evidence ID. Freezing at the cache boundary fixes that producing rule, without copying or parsing the entire inventory for each read. verify-sdk.mjs checks the real provider's string/metadata/function views and complete retained source digest. The fixed probe passes under the bounded runner (1.328 seconds, 1.144 family CPU seconds, 110.6 MiB kernel family peak, zero OOM kills and verified payload cleanup); it is a correctness check, not a model-cost measurement.

Encoded size is another real coverage boundary: two SQLite functions initially lose even their C because pretty JSON exceeds transport capacity. The native patch now uses bounded compact serialization and a graph-only failure when needed. Parser's full graph fits; Pragma retains identical C with an explicit missing graph. The 8 MiB bound remains. See the exact sizes, rebuilt engine digest and verification scope in native-engine.md.

Completion requires actual model Code Mode use, independently checked task answers, shared atomic/direct/Code Mode semantics, needed relation coverage, failure/cancellation/identity tests, and equivalent completed-task measurements including discovery/retries, actual model usage, cold/warm analysis, CPU and process-family peak memory. Negative cases such as the small SQLite function must remain in the comparison.

The outgoing-reference real-model pair now completes with identical required facts and explicit coverage limits. In its single sample, Code Mode uses 264,680 cumulative input / 93.837 seconds versus direct's 285,086 / 106.183. The shared CLI/API also verifies the VM aggregates on unpatched stock Kuna 1.833. These are scoped positive results alongside the earlier mixed adapter comparisons, not proof of general superiority. Details and retained failed/partial attempts are in trials.md.

Recovered calls and their boundary values now survive full-graph capacity failure. The real VM capture preserves 416 call operations and 1,294 boundary values while its graph remains unavailable. Values clearly state their partial scope; definition IDs do not imply readable defining operations. See the boundary question and oracle and its completed real-model pair in trials.md. Both answers pass, using one VM capture each. Code Mode's single-sample latency is lower, with mixed context costs; broader efficiency remains unproven. The focused JS suite at that stage passes 49 tests.

Available snapshot call facets now include by_opcode known/unknown-target counts and their own coverage. The owner derives them once from the same validated immutable calls used by selected reads, including dead operations; target knowledge is independent of opcode. Empty available facets report zeros, while unavailable facets omit counts. This small aggregate addresses an observed cost in the real boundary pair: paging the entire call set just to count it. Both subsequent real-model answers satisfy the unchanged question/oracle without that paging. In the new sample direct is faster and lower on cumulative/uncached input than Code Mode, reversing the preceding latency ordering. Whole-task effects remain mixed; see the complete results. This aggregate is a shared fact-layer improvement, available to either adapter.

The shared owner also checks the native cleanup Result before releasing its lease or clearing retained state. A tagged cleanup failure now stays retryable; MCP errors retain the lease context and reported resources. The real SDK failure-injection verifier first reproduces the old false success, then verifies both close and target replacement retain the staged file/owner until a successful retry. Both public MCP adapters preserve that error and retry successfully. These use an injected tagged failure, not an observed engine cleanup incident; native reads and subsequent cleanup use the real provider. See the recorded scope and resource receipts in native-engine.md.

A subsequent diagnostic probe measures the VM's full 17 MB recovered graph and much smaller selected definition neighborhoods. It supports trying a bounded operation/value facet, while preserving unresolved definitions and unavailable CFG. The following implementation now makes that facet callable; the original diagnostic remains separate from public results.

snapshot.facets.operations distinguishes a complete recovered graph from a bounded call-rooted definition projection. operations, values, selected definition/use relations and slice share the same parsed operation owner in either case. Projection coverage preserves unexpanded definition counts and selected slice frontier reasons; CFG/controller reads remain unavailable. Native operation reads retain is_dead and literal operand/value identities. Coverage returns compact counts and bounded issue text, with complete native details retained in Evidence. The new question/oracle are fixed before real-model trials of the PHI format alternatives.

Both first PHI-format model answers pass the full oracle using one VM analysis and a six-operation selected trace, with the global graph/use limits intact. The trace reveals two format sources at each shared call, verified against byte and machine evidence. Costs and source review also record a direct documentation-lookup miss; named discovery needs fixing before rerunning the same task. No general adapter speedup follows from that first pair.

Named reference lookup now accepts canonical IDs or exact displayed names, including space/comma-separated lists. One presentation map supplies direct registration and documentation names; unmatched terms remain explicit. Selected schemas come from named contracts, use input semantics (defaulted fields stay optional), and are cached immutably by Zod schema identity for the process. Validation stays in the API registry; documentation has its own module. Both actual MCP catalogs and all fourteen reference schemas validate. The unchanged PHI question/oracle were rerun: both answers pass again, but direct has a native timeout followed by a successful retry. Complete costs and the unknown timeout cause remain in trials.md; this does not isolate an adapter speedup.

The next observed atom boundary is a wide PHI. One real VM operation has 156 inputs; its old default single-node slice is about 44 KB and exceeds the guest's 32 KiB result budget. Native operation/slice defaults now return opcode, output and exact input/definition/constant counts. Explicit detailed fields still work. relations({kind:"operands",seeds:[id],input_indices:[0,105,155]}) reads selected slots; omitting the selector pages all slots. Each slot retains value/definition and phi predecessor/edge identity, including null values and definition-free constants/inputs. Counts concern recovered facts, not source function arity.

Full and projected native operations use the same owner. Relationship paging counts existing groups before materializing only the requested page; identity reads deduplicate requested IDs in caller order. Diagnostic SSA explicitly lacks native operand relations. The discriminated relation schema advertises the same seed types and selector applicability enforced at runtime. The fourteen-method API and both adapters remain shared; no new engine or output-budget increase is needed. The seeded wide-operand task/oracle is separate from the earlier PHI-format question and is pinned before model trials.

Both actual wide-operand model answers now pass that fixed oracle, including distinct edges sharing a value and absent-slot versus unretained-seed semantics. Each performs one native analysis and reads only selected slots, distinct values and their immediate definitions. Code Mode takes 95.699 seconds versus direct's 91.817, with nearly equal cumulative input and lower uncached input. Its four discovery lookups prevent fewer MCP requests from becoming fewer model responses. These mixed costs validate the fact boundary while leaving broad efficiency unproven. The current shared JS suite passes 57 tests; the Python cost-ledger suite passes three.

Discovery now also offers reference({view:"index"}): the complete fourteen method names and descriptions, without schemas or return contracts. Follow with view:"contracts" and exact names for only the needed inputs. The default contract view and named/topic lookup stay compatible. Both views use the same registry and presentation map; index access never constructs unrequested schemas. The index describes declared methods, while snapshot remains the live coverage authority. This addresses observed blind discovery without inventing aliases or changing analysis facts; its whole-task effect must be measured on the unchanged wide-operand question.

The identical question now completes correctly through both indexed adapters. Code Mode uses the intended index plus one named lookup; discovery drops from four calls to two, without changing the seven fact reads. The new pair takes about 98.8 seconds each; cumulative input favors direct, uncached input favors Code Mode. All four wide-operand answers pass, with no retry or OOM. Costs remain mixed. A selected-slot read still incurs whole-function native decompilation on a fresh lease; phase profiling is the next computation question, separate from the now finer fact/query boundary.

The native phase study now measures the VM drive at about 34 seconds, C rendering at 0.22 seconds and projection at 0.03 seconds. Repeated new native processes do not gain the API owner's warm result reuse. Its subsequent bounded codec implementation now returns the complete 21,349-operation VM graph in about 3.79 MB, inside the unchanged 8 MiB wire budget. Selected definitions/CFG reads and decoded Evidence work through both real MCP adapters; all graph fields and ordinary C/calls match the prior complete golden. Same-lease reuse is about 67 ms after a 39.7-second fresh capture, with resource limits and no OOM. The next question/oracle start from names/error messages and exercise the new condition/CFG coverage. Two real-model pairs, including a discovery-guided repeat, answer both local guards correctly without a whole-report fetch. Code Mode uses fewer MCP requests, but whole-task costs are mixed and a large capture time variation remains unexplained. Both clients already provide JavaScript orchestration. Source fidelity is not machine equivalence, provider parity or a general speedup. The native switch experiment now exports and independently checks 182 labels over those 149 targets, exposes selected case-to-edge relations and keeps them readable when a 2 MiB native output budget omits the full graph. Both real models now answer the selected-case/default question correctly using one retained capture and focused case/byte reads. Code Mode uses fewer tokens in this pair, but unequal native capture time and extra client polling confound the comparison. Whole-task costs and limits are in the same study; a general efficiency advantage remains unproved. A common long-wait repeat reduces polling to one wait per lane; both answers remain correct, but direct wins cumulative input and wall time while Code Mode uses less uncached input. Runtime correctness, discovery, composition choices and waiting all matter to the completed task.