rea/experiments/atomic-codemode/trials.md

Real-model trials (ongoing)

The first trial used the committed 5aa1c5c64 runtime, requested model alias gpt-6.1-sol with high reasoning, Codex CLI 0.161.0 and the zlib question in this directory. The resolved server-side model build is not inferred. Its actual registered MCP server name was rea-codemode, with only the four Code Mode tools. The runner requests disabled shell/file tools, web search, delegation and client Code Mode feature flags. Actual transcripts nevertheless contain the client's exec dispatch surface in every lane. The earlier inference that these flags established an executor-free client was too strong; it is corrected here. The actual trace separately establishes server-side execute_javascript use.

The model independently found the string and xref, requested the owning function's snapshot, used operations/relations/slices and inspected reconstructed C. Its answer identifies both error-block branches, the message/STORE origins, wrapper bit condition and modulo-31 check, and correctly distinguishes local guards from runtime reachability. It names memory, phi, traversal-budget and CFG limitations. It records the exact target digest and real source Evidence IDs and closes the target. These observations agree with the previous independent machine-byte and arithmetic oracles; this is one stochastic local-question trial, not evidence of general RE superiority.

Measurement	First Code Mode trial
Completed outer MCP calls	14
Actual API calls	30
Reconstructed C page reads	16
Native source captures	3
Cumulative model input tokens	581,617
Cached input tokens (included above)	530,176
Output tokens	3,822
Reasoning output tokens (included in output)	869
Model runner wall seconds	130.348
Bounded family elapsed seconds	130.824
Bounded family CPU seconds	11.842
Kernel process-family memory peak MiB	540.7
OOM kills	0

Artifact bytes were unchanged. The copied-authentication profile was removed; both owner/observer cleanup checks passed. Memory includes charged file cache. Actual token usage is cumulative across turns, including discovery and retries, not a tokenizer estimate of the final answer. Cached and uncached tokens are reported separately without assigning an unverified price.

The first empty workspace was nested beneath the trial output directory in the repository. No source or oracle was copied into it and shell/file tools were disabled. Subsequent trials move the profile under /tmp to remove ancestor repository-instruction discovery as a possible confound. The first trial must not be silently relabeled as having that stronger isolation.

The trace exposes two design shortcomings. Reference lookup supplied parameters but omitted result shapes, making initial calls sequential shape discovery. The C reader's arbitrary 128-line maximum made code compose many page reads even though the complete function easily fits guest memory. Return contracts and byte-budgeted internal C reads are being tested next. The missing native CFG and phi predecessor relationships remain capability gaps, not problems to conceal with reconstructed-C guesses.

Private captures remain under .cache/atomic-codemode/zlib-codemode-v1/, with events.jsonl, codex-tool-context.jsonl, answer.txt, summary.json, explicit trial settings and cleanup proof. The adjacent trace retains actual model code and inner API calls; the *-evidence/ directory retains the three complete native sources and their identity/digest metadata. Credentials and raw captures are not committed.

The second round used the pinned c5cefd622 runtime, the same target, question, model alias/reasoning and client version, with the empty model profiles outside the repository under /tmp. Both atomic modes use the same eight fact methods; only the calling adapter changes. The traditional comparison forces the real Kuna provider and selects nine relevant production tools in the client: open_binary, close_binary, binary_session, search_strings, xrefs, procedure_address, procedure_pseudo_code, procedure_assembly, analyze_function. The production catalog remains complete. This is a favorable focused baseline, not the default full-catalog experience.

Measurement	Atomic direct v2	Atomic + Code Mode v2	Traditional v2
Outer MCP attempts, including failures	27	12	9
Completed / failed outer attempts	27 / 0	12 / 0	7 / 2
Internal Code Mode API reads	—	21	—
Cumulative model input tokens	590,872	453,734	344,105
Cached input tokens (included)	496,512	401,664	297,344
Uncached input tokens	94,360	52,070	46,761
Output tokens	5,874	4,738	3,076
Reasoning tokens (included in output)	2,228	1,716	832
Model runner wall seconds	170.813	148.397	106.352
Bounded family elapsed seconds	171.801	149.585	106.747
Bounded family CPU seconds	11.898	22.129	20.666
Kernel process-family memory peak MiB	578.2	544.3	843.4
OOM kills	0	0	0

The traditional trace includes a client approval-policy rejection for binary_session and a failed name lookup for inflate; the model recovered through string/xref/address discovery. Both attempts and their costs remain in the comparison. A completed transport call does not itself establish a valid analysis result; answers and the retained calls were reviewed separately.

All three answers identify the same two local error branches and modulo-31 condition, with supporting addresses and limitations consistent with the independent instruction/arithmetic checks. This establishes a completed local task comparison. It does not prove global runtime reachability or correctness on arbitrary binaries. All target digests remained unchanged, temporary model profiles were removed, and observed payload cleanup succeeded.

Code Mode reduces direct atomic MCP requests on this trial, but traditional tools win both cumulative input tokens and wall time. The prototype still makes the model assemble incomplete CFG/phi/value relationships and repeatedly inspect operations or C. Fixing the arbitrary C page cap reduced internal C reads from 16 to 2; the overall first-to-second trial changes are not a controlled causal measurement because return contracts and profile placement changed too.

These are single stochastic samples, with fresh provider/session caches, warm and uncontrolled OS file caches, and actual model prompt-cache usage reported. No warm-session paired reruns, confidence interval, server-side model compute or pricing was measured. Reproduce the ledger with summarize-trials.py over the private trial directories; it reads actual model usage and the payload-exit cgroup receipt and deliberately does not grade answers.

The first native-IR model trial pins a7e9f8843, the same zlib question and model settings, with patched Kuna SHA-256 7d357e75ff252d04913fe9e55f3d02c71bceb0fe5cbdac12e9a47ef84151cfcc. Its source is upstream v1.833 plus kuna-native-ir.patch, built at optimization level 1 with embedded specs and a fresh owned per-lease engine cache. The prior stock-release comparison is not an engine/profile-controlled native-IR ablation.

The real model uses relations(kind:"controllers"), incoming edges, blocks, indexed definition slices and native values. The answer agrees with the local guard oracle, and correctly identifies accumulator phi predecessors 42, 43 and 38 while retaining unknown concrete state, path selection, memory aliases and runtime successor coverage. API/transport calls succeed, the runtime input digest is unchanged across the trial, the target digest is unchanged, the copied model profile is removed and observed payload cleanup succeeds.

Measurement	Native IR + Code Mode v3
Outer MCP attempts / failed	15 / 0
Internal API reads	24
Cumulative model input tokens	600,857
Cached input tokens (included)	534,784
Uncached input tokens	66,073
Output / included reasoning tokens	4,131 / 649
Model runner wall seconds	117.773
Bounded family elapsed seconds	119.211
Bounded family CPU seconds	15.339
Kernel process-family memory peak MiB	560.1
OOM kills	0

The capability gap is narrower, but this trial still does not beat the earlier traditional workflow on whole-task costs. The model performs four slices and eleven operation reads. More precise facts allow useful qualification of the answer, but also invite further investigation after the local question is already substantially supported. This is a reason to test task-specific value tracing and small-function counterexamples, rather than to claim that more granular tool catalogs are inherently efficient. Private artifacts are named zlib-native-codemode-v3* beside the earlier trials.

The value-origin task uses the same patched engine in both lanes, with fresh provider/session caches and the pinned f2f029c5d runtime. It asks about the accumulator at the already identified instruction origin 0xd9c9; this is a seeded follow-up investigation, not a repeated string-discovery task. The traditional client retains its nine selected tools. Both traces use the real Kuna engine, complete without API errors, preserve the target/runtime digests and verify profile/payload cleanup with no OOM kills.

Measurement	Native IR + Code Mode	Traditional
Outer MCP attempts	8	5
Cumulative model input tokens	284,638	390,557
Included cached input tokens	220,672	326,016
Uncached input tokens	63,966	64,541
Output / included reasoning tokens	2,601 / 309	2,614 / 531
Model runner wall seconds	72.403	91.832
Bounded family elapsed seconds	72.841	92.254
Bounded family CPU seconds	11.896	20.866
Kernel process-family memory peak MiB	461.6	846.0

Both answers recover the three accumulator alternatives, byte loads, shifts, additions, local bit-count conditions and unknown state. The native answer, however, says that the branch at 0xd97e directly reaches 0xd9c9. This describes the optimized recovered CFG, but the actual bytes 0f87e9080000 encode JA 0xe26d; 0xe270 then encodes JMP 0xd9c9. The traditional answer identifies that machine predecessor and intervening copy/jump. This was checked separately with the shipped machine bytes, rather than accepting either model's answer as the oracle.

The native trace's lower cumulative input and wall time therefore do not yet establish an equal-correctness completed-task win. Uncached input is also almost equal; cached cumulative input alone is not an economic-cost measurement. The representation ambiguity needs a structural fix: distinguish IR origin fields from physical instruction facts and expose selected instruction windows, then rerun the unchanged question. Private artifacts are zlib-value-codemode-v1* and zlib-value-traditional-v1*.

After separating IR origins and adding instruction windows, the unchanged value question was rerun at 4f13bd673. The Code Mode answer now reads actual machine bytes and correctly distinguishes recovered predecessor 38 from the intervening 0xe26d / 0xe270 machine path. Both answers recover the same local value equations, byte-order conditions and unknown invariants. These remain individual stochastic samples, not a general speedup guarantee.

Measurement	Code Mode value v2	Traditional value v2
MCP attempts / failures	7 / 0	6 / 1
Cumulative input tokens	296,888	573,881
Included cached input	252,160	512,128
Uncached input	44,728	61,753
Output / included reasoning	3,176 / 594	2,830 / 547
Model wall seconds	96.662	124.754
Family wall seconds	97.654	125.410
Family CPU seconds	19.495	44.143
Kernel family peak MiB	534.4	797.4

The traditional failure is the client policy rejection of binary_session. Its cost remains included. Both runtime/target digests stay unchanged, cleanup is verified and OOM kills are zero. This is evidence of a useful seeded value-tracing case for the revised interface, with the selected-tool and single-sample qualifications retained.

The initial SQLite diagnostic comparison, also at 4f13bd673, exposes a further capability/comparison gap:

Measurement	Code Mode SQLite v1	Traditional SQLite v1
MCP attempts / failures	14 / 0	17 / 8
Cumulative input tokens	601,445	934,237
Included cached input	535,296	831,744
Uncached input	66,149	102,493
Output / included reasoning	4,790 / 1,668	3,878 / 1,121
Model wall seconds	143.469	231.469
Family CPU seconds	18.996	154.132
Kernel family peak MiB	586.6	1,200.2

Both answers find the two malformed-message conditions and distinguish the depth-error, other-status and success paths. The traditional answer additionally resolves the named formatter implementation at 0x5efc0, tracing free, sqlite3_vmprintf and pointer storage; Code Mode stops at its PLT stub and correctly leaves delivery/allocation unknown. These are unequal-depth answers, so the cost difference is not an equal-completion win.

The selected traditional catalog also omitted list_procedures, despite errors advertising it as the discovery route. The trace retains interior-entry failures, failed resource discovery, a client approval-policy rejection and ambiguous name lookups. This selection confound must be corrected in the next focused baseline; the failed attempts and their costs are not discarded. Both runs still preserve target/runtime identity, clean up their profiles/payloads and have zero OOM kills.

The corrected SQLite comparison pins e9b9a8dd0, the same question and patched engine, and the same requested model alias/reasoning. Both experimental adapters now expose twelve identical fact methods, including the missing function-entry lookup. The direct lane uses explicit direct-API instructions rather than being told to call a nonexistent Code Mode entrypoint. The traditional selected catalog replaces binary_session with list_procedures; exact-name discovery guidance is enabled. These instruction/catalog changes are recorded and are not retroactively applied to earlier trials.

Measurement	Code Mode v2	Atomic direct v1	Traditional cold v2	Traditional reused-cache v1
MCP attempts / failures	12 / 0	24 / 0	14 / 2	11 / 1
Unique model responses	14	14	33	13
Client exec / wait calls	13 / 0	13 / 0	15 / 17	12 / 0
Internal Code Mode API reads	15	—	—	—
Cumulative input tokens	454,914	464,979	1,093,061	432,446
Included cached input	414,080	371,712	1,036,928	377,600
Uncached input	40,834	93,267	56,133	54,846
Output / included reasoning	3,636 / 1,129	4,591 / 1,482	4,056 / 787	2,889 / 488
Model wall seconds	126.250	146.656	346.051	98.210
Family wall seconds	127.171	147.151	346.491	98.690
Family CPU seconds	16.199	18.445	219.655	15.601
Kernel family peak MiB	598.2	565.0	1,128.7	779.2

All four answers cover the two malformed-message selectors, original expression argument, formatter implementation, free/vmprintf/pointer store, missing local allocation-success check, separate depth/status/success paths, and static-only limitations. They agree with the independent instruction checks recorded below. The reused-cache traditional answer supports the closing-parenthesis decrement with reconstructed C; its returned assembly excerpt omits that destination. The other three obtain the decrement's actual instructions. This evidentiary-depth difference is retained rather than claiming byte-for-byte identical answers.

The remaining traditional failures are an interior-address analyze_function request in both runs and an ambiguous function-name lookup in the cold run. list_procedures is available and used. Neither is a policy rejection or a deliberately missing discovery tool. Errors and retries stay in total costs.

Traditional cold analysis creates a validated persistent project containing 3,923 decompiled functions, a 4.4 MB C export and a 17.5 MB assembly export. The next fresh model profile uses the same explicit project cache. Its retained manifest binds the target digest, patched engine digest and one-job setting. The large cold/warm CPU difference is consistent with avoiding this project creation; model strategies also differ, so it is not a controlled measurement of cache speedup alone. Experimental Code Mode/direct analyze selected functions and reuse captures only within a lease; cross-session result caching is absent. Comparing their cold run with the traditional reused-cache run cannot establish an adapter speedup. In this sample, the latter is faster and has fewer cumulative input tokens than Code Mode. All four target/runtime identities, profile/payload cleanup checks pass, with no OOM kills. The private ledger is round-e9b9a8d-ledger.json; each table cell remains one stochastic trial.

The task oracle for this SQLite artifact (SHA-256 2e6eef9a727f081f0d453b4e5e6cbd8b9ef8b6f86cbf7681cbad444d3b0b55c8) is the shipped machine code, checked independently of model answers: wrapper entry 0x65180; status comparisons 0x65230 / 0x65239; context-field load/test 0x65268 / 0x65270; format load 0x65284; emission call 0x65290; parenthesis increment 0x64699 and decrement 0x64c92; formatter implementation 0x5efc0, free call 0x5f031, formatting call 0x5f062 and pointer store 0x5f067. Named PLT entries and runtime binding remain separate from that local static implementation evidence. No SQL input was executed to prove feasibility.

Coverage must be sufficient for the particular question before reporting an equal-completion cost comparison. More tools or a successful MCP transport do not establish this. Current gaps include exact-address-only xrefs, a minimum-five string inventory, unknown runtime successors and memory aliases, and bounded IR extraction. In particular, real sqlite3VdbeExec exceeds the graph limit while retaining 3,654 lines of C. A local successful diagnostic task does not prove full Kuna/Ghidra/IDA parity or arbitrary complex-function coverage.

The separate large-VM diagnostic task pins 5aa78c873, with question and independent instruction anchors committed before running the models in evaluation.md. It uses the corrected traditional selection and guidance, the same patched engine and model settings, and serial bounded trials. The fresh traditional project cache is reused only for its explicitly labeled fourth run. None of these trials uses the later opt-in envelope/stopping guidance.

Measurement	Code Mode VM v1	Atomic direct VM v1	Traditional cold VM v1	Traditional reused-cache VM v1
MCP attempts / failures	14 / 0	20 / 0	10 / 2	14 / 5
Unique model responses	18	17	27	18
Client exec / wait calls	15 / 2	13 / 3	12 / 14	17 / 0
Cumulative input tokens	729,420	594,606	698,971	617,340
Included cached input	664,960	526,720	625,152	543,616
Uncached input	64,460	67,886	73,819	73,724
Output / included reasoning	4,474 / 975	4,287 / 1,045	4,565 / 999	4,446 / 1,290
Model wall seconds	243.082	218.541	342.966	185.580
Family wall seconds	244.298	219.062	343.367	186.321
Family CPU seconds	89.693	79.343	207.596	28.162
Kernel family peak MiB	871.2	858.9	1,408.6	783.2

All four answers correctly distinguish selector zero plus signed positive db+0xd8 for open, found name plus selector one plus the same positive counter for release, the shared 0xf238b emission / local status five, and missing-name precedence with status one. They preserve offsets, real source references and static-only limitations. None falsely promotes local status five into an unconditional final return. Both atomic answers explicitly name the VM's unavailable graph facet while using its valid C and selected machine windows. The cold traditional answer omits the optional formatter implementation detail; the other three include it. The fixed question concerns local message selection and joins, so that extra detail is not required for its completion.

The traditional failures are ambiguous procedure names (and consequent name-selected assembly/pseudocode failures in the reused-cache run), not missing discovery tools or client approval rejections. Costs are retained. The final traditional model does not explicitly call close_binary; teardown still verifies payload/profile cleanup. Both target/runtime identities remain unchanged in all four runs, and OOM kills are zero. These are four single stochastic observations, not confidence intervals.

This larger local task is a useful counterexample to an automatic Code Mode win: atomic direct has fewer cumulative input tokens and lower wall time than Code Mode despite making more MCP requests; traditional with a reused project is fastest. Uncached input is closer across the lanes. Source review shows that the models choose different investigation sequences and optional helper depth. The prototype can complete this task through an unavailable IR facet; a claim of general complex-software superiority is still unsupported.

Actual client dispatch matters to interpreting all these tables. The direct atomic SQLite run performs 24 MCP requests inside 13 client exec calls; Code Mode performs 12 MCP requests inside the same 13 client exec calls. Both have 14 observed model responses. Thus halving MCP requests does not halve model round trips here. The traditional client can also combine calls and filter results in JavaScript. These lanes compare server-side Code Mode with direct MCP surfaces under that same capable client, not JavaScript versus a client incapable of composition. summarize-trials.py now counts unique response/call identities from actual retained events and verifies that per-response usage sums to the runner's declared totals. Missing captures remain unknown. Client waiting during long analysis is part of completed-task cost, not additional analysis facts.

After repairing encoded-output capacity, a model task over the restored Pragma code pins 9e1b39358 and the rebuilt 2290e8b1… engine. Both experimental lanes have identical fact capabilities and common envelope/stopping guidance enabled; this is not a rerun with only one changed variable relative to older tasks. The question and independent instruction anchors were committed before the runs. No traditional cell was run for this new question.

Measurement	Code Mode Pragma v1	Atomic direct Pragma v1
MCP attempts / failures	11 / 0	14 / 0
Unique model responses	13	12
Client exec / wait calls	12 / 0	11 / 0
Cumulative input tokens	370,307	353,364
Included cached input	333,184	278,272
Uncached input	37,123	75,092
Output / included reasoning	3,612 / 803	2,830 / 326
Model wall seconds	121.459	107.625
Family wall seconds	122.256	108.066
Family CPU seconds	20.003	20.261
Kernel family peak MiB	590.1	560.6

Both answers match the machine oracle: null value pointer reads without the state check; a supplied value plus zero D+0x65 selects the diagnostic before the selector-one bypass; allowed updates write the transformed byte at S+0x10 and one at S+0x11; the diagnostic and shared cleanup have the correct physical addresses. Both avoid inventing source member names or a numeric return from the void reconstruction, preserve Evidence/target identity, and explicitly state the unavailable graph and unknown runtime feasibility/binding. Cleanup is verified, runtime/target bytes stay unchanged and OOM kills are zero.

The restored C now supports a completed real-model task in both interfaces. Code Mode has fewer MCP requests and fewer uncached input tokens here; direct atomic has fewer model responses, lower cumulative input/output and shorter wall time. Prompt-cache state and model strategy remain uncontrolled, so no economic or general adapter ranking follows. The native byte-budget repair is a coverage fix, not a demonstrated Code Mode speedup. Private captures and the separate round-9e1b393-ledger.json retain these observations.

The table-dispatch task pins 73db4fad5, the same rebuilt engine and thirteen shared fact methods. This time direct runs first, followed serially by Code Mode; both use common envelope/stopping guidance, fresh analysis leases and isolated empty model profiles. Independent machine instructions and direct file table reads were committed as the oracle before model execution.

Measurement	Atomic direct table v1	Code Mode table v1
MCP attempts / failures	8 / 0	7 / 0
Unique model responses	10	9
Client exec / wait calls	9 / 0	8 / 0
Cumulative input tokens	244,235	225,673
Included cached input	200,704	190,848
Uncached input	43,531	34,825
Output / included reasoning	3,055 / 1,024	2,765 / 355
Model wall seconds	108.774	101.033
Family wall seconds	109.281	101.523
Family CPU seconds	9.683	9.606
Kernel family peak MiB	490.5	495.3

Both models actually read the class and relative-jump tables through bytes. All six class values and destinations match the independent artifact oracle, including unsigned indexing for 0x80, signed 32-bit displacement extension, the correct table base and the shared destination for 0x22 / 0x27. They keep source IDs and complete-for-requested-range coverage, distinguish loaded-image observations from arithmetic derivations, and do not infer final token types, lengths or runtime feasibility. No decompilation is needed for these answers. Code Mode composes functions, instructions and two bytes reads.

This is a successful example of composing small RE facts into a useful answer. Its Code Mode sample has lower observed task token counts and wall time; CPU is almost the same and memory is slightly higher. Each cell is still one stochastic sample, with uncontrolled prompt/OS caches and no traditional cell. It establishes neither a general cost ranking nor complex-program completeness. Target/runtime identities stay unchanged, cleanup verifies and OOM kills are zero. The private ledger is round-73db4fa-ledger.json.

Cross-function call task: preserved rejected launch

Question/oracle and fourteen shared fact methods were fixed at 33c4282b8. The initial sqlite-call-codemode-v1 model sees engine_profile_mismatch at open_binary, then closes and reports that analysis is unavailable. Cargo's non-embedded test build had overwritten the selected mutable executable; the engine digest guard correctly rejects it. No target digest or Evidence is returned. This is an experimental setup failure, not a completed analysis or an accuracy/speed win. Runtime JS and target bytes remain unchanged.

Actual requested gpt-6.1-sol/high usage is 107,489 input (81,280 cached, 26,209 uncached), 565 output including 101 reasoning tokens; five observed model responses, four client exec calls and three MCP requests. MCP transport requests complete, but the open API result is a failure. Model wall time is 40.205 seconds, family CPU 3.619 seconds and memory peak 436.6 MiB. Cleanup is verified, OOM 0. The failed sample and raw receipts remain under .cache/atomic-codemode/ and are excluded from correctness-equal paired costs. Subsequent attempts use an independent preserved executable and explicit pre/post engine validation.

Cross-function call task: initial question pair

sqlite-call-codemode-v2 and sqlite-call-atomic-direct-v2 both run at 01e5c5955, with engine 64c6b3f... from an independent preserved executable. The fourteen fact methods, native profile, artifact, requested model/effort and common guidance match; Code Mode runs first. Runtime, engine and target bytes remain unchanged, per-response usage matches runner totals, cleanup is verified and OOM kills are zero. Both fresh target leases use uncontrolled warm OS caches.

Measurement	Code Mode call v2	Atomic direct call v2
MCP requests / transport failures	10 / 0	27 / 0
Observed model responses	13	16
Client exec calls	12	15
Model wall seconds	168.622	156.033
Family wall seconds	169.190	156.570
Family CPU seconds	16.096	15.854
Family memory peak MiB	514.0	608.1
Cumulative input tokens	454,199	564,158
Cached input tokens	377,216	497,408
Uncached input tokens	76,983	66,750
Output tokens, including reasoning	4,987	4,838
Reasoning tokens, included in output	1,301	935

Both answers correctly connect call 0x65211 -> 0x64060, caller SSA v100000118, passing storage register:0x38/8, and callee input v300000042. They verify caller LEA, helper pointer preservation, character dispatch, field increment/decrement and the caller's post-success field check against selected machine bytes. They preserve storage-match/alias and runtime limitations. Code Mode actually composes calls, values, operations and machine reads; direct uses the same primitives, including a bounded slice. All ten/nine cited Evidence sources respectively have matching retained-source SHA-256 and artifact identity.

Neither final answer includes the original oracle's required separate routing for helper return 1 and 18. The question only explicitly asked to distinguish successful/malformed paths; the oracle was stricter than that wording. Do not retroactively delete this requirement or mark either sample a full-oracle pass. The task wording is now clarified before the next paired round, retaining the same oracle and adding no solution addresses. The table is an observed partial coverage result, not a completed-task efficiency claim. Its mixed costs also do not show a general Code Mode advantage: cumulative input is lower while uncached input and wall time are higher in this sample.

Raw ledger: .cache/atomic-codemode/ledger-call-round-01e5c59.json. Each cell is one stochastic sample. The complete client already provides JavaScript dispatch in both lanes. No new traditional-provider cell is being claimed.

Cross-function call task: explicit return-path pair

Both sqlite-call-atomic-direct-v3 and sqlite-call-codemode-v3 use e9348df0c and the clarified question, fixed before either lane starts. This time direct runs first. Engine, fourteen fact methods, model/effort request and common instruction settings match the prior pair. Runtime, selected engine and target bytes remain unchanged. Target leases and native caches are fresh; OS/page and remote prompt caches are not reset. Per-response token totals match runner usage, process/profile cleanup is verified, and OOM kills are zero.

Measurement	Atomic direct call v3	Code Mode call v3
MCP requests / transport failures	30 / 0	12 / 0
Observed model responses	16	14
Client exec calls	15	13
Model wall seconds	147.424	156.821
Family wall seconds	147.922	157.307
Family CPU seconds	15.678	17.514
Family memory peak MiB	537.9	478.0
Cumulative input tokens	611,321	596,978
Cached input tokens	544,256	528,768
Uncached input tokens	67,065	68,210
Output tokens, including reasoning	4,405	4,954
Reasoning tokens, included in output	673	1,046
Largest observed response input tokens	64,471	65,964

Both answers meet the clarified question's context/field/control-flow and return-code requirements. They identify the exact caller/callee SSA and passing storage, independently verify the actual call/pointer setup, field updates and counter test, and distinguish helper returns 0, 1 and 18. Both correctly separate the wrapper depth diagnostic (argument 12) from the helper nesting limit 1000. They preserve memory-alias/traversal and runtime-feasibility uncertainty. The fspec ownership regression is separately verified at the producer/MCP boundary; neither answer claims a machine target from an annotation offset. All eight/nine cited sources respectively have matching retained-source digests, target identity and selected engine digest.

This is a successful, correctness-equal example of the new call/input facts in real model work. It is not a general Code Mode efficiency win: cumulative input is only about 2.3% lower, while uncached input, output, maximum observed response input, CPU and elapsed time are higher. MCP requests fall 60%, but actual model responses fall only 12.5%. Peak process-family memory is lower in this sample. One stochastic cell per lane, differing chosen excerpts and uncontrolled caches cannot establish causality or a repeatable effect. The existing client offers JavaScript dispatch in both lanes; no new traditional-provider cell was run.

Raw ledger: .cache/atomic-codemode/ledger-call-round-e9348df.json; independent answer/source review: .cache/atomic-codemode/call-v3-answer-review.json. The largest-response metric is the maximum observed per-response input usage, including cached tokens, rather than an estimate from serialized bytes. Memory is the kernel cgroup process-family peak, including charged file cache. Remote model CPU/memory, resolved server model build and monetary cost are unmeasured.

Shallow reference aggregation: real-model pair

Both runs use 3aa685db3, the pre-pinned reference question/oracle, the same preserved 64c6b3f... engine and fourteen fact methods. Code Mode runs first; all other model/effort and common-guidance settings match. Each lease/native cache is fresh; OS/page and remote prompt caches remain uncontrolled. Both have unchanged runtime, engine and target bytes; per-response token totals match the runner. Source capture, process/profile cleanup and OOM-zero receipts verify.

Measurement	Code Mode references v1	Atomic direct references v1
MCP requests / transport failures	8 / 0	20 / 0
Observed model responses	10	11
Client exec calls	9	10
Model wall seconds	93.837	106.183
Family wall seconds	94.371	106.799
Family CPU seconds	12.504	15.934
Family memory peak MiB	518.7	523.5
Cumulative input tokens	264,680	285,086
Cached input tokens	238,720	247,424
Uncached input tokens	25,960	37,662
Output tokens, including reasoning	2,766	3,204
Reasoning tokens, included in output	420	533
Largest observed response input tokens	33,716	33,798

Both answers satisfy all question requirements: parser implementation rather than same-name PLT entry; nine lifted references versus eight IR calls; verified 0x6534b -> 0x29370 stack-canary failure call and guard; VM aggregates 396/171/16; and verified 0xf238b -> 0x296c0. Both compose all four VM pages locally with no full VM decompilation or instruction/source report. Both preserve unknown indirect/discovery coverage, static symbol-binding limits and the non-equivalence of optimized IR to the executable.

Each answer cites six source IDs. Every retained source digest, target digest and selected engine digest verifies. The VM reference pairs in both real-model captures exactly match all independent GNU objdump direct-call pairs. Both reference pages cover source rows 0–395 with a terminal null continuation. Raw Evidence remains in the explicit capture directory after the API lease closes; its IDs are not live handles in a newly opened lease.

This is one positive aggregation sample: Code Mode uses about 7.2% fewer cumulative input tokens and 11.6% less wall time. Uncached input, output and CPU are also lower here; observed peak input context and memory are close. Reducing MCP requests 60% yields only one fewer observed model response. The client's own JavaScript dispatcher lets direct calls aggregate locally too. With one stochastic sample per lane and no reset of remote/file caches, these numbers do not establish a general causal advantage. Earlier cross-function pairs remain mixed; the questions have different required work and must not be treated as a before/after speedup comparison.

Ledger: .cache/atomic-codemode/ledger-reference-round-3aa685d.json; manual answer/source review: reference-answer-review-v1.json in the same cache root. The measured artifact belongs to Debian libsqlite3-0:amd64 version 3.40.1-2+deb12u2. Remote model CPU/memory, resolved server model build, pricing and development/parent-conversation token costs remain unmeasured here.

Across this continuation's seven model launches (one setup rejection, two initial partial-oracle answers, four completed answers), cumulative usage is 2,883,911 input, including 2,515,072 cached and 368,839 uncached; output is 25,719, including 5,009 reasoning. Summed model wall time is 869.125 seconds. This research-attempt cost is separate from the successful paired tables and excludes native builds, scripted verifiers and development conversation. Failed/partial attempts are retained rather than hidden in a selected success result.

Calls beyond graph capacity: real-model pair

The question and acceptance oracle are committed before either launch in f9f9dc209. Both lanes use its fourteen shared fact methods and the preserved e7d3f0cc... embedded-spec engine. Direct runs first this time. Both request gpt-6.1-sol/high, identical common envelope/stopping guidance, fresh leases and native caches, one CPU and the same 4 GiB hard / 3 GiB early-stop bounds. OS and remote prompt caches are uncontrolled. This question requires one VM analysis; the preceding shallow-reference task does not, so their elapsed times are not a before/after adapter comparison.

Measurement	Atomic direct boundary v1	Code Mode boundary v1
MCP requests / transport failures	18 / 0	9 / 0
Observed model responses	13	12
Client exec / wait calls	10 / 2	9 / 2
Model wall seconds	146.865	127.791
Family wall seconds	148.137	128.304
Family CPU seconds	44.640	43.600
Family memory peak MiB	869.2	865.0
Cumulative input tokens	332,318	321,922
Cached input tokens	304,768	294,144
Uncached input tokens	27,550	27,778
Output tokens, including reasoning	3,927	3,188
Reasoning tokens, included in output	870	439
Largest observed response input tokens	34,549	35,170

Both complete answers satisfy the fixed oracle. They select implementation entry 0xea6d0, distinguish available C/calls/boundary values from the unavailable graph, aggregate 395 known direct and 21 unresolved indirect recovered calls, and identify the two selected targets, distinct fspec annotations, argument slots/passing storage and shared input value v30000004b. Focused machine windows verify the selected CALL bytes and essential argument setup. Both state that retained definition IDs do not provide operation bodies, full def/use, aliases or runtime coverage. The Code Mode answer additionally inspects the PLT stub; independent GNU disassembly verifies its JMP [0x15a2f8] at 0x296c0.

Each lane performs exactly one VM decompilation and reuses that source for its call/value reads. Direct composes four count pages and a selected call read; Code Mode retains the selected rows while aggregating its four pages locally. All six/seven captured sources respectively verify their hashes, target and engine identities; all three/four cited IDs exist. The full retained VM function records are identical across lanes. Every recovered direct pair belongs to the independent 396-pair machine oracle, with only 0xf2493 -> 0x29370 absent; all 21 indirect origins match. This corroborates the stated coverage, not complete machine/IR or runtime equivalence.

Runtime/engine/target fingerprints remain unchanged; per-response usage totals match the runner; profile/process cleanup and OOM-zero receipts verify. Code Mode uses about 3.1% less cumulative input and 13.0% less model wall time in this single sample, but slightly more uncached input and a larger maximum response context. Memory peaks are close. Both clients have JavaScript dispatch; halving MCP requests saves only one model response. This is a useful capability result with mixed context costs, not a repeatable causal efficiency claim or provider parity result.

Ledger: .cache/atomic-codemode/ledger-boundary-round-f9f9dc2.json; full-answer and source review: boundary-answer-review-v1.json beside it. Pair totals are 654,240 input tokens (598,912 cached, 55,328 uncached), 7,115 output tokens (1,309 included reasoning) and 274.656 model wall seconds. These totals include discovery and extra reads. Builds, scripted verification, development/parent conversation, resolved server model build, remote CPU/memory and monetary cost remain unmeasured by this pair. Lease/native analysis is cold; subsequent reads are warm within the same lease, not a cross-session result-cache benchmark.

Shared compact counts: same question, new pair

Revision da6f5470b adds snapshot call-count metadata through the shared owner. The question, acceptance oracle, artifact, preserved engine, fourteen methods, model/effort request, common guidance and resource policy remain the same as the preceding pair. Code Mode runs first, direct second; both use fresh leases/native caches. No runtime source or HEAD changes occur during either pair. These are single stochastic runs, with uncontrolled OS and remote prompt caches.

Measurement	Code Mode boundary v2	Atomic direct boundary v2
MCP requests / transport failures	7 / 0	14 / 0
Observed model responses	12	12
Client exec / wait calls	8 / 3	9 / 2
Model wall seconds	149.559	137.633
Family wall seconds	150.643	138.180
Family CPU seconds	46.984	45.417
Family memory peak MiB	849.8	838.2
Cumulative input tokens	268,377	235,640
Cached input tokens	235,136	215,552
Uncached input tokens	33,241	20,088
Output tokens, including reasoning	3,726	3,541
Reasoning tokens, included in output	919	620
Largest observed response input tokens	31,307	27,410

Both answers pass the unchanged full oracle, including selected machine checks, annotation/target separation, input identity, partial value scope and unreadable definition/use/runtime limits. Each uses the shared aggregate and makes only one call read for the two selected origins: neither pages the full 416-call set. Each still makes exactly one VM decompilation. All four cited sources per answer exist; all seven retained sources per lane verify their digests and target/engine contexts. The complete native VM function records equal each other and the v1 captures, so the metadata change preserves the observed native facts. Independent machine-pair/indirect-origin checks retain the same 395/21 coverage and single omitted stack-check call. Runtime/target/engine fingerprints, per-response usage, process/profile cleanup and OOM-zero receipts verify.

Compared with its own v1 sample, direct uses about 29.1% less cumulative input, 27.1% less uncached input and 6.3% less model wall time; its CPU is slightly higher. Code Mode uses about 16.6% less cumulative input, but 19.7% more uncached input and 17.0% more model wall time. Its model-response count stays twelve. The summary eliminates the observed paging work in both lanes, but these runs do not establish a general latency or uncached-token improvement. Across the two new cells, cumulative input is 23.0% lower than v1, while summed model wall time is 4.6% higher. These are observations, not isolated causal effects.

Within v2, direct is lower on every reported token category, elapsed time, CPU, peak response input and family memory, despite twice the MCP requests and the same number of model responses. This reverses the preceding sample's latency ordering. A useful shared aggregate can change the adapter comparison; keeping the underlying direct tools needlessly granular would bias it. Neither RPC count nor smaller cumulative input alone establishes whole-task efficiency. The client has JavaScript dispatch in both modes; no new traditional-provider cell or general complex-software superiority claim follows.

Ledger: .cache/atomic-codemode/ledger-boundary-counts-round-da6f547.json; full-answer/source review: boundary-counts-answer-review-v2.json beside it. Pair totals are 504,017 input (450,688 cached, 53,329 uncached), 7,267 output (1,539 included reasoning) and 287.192 model wall seconds. The four boundary launches together cost 1,158,257 input (1,049,600 cached, 108,657 uncached), 14,382 output (2,848 included reasoning) and 561.848 model wall seconds. Failed/partial earlier experiments remain recorded above. Builds, public scripted checks, development conversation, server model build, remote compute and money remain outside these model-pair totals. The unresolved capability is selected definition bodies/relationships after full-graph failure, not call counting or selected argument identity.

PHI format alternatives: first real-model pair

Both runs use 8114bd675, the pre-pinned projection question/oracle, the same preserved a0685486... engine and fourteen fact methods. Direct runs first. Both request gpt-6.1-sol/high, identical common guidance and resource limits, fresh target leases/native caches, and an isolated workspace without repository or oracle. Runtime, target and engine fingerprints remain unchanged; per-response usage matches the runner; process/profile cleanup and OOM-zero receipts verify.

Measurement	Atomic direct projection v1	Code Mode projection v1
MCP requests / transport failures	13 / 0	11 / 0
Observed model responses	15	13
Client exec / wait calls	11 / 3	10 / 2
Model wall seconds	136.952	135.203
Family wall seconds	137.840	135.656
Family CPU seconds	45.359	45.801
Family memory peak MiB	888.8	852.8
Cumulative input tokens	406,155	365,228
Cached input tokens	337,792	322,304
Uncached input tokens	68,363	42,924
Output tokens, including reasoning	3,856	3,912
Reasoning tokens, included in output	725	556
Largest observed response input tokens	38,008	38,436

Both complete answers pass the unchanged oracle: correct entry/digest, both call/RSI/PHI links, all four operand/value/predecessor-edge/producer links, exact format addresses and strings, selected LEA/CALL bytes and both alternate jump destinations. They distinguish a closed six-operation selected trace from the capture-wide 326-definition frontier and preserve CFG/complete-use/alias/runtime and original-signature uncertainty. Neither reads the entire reconstructed C.

Each performs one VM decompilation and reuses its source. Both capture ten native sources and cite nine; every source hash, subject and preserved-engine identity verifies. The complete native function records equal each other. Machine-window bytes match independent file reads under the verified ELF mapping; GNU decoding independently verifies the selected origins/transfers and the four format strings match read-only loaded bytes. These are selected static facts, not runtime proof.

The tools provide enough facts for this local complex-function task. An actual discovery mismatch also appears: direct queries reference with a list of advertised facts_* names, receives zero matches, then requests the entire 21,332-byte reference. Code Mode requests that complete reference directly. The direct miss is not a transport failure or false analysis fact, but it adds work and makes an isolated adapter-efficiency interpretation inappropriate. Code Mode's lower cumulative/uncached input and nearly equal elapsed time in this sample must retain that limitation and uncontrolled cache/stochastic costs. Fix named discovery for both presentations before another same-question pair.

Ledger: .cache/atomic-codemode/ledger-projection-round-8114bd6.json; full-answer and retained-source review: projection-answer-review-v1.json beside it. The pair costs 771,383 input tokens (660,096 cached, 111,287 uncached), 7,768 output (1,281 included reasoning) and 272.155 model wall seconds. Native builds, scripted checks, development conversation, resolved server model build, remote compute and money remain outside these model-pair totals. There is no new traditional-provider cell, universal RE basis or broad superiority claim.

PHI alternatives after named discovery correction

Both runs use 14bc2bca2, the identical PHI question/oracle, preserved engine, model request and per-lane guidance as v1. Code Mode runs first. Runtime/engine/ target fingerprints, usage reconciliation, cleanup and OOM-zero receipts verify.

Measurement	Atomic direct projection v2	Code Mode projection v2
MCP requests / transport failures	16 / 0	12 / 0
Analysis API failures	1 timeout	0
Observed model responses	26	16
Client exec / wait calls	13 / 12	12 / 3
Model wall seconds	289.080	144.924
Family wall seconds	290.263	145.386
Family CPU seconds	150.551	45.887
Family memory peak MiB	842.2	854.9
Cumulative input tokens	577,633	457,492
Cached input tokens	548,864	413,056
Uncached input tokens	28,769	44,436
Output tokens, including reasoning	4,663	4,216
Reasoning tokens, included in output	973	769
Largest observed response input tokens	33,545	41,702

Both complete answers pass the unchanged oracle and cite nine verified sources. Every retained source hash/context and selected instruction/data byte verifies against the independent ELF file mapping. Successful native function records equal both v1 records. Code Mode retains ten sources; direct retains twelve, including two small instruction probes between analysis attempts. The direct first snapshot returns provider_timeout after 120 seconds with no output; retry succeeds in 53.124 seconds. Code Mode's snapshot succeeds in 37.226 seconds. The timeout cause remains unknown. The ledger now distinguishes a completed MCP transport from a failed direct API result; it previously counted only inner Code Mode failures. A regression preserves both error representations.

Named discovery no longer returns zero or requires an all-reference retry. Direct's seven fact contracts match; session tools open_binary/close_binary are explicitly unmatched because they are outside the fact API. Code Mode first asks for capture topics, then nine named contracts. Its v2 answer composes definition/operation reads instead of invoking slice.

These observations do not isolate an adapter speedup. Direct includes a genuine analysis timeout/retry; removing it would misstate whole-task cost. Direct's uncached input decreases while cumulative input/time increase. Code Mode's input, uncached input and time all increase from v1. Stochastic choices, prompt caching, uncontrolled OS state and one sample per cell remain confounders. Discovery and schemas are more correct; stable whole-task savings are not established.

Ledger: .cache/atomic-codemode/ledger-projection-discovery-round-14bc2bc.json; manual/source review: projection-answer-review-v2.json beside it. Pair totals: 1,035,125 input (961,920 cached, 73,205 uncached), 8,879 output (1,742 included reasoning), 434.004 model wall seconds. All four PHI launches cost 1,806,508 input (1,622,016 cached, 184,492 uncached), 16,647 output (3,023 included reasoning), 706.159 model wall seconds. These totals exclude builds/scripted verification, development conversation, remote compute and money. Memory is the kernel cgroup peak including charged file cache, not RSS; resolved model build is unknown.

Selected wide-PHI input relationships

Both runs use e38a4c439, the separately pre-pinned wide-operand question/oracle, same preserved a0685486... engine, fourteen-method API and model request gpt-6.1-sol/high. Direct runs first. Fresh leases/native caches and common guidance/resource limits remain. Runtime, engine and target fingerprints are unchanged; usage reconciliation, process/profile cleanup and OOM-zero verify.

Measurement	Atomic direct operands v1	Code Mode operands v1
MCP requests / transport failures	10 / 0	9 / 0
Analysis API failures	0	0
Observed model responses	12	12
Client exec / wait calls	8 / 3	8 / 3
Model wall seconds	91.817	95.699
Family wall seconds	92.844	96.267
Family CPU seconds	44.568	43.401
Family memory peak MiB	876.0	840.0
Cumulative input tokens	285,953	285,111
Cached input tokens	241,280	251,648
Uncached input tokens	44,673	33,463
Output tokens, including reasoning	1,654	1,868
Reasoning tokens, included in output	23	49
Largest observed response input tokens	28,546	29,240

Both complete answers pass the unchanged oracle: wide merge identity/opcode/ block/output/arity; all five requested value/predecessor/edge tuples; the two distinct values' exact storage and retained defining operation identities, opcodes and outputs. They preserve four distinct edges sharing one value, and separately prove slot 156 absent for the retained node while 0xffff:0 is unretained/unknown. Both state projection/frontier, CFG/use/alias/runtime and machine-transfer limits. Neither reads the whole input array or C report.

Each performs seven identical fact-method requests: one snapshot, two operation reads, three selected-operand relations and one batch value read. Each captures and cites one native source; every digest/context/reference verifies. Native function records equal each other and the earlier public golden. Native snapshot wall is 39.778 seconds direct and 38.627 Code Mode; neither retries. This is producer-gold/source verification for a seeded relationship task, not independent machine/IR equivalence, unseeded reconstruction or runtime proof.

Code Mode composes four related reads in one execution, then two dependent reads in another. It also makes four reference queries versus direct's one. Its initial natural-language query includes canonical values, so automatic named selection matches only that method and reports other words unmatched; a later lookup of invented rea.operation_inputs is explicitly unmatched. It then discovers the existing relations method correctly. These are retained discovery costs, not failed analysis calls. Direct already sees fact names/descriptions in its catalog. Fewer MCP requests therefore do not reduce observed model responses or client dispatch/wait calls in this sample.

The added operand atom works for both adapters, including selected absence and uncertainty. Code Mode's cumulative input is nearly equal, uncached input lower, and latency/output/largest response input higher. One sample, stochastic discovery and uncontrolled prompt/OS caching do not establish stable cost superiority. No same-question pre-change model cell exists: the old explicit inputs field could retrieve all slots, even though the old default single-node slice exceeds the output budget. Do not compare this task's lower cost to the earlier, broader PHI-format task as a speedup. A new discovery index/search experiment and broader address-unseeded tasks remain useful next tests.

Ledger: .cache/atomic-codemode/ledger-operands-round-e38a4c4.json; full answer/ source review: operands-answer-review-v1.json beside it. Pair totals: 571,064 input (492,928 cached, 78,136 uncached), 3,522 output (72 included reasoning), 187.516 model wall seconds. Builds, scripted verification, development conversation, resolved model build, remote compute and money remain unmeasured or outside these totals. Memory is kernel cgroup peak including charged cache.

Wide operands after the static discovery index

Both runs use a44004902, unchanged wide-operand question/oracle, native engine, fact implementation, model request and per-lane guidance. Code Mode runs first. Runtime/target/engine fingerprints and usage reconciliation pass; process/profile cleanup verifies, with no transport/API failure, retry, abort or OOM.

Measurement	Atomic direct operands v2	Code Mode operands v2
MCP requests / transport failures	10 / 0	7 / 0
Analysis API failures	0	0
Observed model responses	12	11
Client exec / wait calls	8 / 3	8 / 2
Model wall seconds	98.818	98.894
Family wall seconds	99.335	99.498
Family CPU seconds	44.307	44.866
Family memory peak MiB	870.0	851.6
Cumulative input tokens	235,201	268,311
Cached input tokens	196,224	245,376
Uncached input tokens	38,977	22,935
Output tokens, including reasoning	1,686	1,945
Reasoning tokens, included in output	60	35
Largest observed response input tokens	23,943	30,397

Both complete answers again pass every fixed fact and uncertainty requirement. Each uses the same seven fact requests, one native capture and only selected slots/values/immediate definitions. Each cites its sole verified source; function records equal each other, both v1 records and the earlier public golden. The native request takes 39.625 seconds direct and 39.209 Code Mode. This remains a seeded native relationship task, without a whole-input/C read or a runtime claim.

Code Mode uses the new fourteen-method index, then a single exact four-contract lookup, without unmatched names. Discovery falls from four requests to two; overall MCP requests from nine to seven, model responses from twelve to eleven. Direct uses one exact four-contract lookup in both revisions, leaving the two session-tool names explicitly unmatched. No analysis behavior or native coverage was changed to obtain these results.

Whole-task effects remain mixed. Code Mode's cumulative/uncached input decreases from its v1 sample, but elapsed time and maximum response input increase. Direct also uses fewer cumulative/uncached tokens and takes longer despite unchanged query counts. In this second pair, Code Mode has more cumulative input, less uncached input and almost identical wall time to direct. The index's intended discovery behavior works; fewer lookups do not prove general latency/token gains. One sample per cell, stochastic/client behavior, cached prompts and OS state remain uncontrolled. There is no traditional-provider cell for this question.

Ledger: .cache/atomic-codemode/ledger-operands-index-round-a440049.json; manual/source review: operands-answer-review-v2.json beside it. Pair totals: 503,512 input (441,600 cached, 61,912 uncached), 3,631 output (95 included reasoning), 197.712 model wall seconds. All four wide-operand launches cost 1,074,576 input (934,528 cached, 140,048 uncached), 7,153 output (167 included reasoning), 385.228 model wall seconds. Builds, scripted verification, development conversation, resolved model build, remote compute and money remain outside these model totals. Memory is kernel cgroup peak including charged cache.