diff --git a/experiments/atomic-codemode/README.md b/experiments/atomic-codemode/README.md index 450311d9a..e7256b76a 100644 --- a/experiments/atomic-codemode/README.md +++ b/experiments/atomic-codemode/README.md @@ -30,6 +30,12 @@ less cumulative input but more uncached input and peak memory than Direct. Traditional's full inventory is truncated by the client; one original cited payload remains only partially auditable. The study preserves that limitation. +The [string-fact precision study](string-facts-study.md) preserves native NUL +termination as true/false/unknown through the shared domain and public output +contract. Real canonical MCP, SDK, Direct, Code Mode and CLI checks retain both +NUL-terminated strings and a nonterminated printable run without length-based +agent inference. The old coarse descriptor remains compatible. + The [AArch64 reference study](aarch64-xref-study.md) finds and fixes missing ADRP-based string and memory references before the next model comparison. Real SDK and traditional MCP preflights now agree with independent machine diff --git a/experiments/atomic-codemode/api.mjs b/experiments/atomic-codemode/api.mjs index 7fb14a6cf..35c1b2801 100644 --- a/experiments/atomic-codemode/api.mjs +++ b/experiments/atomic-codemode/api.mjs @@ -111,7 +111,7 @@ export const methods = { summary: "Search retained encoded strings by literal text; discovery and reference attribution remain unknown.", description: - "Search retained string inventory by literal text. Paging does not establish artifact completeness.", + "Search retained string inventory by literal text. Encoded byte lengths and provider-reported NUL termination are preserved when known; omitted NUL status is unknown. A false NUL status makes no claim about bytes outside that span. Paging does not establish artifact completeness.", }, { query: z.string(), ...page }, "strings", diff --git a/experiments/atomic-codemode/string-facts-study.md b/experiments/atomic-codemode/string-facts-study.md new file mode 100644 index 000000000..e24aae82f --- /dev/null +++ b/experiments/atomic-codemode/string-facts-study.md @@ -0,0 +1,87 @@ +# Precise encoded-string facts + +This experiment preserves native NUL-termination observations through the shared +string contract. The branch refreshed main to `8451d2839` in `ba0ded9a7` first. +It remains an experiment without a PR or merge into main. + +The [pooled-string trial](string-span-study.md) exposed a real loss of precision: +Kuna's `nul_terminated` boolean was normalized only to the coarser legacy +`termination: present_or_not_required | missing`. The traditional model correctly +kept raw terminator coverage unknown. A second representation in the public MCP +output contract also omitted `encoding_status`, `provider_value` and `decoding`. +The provider already had those facts; agent-side inference would preserve the +producing-boundary defect. + +The domain string definition now includes optional `string.nul_terminated`: + +- `true`: the provider reports a NUL code unit terminating this encoded span. +- `false`: the reported encoded span has no observed terminating NUL; it makes + no claim about bytes outside the span. +- Omitted: that precise provider fact is unknown/unreported. + +Kuna's native parser carries its validated boolean directly. The existing coarse +termination field remains compatible. Other providers keep omission when they +do not report this fact; no Ghidra/Hopper/IDA termination behavior is inferred. +The public addressed-value output now reuses the domain string shape, removing +the duplicate metadata representation. CLI, SDK and both experimental MCP modes +consume that owner. This does not introduce another analysis pass or cache key. + +## Executed checks + +The pinned native engine remains +`2e398e7229e30b6d2387a802db8a0050764119a6b457020d334855e36e69d6c5`. +The real Linux SQLite target remains +`2e6eef9a727f081f0d453b4e5e6cbd8b9ef8b6f86cbf7681cbad444d3b0b55c8`. +Node/npm remain 24.18.0/11.16.0. No target was executed. + +- All 64 focused tests passed, including 38 native string parser cases, + production-boundary string queries and official tool contracts. The runtime + compiled successfully. After a test-helper extraction, the actual SDK output + schema test passed again. It verifies both string tools after conversion, + true/false/omitted states, malformed values and preservation of shared metadata. +- Real SDK, Direct MCP and Code Mode range verifiers passed the earlier eleven + occurrences and new boolean checks. `no such table` and `sqlite_schema` report + true, independently corroborated by selected loaded bytes. +- A real `.rodata` run at `0x11ee21`, text + `aceinouyyacdeeghijklnorstuuwyzouaiou`, reports false and 36 encoded bytes. + The selected byte window matches the text and its following byte is nonzero. + This is a retained printable run, not proof of a source-level string object + or the semantics of its consumer. +- The experimental CLI returned true/false correctly. The traditional real MCP + search also preserved both values, and its complete records authenticated with + the domain `parseEvidence` owner. The first canonical verifier incorrectly + expected substring search to return one row; it was corrected to select the + exact text among all matches. Preserve that failed verifier attempt. +- Changed-file lint passed with the existing large-file warnings. Full root + `check:fast` was not retried after its earlier bounded memory refusal; neither + `check:pr` nor real Ghidra/Hopper/IDA verification was run for this patch. +- The ignored product catalog and MCP test catalog were regenerated from the + built contracts; the converted output-schema checks above inspect the actual + advertised string facts rather than relying on a generated file alone. + +All following successful receipts verify owned payload cleanup and OOM=0: + +| Verification | Wall | Local family CPU | Kernel family peak | +| ---------------------------------- | -------: | ---------------: | -----------------: | +| Focused build + 64 tests | 60.830 s | 58.592501 s | 3,058.8 MiB | +| Final converted output-schema test | 6.569 s | 6.412996 s | 720.4 MiB | +| SDK real verifier | 15.024 s | 14.658184 s | 481.6 MiB | +| Direct real verifier | 15.218 s | 14.882060 s | 313.0 MiB | +| Code Mode real verifier | 15.926 s | 15.661633 s | 325.7 MiB | +| Canonical MCP verifier | 3.947 s | 3.773770 s | 408.3 MiB | +| Experimental CLI | 1.528 s | 1.333069 s | 159.1 MiB | + +The build uses a 3.25 GiB early stop; analysis uses 3 GiB. All are one CPU, +hard memory cap 4 GiB, no swap. Kernel process-family memory includes file cache, +not just RSS. Final exited-cgroup counters are unavailable; the table uses +verified payload-exit observations. These are different verification suites, +not equivalent-task speed comparisons or model token measurements. + +The large unfiltered inventory/client truncation gap from the earlier study +remains unresolved by this precision fix. Selected lookup and serialization +bounds still need exploration. Full Evidence remains retained within the lease; +explicit harness copies remain on disk after close. + +The same frozen pooled-string question and oracle will be used for the next +real Direct/Code Mode check. Record its outcomes separately rather than +attributing any timing difference to this small metadata change alone. diff --git a/experiments/atomic-codemode/verify-reference-ranges.mjs b/experiments/atomic-codemode/verify-reference-ranges.mjs index 1c634c02d..5ef62bab8 100644 --- a/experiments/atomic-codemode/verify-reference-ranges.mjs +++ b/experiments/atomic-codemode/verify-reference-ranges.mjs @@ -120,6 +120,7 @@ try { const string = matches[0]; assert.equal(string.string.encoding, "ascii"); assert.equal(string.string.byte_length, 14); + assert.equal(string.string.nul_terminated, true); ranges.push({ start: string.address, end_inclusive: `0x${(BigInt(string.address) + BigInt(string.string.byte_length) - 1n).toString(16)}`, @@ -196,6 +197,23 @@ try { bytes.results.map((r) => Buffer.from(r.hex, "hex").toString("ascii")), ["no such table\0", "sqlite_schema\0"], ); + // A real .rodata printable run is not necessarily a NUL-terminated literal. + const runText = "aceinouyyacdeeghijklnorstuuwyzouaiou"; + const runs = await read("strings", { query: runText, limit: 128 }); + const selectedRun = runs.items.filter((r) => r.address === "0x11ee21"); + assert.equal(selectedRun.length, 1); + assert.equal(selectedRun[0].value, runText); + assert.equal(selectedRun[0].string.nul_terminated, false); + assert.equal(selectedRun[0].string.byte_length, runText.length); + const runBytes = await read("bytes", { + windows: [{ address: "0x11ee21", length: runText.length + 1 }], + }); + const encodedRun = Buffer.from(runBytes.results[0].hex, "hex"); + assert.equal( + encodedRun.subarray(0, runText.length).toString("ascii"), + runText, + ); + assert.notEqual(encodedRun.at(-1), 0); const machine = await read("instructions", { windows: [ { address: "0x2d706", count: 6 }, diff --git a/src/contracts/toolOutputSchemaPrimitives.ts b/src/contracts/toolOutputSchemaPrimitives.ts index bee7c7eae..58871ab01 100644 --- a/src/contracts/toolOutputSchemaPrimitives.ts +++ b/src/contracts/toolOutputSchemaPrimitives.ts @@ -6,6 +6,7 @@ import { localVariableSchema, functionDossierSchema, functionBodySchema, + analysisStringSchema, } from "../domain/hopperValues.js"; import { analysisProfileSchema } from "../domain/analysisProfile.js"; import { @@ -573,8 +574,7 @@ const availableMemoryPermissions = z.object({ }); export const addressedValue = z.object({ - address: z.string(), - value: z.string(), + ...analysisStringSchema.shape, symbol: z .object({ primary: z.boolean(), @@ -591,13 +591,6 @@ export const addressedValue = z.object({ thunk_target: z.string().nullable(), }) .optional(), - string: z - .object({ - encoding: z.string(), - termination: z.enum(["missing", "present_or_not_required"]), - byte_length: z.number().int().min(0), - }) - .optional(), }); const memoryRegionOutput = z.object({ diff --git a/src/domain/hopperValues.ts b/src/domain/hopperValues.ts index 0f6c42ef4..f8c83e2bb 100644 --- a/src/domain/hopperValues.ts +++ b/src/domain/hopperValues.ts @@ -79,6 +79,12 @@ export const analysisStringSchema = z.object({ "Whether the provider declares the encoding or REA infers it from the typed bytes and native display. This does not establish the original source's intended encoding.", ), termination: z.enum(["missing", "present_or_not_required"]), + nul_terminated: z + .boolean() + .exactOptional() + .describe( + "Whether the provider observed a NUL code unit terminating this encoded span. Omitted means unknown; false makes no claim about bytes outside the span.", + ), byte_length: z.number().int().min(0), }) .exactOptional(), diff --git a/src/kuna/KunaStringInventory.ts b/src/kuna/KunaStringInventory.ts index 978f5b270..d5e6f17ff 100644 --- a/src/kuna/KunaStringInventory.ts +++ b/src/kuna/KunaStringInventory.ts @@ -159,6 +159,7 @@ export class KunaStringInventoryReader extends KunaJsonDocumentReader unknown, +) => { + const canonical = toolContract(name).outputSchema.shape.normalized_result; + const facts = { + address: "0x1000", + value: "hello", + provider_value: '"hello"', + string: { + encoding: "ascii", + encoding_status: "observed", + termination: "present_or_not_required", + byte_length: 6, + }, + }; + for (const status of [true, false, undefined]) { + const rows = [ + { + ...facts, + string: { + ...facts.string, + ...(status === undefined ? {} : { nul_terminated: status }), + }, + }, + ]; + expect(canonical.parse(rows)).toEqual(rows); + expect(validate(rows)).toBe(true); + } + for (const malformed of [null, "false", 0]) { + const rows = [ + { ...facts, string: { ...facts.string, nul_terminated: malformed } }, + ]; + expect(canonical.safeParse(rows).success).toBe(false); + expect(validate(rows)).toBe(false); + } +}; + it("advertises semantic output invariants and keeps producers aligned with Zod", async () => { const contracts = [ toolContract("capture_native_ui_scenario"), toolContract("capture_process_scenario"), toolContract("verify_managed_native_boundaries"), + toolContract("list_strings"), + toolContract("search_strings"), ]; const server = new McpServer({ name: "output-schema-semantics", @@ -96,6 +136,9 @@ it("advertises semantic output invariants and keeps producers aligned with Zod", }; }; + for (const name of ["list_strings", "search_strings"] as const) + verifyStringOutput(name, outputResultSchema(name).validate); + const native = outputResultSchema("capture_native_ui_scenario"); const initial = native.node(record(native.result.properties).initial); const screenshotUnion = native.node(record(initial.properties).screenshot); diff --git a/tests/boundary/providers/kuna/kunaStringInventory.test.ts b/tests/boundary/providers/kuna/kunaStringInventory.test.ts index 4bc1cb40b..20a81c95e 100644 --- a/tests/boundary/providers/kuna/kunaStringInventory.test.ts +++ b/tests/boundary/providers/kuna/kunaStringInventory.test.ts @@ -65,6 +65,7 @@ it("preserves exact u64 string identity, multiline text and typed source facts", encoding: "utf8", encoding_status: "observed", termination: "present_or_not_required", + nul_terminated: true, byte_length: 9, }, }, @@ -225,7 +226,10 @@ it.each([ ok: true, value: { items: [ - { value: facts.text, string: { byte_length: facts.byte_length } }, + { + value: facts.text, + string: { byte_length: facts.byte_length, nul_terminated: true }, + }, ], }, }); diff --git a/tests/boundary/providers/kuna/kunaStrings.test.ts b/tests/boundary/providers/kuna/kunaStrings.test.ts index ebb18a8b8..ac8b5ccbb 100644 --- a/tests/boundary/providers/kuna/kunaStrings.test.ts +++ b/tests/boundary/providers/kuna/kunaStrings.test.ts @@ -29,6 +29,7 @@ it("shares a typed string scan across concurrent searches and exact address read encoding: "ascii", encoding_status: "observed", termination: "present_or_not_required", + nul_terminated: true, byte_length: 19, }, }, @@ -41,6 +42,7 @@ it("shares a typed string scan across concurrent searches and exact address read encoding: "utf8", encoding_status: "observed", termination: "missing", + nul_terminated: false, byte_length: 6, }, },