fix(experiment): preserve precise native string facts

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>

experiments/atomic-codemode/README.md

@@ -30,6 +30,12 @@ less cumulative input but more uncached input and peak memory than Direct.
 Traditional's full inventory is truncated by the client; one original cited
 payload remains only partially auditable. The study preserves that limitation.
 
+The [string-fact precision study](string-facts-study.md) preserves native NUL
+termination as true/false/unknown through the shared domain and public output
+contract. Real canonical MCP, SDK, Direct, Code Mode and CLI checks retain both
+NUL-terminated strings and a nonterminated printable run without length-based
+agent inference. The old coarse descriptor remains compatible.
+
 The [AArch64 reference study](aarch64-xref-study.md) finds and fixes missing
 ADRP-based string and memory references before the next model comparison.
 Real SDK and traditional MCP preflights now agree with independent machine

experiments/atomic-codemode/api.mjs

@@ -111,7 +111,7 @@ export const methods = {
       summary:
         "Search retained encoded strings by literal text; discovery and reference attribution remain unknown.",
       description:
-        "Search retained string inventory by literal text. Paging does not establish artifact completeness.",
+        "Search retained string inventory by literal text. Encoded byte lengths and provider-reported NUL termination are preserved when known; omitted NUL status is unknown. A false NUL status makes no claim about bytes outside that span. Paging does not establish artifact completeness.",
     },
     { query: z.string(), ...page },
     "strings",

experiments/atomic-codemode/string-facts-study.md

@@ -0,0 +1,87 @@
+# Precise encoded-string facts
+
+This experiment preserves native NUL-termination observations through the shared
+string contract. The branch refreshed main to `8451d2839` in `ba0ded9a7` first.
+It remains an experiment without a PR or merge into main.
+
+The [pooled-string trial](string-span-study.md) exposed a real loss of precision:
+Kuna's `nul_terminated` boolean was normalized only to the coarser legacy
+`termination: present_or_not_required | missing`. The traditional model correctly
+kept raw terminator coverage unknown. A second representation in the public MCP
+output contract also omitted `encoding_status`, `provider_value` and `decoding`.
+The provider already had those facts; agent-side inference would preserve the
+producing-boundary defect.
+
+The domain string definition now includes optional `string.nul_terminated`:
+
+- `true`: the provider reports a NUL code unit terminating this encoded span.
+- `false`: the reported encoded span has no observed terminating NUL; it makes
+  no claim about bytes outside the span.
+- Omitted: that precise provider fact is unknown/unreported.
+
+Kuna's native parser carries its validated boolean directly. The existing coarse
+termination field remains compatible. Other providers keep omission when they
+do not report this fact; no Ghidra/Hopper/IDA termination behavior is inferred.
+The public addressed-value output now reuses the domain string shape, removing
+the duplicate metadata representation. CLI, SDK and both experimental MCP modes
+consume that owner. This does not introduce another analysis pass or cache key.
+
+## Executed checks
+
+The pinned native engine remains
+`2e398e7229e30b6d2387a802db8a0050764119a6b457020d334855e36e69d6c5`.
+The real Linux SQLite target remains
+`2e6eef9a727f081f0d453b4e5e6cbd8b9ef8b6f86cbf7681cbad444d3b0b55c8`.
+Node/npm remain 24.18.0/11.16.0. No target was executed.
+
+- All 64 focused tests passed, including 38 native string parser cases,
+  production-boundary string queries and official tool contracts. The runtime
+  compiled successfully. After a test-helper extraction, the actual SDK output
+  schema test passed again. It verifies both string tools after conversion,
+  true/false/omitted states, malformed values and preservation of shared metadata.
+- Real SDK, Direct MCP and Code Mode range verifiers passed the earlier eleven
+  occurrences and new boolean checks. `no such table` and `sqlite_schema` report
+  true, independently corroborated by selected loaded bytes.
+- A real `.rodata` run at `0x11ee21`, text
+  `aceinouyyacdeeghijklnorstuuwyzouaiou`, reports false and 36 encoded bytes.
+  The selected byte window matches the text and its following byte is nonzero.
+  This is a retained printable run, not proof of a source-level string object
+  or the semantics of its consumer.
+- The experimental CLI returned true/false correctly. The traditional real MCP
+  search also preserved both values, and its complete records authenticated with
+  the domain `parseEvidence` owner. The first canonical verifier incorrectly
+  expected substring search to return one row; it was corrected to select the
+  exact text among all matches. Preserve that failed verifier attempt.
+- Changed-file lint passed with the existing large-file warnings. Full root
+  `check:fast` was not retried after its earlier bounded memory refusal; neither
+  `check:pr` nor real Ghidra/Hopper/IDA verification was run for this patch.
+- The ignored product catalog and MCP test catalog were regenerated from the
+  built contracts; the converted output-schema checks above inspect the actual
+  advertised string facts rather than relying on a generated file alone.
+
+All following successful receipts verify owned payload cleanup and OOM=0:
+
+| Verification                       |     Wall | Local family CPU | Kernel family peak |
+| ---------------------------------- | -------: | ---------------: | -----------------: |
+| Focused build + 64 tests           | 60.830 s |      58.592501 s |        3,058.8 MiB |
+| Final converted output-schema test |  6.569 s |       6.412996 s |          720.4 MiB |
+| SDK real verifier                  | 15.024 s |      14.658184 s |          481.6 MiB |
+| Direct real verifier               | 15.218 s |      14.882060 s |          313.0 MiB |
+| Code Mode real verifier            | 15.926 s |      15.661633 s |          325.7 MiB |
+| Canonical MCP verifier             |  3.947 s |       3.773770 s |          408.3 MiB |
+| Experimental CLI                   |  1.528 s |       1.333069 s |          159.1 MiB |
+
+The build uses a 3.25 GiB early stop; analysis uses 3 GiB. All are one CPU,
+hard memory cap 4 GiB, no swap. Kernel process-family memory includes file cache,
+not just RSS. Final exited-cgroup counters are unavailable; the table uses
+verified payload-exit observations. These are different verification suites,
+not equivalent-task speed comparisons or model token measurements.
+
+The large unfiltered inventory/client truncation gap from the earlier study
+remains unresolved by this precision fix. Selected lookup and serialization
+bounds still need exploration. Full Evidence remains retained within the lease;
+explicit harness copies remain on disk after close.
+
+The same frozen pooled-string question and oracle will be used for the next
+real Direct/Code Mode check. Record its outcomes separately rather than
+attributing any timing difference to this small metadata change alone.

experiments/atomic-codemode/verify-reference-ranges.mjs

@@ -120,6 +120,7 @@ try {
     const string = matches[0];
     assert.equal(string.string.encoding, "ascii");
     assert.equal(string.string.byte_length, 14);
+    assert.equal(string.string.nul_terminated, true);
     ranges.push({
       start: string.address,
       end_inclusive: `0x${(BigInt(string.address) + BigInt(string.string.byte_length) - 1n).toString(16)}`,
@@ -196,6 +197,23 @@ try {
     bytes.results.map((r) => Buffer.from(r.hex, "hex").toString("ascii")),
     ["no such table\0", "sqlite_schema\0"],
   );
+  // A real .rodata printable run is not necessarily a NUL-terminated literal.
+  const runText = "aceinouyyacdeeghijklnorstuuwyzouaiou";
+  const runs = await read("strings", { query: runText, limit: 128 });
+  const selectedRun = runs.items.filter((r) => r.address === "0x11ee21");
+  assert.equal(selectedRun.length, 1);
+  assert.equal(selectedRun[0].value, runText);
+  assert.equal(selectedRun[0].string.nul_terminated, false);
+  assert.equal(selectedRun[0].string.byte_length, runText.length);
+  const runBytes = await read("bytes", {
+    windows: [{ address: "0x11ee21", length: runText.length + 1 }],
+  });
+  const encodedRun = Buffer.from(runBytes.results[0].hex, "hex");
+  assert.equal(
+    encodedRun.subarray(0, runText.length).toString("ascii"),
+    runText,
+  );
+  assert.notEqual(encodedRun.at(-1), 0);
   const machine = await read("instructions", {
     windows: [
       { address: "0x2d706", count: 6 },

src/contracts/toolOutputSchemaPrimitives.ts

@@ -6,6 +6,7 @@ import {
   localVariableSchema,
   functionDossierSchema,
   functionBodySchema,
+  analysisStringSchema,
 } from "../domain/hopperValues.js";
 import { analysisProfileSchema } from "../domain/analysisProfile.js";
 import {
@@ -573,8 +574,7 @@ const availableMemoryPermissions = z.object({
 });
 
 export const addressedValue = z.object({
-  address: z.string(),
-  value: z.string(),
+  ...analysisStringSchema.shape,
   symbol: z
     .object({
       primary: z.boolean(),
@@ -591,13 +591,6 @@ export const addressedValue = z.object({
       thunk_target: z.string().nullable(),
     })
     .optional(),
-  string: z
-    .object({
-      encoding: z.string(),
-      termination: z.enum(["missing", "present_or_not_required"]),
-      byte_length: z.number().int().min(0),
-    })
-    .optional(),
 });
 
 const memoryRegionOutput = z.object({

src/domain/hopperValues.ts

@@ -79,6 +79,12 @@ export const analysisStringSchema = z.object({
           "Whether the provider declares the encoding or REA infers it from the typed bytes and native display. This does not establish the original source's intended encoding.",
         ),
       termination: z.enum(["missing", "present_or_not_required"]),
+      nul_terminated: z
+        .boolean()
+        .exactOptional()
+        .describe(
+          "Whether the provider observed a NUL code unit terminating this encoded span. Omitted means unknown; false makes no claim about bytes outside the span.",
+        ),
       byte_length: z.number().int().min(0),
     })
     .exactOptional(),

src/kuna/KunaStringInventory.ts

@@ -159,6 +159,7 @@ export class KunaStringInventoryReader extends KunaJsonDocumentReader<KunaString
         termination: entry.nul_terminated
           ? "present_or_not_required"
           : "missing",
+        nul_terminated: entry.nul_terminated,
         byte_length: entry.byte_length,
       },
     });

tests/boundary/mcp/outputSchemaSemanticConstraints.test.ts

@@ -43,11 +43,51 @@ const schemaNode = (
   return { ...record(referenced), ...schema };
 };
 
+const verifyStringOutput = (
+  name: "list_strings" | "search_strings",
+  validate: (value: unknown) => unknown,
+) => {
+  const canonical = toolContract(name).outputSchema.shape.normalized_result;
+  const facts = {
+    address: "0x1000",
+    value: "hello",
+    provider_value: '"hello"',
+    string: {
+      encoding: "ascii",
+      encoding_status: "observed",
+      termination: "present_or_not_required",
+      byte_length: 6,
+    },
+  };
+  for (const status of [true, false, undefined]) {
+    const rows = [
+      {
+        ...facts,
+        string: {
+          ...facts.string,
+          ...(status === undefined ? {} : { nul_terminated: status }),
+        },
+      },
+    ];
+    expect(canonical.parse(rows)).toEqual(rows);
+    expect(validate(rows)).toBe(true);
+  }
+  for (const malformed of [null, "false", 0]) {
+    const rows = [
+      { ...facts, string: { ...facts.string, nul_terminated: malformed } },
+    ];
+    expect(canonical.safeParse(rows).success).toBe(false);
+    expect(validate(rows)).toBe(false);
+  }
+};
+
 it("advertises semantic output invariants and keeps producers aligned with Zod", async () => {
   const contracts = [
     toolContract("capture_native_ui_scenario"),
     toolContract("capture_process_scenario"),
     toolContract("verify_managed_native_boundaries"),
+    toolContract("list_strings"),
+    toolContract("search_strings"),
   ];
   const server = new McpServer({
     name: "output-schema-semantics",
@@ -96,6 +136,9 @@ it("advertises semantic output invariants and keeps producers aligned with Zod",
       };
     };
 
+    for (const name of ["list_strings", "search_strings"] as const)
+      verifyStringOutput(name, outputResultSchema(name).validate);
+
     const native = outputResultSchema("capture_native_ui_scenario");
     const initial = native.node(record(native.result.properties).initial);
     const screenshotUnion = native.node(record(initial.properties).screenshot);

tests/boundary/providers/kuna/kunaStringInventory.test.ts

@@ -65,6 +65,7 @@ it("preserves exact u64 string identity, multiline text and typed source facts",
             encoding: "utf8",
             encoding_status: "observed",
             termination: "present_or_not_required",
+            nul_terminated: true,
             byte_length: 9,
           },
         },
@@ -225,7 +226,10 @@ it.each([
     ok: true,
     value: {
       items: [
-        { value: facts.text, string: { byte_length: facts.byte_length } },
+        {
+          value: facts.text,
+          string: { byte_length: facts.byte_length, nul_terminated: true },
+        },
       ],
     },
   });

tests/boundary/providers/kuna/kunaStrings.test.ts

@@ -29,6 +29,7 @@ it("shares a typed string scan across concurrent searches and exact address read
           encoding: "ascii",
           encoding_status: "observed",
           termination: "present_or_not_required",
+          nul_terminated: true,
           byte_length: 19,
         },
       },
@@ -41,6 +42,7 @@ it("shares a typed string scan across concurrent searches and exact address read
           encoding: "utf8",
           encoding_status: "observed",
           termination: "missing",
+          nul_terminated: false,
           byte_length: 6,
         },
       },