experiment(re): distinguish point facts from object selectors
Apply the interior-reference counterexample to the facade: label exact-address scope and emit location endpoints while preserving caller selectors separately. Carry canonical target-content provenance through experimental MCP replies and verify it on the real client. Keep original measurements pinned to the v1 response shape. Nine experiment checks and real Kuna lifecycle/context checks pass. Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
This commit is contained in:
1 parent
72b24433ea
commit
d711c55aed
6 files changed
+31
-7
No files matched your search
@@ -120,6 +120,13 @@ shape remain the same; ordinary queries do not materialize the richer facts.
|
||||
Experimental MCP tools have live-validated input schemas, but no canonical
|
||||
output-schema/product-catalog integration. They are not a production SDK/API.
|
||||
|
||||
The v2 facade labels reference scope explicitly and connects **locations** in its
|
||||
point-reference edges. A result's `target` preserves the caller's selector; it
|
||||
is not a separate observation proving that an object exists at that address.
|
||||
All experimental MCP results carry REA's canonical untrusted-target-content
|
||||
marker. The v1 measurement table is retained with its revision label; it is not
|
||||
relabelled as a measurement of this later response shape.
|
||||
|
||||
## Reproduce
|
||||
|
||||
Use the repository's pinned Node/npm and run `npm ci`, then
|
||||
|
||||
@@ -58,6 +58,12 @@ const call = async (name, arguments_ = {}, expectError = false) => {
|
||||
{ timeout: 180000 },
|
||||
);
|
||||
wire.push({ request: { name, arguments: arguments_ }, response, phase });
|
||||
if (name.startsWith("experiment_"))
|
||||
assert.equal(
|
||||
response._meta?.provenance?.untrusted_content,
|
||||
true,
|
||||
"Experimental results lost the canonical target-content marker",
|
||||
);
|
||||
calls.push({ name, phase, elapsed_ms: performance.now() - started });
|
||||
if (expectError) {
|
||||
assert.equal(response.isError, true);
|
||||
|
||||
@@ -104,6 +104,7 @@ export class FactQueries {
|
||||
results.push({
|
||||
target,
|
||||
status: "ok",
|
||||
scope: { kind: "exact-address", address: target.address },
|
||||
...source(evidence),
|
||||
selection: page.selection,
|
||||
coverage: {
|
||||
@@ -113,7 +114,7 @@ export class FactQueries {
|
||||
},
|
||||
items: page.items.map((address) => ({
|
||||
from: { kind: "location", address },
|
||||
to: target,
|
||||
to: { kind: "location", address: target.address },
|
||||
kind: "unknown",
|
||||
kind_reason: "The existing xrefs contract projects addresses only.",
|
||||
})),
|
||||
|
||||
@@ -77,7 +77,7 @@ test("partial batches retain good observations and exact failure reasons", async
|
||||
assert.equal(result.results[0].coverage.status, "unknown");
|
||||
});
|
||||
|
||||
test("duplicate selectors share work but preserve separate string identities", async () => {
|
||||
test("duplicate point lookups share work without turning selectors into observed objects", async () => {
|
||||
let calls = 0;
|
||||
const facts = new FactQueries(
|
||||
port(async () => {
|
||||
@@ -92,9 +92,14 @@ test("duplicate selectors share work but preserve separate string identities", a
|
||||
});
|
||||
assert.equal(calls, 1);
|
||||
assert.deepEqual(
|
||||
result.results.map((row) => row.items[0].to),
|
||||
result.results.map((row) => row.target),
|
||||
[string, otherEncoding, string],
|
||||
);
|
||||
assert.deepEqual(
|
||||
result.results.map((row) => row.items[0].to),
|
||||
Array(3).fill({ kind: "location", address: string.address }),
|
||||
);
|
||||
assert.equal(result.results[0].scope.kind, "exact-address");
|
||||
});
|
||||
|
||||
test("paging and field selection retain identities, source and unknown coverage", async () => {
|
||||
|
||||
@@ -154,10 +154,8 @@ The exact-address composition therefore returns **unknown**, not a guessed
|
||||
nearby line. The mapped-code run cost 8.387 s / 8.091 CPU s / 202.0 MiB, including
|
||||
discovery/references and decompilation. It did not finish the analyst question
|
||||
"which condition triggers the error?" and must not count as a successful
|
||||
context-saving answer. A development attempt also used an unsupported
|
||||
single-function timeout flag; it failed and was corrected after checking actual
|
||||
CLI help. Single-function work uses REA's owned command deadline and native
|
||||
`--max-fn-memory`; it does not accept the batch `--max-fn-seconds` flag.
|
||||
context-saving answer. Single-function work uses REA's owned command deadline
|
||||
and the native `--max-fn-memory` option shown by the tested CLI's actual help.
|
||||
|
||||
The retained C text does contain the error literal at line 164, but a small
|
||||
fixed window there omits the earlier check around lines 122–124. Literal search
|
||||
@@ -190,6 +188,11 @@ does not claim to have completed that trial.
|
||||
paging, duplicate-selector, negative/unknown, retained-Evidence, foreign-context
|
||||
and expired-lease checks. Its live catalog had 212 tools and zero output
|
||||
schemas. Verification runs are excluded from the performance table.
|
||||
- The v2 facade was also exercised through a real Kuna MCP client: edges carry
|
||||
explicit point scope and location endpoints, caller selectors stay separate,
|
||||
and successful/error results retain the canonical untrusted-target marker.
|
||||
The same six additional verification groups passed. This v2 verification is
|
||||
not relabelled as a new performance comparison.
|
||||
- All real-provider measurements reported verified cleanup and zero OOM kills.
|
||||
No default parallel-suite stability, live-model reasoning improvement, general
|
||||
high-address code mapping, overlay support or newer Kuna compatibility is claimed.
|
||||
|
||||
@@ -6,6 +6,7 @@ import { createServer } from "../../dist/server/createServer.js";
|
||||
import { createEvidence } from "../../dist/domain/evidence.js";
|
||||
import { projectAnalysisError } from "../../dist/domain/analysisErrorProjection.js";
|
||||
import { toolContract } from "../../dist/contracts/toolContracts.js";
|
||||
import { TARGET_CONTENT_RESULT_META } from "../../dist/server/toolResultEncoding.js";
|
||||
import { expandProviderCapabilities } from "../../dist/contracts/toolOutputSchemaPrimitives.js";
|
||||
import { FactQueries, contextFor } from "./facts.mjs";
|
||||
|
||||
@@ -63,6 +64,7 @@ const contracts = [
|
||||
];
|
||||
|
||||
const encode = (value) => ({
|
||||
_meta: TARGET_CONTENT_RESULT_META,
|
||||
content: [{ type: "text", text: JSON.stringify(value) }],
|
||||
structuredContent: value,
|
||||
...(value.status === "error" ? { isError: true } : {}),
|
||||
|
||||
Reference in new issue
Block a user