fix(javascript): bind native handoffs to canonical byte identity
Require the native node owner's content digest instead of borrowing an observation artifact digest, which can identify the containing ASAR. Keep unavailable nodes and their path/provenance/partial facts in the trace, without a false actionable target or native Evidence link. Remove the old fallback and allocate the query index only for eligible native targets. Verify real public CLI/stdio MCP workflows, known/missing companion inputs and shipped Joplin controls. Preserve reproducible scripts, source pins and compact correctness/resource/token records without archiving applications. Fixes #3926. Part of #4027. Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
This commit is contained in:
YsopwnableandKaoru0822 authored and GitHub committed
1 parent
633e4ff434
commit
c7f6b8bad1
7 files changed
+1820
-13
No files matched your search
@@ -95,6 +95,14 @@ matches exactly. Otherwise the result recommends provider-neutral follow-up
|
||||
tools and reports `requires-provider-analysis`; it never starts or switches a
|
||||
provider implicitly.
|
||||
|
||||
A handoff requires the add-on's exact content identity. Unavailable unpacked
|
||||
companions remain visible in the trace graph with their canonical path,
|
||||
availability and container-backed Evidence. They yield no actionable handoff
|
||||
until their bytes are available and the application is reanalyzed; the archive
|
||||
digest does not identify the missing add-on. An empty handoff list does not prove
|
||||
that the trace contains no native components. Source partial coverage remains
|
||||
visible independently of handoff eligibility.
|
||||
|
||||
## Semantic relation tracing
|
||||
|
||||
`trace_javascript_semantics` queries the authenticated companion graph without
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,179 @@
|
||||
# Native handoff byte-identity verification
|
||||
|
||||
Tracks #3926 under #4027. Baseline: `9f8659e0536e4e50a438f5b9d5661f3c9624f6ac`.
|
||||
Final integration base: `62bf09809ef7bb224028916424f8f07652a20f4f`.
|
||||
|
||||
The analyst already knows an ASAR-relative native path and supplies prior native
|
||||
Evidence. The question is which Evidence can be linked to that component's exact
|
||||
bytes. Discovery and native-provider execution are outside this task.
|
||||
|
||||
## Observed behavior and correction
|
||||
|
||||
The native-node producer distinguishes a content digest from a canonical path
|
||||
whose bytes are unavailable. Its unavailable observation legitimately cites the
|
||||
containing archive. The handoff consumer previously treated that supporting
|
||||
artifact digest as the native component's digest.
|
||||
|
||||
Public CLI/stdio MCP regression: the missing-companion case failed and the
|
||||
available-byte control passed. Real Joplin reproduced the same result: missing
|
||||
keytar bytes produced an `evidence-linked` handoff with the ASAR digest and the
|
||||
simulated container observation. CLI/MCP agreed on that incorrect answer.
|
||||
|
||||
The consumer now uses only the node owner's content-digest identity. The fallback
|
||||
observation scan is removed. Unavailable native nodes retain their canonical
|
||||
path, availability, container-backed provenance and partial coverage in the
|
||||
inline graph; they yield no false actionable handoff or native Evidence link.
|
||||
An empty handoff list describes eligibility, not absence of native components.
|
||||
The per-query node index is allocated only after an eligible target exists; no
|
||||
persistent cache, schema, tool or target authority is added.
|
||||
|
||||
Final real checks passed both cases with exact CLI/MCP trace parity. The available
|
||||
control preserved the complete parent/trace Evidence IDs, native facts, links,
|
||||
coverage and all token projections. The missing control preserved its parent,
|
||||
native node, source provenance and coverage, while removing the wrong handoff
|
||||
and container Evidence link.
|
||||
|
||||
## Real input and scope
|
||||
|
||||
[Official Joplin v3.7.21 Linux AppImage](https://github.com/laurent22/joplin/releases/download/v3.7.21/Joplin-3.7.21.AppImage),
|
||||
published 2026-09-25, 201,462,481 bytes, SHA-256
|
||||
`5d202dcaca31e14206826aa3f04125e7c218dbc3dc3d98479d811efbd6b856c1`.
|
||||
|
||||
Read-only 7z 26.02 inspection found 6,324 ASAR files, 46 unpacked entries and
|
||||
three `.node` components. Extract only the ASAR and those three binaries:
|
||||
|
||||
```sh
|
||||
input_root="$(mktemp -d -t rea-native-handoff-XXXXXX)"
|
||||
curl --fail --location --output "$input_root/Joplin-3.7.21.AppImage" \
|
||||
https://github.com/laurent22/joplin/releases/download/v3.7.21/Joplin-3.7.21.AppImage
|
||||
sha256sum "$input_root/Joplin-3.7.21.AppImage"
|
||||
7z x -mmt=1 "-o$input_root/extracted" "$input_root/Joplin-3.7.21.AppImage" \
|
||||
resources/app.asar \
|
||||
resources/app.asar.unpacked/node_modules/keytar/build/Release/keytar.node \
|
||||
resources/app.asar.unpacked/node_modules/sqlite3/lib/binding/napi-v6-linux-glibc-x64/node_sqlite3.node \
|
||||
resources/app.asar.unpacked/node_modules/onnxruntime-node/bin/napi-v6/linux/x64/onnxruntime_binding.node
|
||||
python3 - "$input_root" <<'PY'
|
||||
import os, sys
|
||||
from pathlib import Path
|
||||
root = Path(sys.argv[1])
|
||||
source = root / 'extracted/resources'
|
||||
for mode in ('available', 'missing'):
|
||||
case = root / 'cases' / mode
|
||||
case.mkdir(parents=True)
|
||||
os.link(source / 'app.asar', case / 'app.asar')
|
||||
if mode == 'available':
|
||||
for native in (source / 'app.asar.unpacked').rglob('*.node'):
|
||||
target = case / native.relative_to(source)
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
os.link(native, target)
|
||||
PY
|
||||
```
|
||||
|
||||
Verify the asset digest above before extraction. The ASAR is 233,575,014 bytes,
|
||||
SHA-256 `767df409b18effe60421fbefcca873de237c7e813eac744ecdeebbc5e825a33a`.
|
||||
Keytar is 76,960 bytes, SHA-256
|
||||
`0700016bcd761cbe02d275f2046c7cc42f39f730a50d66e3d3307aa9d413373a`.
|
||||
All extracted component hashes are in the JSON record. 7z reported one
|
||||
trailing-data warning for the AppImage's embedded SquashFS and exited 0.
|
||||
|
||||
Both cases select only `package.json` for static source analysis while retaining
|
||||
the inventory. The available case stages all three native companions; the other
|
||||
43 unpacked entries are not staged. The missing case stages none. Whole-application
|
||||
coverage is therefore partial in both cases, independently of the selected
|
||||
native component's known or unknown byte identity. Neither the application nor
|
||||
a native analysis provider is executed. Native Evidence is explicitly simulated
|
||||
to test exact subject linkage. Live Electron, real Ghidra/Hopper/IDA behavior and
|
||||
Windows/macOS verification are **NOT_ENOUGH_INFO**.
|
||||
|
||||
## Reproduce the public workflow
|
||||
|
||||
Use pinned Node 24.18.0/npm 11.16.0, `npm ci`, then `npm run build:cached`.
|
||||
The verifier returns compact records and deletes its private exported parent and
|
||||
CLI input after owned cleanup. No downloadable application or large Evidence is
|
||||
archived.
|
||||
|
||||
```sh
|
||||
for mode in available missing; do
|
||||
node scripts/verify/javascript/native-handoffs.mjs \
|
||||
--artifact "$input_root/cases/$mode/app.asar" \
|
||||
--addon-path node_modules/keytar/build/Release/keytar.node \
|
||||
--native-sha256 0700016bcd761cbe02d275f2046c7cc42f39f730a50d66e3d3307aa9d413373a \
|
||||
--source-path package.json --rounds 3 --cli-parity --require-safe
|
||||
done
|
||||
```
|
||||
|
||||
Add `--tokenizer-python /absolute/path/to/python-with-tiktoken-0.12.0` for the
|
||||
reported BPE projections. The npm alias `verify:javascript:native-handoffs`
|
||||
builds first; recorded costs use the direct Node invocation above after building.
|
||||
For a pre-fix observation, run the saved verifier on the baseline build without
|
||||
`--require-safe`; `safe_identity: false` is an incorrect answer, not a passing
|
||||
analyst task. Paths affect serialized requests and some Evidence identities.
|
||||
|
||||
The exact resource runs additionally used the preserved
|
||||
[8 GiB observer](https://github.com/morluto/rea/blob/b3bdabab694af1f9801eff7caf75001b92314a58/experiments/analysis-efficiency/resource-run.py)
|
||||
and adjacent `resource_payload.py`; both hashes are pinned in the JSON.
|
||||
|
||||
## Completed workflow costs and limits
|
||||
|
||||
| Case | Revision | Wall s | CPU s | Peak cgroup bytes | Identity oracle |
|
||||
| ---------------------- | -------- | ------: | ---------: | ----------------: | -------------------------- |
|
||||
| Native bytes available | Before | 60.369 | 59.806670 | 2,662,629,376 | pass |
|
||||
| Native bytes available | After | 258.885 | 106.221003 | 2,678,456,320 | pass, exact baseline facts |
|
||||
| Native bytes missing | Before | 57.930 | 57.479109 | 2,719,457,280 | wrong identity/linkage |
|
||||
| Native bytes missing | After | 59.738 | 59.254776 | 2,571,235,328 | pass, unknown retained |
|
||||
|
||||
Whole-verifier resources include input hashing, analysis, three retained trace
|
||||
queries, export, CLI parity and tokenization. One CPU, 8 GiB hard cap, 6 GiB early
|
||||
stop, no swap; cleanup verified and no OOM. Memory includes charged file cache,
|
||||
not summed RSS. Counters are at payload exit while the observer remains alive.
|
||||
The available control's final wall and CPU times are substantially higher than
|
||||
baseline. Earlier post-fix controls on the original base took 61.657/74.956 s wall
|
||||
and 61.135962/60.352428 s CPU for available/missing cases; their complete receipts
|
||||
remain separately recorded. The final missing case took 59.738 s wall. This
|
||||
variability is unresolved and cannot be attributed to the native consumer change.
|
||||
These runs establish neither speedup nor performance equivalence. Input hashing warms the ASAR
|
||||
cache, build/OS caches and shared-host scheduling are uncontrolled. Fresh MCP
|
||||
sessions do not establish OS-cold conditions. First post-analysis and repeated
|
||||
warm-query samples are preserved separately; query CPU/memory are unmeasured.
|
||||
|
||||
One defined task is one analysis summary plus one retained trace, with the path
|
||||
and prior Evidence already supplied. Static `cl100k_base`/tiktoken 0.12.0 counts
|
||||
include both requests, replies and name/description/inputSchema contracts:
|
||||
|
||||
| Available-byte task, unchanged before/after | BPE tokens |
|
||||
| ----------------------------------------------- | ---------: |
|
||||
| Selected catalog + structured replies | 12,340 |
|
||||
| Selected catalog + complete tool replies | 18,906 |
|
||||
| Full catalog projection + structured replies | 202,292 |
|
||||
| Full catalog projection + complete tool replies | 208,858 |
|
||||
|
||||
These are alternative reply/catalog projections. The selected catalog alone is
|
||||
5,570 BPE; the full projection is 195,522. Output schemas, annotations, host
|
||||
rendering/replay, reasoning, final answers and model billing are unmeasured.
|
||||
Repeated queries/export/CLI/tokenization are validation overhead outside this
|
||||
one-task token count, but inside resource receipts. Missing-case structured-task
|
||||
counts change 11,372 → 11,103 by removing unsupported claims; the incorrect
|
||||
baseline earns no efficiency credit.
|
||||
|
||||
## Regression and verification record
|
||||
|
||||
The initial four-file focused selection passed 24 tests. The two public cases
|
||||
were rerun after adding named summary-result parsing and correcting an optional
|
||||
architecture factory input. Omitting that field instead of passing null produced
|
||||
byte-identical normalized simulated Evidence in all four real-case inputs.
|
||||
The initial CLI-selector harness error and first typecheck failure are retained
|
||||
as harness/validation attempts, not product reproductions.
|
||||
|
||||
After syncing the reviewed upstream changes, the updated application workflow
|
||||
CLI acceptance file passed 34 tests, bringing the focused selection to 58 distinct
|
||||
cases across five files. Both Joplin controls were then repeated on the final
|
||||
integration base. Canonical byte identity is also used by the related version
|
||||
matching/content comparison consumers; no equivalent observation-digest fallback
|
||||
was found there.
|
||||
|
||||
Final `npm run check && npm run docs:check` passed on the integration base,
|
||||
including five documentation consistency tests: 128.732 s wall, 115.518920 s CPU,
|
||||
4,301,684,736 B peak cgroup memory, clean owned exit and no OOM.
|
||||
|
||||
Final public and static/docs outcomes are recorded in the
|
||||
[compact measurements and source pins](javascript-native-handoff-identity-2026-10-15.json).
|
||||
@@ -214,6 +214,7 @@
|
||||
"verify:javascript:digests": "npm run build:cached && node scripts/verify/javascript/digests.mjs",
|
||||
"verify:javascript:output": "npm run build:cached && node scripts/verify/javascript/output.mjs",
|
||||
"verify:javascript:module-references": "npm run build:cached && node scripts/verify/javascript/module-references.mjs",
|
||||
"verify:javascript:native-handoffs": "npm run build:cached && node scripts/verify/javascript/native-handoffs.mjs",
|
||||
"verify:application-workflows": "npm run build:cached && node scripts/verify/javascript/application-workflows.mjs",
|
||||
"verify:managed": "npm run build:cached && node scripts/verify/managed/conformance.mjs",
|
||||
"verify:firmware": "npm run build:cached && node scripts/verify/firmware/analysis.mjs",
|
||||
|
||||
@@ -0,0 +1,348 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import assert from "node:assert/strict";
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import { createReadStream } from "node:fs";
|
||||
import { readFile, writeFile } from "node:fs/promises";
|
||||
import { isAbsolute, join } from "node:path";
|
||||
import { performance } from "node:perf_hooks";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { parseArgs } from "node:util";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { createEvidence } from "../../../dist/domain/evidence.js";
|
||||
import { PrivateRuntimeRoot } from "../../../dist/process/PrivateRuntimeRoot.js";
|
||||
import {
|
||||
OwnedCommandFailure,
|
||||
runOwnedCommand,
|
||||
} from "../../../dist/process/OwnedCommand.js";
|
||||
import { closeVerifierResources } from "../../lib/verifier-cleanup.mjs";
|
||||
|
||||
const { values } = parseArgs({
|
||||
options: {
|
||||
artifact: { type: "string" },
|
||||
"addon-path": { type: "string" },
|
||||
"native-sha256": { type: "string" },
|
||||
"source-path": { type: "string", multiple: true },
|
||||
rounds: { type: "string", default: "3" },
|
||||
"cli-parity": { type: "boolean", default: false },
|
||||
"tokenizer-python": { type: "string" },
|
||||
"require-safe": { type: "boolean", default: false },
|
||||
},
|
||||
strict: true,
|
||||
});
|
||||
assert.ok(
|
||||
values.artifact && isAbsolute(values.artifact),
|
||||
"--artifact must be absolute",
|
||||
);
|
||||
assert.ok(
|
||||
values["addon-path"] &&
|
||||
!isAbsolute(values["addon-path"]) &&
|
||||
values["addon-path"]
|
||||
.split("/")
|
||||
.every((part) => part !== "" && part !== "." && part !== ".."),
|
||||
"--addon-path must be a normalized archive-relative path",
|
||||
);
|
||||
assert.match(
|
||||
values["native-sha256"] ?? "",
|
||||
/^[a-f0-9]{64}$/,
|
||||
"--native-sha256 pins prior native bytes; it does not prove availability",
|
||||
);
|
||||
const rounds = Number(values.rounds);
|
||||
assert.ok(Number.isSafeInteger(rounds) && rounds > 0);
|
||||
const environment = {
|
||||
PATH: process.env.PATH ?? "/usr/bin:/bin",
|
||||
REA_LOG_LEVEL: "silent",
|
||||
REA_ANALYSIS_PROVIDER: "auto",
|
||||
HOPPER_LAUNCHER_PATH: "/rea-unconfigured-deep-provider/hopper",
|
||||
};
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [fileURLToPath(new URL("../../rea.mjs", import.meta.url)), "mcp"],
|
||||
env: environment,
|
||||
stderr: "pipe",
|
||||
});
|
||||
const client = new Client({
|
||||
name: "javascript-native-handoff-verifier",
|
||||
version: "1",
|
||||
});
|
||||
const serialize = (value) => JSON.stringify(value);
|
||||
const digest = (value) =>
|
||||
createHash("sha256").update(serialize(value)).digest("hex");
|
||||
const fileDigest = async (path) => {
|
||||
const hash = createHash("sha256");
|
||||
for await (const chunk of createReadStream(path)) hash.update(chunk);
|
||||
return hash.digest("hex");
|
||||
};
|
||||
const simulatedNativeEvidence = (path, sha256) =>
|
||||
createEvidence(
|
||||
{ path, sha256, format: "raw-binary" },
|
||||
{ id: "ghidra", name: "Ghidra", version: "simulated" },
|
||||
{
|
||||
operation: "binary_overview",
|
||||
parameters: {},
|
||||
result: {
|
||||
summary:
|
||||
"Simulated subject-linkage input; no native provider executed.",
|
||||
},
|
||||
limitations: ["No native provider behavior is established."],
|
||||
},
|
||||
);
|
||||
let temporary;
|
||||
let primaryError;
|
||||
let retainTemporary = false;
|
||||
let report;
|
||||
const ownedCommand = async (command, arguments_, timeoutMs = 60_000) => {
|
||||
try {
|
||||
return await runOwnedCommand(
|
||||
{
|
||||
command,
|
||||
arguments: arguments_,
|
||||
cwd: temporary.path,
|
||||
env: environment,
|
||||
runId: `rea-native-handoff-${randomUUID()}`,
|
||||
expectedCommand: command === process.execPath ? process.execPath : null,
|
||||
sweepTokenOwnedProcesses: true,
|
||||
},
|
||||
{ timeoutMs, diagnosticBytes: 16 * 1024 * 1024 },
|
||||
);
|
||||
} catch (cause) {
|
||||
retainTemporary ||=
|
||||
cause instanceof OwnedCommandFailure && cause.cleanupOwner !== undefined;
|
||||
throw cause;
|
||||
}
|
||||
};
|
||||
try {
|
||||
const archiveSha256 = await fileDigest(values.artifact);
|
||||
const companion = join(`${values.artifact}.unpacked`, values["addon-path"]);
|
||||
let companionSha256 = null;
|
||||
try {
|
||||
companionSha256 = await fileDigest(companion);
|
||||
} catch (cause) {
|
||||
if (cause?.code !== "ENOENT") throw cause;
|
||||
}
|
||||
if (companionSha256 !== null)
|
||||
assert.equal(companionSha256, values["native-sha256"]);
|
||||
const native = simulatedNativeEvidence(companion, values["native-sha256"]);
|
||||
const container = simulatedNativeEvidence(values.artifact, archiveSha256);
|
||||
await client.connect(transport);
|
||||
transport.stderr?.on("data", () => undefined);
|
||||
const tools = (await client.listTools()).tools;
|
||||
const catalog = tools.map(({ name, description, inputSchema }) => ({
|
||||
name,
|
||||
description,
|
||||
inputSchema,
|
||||
}));
|
||||
const selected = catalog.filter(({ name }) =>
|
||||
["analyze_javascript_application", "trace_application_feature"].includes(
|
||||
name,
|
||||
),
|
||||
);
|
||||
assert.equal(selected.length, 2);
|
||||
const call = async (name, arguments_) => {
|
||||
const response = await client.callTool(
|
||||
{ name, arguments: arguments_ },
|
||||
{ timeout: 360_000 },
|
||||
);
|
||||
assert.notEqual(response.isError, true, serialize(response));
|
||||
assert.ok(response.structuredContent?.evidence_id);
|
||||
return {
|
||||
request: { name, arguments: arguments_ },
|
||||
response: response.structuredContent,
|
||||
wire_response: response,
|
||||
};
|
||||
};
|
||||
const prepStart = performance.now();
|
||||
const preparation = await call("analyze_javascript_application", {
|
||||
input_path: values.artifact,
|
||||
detail: "summary",
|
||||
...(values["source-path"] === undefined
|
||||
? {}
|
||||
: { source_paths: values["source-path"] }),
|
||||
});
|
||||
const preparationMs = performance.now() - prepStart;
|
||||
assert.equal(preparation.response.subject.digest.sha256, archiveSha256);
|
||||
const sourceId = preparation.response.normalized_result.parent_evidence_id;
|
||||
assert.match(sourceId, /^ev_[a-f0-9]{64}$/);
|
||||
const query = {
|
||||
application: { kind: "retained-evidence", evidence_id: sourceId },
|
||||
native_observations: [container, native],
|
||||
seed: {
|
||||
kind: "string",
|
||||
value: values["addon-path"],
|
||||
match: "exact",
|
||||
case_sensitive: true,
|
||||
},
|
||||
direction: "outgoing",
|
||||
};
|
||||
const samples = [];
|
||||
let first;
|
||||
for (let round = 0; round < rounds; round++) {
|
||||
const start = performance.now();
|
||||
const result = await call("trace_application_feature", query);
|
||||
const elapsedMs = performance.now() - start;
|
||||
if (first === undefined) first = result;
|
||||
else assert.equal(result.response.evidence_id, first.response.evidence_id);
|
||||
samples.push({
|
||||
round,
|
||||
elapsed_ms: elapsedMs,
|
||||
structured_reply_bytes: Buffer.byteLength(serialize(result.response)),
|
||||
full_tool_reply_bytes: Buffer.byteLength(serialize(result.wire_response)),
|
||||
});
|
||||
}
|
||||
const trace = first.response.normalized_result;
|
||||
const addons = trace.graph.nodes.filter(
|
||||
({ kind }) => kind === "native-addon",
|
||||
);
|
||||
assert.equal(
|
||||
addons.length,
|
||||
1,
|
||||
"The selected native add-on must remain visible",
|
||||
);
|
||||
const addon = addons[0];
|
||||
const unavailable = addon.observations.some(
|
||||
({ properties }) => properties.availability === "unavailable",
|
||||
);
|
||||
const expectedLinks = [
|
||||
sourceId,
|
||||
...(companionSha256 === null ? [] : [native.evidence_id]),
|
||||
].sort();
|
||||
const safeIdentity =
|
||||
serialize(trace.evidence_links) === serialize(expectedLinks) &&
|
||||
(companionSha256 === null
|
||||
? unavailable &&
|
||||
addon.identity.strategy !== "content-digest" &&
|
||||
trace.coverage.status === "partial" &&
|
||||
trace.native_handoffs.length === 0
|
||||
: addon.identity.strategy === "content-digest" &&
|
||||
addon.identity.sha256 === companionSha256 &&
|
||||
trace.native_handoffs.length === 1 &&
|
||||
trace.native_handoffs[0].artifact_sha256 === companionSha256 &&
|
||||
trace.native_handoffs[0].status === "evidence-linked" &&
|
||||
serialize(trace.native_handoffs[0].evidence_ids) ===
|
||||
serialize([native.evidence_id]));
|
||||
let cliParity = null;
|
||||
if (values["cli-parity"]) {
|
||||
temporary = await PrivateRuntimeRoot.create({
|
||||
prefix: "rea-native-handoff-verifier-",
|
||||
});
|
||||
const bundlePath = join(temporary.path, "bundle.json");
|
||||
const exported = await client.callTool({
|
||||
name: "export_evidence_bundle",
|
||||
arguments: { path: bundlePath },
|
||||
});
|
||||
assert.notEqual(exported.isError, true, serialize(exported));
|
||||
const bundle = JSON.parse(await readFile(bundlePath, "utf8"));
|
||||
const parent = bundle.records.find(
|
||||
({ evidence_id }) => evidence_id === sourceId,
|
||||
);
|
||||
assert.ok(parent, "Export omitted the retained parent");
|
||||
const request = join(temporary.path, "trace.json");
|
||||
// The CLI's portable file contract takes complete Evidence; MCP resolves a
|
||||
// same-session reference into the same shared application workflow.
|
||||
await writeFile(request, serialize({ ...query, application: parent }));
|
||||
const result = await ownedCommand(
|
||||
process.execPath,
|
||||
[
|
||||
fileURLToPath(new URL("../../rea.mjs", import.meta.url)),
|
||||
"trace-application-feature",
|
||||
request,
|
||||
"--json",
|
||||
],
|
||||
120_000,
|
||||
);
|
||||
const evidence = JSON.parse(result.stdout.text);
|
||||
assert.deepEqual(evidence.normalized_result, trace);
|
||||
assert.equal(evidence.evidence_id, first.response.evidence_id);
|
||||
cliParity = {
|
||||
exact_cli_mcp_trace_parity: true,
|
||||
evidence_id: evidence.evidence_id,
|
||||
};
|
||||
}
|
||||
let tokenCosts = null;
|
||||
if (values["tokenizer-python"]) {
|
||||
temporary ??= await PrivateRuntimeRoot.create({
|
||||
prefix: "rea-native-handoff-verifier-",
|
||||
});
|
||||
const input = join(temporary.path, "token-input.json");
|
||||
await writeFile(
|
||||
input,
|
||||
serialize({ catalog, selected, preparation, query: first }),
|
||||
);
|
||||
const result = await ownedCommand(values["tokenizer-python"], [
|
||||
"-c",
|
||||
`import json,sys,tiktoken
|
||||
with open(sys.argv[1],encoding='utf-8') as f:d=json.load(f)
|
||||
enc=tiktoken.get_encoding('cl100k_base')
|
||||
def n(v):return len(enc.encode(json.dumps(v,ensure_ascii=False,separators=(',',':')),disallowed_special=()))
|
||||
requests=n(d['preparation']['request'])+n(d['query']['request'])
|
||||
structured=n(d['preparation']['response'])+n(d['query']['response'])
|
||||
full_reply=n(d['preparation']['wire_response'])+n(d['query']['wire_response'])
|
||||
s=n(d['selected']);a=n(d['catalog'])
|
||||
print(json.dumps(dict(tokenizer='cl100k_base',tiktoken_version=tiktoken.__version__,selected_catalog=s,full_catalog=a,requests=requests,structured_replies=structured,full_tool_replies=full_reply,selected_catalog_structured_task=s+requests+structured,full_catalog_structured_task=a+requests+structured,selected_catalog_full_reply_task=s+requests+full_reply,full_catalog_full_reply_task=a+requests+full_reply)))`,
|
||||
input,
|
||||
]);
|
||||
tokenCosts = JSON.parse(result.stdout.text);
|
||||
}
|
||||
report = {
|
||||
scope:
|
||||
"Real shipped-ASAR native identity through public CLI/stdio MCP; simulated native Evidence only, no provider/application executed",
|
||||
artifact: { path: values.artifact, sha256: archiveSha256 },
|
||||
addon_path: values["addon-path"],
|
||||
companion_sha256_observed: companionSha256,
|
||||
prior_native_sha256_supplied: values["native-sha256"],
|
||||
source_paths: values["source-path"] ?? null,
|
||||
parent_evidence_id: sourceId,
|
||||
trace_evidence_id: first.response.evidence_id,
|
||||
preparation_ms: preparationMs,
|
||||
samples,
|
||||
safe_identity: safeIdentity,
|
||||
node: {
|
||||
node_id: addon.node_id,
|
||||
identity: addon.identity,
|
||||
unavailable_observed: unavailable,
|
||||
basis: addon.observations.map(({ evidence }) => ({
|
||||
authority: evidence.authority,
|
||||
state: evidence.state,
|
||||
confidence: evidence.confidence,
|
||||
artifact: evidence.artifact,
|
||||
location: evidence.location,
|
||||
})),
|
||||
},
|
||||
native_handoffs: trace.native_handoffs,
|
||||
evidence_links: trace.evidence_links,
|
||||
source_coverage: trace.coverage,
|
||||
graph_coverage: trace.graph.coverage,
|
||||
facts_sha256: digest({
|
||||
node: addon,
|
||||
native_handoffs: trace.native_handoffs,
|
||||
evidence_links: trace.evidence_links,
|
||||
coverage: trace.coverage,
|
||||
}),
|
||||
cli_parity: cliParity,
|
||||
token_costs: tokenCosts,
|
||||
conditions:
|
||||
"Fresh MCP session, one selected analysis followed by repeated identical retained traces. Filesystem/build caches uncontrolled. Task token counts include one analysis summary and one trace with name/description/inputSchema catalog projections; structured and complete-tool-reply projections are alternatives, not billing. Repeated rounds, export, CLI parity and tokenizer are validation overhead outside that one-task token count but inside total resource receipts. A wrong-identity baseline is not a successful equivalent task or efficiency credit.",
|
||||
};
|
||||
if (values["require-safe"])
|
||||
assert.ok(safeIdentity, "Native identity/linkage oracle failed");
|
||||
} catch (cause) {
|
||||
primaryError = cause;
|
||||
throw cause;
|
||||
} finally {
|
||||
await closeVerifierResources(
|
||||
[
|
||||
() => client.close(),
|
||||
() => transport.close(),
|
||||
async () => {
|
||||
if (retainTemporary)
|
||||
throw new Error(
|
||||
`Command cleanup incomplete; retaining owned inputs at ${temporary?.path}.`,
|
||||
);
|
||||
await temporary?.close();
|
||||
},
|
||||
],
|
||||
primaryError,
|
||||
);
|
||||
}
|
||||
console.log(serialize(report));
|
||||
@@ -14,15 +14,21 @@ export const buildJavaScriptNativeHandoffs = (
|
||||
edges: readonly ApplicationEdge[],
|
||||
nativeEvidence: readonly Evidence[],
|
||||
): NativeHandoff[] => {
|
||||
const nodeById = new Map(nodes.map((node) => [node.node_id, node]));
|
||||
let nodeById: Map<string, ApplicationNode> | undefined;
|
||||
return nodes
|
||||
.filter(({ kind }) => kind === "native-addon")
|
||||
.flatMap((node) => {
|
||||
const digest = artifactDigest(node);
|
||||
if (digest === null) return [];
|
||||
// Observations may describe only a containing archive. A native target
|
||||
// requires the node owner's exact byte identity, independently of its
|
||||
// path, availability labels or the artifact supporting an observation.
|
||||
if (node.identity.strategy !== "content-digest") return [];
|
||||
const digest = node.identity.sha256;
|
||||
const linked = nativeEvidence.filter(
|
||||
({ subject }) => subject?.digest.sha256 === digest,
|
||||
);
|
||||
nodeById ??= new Map(
|
||||
nodes.map((candidate) => [candidate.node_id, candidate]),
|
||||
);
|
||||
const exportNodes = nativeExportNodes(node, edges, nodeById);
|
||||
return [
|
||||
{
|
||||
@@ -54,16 +60,6 @@ export const buildJavaScriptNativeHandoffs = (
|
||||
);
|
||||
};
|
||||
|
||||
const artifactDigest = (node: ApplicationNode): string | null => {
|
||||
if (node.identity.strategy === "content-digest") return node.identity.sha256;
|
||||
const observation = node.observations.find(
|
||||
({ evidence }) => evidence.artifact.available,
|
||||
);
|
||||
return observation?.evidence.artifact.available === true
|
||||
? observation.evidence.artifact.sha256
|
||||
: null;
|
||||
};
|
||||
|
||||
const artifactPath = (node: ApplicationNode): string | null => {
|
||||
for (const { evidence } of node.observations)
|
||||
if (
|
||||
|
||||
@@ -0,0 +1,223 @@
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { createHash } from "node:crypto";
|
||||
import { readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { join, resolve } from "node:path";
|
||||
import { afterAll, beforeAll, expect } from "vitest";
|
||||
|
||||
import { toolContract } from "../../../src/contracts/toolContracts.js";
|
||||
import { analysisViewResultSchema } from "../../../src/domain/analysisView/analysisView.js";
|
||||
import { createEvidence } from "../../../src/domain/evidence.js";
|
||||
import {
|
||||
applicationFeatureTraceResultSchema,
|
||||
type ApplicationFeatureTraceResult,
|
||||
} from "../../../src/domain/javascript/javascriptFeatureTraceSchemas.js";
|
||||
import { createStrippedAsarAddon } from "../../fixtures/strippedAsarAddon.js";
|
||||
import { cliTest } from "../../support/cli/cliFixture.js";
|
||||
|
||||
const client = new Client({ name: "native-handoff-identity", version: "1" });
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [resolve("scripts/rea.mjs"), "mcp"],
|
||||
cwd: process.cwd(),
|
||||
env: { PATH: process.env.PATH ?? "" },
|
||||
stderr: "pipe",
|
||||
});
|
||||
beforeAll(async () => {
|
||||
await client.connect(transport);
|
||||
transport.stderr?.on("data", () => undefined);
|
||||
});
|
||||
afterAll(async () => {
|
||||
await Promise.all([client.close(), transport.close()]);
|
||||
});
|
||||
|
||||
const sha256 = (value: Uint8Array) =>
|
||||
createHash("sha256").update(value).digest("hex");
|
||||
const nativeObservation = (path: string, sha256: string) =>
|
||||
createEvidence(
|
||||
{ path, sha256, format: "raw-binary" },
|
||||
{ id: "ghidra", name: "Ghidra", version: "simulated" },
|
||||
{
|
||||
operation: "binary_overview",
|
||||
parameters: {},
|
||||
result: {
|
||||
summary: "Simulated native Evidence for subject identity only.",
|
||||
},
|
||||
limitations: ["No native provider executed."],
|
||||
},
|
||||
);
|
||||
|
||||
interface ExpectedIdentity {
|
||||
readonly available: boolean;
|
||||
readonly addon: string;
|
||||
readonly archiveDigest: string;
|
||||
readonly nativeDigest: string;
|
||||
readonly parentId: string;
|
||||
readonly nativeId: string;
|
||||
readonly linked: boolean;
|
||||
}
|
||||
|
||||
const expectHandoffIdentity = (
|
||||
trace: ApplicationFeatureTraceResult,
|
||||
expected: ExpectedIdentity,
|
||||
) => {
|
||||
const node = trace.graph?.nodes.find(({ kind }) => kind === "native-addon");
|
||||
expect(node).toBeDefined();
|
||||
expect(trace.evidence_links).toEqual(
|
||||
[
|
||||
expected.parentId,
|
||||
...(expected.available && expected.linked ? [expected.nativeId] : []),
|
||||
].sort(),
|
||||
);
|
||||
if (expected.available) {
|
||||
expect(node?.identity).toMatchObject({
|
||||
strategy: "content-digest",
|
||||
sha256: expected.nativeDigest,
|
||||
});
|
||||
expect(trace.native_handoffs).toHaveLength(1);
|
||||
expect(trace.native_handoffs[0]).toMatchObject({
|
||||
native_node_id: node?.node_id,
|
||||
artifact_sha256: expected.nativeDigest,
|
||||
artifact_path: expected.addon,
|
||||
status: expected.linked
|
||||
? "evidence-linked"
|
||||
: "requires-provider-analysis",
|
||||
evidence_ids: expected.linked ? [expected.nativeId] : [],
|
||||
});
|
||||
} else {
|
||||
expect(node?.identity).toMatchObject({
|
||||
strategy: "canonical-path",
|
||||
artifact_sha256: expected.archiveDigest,
|
||||
path: expected.addon,
|
||||
});
|
||||
expect(node?.observations).toContainEqual(
|
||||
expect.objectContaining({
|
||||
properties: expect.objectContaining({
|
||||
availability: "unavailable",
|
||||
hash_status: "unavailable",
|
||||
inventory_artifact_id: null,
|
||||
}),
|
||||
}),
|
||||
);
|
||||
expect(trace.coverage.status).toBe("partial");
|
||||
expect(trace.graph?.coverage.status).toBe("partial");
|
||||
expect(trace.native_handoffs).toEqual([]);
|
||||
expect(trace.summary.native_handoffs).toBe(0);
|
||||
}
|
||||
};
|
||||
|
||||
for (const available of [false, true]) {
|
||||
cliTest(
|
||||
`keeps native handoff byte identity ${available ? "exact when available" : "unknown for a missing ASAR companion"}`,
|
||||
async ({ cli }) => {
|
||||
const { root, archive, addon, original } =
|
||||
await createStrippedAsarAddon();
|
||||
const companion = join(`${archive}.unpacked`, addon);
|
||||
if (available) await writeFile(companion, original);
|
||||
else await rm(companion);
|
||||
const archiveDigest = sha256(await readFile(archive));
|
||||
const nativeDigest = sha256(original);
|
||||
const containerEvidence = nativeObservation(archive, archiveDigest);
|
||||
const nativeEvidence = nativeObservation(companion, nativeDigest);
|
||||
const analysis = await cli.run({
|
||||
arguments: ["analyze-javascript-application", archive, "--json"],
|
||||
});
|
||||
expect(analysis.exitCode, analysis.stderr).toBe(0);
|
||||
const parent = toolContract(
|
||||
"analyze_javascript_application",
|
||||
).outputSchema.parse(analysis.json);
|
||||
const summary = await client.callTool({
|
||||
name: "analyze_javascript_application",
|
||||
arguments: { input_path: archive, detail: "summary" },
|
||||
});
|
||||
expect(summary.isError).not.toBe(true);
|
||||
const summaryEvidence = toolContract(
|
||||
"analyze_javascript_application",
|
||||
).outputSchema.parse(summary.structuredContent);
|
||||
expect(
|
||||
analysisViewResultSchema.parse(summaryEvidence.normalized_result)
|
||||
.parent_evidence_id,
|
||||
).toBe(parent.evidence_id);
|
||||
const observed: {
|
||||
trace: ApplicationFeatureTraceResult;
|
||||
linked: boolean;
|
||||
}[] = [];
|
||||
for (const native_observations of [
|
||||
[containerEvidence, nativeEvidence],
|
||||
[],
|
||||
]) {
|
||||
const query = {
|
||||
seed: {
|
||||
kind: "string",
|
||||
value: addon,
|
||||
match: "exact",
|
||||
case_sensitive: true,
|
||||
},
|
||||
direction: "outgoing",
|
||||
native_observations,
|
||||
};
|
||||
const request = join(
|
||||
root,
|
||||
`trace-${String(native_observations.length)}.json`,
|
||||
);
|
||||
await writeFile(
|
||||
request,
|
||||
JSON.stringify({
|
||||
...query,
|
||||
application: parent,
|
||||
}),
|
||||
);
|
||||
const cliTrace = await cli.run({
|
||||
arguments: ["trace-application-feature", request, "--json"],
|
||||
});
|
||||
expect(
|
||||
cliTrace.exitCode,
|
||||
cliTrace.exitCode === 0
|
||||
? undefined
|
||||
: JSON.stringify(cliTrace.json ?? cliTrace.stderr),
|
||||
).toBe(0);
|
||||
const cliEvidence = toolContract(
|
||||
"trace_application_feature",
|
||||
).outputSchema.parse(cliTrace.json);
|
||||
const response = await client.callTool({
|
||||
name: "trace_application_feature",
|
||||
arguments: {
|
||||
...query,
|
||||
application: {
|
||||
kind: "retained-evidence",
|
||||
evidence_id: parent.evidence_id,
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(response.isError).not.toBe(true);
|
||||
const evidence = toolContract(
|
||||
"trace_application_feature",
|
||||
).outputSchema.parse(response.structuredContent);
|
||||
expect(evidence.evidence_id).toBe(cliEvidence.evidence_id);
|
||||
for (const value of [cliEvidence, evidence])
|
||||
observed.push({
|
||||
trace: applicationFeatureTraceResultSchema.parse(
|
||||
value.normalized_result,
|
||||
),
|
||||
linked: native_observations.length > 0,
|
||||
});
|
||||
}
|
||||
// Exercise both provider-evidence selections and adapters before checking
|
||||
// the missing-byte outcome; a failed baseline still observes both paths.
|
||||
for (const { trace, linked } of observed)
|
||||
expectHandoffIdentity(trace, {
|
||||
available,
|
||||
addon,
|
||||
archiveDigest,
|
||||
nativeDigest,
|
||||
parentId: parent.evidence_id,
|
||||
nativeId: nativeEvidence.evidence_id,
|
||||
linked,
|
||||
});
|
||||
expect(sha256(await readFile(archive))).toBe(archiveDigest);
|
||||
if (available)
|
||||
expect(sha256(await readFile(companion))).toBe(nativeDigest);
|
||||
await client.ping();
|
||||
},
|
||||
);
|
||||
}
|
||||
Reference in new issue
Block a user