Files
rea/tests/process-global/stableArtifactConsumers.test.ts
JANG BYUNGKUNandgensokyo-dev 833136507d feat(il2cpp): inventory Unity IL2CPP global-metadata images, types and methods (#3807)
* feat(il2cpp): inventory IL2CPP global-metadata images, types and methods

Add inspect_il2cpp_metadata (CLI inspect-il2cpp-metadata), a read-only
in-process decoder for unencrypted Unity global-metadata.dat version 39.
It reports header sections, serialized index widths, images, type
definitions (nesting from the nestedTypes section) and method names.

Unrecognized magic and other versions are unsupported; damaged headers,
overlapping sections, inconsistent record sizes, out-of-range indices and
invalid strings fail with a reason code and no partial success. MCP
detail summary retains the complete Evidence and pages rows through
inspect_analysis_view kind il2cpp-metadata.

Adds an opt-in verify:il2cpp:metadata -- --metadata PATH check.

* feat(il2cpp): deliver metadata summaries by default over MCP

A 112,232-method file produced a 10.5 MB complete response, above the
10 MiB stdio budget. Default MCP delivery is now the retained summary;
detail complete stays available, and the one-shot CLI requests it.

* fix(il2cpp): reject shifted strings, nesting cycles and method owner drift

Review fuzzing showed that moving the strings section offset by one byte
silently renamed every image, type and method. Name indices must now start
a string, which also bounds total decoding by the section size.

Nesting cycles and methods whose declaring type disagrees with the owning
type range are rejected instead of returned. Adds service, shutdown and
index-width coverage, and moves environment-specific measurements out of
the public guide and tool description.

* feat(il2cpp): decode global-metadata versions 29 and 31

Unity 2022.3 writes version 29 or 31 metadata: a 256-byte header of
(offset, size) pairs, fixed 4-byte indices, type records with
elementTypeIndex, and method records that gained returnParameterToken in
version 31. Record counts come from section sizes and must be whole.

Sections report a null count for these versions and index widths report
fixed-width-version. Image, type and method rows keep the same shape, and
the string, nesting and method-ownership checks apply to every version.

* fix(il2cpp): bound report size before serializing and tighten coverage

Rows share decoded names, so a 130 KB file could expand into gigabytes of
JSON before the 64 MiB budget check, ending in an internal RangeError. The
provider now refuses from a lower bound on the serialized rows first.

The decoder also rejects non-empty sections inside the header, methods
outside their declaring type's range and overlapping image type ranges, and
keeps a leading byte order mark in names. Layouts now carry one indexing
mode instead of two coupled flags, and the tool description explains
nested_type_start.

* fix(il2cpp): refuse oversized metadata before decoding rows

Address review findings on the IL2CPP metadata inventory:

- check a lower bound from header row counts before any row object is
  allocated, so a large rejected file no longer builds every row first;
- validate sealed parent Evidence and count its rows once per report
  instead of on every inspect_analysis_view page;
- decode version 39 fixtures with 2- and 4-byte type-definition,
  generic-container and parameter indices.

* docs(il2cpp): attach the decoder comment to readIl2cppMetadata

* fix(il2cpp): deliver complete rows by default like the other inspection tools

inspect_il2cpp_metadata defaulted to detail summary, the only inspection
tool to do so. Omitted detail now returns every row inline, as
inspect_artifact, decode_interface_builder and the capture tools do;
summary stays one argument away and the guide, CLI reference and skill
reference say to select it first for large games, whose complete MCP
delivery is refused at the stdio budget with retained Evidence.

* test(il2cpp): cover narrow-width -1 sentinels and fix the skill's default

An empty image stores its type start as the all-ones value of a one- or
two-byte index; the decoder reads it as -1, but no fixture exercised it,
so removing either conversion passed the suite. The fixture can now append
an empty image, and both widths decode it. The skill still described a
summary default; it now says to select summary first, as the guide does.

* docs(il2cpp): request summary explicitly in the summary example

* chore(il2cpp): hash the verifier metadata through owned file resources

* fix(il2cpp): count escaped name length without building the escaped string

* fix(il2cpp): keep the verifier's check failure and small-table last page

* feat(domain): count JSON string length without building the escaped string

* fix(il2cpp): keep descriptor cleanup owners and share the JSON length count

* test(domain): give the backslash case real backslashes

* fix(il2cpp): select the CLI path and finish the command like the other tools

Follow main's absoluteCallerPath (#4067) and pass the command signal to finishAnalysisCommand.

* fix(il2cpp): select the verifier input path like the CLI

Use absoluteCallerPath (#4067) so a path with a symlink followed by .. names the same file the CLI and MCP inspect.

* feat(il2cpp): add compact composable metadata queries

Retain source-indexed Evidence for filtered batch and field queries. Bound representation growth before serialization, yield decoding for cancellation, and share CLI Evidence publication with desktop inspection. Verify public Unity metadata against independent byte and managed-DLL oracles and exercise actual stdio SDK workflows.

* fix(il2cpp): retain the validated metadata header magic

* test(il2cpp): remove obsolete execution wording exception

* fix(artifacts): preserve export selection and completed source facts

---------

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-20 07:20:47 +08:00

498 lines
18 KiB
TypeScript

import { writeFile, type FileHandle } from "node:fs/promises";
import { createHash } from "node:crypto";
import { join } from "node:path";
import { afterEach, beforeEach, expect, it, vi } from "vitest";
const fault = vi.hoisted(() => ({
path: "",
blockNew: true,
blocked: new Set<FileHandle>(),
handles: [] as FileHandle[],
onClose: undefined as (() => void) | undefined,
readFailure: undefined as { cause: unknown } | undefined,
}));
vi.mock("node:fs/promises", async (importOriginal) => {
const original = await importOriginal<typeof import("node:fs/promises")>();
return {
...original,
open: async (...arguments_: Parameters<typeof original.open>) => {
const handle = await original.open(...arguments_);
if (String(arguments_[0]) !== fault.path) return handle;
const close = handle.close.bind(handle);
fault.handles.push(handle);
if (fault.blockNew) fault.blocked.add(handle);
if (fault.readFailure !== undefined) {
const failure = fault.readFailure;
handle.stat = async () => {
throw failure.cause;
};
}
handle.close = async () => {
fault.onClose?.();
if (fault.blocked.has(handle))
throw new Error("controlled Rust descriptor close failure");
await close();
};
return handle;
},
};
});
import { RustBinaryProvider } from "../../src/rust/RustBinaryProvider.js";
import { RustBinaryService } from "../../src/application/rust/RustBinaryService.js";
import { AnalysisError } from "../../src/domain/analysisErrorBase.js";
import { parseEvidence } from "../../src/domain/evidence.js";
import { projectAnalysisError } from "../../src/domain/analysisErrorProjection.js";
import {
AnalysisCancelledError,
AnalysisArtifactChangedError,
AnalysisAccessDeniedError,
AnalysisResourceConstraintError,
} from "../../src/domain/analysisErrorCore.js";
import {
ArtifactFileCleanupFailure,
withArtifactRegularFile,
} from "../../src/artifacts/ArtifactFileOperation.js";
import { ArtifactResourceScope } from "../../src/artifacts/ArtifactResourceScope.js";
import { ArtifactReaderFailure } from "../../src/artifacts/ArtifactReader.js";
import { GoBinaryProvider } from "../../src/go/GoBinaryProvider.js";
import { PeResourcesProvider } from "../../src/native/pe/PeResourcesProvider.js";
import { Il2cppMetadataProvider } from "../../src/il2cpp/Il2cppMetadataProvider.js";
import { inspectPeResourcesInputSchema } from "../../src/domain/native/peResources.js";
import { peResourceFixture } from "../../src/native/pe/PeResources.fixture.js";
import { createServer } from "../../src/server/createServer.js";
import { createTestBinarySession } from "../fixtures/binarySession.js";
import { createRustImage } from "../fixtures/rust/image.js";
import { createIl2cppMetadataFixture } from "../fixtures/il2cpp/metadata.js";
import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js";
import { parseMcpToolError } from "../fixtures/mcpToolError.js";
import { mcpTest } from "../support/mcp/mcpFixture.js";
beforeEach(() => {
fault.path = "";
fault.blockNew = true;
fault.blocked.clear();
fault.handles = [];
fault.onClose = undefined;
fault.readFailure = undefined;
});
const staticProviders = [
["Rust", () => new RustBinaryProvider()],
["Go", () => new GoBinaryProvider()],
["PE", () => new PeResourcesProvider()],
["IL2CPP metadata", () => new Il2cppMetadataProvider()],
] as const;
const expectIl2cppSource = (
error: AnalysisError,
path: string,
bytes: Buffer,
) => {
const source = parseEvidence(error.partialObservation);
const sha256 = createHash("sha256").update(bytes).digest("hex");
expect(source.operation).toBe("observe_il2cpp_metadata_source");
expect(source.subject).toMatchObject({
local_path: path,
digest: { sha256 },
});
expect(source.normalized_result).toEqual({
artifact: { path, sha256, bytes: bytes.length },
inventory_status: "not-returned-as-complete",
});
};
it.each(staticProviders)(
"preserves the exact late-close cancellation cause through %s inspection",
async (_name, makeProvider) => {
const path = await selectedImage();
const provider = makeProvider();
if (provider instanceof PeResourcesProvider)
await writeFile(path, peResourceFixture().bytes);
if (provider instanceof Il2cppMetadataProvider)
await writeFile(path, createIl2cppMetadataFixture().bytes);
const controller = new AbortController();
const reason = new Error("cancel during confirmed descriptor close");
fault.blockNew = false;
fault.onClose = () => controller.abort(reason);
try {
const result =
provider instanceof PeResourcesProvider
? await provider.inspect(
inspectPeResourcesInputSchema.parse({ path }),
{ signal: controller.signal },
)
: await provider.inspect({ path }, { signal: controller.signal });
if (result.ok) throw new Error("Expected late cancellation");
expect(result.error).toBeInstanceOf(AnalysisCancelledError);
expect(result.error.cause).toBe(reason);
if (provider instanceof Il2cppMetadataProvider)
expectIl2cppSource(
result.error,
path,
createIl2cppMetadataFixture().bytes,
);
expect(fault.handles.every((handle) => handle.fd === -1)).toBe(true);
} finally {
fault.blocked.clear();
await provider.close();
}
},
);
it.each([false, true])(
"retains completed IL2CPP source facts through failed close (cancel: %s)",
async (cancel) => {
const path = await selectedImage();
const { bytes } = createIl2cppMetadataFixture();
await writeFile(path, bytes);
const provider = new Il2cppMetadataProvider();
const controller = new AbortController();
const reason = new Error("cancel during failed close");
if (cancel) fault.onClose = () => controller.abort(reason);
try {
const result = await provider.inspect(
{ path },
{ signal: controller.signal },
);
if (result.ok) throw new Error("Expected failed close");
if (cancel) {
expect(result.error.cleanupPrimary).toBeInstanceOf(
AnalysisCancelledError,
);
expect(result.error.cleanupPrimary?.cause).toBe(reason);
}
expect(result.error.cleanupIncomplete).toBe(true);
expectIl2cppSource(result.error, path, bytes);
expect(fault.handles).toHaveLength(1);
expect(fault.handles[0]?.fd).toBeGreaterThanOrEqual(0);
fault.blocked.clear();
await provider.close();
expect(result.error.cleanupIncomplete).toBe(false);
expect(fault.handles[0]?.fd).toBe(-1);
expect(
projectAnalysisError(result.error).details?.partial_observation,
).toEqual(result.error.partialObservation);
} finally {
fault.blocked.clear();
await provider.close();
}
},
);
it.each(
staticProviders.flatMap(([name, makeProvider]) => [
[
name,
makeProvider,
new ArtifactReaderFailure("integrity", "source replacement"),
AnalysisArtifactChangedError,
] as const,
[
name,
makeProvider,
Object.assign(new Error("read denied"), { code: "EACCES" }),
AnalysisAccessDeniedError,
] as const,
[
name,
makeProvider,
new ArtifactReaderFailure("limit", "read limit"),
AnalysisResourceConstraintError,
] as const,
]),
)(
"preserves %s read failure after successful close observes cancellation",
async (_name, makeProvider, primary, expected) => {
const path = await selectedImage();
const provider = makeProvider();
fault.blockNew = false;
fault.readFailure = { cause: primary };
const controller = new AbortController();
fault.onClose = () => controller.abort(new Error("later cancellation"));
try {
const result =
provider instanceof PeResourcesProvider
? await provider.inspect(
inspectPeResourcesInputSchema.parse({ path }),
{ signal: controller.signal },
)
: await provider.inspect({ path }, { signal: controller.signal });
if (result.ok) throw new Error("Expected primary read failure");
expect(result.error).toBeInstanceOf(expected);
expect(result.error.cause).toBe(primary);
if (provider instanceof Il2cppMetadataProvider)
expect(result.error.partialObservation).toBeUndefined();
expect(fault.handles.every((handle) => handle.fd === -1)).toBe(true);
} finally {
await provider.close();
}
},
);
it.each(staticProviders)(
"keeps the primary %s read failure when cancellation arrives during failed close",
async (_name, makeProvider) => {
const path = await selectedImage();
const provider = makeProvider();
const primary = new ArtifactReaderFailure(
"integrity",
"controlled source replacement",
);
fault.readFailure = { cause: primary };
const controller = new AbortController();
fault.onClose = () => controller.abort(new Error("later cancellation"));
try {
const result =
provider instanceof PeResourcesProvider
? await provider.inspect(
inspectPeResourcesInputSchema.parse({ path }),
{ signal: controller.signal },
)
: await provider.inspect({ path }, { signal: controller.signal });
if (result.ok)
throw new Error("Expected primary failure and retained descriptor");
expect(result.error.cleanupIncomplete).toBe(true);
const aggregate = result.error.cause;
if (!(aggregate instanceof AggregateError))
throw new Error("Missing primary/cleanup aggregate");
const [original] = aggregate.errors;
expect(original).toBeInstanceOf(AnalysisArtifactChangedError);
expect((original as AnalysisArtifactChangedError).cause).toBe(primary);
if (provider instanceof Il2cppMetadataProvider)
expect(result.error.partialObservation).toBeUndefined();
fault.blocked.clear();
await provider.close();
expect(result.error.cleanupIncomplete).toBe(false);
} finally {
fault.blocked.clear();
await provider.close();
}
},
);
afterEach(async () => {
fault.blockNew = false;
fault.blocked.clear();
fault.onClose = undefined;
for (const handle of fault.handles) if (handle.fd >= 0) await handle.close();
});
const selectedImage = async () => {
const root = await createTestTempDirectory("rea-rust-artifact-owner-");
const path = join(root, "image.elf");
await writeFile(path, createRustImage({ format: "elf" }));
fault.path = path;
return path;
};
mcpTest(
"retains Rust descriptor cleanup and recovers in the same MCP connection",
async ({ mcp }) => {
const path = await selectedImage();
const service = new RustBinaryService(new RustBinaryProvider());
const failures: AnalysisError[] = [];
const inspect = service.inspect.bind(service);
vi.spyOn(service, "inspect").mockImplementation(async (...arguments_) => {
const result = await inspect(...arguments_);
if (!result.ok) failures.push(result.error);
return result;
});
const session = createTestBinarySession(() => {
throw new Error("Unexpected native provider");
});
const server = createServer(
{ kind: "session", session },
{ rustBinary: service },
);
const client = await mcp.connect(server);
try {
for (let attempt = 0; attempt < 2; attempt++) {
const response = await client.callTool({
name: "inspect_rust_binary",
arguments: { path },
});
expect(response.isError).toBe(true);
expect(parseMcpToolError(response).error).toMatchObject({
code: "cleanup_incomplete",
details: { resources: [path] },
});
}
expect(fault.handles).toHaveLength(1);
expect(fault.handles[0]?.fd).toBeGreaterThanOrEqual(0);
expect(failures.map((error) => error.cleanupIncomplete)).toEqual([
true,
true,
]);
fault.blockNew = false;
fault.blocked.clear();
const recovered = await client.callTool({
name: "inspect_rust_binary",
arguments: { path },
});
expect(recovered.isError).not.toBe(true);
expect(failures.map((error) => error.cleanupIncomplete)).toEqual([
false,
false,
]);
expect(fault.handles.every((handle) => handle.fd === -1)).toBe(true);
} finally {
fault.blockNew = false;
fault.blocked.clear();
await server.close();
await session.close();
}
},
);
it("keeps mixed descriptor cleanup and its aggregate bound to the exact unresolved owners", async () => {
const path = await selectedImage();
const service = new RustBinaryService(new RustBinaryProvider());
try {
const inspections = await Promise.all([
service.inspect({ path }),
service.inspect({ path }),
]);
expect(fault.handles).toHaveLength(2);
const errors = inspections.map((result) => {
if (result.ok) throw new Error("Expected retained cleanup");
return result.error;
});
const firstClose = await service.close().catch((cause: unknown) => cause);
if (!(firstClose instanceof AnalysisError))
throw new Error("Missing typed shutdown failure");
expect(firstClose.cleanupIncomplete).toBe(true);
const handlesByError = errors.map((error) => {
const aggregate = error.cause;
if (!(aggregate instanceof AggregateError))
throw new Error("Missing cleanup aggregate");
const cleanup = aggregate.errors[1];
if (
!(cleanup instanceof AnalysisError) ||
!(cleanup.cause instanceof ArtifactFileCleanupFailure)
)
throw new Error("Missing exact descriptor owner");
return cleanup.cause.fileCleanupFailure.owner.handle;
});
expect(new Set(handlesByError)).toEqual(new Set(fault.handles));
const [first, second] = handlesByError;
if (first === undefined || second === undefined)
throw new Error("Missing distinct descriptors");
fault.blocked.delete(first);
const mixedClose = await service.close().catch((cause: unknown) => cause);
if (!(mixedClose instanceof AnalysisError))
throw new Error("Missing mixed shutdown failure");
expect(first.fd).toBe(-1);
expect(second.fd).toBeGreaterThanOrEqual(0);
expect(errors.map((error) => error.cleanupIncomplete)).toEqual([
false,
true,
]);
expect(firstClose.cleanupIncomplete).toBe(true);
expect(mixedClose.cleanupIncomplete).toBe(true);
fault.blocked.clear();
await service.close();
expect(errors.map((error) => error.cleanupIncomplete)).toEqual([
false,
false,
]);
expect(firstClose.cleanupIncomplete).toBe(false);
expect(mixedClose.cleanupIncomplete).toBe(false);
} finally {
fault.blockNew = false;
fault.blocked.clear();
await service.close();
}
});
mcpTest(
"server shutdown releases Rust descriptors retained by a failed tool call",
async ({ mcp }) => {
const path = await selectedImage();
const session = createTestBinarySession(() => {
throw new Error("Unexpected native provider");
});
const server = createServer({ kind: "session", session });
const client = await mcp.connect(server);
try {
const response = await client.callTool({
name: "inspect_rust_binary",
arguments: { path },
});
expect(parseMcpToolError(response).error.code).toBe("cleanup_incomplete");
expect(fault.handles[0]?.fd).toBeGreaterThanOrEqual(0);
fault.blockNew = false;
fault.blocked.clear();
await server.close();
expect(fault.handles[0]?.fd).toBe(-1);
} finally {
fault.blockNew = false;
fault.blocked.clear();
await server.close();
await session.close();
}
},
);
it("preserves the caller's cancellation cause when descriptor close also fails", async () => {
const path = await selectedImage();
const provider = new RustBinaryProvider();
const controller = new AbortController();
const reason = new Error("cancel during Rust descriptor close");
fault.onClose = () => controller.abort(reason);
try {
const result = await provider.inspect(
{ path },
{ signal: controller.signal },
);
if (result.ok)
throw new Error("Expected cancellation with retained cleanup");
expect(result.error.cleanupIncomplete).toBe(true);
const aggregate = result.error.cause;
if (!(aggregate instanceof AggregateError))
throw new Error("Missing primary/cleanup aggregate");
const [primary] = aggregate.errors;
expect(primary).toBeInstanceOf(AnalysisCancelledError);
expect((primary as AnalysisCancelledError).cause).toBe(reason);
fault.blockNew = false;
fault.blocked.clear();
await provider.close();
expect(result.error.cleanupIncomplete).toBe(false);
} finally {
fault.blockNew = false;
fault.blocked.clear();
await provider.close();
fault.onClose = undefined;
}
});
it("preserves upstream resource-bound facts through an unconfirmed descriptor close", async () => {
const path = await selectedImage();
const scope = new ArtifactResourceScope();
const primary = new ArtifactReaderFailure(
"limit",
"Controlled input budget refusal",
{ boundary: "file-size" },
);
try {
const failure = await scope
.run(() =>
withArtifactRegularFile(path, scope, undefined, async () => {
throw primary;
}),
)
.catch((cause: unknown) => cause);
expect(failure).toBeInstanceOf(ArtifactFileCleanupFailure);
expect(failure).toMatchObject({ reason: "limit", boundary: "file-size" });
if (!(failure instanceof ArtifactFileCleanupFailure))
throw new Error("Expected exact descriptor cleanup failure");
expect(failure.fileCleanupFailure.outcome).toEqual({
kind: "failed",
cause: primary,
});
} finally {
fault.blocked.clear();
await scope.close();
}
});