rea/docs/specs/agent-environments/scenario-capture/verification.md

Verification

Verified commit: 3c2a8db2, this increment on top of the environment-export increment (481c9883).

Gates

Run on macOS 26.5.1 with Node 22.23.1 on that tree:

• npx tsc --noEmit: no errors.
• oxfmt --check: all files formatted. oxlint on the changed files: no new findings. npm run verify:module-boundaries: 2133 files checked.
• npm run jscpd: 3.35% duplicated lines, unchanged. npm run scan:todos: none. npm run mcp-catalog:check: passed. npx knip: one unresolved import that predates this change (#javascript-analysis-worker).
• npm run test:prepare, then vitest over src/domain/analysisView, src/application, src/contracts, src/server/environment, tests/conformance and the analysis-view, environment-serve, environment-export and browser-scenario boundary tests: 1946 tests in 157 files, 1941 passed, 5 skipped, 0 failed.

Real browser captures

npm pack of this commit (rea-agents-6.7.0.tgz) was installed on Ubuntu 24.04 amd64 hosts (8 vCPU, 32 GB) with Chromium 155.0.8059.79 at /snap/bin/chromium, Node 24, Docker and agentenv-framework 0.9.1334. Every capture below is rea capture-browser-scenario output; no HAR was recorded, scoped or edited.

Fixture

A loopback fixture served a page that lists photos from /api/photos, three PNG thumbnails, and a form that POSTs JSON to /api/albums. The scenario waited for the list, opened /api/photos/1..3 with goto, filled and submitted the form.

• Capture: 14 transactions, 70 events, 0 dropped, authority controlled-replay.
• environment-manifest: parent_operation capture_browser_scenario, origin http://127.0.0.1:3199, confidence inferred. Tools get_photos (records 1, 9), get_photos_by_id ({numeric} id, records 5, 6, 7) and post_albums (record 13, body shape title: string). The page and the three PNGs were excluded as ui-page-or-asset.
• environment-serve: get_photos_by_id {id: "3"} returned the recorded JSON with record_ordinal 7 and exact_match true.
• The POST's response body was missing: the scenario ended at the action that sent it, the transaction was still unfinished, and the observer does not read bodies of unfinished responses. The manifest stated "1 of 14 scenario transactions retained no response body" and post_albums replayed status 201 with no body. Ending the scenario with a 1.5 s wait_for_timeout finished all 14 transactions and retained the body.
• environment-export of that capture wrote a README naming the browser scenario capture and the scenario definition's SHA-256; agent-env run <bundle> --task smoke passed in 30.4 s.

Five applications

Each environment ran manifest, export, the bundle's smoke task and 6 to 12 generated retrieval tasks, with one run per task for two models through the Hugging Face router.

Application	Capture	Actions	Transactions	Tools (rules / none)	Smoke	Retrieval tasks	gpt-oss-120b	GLM-5.3-Flash
PhotoPrism 261007	self-hosted, no login (public mode)	55	277	13 / 28	passed	10	10/10	10/10
Lychee 6.10.4	self-hosted, session cookie seeded from a declared secret	152	134	15 / 27	passed	12	11/12	12/12
Pinry 2.1.13	self-hosted, session and CSRF cookies seeded from declared secrets	148	568	12 / 12	passed	6	6/6	6/6
Art Institute of Chicago API	public, goto per URL	52	26	11 / 12	passed	12	11/12	12/12
Library of Congress JSON API	public, goto per URL	55	29	11 / 11	passed	12	11/12	12/12

Every scenario step completed and no capture dropped events. Each self-hosted capture includes one create, one edit and one delete made through the application's own forms; all three became tools with their recorded statuses, including a 204 DELETE without a content type. Declared secrets did not appear in any capture; where the application echoed a declared value in a response body (PhotoPrism's session reply, Pinry's owner token) the capture held the redaction marker instead.

Two other targets were tried and replaced. The Met Collection API projected 3 tools: its data paths are /departments, /objects/{id} and a search, and numeric object ids merge into one tool. The NASA Image and Video Library projected 16 tools, but its Collection+JSON bodies nest record fields deeper than the task generator reads, so no tasks could be derived.

Observed limits

These were observed during the runs and are outside this increment:

• A scenario's last action can lose its response body unless the scenario ends with a wait (above).
• Large asset bodies count against the analysis-view budget. One Pinry capture at max_network_body_bytes 262144 held 12.5 MB of Evidence, mostly CSS and JavaScript reloaded on 33 page loads, and the manifest view returned resource_constraint. A 20,000-byte body limit kept it within budget. Scenarios have no per-origin or per-media-type body selection.
• Image responses outside an API path are excluded as UI assets, so their bytes are replayable only when an API path serves them (PhotoPrism's /api/v1/t/... tiles became tools).
• With the default max_total_event_allocation_bytes, one PhotoPrism capture dropped its last 68 transactions, including all three writes, while the command exited 0; 64 MiB dropped none.
• Actions have no default deadline: two runs waited on a locator for 277 s and 7.5 min until cancelled. Every scenario above sets timeout_ms on its actions.
• A redirect is its own transaction; Django's trailing-slash 301 produced a second tool (get_pins_by_id and get_pins_by_id_2) for one resource.

Linux followup verification (2026-10-19)

The original PR was merged with current main 7a0c47bfa, after #4955, #4975 and #4999 merged. The export filesystem ownership, mode, tarball, cleanup and caller-path fixes from main are retained. Default MCP remains stdio.

The retained-parent projection previously decoded every request body before per-record budget checks. It now reserves copied/transformed text through the shared admission owner before schema validation or decoding, including retained parents. Replay projection omits request text used only for body-shape inference; response bytes still undergo the producer schema's integrity validation. The intermediate projected-array copy and unused capture wrapper were removed. Windows test inventories normalize filesystem separators to the portable bundle path representation. CI selects the agentenv lane for the new projection and its shared admission dependencies.

Executed on Linux x64, Node 24.18.0 / npm 11.16.0, Chrome for Testing 156.0.8078.4:

• turbo run typecheck lint:built format:check docs:check:built knip artifacts:skills artifacts:mcp-catalog --concurrency=1 --summarize: eleven tasks passed, one cached, 210.023 s elapsed / 209.860 s CPU / 4562.07 MiB peak process-family RSS. This first complete gate preceded the final lint cleanup and CI-selection additions; later checks are reported on the PR.
• Initial source verification passed 111 tests in 13 files, including HAR/manifest regressions, scenario loading/replay/export and new capacity/replay-mode tests. CI-selection verification passed 235 cases.
• Real public CLI capture of the source-owned interactive browser network site: ten transactions, fifty events, two tools, no dropped events. Its POST response replayed twice through the official HTTP SDK with exact body/status/Evidence ordinal and untrusted-content provenance. Declared secrets and transport credentials were excluded; the ordinary selected password field value remained in retained body bytes.
• Real public CLI capture of three live GitHub API responses in Chromium, without a HAR: three transactions, fifteen events, three tools, no dropped events. Every response replayed twice through the HTTP SDK with exact body/status/Evidence/provenance. Explicit navigation waits let response-body capture settle. Complete two-scenario capture/projection/serve task: 22.527 s / 16.678 s CPU / 1397.93 MiB peak process-family RSS, including owned browser processes. Both listeners exited 143 and newly owned browser profiles were gone.
• The same two actual captures produced identical manifest results through CLI and official SDK stdio MCP, with HTTP environment variables present: 2.795 s / 2.778 s CPU / 476.67 MiB. This verifies adapter parity and default transport independently of explicit environment HTTP.

The first local probe mistakenly searched base64 body content for plaintext; its assertion was corrected to inspect decoded retained bytes. A later SDK probe initially imported stdio transport from the SDK root instead of its documented subpath; the corrected probe passed. An early live navigation also reported one body-read-failed response, which the projection preserved as unavailable. No browser-provider behavior was changed; the successful run used explicit waits. These earlier attempts are not counted as passing runs.

A matched retained-scenario projection comparison bundles the original helper from 3aae3579e and the followup helper, supplies the same 72,771-byte capture with three JSON writes, and completes one cold plus 100 warm full manifest projections. Both outputs have the same SHA-256 990ac431670374ff68c77a20d77985f626e4681bb0e2f491e5af18f45ce79991. Before/after process totals: 0.970/0.876 s elapsed, 0.934/0.865 s CPU, 184.89/184.39 MiB RSS. First task: 17.378/13.073 ms; 100 warm tasks: 261.419/236.131 ms. Single samples do not establish a speed or memory improvement. Bundle preparation is excluded; process totals include fixture startup. CLI, provider and transport costs are excluded from this comparison.

Heavy work is serialized on one CPU with one test worker. RSS sums sampled process descendants every 50 ms and can double-count shared pages. Agent token usage is unmeasured. These followups do not rerun the author's five application reconstruction/model tasks, nor establish Windows or macOS behavior. Historical response replay does not reconstruct application state.

A retained-parent counterexample used a 128 KiB projection budget and one 16 KiB JSON request: the original helper admitted it and decoded once; the repaired helper returned AnalysisResourceConstraintError with zero UTF-8 decodes. The test intentionally uses a small budget without inducing an OOM. Final targeted verification passed 242 tests (including 235 CI-selection cases): 41.042 s / 40.666 s CPU / 644.37 MiB peak process-family RSS.

The broader d2416ca54 regression run passed 657 tests in 47 files plus ten product-catalog cases (295.763 s / 293.370 s CPU / 1921.69 MiB family RSS; catalog checked separately). The author then pushed 655e62334 with redaction/body-budget documentation and withheld-empty-body behavior. That commit was retained; the followup counts bodiless/redacted observations against represented transactions, preserving the complete request ordinals. The related source run passed 145 tests in 14 files (9.793 s / 9.618 s CPU / 688.66 MiB). Docker execution was held at this author-update checkpoint, so no earlier bundle is attributed to the new source. Final committed checks and actual scenario-container/framework results are reported on the PR.