Verification¶
Executed on macOS with Node v24.22.1 at feat/environment-prompt 8dfd5a4d (stacked on the environment-build branch at
eeab0bda), and on Ubuntu 24.04 VMs for the end-to-end runs.
Gates¶
npm run -s typecheck,lint,format:check,docs:check,mcp-catalog:check,verify:test-discoveryandnpx knip: passed.npx vitest run --project domain --project services --project adapters: 16,347 passed and 1 failed, "stops Inspector target authorization after the current stat settles", which fails the same way on the base fce1aba8 in this environment (16,326 passed, 2 failed there); none is new.- Boundaries: the bundle tool, analysis view, tool schema validity, environment-build CLI, setup skill and client lifecycle suites (123 passed), and every browser and scenario MCP and CLI boundary suite (68 passed).
- Real Chrome:
verify:browser:request(new: 22 request steps, the CSRF cookie and stored token authenticate seven writes and never reach the Evidence, two queued writes are refused, the build serves the three write tools, an unexpected status fails the step and cancels the rest, and a redirect to another origin is not followed, the other origin receiving nothing) andverify:browser:networkpassed.verify:browserstops at "Real Chrome stable capture identities did not reconcile", aninspect_web_pagecomparison this change does not touch, and fails the same way on the base. verify:browser:requestruns each CLI call under the shared package verifier command owner, which stops the CLI and the browser it launched on timeout, output overflow or failure before the lane's private root is removed, and reports a failed stop with the root retained;browserRequestVerifierOwnership.test.tsproves a still-running CLI's separately grouped browser is stopped before the root goes.
Scenarios without request steps are unchanged: the action union gains a member, the secrets refactor compiles the same patterns, the request-step limitation appears only with request steps, and the browser suites and the network lane pass.
End to end: environments from a prompt¶
Five self-hosted applications were brought up fresh and seeded on four disposable VMs, loopback only. Claude Code 2.1.310
(claude-sonnet-5-5 through a LiteLLM proxy) got this branch's packed CLI, registered with rea setup --client
claude_code (the MCP server and the skill), and one prompt per application: build an agent environment of the app at
this URL, cover these entities, the instance is disposable but leave the seeded data as found, sign in as this user, the
password is in APP_PASSWORD, Chromium is here, write the bundle there. The prompt did not mention the skill. The agent
could use the REA tools, the skill and file tools; Bash and web tools were disabled.
Each bundle was then rebuilt with environment-build from its own evidence.json and this branch's tarball (the
registry's rea-agents@6.7.0 has no environment-serve); manifest.json and evidence.json were byte-identical to
the agent's in all five. It ran its smoke task, a retrieval eval generated from its manifest and capture, and the
application's recorded write eval with fireworks_ai/gpt-oss-120b and fireworks_ai/glm-5p3-flash, one run per task.
| App | Agent run | Recordings | Records | Tools (state / replayed reads) | Served writes | Smoke | Retrieval gpt-oss / GLM | Writes gpt-oss / GLM |
|---|---|---|---|---|---|---|---|---|
| Linkding | 182 s | 9 | 66 | 8 (7 / 1) | 0 | passed | 9/9 / 8/9 | 0/3 / 0/3 |
| Grocy | 253 s | 7 | 154 | 22 (16 / 0) | 6 | passed | 10/11 / 10/11 | 3/3 / 3/3 |
| Homebox | 391 s | 9 | 268 | 13 (11 / 2) | 0 | passed | 9/9 / 9/9 | 0/2 / 0/2 |
| Directus | 202 s | 7 | 100 | 25 (16 / 4) | 5 | passed | 12/12 / 11/12 | 2/2 / 2/2 * |
| Grafana | 143 s | 4 | 100 | 15 (12 / 2) | 1 | passed | 5/5 / 5/5 | 2/2 / 2/2 |
* The Directus write eval checks that the state holds exactly the provisions the scripted recording read (2); the agent's recording read all 5, so its count checks fail although every field check passes (0/2 with both models). With the counts relative to the state's own provisions it passes 2/2 with both.
- Write tasks passed exactly when the build served their tool, with both models.
- The skill loaded unprompted in every run. All five agents signed in through the form with the declared secret,
recorded the API with request steps, checked coverage, stopped by the withheld reasons and exported with
export_environment_bundle; each reported the credential-named warning pointers, and none published anything. - The scripted recordings the published environments were built from, built with this branch, read more (11, 39, 21, 31 and 35 state-backed reads) and serve 7, 3, 2, 2 and 0 writes; the prompted recordings serve 0, 6, 0, 5 and 1.
- Linkding's and Homebox's updates are withheld because a browser scenario carries no request times: the state learns
no clock, so an update the application stamps with a modification time is
unexplained-change. Linkding's scripted HAR serves the same PATCH with a clock rule. The coverage view now says so; recording request times on scenario network events would lift it. Linkding derives each bookmark's archive URL from its creation time, so its create iscontradictedin both recordings.
Earlier rounds: in the first, two agents hung on steps without timeout_ms and most guessed the scenario schema, so the
reference now starts from a whole scenario with a timeout on every step; the second (same five apps, before the review
fixes) served 1, 6, 0, 5 and 0 writes with the same eval pattern.
Review¶
Two independent reviews tried to refute the change.
- The first found a 302 that took a request step to another origin, page values left unredacted under a repeated or
colliding header label, short page values that rewrote ids and hosts across the capture,
User-Agentaccepted but dropped by Chrome, a request that could still be sent after its step timed out, an origin check separated from the send, a literal query re-encoded, line breaks accepted in header values, and CLI wording in MCP refusals. All are fixed and tested; redirects are now checked in real Chrome. - The second confirmed those fixes and found query entries mishandled after a path ending in
?or#, 3xx expected statuses that can never match, and padded stored values escaping redaction (fixed), plus the body cost of reading a page value, short values kept in plain text and the MCP dry run checking its directory (documented). In the prompted captures, only static assets lost bodies to the size limits.
本页原文 Markdown:在 AtomGit 查看·内容源自开源项目 rea2/rea