rapp-playwright/docs/evidence/2026-10-13-independent-local-consumer.md

141 lines
9.2 KiB
Markdown
Raw Permalink Normal View History

# Independent local consumer trial — 2026-10-13
The fresh consumer agent completed the documented Python/HTTP trial against
`http://127.0.0.1:18081` without installing Playwright or a browser, reading
provider source, or copying sibling fixtures. This proves the **local Docker
container** consumer path only. Preflight and every receipt identify
`execution_location: workstation-container`, `release: development`, service
0.1.0, Playwright 1.64.0 and Chromium 156.0.8078.4. It does not prove a Railiance
deployment or accept content-explorer's own task.
The agent first read the repository instructions and
[consumer contract](../consumer-contract.md), then ran:
```sh
python3 client/rapp_playwright.py preflight
python3 client/rapp_playwright.py trial --out .local/independent-trial-01
```
The fresh output directory was created by the documented client. Preflight
reported browser availability; the retained follow-up
[preflight](2026-10-13-independent-local-consumer/preflight.json) records the
same runtime identity. The successful run
`6b97d75c-0c39-415f-9cf4-f1813c205066` ran from
2026-10-13T16:43:17.124Z to 16:43:30.498Z.
## Evidence
| Check | Observed result |
| --- | --- |
| Keyboard and assertions | All 14 steps passed across desktop and narrow contexts: visible heading, count 0, Tab, focused increment button, Enter, count 1, screenshot. |
| PNG dimensions | Both desktop files are 1280×800; both narrow files are 390×844. Parsed the PNG IHDR directly and independently recomputed each download SHA-256. |
| Screenshot inspection | Narrow final image visibly shows Count: 1 and an orange focus outline around Increment; no horizontal overflow is visible. This is inspection, not automated visual-baseline acceptance. |
| Capability boundary | GET of the successful run with a deliberately incorrect bearer token returned HTTP 404 and `not_found`. |
| Deliberate UI failure | Run `d6b71146-4f82-4d9c-bf72-4d2b672080d2` expected impossible count 99999: `ui_assertion_failed`, client exit 2, failed desktop step index 1/action text, one prior screenshot preserved. |
| Target failure | Run `57b03fc8-4947-43a9-8c9b-058f1c46a9e0` navigated to an intentionally missing path on allowed `http://host.docker.internal:3300`: `target_unreachable`, client exit 3, zero completed steps/artifacts. Receipt does not distinguish HTTP status, connection, DNS or timeout cause. |
| Service connection failure | Preflight against unused local service port 18089 returned `service_unreachable: establish the documented tunnel and retry`, exit 3. The real service was not stopped. |
| Private handles | Each client-created output directory is 0700 and its run.json is 0600. No run handles or bearer capabilities are copied into retained evidence. |
Retained evidence:
- [Successful receipt](2026-10-13-independent-local-consumer/receipt.json),
[invocation](2026-10-13-independent-local-consumer/invocation.json),
[server trial specification](2026-10-13-independent-local-consumer/trial-spec.json)
and [independent artifact checks](2026-10-13-independent-local-consumer/artifact-checks.json).
- [Desktop screenshot](2026-10-13-independent-local-consumer/desktop-final.png)
and [narrow screenshot](2026-10-13-independent-local-consumer/narrow-final.png).
- [Wrong-token response](2026-10-13-independent-local-consumer/wrong-token.json).
- [UI failure receipt](2026-10-13-independent-local-consumer/ui-failure-receipt.json),
[specification](2026-10-13-independent-local-consumer/failure-spec.json),
[invocation](2026-10-13-independent-local-consumer/ui-failure-invocation.json)
and [preserved screenshot](2026-10-13-independent-local-consumer/ui-failure-prior-screenshot.png).
- [Target failure receipt](2026-10-13-independent-local-consumer/target-failure-receipt.json),
[specification](2026-10-13-independent-local-consumer/target-spec.json)
and [invocation](2026-10-13-independent-local-consumer/target-failure-invocation.json).
- [Client exits, unavailable service probe and permissions](2026-10-13-independent-local-consumer/client-outcomes.json).
## Usability findings
1. The first-command path worked without source inspection or dependency setup.
Error categories distinguish assertion, target and service connection failure.
2. At trial time, the contract's general “Client exit codes” wording did not
explain that `status --handle` returns 0 when successfully retrieving a failed
run. Both failed runs above reproduced this. A script must inspect JSON status;
clarify that command-specific behavior or align its exit codes.
3. Target diagnosis identifies the correct failure category but supplies no
sanitized reason such as HTTP status or connection failure. The documentation
points to browser-network reachability; more precise safe diagnostics would
reduce operator round trips. This test makes no claim about which transport
cause produced the observed error.
4. At initial reading, SCOPE.md, AGENTS.md and workplans/README.md still described
a planning-only repository despite the working client and service. The parent
delivery session was informed; those status statements need reconciliation
before publishing managed availability.
This report does not establish lifecycle cleanup, cancellation, restart recovery,
cross-run browser state isolation, hostile-page safety, a managed endpoint, or
consumer application acceptance; those require separate evidence.
## Final release verification — 2026-10-13, 17:14–17:16 UTC
Repeated the independent client trial against the corrected immutable release at
`http://127.0.0.1:18083`, using fresh client output directories:
```sh
python3 client/rapp_playwright.py --url http://127.0.0.1:18083 preflight
python3 client/rapp_playwright.py --url http://127.0.0.1:18083 trial --out .local/independent-final-01
python3 client/rapp_playwright.py --url http://127.0.0.1:18083 run --spec .local/independent-final-01/failure-spec.json --out .local/independent-final-ui-failure-01
```
The release recorded by preflight and both receipts is
`sha256:8d6fd5a8172cf906bb503642ed620761945c5138b86f6396c36f65cbe6def441`.
Execution location is `workstation-release-check`; service 0.1.0, Playwright
1.64.0 and Chromium 156.0.8078.4 remain unchanged. Preflight lists no allowed
external origins, and this repeat used only the provider fixture. No external
target, redirect diagnosis or target-unreachable probe was attempted against
this fixture-only container.
Run `35124424-950a-4339-adea-45525b470cab` passed all 14 steps, from
2026-10-13T17:14:50.055Z through 17:15:01.861Z, and the client exited 0. Independent
PNG header/hash checks again confirmed four screenshots: two at 1280×800 and two
at 390×844. Their hashes match the initial successful trial. The run-token
rejection probe again returned HTTP 404 / `not_found`. The new output directory
is 0700 and its private handle is 0600.
Run `75c6b50e-94a1-467f-98fc-7e8937f4362e` deliberately failed the count assertion
at desktop step index 1 with `ui_assertion_failed`, client exit 2, and the prior
1280×800 screenshot preserved with a verified hash. Preflight against unused
service port 18089 again produced `service_unreachable` and exit 3. Retrieving the
failed run with `--url http://127.0.0.1:18083 status --handle` succeeded with exit 0, now explicitly documented
in the consumer contract. AGENTS.md and README.md now describe implemented
verification commands and distinguish local evidence from managed availability.
Recovery usability detail: `--url` must also be repeated on `status` for a
nondefault endpoint. The first status invocation omitted it, queried the default
18081 service and returned HTTP 404 / exit 3; repeating with the correct 18083
override returned the expected failed-run receipt / exit 0. The private handle
does not automatically select the originating service URL.
Retained final evidence:
- [Preflight](2026-10-13-independent-final-consumer/preflight.json),
[successful receipt](2026-10-13-independent-final-consumer/receipt.json),
[invocation](2026-10-13-independent-final-consumer/invocation.json),
[artifact checks](2026-10-13-independent-final-consumer/artifact-checks.json),
[wrong-token response](2026-10-13-independent-final-consumer/wrong-token.json)
and [client outcomes](2026-10-13-independent-final-consumer/client-outcomes.json).
- [Desktop final](2026-10-13-independent-final-consumer/desktop-final.png),
[desktop keyboard](2026-10-13-independent-final-consumer/desktop-6-keyboard-result.png),
[narrow final](2026-10-13-independent-final-consumer/narrow-final.png)
and [narrow keyboard](2026-10-13-independent-final-consumer/narrow-6-keyboard-result.png).
- [Failed assertion receipt](2026-10-13-independent-final-consumer/ui-failure-receipt.json),
[specification](2026-10-13-independent-final-consumer/failure-spec.json),
[invocation](2026-10-13-independent-final-consumer/ui-failure-invocation.json)
and [preserved screenshot](2026-10-13-independent-final-consumer/ui-failure-prior-screenshot.png).
These are local final-release consumer checks. Read-only root configuration,
worker temporary-directory isolation and crash cleanup were reported by the
provider; this consumer trial does not independently inspect or establish those
properties. Managed platform admission remains pending, and no Railiance
deployment or content-explorer acceptance is claimed.