coulomb/rapp-playwright
5
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add repeatable browser acceptance for real consumer connections

Browse source 
Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a13590-a16f-7f60-be60-fd71627bc99d
This commit is contained in:
tegwick 2026-10-13 22:11:40 +02:00
parent 3a30ef159a
commit 5b8e798419
7 changed files with 591 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

2
Makefile
View file

@ -10,7 +10,7 @@ install:
npm ci --ignore-scripts
check:
npm run check
python3 -m py_compile client/rapp_playwright.py tools/render.py
python3 -m py_compile client/rapp_playwright.py tools/render.py tools/verify_consumer.py
test: check
npm test
test-integration:

11
SCOPE.md
View file

@ -55,7 +55,9 @@ Platform admitted and deployed the reviewed package under RPF-WP-0056. The
eligibility includes all identified platform humans and coding agents; no further
membership decision is needed. Registration, native JWT authentication,
per-caller API transport, renewal/revocation and real consumer receipts are still
open in RPF-WP-0056-T03. Custodian coordination is requested in T06.
open in RPF-WP-0056-T03. Custodian accepted coordination in CUST-WP-0080-T05 (message
`9defa19c-e812-4960-b563-2398688f562a`); concrete identity/transport owner
implementation returns remain pending.
Existing capability `capability.infotech.headless-browser` is now published at
D3/A5/C2/R2 with pinned client/contract/release links and runnable local guidance.
@ -72,6 +74,11 @@ are prepared but conditional on identity-owner activation. The earlier
admission, deployment and the binding are no longer the outstanding decision.
Abrupt-interruption recovery and actual cross-version rollback remain unverified.
The repeatable [consumer API acceptance command](docs/operations.md#repeatable-consumer-api-acceptance)
now passes six local API checks and refuses release/location mismatches. It is
ready to run through each real consumer connection after Platform activation;
its local receipts do not establish consumer authentication.
## Deliberate limits
V1 provides trusted-target Chromium jobs and a finite action/assertion vocabulary.
@ -81,5 +88,5 @@ sandbox is disabled; hostile pages and mutually untrusted tenants are unsupporte
Target failures identify the category but not detailed DNS/TLS/HTTP causes.
The OnCo pilot covers the home page, not all consumer routes or accessibility.
See the [timestamped intent assessment](history/2026-10-13-2142-intent-scope-assessment.md)
See the [timestamped intent assessment](history/2026-10-13-2211-intent-scope-assessment.md)
for the prioritized gaps and their acceptance evidence.

347
docs/evidence/2026-10-13-consumer-verifier.json Normal file
View file

@ -0,0 +1,347 @@
{
"local_pass": {
"started_at": "2026-10-13T20:10:02.963127+00:00",
"service_url": "http://127.0.0.1:18083",
"argv": [
"tools/verify_consumer.py",
"--url",
"http://127.0.0.1:18083",
"--expected-release",
"sha256:8d6fd5a8172cf906bb503642ed620761945c5138b86f6396c36f65cbe6def441",
"--expected-location",
"workstation-release-check",
"--out",
".local/acceptance-tool-02"
],
"expected_release": "sha256:8d6fd5a8172cf906bb503642ed620761945c5138b86f6396c36f65cbe6def441",
"expected_location": "workstation-release-check",
"checks": [
"preflight_identity",
"responsive_keyboard_trial_and_downloads",
"foreign_run_and_artifact_denied",
"ui_failure_preserves_evidence",
"private_target_denied",
"running_job_cancellation"
],
"status": "passed",
"identity_authentication_verified": false,
"scope": "browser API only; pair with native consumer identity and permission receipts",
"finished_at": "2026-10-13T20:10:20.767858+00:00"
},
"wrong_release": {
"started_at": "2026-10-13T20:10:04.425524+00:00",
"service_url": "http://127.0.0.1:18083",
"argv": [
"tools/verify_consumer.py",
"--url",
"http://127.0.0.1:18083",
"--expected-release",
"deliberately-wrong",
"--expected-location",
"workstation-release-check",
"--out",
".local/acceptance-tool-wrong-release"
],
"expected_release": "deliberately-wrong",
"expected_location": "workstation-release-check",
"checks": [],
"status": "failed",
"identity_authentication_verified": false,
"scope": "browser API only; pair with native consumer identity and permission receipts",
"error": "release_mismatch",
"finished_at": "2026-10-13T20:10:04.527311+00:00"
},
"wrong_location": {
"started_at": "2026-10-13T20:10:15.402989+00:00",
"service_url": "http://127.0.0.1:18083",
"argv": [
"tools/verify_consumer.py",
"--url",
"http://127.0.0.1:18083",
"--expected-release",
"sha256:8d6fd5a8172cf906bb503642ed620761945c5138b86f6396c36f65cbe6def441",
"--expected-location",
"railiance01/playwright",
"--out",
".local/acceptance-tool-wrong-location"
],
"expected_release": "sha256:8d6fd5a8172cf906bb503642ed620761945c5138b86f6396c36f65cbe6def441",
"expected_location": "railiance01/playwright",
"checks": [],
"status": "failed",
"identity_authentication_verified": false,
"scope": "browser API only; pair with native consumer identity and permission receipts",
"error": "execution_location_mismatch",
"finished_at": "2026-10-13T20:10:15.452794+00:00"
},
"trial_receipt": {
"id": "be7f8e4a-c0d3-4728-b290-06c08dc224f4",
"api_version": "1",
"status": "passed",
"error": null,
"label": "independent-browser-trial",
"target": "fixture",
"viewports": [
{
"name": "desktop",
"width": 1280,
"height": 800
},
{
"name": "narrow",
"width": 390,
"height": 844
}
],
"timeout_ms": 60000,
"created_at": "2026-10-13T20:10:03.005Z",
"expires_at": "2026-10-13T21:10:03.005Z",
"service_version": "0.1.0",
"playwright_version": "1.64.0",
"release": "sha256:8d6fd5a8172cf906bb503642ed620761945c5138b86f6396c36f65cbe6def441",
"execution_location": "workstation-release-check",
"browser_version": "156.0.8078.4",
"completed_steps": 14,
"steps": [
{
"viewport": "desktop",
"index": 0,
"action": "visible",
"status": "passed",
"at": "2026-10-13T20:10:06.924Z",
"selector": "h1"
},
{
"viewport": "desktop",
"index": 1,
"action": "text",
"status": "passed",
"at": "2026-10-13T20:10:07.220Z",
"selector": "#count"
},
{
"viewport": "desktop",
"index": 2,
"action": "press",
"status": "passed",
"at": "2026-10-13T20:10:07.332Z",
"key": "Tab"
},
{
"viewport": "desktop",
"index": 3,
"action": "focused",
"status": "passed",
"at": "2026-10-13T20:10:07.410Z",
"selector": "#increment"
},
{
"viewport": "desktop",
"index": 4,
"action": "press",
"status": "passed",
"at": "2026-10-13T20:10:07.420Z",
"key": "Enter"
},
{
"viewport": "desktop",
"index": 5,
"action": "text",
"status": "passed",
"at": "2026-10-13T20:10:07.429Z",
"selector": "#count"
},
{
"viewport": "desktop",
"index": 6,
"action": "screenshot",
"status": "passed",
"at": "2026-10-13T20:10:07.510Z"
},
{
"viewport": "narrow",
"index": 0,
"action": "visible",
"status": "passed",
"at": "2026-10-13T20:10:09.121Z",
"selector": "h1"
},
{
"viewport": "narrow",
"index": 1,
"action": "text",
"status": "passed",
"at": "2026-10-13T20:10:09.324Z",
"selector": "#count"
},
{
"viewport": "narrow",
"index": 2,
"action": "press",
"status": "passed",
"at": "2026-10-13T20:10:09.431Z",
"key": "Tab"
},
{
"viewport": "narrow",
"index": 3,
"action": "focused",
"status": "passed",
"at": "2026-10-13T20:10:09.438Z",
"selector": "#increment"
},
{
"viewport": "narrow",
"index": 4,
"action": "press",
"status": "passed",
"at": "2026-10-13T20:10:09.514Z",
"key": "Enter"
},
{
"viewport": "narrow",
"index": 5,
"action": "text",
"status": "passed",
"at": "2026-10-13T20:10:09.521Z",
"selector": "#count"
},
{
"viewport": "narrow",
"index": 6,
"action": "screenshot",
"status": "passed",
"at": "2026-10-13T20:10:09.529Z"
}
],
"artifacts": [
{
"name": "desktop-6-keyboard-result.png",
"bytes": 26622
},
{
"name": "desktop-final.png",
"bytes": 26622
},
{
"name": "narrow-6-keyboard-result.png",
"bytes": 24280
},
{
"name": "narrow-final.png",
"bytes": 24280
}
],
"started_at": "2026-10-13T20:10:03.007Z",
"blocked_requests": 13,
"finished_at": "2026-10-13T20:10:10.349Z",
"downloads": [
{
"name": "desktop-6-keyboard-result.png",
"bytes": 26622,
"sha256": "b15998824256b86c74f2bbb75f061061be65db0b4f3bb5444529ac69a723cdad"
},
{
"name": "desktop-final.png",
"bytes": 26622,
"sha256": "b15998824256b86c74f2bbb75f061061be65db0b4f3bb5444529ac69a723cdad"
},
{
"name": "narrow-6-keyboard-result.png",
"bytes": 24280,
"sha256": "49d27cb169b8e180ce93969c6d2b8619a7684cf035bb9dcae9cd91d8bd9952e8"
},
{
"name": "narrow-final.png",
"bytes": 24280,
"sha256": "49d27cb169b8e180ce93969c6d2b8619a7684cf035bb9dcae9cd91d8bd9952e8"
}
]
},
"ui_failure_receipt": {
"id": "fc5aa6c0-6335-4b39-a321-5c24c4dea3c4",
"api_version": "1",
"status": "failed",
"error": "ui_assertion_failed",
"label": "independent-browser-trial",
"target": "fixture",
"viewports": [
{
"name": "desktop",
"width": 1280,
"height": 800
}
],
"timeout_ms": 60000,
"created_at": "2026-10-13T20:10:10.377Z",
"expires_at": "2026-10-13T21:10:10.377Z",
"service_version": "0.1.0",
"playwright_version": "1.64.0",
"release": "sha256:8d6fd5a8172cf906bb503642ed620761945c5138b86f6396c36f65cbe6def441",
"execution_location": "workstation-release-check",
"browser_version": "156.0.8078.4",
"completed_steps": 1,
"steps": [
{
"viewport": "desktop",
"index": 0,
"action": "screenshot",
"status": "passed",
"at": "2026-10-13T20:10:14.611Z"
},
{
"viewport": "desktop",
"index": 1,
"action": "text",
"status": "failed",
"at": "2026-10-13T20:10:15.009Z",
"selector": "#count"
}
],
"artifacts": [
{
"name": "desktop-0-before-failure.png",
"bytes": 25740
}
],
"started_at": "2026-10-13T20:10:10.380Z",
"blocked_requests": 9,
"finished_at": "2026-10-13T20:10:20.537Z",
"downloads": [
{
"name": "desktop-0-before-failure.png",
"bytes": 25740,
"sha256": "141dcadca4774da5fefec835bb789c5221685d3f453070a593e69aa6ad4782ce"
}
]
},
"cancellation_receipt": {
"id": "6af97df4-d8eb-4c3f-897f-47ec8568634a",
"api_version": "1",
"status": "cancelled",
"error": "cancelled",
"label": "independent-browser-trial",
"target": "fixture",
"viewports": [
{
"name": "desktop",
"width": 1280,
"height": 800
}
],
"timeout_ms": 60000,
"created_at": "2026-10-13T20:10:20.636Z",
"expires_at": "2026-10-13T21:10:20.636Z",
"service_version": "0.1.0",
"playwright_version": "1.64.0",
"release": "sha256:8d6fd5a8172cf906bb503642ed620761945c5138b86f6396c36f65cbe6def441",
"execution_location": "workstation-release-check",
"browser_version": null,
"completed_steps": 0,
"steps": [],
"artifacts": [],
"started_at": "2026-10-13T20:10:20.638Z",
"finished_at": "2026-10-13T20:10:20.658Z",
"downloads": []
}
}

29
docs/operations.md
View file

@ -82,3 +82,32 @@ imagetools inspect --raw <image-reference>` and hash the exact returned bytes.
Require the expected digest, then pull that immutable reference with
`sudo k3s crictl pull <image-reference>` on railiance01 and verify its recorded
repo digest. A successful upload alone does not establish node pull access.
## Repeatable consumer API acceptance
After Platform establishes and verifies the caller's own identity, permission
boundary and port-forward, run this from the provider checkout with Python 3.10+:
```sh
python3 tools/verify_consumer.py --url http://127.0.0.1:18081 \
--expected-release forgejo.coulomb.social/coulomb/rapp-playwright@sha256:8d6fd5a8172cf906bb503642ed620761945c5138b86f6396c36f65cbe6def441 \
--expected-location railiance01/playwright --out .local/consumer-acceptance-01
```
Use a fresh output directory for each caller/run. Exit 0 means browser API checks
passed; exit 3 means verification failed. `verification.json` lists checks and
errors. The command verifies exact release/location, desktop/narrow PNG dimensions
and hashed downloads, keyboard assertions, foreign-token receipt/artifact denial,
failed-assertion evidence, private-target refusal and running-job cancellation.
It submits only fixture jobs and one denied private-target request, cancels its
own runs on exit and does not restart the service or alter access grants.
Keep all `run.json` files private. Pair the result with Platform's native consumer
identity/permission receipt for each real human and registered coding agent.
The result deliberately says `identity_authentication_verified: false`: neither
operator impersonation nor this browser check alone proves self-service access.
Target-unreachable diagnostics, abrupt interruption/recovery, revocation/stream
termination and cross-version rollback require their separate evidence. The
command's local validation is retained in
[evidence](evidence/2026-10-13-consumer-verifier.json).

31
history/2026-10-13-2211-intent-scope-assessment.md Normal file
View file

@ -0,0 +1,31 @@
# INTENT against SCOPE — 2026-10-13 22:11 Europe/Berlin
The provider can move forward on repeatable acceptance while platform identity
activation is pending. `tools/verify_consumer.py` now executes the same browser
API checks through any already established consumer connection, with mandatory
release and execution-location expectations. The local real-browser run passed
all six checks; incorrect release and location were refused before submission.
See [retained receipts](../docs/evidence/2026-10-13-consumer-verifier.json) and
[command documentation](../docs/operations.md#repeatable-consumer-api-acceptance).
Custodian accepted coordination in CUST-WP-0080-T05, source
`3a850beb06bd1e753dd44f084e4b383983113706`. Concrete registration, custody,
JWT-authenticator and per-caller transport owner returns remain pending. This
acceptance is progress in accountability, not evidence that login works.
Most important remaining gaps:
1. Real human and coding-agent authentication/transport and their native identity,
permission and browser receipts. RPF-WP-0056-T03 owns delivery; provider
WP-0004-T01 accepts the browser/runtime result. Membership is already decided.
2. Abrupt-interruption recovery, revocation/connection termination and actual
cross-version rollback. The new command checks cancellation, not those cases.
3. Final managed availability and discovery wording after consumer acceptance.
Local-provider A5 discovery is already published; WP-0004-T03 remains open.
4. Content-explorer's broader route/visual acceptance remains consumer-owned.
The deployed pilot, published image, operator restart proof and A5 discovery
from the [previous assessment](2026-10-13-2142-intent-scope-assessment.md) remain
the latest inspected managed evidence. This pass changed only acceptance tooling
and documentation; it did not mutate the managed runtime. SCOPE reflects the new
command and accepted coordinator. The overall implementation is not complete.

156
tools/verify_consumer.py Normal file
View file

@ -0,0 +1,156 @@
#!/usr/bin/env python3
"""Verify the browser API through an already established consumer connection.
This command does not establish identity, create access grants or restart services.
"""
import argparse
from datetime import datetime, timezone
import json
from pathlib import Path
import struct
import sys
import time
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'client'))
from rapp_playwright import Client, ClientError, collect, save
TERMINAL = {'passed', 'failed', 'cancelled', 'interrupted'}
def require(condition, message):
if not condition:
raise ClientError(message)
def denied(operation, expected):
try:
operation()
except ClientError as exc:
require(str(exc) == expected, 'unexpected denial: ' + str(exc))
else:
raise ClientError('request unexpectedly permitted')
def wait_for(client, run, *, running=False):
deadline = time.monotonic() + 90
while time.monotonic() < deadline:
result = client.request('/v1/runs/' + run['id'], token=run['token'])
if running and result['status'] == 'running':
return result
if result['status'] in TERMINAL:
require(not running, 'cancellation case terminated before running')
return result
time.sleep(0.1)
raise ClientError('acceptance_wait_timeout')
def verify(args):
args.out.mkdir(parents=True, mode=0o700, exist_ok=False)
report = {'started_at': datetime.now(timezone.utc).isoformat(),
'service_url': args.url, 'argv': sys.argv, 'expected_release': args.expected_release,
'expected_location': args.expected_location, 'checks': [],
'status': 'failed', 'identity_authentication_verified': False,
'scope': 'browser API only; pair with native consumer identity and permission receipts'}
client = Client(args.url)
runs = []
def passed(name):
report['checks'].append(name)
def submit(spec, name):
out = args.out / name
out.mkdir(mode=0o700)
run = client.request('/v1/runs', 'POST', spec)
runs.append(run)
save(out / 'run.json', run)
return run, out
try:
pf = client.request('/v1/preflight')
save(args.out / 'preflight.json', pf)
require(pf.get('available') is True and pf.get('api_version') == '1', 'provider_not_ready')
require(pf.get('release') == args.expected_release, 'release_mismatch')
require(pf.get('execution_location') == args.expected_location, 'execution_location_mismatch')
passed('preflight_identity')
spec = client.request('/v1/trial')
spec['viewports'] = [{'name': 'desktop', 'width': 1280, 'height': 800},
{'name': 'narrow', 'width': 390, 'height': 844}]
save(args.out / 'trial-spec.json', spec)
run, out = submit(spec, 'trial')
wait_for(client, run)
receipt = collect(client, run, out)
require(receipt['status'] == 'passed' and receipt['error'] is None, 'trial_failed')
require(receipt['completed_steps'] == len(spec['steps']) * len(spec['viewports']), 'incomplete_trial')
require(receipt['release'] == args.expected_release and receipt['execution_location'] == args.expected_location,
'trial_identity_mismatch')
require(len(receipt['downloads']) == 4, 'expected_four_trial_screenshots')
sizes = {v['name']: (v['width'], v['height']) for v in spec['viewports']}
for item in receipt['downloads']:
data = (out / item['name']).read_bytes()
require(data[:8] == b'\x89PNG\r\n\x1a\n' and data[12:16] == b'IHDR', 'invalid_png')
viewport = next((v for v in sizes if item['name'].startswith(v + '-')), None)
require(viewport is not None and struct.unpack('>II', data[16:24]) == sizes[viewport], 'wrong_png_dimensions')
passed('responsive_keyboard_trial_and_downloads')
path = '/v1/runs/' + run['id']
denied(lambda: client.request(path, token='0' * 64), 'HTTP 404: not_found')
denied(lambda: client.request(path + '/artifacts/' + receipt['downloads'][0]['name'], token='0' * 64),
'HTTP 404: not_found')
passed('foreign_run_and_artifact_denied')
bad = {**spec, 'viewports': spec['viewports'][:1], 'steps': [
{'action': 'screenshot', 'name': 'before-failure'},
{'action': 'text', 'selector': '#count', 'value': 'acceptance-deliberately-wrong'}]}
failed, failure_out = submit(bad, 'ui-failure')
wait_for(client, failed)
failure = collect(client, failed, failure_out)
require(failure['status'] == 'failed' and failure['error'] == 'ui_assertion_failed', 'failure_not_classified')
require(len(failure['downloads']) >= 1, 'failure_artifact_missing')
passed('ui_failure_preserves_evidence')
# An undeclared private origin must fail before browser execution.
denied(lambda: client.request('/v1/runs', 'POST', {**spec, 'target': 'http://127.0.0.1:1/'}),
'HTTP 403: target_not_allowed')
passed('private_target_denied')
slow = {**spec, 'viewports': spec['viewports'][:1], 'timeout_ms': 60000,
'steps': [{'action': 'visible', 'selector': '#acceptance-never-present'}]}
cancelled, cancel_out = submit(slow, 'cancel')
wait_for(client, cancelled, running=True)
start = time.monotonic()
client.request('/v1/runs/' + cancelled['id'], 'DELETE', token=cancelled['token'])
result = wait_for(client, cancelled)
require(result['status'] == 'cancelled', 'cancellation_failed')
require(time.monotonic() - start < 10, 'cancellation_exceeded_10s')
collect(client, cancelled, cancel_out)
passed('running_job_cancellation')
report['status'] = 'passed'
except KeyboardInterrupt:
report['error'] = 'cancelled_by_client'
except (ClientError, OSError, ValueError, KeyError, struct.error) as exc:
report['error'] = str(exc)
finally:
# Cancel only runs created here. Completed runs remain downloadable until expiry.
for run in runs:
try:
client.request('/v1/runs/' + run['id'], 'DELETE', token=run['token'])
except ClientError:
pass
report['finished_at'] = datetime.now(timezone.utc).isoformat()
save(args.out / 'verification.json', report)
print(json.dumps(report, indent=2))
return 0 if report['status'] == 'passed' else 3
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--url', required=True)
parser.add_argument('--expected-release', required=True)
parser.add_argument('--expected-location', required=True)
parser.add_argument('--out', required=True, type=Path)
return verify(parser.parse_args())
if __name__ == '__main__':
try:
sys.exit(main())
except (ClientError, OSError, ValueError) as exc:
print(json.dumps({'status': 'failed', 'error': str(exc)}), file=sys.stderr)
sys.exit(3)

18
workplans/RAPP-PLAYWRIGHT-WP-0004-consumer-verification.md
View file

@ -147,3 +147,21 @@ update, rather than an unanswered local-provider registry handoff. The current
published admission-pending wording lags the deployed pilot and should distinguish
pilot runtime from consumer readiness in that update. T03 stays open as requested
by platform until the native consumer receipts are accepted.
## Acceptance command and coordination return — 2026-10-13 22:11 Europe/Berlin
Custodian accepted coordinating ownership in CUST-WP-0080-T05, source commit
`3a850beb06bd1e753dd44f084e4b383983113706`, message
`9defa19c-e812-4960-b563-2398688f562a`. Its delivery case preserves Platform T03
and provider T01/T03 acceptance. Owner registration, custody, native authenticator
and per-caller transport returns are still missing; coordination is not activation.
Implemented `tools/verify_consumer.py` for Platform to run through each actual
consumer connection. Six local API checks pass: exact provider identity,
responsive/keyboard trial and four PNG downloads, foreign-token denials,
failed-assertion artifact retrieval, private-target refusal and bounded running
cancellation. Wrong release and wrong execution location each fail before job
submission. Retained [verification](../docs/evidence/2026-10-13-consumer-verifier.json).
The command does not establish authentication, restart workloads or close T01;
pair it with native identity and permission receipts plus remaining recovery tests.