From ecb1ba47f6ec959b41d3a67986a3c391a0159cfa Mon Sep 17 00:00:00 2001 From: tegwick Date: Mon, 19 Oct 2026 12:49:15 +0200 Subject: [PATCH] MAS-WP-0013 finished: mode contract flows runnable from the bundle alone conformance/FLOWS.md defines setup, steps, captures and expectations, all observable at the contract boundary. flows.json carries setups (peer ids), fixed test keys, statute, scenario and explicit assign steps; scenarios.json holds the callback scenarios (the stub loads them). CONTRACT.md defines the case.assign response. Runner reads only the bundle and rejects undefined keys; new flow s-006 (held over cap). Co-Authored-By: Claude Opus 5.5 Assistant: claude-code Assistant-Model: opus Assistant-Process: 5787@bnt-lap001 Assistant-Session: 5db26b82-138b-40cc-9856-4d5eb0f752fd --- mas-modes-v0.1-spec/CONTRACT.md | 11 +- mas-modes-v0.1-spec/README.md | 5 +- mas-modes-v0.1-spec/conformance/FLOWS.md | 93 ++++++ mas-modes-v0.1-spec/conformance/flows.json | 314 +++++++++++++----- .../conformance/scenarios.json | 120 +++++++ mas_modes/conformance/FLOWS.md | 93 ++++++ mas_modes/conformance/flows.json | 314 +++++++++++++----- mas_modes/conformance/scenarios.json | 120 +++++++ mas_modes/harness.py | 4 +- mas_modes/runner.py | 213 ++++++++---- mas_modes/stubs/callback.py | 46 +-- pyproject.toml | 2 +- tests/test_modes_conformance.py | 16 +- .../MAS-WP-0013-harness-independent-flows.md | 36 +- 14 files changed, 1118 insertions(+), 269 deletions(-) create mode 100644 mas-modes-v0.1-spec/conformance/FLOWS.md create mode 100644 mas-modes-v0.1-spec/conformance/scenarios.json create mode 100644 mas_modes/conformance/FLOWS.md create mode 100644 mas_modes/conformance/scenarios.json diff --git a/mas-modes-v0.1-spec/CONTRACT.md b/mas-modes-v0.1-spec/CONTRACT.md index 1dd9fc4..3bfc41d 100644 --- a/mas-modes-v0.1-spec/CONTRACT.md +++ b/mas-modes-v0.1-spec/CONTRACT.md @@ -150,6 +150,11 @@ A receiver MUST refuse a transition that the table does not allow `automated-sender`, `send-cap-reached`, `unknown-message`. With `send-cap-reached` the response also carries `held` (bool, §6). +`case.assign` responds `{"ok": true, "case": {address, key}, +"message_ref": M}`, where M is the intake message, now recorded on the +case. When `autonomous_send` is on, the response also has `acknowledged` +(bool). When the intake requested a mode, it has `handoff_id`. + ## 6. Principle 10 in both roles Email MUST NOT send mail to a sender that did not pass DMARC (as read @@ -252,8 +257,10 @@ A sender MUST NOT retry `401` or `403` unchanged. ## 8. Conformance `conformance/` holds valid and invalid messages, the transition table as -data, and scripted flows. Each flow is a sequence of inbound mail, -contract messages and clock ticks, with the expected email-side reaction. +data, the callback scenarios (`scenarios.json`), and scripted flows. Each +flow is a sequence of inbound mail, contract messages and clock ticks, +with the expected email-side reaction, defined in `conformance/FLOWS.md` +and runnable from the bundle alone. `conformance/auth.json` holds signing vectors and verification cases for section 7; every party MUST pass them. Flows `a-*` check authorization at the email side. An email-side implementation claims `MAS-MODES-EMAIL-MASTER` diff --git a/mas-modes-v0.1-spec/README.md b/mas-modes-v0.1-spec/README.md index 9ab6f67..3d5ee76 100644 --- a/mas-modes-v0.1-spec/README.md +++ b/mas-modes-v0.1-spec/README.md @@ -15,7 +15,10 @@ separately from the MAS address specification (`mas-v0.2-spec/`, §7). - `conformance/transitions.json`: the status machine as data - `conformance/messages.json`: valid and invalid messages - `conformance/flows.json`: scripted flows for `MAS-MODES-EMAIL-MASTER` and - `MAS-MODES-EMAIL-SLAVE` + `MAS-MODES-EMAIL-SLAVE`, with setups and test keys; runnable from the + bundle alone +- `conformance/FLOWS.md`: the flow format: steps, captures and expectations +- `conformance/scenarios.json`: the callback stub's scenarios - `conformance/auth.json`: signing vectors and verification cases (§7) - `conformance/statute.json`: the statute the flows run under diff --git a/mas-modes-v0.1-spec/conformance/FLOWS.md b/mas-modes-v0.1-spec/conformance/FLOWS.md new file mode 100644 index 0000000..18065f3 --- /dev/null +++ b/mas-modes-v0.1-spec/conformance/FLOWS.md @@ -0,0 +1,93 @@ +# Conformance flows (mode contract v0.1) + +`flows.json` describes complete conversations that an email-side +implementation must handle. A run needs only this bundle: `flows.json`, +`scenarios.json` and `statute.json`. Every expectation is observable at +the contract boundary: messages the stub parties receive, responses to +contract requests, and mail the email side sends. Nothing depends on how an +implementation stores data or answers the gateway. + +## Running a flow + +For each flow, start fresh: + +1. **Email side.** An email side for the statute in `statute.json` + (namespace `modes.example`), configured with the flow's `setup`, i.e. + `setups[flow.setup].roles`. Its own peer id is `email`, and it knows the + keys in `peers`. A master role needs a store for its cases. The + reference uses its ticket service, but any store that creates a case on + intake mail and correlates mail to `ticket=ID` works. +2. **Callback stub.** Peer `callback-1` speaks the contract and plays + `scenarios.json`, with `flow.scenario` as its default. It sends nothing + until a `mode` step. +3. **Outside master stub.** Peer `cms-1` records every message it gets, + and sends commands only in `master` steps. +4. **Clock.** A controllable clock for the email side's timeouts. Only + `advance` steps move it. Signatures use real time. +5. **Send cap.** When `flow.max_sent` is set, the email side's send cap is + that number, and an SMTP sink counts what is actually sent. Without it + there is no cap. + +Role configuration fields are those of CONTRACT.md §3. In a setup: + +- `modes: {MODE: {"peer": ID}}` means the mode endpoint of that peer; +- `master_peer` is the outside master, which implies its event URL; +- `""` stands for the handoff URL of peer ID. + +The implementation wires peer ids to its own URLs. The `peers` keys are +public **test keys**, never to be used in a deployment. + +## Steps + +| Step | Meaning | +|---|---| +| `{"mail": {to, text, dmarc?, subject?}}` | Inbound mail from `customer@example.net` to `TO@modes.example`, through the normal gateway path. `dmarc` (default `"pass"`) is the trusted Authentication-Results verdict; `null` means no verdict. | +| `{"mode": "run"}` | The callback stub sends every status that is due, until it is waiting, silent or terminal. | +| `{"advance": N}` | Move the email side's clock N seconds and let it run its timeouts. | +| `{"master": {"command": C, ...}}` | The stub master sends a signed contract command: `assign` (`case.assign`; `resource_id`, plus `intake_ref`, by default the captured one), `send` (`mail.send`; `in_reply_to`, `body`), `cancel` (`handoff.cancel`; `handoff_id`), or `policy` (`case.policy`). The case is the captured case. | +| `{"inject": {endpoint, as?, sign?, token?, message}}` | Send a crafted request straight to the email side: `endpoint` is `master`, `status` (the captured handoff's status URL) or `events`. `as` is the claimed peer id. `sign` is `valid` (default), `none`, `stale` (timestamp 400 s old) or `wrong-key`. `token: "bogus"` replaces the status URL token. In `message`, `"case": "current"` means the captured case, and the envelope (`mas_modes_version`, `id`) is added. | + +Strings in steps may contain the captures `{case_id}`, `{message_ref}`, +`{intake_ref}` and `{handoff_id}`. + +## Captures + +Captures are updated after every step, from what that step produced, in +this order: + +1. every message the mode stub and the master stub received; +2. the response to a `master` or `inject` step. + +The latest value wins. + +| Capture | From | +|---|---| +| case (`address`, `key`) | the `case` of any of those messages or responses | +| `{case_id}` | the resource id in the captured case address (`TYPE=ID@D`) | +| `{message_ref}` | `message_ref` of `case.message`, or of the `case.assign` response | +| `{intake_ref}` | `intake_ref` of `case.intake` | +| `{handoff_id}` | `handoff_id` of `handoff.request` (mode) or `handoff.requested` (master) | + +## Expectations + +`expect` is checked after the step. Every key listed must match. + +| Key | Value | +|---|---| +| `event_accepted` | the email side answered the step's inbound mail event with 2xx | +| `mode_statuses` | statuses of every `handoff.status` the mode stub sent, in order | +| `mode_received` | number of contract messages the mode stub received | +| `mode_received_last` | `type` of the last one | +| `master_received` | `{type: count}` of messages the master stub received; only the listed types are compared | +| `relayed` | statuses of every `handoff.status` the master stub received, in order | +| `mails_sent` | number of mails the email side sent (held and refused mail excluded) | +| `last_mail_contains` | the body of the last mail sent contains this text | +| `response_ok` | `ok` in the response to the step's command | +| `response_reason` | `reason` in that response | +| `response_held` | `held` in that response (CONTRACT.md §6) | +| `case_address` | `case.address` in that response | +| `http_status` | HTTP status of that response | +| `error` | `error` in that response | + +A runner MUST reject a flow that uses a step or expectation key not +defined here. diff --git a/mas-modes-v0.1-spec/conformance/flows.json b/mas-modes-v0.1-spec/conformance/flows.json index 9c0a83a..478f1d9 100644 --- a/mas-modes-v0.1-spec/conformance/flows.json +++ b/mas-modes-v0.1-spec/conformance/flows.json @@ -1,9 +1,58 @@ { - "description": "Scripted flows (CONTRACT.md section 7). Each runs against a fresh email side with the callback stub and, for 'slave', a stub outside master that assigns EXT-100, EXT-101, ... The statute is statute.json. Placeholders: {case_id}, {message_ref}, {handoff_id} come from the latest response that carried them; {callback} is the stub's handoff URL. Peers: email, callback-1 (the stub), cms-1 (the stub master); every request is signed (CONTRACT.md section 7). Master role config: ticket, modes {callback}, fields phone/time, timeout 3600 s. Slave role config: case, defaults autonomous_send false, transfers block, overridden by 'config'.", + "description": "Scripted flows (CONTRACT.md section 8). The format, steps, expectations and captures are defined in conformance/FLOWS.md. Everything a run needs is in this file, scenarios.json and statute.json.", + "statute": "statute.json", + "peers": { + "callback-1": "6d61732d6d6f6465732d76302e312d746573742d6b65792d63616c6c6261636b31", + "cms-1": "6d61732d6d6f6465732d76302e312d746573742d6b65792d636d732d312d2d2d2d", + "gateway": "6d61732d6d6f6465732d76302e312d746573742d6b65792d676174657761792d2d" + }, + "setups": { + "master": { + "roles": { + "ticket": { + "role": "master", + "modes": { + "callback": { + "peer": "callback-1" + } + }, + "fields": [ + "phone", + "time", + "scenario" + ], + "timeout": 3600 + } + } + }, + "slave": { + "roles": { + "case": { + "role": "slave", + "master_peer": "cms-1", + "autonomous_send": false, + "transfers": "block" + } + } + }, + "slave-webhook": { + "roles": { + "case": { + "role": "slave", + "master_peer": "cms-1", + "autonomous_send": false, + "transfers": { + "webhook": "", + "peer": "callback-1" + } + } + } + } + }, "flows": [ { "id": "m-001", - "role": "master", + "setup": "master", "title": "callback completes", "scenario": "completed", "steps": [ @@ -13,13 +62,14 @@ "text": "Phone: +49 30 1234567\nTime: tomorrow 10:00" }, "expect": { - "response_status": "created" + "event_accepted": true, + "mode_received": 1 } }, { "mode": "run", "expect": { - "statuses": [ + "mode_statuses": [ "scheduled", "running", "completed" @@ -31,7 +81,7 @@ }, { "id": "m-002", - "role": "master", + "setup": "master", "title": "negotiation over email", "scenario": "negotiate", "steps": [ @@ -41,13 +91,13 @@ "text": "Please call me." }, "expect": { - "response_status": "created" + "event_accepted": true } }, { "mode": "run", "expect": { - "statuses": [ + "mode_statuses": [ "negotiating" ], "last_mail_contains": "Phone: ..." @@ -59,13 +109,14 @@ "text": "Phone: +49 30 1234567\nTime: tomorrow 10:00" }, "expect": { - "response_status": "correlated" + "event_accepted": true, + "mode_received_last": "handoff.update" } }, { "mode": "run", "expect": { - "statuses": [ + "mode_statuses": [ "negotiating", "scheduled", "running", @@ -78,7 +129,7 @@ }, { "id": "m-003", - "role": "master", + "setup": "master", "title": "no answer, retried, completed", "scenario": "no-answer-retry", "steps": [ @@ -91,7 +142,7 @@ { "mode": "run", "expect": { - "statuses": [ + "mode_statuses": [ "scheduled", "attempted", "scheduled", @@ -104,7 +155,7 @@ }, { "id": "m-004", - "role": "master", + "setup": "master", "title": "no answer twice, failed", "scenario": "no-answer-fail", "steps": [ @@ -117,7 +168,7 @@ { "mode": "run", "expect": { - "statuses": [ + "mode_statuses": [ "scheduled", "attempted", "scheduled", @@ -131,7 +182,7 @@ }, { "id": "m-005", - "role": "master", + "setup": "master", "title": "refused out of hours", "scenario": "out-of-hours", "steps": [ @@ -144,7 +195,7 @@ { "mode": "run", "expect": { - "statuses": [ + "mode_statuses": [ "refused" ], "last_mail_contains": "out-of-hours" @@ -154,7 +205,7 @@ }, { "id": "m-006", - "role": "master", + "setup": "master", "title": "email owns the timeout", "scenario": "silent", "steps": [ @@ -162,33 +213,44 @@ "mail": { "to": "ticket!callback", "text": "Phone: +49 30 1234567\nTime: tomorrow 10:00" + }, + "expect": { + "mails_sent": 1 } }, { "mode": "run", "expect": { - "statuses": [] + "mode_statuses": [] } }, { "advance": 3599, "expect": { - "timed_out": 0 + "mode_received": 1, + "mails_sent": 1 } }, { "advance": 2, "expect": { - "timed_out": 1, "mode_received_last": "handoff.cancel", + "mails_sent": 2, "last_mail_contains": "could not arrange" } + }, + { + "advance": 100000, + "expect": { + "mode_received": 2, + "mails_sent": 2 + } } ] }, { "id": "m-007", - "role": "master", + "setup": "master", "title": "mode not enabled by the statute", "scenario": "completed", "steps": [ @@ -198,16 +260,16 @@ "text": "hi" }, "expect": { - "response_status": "created", + "event_accepted": true, "mode_received": 0, - "outbox_count": 1 + "mails_sent": 1 } } ] }, { "id": "m-008", - "role": "master", + "setup": "master", "title": "unauthenticated sender gets no mail", "scenario": "negotiate", "steps": [ @@ -218,24 +280,25 @@ "dmarc": "fail" }, "expect": { - "response_status": "created" + "event_accepted": true } }, { "mode": "run", "expect": { - "statuses": [ + "mode_statuses": [ "negotiating" ], - "outbox_count": 0 + "mails_sent": 0 } } ] }, { "id": "s-001", - "role": "slave", + "setup": "slave", "title": "master issues the id; email sends only on instruction", + "scenario": "completed", "steps": [ { "mail": { @@ -243,16 +306,20 @@ "text": "My order is late" }, "expect": { - "response_status": "intake-reported", + "event_accepted": true, "master_received": { "case.intake": 1 }, - "outbox_count": 0 + "mails_sent": 0 } }, { - "pump": true, + "master": { + "command": "assign", + "resource_id": "EXT-100" + }, "expect": { + "response_ok": true, "case_address": "case=EXT-100@modes.example" } }, @@ -262,11 +329,10 @@ "text": "more" }, "expect": { - "response_status": "reported", "master_received": { "case.message": 1 }, - "outbox_count": 0 + "mails_sent": 0 } }, { @@ -276,16 +342,17 @@ "body": "We are on it." }, "expect": { - "command_ok": true, - "outbox_count": 1 + "response_ok": true, + "mails_sent": 1 } } ] }, { "id": "s-002", - "role": "slave", + "setup": "slave", "title": "principle 10 refuses a master send", + "scenario": "completed", "steps": [ { "mail": { @@ -295,7 +362,10 @@ } }, { - "pump": true + "master": { + "command": "assign", + "resource_id": "EXT-100" + } }, { "master": { @@ -304,31 +374,43 @@ "body": "x" }, "expect": { - "command_ok": false, - "command_reason": "unauthenticated-sender", - "outbox_count": 0 + "response_ok": false, + "response_reason": "unauthenticated-sender", + "mails_sent": 0 } } ] }, { "id": "s-003", - "role": "slave", + "setup": "slave", "title": "transfer blocked", + "scenario": "completed", "steps": [ { "mail": { "to": "case!callback", "text": "Phone: +49 30 1234567\nTime: tomorrow 10:00" + }, + "expect": { + "master_received": { + "case.intake": 1 + } } }, { - "pump": true, + "master": { + "command": "assign", + "resource_id": "EXT-100" + }, "expect": { "master_received": { "handoff.requested": 1, "handoff.status": 1 }, + "relayed": [ + "refused" + ], "mode_received": 0 } } @@ -336,14 +418,8 @@ }, { "id": "s-004", - "role": "slave", + "setup": "slave-webhook", "title": "transfer to the primed webhook; statuses relayed", - "config": { - "transfers": { - "webhook": "{callback}", - "peer": "callback-1" - } - }, "scenario": "completed", "steps": [ { @@ -353,7 +429,10 @@ } }, { - "pump": true, + "master": { + "command": "assign", + "resource_id": "EXT-100" + }, "expect": { "mode_received": 1 } @@ -366,27 +445,21 @@ "running", "completed" ], - "outbox_count": 0 + "mails_sent": 0 } }, { "advance": 1000000, "expect": { - "timed_out": 0 + "mode_received": 1 } } ] }, { "id": "s-005", - "role": "slave", + "setup": "slave-webhook", "title": "the master owns the timeout", - "config": { - "transfers": { - "webhook": "{callback}", - "peer": "callback-1" - } - }, "scenario": "silent", "steps": [ { @@ -396,7 +469,10 @@ } }, { - "pump": true + "master": { + "command": "assign", + "resource_id": "EXT-100" + } }, { "mode": "run", @@ -404,13 +480,19 @@ "relayed": [] } }, + { + "advance": 1000000, + "expect": { + "mode_received": 1 + } + }, { "master": { "command": "cancel", "handoff_id": "{handoff_id}" }, "expect": { - "command_ok": true, + "response_ok": true, "mode_received_last": "handoff.cancel" } }, @@ -425,9 +507,10 @@ ] }, { - "id": "a-001", - "role": "slave", - "title": "unsigned master command refused", + "id": "s-006", + "setup": "slave", + "title": "over the cap a master send is held", + "scenario": "completed", "steps": [ { "mail": { @@ -436,7 +519,55 @@ } }, { - "pump": true + "master": { + "command": "assign", + "resource_id": "EXT-100" + } + }, + { + "master": { + "command": "send", + "in_reply_to": "{message_ref}", + "body": "one" + }, + "expect": { + "response_ok": true, + "mails_sent": 1 + } + }, + { + "master": { + "command": "send", + "in_reply_to": "{message_ref}", + "body": "two" + }, + "expect": { + "response_ok": false, + "response_reason": "send-cap-reached", + "response_held": true, + "mails_sent": 1 + } + } + ], + "max_sent": 1 + }, + { + "id": "a-001", + "setup": "slave", + "title": "unsigned master command refused", + "scenario": "completed", + "steps": [ + { + "mail": { + "to": "case", + "text": "hi" + } + }, + { + "master": { + "command": "assign", + "resource_id": "EXT-100" + } }, { "inject": { @@ -452,23 +583,21 @@ "expect": { "http_status": 401, "error": "unauthenticated-peer", - "outbox_count": 0 + "mails_sent": 0 } } ] }, { "id": "a-002", - "role": "slave", + "setup": "slave", "title": "a mode cannot assign case ids", + "scenario": "completed", "steps": [ { "mail": { "to": "case", "text": "hi" - }, - "expect": { - "response_status": "intake-reported" } }, { @@ -487,7 +616,10 @@ } }, { - "pump": true, + "master": { + "command": "assign", + "resource_id": "EXT-100" + }, "expect": { "case_address": "case=EXT-100@modes.example" } @@ -496,8 +628,9 @@ }, { "id": "a-003", - "role": "slave", + "setup": "slave", "title": "webhook to a peer email has no key for", + "scenario": "completed", "steps": [ { "mail": { @@ -506,7 +639,10 @@ } }, { - "pump": true + "master": { + "command": "assign", + "resource_id": "EXT-100" + } }, { "inject": { @@ -530,8 +666,9 @@ }, { "id": "a-004", - "role": "slave", + "setup": "slave", "title": "replayed master command", + "scenario": "completed", "steps": [ { "mail": { @@ -540,7 +677,10 @@ } }, { - "pump": true + "master": { + "command": "assign", + "resource_id": "EXT-100" + } }, { "inject": { @@ -557,15 +697,16 @@ "expect": { "http_status": 401, "error": "replayed", - "outbox_count": 0 + "mails_sent": 0 } } ] }, { "id": "a-005", - "role": "slave", + "setup": "slave", "title": "master command signed with the wrong key", + "scenario": "completed", "steps": [ { "mail": { @@ -574,7 +715,10 @@ } }, { - "pump": true + "master": { + "command": "assign", + "resource_id": "EXT-100" + } }, { "inject": { @@ -591,7 +735,7 @@ "expect": { "http_status": 401, "error": "unauthenticated-peer", - "outbox_count": 0 + "mails_sent": 0 } }, { @@ -607,14 +751,14 @@ }, "expect": { "http_status": 200, - "outbox_count": 1 + "mails_sent": 1 } } ] }, { "id": "a-006", - "role": "master", + "setup": "master", "title": "statuses only from the handoff's peer, at its URL", "scenario": "silent", "steps": [ @@ -624,7 +768,7 @@ "text": "Phone: +49 30 1234567\nTime: tomorrow 10:00" }, "expect": { - "outbox_count": 1 + "mails_sent": 1 } }, { @@ -634,6 +778,7 @@ "message": { "type": "handoff.status", "case": "current", + "handoff_id": "{handoff_id}", "seq": 1, "status": "completed" } @@ -651,6 +796,7 @@ "message": { "type": "handoff.status", "case": "current", + "handoff_id": "{handoff_id}", "seq": 1, "status": "completed" } @@ -658,7 +804,7 @@ "expect": { "http_status": 404, "error": "unknown-handoff", - "outbox_count": 1 + "mails_sent": 1 } }, { @@ -668,6 +814,7 @@ "message": { "type": "handoff.status", "case": "current", + "handoff_id": "{handoff_id}", "seq": 1, "status": "scheduled", "at": "x" @@ -681,8 +828,9 @@ }, { "id": "a-007", - "role": "master", + "setup": "master", "title": "only the gateway posts events", + "scenario": "completed", "steps": [ { "inject": { diff --git a/mas-modes-v0.1-spec/conformance/scenarios.json b/mas-modes-v0.1-spec/conformance/scenarios.json new file mode 100644 index 0000000..caeae4b --- /dev/null +++ b/mas-modes-v0.1-spec/conformance/scenarios.json @@ -0,0 +1,120 @@ +{ + "description": "Callback stub scenarios (CONTRACT.md section 8). Each is the sequence of handoff.status messages the stub sends, one per step. A step {'wait_for_details': [fields]} sends 'negotiating' with needs = the missing fields and waits until handoff.update supplies them; it is skipped when the handoff.request already carried them. After handoff.cancel the stub sends 'cancelled' next. 'silent' never answers.", + "default": "completed", + "scenarios": { + "completed": [ + { + "status": "scheduled", + "at": "2026-10-20T10:00:00Z" + }, + { + "status": "running" + }, + { + "status": "completed" + } + ], + "negotiate": [ + { + "wait_for_details": [ + "phone", + "time" + ] + }, + { + "status": "scheduled", + "at": "2026-10-20T10:00:00Z" + }, + { + "status": "running" + }, + { + "status": "completed" + } + ], + "no-answer-retry": [ + { + "status": "scheduled", + "at": "2026-10-20T10:00:00Z" + }, + { + "status": "attempted", + "outcome": "no-answer" + }, + { + "status": "scheduled", + "at": "2026-10-20T14:00:00Z" + }, + { + "status": "running" + }, + { + "status": "completed" + } + ], + "no-answer-fail": [ + { + "status": "scheduled", + "at": "2026-10-20T10:00:00Z" + }, + { + "status": "attempted", + "outcome": "no-answer" + }, + { + "status": "scheduled", + "at": "2026-10-20T14:00:00Z" + }, + { + "status": "attempted", + "outcome": "no-answer" + }, + { + "status": "failed", + "detail": "two attempts, no answer" + } + ], + "call-failed": [ + { + "status": "scheduled", + "at": "2026-10-20T10:00:00Z" + }, + { + "status": "running" + }, + { + "status": "failed", + "detail": "line dropped" + } + ], + "expired": [ + { + "status": "scheduled", + "at": "2026-10-20T10:00:00Z" + }, + { + "status": "expired", + "detail": "slot passed" + } + ], + "unavailable": [ + { + "status": "refused", + "reason": "mode-unavailable" + } + ], + "declined": [ + { + "status": "refused", + "reason": "declined" + } + ], + "out-of-hours": [ + { + "status": "refused", + "reason": "out-of-hours" + } + ], + "silent": [] + } +} diff --git a/mas_modes/conformance/FLOWS.md b/mas_modes/conformance/FLOWS.md new file mode 100644 index 0000000..18065f3 --- /dev/null +++ b/mas_modes/conformance/FLOWS.md @@ -0,0 +1,93 @@ +# Conformance flows (mode contract v0.1) + +`flows.json` describes complete conversations that an email-side +implementation must handle. A run needs only this bundle: `flows.json`, +`scenarios.json` and `statute.json`. Every expectation is observable at +the contract boundary: messages the stub parties receive, responses to +contract requests, and mail the email side sends. Nothing depends on how an +implementation stores data or answers the gateway. + +## Running a flow + +For each flow, start fresh: + +1. **Email side.** An email side for the statute in `statute.json` + (namespace `modes.example`), configured with the flow's `setup`, i.e. + `setups[flow.setup].roles`. Its own peer id is `email`, and it knows the + keys in `peers`. A master role needs a store for its cases. The + reference uses its ticket service, but any store that creates a case on + intake mail and correlates mail to `ticket=ID` works. +2. **Callback stub.** Peer `callback-1` speaks the contract and plays + `scenarios.json`, with `flow.scenario` as its default. It sends nothing + until a `mode` step. +3. **Outside master stub.** Peer `cms-1` records every message it gets, + and sends commands only in `master` steps. +4. **Clock.** A controllable clock for the email side's timeouts. Only + `advance` steps move it. Signatures use real time. +5. **Send cap.** When `flow.max_sent` is set, the email side's send cap is + that number, and an SMTP sink counts what is actually sent. Without it + there is no cap. + +Role configuration fields are those of CONTRACT.md §3. In a setup: + +- `modes: {MODE: {"peer": ID}}` means the mode endpoint of that peer; +- `master_peer` is the outside master, which implies its event URL; +- `""` stands for the handoff URL of peer ID. + +The implementation wires peer ids to its own URLs. The `peers` keys are +public **test keys**, never to be used in a deployment. + +## Steps + +| Step | Meaning | +|---|---| +| `{"mail": {to, text, dmarc?, subject?}}` | Inbound mail from `customer@example.net` to `TO@modes.example`, through the normal gateway path. `dmarc` (default `"pass"`) is the trusted Authentication-Results verdict; `null` means no verdict. | +| `{"mode": "run"}` | The callback stub sends every status that is due, until it is waiting, silent or terminal. | +| `{"advance": N}` | Move the email side's clock N seconds and let it run its timeouts. | +| `{"master": {"command": C, ...}}` | The stub master sends a signed contract command: `assign` (`case.assign`; `resource_id`, plus `intake_ref`, by default the captured one), `send` (`mail.send`; `in_reply_to`, `body`), `cancel` (`handoff.cancel`; `handoff_id`), or `policy` (`case.policy`). The case is the captured case. | +| `{"inject": {endpoint, as?, sign?, token?, message}}` | Send a crafted request straight to the email side: `endpoint` is `master`, `status` (the captured handoff's status URL) or `events`. `as` is the claimed peer id. `sign` is `valid` (default), `none`, `stale` (timestamp 400 s old) or `wrong-key`. `token: "bogus"` replaces the status URL token. In `message`, `"case": "current"` means the captured case, and the envelope (`mas_modes_version`, `id`) is added. | + +Strings in steps may contain the captures `{case_id}`, `{message_ref}`, +`{intake_ref}` and `{handoff_id}`. + +## Captures + +Captures are updated after every step, from what that step produced, in +this order: + +1. every message the mode stub and the master stub received; +2. the response to a `master` or `inject` step. + +The latest value wins. + +| Capture | From | +|---|---| +| case (`address`, `key`) | the `case` of any of those messages or responses | +| `{case_id}` | the resource id in the captured case address (`TYPE=ID@D`) | +| `{message_ref}` | `message_ref` of `case.message`, or of the `case.assign` response | +| `{intake_ref}` | `intake_ref` of `case.intake` | +| `{handoff_id}` | `handoff_id` of `handoff.request` (mode) or `handoff.requested` (master) | + +## Expectations + +`expect` is checked after the step. Every key listed must match. + +| Key | Value | +|---|---| +| `event_accepted` | the email side answered the step's inbound mail event with 2xx | +| `mode_statuses` | statuses of every `handoff.status` the mode stub sent, in order | +| `mode_received` | number of contract messages the mode stub received | +| `mode_received_last` | `type` of the last one | +| `master_received` | `{type: count}` of messages the master stub received; only the listed types are compared | +| `relayed` | statuses of every `handoff.status` the master stub received, in order | +| `mails_sent` | number of mails the email side sent (held and refused mail excluded) | +| `last_mail_contains` | the body of the last mail sent contains this text | +| `response_ok` | `ok` in the response to the step's command | +| `response_reason` | `reason` in that response | +| `response_held` | `held` in that response (CONTRACT.md §6) | +| `case_address` | `case.address` in that response | +| `http_status` | HTTP status of that response | +| `error` | `error` in that response | + +A runner MUST reject a flow that uses a step or expectation key not +defined here. diff --git a/mas_modes/conformance/flows.json b/mas_modes/conformance/flows.json index 9c0a83a..478f1d9 100644 --- a/mas_modes/conformance/flows.json +++ b/mas_modes/conformance/flows.json @@ -1,9 +1,58 @@ { - "description": "Scripted flows (CONTRACT.md section 7). Each runs against a fresh email side with the callback stub and, for 'slave', a stub outside master that assigns EXT-100, EXT-101, ... The statute is statute.json. Placeholders: {case_id}, {message_ref}, {handoff_id} come from the latest response that carried them; {callback} is the stub's handoff URL. Peers: email, callback-1 (the stub), cms-1 (the stub master); every request is signed (CONTRACT.md section 7). Master role config: ticket, modes {callback}, fields phone/time, timeout 3600 s. Slave role config: case, defaults autonomous_send false, transfers block, overridden by 'config'.", + "description": "Scripted flows (CONTRACT.md section 8). The format, steps, expectations and captures are defined in conformance/FLOWS.md. Everything a run needs is in this file, scenarios.json and statute.json.", + "statute": "statute.json", + "peers": { + "callback-1": "6d61732d6d6f6465732d76302e312d746573742d6b65792d63616c6c6261636b31", + "cms-1": "6d61732d6d6f6465732d76302e312d746573742d6b65792d636d732d312d2d2d2d", + "gateway": "6d61732d6d6f6465732d76302e312d746573742d6b65792d676174657761792d2d" + }, + "setups": { + "master": { + "roles": { + "ticket": { + "role": "master", + "modes": { + "callback": { + "peer": "callback-1" + } + }, + "fields": [ + "phone", + "time", + "scenario" + ], + "timeout": 3600 + } + } + }, + "slave": { + "roles": { + "case": { + "role": "slave", + "master_peer": "cms-1", + "autonomous_send": false, + "transfers": "block" + } + } + }, + "slave-webhook": { + "roles": { + "case": { + "role": "slave", + "master_peer": "cms-1", + "autonomous_send": false, + "transfers": { + "webhook": "", + "peer": "callback-1" + } + } + } + } + }, "flows": [ { "id": "m-001", - "role": "master", + "setup": "master", "title": "callback completes", "scenario": "completed", "steps": [ @@ -13,13 +62,14 @@ "text": "Phone: +49 30 1234567\nTime: tomorrow 10:00" }, "expect": { - "response_status": "created" + "event_accepted": true, + "mode_received": 1 } }, { "mode": "run", "expect": { - "statuses": [ + "mode_statuses": [ "scheduled", "running", "completed" @@ -31,7 +81,7 @@ }, { "id": "m-002", - "role": "master", + "setup": "master", "title": "negotiation over email", "scenario": "negotiate", "steps": [ @@ -41,13 +91,13 @@ "text": "Please call me." }, "expect": { - "response_status": "created" + "event_accepted": true } }, { "mode": "run", "expect": { - "statuses": [ + "mode_statuses": [ "negotiating" ], "last_mail_contains": "Phone: ..." @@ -59,13 +109,14 @@ "text": "Phone: +49 30 1234567\nTime: tomorrow 10:00" }, "expect": { - "response_status": "correlated" + "event_accepted": true, + "mode_received_last": "handoff.update" } }, { "mode": "run", "expect": { - "statuses": [ + "mode_statuses": [ "negotiating", "scheduled", "running", @@ -78,7 +129,7 @@ }, { "id": "m-003", - "role": "master", + "setup": "master", "title": "no answer, retried, completed", "scenario": "no-answer-retry", "steps": [ @@ -91,7 +142,7 @@ { "mode": "run", "expect": { - "statuses": [ + "mode_statuses": [ "scheduled", "attempted", "scheduled", @@ -104,7 +155,7 @@ }, { "id": "m-004", - "role": "master", + "setup": "master", "title": "no answer twice, failed", "scenario": "no-answer-fail", "steps": [ @@ -117,7 +168,7 @@ { "mode": "run", "expect": { - "statuses": [ + "mode_statuses": [ "scheduled", "attempted", "scheduled", @@ -131,7 +182,7 @@ }, { "id": "m-005", - "role": "master", + "setup": "master", "title": "refused out of hours", "scenario": "out-of-hours", "steps": [ @@ -144,7 +195,7 @@ { "mode": "run", "expect": { - "statuses": [ + "mode_statuses": [ "refused" ], "last_mail_contains": "out-of-hours" @@ -154,7 +205,7 @@ }, { "id": "m-006", - "role": "master", + "setup": "master", "title": "email owns the timeout", "scenario": "silent", "steps": [ @@ -162,33 +213,44 @@ "mail": { "to": "ticket!callback", "text": "Phone: +49 30 1234567\nTime: tomorrow 10:00" + }, + "expect": { + "mails_sent": 1 } }, { "mode": "run", "expect": { - "statuses": [] + "mode_statuses": [] } }, { "advance": 3599, "expect": { - "timed_out": 0 + "mode_received": 1, + "mails_sent": 1 } }, { "advance": 2, "expect": { - "timed_out": 1, "mode_received_last": "handoff.cancel", + "mails_sent": 2, "last_mail_contains": "could not arrange" } + }, + { + "advance": 100000, + "expect": { + "mode_received": 2, + "mails_sent": 2 + } } ] }, { "id": "m-007", - "role": "master", + "setup": "master", "title": "mode not enabled by the statute", "scenario": "completed", "steps": [ @@ -198,16 +260,16 @@ "text": "hi" }, "expect": { - "response_status": "created", + "event_accepted": true, "mode_received": 0, - "outbox_count": 1 + "mails_sent": 1 } } ] }, { "id": "m-008", - "role": "master", + "setup": "master", "title": "unauthenticated sender gets no mail", "scenario": "negotiate", "steps": [ @@ -218,24 +280,25 @@ "dmarc": "fail" }, "expect": { - "response_status": "created" + "event_accepted": true } }, { "mode": "run", "expect": { - "statuses": [ + "mode_statuses": [ "negotiating" ], - "outbox_count": 0 + "mails_sent": 0 } } ] }, { "id": "s-001", - "role": "slave", + "setup": "slave", "title": "master issues the id; email sends only on instruction", + "scenario": "completed", "steps": [ { "mail": { @@ -243,16 +306,20 @@ "text": "My order is late" }, "expect": { - "response_status": "intake-reported", + "event_accepted": true, "master_received": { "case.intake": 1 }, - "outbox_count": 0 + "mails_sent": 0 } }, { - "pump": true, + "master": { + "command": "assign", + "resource_id": "EXT-100" + }, "expect": { + "response_ok": true, "case_address": "case=EXT-100@modes.example" } }, @@ -262,11 +329,10 @@ "text": "more" }, "expect": { - "response_status": "reported", "master_received": { "case.message": 1 }, - "outbox_count": 0 + "mails_sent": 0 } }, { @@ -276,16 +342,17 @@ "body": "We are on it." }, "expect": { - "command_ok": true, - "outbox_count": 1 + "response_ok": true, + "mails_sent": 1 } } ] }, { "id": "s-002", - "role": "slave", + "setup": "slave", "title": "principle 10 refuses a master send", + "scenario": "completed", "steps": [ { "mail": { @@ -295,7 +362,10 @@ } }, { - "pump": true + "master": { + "command": "assign", + "resource_id": "EXT-100" + } }, { "master": { @@ -304,31 +374,43 @@ "body": "x" }, "expect": { - "command_ok": false, - "command_reason": "unauthenticated-sender", - "outbox_count": 0 + "response_ok": false, + "response_reason": "unauthenticated-sender", + "mails_sent": 0 } } ] }, { "id": "s-003", - "role": "slave", + "setup": "slave", "title": "transfer blocked", + "scenario": "completed", "steps": [ { "mail": { "to": "case!callback", "text": "Phone: +49 30 1234567\nTime: tomorrow 10:00" + }, + "expect": { + "master_received": { + "case.intake": 1 + } } }, { - "pump": true, + "master": { + "command": "assign", + "resource_id": "EXT-100" + }, "expect": { "master_received": { "handoff.requested": 1, "handoff.status": 1 }, + "relayed": [ + "refused" + ], "mode_received": 0 } } @@ -336,14 +418,8 @@ }, { "id": "s-004", - "role": "slave", + "setup": "slave-webhook", "title": "transfer to the primed webhook; statuses relayed", - "config": { - "transfers": { - "webhook": "{callback}", - "peer": "callback-1" - } - }, "scenario": "completed", "steps": [ { @@ -353,7 +429,10 @@ } }, { - "pump": true, + "master": { + "command": "assign", + "resource_id": "EXT-100" + }, "expect": { "mode_received": 1 } @@ -366,27 +445,21 @@ "running", "completed" ], - "outbox_count": 0 + "mails_sent": 0 } }, { "advance": 1000000, "expect": { - "timed_out": 0 + "mode_received": 1 } } ] }, { "id": "s-005", - "role": "slave", + "setup": "slave-webhook", "title": "the master owns the timeout", - "config": { - "transfers": { - "webhook": "{callback}", - "peer": "callback-1" - } - }, "scenario": "silent", "steps": [ { @@ -396,7 +469,10 @@ } }, { - "pump": true + "master": { + "command": "assign", + "resource_id": "EXT-100" + } }, { "mode": "run", @@ -404,13 +480,19 @@ "relayed": [] } }, + { + "advance": 1000000, + "expect": { + "mode_received": 1 + } + }, { "master": { "command": "cancel", "handoff_id": "{handoff_id}" }, "expect": { - "command_ok": true, + "response_ok": true, "mode_received_last": "handoff.cancel" } }, @@ -425,9 +507,10 @@ ] }, { - "id": "a-001", - "role": "slave", - "title": "unsigned master command refused", + "id": "s-006", + "setup": "slave", + "title": "over the cap a master send is held", + "scenario": "completed", "steps": [ { "mail": { @@ -436,7 +519,55 @@ } }, { - "pump": true + "master": { + "command": "assign", + "resource_id": "EXT-100" + } + }, + { + "master": { + "command": "send", + "in_reply_to": "{message_ref}", + "body": "one" + }, + "expect": { + "response_ok": true, + "mails_sent": 1 + } + }, + { + "master": { + "command": "send", + "in_reply_to": "{message_ref}", + "body": "two" + }, + "expect": { + "response_ok": false, + "response_reason": "send-cap-reached", + "response_held": true, + "mails_sent": 1 + } + } + ], + "max_sent": 1 + }, + { + "id": "a-001", + "setup": "slave", + "title": "unsigned master command refused", + "scenario": "completed", + "steps": [ + { + "mail": { + "to": "case", + "text": "hi" + } + }, + { + "master": { + "command": "assign", + "resource_id": "EXT-100" + } }, { "inject": { @@ -452,23 +583,21 @@ "expect": { "http_status": 401, "error": "unauthenticated-peer", - "outbox_count": 0 + "mails_sent": 0 } } ] }, { "id": "a-002", - "role": "slave", + "setup": "slave", "title": "a mode cannot assign case ids", + "scenario": "completed", "steps": [ { "mail": { "to": "case", "text": "hi" - }, - "expect": { - "response_status": "intake-reported" } }, { @@ -487,7 +616,10 @@ } }, { - "pump": true, + "master": { + "command": "assign", + "resource_id": "EXT-100" + }, "expect": { "case_address": "case=EXT-100@modes.example" } @@ -496,8 +628,9 @@ }, { "id": "a-003", - "role": "slave", + "setup": "slave", "title": "webhook to a peer email has no key for", + "scenario": "completed", "steps": [ { "mail": { @@ -506,7 +639,10 @@ } }, { - "pump": true + "master": { + "command": "assign", + "resource_id": "EXT-100" + } }, { "inject": { @@ -530,8 +666,9 @@ }, { "id": "a-004", - "role": "slave", + "setup": "slave", "title": "replayed master command", + "scenario": "completed", "steps": [ { "mail": { @@ -540,7 +677,10 @@ } }, { - "pump": true + "master": { + "command": "assign", + "resource_id": "EXT-100" + } }, { "inject": { @@ -557,15 +697,16 @@ "expect": { "http_status": 401, "error": "replayed", - "outbox_count": 0 + "mails_sent": 0 } } ] }, { "id": "a-005", - "role": "slave", + "setup": "slave", "title": "master command signed with the wrong key", + "scenario": "completed", "steps": [ { "mail": { @@ -574,7 +715,10 @@ } }, { - "pump": true + "master": { + "command": "assign", + "resource_id": "EXT-100" + } }, { "inject": { @@ -591,7 +735,7 @@ "expect": { "http_status": 401, "error": "unauthenticated-peer", - "outbox_count": 0 + "mails_sent": 0 } }, { @@ -607,14 +751,14 @@ }, "expect": { "http_status": 200, - "outbox_count": 1 + "mails_sent": 1 } } ] }, { "id": "a-006", - "role": "master", + "setup": "master", "title": "statuses only from the handoff's peer, at its URL", "scenario": "silent", "steps": [ @@ -624,7 +768,7 @@ "text": "Phone: +49 30 1234567\nTime: tomorrow 10:00" }, "expect": { - "outbox_count": 1 + "mails_sent": 1 } }, { @@ -634,6 +778,7 @@ "message": { "type": "handoff.status", "case": "current", + "handoff_id": "{handoff_id}", "seq": 1, "status": "completed" } @@ -651,6 +796,7 @@ "message": { "type": "handoff.status", "case": "current", + "handoff_id": "{handoff_id}", "seq": 1, "status": "completed" } @@ -658,7 +804,7 @@ "expect": { "http_status": 404, "error": "unknown-handoff", - "outbox_count": 1 + "mails_sent": 1 } }, { @@ -668,6 +814,7 @@ "message": { "type": "handoff.status", "case": "current", + "handoff_id": "{handoff_id}", "seq": 1, "status": "scheduled", "at": "x" @@ -681,8 +828,9 @@ }, { "id": "a-007", - "role": "master", + "setup": "master", "title": "only the gateway posts events", + "scenario": "completed", "steps": [ { "inject": { diff --git a/mas_modes/conformance/scenarios.json b/mas_modes/conformance/scenarios.json new file mode 100644 index 0000000..caeae4b --- /dev/null +++ b/mas_modes/conformance/scenarios.json @@ -0,0 +1,120 @@ +{ + "description": "Callback stub scenarios (CONTRACT.md section 8). Each is the sequence of handoff.status messages the stub sends, one per step. A step {'wait_for_details': [fields]} sends 'negotiating' with needs = the missing fields and waits until handoff.update supplies them; it is skipped when the handoff.request already carried them. After handoff.cancel the stub sends 'cancelled' next. 'silent' never answers.", + "default": "completed", + "scenarios": { + "completed": [ + { + "status": "scheduled", + "at": "2026-10-20T10:00:00Z" + }, + { + "status": "running" + }, + { + "status": "completed" + } + ], + "negotiate": [ + { + "wait_for_details": [ + "phone", + "time" + ] + }, + { + "status": "scheduled", + "at": "2026-10-20T10:00:00Z" + }, + { + "status": "running" + }, + { + "status": "completed" + } + ], + "no-answer-retry": [ + { + "status": "scheduled", + "at": "2026-10-20T10:00:00Z" + }, + { + "status": "attempted", + "outcome": "no-answer" + }, + { + "status": "scheduled", + "at": "2026-10-20T14:00:00Z" + }, + { + "status": "running" + }, + { + "status": "completed" + } + ], + "no-answer-fail": [ + { + "status": "scheduled", + "at": "2026-10-20T10:00:00Z" + }, + { + "status": "attempted", + "outcome": "no-answer" + }, + { + "status": "scheduled", + "at": "2026-10-20T14:00:00Z" + }, + { + "status": "attempted", + "outcome": "no-answer" + }, + { + "status": "failed", + "detail": "two attempts, no answer" + } + ], + "call-failed": [ + { + "status": "scheduled", + "at": "2026-10-20T10:00:00Z" + }, + { + "status": "running" + }, + { + "status": "failed", + "detail": "line dropped" + } + ], + "expired": [ + { + "status": "scheduled", + "at": "2026-10-20T10:00:00Z" + }, + { + "status": "expired", + "detail": "slot passed" + } + ], + "unavailable": [ + { + "status": "refused", + "reason": "mode-unavailable" + } + ], + "declined": [ + { + "status": "refused", + "reason": "declined" + } + ], + "out-of-hours": [ + { + "status": "refused", + "reason": "out-of-hours" + } + ], + "silent": [] + } +} diff --git a/mas_modes/harness.py b/mas_modes/harness.py index ef19b81..444b8b7 100644 --- a/mas_modes/harness.py +++ b/mas_modes/harness.py @@ -52,11 +52,11 @@ class Harness: def __init__(self, statute_file, roles: dict, scenario: str = "completed", auto_assign: bool = True, max_sent: Optional[int] = None, ticket_ids=None, - sign_events: bool = False): + sign_events: bool = False, keys: Optional[dict] = None): self.resolver = Resolver.with_statute_files([statute_file]) [self.domain] = self.resolver.pinned self.clock = FakeClock() - self.keys = {peer: secrets.token_bytes(32) for peer in ("callback-1", "cms-1", "gateway")} + self.keys = keys or {peer: secrets.token_bytes(32) for peer in ("callback-1", "cms-1", "gateway")} self.callback = CallbackStub({"email": self.keys["callback-1"]}, default=scenario) self._servers = [] cb_url = self.callback_url = self._serve(self.callback.server()) + "/handoffs" diff --git a/mas_modes/runner.py b/mas_modes/runner.py index 3f48fe8..cf15330 100644 --- a/mas_modes/runner.py +++ b/mas_modes/runner.py @@ -16,16 +16,13 @@ import uuid from collections import Counter from dataclasses import dataclass, field from importlib import resources +from typing import Optional -from . import _load, flow_fixtures, message_errors, message_fixtures, transitions +from . import _load, message_errors, message_fixtures, transitions from .contract import allowed from .harness import Harness from .signing import AuthError, sign, signature, verify -MASTER = {"role": "master", "fields": ("phone", "time", "scenario"), "timeout": 3600} -SLAVE = {"role": "slave"} - - @dataclass class Report: passed: int = 0 @@ -107,65 +104,141 @@ def _inject(h, spec, ctx): return e.code, json.loads(e.read().decode() or "null") +STEPS = {"mail", "mode", "advance", "master", "inject"} +EXPECT = {"event_accepted", "mode_statuses", "mode_received", "mode_received_last", + "master_received", "relayed", "mails_sent", "last_mail_contains", "response_ok", + "response_reason", "response_held", "case_address", "http_status", "error"} + + +def undefined_keys(flow: dict) -> list[str]: + """Steps or expectations not defined in conformance/FLOWS.md.""" + bad = [] + for n, step in enumerate(flow["steps"], 1): + bad += [f"step {n}: {k}" for k in set(step) - STEPS - {"expect"}] + bad += [f"step {n}: expect {k}" for k in set(step.get("expect", {})) - EXPECT] + return bad + + def _fill(value, ctx): if isinstance(value, str): for k, v in ctx.items(): - value = value.replace("{" + k + "}", str(v)) + if isinstance(v, str): + value = value.replace("{" + k + "}", v) return value if isinstance(value, dict): return {k: _fill(v, ctx) for k, v in value.items()} return value -def _statute_path(): - return resources.files(__package__).joinpath("conformance/statute.json") - - -def run_flow(flow: dict) -> list[str]: - """Returns the problems found; empty when the flow passes.""" - with resources.as_file(_statute_path()) as statute: - if flow["role"] == "master": - roles = {"ticket": MASTER} +def _roles(setup: dict) -> dict: + """A flow setup (peer ids only) as harness role configs (with stub URLs).""" + def url_for(v): + if isinstance(v, str) and v.startswith(" list[str]: + """Run one flow from the bundle (FLOWS.md). Returns the problems found.""" + doc = doc or _load("conformance/flows.json") + bad = undefined_keys(flow) + if bad: + return [f"undefined key(s): {bad}"] + keys = {p: bytes.fromhex(k) for p, k in doc["peers"].items()} + with resources.as_file(resources.files(__package__).joinpath("conformance", doc["statute"])) as st: + h = Harness(st, _roles(doc["setups"][flow["setup"]]), scenario=flow["scenario"], + auto_assign=False, max_sent=flow.get("max_sent"), keys=keys) + ctx: dict = {} + problems = [] try: for n, step in enumerate(flow["steps"], 1): - resp = None + seen = (len(h.callback.received), len(h.master.received)) + event, response = None, None if "mail" in step: m = _fill(step["mail"], ctx) - d = h.mail(m["to"], m.get("text", ""), dmarc=m.get("dmarc", "pass")) - resp = d.response if isinstance(d.response, dict) else {} + event = h.mail(m["to"], m.get("text", ""), subject=m.get("subject", "Help needed"), + dmarc=m.get("dmarc", "pass")) elif "mode" in step: h.callback.run() elif "advance" in step: - timed_out = h.advance(step["advance"]) - elif "pump" in step: - results = h.master.pump() - resp = results[-1][1] if results else {} - elif "inject" in step: - last_command = _inject(h, step["inject"], ctx) + h.advance(step["advance"]) elif "master" in step: - cmd = _fill(step["master"], {**ctx, "callback": ""}) - case = {"address": ctx["case_address"], "key": ctx["case_key"]} - if cmd["command"] == "send": - last_command = h.master.send(case, cmd["in_reply_to"], cmd["body"]) - elif cmd["command"] == "cancel": - last_command = h.master.cancel(case, cmd["handoff_id"]) - if isinstance(resp, dict): - if resp.get("ticket_id"): - ctx["case_id"] = resp["ticket_id"] - if isinstance(resp.get("case"), dict): - ctx["case_id"] = resp["case"]["address"].split("=", 1)[1].split("@")[0] - ctx["case_address"], ctx["case_key"] = resp["case"]["address"], resp["case"]["key"] - for k in ("message_ref", "handoff_id", "intake_ref"): - if resp.get(k): - ctx[k] = resp[k] + response = _master(h, _fill(step["master"], ctx), ctx) + elif "inject" in step: + response = _inject(h, step["inject"], ctx) + _capture(ctx, h.callback.received[seen[0]:] + h.master.received[seen[1]:]) + if response is not None: + _capture(ctx, [response[1]]) for key, want in step.get("expect", {}).items(): - got = _observe(h, key, resp, last_command, timed_out) + got = _observe(h, key, event, response) if key == "master_received": got = {t: got.get(t, 0) for t in want} elif key == "last_mail_contains": @@ -177,36 +250,37 @@ def run_flow(flow: dict) -> list[str]: return problems -def _observe(h, key, resp, last_command, timed_out): - if key == "response_status": - return (resp or {}).get("status") - if key == "statuses": +def _observe(h, key, event, response): + body = (response[1] if response else None) or {} + if key == "event_accepted": + return event is not None and event.status is not None and 200 <= event.status < 300 + if key == "mode_statuses": return [m["status"] for hd in h.callback.handoffs.values() for m in hd.sent] - if key == "relayed": - return [m["status"] for m in h.master.of_type("handoff.status")] - if key == "outbox_count": - return len(h.outbox()) - if key == "last_mail_contains": - mails = h.outbox() - return mails[-1]["body"] if mails else None - if key == "master_received": - return dict(Counter(m["type"] for m in h.master.received)) if key == "mode_received": return len(h.callback.received) if key == "mode_received_last": return h.callback.received[-1]["type"] if h.callback.received else None - if key == "timed_out": - return len(timed_out) + if key == "master_received": + return dict(Counter(m["type"] for m in h.master.received)) + if key == "relayed": + return [m["status"] for m in h.master.of_type("handoff.status")] + if key == "mails_sent": + return len(h.outbox()) + if key == "last_mail_contains": + mails = h.outbox() + return mails[-1]["body"] if mails else None + if key == "response_ok": + return body.get("ok") + if key == "response_reason": + return body.get("reason") + if key == "response_held": + return body.get("held") if key == "case_address": - return (resp or {}).get("case", {}).get("address") - if key == "command_ok": - return (last_command[1] or {}).get("ok") - if key == "command_reason": - return (last_command[1] or {}).get("reason") + return (body.get("case") or {}).get("address") if key == "http_status": - return last_command[0] + return response[0] if response else None if key == "error": - return (last_command[1] or {}).get("error") + return body.get("error") raise KeyError(f"unknown expectation {key!r}") @@ -214,8 +288,9 @@ def run_mode_corpus() -> Report: report = Report() check_messages(report) check_auth(report) - for flow in flow_fixtures(): - problems = run_flow(flow) + doc = _load("conformance/flows.json") + for flow in doc["flows"]: + problems = run_flow(flow, doc) if problems: report.failures.append((flow["id"], "; ".join(problems))) else: diff --git a/mas_modes/stubs/callback.py b/mas_modes/stubs/callback.py index 726d01e..6a49268 100644 --- a/mas_modes/stubs/callback.py +++ b/mas_modes/stubs/callback.py @@ -18,32 +18,32 @@ import threading from dataclasses import dataclass, field from typing import Optional -from .. import message_errors +from .. import _load, message_errors from ..contract import TERMINAL, Idempotent, json_server, message, post, refuse -AT = "2026-10-20T10:00:00Z" -NEEDED = ("phone", "time") -WAIT = "wait-for-details" # negotiation: block until handoff.update supplies NEEDED +WAIT = "wait-for-details" # negotiation: block until handoff.update supplies the needed fields -# Each step is (status, extra fields), or WAIT. -SCENARIOS: dict[str, list] = { - "completed": [("scheduled", {"at": AT}), ("running", {}), ("completed", {})], - "negotiate": [WAIT, ("scheduled", {"at": AT}), ("running", {}), ("completed", {})], - "no-answer-retry": [("scheduled", {"at": AT}), ("attempted", {"outcome": "no-answer"}), - ("scheduled", {"at": "2026-10-20T14:00:00Z"}), ("running", {}), - ("completed", {})], - "no-answer-fail": [("scheduled", {"at": AT}), ("attempted", {"outcome": "no-answer"}), - ("scheduled", {"at": "2026-10-20T14:00:00Z"}), - ("attempted", {"outcome": "no-answer"}), - ("failed", {"detail": "two attempts, no answer"})], - "call-failed": [("scheduled", {"at": AT}), ("running", {}), - ("failed", {"detail": "line dropped"})], - "expired": [("scheduled", {"at": AT}), ("expired", {"detail": "slot passed"})], - "unavailable": [("refused", {"reason": "mode-unavailable"})], - "declined": [("refused", {"reason": "declined"})], - "out-of-hours": [("refused", {"reason": "out-of-hours"})], - "silent": [], # never answers: the timeout owner has to act -} + +def _scenarios() -> tuple[dict, tuple]: + """The scenarios from the bundle (conformance/scenarios.json), as steps of + (status, extra fields) or WAIT.""" + doc = _load("conformance/scenarios.json") + needed = () + out = {} + for name, steps in doc["scenarios"].items(): + script = [] + for step in steps: + if "wait_for_details" in step: + needed = tuple(step["wait_for_details"]) + script.append(WAIT) + else: + extra = {k: v for k, v in step.items() if k != "status"} + script.append((step["status"], extra)) + out[name] = script + return out, needed + + +SCENARIOS, NEEDED = _scenarios() @dataclass diff --git a/pyproject.toml b/pyproject.toml index c0e33e4..3cd9b77 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -22,7 +22,7 @@ packages = ["mas_core", "mas_conformance", "mas_cli", "mas_gateway", "mas_modes" [tool.setuptools.package-data] mas_core = ["schema/*.json"] mas_conformance = ["*.json", "statutes/*.json"] -mas_modes = ["schema/*.json", "conformance/*.json"] +mas_modes = ["schema/*.json", "conformance/*.json", "conformance/*.md"] [tool.pytest.ini_options] testpaths = ["tests"] diff --git a/tests/test_modes_conformance.py b/tests/test_modes_conformance.py index d6ebc77..89ce0c2 100644 --- a/tests/test_modes_conformance.py +++ b/tests/test_modes_conformance.py @@ -7,18 +7,30 @@ import pytest from mas_core.discovery import Response, discover from mas_gateway.core import Resolver, build_event, message_meta_from_rfc822 from mas_modes import event_errors -from mas_modes.runner import run_mode_corpus +from mas_modes import _load +from mas_modes.runner import EXPECT, STEPS, run_mode_corpus, undefined_keys ROOT = Path(__file__).resolve().parents[1] BUNDLE = ROOT / "mas-modes-v0.1-spec" VENDORED = ROOT / "mas_modes" -@pytest.mark.parametrize("rel", sorted(str(p.relative_to(BUNDLE)) for p in BUNDLE.glob("*/*.json"))) +@pytest.mark.parametrize("rel", sorted(str(p.relative_to(BUNDLE)) for p in BUNDLE.glob("*/*.*"))) def test_vendored_copy_matches_bundle(rel): assert (BUNDLE / rel).read_bytes() == (VENDORED / rel).read_bytes() +def test_flows_are_self_describing(): + doc = _load("conformance/flows.json") + scenarios = _load("conformance/scenarios.json")["scenarios"] + for flow in doc["flows"]: + assert undefined_keys(flow) == [], flow["id"] + assert flow["setup"] in doc["setups"] and flow["scenario"] in scenarios, flow["id"] + text = (BUNDLE / "conformance/FLOWS.md").read_text() + for key in STEPS | EXPECT: + assert f"`{key}`" in text or f'"{key}"' in text, f"{key} is not defined in FLOWS.md" + + def test_mode_corpus(): rep = run_mode_corpus() assert rep.ok, rep.failures diff --git a/workplans/MAS-WP-0013-harness-independent-flows.md b/workplans/MAS-WP-0013-harness-independent-flows.md index 22d4fb2..3ba188a 100644 --- a/workplans/MAS-WP-0013-harness-independent-flows.md +++ b/workplans/MAS-WP-0013-harness-independent-flows.md @@ -4,7 +4,7 @@ type: workplan title: "Mode contract flows independent of the Python harness" domain: communication repo: mail-address-semantics -status: proposed +status: finished owner: claude topic_slug: coulomb-social created: "2026-10-19" @@ -27,7 +27,7 @@ not run the flows from the bundle alone. ```task id: MAS-WP-0013-T01 -status: todo +status: done priority: medium state_hub_task_id: "95fe186c-d0af-5755-a78f-065e7da7afd1" ``` @@ -41,7 +41,7 @@ Remove any expectation that only the Python harness can observe. ```task id: MAS-WP-0013-T02 -status: todo +status: done priority: medium state_hub_task_id: "2e71569f-739a-513a-970f-0a0c867604d6" ``` @@ -49,3 +49,33 @@ state_hub_task_id: "2e71569f-739a-513a-970f-0a0c867604d6" Update `mas_modes.runner` and the stubs to read everything from the flow. Keep `mas test` green. Add a fixture-level check that the bundle's flows do not use undefined keys. + +## Outcome + +- **`conformance/FLOWS.md`** defines the format: how to set up a run (email + side, callback stub `callback-1`, master stub `cms-1`, controllable + clock, optional send cap), the five step types, captures and the 14 + expectation keys. All of it is observable at the contract boundary: + messages the stubs receive, command responses, and sent mail. +- **`flows.json`** carries everything a run needs: + - `setups` with role configs by peer id (no URLs; `` for a + primed webhook); + - fixed public test keys in `peers`, plus `statute`; + - per flow, a `scenario` and an optional `max_sent`; + - explicit `assign` steps with `EXT-100` instead of the stub's + auto-numbering. +- **Removed harness-only observables:** the ticket service's response + strings (now `event_accepted`, the 2xx) and `tick()`'s return value (now + `mode_received`, `mode_received_last` and `mails_sent`). +- **`conformance/scenarios.json`:** the callback scenarios as data. The + Python stub loads them from there. +- **CONTRACT.md §5** now defines the `case.assign` response, which flows + capture from. +- **New flow** `s-006` (a held send over the cap): 21 flows in all. +- **Runner** (`mas_modes.runner`) reads only the bundle and rejects + undefined keys. `tests/test_modes_conformance.py` checks that every flow + uses defined keys, existing setups and scenarios, and that every key is + documented in FLOWS.md. The drift test now also covers the vendored + FLOWS.md. +- **Checks.** A mutation check over 5 expectations, plus an undefined key, + was detected every time. `mas test`: mode corpus 195/195.