EMAIL-WP-0005 T05/T06: evidence realism and harness documentation
Adds harness.deliver_raw() to inject crafted .eml fixtures through the harness, so they pick up the real Received: headers an MTA adds, and harness.inject_maildir() as the byte-exact offline counterpart. tests/test_evidence_realism.py asserts all ten recognized evidence classes after passing through a real MTA, proves MTA delivery does not change which evidence is produced, and covers Maildir injection offline. Documents what no local server can honestly produce -- provider-generated DSNs, real 4xx deferral and retry, ISP feedback loops, provider suppression behavior, MX acceptance as distinct from provider acceptance -- and names SES simulator addresses as the staging path. That tier stays out of the test run because it needs real credentials. Adds docs/test-harness-tutorial.md covering the three test tiers and the start/send/scan/assert/reset walkthrough, with an explicit assertable/not-assertable list so the evidence ceiling is stated where tests get written. Adds a Maildir section to the mailbox report tutorial. Completes EMAIL-WP-0005. Suite: 85 passed with the harness up, 64 passed + 21 skipped with it down. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
e9609b4024
commit
3497ca88bf
7 changed files with 431 additions and 4 deletions
|
|
@ -4,7 +4,7 @@ type: workplan
|
|||
title: "Test Mailbox Harness for Automated Test Environments"
|
||||
domain: infotech
|
||||
repo: email-connect
|
||||
status: proposed
|
||||
status: finished
|
||||
owner: claude
|
||||
topic_slug: custodian
|
||||
created: "2026-08-14"
|
||||
|
|
@ -322,7 +322,7 @@ Two production defects surfaced, both invisible to the fixture-only suite:
|
|||
|
||||
```task
|
||||
id: EMAIL-WP-0005-T05
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "48b8eaf9-5a4b-437b-ad2c-952907340ed7"
|
||||
```
|
||||
|
|
@ -345,11 +345,26 @@ path, and the limits of the local harness are stated explicitly rather than
|
|||
papered over.
|
||||
```
|
||||
|
||||
Done 2026-08-14:
|
||||
|
||||
* `harness.deliver_raw()` injects a crafted `.eml` through the harness, so the
|
||||
message picks up real `Received:` headers on the way in.
|
||||
`harness.inject_maildir()` is the offline counterpart, byte-exact with no MTA.
|
||||
* `tests/test_evidence_realism.py`, 13 cases: all ten recognized classes are
|
||||
asserted after passing through a real MTA, plus a parity case proving MTA
|
||||
delivery does not change which evidence is produced, plus two offline Maildir
|
||||
injection cases.
|
||||
* Documented what no local server can honestly produce — provider-generated
|
||||
DSNs, real 4xx deferral and retry, ISP feedback loops, provider suppression
|
||||
behavior, MX acceptance as distinct from provider acceptance — and named SES
|
||||
simulator addresses as the staging path. That tier is deliberately not wired
|
||||
into the test run: it needs real credentials, which stay in OpenBao.
|
||||
|
||||
## T06 - Documentation and decision record
|
||||
|
||||
```task
|
||||
id: EMAIL-WP-0005-T06
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "0599de98-c0ff-4160-a2c8-54265b31ca5d"
|
||||
```
|
||||
|
|
@ -371,6 +386,21 @@ A new contributor can start the harness, run the integration tier, and
|
|||
correctly state what harness-observed mail does and does not prove.
|
||||
```
|
||||
|
||||
Done 2026-08-14:
|
||||
|
||||
* `docs/test-harness-tutorial.md`: three test tiers, start, mailbox, send, scan,
|
||||
assert, reset, fixture injection, teardown. Includes an explicit
|
||||
assertable/not-assertable list, so the evidence ceiling is stated where tests
|
||||
get written rather than only in the canon.
|
||||
* `docs/mailbox-report-tutorial.md` gained a Maildir section covering
|
||||
`protocol: maildir`, `source.maildir_dir`, identity across `new/`→`cur/`, and
|
||||
the read-only guarantees.
|
||||
* `tests/harness/README.md` carries the reference material: ports, addressing
|
||||
convention, reset contract, skip gating, injection, and the
|
||||
provider-simulator table.
|
||||
* Decisions were recorded as the work happened: s/qmail rejection and the
|
||||
GreenMail plus loopback-plaintext choice are both in `DECISIONS.md`.
|
||||
|
||||
## 8. Completion Criteria
|
||||
|
||||
This workplan is complete when:
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue