EMAIL-WP-0005 T05/T06: evidence realism and harness documentation

Adds harness.deliver_raw() to inject crafted .eml fixtures through the harness,
so they pick up the real Received: headers an MTA adds, and
harness.inject_maildir() as the byte-exact offline counterpart.

tests/test_evidence_realism.py asserts all ten recognized evidence classes
after passing through a real MTA, proves MTA delivery does not change which
evidence is produced, and covers Maildir injection offline.

Documents what no local server can honestly produce -- provider-generated DSNs,
real 4xx deferral and retry, ISP feedback loops, provider suppression behavior,
MX acceptance as distinct from provider acceptance -- and names SES simulator
addresses as the staging path. That tier stays out of the test run because it
needs real credentials.

Adds docs/test-harness-tutorial.md covering the three test tiers and the
start/send/scan/assert/reset walkthrough, with an explicit
assertable/not-assertable list so the evidence ceiling is stated where tests
get written. Adds a Maildir section to the mailbox report tutorial.

Completes EMAIL-WP-0005. Suite: 85 passed with the harness up, 64 passed +
21 skipped with it down.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

WORK-RECORDS.md

@@ -43,6 +43,6 @@
 | task | EMAIL-WP-0005-T01 | done | — | workplans/EMAIL-WP-0005-test-mailbox-harness.md |
 | task | EMAIL-WP-0005-T02 | done | — | workplans/EMAIL-WP-0005-test-mailbox-harness.md |
 | task | EMAIL-WP-0005-T03 | done | — | workplans/EMAIL-WP-0005-test-mailbox-harness.md |
-| task | EMAIL-WP-0005-T04 | todo | — | workplans/EMAIL-WP-0005-test-mailbox-harness.md |
+| task | EMAIL-WP-0005-T04 | done | — | workplans/EMAIL-WP-0005-test-mailbox-harness.md |
 | task | EMAIL-WP-0005-T05 | todo | — | workplans/EMAIL-WP-0005-test-mailbox-harness.md |
 | task | EMAIL-WP-0005-T06 | todo | — | workplans/EMAIL-WP-0005-test-mailbox-harness.md |

docs/mailbox-report-tutorial.md

@@ -50,6 +50,32 @@ export EMAIL_CONNECT_IMAP_PASSWORD='app-password'
 IMAP scans select the folder read-only and fetch messages with `BODY.PEEK[]`.
 The scanner does not mark messages seen, move messages, or delete messages.
 
+## 3b. Scan A Maildir Instead
+
+When mail is delivered locally — an MTA writing to disk, or the test harness —
+point the scanner at the Maildir root instead of an IMAP server:
+
+```yaml
+mailbox:
+  protocol: maildir
+  folder: INBOX
+
+source:
+  maildir_dir: /var/mail/returns/Maildir
+```
+
+The directory is the mailbox root containing `new/` and `cur/`; both are read.
+No credentials are involved. Message identity is the Maildir unique name without
+its `:2,FLAGS` suffix, so a message stays the same message when the MTA or a
+mail client moves it from `new/` to `cur/`.
+
+Like IMAP, Maildir scans are read-only: `mark_seen` is rejected, and the scanner
+never moves, flags, or deletes a message. `include_seen: false` skips messages
+already flagged `S`.
+
+For a local Maildir or IMAP server to test against, see
+`docs/test-harness-tutorial.md`.
+
 ## 4. Add Expected Recipients
 
 Expected recipients are optional. A newline-separated file can look like:

docs/test-harness-tutorial.md

@@ -0,0 +1,164 @@
+# Test Harness Tutorial
+
+How to run `email-connect` against a local mail server: start it, send through
+it, scan it back, assert on the result, and reset between tests.
+
+Reference material lives in `tests/harness/README.md`. This page is the
+walkthrough.
+
+## 1. The three test tiers
+
+`email-connect` tests at three levels, and it matters which one a given
+assertion belongs to.
+
+| Tier | Source | Needs | What it is for |
+| --- | --- | --- | --- |
+| Unit | `.eml` fixtures, temporary Maildir trees | nothing | Parsing, classification, reporting. Deterministic and offline. |
+| Integration | local GreenMail harness | Docker | Send-to-scan continuity through a real SMTP/IMAP server. |
+| Provider staging | real provider account | credentials, operator | Behavior only a real provider and remote MX produce. |
+
+The default `pytest` run is the unit tier plus any integration tests whose
+harness happens to be up. Nothing in the default run requires a container: with
+the harness down the integration cases skip.
+
+The staging tier is not wired into the test run at all. It needs real
+credentials, which are routed through OpenBao — see
+`.claude/rules/credential-routing.md` — and must never be pointed at the
+harness.
+
+## 2. Start the harness
+
+```bash
+docker compose -f tests/harness/docker-compose.yml up -d
+docker inspect --format '{{.State.Health.Status}}' email-connect-harness
+```
+
+Wait for `healthy`. The container serves SMTP on 3025 and IMAP on 3143, both
+bound to `127.0.0.1` only.
+
+## 3. Give the test a mailbox
+
+Authentication is disabled and mailboxes are created on first login, so there is
+no provisioning step — pick an address and use it:
+
+```python
+import harness
+
+recipient = harness.address("invitation delivered")
+# -> invitation-delivered@harness.email-connect.test
+```
+
+Slugging the test's own name keeps two tests from colliding. The domain is under
+`.test`, reserved by RFC 2606, so a stray send cannot leave the host.
+
+## 4. Send through the service
+
+```python
+from email_connect.transactional import SQLiteDeliveryStore, TransactionalApplication
+
+app = TransactionalApplication(
+    SQLiteDeliveryStore(str(tmp_path / "mail.db")),
+    harness.smtp_provider(),
+    "harness-ingest-token",
+    "https://users.harness.email-connect.test",
+)
+```
+
+`harness.smtp_provider()` returns a real `SMTPProvider` in plaintext transport
+mode. GreenMail offers no STARTTLS, and `SMTPProvider` permits plaintext only
+for loopback hosts, so this cannot be pointed at a real provider by accident.
+
+## 5. Scan it back
+
+```python
+from email_connect.scanner import scan_mailbox
+
+config = harness.mailbox_config(
+    recipient,
+    storage_path=str(tmp_path / "state.sqlite"),
+    reports_dir=str(tmp_path / "reports"),
+)
+result = scan_mailbox(config)
+```
+
+From the command line:
+
+```bash
+export EMAIL_CONNECT_IMAP_USER=invitation-delivered@harness.email-connect.test
+export EMAIL_CONNECT_IMAP_PASSWORD=harness
+email-connect scan-mailbox --config config/harness-imap.yml --out reports/
+```
+
+## 6. Assert — and what you may not assert
+
+An accepted send is `provider_accepted`. A message sitting in a harness mailbox
+is a message sitting in a harness mailbox. Neither is evidence of inbox
+placement, recipient awareness, identity, or authorization, and a test that
+asserts otherwise is wrong even when it passes.
+
+Assertable:
+
+```python
+assert body["evidence_ceiling"] == "provider_accepted"
+assert body["reference"] == str(messages[0]["Message-ID"]).strip()
+assert len(delivered_messages(recipient)) == 1
+```
+
+Not assertable from harness mail — these are the claims the repo exists to
+avoid:
+
+```text
+the recipient received it
+the recipient read it
+the recipient's identity is confirmed
+the recipient is authorized
+the intended result was satisfied
+```
+
+`coordination-engine` evaluates results. `email-connect` reports what the email
+channel observed.
+
+## 7. Reset between tests
+
+```python
+def test_something():
+    harness.reset()
+    ...
+```
+
+`harness.reset()` drops every user and message; the next login recreates the
+mailbox empty. Call it at the **start** of a test so a crashed or interrupted
+run cannot leave state for the next one. It is global, so tests that reset
+cannot run in parallel against a single harness.
+
+## 8. Bounces, complaints and deferrals
+
+The harness accepts everything, so it never produces these on its own. Inject a
+crafted fixture instead:
+
+```python
+harness.deliver_raw(recipient, (FIXTURES / "hard_bounce.eml").read_bytes())
+```
+
+Injected messages still travel through a real MTA and pick up real `Received:`
+headers — which is how `EMAIL-WP-0005-T04` caught a reply heuristic that was
+matching those headers and labelling ordinary mail a human reply.
+
+`harness.inject_maildir()` is the offline counterpart, writing straight into a
+Maildir tree with no MTA and no harness.
+
+Provider-generated DSNs, real deferral and retry, ISP feedback loops, and
+provider suppression behavior are **not** reachable locally. See the
+provider-simulator table in `tests/harness/README.md`.
+
+## 9. Stop the harness
+
+```bash
+docker compose -f tests/harness/docker-compose.yml down -v
+```
+
+Then confirm the default suite is still offline and green:
+
+```bash
+pytest tests/ -q     # integration cases skip
+```

tests/harness/README.md

@@ -122,6 +122,50 @@ export EMAIL_CONNECT_SENDER=noreply@harness.email-connect.test
 for any non-loopback host, so this setting cannot weaken a real deployment: it
 fails at startup rather than sending credentials in the clear.
 
+## Exercising bounces, complaints and deferrals
+
+The harness accepts every message, so it cannot *generate* a bounce, complaint,
+or deferral. Those classes are exercised by injecting the crafted fixtures in
+`tests/fixtures/mailbox/` and letting them travel through the harness:
+
+```python
+harness.deliver_raw(recipient, (FIXTURES / "hard_bounce.eml").read_bytes())
+```
+
+This matters more than reading the fixture off disk. Delivery adds the real
+`Received:` headers an MTA-handled message carries — the exact difference that
+hid an overclaiming reply heuristic until `EMAIL-WP-0005-T04`.
+
+`harness.inject_maildir(maildir_dir, raw_bytes)` is the offline counterpart: it
+writes the message straight into a Maildir tree, byte for byte, with no MTA
+involved and no harness required.
+
+`tests/test_evidence_realism.py` runs every recognized class both ways and
+asserts the two agree.
+
+## What is only reachable in a provider staging tier
+
+Some behavior no local server can honestly reproduce, because it belongs to a
+real provider and a real remote MX:
+
+| Not reachable locally | Why |
+| --- | --- |
+| Provider-generated DSNs | Local injection uses a DSN we wrote ourselves |
+| Real 4xx deferral and retry over time | The harness never defers or retries |
+| ISP complaint feedback loops (ARF) | Needs a real feedback-loop subscription |
+| Provider suppression-list behavior | Belongs to the provider account |
+| MX acceptance as distinct from provider acceptance | Needs a real remote MX |
+
+For those, use a provider staging tier with simulator addresses rather than
+inventing local equivalents. On SES, for example, `bounce@simulator.amazonses.com`,
+`complaint@simulator.amazonses.com`, `ooto@simulator.amazonses.com` and
+`suppressionlist@simulator.amazonses.com` drive the corresponding paths without
+touching a real recipient.
+
+That tier is deliberately **not** wired into this repo's test run: it needs real
+credentials, which are routed through OpenBao and must never reach the harness.
+Treat it as an operator-run check against a staging provider account.
+
 ## What harness mail proves
 
 Nothing beyond `provider_accepted`. A message sitting in a GreenMail mailbox is

tests/harness/__init__.py

@@ -104,6 +104,40 @@ def smtp_provider(sender: str = DEFAULT_SENDER):
     return SMTPProvider(HOST, SMTP_PORT, sender, PASSWORD, sender, security="plaintext")
 
 
+def deliver_raw(recipient: str, raw_bytes: bytes, sender: str = DEFAULT_SENDER) -> None:
+    """Deliver a crafted message verbatim to a harness mailbox.
+
+    The harness accepts everything, so it cannot *generate* a bounce, complaint,
+    or deferral. Injecting a crafted `.eml` is how those classes are exercised:
+    the message still travels through a real MTA and picks up real `Received:`
+    headers, which a fixture read off disk never does.
+    """
+
+    import smtplib
+
+    with smtplib.SMTP(HOST, SMTP_PORT, timeout=10) as smtp:
+        smtp.sendmail(sender, [recipient], raw_bytes)
+
+
+def inject_maildir(maildir_dir, raw_bytes: bytes, *, name: str | None = None, subdir: str = "new"):
+    """Write a crafted message straight into a Maildir tree.
+
+    The offline counterpart to `deliver_raw`: no MTA involved, so the message
+    arrives byte-for-byte with no added headers.
+    """
+
+    import time
+    from pathlib import Path
+
+    root = Path(maildir_dir)
+    for required in ("new", "cur", "tmp"):
+        (root / required).mkdir(parents=True, exist_ok=True)
+    filename = name or f"{int(time.time())}.M{len(raw_bytes)}P{id(raw_bytes) % 100000}.harness"
+    path = root / subdir / filename
+    path.write_bytes(raw_bytes)
+    return path
+
+
 def mailbox_config(
     user_address: str,
     *,

tests/test_evidence_realism.py

@@ -0,0 +1,129 @@
+"""Every recognized evidence class, exercised through a real MTA.
+
+The harness accepts everything, so it cannot generate a bounce, complaint, or
+deferral on its own. Those classes are exercised by injecting the crafted
+fixtures and letting them travel through GreenMail, which adds the real
+`Received:` headers that a fixture read off disk never has — the difference that
+hid the reply-heuristic overclaim until EMAIL-WP-0005-T04.
+
+Classes that no local server can produce are listed in
+`tests/harness/README.md` under the provider-simulator tier.
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import pytest
+
+import harness
+from email_connect.config import AppConfig, MailboxConfig, ReportsConfig, ScanConfig, SourceConfig, StorageConfig
+from email_connect.scanner import scan_mailbox
+
+FIXTURES = Path(__file__).parent / "fixtures" / "mailbox"
+
+requires_harness = pytest.mark.skipif(
+    not harness.available(),
+    reason="mail harness not running: docker compose -f tests/harness/docker-compose.yml up -d",
+)
+
+#: The evidence each crafted fixture must still yield after passing through an
+#: MTA. Anything absent here is unreachable locally, not merely untested.
+EXPECTED_EVENTS = {
+    "hard_bounce.eml": "notification.endpoint.rejected_permanent",
+    "soft_bounce.eml": "notification.endpoint.rejected_temporary",
+    "delayed_delivery.eml": "notification.endpoint.deferred",
+    "complaint.eml": "notification.channel.complaint_received",
+    "unsubscribe.eml": "notification.channel.unsubscribe_received",
+    "out_of_office.eml": "interaction.out_of_office_received",
+    "challenge_response.eml": "interaction.unverified_actor_interaction",
+    "human_reply.eml": "interaction.reply_received",
+    "final_failure.eml": "notification.endpoint.rejected_permanent",
+    "unknown_return.eml": "notification.endpoint.unknown",
+}
+
+
+def scan_events(config) -> set[str]:
+    result = scan_mailbox(config, full_rescan=True)
+    rows = result.report_path.read_text(encoding="utf-8").splitlines()
+    header = rows[0].split(",")
+    index = header.index("normalized_event_type")
+    return {line.split(",")[index] for line in rows[1:] if line.strip()}
+
+
+@pytest.mark.parametrize(("fixture", "expected"), sorted(EXPECTED_EVENTS.items()))
+@requires_harness
+def test_crafted_evidence_survives_a_real_mta(fixture, expected, tmp_path):
+    harness.reset()
+    recipient = harness.address(f"realism {Path(fixture).stem}")
+    harness.deliver_raw(recipient, (FIXTURES / fixture).read_bytes())
+
+    config = harness.mailbox_config(
+        recipient,
+        storage_path=str(tmp_path / "state.sqlite"),
+        reports_dir=str(tmp_path / "reports"),
+    )
+
+    assert expected in scan_events(config)
+
+
+@requires_harness
+def test_injected_fixtures_match_the_offline_fixture_scan(tmp_path):
+    """Delivery through an MTA must not change which evidence is produced."""
+
+    harness.reset()
+    recipient = harness.address("realism parity")
+    for fixture in EXPECTED_EVENTS:
+        harness.deliver_raw(recipient, (FIXTURES / fixture).read_bytes())
+
+    delivered = scan_events(
+        harness.mailbox_config(
+            recipient,
+            storage_path=str(tmp_path / "delivered.sqlite"),
+            reports_dir=str(tmp_path / "delivered-reports"),
+        )
+    )
+
+    offline_dir = tmp_path / "offline-fixtures"
+    offline_dir.mkdir()
+    for fixture in EXPECTED_EVENTS:
+        (offline_dir / fixture).write_bytes((FIXTURES / fixture).read_bytes())
+    offline = scan_events(
+        AppConfig(
+            mailbox=MailboxConfig(id="offline", protocol="fixture"),
+            scan=ScanConfig(),
+            storage=StorageConfig(path=str(tmp_path / "offline.sqlite")),
+            reports=ReportsConfig(output_dir=str(tmp_path / "offline-reports")),
+            source=SourceConfig(fixture_dir=str(offline_dir)),
+        )
+    )
+
+    assert delivered == offline
+
+
+def test_maildir_injection_is_byte_exact(tmp_path):
+    """The offline injection path adds nothing to the message."""
+
+    raw = (FIXTURES / "hard_bounce.eml").read_bytes()
+    path = harness.inject_maildir(tmp_path / "Maildir", raw)
+
+    assert path.read_bytes() == raw
+    assert path.parent.name == "new"
+
+
+def test_maildir_injection_feeds_the_scanner(tmp_path):
+    maildir = tmp_path / "Maildir"
+    harness.inject_maildir(maildir, (FIXTURES / "hard_bounce.eml").read_bytes(), name="1749000001.M1P1.harness")
+    harness.inject_maildir(maildir, (FIXTURES / "complaint.eml").read_bytes(), name="1749000002.M2P2.harness")
+
+    config = AppConfig(
+        mailbox=MailboxConfig(id="injected", protocol="maildir"),
+        scan=ScanConfig(),
+        storage=StorageConfig(path=str(tmp_path / "state.sqlite")),
+        reports=ReportsConfig(output_dir=str(tmp_path / "reports")),
+        source=SourceConfig(maildir_dir=str(maildir)),
+    )
+    events = scan_events(config)
+
+    assert "notification.endpoint.rejected_permanent" in events
+    assert "notification.channel.complaint_received" in events

workplans/EMAIL-WP-0005-test-mailbox-harness.md

@@ -4,7 +4,7 @@ type: workplan
 title: "Test Mailbox Harness for Automated Test Environments"
 domain: infotech
 repo: email-connect
-status: proposed
+status: finished
 owner: claude
 topic_slug: custodian
 created: "2026-08-14"
@@ -322,7 +322,7 @@ Two production defects surfaced, both invisible to the fixture-only suite:
 
 ```task
 id: EMAIL-WP-0005-T05
-status: todo
+status: done
 priority: medium
 state_hub_task_id: "48b8eaf9-5a4b-437b-ad2c-952907340ed7"
 ```
@@ -345,11 +345,26 @@ path, and the limits of the local harness are stated explicitly rather than
 papered over.
 ```
 
+Done 2026-08-14:
+
+* `harness.deliver_raw()` injects a crafted `.eml` through the harness, so the
+  message picks up real `Received:` headers on the way in.
+  `harness.inject_maildir()` is the offline counterpart, byte-exact with no MTA.
+* `tests/test_evidence_realism.py`, 13 cases: all ten recognized classes are
+  asserted after passing through a real MTA, plus a parity case proving MTA
+  delivery does not change which evidence is produced, plus two offline Maildir
+  injection cases.
+* Documented what no local server can honestly produce — provider-generated
+  DSNs, real 4xx deferral and retry, ISP feedback loops, provider suppression
+  behavior, MX acceptance as distinct from provider acceptance — and named SES
+  simulator addresses as the staging path. That tier is deliberately not wired
+  into the test run: it needs real credentials, which stay in OpenBao.
+
 ## T06 - Documentation and decision record
 
 ```task
 id: EMAIL-WP-0005-T06
-status: todo
+status: done
 priority: medium
 state_hub_task_id: "0599de98-c0ff-4160-a2c8-54265b31ca5d"
 ```
@@ -371,6 +386,21 @@ A new contributor can start the harness, run the integration tier, and
 correctly state what harness-observed mail does and does not prove.
 ```
 
+Done 2026-08-14:
+
+* `docs/test-harness-tutorial.md`: three test tiers, start, mailbox, send, scan,
+  assert, reset, fixture injection, teardown. Includes an explicit
+  assertable/not-assertable list, so the evidence ceiling is stated where tests
+  get written rather than only in the canon.
+* `docs/mailbox-report-tutorial.md` gained a Maildir section covering
+  `protocol: maildir`, `source.maildir_dir`, identity across `new/`→`cur/`, and
+  the read-only guarantees.
+* `tests/harness/README.md` carries the reference material: ports, addressing
+  convention, reset contract, skip gating, injection, and the
+  provider-simulator table.
+* Decisions were recorded as the work happened: s/qmail rejection and the
+  GreenMail plus loopback-plaintext choice are both in `DECISIONS.md`.
+
 ## 8. Completion Criteria
 
 This workplan is complete when: