email-connect/docs/test-harness-tutorial.md

# Test Harness Tutorial

How to run `email-connect` against a local mail server: start it, send through
it, scan it back, assert on the result, and reset between tests.

Reference material lives in `tests/harness/README.md`. This page is the
walkthrough.

## 1. The three test tiers

`email-connect` tests at three levels, and it matters which one a given
assertion belongs to.

| Tier | Source | Needs | What it is for |
| --- | --- | --- | --- |
| Unit | `.eml` fixtures, temporary Maildir trees | nothing | Parsing, classification, reporting. Deterministic and offline. |
| Integration | local GreenMail harness | Docker | Send-to-scan continuity through a real SMTP/IMAP server. |
| Provider staging | real provider account | credentials, operator | Behavior only a real provider and remote MX produce. |

The default `pytest` run is the unit tier plus any integration tests whose
harness happens to be up. Nothing in the default run requires a container: with
the harness down the integration cases skip.

The staging tier is not wired into the test run at all. It needs real
credentials, which are routed through OpenBao — see
`.claude/rules/credential-routing.md` — and must never be pointed at the
harness.

## 2. Start the harness

```bash
docker compose -f tests/harness/docker-compose.yml up -d
docker inspect --format '{{.State.Health.Status}}' email-connect-harness
```

Wait for `healthy`. The container serves SMTP on 3025 and IMAP on 3143, both
bound to `127.0.0.1` only.

## 3. Give the test a mailbox

Authentication is disabled and mailboxes are created on first login, so there is
no provisioning step — pick an address and use it:

```python
import harness

recipient = harness.address("invitation delivered")
# -> invitation-delivered@harness.email-connect.test
```

Slugging the test's own name keeps two tests from colliding. The domain is under
`.test`, reserved by RFC 2606, so a stray send cannot leave the host.

## 4. Send through the service

```python
from email_connect.transactional import SQLiteDeliveryStore, TransactionalApplication

app = TransactionalApplication(
    SQLiteDeliveryStore(str(tmp_path / "mail.db")),
    harness.smtp_provider(),
    "harness-ingest-token",
    "https://users.harness.email-connect.test",
)
```

`harness.smtp_provider()` returns a real `SMTPProvider` in plaintext transport
mode. GreenMail offers no STARTTLS, and `SMTPProvider` permits plaintext only
for loopback hosts, so this cannot be pointed at a real provider by accident.

## 5. Scan it back

```python
from email_connect.scanner import scan_mailbox

config = harness.mailbox_config(
    recipient,
    storage_path=str(tmp_path / "state.sqlite"),
    reports_dir=str(tmp_path / "reports"),
)
result = scan_mailbox(config)
```

From the command line:

```bash
export EMAIL_CONNECT_IMAP_USER=invitation-delivered@harness.email-connect.test
export EMAIL_CONNECT_IMAP_PASSWORD=harness
email-connect scan-mailbox --config config/harness-imap.yml --out reports/
```

## 6. Assert — and what you may not assert

An accepted send is `provider_accepted`. A message sitting in a harness mailbox
is a message sitting in a harness mailbox. Neither is evidence of inbox
placement, recipient awareness, identity, or authorization, and a test that
asserts otherwise is wrong even when it passes.

Assertable:

```python
assert body["evidence_ceiling"] == "provider_accepted"
assert body["reference"] == str(messages[0]["Message-ID"]).strip()
assert len(delivered_messages(recipient)) == 1
```

Not assertable from harness mail — these are the claims the repo exists to
avoid:

```text
the recipient received it
the recipient read it
the recipient's identity is confirmed
the recipient is authorized
the intended result was satisfied
```

`coordination-engine` evaluates results. `email-connect` reports what the email
channel observed.

## 7. Reset between tests

```python
def test_something():
    harness.reset()
    ...
```

`harness.reset()` drops every user and message; the next login recreates the
mailbox empty. Call it at the **start** of a test so a crashed or interrupted
run cannot leave state for the next one. It is global, so tests that reset
cannot run in parallel against a single harness.

## 8. Bounces, complaints and deferrals

The harness accepts everything, so it never produces these on its own. Inject a
crafted fixture instead:

```python
harness.deliver_raw(recipient, (FIXTURES / "hard_bounce.eml").read_bytes())
```

Injected messages still travel through a real MTA and pick up real `Received:`
headers — which is how `EMAIL-WP-0005-T04` caught a reply heuristic that was
matching those headers and labelling ordinary mail a human reply.

`harness.inject_maildir()` is the offline counterpart, writing straight into a
Maildir tree with no MTA and no harness.

Provider-generated DSNs, real deferral and retry, ISP feedback loops, and
provider suppression behavior are **not** reachable locally. See the
provider-simulator table in `tests/harness/README.md`.

## 9. Stop the harness

```bash
docker compose -f tests/harness/docker-compose.yml down -v
```

Then confirm the default suite is still offline and green:

```bash
pytest tests/ -q     # integration cases skip
```