EMAIL-WP-0005-T02: add GreenMail test harness
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s

Adds tests/harness/docker-compose.yml running GreenMail 2.1.12 (digest-pinned)
with SMTP 3025, IMAP 3143 and the API bound to 127.0.0.1 only, plus a
config/harness-imap.yml scanner profile and harness README. Auth is disabled
and no users are declared, so a mailbox is created on first login and per-test
users need no provisioning.

GreenMail standalone offers no STARTTLS, only plaintext or implicit TLS, while
SMTPProvider hardcoded starttls() -- so no send could reach it. SMTPProvider
now takes a security mode via EMAIL_CONNECT_SMTP_SECURITY, defaulting to
starttls. plaintext is refused for any non-loopback host, and hostnames are
never resolved to decide that, so a misconfigured deployment fails at startup
rather than sending credentials in the clear. Trusting GreenMail's self-signed
cert was rejected as the wider risk; see DECISIONS.md.

Verified end to end against the live harness: SMTPProvider.send -> GreenMail ->
ImapMailboxSource, and the documented scan-mailbox CLI. Suite: 52 passed with
the harness down.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-08-14 01:41:38 +02:00
parent 86f22c2e65
commit 89fd13ac2d
8 changed files with 329 additions and 4 deletions

View file

@ -160,3 +160,31 @@ local server; those classes stay on crafted fixtures plus an optional provider
simulator tier.
---
## Test harness mail server: GreenMail, and plaintext SMTP for loopback only
**Date:** 2026-08-14
**Decided by:** codex (EMAIL-WP-0005-T02)
GreenMail `2.1.12` (digest-pinned) is the harness mail server. It serves SMTP
and IMAP from one container and, with `greenmail.auth.disabled`, creates a
mailbox on first login — so per-test users need no provisioning step. Mailpit
was the alternative but offers no IMAP, which would leave the scanner's IMAP
source untested.
GreenMail standalone supports only plaintext or implicit-TLS setups; it has no
STARTTLS option (verified against the shipped jar's property builder). The
transactional service previously hardcoded `starttls()`, so no send could reach
it.
`SMTPProvider` therefore takes a `security` mode, `EMAIL_CONNECT_SMTP_SECURITY`,
defaulting to `starttls`. `plaintext` is refused for any non-loopback host, and
hostnames are never resolved to decide this — a name pointing at 127.0.0.1 today
is not a durable guarantee. A misconfigured deployment fails at startup instead
of putting credentials on the wire.
The alternative was trusting GreenMail's self-signed certificate for implicit
TLS, which would have meant shipping certificate-verification bypass code in the
send path. A loopback-guarded plaintext mode is the narrower risk.
---

View file

@ -40,7 +40,7 @@
| task | EMAIL-WP-0004-T02 | done | — | workplans/EMAIL-WP-0004-transactional-mail-delivery-service.md |
| task | EMAIL-WP-0004-T03 | done | — | workplans/EMAIL-WP-0004-transactional-mail-delivery-service.md |
| task | EMAIL-WP-0004-T04 | done | — | workplans/EMAIL-WP-0004-transactional-mail-delivery-service.md |
| task | EMAIL-WP-0005-T01 | todo | — | workplans/EMAIL-WP-0005-test-mailbox-harness.md |
| task | EMAIL-WP-0005-T01 | done | — | workplans/EMAIL-WP-0005-test-mailbox-harness.md |
| task | EMAIL-WP-0005-T02 | todo | — | workplans/EMAIL-WP-0005-test-mailbox-harness.md |
| task | EMAIL-WP-0005-T03 | todo | — | workplans/EMAIL-WP-0005-test-mailbox-harness.md |
| task | EMAIL-WP-0005-T04 | todo | — | workplans/EMAIL-WP-0005-test-mailbox-harness.md |

46
config/harness-imap.yml Normal file
View file

@ -0,0 +1,46 @@
# Scanner profile for the local GreenMail test harness (EMAIL-WP-0005-T02).
#
# Start the harness first:
# docker compose -f tests/harness/docker-compose.yml up -d
#
# The harness runs with authentication disabled and creates a mailbox on first
# login, so these credentials are placeholders that only need to be set:
# export EMAIL_CONNECT_IMAP_USER=t01@harness.email-connect.test
# export EMAIL_CONNECT_IMAP_PASSWORD=harness
#
# Then:
# email-connect scan-mailbox --config config/harness-imap.yml --out reports/
mailbox:
id: harness-mailbox
protocol: imap
host: 127.0.0.1
port: 3143
tls: false
username_env: EMAIL_CONNECT_IMAP_USER
password_env: EMAIL_CONNECT_IMAP_PASSWORD
folder: INBOX
scan:
mode: incremental
max_messages_per_run: 5000
since: null
from: null
to: null
include_seen: true
mark_seen: false
store_raw_headers: true
store_raw_body: false
store_raw_message_ref: true
expected_recipients:
path: null
csv_column: email
storage:
path: .email-connect/harness-state.sqlite
reports:
output_dir: reports
include_all_evidence: true
include_unknown_messages: true
timestamp_timezone: UTC

View file

@ -4,6 +4,7 @@ from __future__ import annotations
import hmac
import hashlib
import ipaddress
import json
import os
import smtplib
@ -148,7 +149,25 @@ class SQLiteDeliveryStore:
class SMTPProvider:
def __init__(self, host: str, port: int, username: str, password: str, sender: str) -> None:
#: STARTTLS is the only transport allowed against a remote provider.
#: `plaintext` exists for loopback test servers that cannot offer TLS and is
#: refused for any other host, so a misconfiguration cannot put credentials
#: on the wire in the clear.
SECURITY_MODES = ("starttls", "plaintext")
def __init__(
self,
host: str,
port: int,
username: str,
password: str,
sender: str,
security: str = "starttls",
) -> None:
if security not in self.SECURITY_MODES:
raise ValueError(f"Unsupported SMTP security mode: {security}")
if security == "plaintext" and not _is_loopback_host(host):
raise ValueError("Plaintext SMTP is only permitted for loopback hosts.")
self.host, self.port, self.username, self.password, self.sender = (
host,
port,
@ -156,6 +175,7 @@ class SMTPProvider:
password,
sender,
)
self.security = security
def send(self, recipient: str, subject: str, text: str) -> str:
message = EmailMessage()
@ -163,7 +183,8 @@ class SMTPProvider:
message.set_content(text)
try:
with smtplib.SMTP(self.host, self.port, timeout=10) as smtp:
smtp.starttls()
if self.security == "starttls":
smtp.starttls()
smtp.login(self.username, self.password)
smtp.send_message(message)
except TimeoutError as exc:
@ -352,6 +373,18 @@ class TransactionalApplication:
return [body]
def _is_loopback_host(host: str) -> bool:
candidate = (host or "").strip().strip("[]")
if candidate.lower() == "localhost":
return True
try:
return ipaddress.ip_address(candidate).is_loopback
except ValueError:
# Never resolve names here: a hostname that happens to point at 127.0.0.1
# today is not a durable guarantee that credentials stay on the host.
return False
def _valid_address(value: str) -> bool:
_, address = parseaddr(value)
return address == value and "@" in address and "\n" not in address and "\r" not in address
@ -364,6 +397,7 @@ def main() -> None:
os.environ["EMAIL_CONNECT_SMTP_USERNAME"],
os.environ["EMAIL_CONNECT_SMTP_PASSWORD"],
os.environ["EMAIL_CONNECT_SENDER"],
security=os.environ.get("EMAIL_CONNECT_SMTP_SECURITY", "starttls"),
)
app = TransactionalApplication(
SQLiteDeliveryStore(os.environ.get("EMAIL_CONNECT_DATABASE_PATH", "/data/email-connect.db")),

86
tests/harness/README.md Normal file
View file

@ -0,0 +1,86 @@
# Local mail test harness
A single GreenMail container serving SMTP and IMAP on loopback, so integration
tests can drive the transactional send path and read the result back through the
scanner without a live provider.
Part of `EMAIL-WP-0005`. Unit tests do not need this harness — they run against
fixture directories and temporary Maildir trees and stay offline.
## Start and stop
```bash
docker compose -f tests/harness/docker-compose.yml up -d
docker compose -f tests/harness/docker-compose.yml down -v
```
Readiness (the container healthcheck probes both mail ports; the API also
answers):
```bash
docker inspect --format '{{.State.Health.Status}}' email-connect-harness
curl -s http://127.0.0.1:8080/api/service/readiness
```
## Ports
| Port | Protocol | Notes |
| --- | --- | --- |
| 3025 | SMTP | plaintext, no STARTTLS |
| 3143 | IMAP | plaintext |
| 8080 | HTTP | GreenMail API — readiness, user and mail management |
All three bind `127.0.0.1` only. The harness accepts any credentials and must
never be exposed beyond the host.
## Accounts
The harness runs with `greenmail.auth.disabled`, so **any** login is accepted and
the mailbox is created on first use. No provisioning step is needed: pick an
address under `harness.email-connect.test` and log in.
```bash
export EMAIL_CONNECT_IMAP_USER=t01@harness.email-connect.test
export EMAIL_CONNECT_IMAP_PASSWORD=harness # ignored, but must be set
```
These credentials are deliberately non-secret and belong in the repo. Real
provider material is routed through OpenBao — see
`.claude/rules/credential-routing.md`. Never point this harness at real
credentials or a real mailbox.
## Scanning the harness
```bash
docker compose -f tests/harness/docker-compose.yml up -d
export EMAIL_CONNECT_IMAP_USER=t01@harness.email-connect.test
export EMAIL_CONNECT_IMAP_PASSWORD=harness
email-connect scan-mailbox --config config/harness-imap.yml --out reports/
```
## Sending through the harness
GreenMail offers no STARTTLS, only plaintext or implicit TLS with a self-signed
certificate. The transactional service therefore needs its transport mode set
explicitly:
```bash
export EMAIL_CONNECT_SMTP_HOST=127.0.0.1
export EMAIL_CONNECT_SMTP_PORT=3025
export EMAIL_CONNECT_SMTP_SECURITY=plaintext
export EMAIL_CONNECT_SMTP_USERNAME=sender@harness.email-connect.test
export EMAIL_CONNECT_SMTP_PASSWORD=harness
export EMAIL_CONNECT_SENDER=noreply@harness.email-connect.test
```
`EMAIL_CONNECT_SMTP_SECURITY` defaults to `starttls`. `plaintext` is rejected
for any non-loopback host, so this setting cannot weaken a real deployment: it
fails at startup rather than sending credentials in the clear.
## What harness mail proves
Nothing beyond `provider_accepted`. A message sitting in a GreenMail mailbox is
not evidence of inbox placement, recipient awareness, identity, or
authorization. Bounces, complaints, and deferrals are **not** reproducible here
— GreenMail accepts everything. Those classes stay on crafted `.eml` fixtures
and an optional provider-simulator tier (`EMAIL-WP-0005-T05`).

View file

@ -0,0 +1,38 @@
# Local mail server for email-connect integration tests (EMAIL-WP-0005-T02).
#
# GreenMail serves SMTP and IMAP from one container and creates the declared
# accounts at startup, which is what the scanner's IMAP source and the
# transactional service's SMTP provider need.
#
# Everything here is test-only. The credentials below are deliberately
# non-secret and belong in the repo; real provider material is routed through
# OpenBao (see .claude/rules/credential-routing.md) and must never appear here.
services:
greenmail:
image: greenmail/standalone:2.1.12@sha256:9f32971b4f25d32b4de6fa2e297423768441c65e4541f6aecd7631c890a229a7
container_name: email-connect-harness
# Loopback-only bindings. The harness accepts unauthenticated-ish test
# traffic and must never be reachable from the network.
ports:
- "127.0.0.1:3025:3025" # SMTP
- "127.0.0.1:3143:3143" # IMAP
- "127.0.0.1:8080:8080" # GreenMail API (readiness, purge)
environment:
GREENMAIL_OPTS: >-
-Dgreenmail.setup.test.smtp
-Dgreenmail.setup.test.imap
-Dgreenmail.hostname=0.0.0.0
-Dgreenmail.auth.disabled
-Dgreenmail.verbose
# The image ships no curl/wget/nc, so readiness is probed with bash's
# /dev/tcp against both mail ports.
healthcheck:
test:
- CMD
- bash
- -c
- "exec 3<>/dev/tcp/127.0.0.1/3025 && exec 4<>/dev/tcp/127.0.0.1/3143"
interval: 2s
timeout: 3s
retries: 30
start_period: 5s

View file

@ -2,8 +2,11 @@ import io
import json
import smtplib
import pytest
from email_connect.transactional import (
ProviderError,
SMTPProvider,
SQLiteDeliveryStore,
TransactionalApplication,
)
@ -276,3 +279,69 @@ def test_mailbox_evidence_is_not_authorization(tmp_path):
assert accepted["evidence_ceiling"] == "provider_accepted"
# Ceiling is explicit; user-engine must not elevate this to authz.
assert "authorized" not in accepted
class RecordingSMTP:
"""Minimal smtplib.SMTP stand-in that records the handshake it was given."""
instances = []
def __init__(self, host, port, timeout=None):
self.host, self.port, self.timeout = host, port, timeout
self.starttls_calls = 0
self.logins = []
self.sent = []
RecordingSMTP.instances.append(self)
def __enter__(self):
return self
def __exit__(self, *exc):
return False
def starttls(self):
self.starttls_calls += 1
def login(self, username, password):
self.logins.append((username, password))
def send_message(self, message):
self.sent.append(message)
def test_smtp_provider_defaults_to_starttls(monkeypatch):
monkeypatch.setattr(smtplib, "SMTP", RecordingSMTP)
RecordingSMTP.instances = []
provider = SMTPProvider("smtp.example.com", 587, "user", "pass", "noreply@example.com")
assert provider.security == "starttls"
provider.send("person@example.test", "Subject", "body")
assert RecordingSMTP.instances[0].starttls_calls == 1
def test_plaintext_security_skips_starttls_on_loopback(monkeypatch):
monkeypatch.setattr(smtplib, "SMTP", RecordingSMTP)
RecordingSMTP.instances = []
provider = SMTPProvider("127.0.0.1", 3025, "user", "pass", "noreply@example.com", security="plaintext")
provider.send("person@example.test", "Subject", "body")
smtp = RecordingSMTP.instances[0]
assert smtp.starttls_calls == 0
assert smtp.logins == [("user", "pass")]
assert len(smtp.sent) == 1
@pytest.mark.parametrize("host", ["127.0.0.1", "127.0.1.1", "::1", "[::1]", "localhost", "LOCALHOST"])
def test_plaintext_security_is_allowed_for_loopback_hosts(host):
assert SMTPProvider(host, 3025, "user", "pass", "noreply@example.com", security="plaintext")
@pytest.mark.parametrize("host", ["smtp.example.com", "10.0.0.5", "::ffff:10.0.0.5", "", "localhost.example.com"])
def test_plaintext_security_is_refused_for_non_loopback_hosts(host):
with pytest.raises(ValueError, match="loopback"):
SMTPProvider(host, 3025, "user", "pass", "noreply@example.com", security="plaintext")
def test_unsupported_security_mode_is_refused():
with pytest.raises(ValueError, match="security mode"):
SMTPProvider("127.0.0.1", 3025, "user", "pass", "noreply@example.com", security="none")

View file

@ -150,7 +150,7 @@ Done 2026-08-14:
```task
id: EMAIL-WP-0005-T02
status: todo
status: done
priority: high
state_hub_task_id: "967d2bab-43e3-4240-8bd5-c657ba4178ff"
```
@ -183,6 +183,30 @@ accounts on demand. Mailpit is the fallback for send-side-only inspection and
does not exercise the IMAP source. Record the choice in DECISIONS.md.
```
Done 2026-08-14:
* `tests/harness/docker-compose.yml` runs GreenMail `2.1.12`, digest-pinned,
SMTP 3025 / IMAP 3143 / API 8080, all bound to `127.0.0.1` only. The image
ships no curl/wget/nc, so the healthcheck probes both mail ports with bash
`/dev/tcp`. Verified healthy.
* `greenmail.auth.disabled` with no declared users: any login is accepted and
the mailbox is created on first use, so T03 needs no provisioning step.
Declaring users *and* enabling auth-disabled conflicts — GreenMail tries to
auto-create the login and collides with the declared address.
* `config/harness-imap.yml` scanner profile; env-var recipe for the send side in
`tests/harness/README.md`.
* GreenMail standalone has no STARTTLS support (confirmed against the shipped
jar's property builder — only plain and implicit-TLS setups exist), and
`SMTPProvider` hardcoded `starttls()`, so no send could reach it. Added an
`EMAIL_CONNECT_SMTP_SECURITY` mode defaulting to `starttls`; `plaintext` is
refused for any non-loopback host and hostnames are never resolved to decide
it. See DECISIONS.md.
* Verified end to end: `SMTPProvider.send` → GreenMail → `ImapMailboxSource`
fetch, then the documented CLI (`scan-mailbox --config config/harness-imap.yml`)
against the live harness — 1 message seen, parsed, 1 evidence event.
* Tests: 14 offline cases for the transport-security guard in
`tests/test_transactional.py`. Full suite 52 passed, still container-free.
## T03 - Test-user account provisioning and reset
```task