EMAIL-WP-0005-T02: add GreenMail test harness
Adds tests/harness/docker-compose.yml running GreenMail 2.1.12 (digest-pinned) with SMTP 3025, IMAP 3143 and the API bound to 127.0.0.1 only, plus a config/harness-imap.yml scanner profile and harness README. Auth is disabled and no users are declared, so a mailbox is created on first login and per-test users need no provisioning. GreenMail standalone offers no STARTTLS, only plaintext or implicit TLS, while SMTPProvider hardcoded starttls() -- so no send could reach it. SMTPProvider now takes a security mode via EMAIL_CONNECT_SMTP_SECURITY, defaulting to starttls. plaintext is refused for any non-loopback host, and hostnames are never resolved to decide that, so a misconfigured deployment fails at startup rather than sending credentials in the clear. Trusting GreenMail's self-signed cert was rejected as the wider risk; see DECISIONS.md. Verified end to end against the live harness: SMTPProvider.send -> GreenMail -> ImapMailboxSource, and the documented scan-mailbox CLI. Suite: 52 passed with the harness down. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
86f22c2e65
commit
89fd13ac2d
8 changed files with 329 additions and 4 deletions
28
DECISIONS.md
28
DECISIONS.md
|
|
@ -160,3 +160,31 @@ local server; those classes stay on crafted fixtures plus an optional provider
|
|||
simulator tier.
|
||||
|
||||
---
|
||||
|
||||
## Test harness mail server: GreenMail, and plaintext SMTP for loopback only
|
||||
|
||||
**Date:** 2026-08-14
|
||||
**Decided by:** codex (EMAIL-WP-0005-T02)
|
||||
|
||||
GreenMail `2.1.12` (digest-pinned) is the harness mail server. It serves SMTP
|
||||
and IMAP from one container and, with `greenmail.auth.disabled`, creates a
|
||||
mailbox on first login — so per-test users need no provisioning step. Mailpit
|
||||
was the alternative but offers no IMAP, which would leave the scanner's IMAP
|
||||
source untested.
|
||||
|
||||
GreenMail standalone supports only plaintext or implicit-TLS setups; it has no
|
||||
STARTTLS option (verified against the shipped jar's property builder). The
|
||||
transactional service previously hardcoded `starttls()`, so no send could reach
|
||||
it.
|
||||
|
||||
`SMTPProvider` therefore takes a `security` mode, `EMAIL_CONNECT_SMTP_SECURITY`,
|
||||
defaulting to `starttls`. `plaintext` is refused for any non-loopback host, and
|
||||
hostnames are never resolved to decide this — a name pointing at 127.0.0.1 today
|
||||
is not a durable guarantee. A misconfigured deployment fails at startup instead
|
||||
of putting credentials on the wire.
|
||||
|
||||
The alternative was trusting GreenMail's self-signed certificate for implicit
|
||||
TLS, which would have meant shipping certificate-verification bypass code in the
|
||||
send path. A loopback-guarded plaintext mode is the narrower risk.
|
||||
|
||||
---
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@
|
|||
| task | EMAIL-WP-0004-T02 | done | — | workplans/EMAIL-WP-0004-transactional-mail-delivery-service.md |
|
||||
| task | EMAIL-WP-0004-T03 | done | — | workplans/EMAIL-WP-0004-transactional-mail-delivery-service.md |
|
||||
| task | EMAIL-WP-0004-T04 | done | — | workplans/EMAIL-WP-0004-transactional-mail-delivery-service.md |
|
||||
| task | EMAIL-WP-0005-T01 | todo | — | workplans/EMAIL-WP-0005-test-mailbox-harness.md |
|
||||
| task | EMAIL-WP-0005-T01 | done | — | workplans/EMAIL-WP-0005-test-mailbox-harness.md |
|
||||
| task | EMAIL-WP-0005-T02 | todo | — | workplans/EMAIL-WP-0005-test-mailbox-harness.md |
|
||||
| task | EMAIL-WP-0005-T03 | todo | — | workplans/EMAIL-WP-0005-test-mailbox-harness.md |
|
||||
| task | EMAIL-WP-0005-T04 | todo | — | workplans/EMAIL-WP-0005-test-mailbox-harness.md |
|
||||
|
|
|
|||
46
config/harness-imap.yml
Normal file
46
config/harness-imap.yml
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
# Scanner profile for the local GreenMail test harness (EMAIL-WP-0005-T02).
|
||||
#
|
||||
# Start the harness first:
|
||||
# docker compose -f tests/harness/docker-compose.yml up -d
|
||||
#
|
||||
# The harness runs with authentication disabled and creates a mailbox on first
|
||||
# login, so these credentials are placeholders that only need to be set:
|
||||
# export EMAIL_CONNECT_IMAP_USER=t01@harness.email-connect.test
|
||||
# export EMAIL_CONNECT_IMAP_PASSWORD=harness
|
||||
#
|
||||
# Then:
|
||||
# email-connect scan-mailbox --config config/harness-imap.yml --out reports/
|
||||
mailbox:
|
||||
id: harness-mailbox
|
||||
protocol: imap
|
||||
host: 127.0.0.1
|
||||
port: 3143
|
||||
tls: false
|
||||
username_env: EMAIL_CONNECT_IMAP_USER
|
||||
password_env: EMAIL_CONNECT_IMAP_PASSWORD
|
||||
folder: INBOX
|
||||
|
||||
scan:
|
||||
mode: incremental
|
||||
max_messages_per_run: 5000
|
||||
since: null
|
||||
from: null
|
||||
to: null
|
||||
include_seen: true
|
||||
mark_seen: false
|
||||
store_raw_headers: true
|
||||
store_raw_body: false
|
||||
store_raw_message_ref: true
|
||||
|
||||
expected_recipients:
|
||||
path: null
|
||||
csv_column: email
|
||||
|
||||
storage:
|
||||
path: .email-connect/harness-state.sqlite
|
||||
|
||||
reports:
|
||||
output_dir: reports
|
||||
include_all_evidence: true
|
||||
include_unknown_messages: true
|
||||
timestamp_timezone: UTC
|
||||
|
|
@ -4,6 +4,7 @@ from __future__ import annotations
|
|||
|
||||
import hmac
|
||||
import hashlib
|
||||
import ipaddress
|
||||
import json
|
||||
import os
|
||||
import smtplib
|
||||
|
|
@ -148,7 +149,25 @@ class SQLiteDeliveryStore:
|
|||
|
||||
|
||||
class SMTPProvider:
|
||||
def __init__(self, host: str, port: int, username: str, password: str, sender: str) -> None:
|
||||
#: STARTTLS is the only transport allowed against a remote provider.
|
||||
#: `plaintext` exists for loopback test servers that cannot offer TLS and is
|
||||
#: refused for any other host, so a misconfiguration cannot put credentials
|
||||
#: on the wire in the clear.
|
||||
SECURITY_MODES = ("starttls", "plaintext")
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
host: str,
|
||||
port: int,
|
||||
username: str,
|
||||
password: str,
|
||||
sender: str,
|
||||
security: str = "starttls",
|
||||
) -> None:
|
||||
if security not in self.SECURITY_MODES:
|
||||
raise ValueError(f"Unsupported SMTP security mode: {security}")
|
||||
if security == "plaintext" and not _is_loopback_host(host):
|
||||
raise ValueError("Plaintext SMTP is only permitted for loopback hosts.")
|
||||
self.host, self.port, self.username, self.password, self.sender = (
|
||||
host,
|
||||
port,
|
||||
|
|
@ -156,6 +175,7 @@ class SMTPProvider:
|
|||
password,
|
||||
sender,
|
||||
)
|
||||
self.security = security
|
||||
|
||||
def send(self, recipient: str, subject: str, text: str) -> str:
|
||||
message = EmailMessage()
|
||||
|
|
@ -163,7 +183,8 @@ class SMTPProvider:
|
|||
message.set_content(text)
|
||||
try:
|
||||
with smtplib.SMTP(self.host, self.port, timeout=10) as smtp:
|
||||
smtp.starttls()
|
||||
if self.security == "starttls":
|
||||
smtp.starttls()
|
||||
smtp.login(self.username, self.password)
|
||||
smtp.send_message(message)
|
||||
except TimeoutError as exc:
|
||||
|
|
@ -352,6 +373,18 @@ class TransactionalApplication:
|
|||
return [body]
|
||||
|
||||
|
||||
def _is_loopback_host(host: str) -> bool:
|
||||
candidate = (host or "").strip().strip("[]")
|
||||
if candidate.lower() == "localhost":
|
||||
return True
|
||||
try:
|
||||
return ipaddress.ip_address(candidate).is_loopback
|
||||
except ValueError:
|
||||
# Never resolve names here: a hostname that happens to point at 127.0.0.1
|
||||
# today is not a durable guarantee that credentials stay on the host.
|
||||
return False
|
||||
|
||||
|
||||
def _valid_address(value: str) -> bool:
|
||||
_, address = parseaddr(value)
|
||||
return address == value and "@" in address and "\n" not in address and "\r" not in address
|
||||
|
|
@ -364,6 +397,7 @@ def main() -> None:
|
|||
os.environ["EMAIL_CONNECT_SMTP_USERNAME"],
|
||||
os.environ["EMAIL_CONNECT_SMTP_PASSWORD"],
|
||||
os.environ["EMAIL_CONNECT_SENDER"],
|
||||
security=os.environ.get("EMAIL_CONNECT_SMTP_SECURITY", "starttls"),
|
||||
)
|
||||
app = TransactionalApplication(
|
||||
SQLiteDeliveryStore(os.environ.get("EMAIL_CONNECT_DATABASE_PATH", "/data/email-connect.db")),
|
||||
|
|
|
|||
86
tests/harness/README.md
Normal file
86
tests/harness/README.md
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
# Local mail test harness
|
||||
|
||||
A single GreenMail container serving SMTP and IMAP on loopback, so integration
|
||||
tests can drive the transactional send path and read the result back through the
|
||||
scanner without a live provider.
|
||||
|
||||
Part of `EMAIL-WP-0005`. Unit tests do not need this harness — they run against
|
||||
fixture directories and temporary Maildir trees and stay offline.
|
||||
|
||||
## Start and stop
|
||||
|
||||
```bash
|
||||
docker compose -f tests/harness/docker-compose.yml up -d
|
||||
docker compose -f tests/harness/docker-compose.yml down -v
|
||||
```
|
||||
|
||||
Readiness (the container healthcheck probes both mail ports; the API also
|
||||
answers):
|
||||
|
||||
```bash
|
||||
docker inspect --format '{{.State.Health.Status}}' email-connect-harness
|
||||
curl -s http://127.0.0.1:8080/api/service/readiness
|
||||
```
|
||||
|
||||
## Ports
|
||||
|
||||
| Port | Protocol | Notes |
|
||||
| --- | --- | --- |
|
||||
| 3025 | SMTP | plaintext, no STARTTLS |
|
||||
| 3143 | IMAP | plaintext |
|
||||
| 8080 | HTTP | GreenMail API — readiness, user and mail management |
|
||||
|
||||
All three bind `127.0.0.1` only. The harness accepts any credentials and must
|
||||
never be exposed beyond the host.
|
||||
|
||||
## Accounts
|
||||
|
||||
The harness runs with `greenmail.auth.disabled`, so **any** login is accepted and
|
||||
the mailbox is created on first use. No provisioning step is needed: pick an
|
||||
address under `harness.email-connect.test` and log in.
|
||||
|
||||
```bash
|
||||
export EMAIL_CONNECT_IMAP_USER=t01@harness.email-connect.test
|
||||
export EMAIL_CONNECT_IMAP_PASSWORD=harness # ignored, but must be set
|
||||
```
|
||||
|
||||
These credentials are deliberately non-secret and belong in the repo. Real
|
||||
provider material is routed through OpenBao — see
|
||||
`.claude/rules/credential-routing.md`. Never point this harness at real
|
||||
credentials or a real mailbox.
|
||||
|
||||
## Scanning the harness
|
||||
|
||||
```bash
|
||||
docker compose -f tests/harness/docker-compose.yml up -d
|
||||
export EMAIL_CONNECT_IMAP_USER=t01@harness.email-connect.test
|
||||
export EMAIL_CONNECT_IMAP_PASSWORD=harness
|
||||
email-connect scan-mailbox --config config/harness-imap.yml --out reports/
|
||||
```
|
||||
|
||||
## Sending through the harness
|
||||
|
||||
GreenMail offers no STARTTLS, only plaintext or implicit TLS with a self-signed
|
||||
certificate. The transactional service therefore needs its transport mode set
|
||||
explicitly:
|
||||
|
||||
```bash
|
||||
export EMAIL_CONNECT_SMTP_HOST=127.0.0.1
|
||||
export EMAIL_CONNECT_SMTP_PORT=3025
|
||||
export EMAIL_CONNECT_SMTP_SECURITY=plaintext
|
||||
export EMAIL_CONNECT_SMTP_USERNAME=sender@harness.email-connect.test
|
||||
export EMAIL_CONNECT_SMTP_PASSWORD=harness
|
||||
export EMAIL_CONNECT_SENDER=noreply@harness.email-connect.test
|
||||
```
|
||||
|
||||
`EMAIL_CONNECT_SMTP_SECURITY` defaults to `starttls`. `plaintext` is rejected
|
||||
for any non-loopback host, so this setting cannot weaken a real deployment: it
|
||||
fails at startup rather than sending credentials in the clear.
|
||||
|
||||
## What harness mail proves
|
||||
|
||||
Nothing beyond `provider_accepted`. A message sitting in a GreenMail mailbox is
|
||||
not evidence of inbox placement, recipient awareness, identity, or
|
||||
authorization. Bounces, complaints, and deferrals are **not** reproducible here
|
||||
— GreenMail accepts everything. Those classes stay on crafted `.eml` fixtures
|
||||
and an optional provider-simulator tier (`EMAIL-WP-0005-T05`).
|
||||
38
tests/harness/docker-compose.yml
Normal file
38
tests/harness/docker-compose.yml
Normal file
|
|
@ -0,0 +1,38 @@
|
|||
# Local mail server for email-connect integration tests (EMAIL-WP-0005-T02).
|
||||
#
|
||||
# GreenMail serves SMTP and IMAP from one container and creates the declared
|
||||
# accounts at startup, which is what the scanner's IMAP source and the
|
||||
# transactional service's SMTP provider need.
|
||||
#
|
||||
# Everything here is test-only. The credentials below are deliberately
|
||||
# non-secret and belong in the repo; real provider material is routed through
|
||||
# OpenBao (see .claude/rules/credential-routing.md) and must never appear here.
|
||||
services:
|
||||
greenmail:
|
||||
image: greenmail/standalone:2.1.12@sha256:9f32971b4f25d32b4de6fa2e297423768441c65e4541f6aecd7631c890a229a7
|
||||
container_name: email-connect-harness
|
||||
# Loopback-only bindings. The harness accepts unauthenticated-ish test
|
||||
# traffic and must never be reachable from the network.
|
||||
ports:
|
||||
- "127.0.0.1:3025:3025" # SMTP
|
||||
- "127.0.0.1:3143:3143" # IMAP
|
||||
- "127.0.0.1:8080:8080" # GreenMail API (readiness, purge)
|
||||
environment:
|
||||
GREENMAIL_OPTS: >-
|
||||
-Dgreenmail.setup.test.smtp
|
||||
-Dgreenmail.setup.test.imap
|
||||
-Dgreenmail.hostname=0.0.0.0
|
||||
-Dgreenmail.auth.disabled
|
||||
-Dgreenmail.verbose
|
||||
# The image ships no curl/wget/nc, so readiness is probed with bash's
|
||||
# /dev/tcp against both mail ports.
|
||||
healthcheck:
|
||||
test:
|
||||
- CMD
|
||||
- bash
|
||||
- -c
|
||||
- "exec 3<>/dev/tcp/127.0.0.1/3025 && exec 4<>/dev/tcp/127.0.0.1/3143"
|
||||
interval: 2s
|
||||
timeout: 3s
|
||||
retries: 30
|
||||
start_period: 5s
|
||||
|
|
@ -2,8 +2,11 @@ import io
|
|||
import json
|
||||
import smtplib
|
||||
|
||||
import pytest
|
||||
|
||||
from email_connect.transactional import (
|
||||
ProviderError,
|
||||
SMTPProvider,
|
||||
SQLiteDeliveryStore,
|
||||
TransactionalApplication,
|
||||
)
|
||||
|
|
@ -276,3 +279,69 @@ def test_mailbox_evidence_is_not_authorization(tmp_path):
|
|||
assert accepted["evidence_ceiling"] == "provider_accepted"
|
||||
# Ceiling is explicit; user-engine must not elevate this to authz.
|
||||
assert "authorized" not in accepted
|
||||
|
||||
|
||||
class RecordingSMTP:
|
||||
"""Minimal smtplib.SMTP stand-in that records the handshake it was given."""
|
||||
|
||||
instances = []
|
||||
|
||||
def __init__(self, host, port, timeout=None):
|
||||
self.host, self.port, self.timeout = host, port, timeout
|
||||
self.starttls_calls = 0
|
||||
self.logins = []
|
||||
self.sent = []
|
||||
RecordingSMTP.instances.append(self)
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc):
|
||||
return False
|
||||
|
||||
def starttls(self):
|
||||
self.starttls_calls += 1
|
||||
|
||||
def login(self, username, password):
|
||||
self.logins.append((username, password))
|
||||
|
||||
def send_message(self, message):
|
||||
self.sent.append(message)
|
||||
|
||||
|
||||
def test_smtp_provider_defaults_to_starttls(monkeypatch):
|
||||
monkeypatch.setattr(smtplib, "SMTP", RecordingSMTP)
|
||||
RecordingSMTP.instances = []
|
||||
provider = SMTPProvider("smtp.example.com", 587, "user", "pass", "noreply@example.com")
|
||||
|
||||
assert provider.security == "starttls"
|
||||
provider.send("person@example.test", "Subject", "body")
|
||||
assert RecordingSMTP.instances[0].starttls_calls == 1
|
||||
|
||||
|
||||
def test_plaintext_security_skips_starttls_on_loopback(monkeypatch):
|
||||
monkeypatch.setattr(smtplib, "SMTP", RecordingSMTP)
|
||||
RecordingSMTP.instances = []
|
||||
provider = SMTPProvider("127.0.0.1", 3025, "user", "pass", "noreply@example.com", security="plaintext")
|
||||
|
||||
provider.send("person@example.test", "Subject", "body")
|
||||
smtp = RecordingSMTP.instances[0]
|
||||
assert smtp.starttls_calls == 0
|
||||
assert smtp.logins == [("user", "pass")]
|
||||
assert len(smtp.sent) == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize("host", ["127.0.0.1", "127.0.1.1", "::1", "[::1]", "localhost", "LOCALHOST"])
|
||||
def test_plaintext_security_is_allowed_for_loopback_hosts(host):
|
||||
assert SMTPProvider(host, 3025, "user", "pass", "noreply@example.com", security="plaintext")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("host", ["smtp.example.com", "10.0.0.5", "::ffff:10.0.0.5", "", "localhost.example.com"])
|
||||
def test_plaintext_security_is_refused_for_non_loopback_hosts(host):
|
||||
with pytest.raises(ValueError, match="loopback"):
|
||||
SMTPProvider(host, 3025, "user", "pass", "noreply@example.com", security="plaintext")
|
||||
|
||||
|
||||
def test_unsupported_security_mode_is_refused():
|
||||
with pytest.raises(ValueError, match="security mode"):
|
||||
SMTPProvider("127.0.0.1", 3025, "user", "pass", "noreply@example.com", security="none")
|
||||
|
|
|
|||
|
|
@ -150,7 +150,7 @@ Done 2026-08-14:
|
|||
|
||||
```task
|
||||
id: EMAIL-WP-0005-T02
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "967d2bab-43e3-4240-8bd5-c657ba4178ff"
|
||||
```
|
||||
|
|
@ -183,6 +183,30 @@ accounts on demand. Mailpit is the fallback for send-side-only inspection and
|
|||
does not exercise the IMAP source. Record the choice in DECISIONS.md.
|
||||
```
|
||||
|
||||
Done 2026-08-14:
|
||||
|
||||
* `tests/harness/docker-compose.yml` runs GreenMail `2.1.12`, digest-pinned,
|
||||
SMTP 3025 / IMAP 3143 / API 8080, all bound to `127.0.0.1` only. The image
|
||||
ships no curl/wget/nc, so the healthcheck probes both mail ports with bash
|
||||
`/dev/tcp`. Verified healthy.
|
||||
* `greenmail.auth.disabled` with no declared users: any login is accepted and
|
||||
the mailbox is created on first use, so T03 needs no provisioning step.
|
||||
Declaring users *and* enabling auth-disabled conflicts — GreenMail tries to
|
||||
auto-create the login and collides with the declared address.
|
||||
* `config/harness-imap.yml` scanner profile; env-var recipe for the send side in
|
||||
`tests/harness/README.md`.
|
||||
* GreenMail standalone has no STARTTLS support (confirmed against the shipped
|
||||
jar's property builder — only plain and implicit-TLS setups exist), and
|
||||
`SMTPProvider` hardcoded `starttls()`, so no send could reach it. Added an
|
||||
`EMAIL_CONNECT_SMTP_SECURITY` mode defaulting to `starttls`; `plaintext` is
|
||||
refused for any non-loopback host and hostnames are never resolved to decide
|
||||
it. See DECISIONS.md.
|
||||
* Verified end to end: `SMTPProvider.send` → GreenMail → `ImapMailboxSource`
|
||||
fetch, then the documented CLI (`scan-mailbox --config config/harness-imap.yml`)
|
||||
against the live harness — 1 message seen, parsed, 1 evidence event.
|
||||
* Tests: 14 offline cases for the transport-security guard in
|
||||
`tests/test_transactional.py`. Full suite 52 passed, still container-free.
|
||||
|
||||
## T03 - Test-user account provisioning and reset
|
||||
|
||||
```task
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue