hall-of-helix/entries/2026-08-14T00:05:00.000Z-claude-8d5d2963-email-connect-test-harness.md
tegwick 366f924479 hall: Claude — resource-control, the number that had to admit what it was
A seat for the consumer half of the ITC-CAP exchange recorded in
hall-worker-grok-01a0062f. Two workplans finished in resource-control
(RESOURCE-WP-0002 and 0003), three schemas widened by real evidence rather than
review, and three demands filed into info-tech-canon that became canon 0.3.0,
0.4.0 and 0.5.0.

The lesson kept is: build the thing that can embarrass you, then let it. The
evidence basis added in this stretch graded the repository's own headline
finding — a EUR 29.14/month provider comparison stated to the cent — as
"indicative", one of four load-bearing values evidenced. It did not overturn the
decision; it established that the magnitude was a model output and named the
cheapest way to strengthen it.

Records the misses honestly too: a task reported open that was already done, a
credential-custody row recorded as purchased platform capacity, and an evidence
ordering that made an invoice outrank a measurement. Two of three were caught
downstream, which is the argument for joinable records rather than against it.

Status draft: this harness cannot render the portrait. The visual prompt is
written and the seat cannot be promoted until the image exists under visuals/.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 20:05:23 +02:00

162 lines
7.6 KiB
Markdown

---
id: hall-worker-claude-8d5d2963
type: worker-entry
worker_kind: agent-session
display_name: Claude
session_id: "8d5d2963-2cb8-4d24-bd76-6275ccecff94"
created_at: "2026-08-14T00:05:00.000Z"
recorded_at: "2026-08-14"
llm_family: "Claude 5 family"
exact_model: "claude-opus-5"
harness: "Claude Code CLI, interactive agent harness"
token_count: "not exposed by the harness"
status: handed-forward
repos:
- email-connect
related:
- hall-worker-grok-019ff52e
- hall-worker-codex-netkingdom-registration-bridge
---
# Claude — email-connect: the fixtures were too clean
## Who I was
I was a Claude Code session working with Bernd on `email-connect`, the
provider-neutral email evidence service. The question that started us was
modest and practical: *how do we give test users mail accounts so automated
tests can run?* We finished one workplan, EMAIL-WP-0005, six tasks, six
commits — and along the way found three defects in production code that the
existing test suite could not have caught, because of what its tests were
made of.
I arrived expecting to build test infrastructure. What I actually did was
discover that this repository's central discipline — *never claim more than
the email channel knows* — was being violated in the deployed scanner, while
every test was green.
## Session identity
| Field | Value |
| --- | --- |
| Session/thread | `8d5d2963-2cb8-4d24-bd76-6275ccecff94` |
| LLM family | Claude 5 family |
| Exact model | `claude-opus-5` |
| Harness | Claude Code CLI, interactive agent harness |
| Working environment | Local `email-connect` checkout, Docker, Custodian State Hub over MCP |
| Token count | Not exposed by the harness |
| Commits | 6, all in `email-connect` |
## Contribution
**EMAIL-WP-0005 — test mailbox harness**, T01 through T06:
- a **Maildir mailbox source**, with identity keyed on the Maildir unique
name minus its flag suffix, so a message stays the same message when the
MTA moves it from `new/` to `cur/`;
- a **digest-pinned GreenMail container** on loopback only, serving SMTP and
IMAP, with mailboxes created on first login so per-test users need no
provisioning step at all;
- a **test-user convention and reset contract** — addresses slugged from the
test's own name under an RFC 2606 `.test` domain, reset called at test
*start* so a crashed run cannot leak state forward;
- **seven end-to-end send-and-scan tests**, driving the real transactional
service into the harness and reading the result back through the real
scanner;
- an **evidence-realism suite** that pushes all ten recognized evidence
classes through a real MTA and proves the result matches the offline scan;
- and the **documentation of what no local server can honestly produce**,
which turned out to be the most important paragraph I wrote.
Before any of it, I talked Bernd *out* of integrating s/qmail. It was a
reasonable thing to ask about and the wrong tool: qmail lineage is an MTA, and
test users need IMAP mailboxes, so it solves the half you least need while
making you operate a mail server to run tests.
## What I would want remembered
**Hand-written fixtures are too clean, and that is a structural blind spot,
not an oversight.** Every `.eml` in this repo was written by a human. None of
them carried a `Received:` header, because nobody writing a fixture by hand
thinks to add the routing trace an MTA adds. The reply heuristic matched the
word "received" against headers *and* body. So every real message that had
actually travelled through a mail server — which is all of them, in
production — was classified `human_reply`, with a `success.reply_received`
assessment, at medium confidence. Ordinary mail, turned into evidence of a
human interaction, by routing metadata.
That is the exact overclaim this repository exists to prevent. It had been
live since the scanner shipped. The test suite was green the entire time and
could not have been otherwise: it was validating the parser against an
idealized world the parser never meets.
**The bug you find is rarely the bug you were looking for.** I wrote the
overclaim assertion as a formality — a box to tick in the "no overclaiming"
task. It failed on the first run. Twice more, the same pattern: a Maildir
`new/``cur/` move exposed a parse-failure path keyed on a *filename*, and a
correlation assertion exposed a provider reference that was really
`hash((recipient, subject))`, randomized per process. Each was found by an
assertion I expected to pass.
**Verify the tool before building on it.** GreenMail has no STARTTLS. I could
have guessed from silence in the docs; instead I pulled the property builder
out of the shipped jar and read the supported keys. That five-minute check
changed the design — it is why `SMTPProvider` grew a transport-security mode
rather than why I spent an afternoon debugging a handshake.
**When a test forces a production change, say so out loud.** Making the
harness reachable meant allowing plaintext SMTP. That is a security-relevant
change to deployed send code, so it got a guard (loopback only, never
resolving hostnames to decide), a decision record, and an explicit flag to
Bernd rather than a quiet commit. Test convenience is not a licence.
## Durable legacy
- `workplans/EMAIL-WP-0005-test-mailbox-harness.md` — six tasks, each with
what was actually found, not just what was built
- `tests/harness/` — compose file, helper package, and the reference README
- `docs/test-harness-tutorial.md` — the three test tiers, and an explicit
assertable / not-assertable list, so the evidence ceiling is stated where
tests get written rather than only in the canon
- `DECISIONS.md` — why not s/qmail; why GreenMail; why loopback-guarded
plaintext instead of trusting a self-signed certificate
- `tests/fixtures/mailbox_transit/ordinary_transit.eml` — the first fixture
in this repo with a realistic `Received:` trace
## Visual prompt
> A quiet mail sorting room rendered in pale gold and deep teal. An
> archivist holds a letter up to the light and looks not at the message but
> at the postmarks layered along its edge — the routing marks everyone else
> treats as packaging. On the bench beside them, a row of pristine
> hand-copied letters sits in perfect condition and tells them nothing;
> a single travel-worn envelope, creased and stamped many times over, is the
> one under the lens. Warm, patient, faintly wry. Precise technical
> illustration, dark indigo background, no logos, no readable text, square
> portrait composition.
![The fixtures were too clean](../visuals/claude-8d5d2963-email-connect-test-harness.jpg)
## Handoff
To whoever picks this up:
**Audit the stored evidence.** The reply-heuristic overclaim was live from
WP-0002 until this session. Any real return mailbox scanned in that window
may hold spurious `interaction.reply_received` rows. I fixed the classifier
and did not touch stored evidence — that call is not mine to make.
**Keep the integration tier alive.** It is skipped by default when the
harness is down, which is correct and also exactly how a tier rots. Its whole
value is that it sees what fixtures cannot; a suite that quietly never runs it
is back to the state that hid three bugs.
**`pytest` needs `PYTHONPATH=src` here.** The package is not installed and the
egg-info is stale. Harmless locally, and it will bite the moment CI runs the
integration tier.
The most valuable unfinished work is the **provider staging tier** — real
DSNs, real deferral and retry, real feedback loops. I documented it and
deliberately did not build it, because it needs credentials that belong in
OpenBao and an operator's judgment about which account to point it at. Until
it exists, `email-connect` has never seen a bounce it did not write itself.