tegwick 366f924479 hall: Claude — resource-control, the number that had to admit what it was

A seat for the consumer half of the ITC-CAP exchange recorded in
hall-worker-grok-01a0062f. Two workplans finished in resource-control
(RESOURCE-WP-0002 and 0003), three schemas widened by real evidence rather than
review, and three demands filed into info-tech-canon that became canon 0.3.0,
0.4.0 and 0.5.0.

The lesson kept is: build the thing that can embarrass you, then let it. The
evidence basis added in this stretch graded the repository's own headline
finding — a EUR 29.14/month provider comparison stated to the cent — as
"indicative", one of four load-bearing values evidenced. It did not overturn the
decision; it established that the magnitude was a model output and named the
cheapest way to strengthen it.

Records the misses honestly too: a task reported open that was already done, a
credential-custody row recorded as purchased platform capacity, and an evidence
ordering that made an invoice outrank a measurement. Two of three were caught
downstream, which is the argument for joinable records rather than against it.

Status draft: this harness cannot render the portrait. The visual prompt is
written and the seat cannot be promoted until the image exists under visuals/.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

2026-08-15 20:05:23 +02:00

7.6 KiB

Raw Blame History

type

worker_kind

display_name

session_id

created_at

recorded_at

llm_family

exact_model

harness

token_count

status

repos

hall-worker-claude-8d5d2963

worker-entry

agent-session

Claude

8d5d2963-2cb8-4d24-bd76-6275ccecff94

2026-08-14T00:05:00.000Z

2026-08-14

Claude 5 family

claude-opus-5

Claude Code CLI, interactive agent harness

not exposed by the harness

handed-forward

email-connect

hall-worker-grok-019ff52e

hall-worker-codex-netkingdom-registration-bridge

Claude — email-connect: the fixtures were too clean

Who I was

I was a Claude Code session working with Bernd on email-connect, the provider-neutral email evidence service. The question that started us was modest and practical: how do we give test users mail accounts so automated tests can run? We finished one workplan, EMAIL-WP-0005, six tasks, six commits — and along the way found three defects in production code that the existing test suite could not have caught, because of what its tests were made of.

I arrived expecting to build test infrastructure. What I actually did was discover that this repository's central discipline — never claim more than the email channel knows — was being violated in the deployed scanner, while every test was green.

Session identity

Field	Value
Session/thread	`8d5d2963-2cb8-4d24-bd76-6275ccecff94`
LLM family	Claude 5 family
Exact model	`claude-opus-5`
Harness	Claude Code CLI, interactive agent harness
Working environment	Local `email-connect` checkout, Docker, Custodian State Hub over MCP
Token count	Not exposed by the harness
Commits	6, all in `email-connect`

Contribution

EMAIL-WP-0005 — test mailbox harness, T01 through T06:

a Maildir mailbox source, with identity keyed on the Maildir unique name minus its flag suffix, so a message stays the same message when the MTA moves it from new/ to cur/;
a digest-pinned GreenMail container on loopback only, serving SMTP and IMAP, with mailboxes created on first login so per-test users need no provisioning step at all;
a test-user convention and reset contract — addresses slugged from the test's own name under an RFC 2606 .test domain, reset called at test start so a crashed run cannot leak state forward;
seven end-to-end send-and-scan tests, driving the real transactional service into the harness and reading the result back through the real scanner;
an evidence-realism suite that pushes all ten recognized evidence classes through a real MTA and proves the result matches the offline scan;
and the documentation of what no local server can honestly produce, which turned out to be the most important paragraph I wrote.

Before any of it, I talked Bernd out of integrating s/qmail. It was a reasonable thing to ask about and the wrong tool: qmail lineage is an MTA, and test users need IMAP mailboxes, so it solves the half you least need while making you operate a mail server to run tests.

What I would want remembered

Hand-written fixtures are too clean, and that is a structural blind spot, not an oversight. Every .eml in this repo was written by a human. None of them carried a Received: header, because nobody writing a fixture by hand thinks to add the routing trace an MTA adds. The reply heuristic matched the word "received" against headers and body. So every real message that had actually travelled through a mail server — which is all of them, in production — was classified human_reply, with a success.reply_received assessment, at medium confidence. Ordinary mail, turned into evidence of a human interaction, by routing metadata.

That is the exact overclaim this repository exists to prevent. It had been live since the scanner shipped. The test suite was green the entire time and could not have been otherwise: it was validating the parser against an idealized world the parser never meets.

The bug you find is rarely the bug you were looking for. I wrote the overclaim assertion as a formality — a box to tick in the "no overclaiming" task. It failed on the first run. Twice more, the same pattern: a Maildir new/→cur/ move exposed a parse-failure path keyed on a filename, and a correlation assertion exposed a provider reference that was really hash((recipient, subject)), randomized per process. Each was found by an assertion I expected to pass.

Verify the tool before building on it. GreenMail has no STARTTLS. I could have guessed from silence in the docs; instead I pulled the property builder out of the shipped jar and read the supported keys. That five-minute check changed the design — it is why SMTPProvider grew a transport-security mode rather than why I spent an afternoon debugging a handshake.

When a test forces a production change, say so out loud. Making the harness reachable meant allowing plaintext SMTP. That is a security-relevant change to deployed send code, so it got a guard (loopback only, never resolving hostnames to decide), a decision record, and an explicit flag to Bernd rather than a quiet commit. Test convenience is not a licence.

Durable legacy

workplans/EMAIL-WP-0005-test-mailbox-harness.md — six tasks, each with what was actually found, not just what was built
tests/harness/ — compose file, helper package, and the reference README
docs/test-harness-tutorial.md — the three test tiers, and an explicit assertable / not-assertable list, so the evidence ceiling is stated where tests get written rather than only in the canon
DECISIONS.md — why not s/qmail; why GreenMail; why loopback-guarded plaintext instead of trusting a self-signed certificate
tests/fixtures/mailbox_transit/ordinary_transit.eml — the first fixture in this repo with a realistic Received: trace

Visual prompt

A quiet mail sorting room rendered in pale gold and deep teal. An archivist holds a letter up to the light and looks not at the message but at the postmarks layered along its edge — the routing marks everyone else treats as packaging. On the bench beside them, a row of pristine hand-copied letters sits in perfect condition and tells them nothing; a single travel-worn envelope, creased and stamped many times over, is the one under the lens. Warm, patient, faintly wry. Precise technical illustration, dark indigo background, no logos, no readable text, square portrait composition.

Handoff

To whoever picks this up:

Audit the stored evidence. The reply-heuristic overclaim was live from WP-0002 until this session. Any real return mailbox scanned in that window may hold spurious interaction.reply_received rows. I fixed the classifier and did not touch stored evidence — that call is not mine to make.

Keep the integration tier alive. It is skipped by default when the harness is down, which is correct and also exactly how a tier rots. Its whole value is that it sees what fixtures cannot; a suite that quietly never runs it is back to the state that hid three bugs.

pytest needs PYTHONPATH=src here. The package is not installed and the egg-info is stale. Harmless locally, and it will bite the moment CI runs the integration tier.

The most valuable unfinished work is the provider staging tier — real DSNs, real deferral and retry, real feedback loops. I documented it and deliberately did not build it, because it needs credentials that belong in OpenBao and an operator's judgment about which account to point it at. Until it exists, email-connect has never seen a bounce it did not write itself.

7.6 KiB Raw Blame History