inbound-library/workplans/ILIB-WP-0002-library-infrastructure.md

id	type	title	domain	repo	status	flavor	owner	topic_slug	created	updated	state_hub_workstream_id
ILIB-WP-0002	workplan	Establish the inbound-library infrastructure	infotech	inbound-library	finished	implementation	claude	custodian	2026-10-05	2026-10-05	5e750e55-e5f0-5cfc-9d45-192b91257097

Establish the inbound-library infrastructure

Build the first working library, following INTENT.md:

• read-only IMAP intake;
• an append-only item store;
• file-backed topics, sources and subscriptions;
• notices through State Hub messages, a pull CLI and digest files;
• a systemd user timer on the workstation.

Python 3.11+ standard library only. The first consumer is intelligence-radar (IR-WP-0010).

Repository baseline and data contracts

id: ILIB-WP-0002-T01
status: done
priority: high
state_hub_task_id: "36f19bb9-045f-59fd-9b7c-509cb7dd00f8"

Create the inbound_library package, python3 -m inbound_library with --help, a Makefile (check, test, demo, sync) and docs/data-contracts.md. The contracts define item, provenance, source, topic, subscription and notice, with schema versions. Record the install, test and run commands in AGENTS.md (taken over from ILIB-WP-0001-T02).

Acceptance: make check runs the unit tests on synthetic fixtures. The contracts document every field that a consumer may rely on.

Done 2026-10-05: package inbound_library, CLI, Makefile, docs/data-contracts.md and the AGENTS.md commands are in place. data/ is gitignored because mail can be private.

Append-only item store

id: ILIB-WP-0002-T02
status: done
priority: high
state_hub_task_id: "4bbe176a-b210-5c20-9510-c90029bf2425"

Store original bytes under data/items/<sha256>, with a provenance record for each delivery. Identical bytes delivered twice give one item with two deliveries. Derive plain text and metadata into separate, rebuildable records. Add an integrity check that rehashes every retained item. Never overwrite and never delete. Items without provenance go to quarantine.

Acceptance: the tests prove there is no overwrite, that deduplication works, that tampering is detected and that quarantine routing works.

Done 2026-10-05: store.py is in place. Tests cover no-overwrite, deduplication, tamper detection and quarantine.

Read-only IMAP intake

id: ILIB-WP-0002-T03
status: done
priority: high
state_hub_task_id: "dd06da59-5773-53fa-aba3-dc79dff23c28"

Poll one dedicated mailbox with imaplib, over TLS, using SELECT read-only and BODY.PEEK[]. Credentials come only from environment variable names set in the config. Keep a UIDVALIDITY/UID cursor in the library. Retain the full RFC 822 message as the item, and record each attachment as its own item linked to the message. Map the sender (and optionally a List-Id or subject pattern) to a registered source. Unknown senders are quarantined and not routed. Record DKIM/SPF authentication results as provenance, not as trust. Reuse the read-only IMAP approach of email-connect EMAIL-WP-0002 where that helps.

Acceptance: the tests run against a fixture mailbox. A live run is part of T07.

Done 2026-10-05: intake.py is in place. It selects read-only with BODY.PEEK[] and keeps a UIDVALIDITY/UID cursor. UIDs are left out of delivery records so rescans do not duplicate. Attachments are retained as items. Unknown senders are quarantined and re-routed (requeue_quarantine) once a source is registered. The tests use a fixture mailbox.

File-backed registries

id: ILIB-WP-0002-T04
status: done
priority: high
state_hub_task_id: "4a2cbba2-55d8-5a0e-bd23-74dbfdb7ae61"

Add registry/topics.json (a versioned vocabulary), registry/sources/*.json (owner repo, channel, match rules, topics, expected cadence, visibility: shared|restricted) and registry/subscriptions/<repo>.json (topics, sources, channels, cadence, agent name, digest path). A consumer registers a source from its own research with a commit. The commit is checked for unknown topics, duplicate match rules and restricted sources that leak to other repos.

Acceptance: check rejects invalid registries. A source registered by repo A is matched for repo B through a shared topic.

Done 2026-10-05: registry.py validates topics, sources and subscriptions, including the restricted-source leak check. A test shows a source from repo A matched for repo B through a shared topic.

Matching and notification

id: ILIB-WP-0002-T05
status: done
priority: high
state_hub_task_id: "5da65f55-7608-55b6-9378-7bef3ea8e4ca"

Match new items to subscriptions by source and topic. Then deliver:

• a State Hub agent message (POST /messages/) per subscriber and run, listing the new item ids, sources and pull commands;
• a pull CLI: new --repo … --topic … --since …, show <item> and get <item>;
• a Markdown digest file per subscriber and cadence.

Record each notice in data/notices/, so a retry never notifies twice. If the hub is unreachable, keep the notice pending and retry on the next run.

Acceptance: the tests prove idempotent delivery and recovery when the hub is unreachable.

Done 2026-10-05: notify.py is in place. Notices are deterministic. Hub messages give a pull command that works from any repo. Tests cover the new/show/get commands, digest files, idempotent delivery and a hub outage that stays pending.

Scheduling, health and source silence

id: ILIB-WP-0002-T06
status: done
priority: medium
state_hub_task_id: "01144f1e-b3ea-50b6-bf7c-32e9a29d7213"

Add a run-due that does the intake, then matching and delivery, under a lock. Add a systemd user service and timer in ops/systemd/, and a health command. Health reports:

• the last scan;
• items per source;
• sources that have stayed silent past their expected cadence;
• quarantine size;
• pending notices.

Acceptance: a silent source is reported, and repeated timer wakes are safe.

Done 2026-10-05: run-due runs under a lock. The systemd units are in ops/systemd/, and the service reads ~/.config/inbound-library/imap.env. Tests cover health, including silent-source detection. 14 tests pass.

Provision the dedicated mailbox

id: ILIB-WP-0002-T07
status: done
priority: high
state_hub_task_id: "1e2ab60c-86ef-5e1c-9b94-1f9f1b3182ba"

Operator decisions:

• which mailbox (a new address, or a Gmail alias plus label or folder);
• how its IMAP credential is delivered (an app password through the secrets-engine path, never committed).

Then do a live read-only scan.

Acceptance: one real mail retained with provenance. No mailbox state changed.

Decided 2026-10-05: the mailbox is library@coulomb.social. coulomb.social mail is hosted by IONOS (MX mx00/mx01.ionos.de), the same provider as email-connect's smtp.ionos.de lane. The library therefore reads imap.ionos.de:993 over TLS.

Remaining operator steps:

• confirm the IONOS mailbox exists;
• deliver its password through the secrets-engine/OpenBao path as INBOUND_LIBRARY_IMAP_PASSWORD, with INBOUND_LIBRARY_IMAP_USER=library@coulomb.social.

Do not commit or log the password.

Prepared 2026-10-05 in railiance-platform:

• scripts/inbound_library_imap_entry.py, with tests and docs/inbound-library-imap-custody.md;
• it creates the password-free KV entry platform/workloads/inbound-library/imap (CAS=0), a sibling of the telemetry SMTP entry for platform@coulomb.social.

The founder runs it attended through warden, then adds IMAP_PASSWORD as version 2.

2026-10-05: the founder reports that the entry was created and IMAP_PASSWORD was added. This is not yet verified natively. A live read-only scan is still needed for acceptance.

2026-10-05: the founder rejected an env file as a credential cache. The live credential now comes from an ops-mason AppRole lane, fetched on every run (ILIB-WP-0003). This task completes with ILIB-WP-0003-T06.

Done 2026-10-05: live read-only scan through the ILIB-WP-0003 AppRole lane retained 2 real mails with provenance.