--- id: ILIB-WP-0002 type: workplan title: "Establish the inbound-library infrastructure" domain: infotech repo: inbound-library status: finished flavor: implementation owner: claude topic_slug: custodian created: "2026-10-05" updated: "2026-10-05" state_hub_workstream_id: "5e750e55-e5f0-5cfc-9d45-192b91257097" --- # Establish the inbound-library infrastructure Build the first working library, following INTENT.md: - read-only IMAP intake; - an append-only item store; - file-backed topics, sources and subscriptions; - notices through State Hub messages, a pull CLI and digest files; - a systemd user timer on the workstation. Python 3.11+ standard library only. The first consumer is intelligence-radar (IR-WP-0010). ## Repository baseline and data contracts ```task id: ILIB-WP-0002-T01 status: done priority: high state_hub_task_id: "36f19bb9-045f-59fd-9b7c-509cb7dd00f8" ``` Create the `inbound_library` package, `python3 -m inbound_library` with `--help`, a Makefile (`check`, `test`, `demo`, `sync`) and `docs/data-contracts.md`. The contracts define item, provenance, source, topic, subscription and notice, with schema versions. Record the install, test and run commands in AGENTS.md (taken over from ILIB-WP-0001-T02). Acceptance: `make check` runs the unit tests on synthetic fixtures. The contracts document every field that a consumer may rely on. Done 2026-10-05: package `inbound_library`, CLI, Makefile, `docs/data-contracts.md` and the AGENTS.md commands are in place. `data/` is gitignored because mail can be private. ## Append-only item store ```task id: ILIB-WP-0002-T02 status: done priority: high state_hub_task_id: "4bbe176a-b210-5c20-9510-c90029bf2425" ``` Store original bytes under `data/items/`, with a provenance record for each delivery. Identical bytes delivered twice give one item with two deliveries. Derive plain text and metadata into separate, rebuildable records. Add an integrity check that rehashes every retained item. Never overwrite and never delete. Items without provenance go to quarantine. Acceptance: the tests prove there is no overwrite, that deduplication works, that tampering is detected and that quarantine routing works. Done 2026-10-05: `store.py` is in place. Tests cover no-overwrite, deduplication, tamper detection and quarantine. ## Read-only IMAP intake ```task id: ILIB-WP-0002-T03 status: done priority: high state_hub_task_id: "dd06da59-5773-53fa-aba3-dc79dff23c28" ``` Poll one dedicated mailbox with `imaplib`, over TLS, using SELECT read-only and BODY.PEEK[]. Credentials come only from environment variable names set in the config. Keep a UIDVALIDITY/UID cursor in the library. Retain the full RFC 822 message as the item, and record each attachment as its own item linked to the message. Map the sender (and optionally a List-Id or subject pattern) to a registered source. Unknown senders are quarantined and not routed. Record DKIM/SPF authentication results as provenance, not as trust. Reuse the read-only IMAP approach of email-connect EMAIL-WP-0002 where that helps. Acceptance: the tests run against a fixture mailbox. A live run is part of T07. Done 2026-10-05: `intake.py` is in place. It selects read-only with BODY.PEEK[] and keeps a UIDVALIDITY/UID cursor. UIDs are left out of delivery records so rescans do not duplicate. Attachments are retained as items. Unknown senders are quarantined and re-routed (`requeue_quarantine`) once a source is registered. The tests use a fixture mailbox. ## File-backed registries ```task id: ILIB-WP-0002-T04 status: done priority: high state_hub_task_id: "4a2cbba2-55d8-5a0e-bd23-74dbfdb7ae61" ``` Add `registry/topics.json` (a versioned vocabulary), `registry/sources/*.json` (owner repo, channel, match rules, topics, expected cadence, `visibility: shared|restricted`) and `registry/subscriptions/.json` (topics, sources, channels, cadence, agent name, digest path). A consumer registers a source from its own research with a commit. The commit is checked for unknown topics, duplicate match rules and restricted sources that leak to other repos. Acceptance: `check` rejects invalid registries. A source registered by repo A is matched for repo B through a shared topic. Done 2026-10-05: `registry.py` validates topics, sources and subscriptions, including the restricted-source leak check. A test shows a source from repo A matched for repo B through a shared topic. ## Matching and notification ```task id: ILIB-WP-0002-T05 status: done priority: high state_hub_task_id: "5da65f55-7608-55b6-9378-7bef3ea8e4ca" ``` Match new items to subscriptions by source and topic. Then deliver: - a State Hub agent message (`POST /messages/`) per subscriber and run, listing the new item ids, sources and pull commands; - a pull CLI: `new --repo … --topic … --since …`, `show ` and `get `; - a Markdown digest file per subscriber and cadence. Record each notice in `data/notices/`, so a retry never notifies twice. If the hub is unreachable, keep the notice pending and retry on the next run. Acceptance: the tests prove idempotent delivery and recovery when the hub is unreachable. Done 2026-10-05: `notify.py` is in place. Notices are deterministic. Hub messages give a pull command that works from any repo. Tests cover the `new`/`show`/`get` commands, digest files, idempotent delivery and a hub outage that stays pending. ## Scheduling, health and source silence ```task id: ILIB-WP-0002-T06 status: done priority: medium state_hub_task_id: "01144f1e-b3ea-50b6-bf7c-32e9a29d7213" ``` Add a `run-due` that does the intake, then matching and delivery, under a lock. Add a systemd user service and timer in `ops/systemd/`, and a `health` command. Health reports: - the last scan; - items per source; - sources that have stayed silent past their expected cadence; - quarantine size; - pending notices. Acceptance: a silent source is reported, and repeated timer wakes are safe. Done 2026-10-05: `run-due` runs under a lock. The systemd units are in `ops/systemd/`, and the service reads `~/.config/inbound-library/imap.env`. Tests cover `health`, including silent-source detection. 14 tests pass. ## Provision the dedicated mailbox ```task id: ILIB-WP-0002-T07 status: done priority: high state_hub_task_id: "1e2ab60c-86ef-5e1c-9b94-1f9f1b3182ba" ``` Operator decisions: - which mailbox (a new address, or a Gmail alias plus label or folder); - how its IMAP credential is delivered (an app password through the secrets-engine path, never committed). Then do a live read-only scan. Acceptance: one real mail retained with provenance. No mailbox state changed. Decided 2026-10-05: the mailbox is `library@coulomb.social`. coulomb.social mail is hosted by IONOS (MX `mx00/mx01.ionos.de`), the same provider as email-connect's `smtp.ionos.de` lane. The library therefore reads `imap.ionos.de:993` over TLS. Remaining operator steps: - confirm the IONOS mailbox exists; - deliver its password through the secrets-engine/OpenBao path as `INBOUND_LIBRARY_IMAP_PASSWORD`, with `INBOUND_LIBRARY_IMAP_USER=library@coulomb.social`. Do not commit or log the password. Prepared 2026-10-05 in railiance-platform: - `scripts/inbound_library_imap_entry.py`, with tests and `docs/inbound-library-imap-custody.md`; - it creates the password-free KV entry `platform/workloads/inbound-library/imap` (CAS=0), a sibling of the telemetry SMTP entry for platform@coulomb.social. The founder runs it attended through warden, then adds `IMAP_PASSWORD` as version 2. 2026-10-05: the founder reports that the entry was created and `IMAP_PASSWORD` was added. This is not yet verified natively. A live read-only scan is still needed for acceptance. 2026-10-05: the founder rejected an env file as a credential cache. The live credential now comes from an ops-mason AppRole lane, fetched on every run (ILIB-WP-0003). This task completes with ILIB-WP-0003-T06. Done 2026-10-05: live read-only scan through the ILIB-WP-0003 AppRole lane retained 2 real mails with provenance.