inbound-library/workplans/ILIB-WP-0002-library-infrastructure.md

---
id: ILIB-WP-0002
type: workplan
title: "Establish the inbound-library infrastructure"
domain: infotech
repo: inbound-library
status: active
flavor: implementation
owner: claude
topic_slug: custodian
created: "2026-10-05"
updated: "2026-10-05"
state_hub_workstream_id: "5e750e55-e5f0-5cfc-9d45-192b91257097"
---

# Establish the inbound-library infrastructure

Build the first working library, following INTENT.md:
- read-only IMAP intake;
- an append-only item store;
- file-backed topics, sources and subscriptions;
- notices through State Hub messages, a pull CLI and digest files;
- a systemd user timer on the workstation.

Python 3.11+ standard library only. The first consumer is intelligence-radar
(IR-WP-0010).

## Repository baseline and data contracts

```task
id: ILIB-WP-0002-T01
status: done
priority: high
state_hub_task_id: "36f19bb9-045f-59fd-9b7c-509cb7dd00f8"
```

Create the `inbound_library` package, `python3 -m inbound_library` with
`--help`, a Makefile (`check`, `test`, `demo`, `sync`) and `docs/data-contracts.md`.
The contracts define item, provenance, source, topic, subscription and notice,
with schema versions. Record the install, test and run commands in AGENTS.md
(taken over from ILIB-WP-0001-T02).

Acceptance: `make check` runs the unit tests on synthetic fixtures. The contracts
document every field that a consumer may rely on.

Done 2026-10-05: package `inbound_library`, CLI, Makefile, `docs/data-contracts.md` and the AGENTS.md commands are in place. `data/` is gitignored because mail can be private.

## Append-only item store

```task
id: ILIB-WP-0002-T02
status: done
priority: high
state_hub_task_id: "4bbe176a-b210-5c20-9510-c90029bf2425"
```

Store original bytes under `data/items/<sha256>`, with a provenance record for
each delivery. Identical bytes delivered twice give one item with two
deliveries. Derive plain text and metadata into separate, rebuildable records.
Add an integrity check that rehashes every retained item. Never overwrite and
never delete. Items without provenance go to quarantine.

Acceptance: the tests prove there is no overwrite, that deduplication works,
that tampering is detected and that quarantine routing works.

Done 2026-10-05: `store.py` is in place. Tests cover no-overwrite, deduplication, tamper detection and quarantine.

## Read-only IMAP intake

```task
id: ILIB-WP-0002-T03
status: done
priority: high
state_hub_task_id: "dd06da59-5773-53fa-aba3-dc79dff23c28"
```

Poll one dedicated mailbox with `imaplib`, over TLS, using SELECT read-only and
BODY.PEEK[]. Credentials come only from environment variable names set in the
config. Keep a UIDVALIDITY/UID cursor in the library. Retain the full RFC 822
message as the item, and record each attachment as its own item linked to the
message. Map the sender (and optionally a List-Id or subject pattern) to a
registered source. Unknown senders are quarantined and not routed. Record
DKIM/SPF authentication results as provenance, not as trust. Reuse the
read-only IMAP approach of email-connect EMAIL-WP-0002 where that helps.

Acceptance: the tests run against a fixture mailbox. A live run is part of T07.

Done 2026-10-05: `intake.py` is in place. It selects read-only with BODY.PEEK[] and keeps a UIDVALIDITY/UID cursor. UIDs are left out of delivery records so rescans do not duplicate. Attachments are retained as items. Unknown senders are quarantined and re-routed (`requeue_quarantine`) once a source is registered. The tests use a fixture mailbox.

## File-backed registries

```task
id: ILIB-WP-0002-T04
status: done
priority: high
state_hub_task_id: "4a2cbba2-55d8-5a0e-bd23-74dbfdb7ae61"
```

Add `registry/topics.json` (a versioned vocabulary), `registry/sources/*.json`
(owner repo, channel, match rules, topics, expected cadence,
`visibility: shared|restricted`) and `registry/subscriptions/<repo>.json`
(topics, sources, channels, cadence, agent name, digest path). A consumer
registers a source from its own research with a commit. The commit is checked
for unknown topics, duplicate match rules and restricted sources that leak to
other repos.

Acceptance: `check` rejects invalid registries. A source registered by repo A
is matched for repo B through a shared topic.

Done 2026-10-05: `registry.py` validates topics, sources and subscriptions, including the restricted-source leak check. A test shows a source from repo A matched for repo B through a shared topic.

## Matching and notification

```task
id: ILIB-WP-0002-T05
status: done
priority: high
state_hub_task_id: "5da65f55-7608-55b6-9378-7bef3ea8e4ca"
```

Match new items to subscriptions by source and topic. Then deliver:
- a State Hub agent message (`POST /messages/`) per subscriber and run, listing
  the new item ids, sources and pull commands;
- a pull CLI: `new --repo … --topic … --since …`, `show <item>` and `get <item>`;
- a Markdown digest file per subscriber and cadence.

Record each notice in `data/notices/`, so a retry never notifies twice. If the
hub is unreachable, keep the notice pending and retry on the next run.

Acceptance: the tests prove idempotent delivery and recovery when the hub is
unreachable.

Done 2026-10-05: `notify.py` is in place. Notices are deterministic. Hub messages give a pull command that works from any repo. Tests cover the `new`/`show`/`get` commands, digest files, idempotent delivery and a hub outage that stays pending.

## Scheduling, health and source silence

```task
id: ILIB-WP-0002-T06
status: done
priority: medium
state_hub_task_id: "01144f1e-b3ea-50b6-bf7c-32e9a29d7213"
```

Add a `run-due` that does the intake, then matching and delivery, under a lock.
Add a systemd user service and timer in `ops/systemd/`, and a `health` command.
Health reports:
- the last scan;
- items per source;
- sources that have stayed silent past their expected cadence;
- quarantine size;
- pending notices.

Acceptance: a silent source is reported, and repeated timer wakes are safe.

Done 2026-10-05: `run-due` runs under a lock. The systemd units are in `ops/systemd/`, and the service reads `~/.config/inbound-library/imap.env`. Tests cover `health`, including silent-source detection. 14 tests pass.

## Provision the dedicated mailbox

```task
id: ILIB-WP-0002-T07
status: wait
blocking_reason: "Needs the unattended OpenBao AppRole lane (ILIB-WP-0003)."
priority: high
state_hub_task_id: "1e2ab60c-86ef-5e1c-9b94-1f9f1b3182ba"
```

Operator decisions:
- which mailbox (a new address, or a Gmail alias plus label or folder);
- how its IMAP credential is delivered (an app password through the
  secrets-engine path, never committed).

Then do a live read-only scan.

Acceptance: one real mail retained with provenance. No mailbox state changed.

Decided 2026-10-05: the mailbox is `library@coulomb.social`. coulomb.social mail is
hosted by IONOS (MX `mx00/mx01.ionos.de`), the same provider as email-connect's
`smtp.ionos.de` lane. The library therefore reads `imap.ionos.de:993` over TLS.

Remaining operator steps:
- confirm the IONOS mailbox exists;
- deliver its password through the secrets-engine/OpenBao path as
  `INBOUND_LIBRARY_IMAP_PASSWORD`, with `INBOUND_LIBRARY_IMAP_USER=library@coulomb.social`.

Do not commit or log the password.

Prepared 2026-10-05 in railiance-platform:
- `scripts/inbound_library_imap_entry.py`, with tests and
  `docs/inbound-library-imap-custody.md`;
- it creates the password-free KV entry `platform/workloads/inbound-library/imap`
  (CAS=0), a sibling of the telemetry SMTP entry for platform@coulomb.social.

The founder runs it attended through warden, then adds `IMAP_PASSWORD` as
version 2.

2026-10-05: the founder reports that the entry was created and `IMAP_PASSWORD`
was added. This is not yet verified natively. A live read-only scan is still
needed for acceptance.

2026-10-05: the founder rejected an env file as a credential cache. The live
credential now comes from an ops-mason AppRole lane, fetched on every run
(ILIB-WP-0003). This task completes with ILIB-WP-0003-T06.