Repointing the production hub's 50 Gitea-hosted federation sources to Forgejo ahead of the 2026-08-31 CoulombCore retirement took /v1/federated to HTTP 500. One member index (evidence-binder) has capability rows with no `id`, and compose_federated_index dereferenced item["id"] unguarded. Its Gitea copy was a stale snapshot returning a non-mapping, so those rows had never been parsed. A single malformed member index must not take down the whole endpoint. Extract _read_index_entries(): unparseable YAML, a non-mapping body, an empty file, and a non-list `capabilities` each degrade to a warning and an empty row list, and rows without an `id` are skipped individually. A failed source stays listed with count 0 so it remains visible to operators rather than silently disappearing. Also fix wall-clock rot in tests/test_plan_check.py, which was already failing at clean HEAD: three tests pinned the compose date to a literal that has now aged past STALE_DAYS. Add workplan REUSE-WP-0020 covering the full cutover. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
241 lines
9.2 KiB
Markdown
241 lines
9.2 KiB
Markdown
---
|
|
id: REUSE-WP-0020
|
|
type: workplan
|
|
title: "CoulombCore retirement cutover: federation sources, image, and compose resilience"
|
|
domain: infotech
|
|
repo: reuse-surface
|
|
status: active
|
|
owner: claude
|
|
topic_slug: helix-forge
|
|
created: "2026-08-20"
|
|
updated: "2026-08-20"
|
|
---
|
|
|
|
# CoulombCore retirement cutover
|
|
|
|
CoulombCore is switched off **2026-08-31** (operator decision, 2026-08-20).
|
|
`gitea.coulomb.social` runs on it. Raised by `prj-state-hub-retirement`
|
|
(SHR-WP-0002-T07) as a stale-image report; investigation found a second,
|
|
more urgent problem underneath it.
|
|
|
|
**Two independent failures**, not one:
|
|
|
|
1. **Federation source data** — the production hub's *registrations* pointed
|
|
50 of 61 sources at `gitea.coulomb.social`. This breaks on 08-31 with
|
|
nobody restarting anything. It is database state, not code:
|
|
`registry/federation/sources.yaml` at HEAD has been 61/61 Forgejo since
|
|
`d1c1313` (RAIL-HO-WP-0006).
|
|
2. **Deployed image** — `railiance-apps/helm/reuse-surface-values.yaml` pins
|
|
`image.tag: "e3ae22e"`. The chart's `repository` was already migrated to
|
|
Forgejo in `railiance-apps@04be416`, but `e3ae22e` is commit-dated
|
|
2026-07-07 18:25 and CI only began publishing to Forgejo at 21:25 the same
|
|
day — so a Forgejo `:e3ae22e` tag most likely never existed. Any restart,
|
|
reschedule, or node reboot risks `ImagePullBackOff` **now**, not on 08-31.
|
|
|
|
Item 1 was resolved on 2026-08-20 (see T01). Doing so exposed a third issue:
|
|
a single malformed member index takes the whole federated endpoint down.
|
|
|
|
**Baseline vector:** `D5 / A4 / C5 / R3` — unchanged by this workplan;
|
|
this is operational continuity, not capability growth.
|
|
|
|
## Scope boundary
|
|
|
|
The Deployment manifest is **not in this repo**. It lives in `railiance-apps`
|
|
(`charts/reuse-surface/`, `helm/reuse-surface-values.yaml`, RAILIANCE-WP-0007).
|
|
This repo owns the image contents, the deploy guide, and the federation data;
|
|
applying the manifest change requires a working `KUBECONFIG=~/.kube/config-hosteurope`,
|
|
which is not reachable from the workstation without an ops-bridge tunnel.
|
|
|
|
---
|
|
|
|
## Repoint Production Federation Sources To Forgejo
|
|
|
|
```task
|
|
id: REUSE-WP-0020-T01
|
|
status: done
|
|
priority: high
|
|
```
|
|
|
|
**Completed 2026-08-20.** Updated 50 enabled hub registrations from
|
|
`gitea.coulomb.social/<repo>/raw/main/...` to
|
|
`forgejo.coulomb.social/<repo>/raw/branch/main/...` via
|
|
`reuse-surface hub update --repo <r> --url <new>`.
|
|
|
|
Notes for the record:
|
|
|
|
- All 50 new URLs were pre-verified `200` before any write.
|
|
- The canonical Forgejo raw form is `/raw/branch/main/`. `/raw/main/` answers
|
|
`303` and works only because `urllib.request.urlopen` follows redirects —
|
|
`registry/federation/sources.yaml` still uses the redirecting form. Harmless,
|
|
but prefer the canonical form in new writes.
|
|
- One registration (`inter-hub`, already `disabled`) still carries a Gitea URL.
|
|
Its Forgejo equivalent answers `307`, so it was left alone rather than
|
|
repointed to something unverified. It contributes nothing while disabled.
|
|
|
|
Verified: `GET /v1/federated` → 60 sources, all Forgejo, 62 capabilities.
|
|
|
|
## Harden Compose Against A Malformed Member Index
|
|
|
|
```task
|
|
id: REUSE-WP-0020-T02
|
|
status: done
|
|
priority: high
|
|
```
|
|
|
|
T01 briefly took `/v1/federated` to HTTP 500. Root cause: `evidence-binder`'s
|
|
live index has two capability entries with **no `id` field**, and
|
|
`compose_federated_index` does `cap_id = item["id"]` unguarded. Its Gitea copy
|
|
was a stale snapshot returning a non-mapping, so the entries had never been
|
|
parsed before.
|
|
|
|
**One bad member index must not take down the federated endpoint.** In
|
|
`reuse_surface/federation.py::compose_federated_index`, guard the per-source
|
|
body so a failure degrades to a warning and the remaining sources still
|
|
compose:
|
|
|
|
| Failure | Current | Wanted |
|
|
|---|---|---|
|
|
| `item["id"]` missing | `KeyError`, 500 | skip entry, warn, keep source |
|
|
| `yaml.safe_load` → non-mapping | `AttributeError`, 500 | skip source, warn |
|
|
| `yaml.safe_load` → `None` (empty file) | `AttributeError`, 500 | skip source, warn |
|
|
| YAML parse error | `YAMLError`, 500 | skip source, warn |
|
|
| `sorted(key=item["id"])` | `KeyError` | unreachable once entries are filtered |
|
|
|
|
Confirmed present at HEAD, not only in the deployed image — deploying HEAD
|
|
would **not** have fixed this. Land T02 before T04.
|
|
|
|
Pytest in `tests/test_federation.py`: member index with a missing `id`, an
|
|
empty file, a non-mapping body, and unparseable YAML — each composes the
|
|
other sources successfully and emits a warning naming the source.
|
|
|
|
**Done 2026-08-20.** Extracted `_read_index_entries()`; every read failure now
|
|
returns an empty row list plus a warning, and rows without an `id` are skipped
|
|
individually. A failed source stays in `sources` with `count: 0` rather than
|
|
vanishing, so operators can still see it. Six tests added. Verified against the
|
|
real `evidence-binder` index: composes clean with two skip warnings instead of
|
|
raising.
|
|
|
|
## Fix The evidence-binder Capability Index
|
|
|
|
```task
|
|
id: REUSE-WP-0020-T03
|
|
status: todo
|
|
priority: medium
|
|
```
|
|
|
|
Cross-repo, in `~/evidence-binder`. Both entries in
|
|
`registry/indexes/capabilities.yaml` ("Evidence-to-target binding",
|
|
"Visual-guide rect registry") lack the required `id`. Add ids consistent with
|
|
the registry naming convention, confirm against `schemas/capability.schema.yaml`,
|
|
and check whether `registry/capabilities/*.md` entries exist behind them or
|
|
whether the index rows are orphans.
|
|
|
|
Route rather than edit directly if that repo has an owning agent.
|
|
|
|
## Re-Enable The evidence-binder Source
|
|
|
|
```task
|
|
id: REUSE-WP-0020-T04
|
|
status: wait
|
|
priority: medium
|
|
```
|
|
|
|
Blocked on T02 and T03. `evidence-binder` was set `--no-enabled` on the
|
|
production hub to restore service. Once its index is valid and compose is
|
|
hardened, re-enable and confirm `/v1/federated` returns 61 sources.
|
|
|
|
## Repoint The Production Deployment To A Forgejo Image
|
|
|
|
```task
|
|
id: REUSE-WP-0020-T05
|
|
status: todo
|
|
priority: high
|
|
```
|
|
|
|
Deadline-bound: must land before **2026-08-31**.
|
|
|
|
1. Verify which tags actually exist in
|
|
`forgejo.coulomb.social/coulomb/reuse-surface` (registry API needs auth;
|
|
anonymous `GET /v2/.../tags/list` returns `401`) and that
|
|
`.forgejo/workflows/image.yaml` is green on `main`.
|
|
2. Bump `image.tag` in `railiance-apps/helm/reuse-surface-values.yaml` from
|
|
`e3ae22e` to a verified Forgejo tag at current HEAD.
|
|
3. Correct `docs/deploy/reuse-kubernetes.md`, which still describes Gitea as
|
|
the live registry and its manual `docker build`/`push` commands still
|
|
target `gitea.coulomb.social`.
|
|
4. Apply from a host with cluster access; verify rollout, `/v1/federated`,
|
|
and that the PVC at `/data` survived.
|
|
|
|
Ships REUSE-WP-0019 **T04/T05/T06** as a side effect — telemetry store,
|
|
R-axis aggregation, and hub freshness monitoring were closed as finished but
|
|
landed after the deployed commit and have never run in production.
|
|
`GET /v1/reuse-events` returning `404` on production confirms this.
|
|
|
|
Review the six weeks of change between `e3ae22e` and HEAD before applying.
|
|
|
|
## Correct Hub Freshness And Health Routing
|
|
|
|
```task
|
|
id: REUSE-WP-0020-T06
|
|
status: todo
|
|
priority: low
|
|
```
|
|
|
|
Two smaller production inconsistencies found while verifying T01:
|
|
|
|
- `GET /health` returns nginx `404` through the ingress, though
|
|
`docs/deploy/reuse-kubernetes.md` documents it as the liveness path and the
|
|
landing page links it. Only `/v1/*` routes. Pod probes hit the container
|
|
directly so the service is unaffected, but the documented URL is wrong —
|
|
fix the ingress route or the docs, whichever matches intent.
|
|
- `composed_at` stayed at `2026-08-20T20:44:12` with `stale: false` across a
|
|
recompose that demonstrably changed output (61 → 62 capabilities). The
|
|
freshness timestamp is not tracking recomposes. Re-check after T05, since
|
|
REUSE-WP-0019-T06 touches exactly this and is not deployed.
|
|
|
|
## Refresh SCOPE.md Standard Sections
|
|
|
|
```task
|
|
id: REUSE-WP-0020-T07
|
|
status: todo
|
|
priority: low
|
|
```
|
|
|
|
Unrelated to the retirement, recorded so it is not lost. The hub's repo scope
|
|
check reports `C5b`/`C5c` warnings: `SCOPE.md` is missing the standard H2
|
|
sections *Relevant When*, *Not Relevant When*, *How It Fits*, *Terminology*,
|
|
*Related / Overlapping*, *Provided Capabilities*, and carries no fenced
|
|
capability block.
|
|
|
|
## Fix Wall-Clock Rot In The Plan-Check Test Suite
|
|
|
|
```task
|
|
id: REUSE-WP-0020-T08
|
|
status: done
|
|
priority: medium
|
|
```
|
|
|
|
Found while verifying T02: `tests/test_plan_check.py` was already failing at
|
|
clean HEAD, independent of any change here. Three tests pinned the federated
|
|
index compose date to the literal `"2026-07-06"`, and
|
|
`test_run_plan_check_reuse_verdict` asserts no staleness warning — so the test
|
|
passed when written and started failing once wall-clock time drifted past
|
|
`STALE_DAYS`. **CI is red today for this reason alone**, which matters because
|
|
T05 calls for verifying the image workflow is green.
|
|
|
|
**Done 2026-08-20.** Replaced the literals with a `_recent_date()` helper
|
|
derived from `STALE_DAYS`. Full suite: 179 passed.
|
|
|
|
---
|
|
|
|
## Verification
|
|
|
|
```bash
|
|
python3 -m pytest tests/test_federation.py -q
|
|
python3 -m reuse_surface.cli validate
|
|
python3 -m reuse_surface.cli federation compose --refresh
|
|
REUSE_SURFACE_URL=https://reuse.coulomb.social python3 -m reuse_surface.cli hub list
|
|
curl -s https://reuse.coulomb.social/v1/federated | python3 -m json.tool | head -20
|
|
```
|
|
|
|
No source may reference `gitea.coulomb.social` after this workplan closes.
|