REUSE-WP-0019-T06: hub freshness monitoring, docs, close workplan
Some checks failed
CI Smoke / host-smoke (push) Successful in 1s
CI Smoke / container-smoke (push) Successful in 2s
ci / validate-registry (push) Has been cancelled
Build and Publish Container Image / build-and-push (push) Successful in 1m4s

reuse_surface/stats.py: _hub_summary() now reports composed_at, stale,
age_days, freshness_threshold_days (REUSE_SURFACE_FRESHNESS_DAYS env,
default 7), and a computed stale_warning. New hub_client.hub_federated()
backs it. format_stats_markdown surfaces a STALE marker when triggered.

.forgejo/workflows/ci.yml: new informational (non-failing) hub freshness
check against the live production hub on every push -- prints a
:⚠️: annotation when stale, never fails the build.

docs/RegistryFederation.md: new section tying together the webhook (T02),
scheduled fallback (T03), and freshness visibility (T06) into one
explanation. docs/deploy/reuse-kubernetes.md: updated for the T03 Forgejo
migration and the now-automated image.yaml build; image promotion
checklist updated for the known /health ingress bug (verify via
/v1/repos or /v1/federated instead).

14 new pytest cases, 173 total pass. Live-verified against production:
reuse-surface stats correctly showed composed_at/age_days for the real
federated index. Separately discovered and confirmed (via a live signed
webhook test) that reuse-surface-env moving to ExternalSecret/OpenBao
custody (railiance-apps commit 706f6c7, found while updating these docs)
did not break the T02/T03 webhook -- the synced value still matches what
the hub actually uses.

REUSE-WP-0019 is now fully complete (T01-T06). SCOPE.md and
docs/IntentScopeGapAnalysis.md updated to reflect closure.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-07-08 00:09:58 +02:00
parent 706f6c70fe
commit f9d957a221
10 changed files with 328 additions and 23 deletions

View file

@ -240,4 +240,5 @@ See §4 and archived workplans `workplans/archived/`.
| 2026-06-16 | **REUSE-WP-0016** closed priority 28 (interactive `maintain`, `--auto`, templates) |
| 2026-07-07 | **REUSE-WP-0018** partially closed priority 29 (`plan-check` deterministic matching + State Hub capability-request bridge; LLM rerank and ecosystem rollout remain open) |
| 2026-07-07 | **REUSE-WP-0018-T05** closed following REUSE-WP-0017 completion (61 published capabilities); only T03 (LLM rerank) remains open under priority 29 |
| 2026-07-07 | **REUSE-WP-0018-T03** closed once `llm-connect` came up locally; priority 29 fully closed (all six T01T06 tasks shipped) |
| 2026-07-07 | **REUSE-WP-0018-T03** closed once `llm-connect` came up locally; priority 29 fully closed (all six T01T06 tasks shipped) |
| 2026-07-08 | **REUSE-WP-0019** finished (all six T01T06 tasks shipped): forge host abstraction + `migrate-host`, hub webhook/staleness tracking, this repo's own Forgejo migration + CI, reuse telemetry store/recording, telemetry aggregation into R-axis evidence, and freshness monitoring in `stats`/CI. Closes SCOPE "not possible yet" item *automatic hub refresh* |

View file

@ -262,6 +262,32 @@ curl -fsS "$REUSE_SURFACE_URL/v1/federated" | jq '.capabilities | length'
Read endpoints are public; writes require `REUSE_SURFACE_TOKEN` (Bearer). API
spec: `specs/FederationHubAPI.md`.
### Automatic recompose and freshness (REUSE-WP-0019-T02/T03/T06)
The hub recomposes automatically rather than waiting for a manual
`reuse-surface federation compose --refresh`:
- **Forgejo org webhook** (primary): a single org-level webhook on `coulomb`
fires on every push, HMAC-signed. `POST /v1/webhooks/forgejo` verifies the
signature, checks whether the push touched `registry/indexes/`, and — only
if so — recomposes. It never parses pushed file content, only paths.
- **Scheduled fallback**: `.forgejo/workflows/recompose-fallback.yaml` in
this repo calls `POST /v1/federated/compose` (token-auth) every 6 hours,
in case a webhook delivery is ever missed.
- **Freshness visibility**: `GET /v1/federated` carries `composed_at`
(timestamp of the last *forced* recompose) and `stale` (set by the
webhook, cleared on the next successful recompose). `reuse-surface stats`
surfaces both plus an `age_days`/`stale_warning` computed against a
threshold (`REUSE_SURFACE_FRESHNESS_DAYS`, default 7 days) — this repo's
own CI runs an informational (non-failing) freshness check on every push.
Setting up the org webhook or rotating its secret is an operator action on
the Forgejo instance and the cluster Secret, not something `reuse-surface`
itself automates — see `railiance-apps/docs/reuse-surface-on-railiance01.md`
(the authoritative operator runbook) and `docs/deploy/reuse-kubernetes.md`
for the current secret custody chain (OpenBao → `ExternalSecret`
`reuse-surface-env`).
### Hub vs local `sources.yaml`
| Workflow | When to use |

View file

@ -4,7 +4,16 @@ Companion to **RAILIANCE-WP-0007** (`railiance-apps` Helm release).
## Image
Repository: `gitea.coulomb.social/coulomb/reuse-surface` (Gitea org `coulomb`, repo `reuse-surface`).
This repo's own canonical remote migrated to Forgejo in REUSE-WP-0019-T03
(`origin``forgejo-remote:coulomb/reuse-surface.git`; the old Gitea copy
is kept read-only, not deleted). Currently deployed production image is
still `gitea.coulomb.social/coulomb/reuse-surface:e3ae22e` (built manually
before the migration, per RAILIANCE-WP-0007) — this doc's manual build
commands still target that registry since that's what's actually live.
The repo also builds `forgejo.coulomb.social/coulomb/reuse-surface:latest`
automatically on every push via `.forgejo/workflows/image.yaml`; switching
the deployed registry over is a deliberate follow-up, not done as part of
T06.
```bash
docker build -t gitea.coulomb.social/coulomb/reuse-surface:<tag> .
@ -103,13 +112,31 @@ cert-manager / companion operator logs.
### Image promotion checklist
1. Tag image from CI commit: `gitea.coulomb.social/coulomb/reuse-surface:<sha>`.
2. Run `pytest -q` and `reuse-surface validate` on that commit.
3. Update Helm values image tag in `railiance-apps`.
4. Deploy to Railiance01; verify `GET /health` and `GET /v1/repos`.
5. Smoke `reuse-surface hub list` and `GET /v1/federated` capability count.
1. Tag image from CI commit. `.forgejo/workflows/image.yaml` already builds
and pushes `forgejo.coulomb.social/coulomb/reuse-surface:main-<short-sha>`
automatically on every push that touches `Dockerfile`/`reuse_surface/**`/
`schemas/**`/`pyproject.toml` — verify it's green rather than building
by hand, unless promoting from the still-live `gitea.coulomb.social`
registry (current production posture).
2. Run `pytest -q` and `reuse-surface validate` on that commit (CI already
does this; re-verify locally if promoting outside CI).
3. Update Helm values image tag in `railiance-apps`
(`helm/reuse-surface-values.yaml`).
4. Deploy to Railiance01 (`make reuse-deploy`); verify `GET /v1/federated`
and `GET /v1/repos` (not `GET /health` — see the known ingress routing
issue below).
5. Smoke `reuse-surface hub list` and `GET /v1/federated` capability count;
check `reuse-surface stats` shows a fresh `composed_at` post-deploy.
6. Record image digest in workplan or progress log.
**Known issue (found 2026-07-07, not fixed):** the public ingress's
exact-path `/health` rule 404s (shadowed by the catch-all `/` rule to the
landing page) — confirmed ingress-layer only via direct port-forward and
the Deployment's own passing readiness/liveness probes. Use
`GET /v1/repos` or `GET /v1/federated` for external verification instead.
Flagged to `railiance-apps`; not this repo's fix to make (shared ingress
template).
### SQLite vs Postgres (cnpg) — decision criteria
Stay on SQLite while: