the-custodian/workplans/CUST-WP-0068-cache-only-work-record-recovery.md
codex 262ac4a6bb
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s
fix(workplan): rescope CUST-WP-0068-T07 — the orphan count rests on a broken field
backing_filename is written only by the bindings sync, which fix-consistency
runs before the registrar mints the UUID. Newly registered workplans therefore
record no backing file. Reproduced on CUST-WP-0068 itself.

278 of 800 workplans on central record no backing file, which is what ADR-010
predicted as the "broken links" class. The 23 cache-only orphans were counted
with the same field, so that number is an investigation starting point rather
than a work list.

T07 now repairs the ordering and re-measures before dispositioning anything.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 15:24:14 +02:00

256 lines
10 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
id: CUST-WP-0068
type: workplan
title: "Recover or disposition the cache-only work records so the retired hub can be discarded"
domain: infotech
repo: the-custodian
status: ready
owner: codex
created: "2026-08-25"
updated: "2026-08-25"
quality_dor: DoR-Ok
quality_dor_at: "2026-08-25"
quality_dor_by: codex
quality_dor_note: "Divergence measured exactly (297 workplans, 23 of them orphans), every remaining record classified with a named cause, per-repository blockers captured from real error messages, and the mechanism proven on live repositories before rollout."
state_hub_workstream_id: "c930b3ec-68b5-596e-ae72-d4ecaf7380e7"
---
# Recover or disposition the cache-only work records so the retired hub can be discarded
## Goal
Finish what `CUST-WP-0067-T10` started: get every work record that exists only
in the retired local hub either onto central or into a written disposition, so
the cache database stops being load-bearing and can be discarded. `ADR-010`
described this divergence in August 2026 and it has never been closed.
## Context
The local hub instance was retired under `CUST-WP-0067`, and its repository
records were fully recovered. Its **work records were not**. The database is
still running solely because 297 workplans and their tasks exist nowhere else.
Measured 2026-08-25, after recovering 129 of the original 426:
| Count | Class | Needs |
|---|---|---|
| 176 | non-empty projection on central | C-06 for the deterministic subset; the rest await an identifier decision |
| 58 | file does not parse as a workplan | content fix — 50 are `clay-borg` |
| 24 | bootstrap-eligible but blocked | per-repository unblocking |
| 23 | orphan, no backing file | written disposition |
| 16 | flagged as non-conforming ids | see the correction below |
**Correction carried forward.** Ten of those 16 are `ADHOC-YYYY-MM-DD`
identifiers, which are the documented ad-hoc convention — the classifier's
pattern was wrong, not the files. Only six are genuinely non-conforming:
`GLAS-0001`, `REIN-A-0001``0004`, `REIN-OW-0001`, plus
`RAIL-BS-ADHOC-2026-07-27` on a prefix the registry has retired. The ten adhocs
should be re-tested and will most likely fall to the class-F treatment.
**What already works.** `--bootstrap-empty-projection` registers a repository's
workplans while **preserving their existing identifiers**, verified on a live
repository before rollout: workplan and all four tasks landed with original
UUIDs and statuses, no file rewritten and no commit. All 403 cache-only
workplans that record a backing path still have that file on disk, so this is
recovery, not reconstruction.
**What does not work, and why.** Only 57 of 403 carry deterministic UUIDv5
identifiers. C-06 refuses non-derived identifiers as `C-03 FAIL — stale
reference`, so the ordinary consistency path cannot re-derive most of them.
Rewriting 345 identifiers fleet-wide was considered and rejected: it overlaps
`RMGR-WP-0005`'s ownership of identifier migration and would churn IDs across
~40 repositories to solve a problem bootstrap already solves for part of the set.
## Repair the registrar's self-defeating precondition
```task
id: CUST-WP-0068-T01
status: todo
priority: high
state_hub_task_id: "e8fbfa1a-3abf-5ec0-8abb-f63604dc23d5"
```
The registrar generates `WORK-RECORDS.md`, which dirties the worktree and then
fails its own git precondition. Several runs registered their records
successfully and *then* reported failure for the file they had just created;
retries report `git_precondition_failed` instead, which points away from the
real cause. Any operator running this at scale will misread it exactly as this
workplan's predecessor did.
Also surface error *messages*, not only codes. `bootstrap_source_invalid` covers
at least two unrelated causes — a repository with no parseable workplan, and a
task missing its projection UUID — and reading only the code merged them into
one wrong diagnosis.
Acceptance: a repository whose only change is the generated index completes in
one run; failure reports name the specific record and cause.
## Unblock the bootstrap-eligible repositories
```task
id: CUST-WP-0068-T02
status: todo
priority: high
state_hub_task_id: "a3564d51-2a9a-511a-8053-1a346a979bef"
```
Twenty-four workplans sit in repositories whose projection is empty and would
bootstrap cleanly but for a specific blocker:
| Repository | Cause |
|---|---|
| `rapp-openbao`, `policy-nexus`, `whitehat-security` | a task has no authoritative projection UUID; task ids must be minted first |
| `ops-mason` | projection is no longer empty — partly registered already, so it needs the class-F path |
| `ground-game` | genuine uncommitted work (`SCOPE.md`, `Modes.csv`); owner must land or stash it |
| `zone-engine` | untracked `DECISIONS.md`; owner content, not ours to commit |
Acceptance: each repository either bootstraps or has its blocker recorded as an
owner action.
## Rule the six non-conforming record identifiers
```task
id: CUST-WP-0068-T03
status: todo
priority: medium
state_hub_task_id: "e1eb31d5-cece-59ac-9ccc-8631fc025870"
```
`GLAS-0001`, `REIN-A-0001``0004`, `REIN-OW-0001` and `RAIL-BS-ADHOC-2026-07-27`
are rejected by the canon registry. Renaming a work record changes its identity,
so this is an owner decision per repository, not a rewrite.
Note `rein-aharness` and `rein-openweights` are the `rein` concept that
`agent-harness` was dropped in favour of (`CUST-WP-0067-T05`), so their records
matter more than their age suggests.
Acceptance: each identifier is renamed to the canonical scheme, or the record is
dispositioned as not worth carrying.
## Bring clay-borg's workplans onto the convention
```task
id: CUST-WP-0068-T04
status: todo
priority: medium
state_hub_task_id: "554bf550-3357-580b-a54c-78e4be777911"
```
Fifty workplans — the single largest block — use `kind: product` instead of
`type: workplan` and `status: done`, which is not a workplan status. Nothing
parses them as workplans, so no tooling sees them at all.
This is a content pass with the repository owner, not a mechanical rewrite: the
status mapping (`done``finished`?) and the missing `domain`, `repo` and
`owner` fields are judgements about what those records mean.
Acceptance: `clay-borg` workplans parse, and either bootstrap or take the
class-F path.
## Recover the 176 with a non-empty projection
```task
id: CUST-WP-0068-T05
status: todo
priority: high
state_hub_task_id: "0947f7fe-9432-567b-8fc0-4b5e3fadccf4"
```
Thirty-two repositories already have workplans on central, so bootstrap refuses
them. The deterministic subset can go through C-06 unchanged. Largest:
`railiance-platform` (18), `activity-core` (17), `net-kingdom` (13),
`state-hub` (12), `flex-auth` (11), `the-custodian` (10), `ops-warden` (10).
Run C-06 first and measure what remains before assuming any of it needs T06.
Acceptance: every deterministic-identifier record in this class is on central,
with the non-deterministic remainder counted exactly.
## Rule the identifier question for the remainder
```task
id: CUST-WP-0068-T06
status: todo
priority: medium
state_hub_task_id: "fd1bf8e8-d03c-5f26-99c4-aef676b39644"
```
Whatever survives T05 has a non-deterministic identifier in a repository whose
projection is not empty — the one case neither bootstrap nor C-06 handles.
The options are to migrate those identifiers to derived UUIDv5 (`ADR-007`
decision 2, and `state-hub` already has
`apply_repository_identifier_migration` with a reversible plan), or to relax
C-06 for records whose backing file is committed and pushed, or to disposition
them as closed history. **This overlaps `RMGR-WP-0005` and must be agreed with
Repo Manager rather than decided here.**
Do not begin before T05 reports its count: the population may be small enough
that disposition beats migration.
Acceptance: a ruling recorded as a decision, with the chosen path executed or
explicitly deferred.
## Repair the binding gap before counting any orphans
```task
id: CUST-WP-0068-T07
status: todo
priority: high
state_hub_task_id: "f44537f2-a738-5f39-b4c1-3711de06632e"
```
**The orphan count is not trustworthy and must be re-measured first.**
`backing_filename` is written only by `PUT /workplans/index/bindings`, which
`fix-consistency` calls for workplans that already carry a UUID. The documented
registration flow mints the UUID *after* that sync — `fix-consistency` runs,
C-06 skips because the workstation is not the registrar, then
`registrar-reconcile` mints — so a newly registered workplan is left with no
backing file recorded. Reproduced on `CUST-WP-0068` itself: it registered with
`backing_filename: None`, and a second `fix-consistency` run populated it.
Consequence: **278 of 800 workplans on central (35%) record no backing file**,
including 8 `proposed`, 4 `ready` and 4 `active`. By the read model's own
definition those are hub-only records, and `ADR-010` predicted exactly this as
"broken links — a file exists but `backing_filename` was never recorded …
likely the largest class". The 23 cache-only records classified as orphans were
counted with the same unreliable field, so that number is a starting point for
investigation, not a work list.
Order of work:
1. Fix the ordering so registration binds in one pass — either the registrar
syncs bindings after minting, or the session protocol requires the second
`fix-consistency` run explicitly. The former is preferable; a protocol step
everyone must remember is how this gap opened.
2. Re-sync bindings fleet-wide and re-measure. Also explain
`bindings: synced 63/73` — ten workplans did not bind and the reason is
unknown.
3. Only then apply `ADR-010`'s orphan rules to whatever genuinely has no file:
open hub-first records are `ADR-001` violations needing a file written or an
explicit drop; closed ones may be retained as provenance where cheap.
Acceptance: registration records a backing file in one pass; the fleet-wide
count of file-less workplans is re-measured and explained; every remaining true
orphan is repaired, given a file, or dropped in writing.
## Converge and retire the cache database
```task
id: CUST-WP-0068-T08
status: todo
priority: high
state_hub_task_id: "cbb8bf98-9b66-5827-992c-f22c3a6c1bb5"
```
Confirm no work record exists only in the cache, then stop and remove the local
Postgres instance and its volume.
Verify by direct comparison against the database rather than by assuming the
preceding tasks covered everything — the predecessor workplan twice reported
convergence that a direct count contradicted.
Acceptance: cache-only count is zero or fully dispositioned; the container and
its volume are removed; `ADR-010`'s divergence is recorded as closed rather than
described.