Assistant: codex Assistant-Model: gpt-5.6-sol Assistant-Session: 01a053ff-1d6f-7fe2-ac1c-a6eb40a42a0c
772 lines
35 KiB
Markdown
772 lines
35 KiB
Markdown
---
|
|
id: STATE-WP-0083
|
|
type: workplan
|
|
title: "Forge-derived projection reset, per repository"
|
|
domain: infotech
|
|
repo: state-hub
|
|
status: finished
|
|
owner: codex
|
|
topic_slug: infotech
|
|
created: "2026-08-25"
|
|
updated: "2026-08-31"
|
|
related:
|
|
- CUST-ADR-012
|
|
- CUST-WP-0068
|
|
state_hub_workstream_id: "40e36360-6b2a-5903-b769-47507d866ac8"
|
|
---
|
|
|
|
# Forge-derived projection reset, per repository
|
|
|
|
## Goal
|
|
|
|
Implement `ADR-012` decision 7: discard a repository's projection and rebuild it
|
|
from what Forgejo holds. Per repository, routine, idempotent, and verifiable.
|
|
|
|
This is the only sanctioned way to remove a hub record. The hub has no
|
|
hard-delete for work records — `DELETE /tasks/{id}` is `cancel_task`,
|
|
`DELETE /workstreams/{id}` is `410 Gone`, and no `session.delete` exists for
|
|
either. Re-derivation is not one option among several; it is the mechanism.
|
|
|
|
## Why here and not in Repo Manager
|
|
|
|
`ADR-012` decision 1 says central derives from the forge, and the pod can reach
|
|
it: `git ls-remote https://forgejo.coulomb.social/...` succeeds anonymously from
|
|
inside the cluster, and `git` is already in the image.
|
|
|
|
Driving the reset from Repo Manager would mean computing a projection on a
|
|
workstation and pushing it into central — precisely the injection of derived
|
|
state `ADR-010` decision 5 forbids. Central must do its own reading. The cost is
|
|
that workplan parsing exists in two places; `scripts/consistency_check.py`
|
|
already parses frontmatter and task blocks here, so the capability is present
|
|
and should be extracted rather than rewritten.
|
|
|
|
## What it must not do
|
|
|
|
- **Not restore preliminary overlay records.** They exist because the forge does
|
|
not hold them; a rebuild from the forge cannot reproduce them and must not
|
|
pretend to.
|
|
- **Not destroy hub-native records.** Progress events, decisions and inbox
|
|
messages originate in the hub (`ADR-010` decision 4). They are not
|
|
forge-derived and must survive a rebuild of forge-derived state.
|
|
- **Not proceed silently when records would be lost.** Stop, name them, and
|
|
require an explicit acknowledgement.
|
|
|
|
## Derive a repository's projection from the forge
|
|
|
|
```task
|
|
id: STATE-WP-0083-T01
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "ca3e8578-6855-55a2-9042-f790c8997628"
|
|
```
|
|
|
|
Given a repository slug, clone or fetch its default branch from Forgejo into a
|
|
temporary location, parse the workplan files, and compute the projection that
|
|
state would produce: workplans, tasks, their identifiers, statuses and backing
|
|
paths.
|
|
|
|
Read-only and side-effect free. Record the commit the derivation came from —
|
|
`ADR-012` decision 2 requires provenance, and the existing `git_fingerprint` is
|
|
worthless precisely because nothing ever wrote it correctly.
|
|
|
|
Acceptance: deriving `the-custodian` twice from the same commit yields identical
|
|
output, and the commit is reported.
|
|
|
|
**Done (2026-08-25).** `api/services/forge_projection.py`. Central clones the
|
|
default branch from Forgejo and derives 69 workplans and 459 tasks from
|
|
`the-custodian` at commit `d5013ae`, identically across runs. Identifiers are
|
|
derived from the canonical record id in the `ADR-007` namespace, verified against
|
|
live records — so a forge-derived projection and a preliminary overlay agree on
|
|
identity without reconciliation. Eight tests, including that identifiers are
|
|
derived rather than inherited from whatever a file happens to carry.
|
|
|
|
A fresh shallow clone each time is deliberate: reusing a working copy is how a
|
|
projection ends up reflecting someone's local state instead of the forge.
|
|
|
|
## Report the difference before changing anything
|
|
|
|
```task
|
|
id: STATE-WP-0083-T02
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "3122ae66-1a9a-5102-a7ff-b7ba1564bf3c"
|
|
```
|
|
|
|
Compare the derived projection against what the hub currently holds for that
|
|
repository, and report: records the forge has that the hub lacks, records the
|
|
hub has that the forge does not, and records whose fields differ.
|
|
|
|
This is the half that is immediately useful without any destructive capability,
|
|
and it is what makes decision 7's "verifiable" real. It also answers the
|
|
question `CUST-WP-0068-T09` is waiting on — which stale rows would actually
|
|
clear — before anyone commits to clearing them.
|
|
|
|
Acceptance: a dry-run diff for a repository with known drift matches what
|
|
manual inspection shows.
|
|
|
|
**Workplan-level diff done; task-level blocked (2026-08-25).**
|
|
|
|
The first implementation matched hub records to derived ones *by UUID* and was
|
|
badly wrong: most hub records still carry pre-`ADR-007` random identifiers, so
|
|
nearly every record appeared simultaneously missing and stale. A reset built on
|
|
that comparison would have destroyed and recreated the whole projection. It now
|
|
matches on canonical record id, falling back to the backing file.
|
|
|
|
Workplan-level results are trustworthy and inspectable:
|
|
|
|
| Repository | missing | stale | differing |
|
|
|---|---|---|---|
|
|
| `whitehat-security` | 0 | 0 | 0 |
|
|
| `the-custodian` | 3 | 4 | 52 |
|
|
| `railiance-platform` | 18 | 25 | 2 |
|
|
|
|
`whitehat-security` reporting clean is the control: it was bootstrapped directly
|
|
from its files, so a forge derivation must agree with it exactly. The four stale
|
|
workplans on `the-custodian` were checked by hand and are real —
|
|
`CUST-WP-0023`/`0024` have no file in the repository at all, and
|
|
`state-hub-v0.1`/`v0.2` carry slugs that were never canonical ids. All four are
|
|
genuine hub-first records, the class `ADR-010` says to disposition.
|
|
|
|
**Blocked: hub tasks carry no canonical record id.** The task schema is
|
|
`id, workplan_id, title, status, priority, …` with nothing holding
|
|
`CUST-WP-0067-T01`. A file task and a hub task can therefore only be matched by
|
|
title, which is why the task diff reports 149 missing and 131 stale for
|
|
`the-custodian` where the workplan diff reports 3 and 4. That is the matching
|
|
failing, not drift.
|
|
|
|
Task-level reset must not be built on title matching — renaming a task heading
|
|
would silently destroy and recreate its record. `T03` is limited to workplans
|
|
until tasks carry their canonical id, which is `T06`.
|
|
|
|
## Apply the reset transactionally
|
|
|
|
```task
|
|
id: STATE-WP-0083-T03
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "95496a43-9e72-53d0-ad89-1a43713b4be9"
|
|
```
|
|
|
|
Replace the repository's forge-derived records with the derived projection in
|
|
one transaction: create what is missing, update what differs, remove what no
|
|
longer derives. Hub-native records are untouched.
|
|
|
|
Refuse by default when removal would destroy a record with no counterpart in the
|
|
forge; report what would be lost and require an explicit acknowledgement to
|
|
proceed. That refusal is evaluated per repository, so one unresolved repository
|
|
never blocks the rest.
|
|
|
|
Acceptance: reset twice produces the same projection; a repository holding
|
|
records the forge lacks is refused with those records named; hub-native record
|
|
counts are unchanged across a reset.
|
|
|
|
**Done (2026-08-26).** `reset_repository_projection()` in
|
|
`api/services/forge_projection.py`. Verified against live data, rolled back:
|
|
`whitehat-security` applied 5 updates and 0 retirements; `the-custodian`
|
|
**refused**, naming `cust-wp-0023`, `cust-wp-0024`, `state-hub-v0.1` and
|
|
`state-hub-v0.2` — the four records confirmed by hand as genuine hub-first
|
|
records with no file — and changed nothing.
|
|
|
|
Refusal is the default because a record that stops deriving is ambiguous: the
|
|
file may have been deleted deliberately, or the caller may have pointed at the
|
|
wrong branch. Only the caller can say which, so only the caller may authorise it.
|
|
|
|
Retirement never deletes, with a test asserting the row survives and
|
|
`session.delete` is never called. A record that derives again is un-retired
|
|
rather than left contradicting the forge.
|
|
|
|
The first execution of this write path was run against the **cache** database
|
|
rather than central, and rolled back. A write path's first run belongs on the
|
|
discardable copy.
|
|
|
|
**Scope limit, enforced in the code rather than documented beside it.** Tasks of
|
|
existing workplans are untouched; tasks are created only alongside a *new*
|
|
workplan, where nothing exists to mis-match. The outcome carries that as a note
|
|
so partial convergence cannot be mistaken for full. 678 tests pass.
|
|
|
|
## Fleet form as a loop over the repository form
|
|
|
|
```task
|
|
id: STATE-WP-0083-T04
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "3935334e-d5bd-5667-a862-0049c1c2c5b0"
|
|
```
|
|
|
|
The fleet-wide reset iterates the per-repository reset and shares its
|
|
implementation, so the rarely-used dangerous path is exercised by the frequently
|
|
used safe one. Repositories that refuse are skipped and reported, never aborting
|
|
the pass.
|
|
|
|
Acceptance: a fleet dry-run reports per-repository outcomes including refusals,
|
|
and completes despite them.
|
|
|
|
**Built (2026-08-26).** `reset_fleet_projection()` is a loop over the
|
|
per-repository form and nothing else, so the rarely-run wide operation is the
|
|
frequently-run narrow one rather than a second implementation trusted on the
|
|
strength of never having been exercised.
|
|
|
|
Its behaviour on failure is the part that matters, and all three are tested:
|
|
|
|
- **A refusal does not stop the pass.** Aborting on the first refusal means one
|
|
unresolved repository blocks reconstruction everywhere, which in practice means
|
|
permanently.
|
|
- **An error does not stop the pass.** A repository that cannot be cloned is
|
|
recorded and the loop continues.
|
|
- **Each repository gets its own session**, so one failure cannot roll back
|
|
another's work or leave a poisoned transaction behind. Only repositories that
|
|
applied are committed.
|
|
|
|
694 tests pass. Needs deployment before it can run against central.
|
|
|
|
**Ran against central (2026-08-28).** Image `main-54b09ee`. Fleet
|
|
refuse-mode over 125 repositories: 52 applied, 69 noop, 2 refused
|
|
(identifier collisions on `railiance-bootstrap` and `railiance-hosts`;
|
|
acknowledgement does not authorise those), 2 unreadable
|
|
(`vergabe_teilnahme`, `markitect-project`), 0 errors. Zero workplans
|
|
retired, zero tasks cancelled. `the-custodian` had already been
|
|
reset-acked and reported `noop` on the fleet pass. Evidence lives in
|
|
`the-custodian/docs/recovery/fleet-projection-reset-2026-08-28.md`.
|
|
|
|
## Retire what the reset replaces
|
|
|
|
```task
|
|
id: STATE-WP-0083-T05
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "b63c97a8-ceba-551a-a3df-5253a9de409a"
|
|
```
|
|
|
|
Once reset is trusted, `CUST-WP-0068-T09` can clear its stale rows — 4 workplan
|
|
rows from the prefix migration and 305 task rows across 44 workplans.
|
|
|
|
`ADR-003` decision 2's `mtime`-based fingerprints should also be replaced by the
|
|
source commit at this point; `ADR-012` invalidated that composition and the
|
|
replacement belongs with the provenance work in `T01`.
|
|
|
|
Acceptance: `CUST-WP-0068-T09` closes; no fingerprint input depends on a local
|
|
filesystem.
|
|
|
|
|
|
## Give hub tasks their canonical record id
|
|
|
|
```task
|
|
id: STATE-WP-0083-T06
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "6172bdd6-3236-5f85-8c02-b0f149e373b8"
|
|
```
|
|
|
|
Hub task rows hold no canonical identifier — only a title — so nothing reliably
|
|
connects `CUST-WP-0067-T01` in a file to its row. Every other record type has a
|
|
stable identity; tasks do not, and that gap is what stops the reset from covering
|
|
them.
|
|
|
|
Add the canonical id to the task record and populate it during derivation and
|
|
registration. Once present, task matching becomes identity-based like workplans,
|
|
and `T03` can extend to tasks safely.
|
|
|
|
Until then a task's identity is its title, which changes whenever someone edits a
|
|
heading. That is not a foundation for deletion.
|
|
|
|
Acceptance: task rows carry their canonical record id; the task diff for
|
|
`whitehat-security` reports clean; `the-custodian`'s task diff falls to something
|
|
manual inspection confirms.
|
|
|
|
**Built (2026-08-26).** Migration `e2b3c4d5f6a7` adds `tasks.record_id`,
|
|
nullable because no migration can invent an identity for an existing row — only
|
|
the repository files hold the mapping.
|
|
|
|
The backfill *reads* that mapping rather than inferring it: a file task declares
|
|
both its canonical id and the projection UUID it was registered under, so the
|
|
pairing is stated, not guessed. Across 121 repositories and 1104 files it
|
|
recovered **5516 pairs with zero conflicts**, and identified **4456 of 6073**
|
|
task rows in the cache database.
|
|
|
|
Diff and reset now key on `record_id` where present, falling back to a
|
|
`title:`-prefixed key so an unidentified row is *visibly* unidentified rather
|
|
than silently title-matched.
|
|
|
|
Two refusals are tested rather than documented: a row the files do not claim
|
|
keeps no identity, and a row that already has one is never overwritten — a
|
|
mismatch is recorded as a conflict, not resolved. Title matching would have
|
|
"worked" and destroyed a record every time someone edited a heading.
|
|
|
|
**Remaining, in order.** Deploy the migration to central; then run the backfill
|
|
against central as an explicit operation — deliberately *not* inside the Helm
|
|
hook, where a partial failure would silently leave half the tasks identified.
|
|
The cache matched 4456 of 6073 and central has a different history, so the
|
|
numbers will differ and a dry run should be compared before applying.
|
|
|
|
Only then may `T03` be extended to tasks of existing workplans, which is a
|
|
further change with its own verification. `CUST-WP-0068-T09` is therefore two
|
|
moves away, not one.
|
|
|
|
**Backfill run against central (2026-08-26), and a defect it exposed.**
|
|
|
|
The first implementation read local filesystem paths. Central has no workstation
|
|
checkout and must not depend on one — `ADR-012` decision 1 makes the forge the
|
|
projection source — and the point became concrete rather than theoretical:
|
|
central's Postgres is unreachable from the workstation, since the tunnel carries
|
|
HTTP only. A forge-sourced variant now clones each repository in-cluster and
|
|
reads the pairing from what the forge holds.
|
|
|
|
Dry run across 121 repositories: 1094 files, 5620 pairs, 5248 to update, 372
|
|
file-declared UUIDs central does not hold, **zero conflicts**. The count
|
|
differing from the cache's 4456 of 6073 was the check that mattered — central
|
|
holds the re-keyed records the cache never had, so a *matching* number would have
|
|
meant cache state was leaking in. The forge scan reading 1094 files against the
|
|
workstation's 1104 is the preliminary-overlay distinction showing up in the
|
|
numbers.
|
|
|
|
**Then applied, and wrong.** 5248 rows were identified but only 5100 identities
|
|
were distinct. A task id written as a bare `T01` is unique only inside its own
|
|
workplan; stored as canonical it gave every workplan's first task one identity.
|
|
51 such ids landed on 148 rows.
|
|
|
|
The backfill's own conflict detection could not see it: it checks one UUID
|
|
claimed by two ids, and this was the inverse. It surfaced only because the
|
|
identified count and the distinct count did not reconcile — a check that could
|
|
easily have been skipped after a dry run reporting zero conflicts.
|
|
|
|
Short ids are now qualified as `WORKPLAN-ID-T01`, and a short id in a file with
|
|
no workplan id gets no identity at all. The 136 affected rows on central were
|
|
cleared for reassignment, since the backfill never overwrites an existing
|
|
identity.
|
|
|
|
**Central now: 5112 of 5974 identified, 5077 distinct.** The remaining 34
|
|
duplicates are not from this work. They are unqualified ad-hoc ids reused across
|
|
repositories — `ADHOC-2026-07-02-T01` exists in three workplans — which is the
|
|
problem `CUST-WP-0066` closed for new records while existing ones kept their
|
|
unqualified ids. They need their own disposition.
|
|
|
|
**Done (2026-08-26).** Corrected backfill applied to central: **5248 of 5974
|
|
tasks identified, 5213 distinct, 35 duplicate identities.**
|
|
|
|
The check before applying was uniqueness, not volume — the lesson from the first
|
|
attempt, where a clean dry run reporting zero conflicts concealed 51 non-unique
|
|
ids. All 136 pending updates were verified to take identities held by nothing
|
|
else before a single row was written.
|
|
|
|
The 35 remaining duplicates predate this work and fall into three classes:
|
|
|
|
| Count | Class | Origin |
|
|
|---|---|---|
|
|
| 18 | `KONT-WP-0013-T00N` and similar | a workplan registered twice, so both copies claim the same task ids |
|
|
| 12 | `RAILIANCE-WP-00NN-T0N` | archived RAILIANCE-WP files, left unrenamed by the owner decision of 2026-08-25 — the same numbers exist in three repositories, so their task ids collide |
|
|
| 4 | `ADHOC-*-T0N` | unqualified ad-hoc ids reused across repositories, the class `CUST-WP-0066` closed for new records only |
|
|
|
|
The `RAILIANCE-WP` twelve are worth noting as a consequence rather than a
|
|
surprise: migrating only the *active* workplans was the right call for blast
|
|
radius, and this is the price — the archived copies keep colliding identities
|
|
until they are migrated too, and task-level reset cannot act on those workplans
|
|
in the meantime.
|
|
|
|
**Task-level reset remains blocked**, now on a narrower and better-understood
|
|
problem: 35 identities, not 5974 unidentified rows.
|
|
|
|
**Mixed identifier convergence support (2026-08-31).** The fast forge path
|
|
exposed a second, separate consistency shape: one repository can contain legacy
|
|
UUID rows and already-derived UUID rows at the same time. State Hub's sealed
|
|
identifier transaction now classifies every mapping, migrates only verified
|
|
legacy sources, accepts already-derived targets only when their canonical
|
|
record identity and repository match, and records durable aliases for both.
|
|
Both-present and neither-present mappings fail the whole repository transaction.
|
|
The primary-only HTTP surface has explicit plan-hash confirmation and a reverse
|
|
operation for a failed file phase. This does not resolve the 35 duplicate
|
|
canonical task identities, so T02 remains `progress`.
|
|
|
|
**Duplicate identity disposition (2026-08-31).** A fresh primary audit found
|
|
131 duplicate task `record_id` groups across 274 rows. The global count is not
|
|
the reset safety boundary: all 131 cross workplan boundaries, while none repeats
|
|
inside one workplan. A task has no projection-retirement field, so workplan slug
|
|
tombstones are only a diagnostic clue; current task membership is whether the
|
|
Forge still derives that id for that owning workplan.
|
|
|
|
The 12 groups with multiple untombstoned owning workplan slugs come from two
|
|
earlier wrong workplan/backing-file projections:
|
|
`CUST-WP-0010b-T01..T02` and `SECRETS-WP-0002-T01..T10`. They must be resolved
|
|
by reconciling the owning workplans while preserving their completed historical
|
|
task rows, not by deleting or re-keying those rows. Live exact-commit reconciles
|
|
did that with zero creates, retirements, cancellations, or refusals. Evidence:
|
|
`docs/evidence/STATE-WP-0083-task-identity-audit-2026-08-31.json`.
|
|
|
|
The diff now reports an explicit `ambiguous` class instead of silently choosing
|
|
the last row in a dictionary. Reset refuses duplicate identities declared by
|
|
the forge, duplicate identified rows inside the matched current workplan, and
|
|
any task creation whose derived UUID is held by another current workplan. A
|
|
repeated human-readable id outside the owning workplan's current desired set is
|
|
historical evidence, not a competing projection. These checks are batched;
|
|
retired/displaced workplan tasks remain preserved. The second exact-commit pass
|
|
for both affected repositories returned `noop` with every count zero. Final
|
|
deployment of the refined boundary remains before T02 can close.
|
|
|
|
**Done (2026-08-31).** Image `main-a32112e`, Helm revision 53, is healthy on
|
|
the `railliance01` primary. State Hub itself converged at exact Forge commit
|
|
`a32112e5abc18dfbecbbf1320cf5b04dce814920`; the immediately repeated pass was
|
|
`noop` with every count zero. The task diff/reset boundary is now complete.
|
|
|
|
Fleet identifier rollout remains live under `RMGR-WP-0005`; this workplan does
|
|
not duplicate that residual. Historical terminal task labels remain preserved
|
|
by decision and are not pending cleanup work.
|
|
|
|
|
|
## Restore a migration mechanism for central
|
|
|
|
```task
|
|
id: STATE-WP-0083-T07
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "27e12170-486e-5669-8658-f7acc80d81f8"
|
|
```
|
|
|
|
**Central's schema is behind the code it is running, and nothing detects it.**
|
|
Discovered 2026-08-25 while preparing the retirement columns.
|
|
|
|
Central is at revision `b8d4f0a2c6e1`; the chain is
|
|
`b8d4f0a2c6e1 → c9e5a1b3d7f2 → d1a2b3c4e5f6`. The `review_contracts` table does
|
|
not exist on central even though the migration creating it ships inside the image
|
|
currently serving traffic. The code and the database disagree silently.
|
|
|
|
There is no migration mechanism. The image `CMD` is bare `uvicorn`, the Helm
|
|
chart declares no migration job, and the only evidence migrations ever ran was an
|
|
ad-hoc `state-hub-alembic-init` Job created outside the chart — deleted earlier
|
|
the same day as a retired-forge leftover. Deleting a completed Job removed a
|
|
historical record rather than a working mechanism, since Jobs do not re-run, but
|
|
it also removed the last visible sign that this gap existed.
|
|
|
|
This blocks `T03`: the retirement columns cannot reach central without it.
|
|
|
|
Needed: migrations as a declared part of the release — a chart-managed job or
|
|
init container that runs `alembic upgrade head` before the API serves — and a
|
|
check that surfaces a schema/code mismatch instead of leaving it silent. A
|
|
deployment that can run against a schema it was not built for is the same class
|
|
of defect as a projection that cannot name its source commit.
|
|
|
|
Acceptance: central reaches `head`; `review_contracts` exists; the mechanism is
|
|
declared in the chart; a mismatch between code and schema is reported rather than
|
|
tolerated.
|
|
|
|
**Done (2026-08-26).** Central reports
|
|
`schema: {status: ok, applied: d1a2b3c4e5f6, expected: d1a2b3c4e5f6}`.
|
|
`review_contracts` exists. Both pods run `main-97c8762`.
|
|
|
|
The first execution is worth recording, because it failed in a way that proved
|
|
the design. The `pre-upgrade` hook ran, applied both migrations and deleted
|
|
itself on success — then the k3s API tunnel dropped (`unexpected EOF` →
|
|
`connection refused`) and Helm could not read the deployment to roll the API.
|
|
The release wedged at `pending-upgrade`; no pod rolled.
|
|
|
|
The schema was therefore briefly *ahead* of the running code, which is the safe
|
|
direction and the reason additive migrations were the right shape: the old image
|
|
served correctly against the new schema throughout. A destructive migration in
|
|
the same circumstance would have taken the service down.
|
|
|
|
Recovery was `helm rollback` to the last deployed revision — which clears the
|
|
wedge without touching the schema, there being nothing to undo — then re-running
|
|
the upgrade. The hook is idempotent, so the second run's `alembic upgrade head`
|
|
was a no-op.
|
|
|
|
Two things this exposed, neither yet addressed:
|
|
|
|
- The hook's `hook-succeeded` delete policy removes the Job on success, so a
|
|
*successful* migration leaves no trace. That made it look as though nothing had
|
|
run when in fact everything had. Retaining a short-lived record of successful
|
|
migrations, not only failed ones, would have answered the question immediately.
|
|
- `bridge status` reported `k3s-api-railiance01` as `connected` while holding a
|
|
stale pid, so its health signal did not reflect the dropped connection. A
|
|
tunnel monitor that cannot detect the failure it exists to detect is the same
|
|
defect class as a projection that cannot name its source commit.
|
|
|
|
|
|
## Repair workplan files whose frontmatter never terminates
|
|
|
|
```task
|
|
id: STATE-WP-0083-T08
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "a27b7130-70f5-59a0-9e1e-5ec048737c68"
|
|
```
|
|
|
|
**Blocks the first production reset.** Found 2026-08-26 by reviewing the reset's
|
|
diff before applying it — the diff proposed retiring `kont-wp-0015`, a live and
|
|
correctly registered workplan, which is what prompted the investigation.
|
|
|
|
Eighteen workplan files across the fleet have a closing `---` fused onto the last
|
|
frontmatter value:
|
|
|
|
```yaml
|
|
depends_on_workplans:
|
|
- KONT-WP-0016---
|
|
```
|
|
|
|
There is no delimiter line, so the frontmatter never terminates. The file
|
|
declares a dependency on a workplan literally named `KONT-WP-0016---` and the
|
|
entire body is swallowed as frontmatter. Any parser selecting on `type: workplan`
|
|
sees nothing, so the file is not merely unparsed but *invisible* — it appears as
|
|
neither a workplan nor an error.
|
|
|
|
Affected: `citation-evidence` (5), `infospace-bench` (2), `kontextual-engine`,
|
|
`net-kingdom` (2 incl. archived), and others — 18 in total.
|
|
|
|
The consequence for reset is severe and must not be worked around: a repository
|
|
containing one of these files derives a projection missing that workplan, so the
|
|
reset proposes retiring a record that is entirely correct. Running it before this
|
|
is repaired would retire live work.
|
|
|
|
That the refusal-by-default caught this is the design working, but the diff would
|
|
have looked plausible to anyone not checking a specific record — `created=0,
|
|
would_retire=N` reads like a tidy-up.
|
|
|
|
Repair the frontmatter, then re-derive and confirm each affected repository's
|
|
diff no longer proposes retiring live workplans.
|
|
|
|
Acceptance: no workplan file yields empty frontmatter while starting with `---`;
|
|
the reset diff for each affected repository proposes retiring only records whose
|
|
backing file genuinely no longer exists.
|
|
|
|
**Done (2026-08-26).** All 18 repaired across 8 repositories; **zero workplan
|
|
files now yield empty frontmatter**. Previously invisible workplans derive again:
|
|
`KONT-WP-0015`, `CUST-WP-0000`, `CE-WP-0001` and the rest.
|
|
|
|
Two shapes of one defect, both a missing newline:
|
|
|
|
| Count | Shape | Example |
|
|
|---|---|---|
|
|
| 16 | closing delimiter fused onto the last value | `- KONT-WP-0016---` |
|
|
| 2 | a value fused onto the following key | `…v1.3.0state_hub_task_id: …` |
|
|
|
|
Only the newline was inserted; no value was altered, and the diff on each file is
|
|
one line split into two.
|
|
|
|
**A correction this forces.** `state-hub-v0.1` and `state-hub-v0.2` were reported
|
|
earlier as genuine hub-first records with no backing file, and dispositioned on
|
|
that basis. They are `CUST-WP-0000` and `CUST-WP-0000b` — files that existed all
|
|
along and were unreadable. The claim was wrong because "no file backs this
|
|
record" was inferred from a parser's silence rather than checked against the
|
|
directory.
|
|
|
|
**The ordering bug in the repair itself is worth recording**, since it is the
|
|
same mistake in miniature. The first version only looked for a fused delimiter
|
|
when no `\n---` existed anywhere in the file. Five `citation-evidence` files have
|
|
a horizontal rule in the body, so a delimiter *was* found — just the wrong one —
|
|
and the real defect was hidden behind it. Checking whether the frontmatter
|
|
*parses* before trusting the delimiter position fixed it, taking the count from
|
|
11 repairable to 18.
|
|
|
|
|
|
## Reset diff after the frontmatter repairs (2026-08-26)
|
|
|
|
Re-run against central once `T08` landed. The false retirements are gone:
|
|
|
|
| Repository | before | after |
|
|
|---|---|---|
|
|
| `kontextual-engine` | 1 (a live workplan) | 0 retire, 18 updated |
|
|
| `citation-evidence` | not yet measured | 0 retire, 12 updated |
|
|
| `the-custodian` | 4 | 2 retire |
|
|
| `net-kingdom` | — | **errors** |
|
|
|
|
Two findings the re-run produced, both worth more than the diff itself.
|
|
|
|
### The four "hub-first records" were never hub-first
|
|
|
|
`cust-wp-0023` and `cust-wp-0024` were reported as records with no backing file,
|
|
verified by hand, and dispositioned on that basis. They have files —
|
|
`CUST-WP-0023-tpsc.md` and `CUST-WP-0024-repo-doi-gate.md` — which declare
|
|
`type: feature`, so the derivation correctly excludes them while the hub holds
|
|
them as workplans.
|
|
|
|
Together with `state-hub-v0.1`/`v0.2`, which turned out to be `CUST-WP-0000` and
|
|
`CUST-WP-0000b` with unterminated frontmatter, **all four claims were wrong, by
|
|
three different mechanisms**: broken frontmatter, a mismatched `type`, and in
|
|
both cases an inference from a parser's silence rather than a check against the
|
|
directory. "The tool did not find it" was repeatedly reported as "it does not
|
|
exist".
|
|
|
|
Eleven files in `workplans/` directories declared something other than
|
|
`type: workplan` — `feature`, `runbook`, `bug-report`, `extension-workplan`,
|
|
`scorecard`, `session-note`, `adhoc` — while the hub held every one of them as a
|
|
workplan.
|
|
|
|
**Resolved 2026-08-26 by owner decision.** Canon defines six kinds — workplan,
|
|
task, intake, decision, engagement, register-entry — and none of these was among
|
|
them, so the declarations were simply wrong rather than a missing vocabulary.
|
|
Eight files with valid workplan identifiers had their type corrected. Three that
|
|
lacked an identifier as well were given one continuing their repository's
|
|
sequence, with existing hub identifiers preserved so no record was orphaned.
|
|
|
|
Zero non-workplan types now remain in any `workplans/` directory.
|
|
|
|
### The reset crashes where it should refuse
|
|
|
|
`net-kingdom` raises `IntegrityError: Key (id)=(e25ba5fa…) already exists`. Its
|
|
`ADHOC-2026-08-23` derives to an identifier another repository already holds —
|
|
the exact collision `CUST-WP-0066` was written about, where two repositories
|
|
created the same daily identifier on the same day.
|
|
|
|
The reset must detect that a derived identifier belongs to another repository and
|
|
refuse with that stated, rather than failing on a database constraint. A refusal
|
|
is a decision the caller can act on; a constraint violation is a stack trace.
|
|
|
|
**Fixed and verified live (2026-08-26, `main-5e4d0be`).** `net-kingdom` now
|
|
returns `refused` with reason *"derived identifier already belongs to another
|
|
repository"*, naming `ADHOC-2026-08-23` and the holder.
|
|
|
|
Acknowledging retirements deliberately does not authorise a collision: those are
|
|
different decisions — one says the work is gone, the other says take an
|
|
identifier another repository owns — and conflating them would let a routine
|
|
acknowledgement carry an identity change through. Covered by test.
|
|
|
|
Worth recording how nearly this was mis-reported. A first check run against the
|
|
*cache* returned `refused` and was almost reported as proof; it had exercised the
|
|
retirement path instead, because the cache does not hold the colliding record.
|
|
The output looked like success while testing nothing of the sort.
|
|
|
|
|
|
## Reset diff, clean (2026-08-26)
|
|
|
|
After the frontmatter repairs (`T08`) and the type corrections, the diff proposes
|
|
**no retirements at all**:
|
|
|
|
| Repository | created | updated | would retire |
|
|
|---|---|---|---|
|
|
| `the-custodian` | 3 | 71 | **0** |
|
|
| `activity-core` | 3 | 40 | **0** |
|
|
| `phase-memory` | 0 | 18 | **0** |
|
|
| `ground-game` | 0 | 9 | **0** |
|
|
| `open-cmis-tck` | 0 | 3 | **0** |
|
|
|
|
Every hub record in these repositories now derives from a file. `the-custodian`
|
|
began this sequence proposing to retire four live workplans; it proposes none.
|
|
|
|
Worth stating what changed and what did not. Nothing about the reset's logic was
|
|
altered to reach this: the retirements disappeared because the *sources* were
|
|
repaired — unterminated frontmatter, and types that made files invisible. The
|
|
reset was reporting the truth about a fleet whose files could not be read.
|
|
|
|
That is the argument for `T02` existing separately from `T03`. Had the reset
|
|
applied on first run, it would have retired live work in at least five
|
|
repositories, and every one of those retirements would have looked like tidy-up.
|
|
|
|
## First fleet-wide pass (2026-08-26)
|
|
|
|
121 repositories, refuse mode. The first measured answer to whether the hub
|
|
matches the forge.
|
|
|
|
| | |
|
|
|---|---|
|
|
| applied | 91 — 737 updated, 8 created |
|
|
| refused | 16 repositories, **64 records** |
|
|
| noop | 2 |
|
|
| errored | 12 |
|
|
|
|
**745 workplans now carry the commit they derived from.** `ADR-012` decision 2
|
|
is satisfied for the first time: until today no record could name its source, and
|
|
`git_fingerprint` had held the initial commit since the repository began. Zero
|
|
records were retired and hub-native progress events were untouched, as intended.
|
|
|
|
**64 is the real size of the stale-row problem** that `CUST-WP-0068-T09` has been
|
|
waiting on — measured rather than estimated. `railiance-platform` (20),
|
|
`vergabe-teilnahme` (16) and `railiance-apps` (8) hold more than two thirds.
|
|
|
|
### Private repositories are invisible to central
|
|
|
|
Eleven of the twelve errors are the same:
|
|
|
|
```text
|
|
fatal: could not read Username for 'https://forgejo.coulomb.social'
|
|
```
|
|
|
|
Those repositories are private, and the pod clones anonymously. `rapp-core-hub`,
|
|
`rapp-issue-core`, `rapp-openbao`, `rapp-policy-nexus` and seven others cannot be
|
|
derived at all.
|
|
|
|
This is a genuine limit on `ADR-012` decision 1: *the forge is the projection
|
|
source* holds only for repositories central can read. Until it has a deploy token
|
|
or equivalent, a whole class of repositories can never be reset — and, worse,
|
|
their absence looks like an error rather than a policy, so nothing distinguishes
|
|
"cannot read" from "does not exist".
|
|
|
|
### A second collision dimension
|
|
|
|
`disaster-control` raised `IntegrityError: Key (slug)=(repo-wp-0001) already
|
|
exists`. The identifier refusal added earlier checks `id`; `slug` carries its own
|
|
unique constraint across the whole table, so two repositories can derive
|
|
different identifiers whose slugs still collide. Now refused with the holder
|
|
named, and covered by test.
|
|
|
|
|
|
## Second fleet pass — idempotent (2026-08-26)
|
|
|
|
Re-run after the slug-collision refusal deployed.
|
|
|
|
| | first pass | second |
|
|
|---|---|---|
|
|
| applied | 91 | 0 |
|
|
| **noop** | 2 | **93** |
|
|
| refused | 16 repos / 64 | 19 repos / 69 |
|
|
| errored | 12 | **9** |
|
|
| created / updated / retired | 8 / 737 / 0 | **0 / 0 / 0** |
|
|
|
|
**Idempotence demonstrated rather than asserted.** 93 repositories changed
|
|
nothing on the second run: `ADR-012` decision 7 requires that reset twice
|
|
produces the same projection, and it does.
|
|
|
|
The slug fix helped more than predicted. Errors fell 12 → 9, not 12 → 11 — three
|
|
repositories were erroring on slug collisions rather than only
|
|
`disaster-control`. They now refuse with the holder named:
|
|
|
|
| Refusal reason | Count |
|
|
|---|---|
|
|
| would be retired; the forge no longer derives it | 60 |
|
|
| slug already belongs to another repository | 9 |
|
|
|
|
**60 records across 13 repositories no longer derive from any file.** That is the
|
|
retirement work `CUST-WP-0068-T09` waits on, now enumerated per repository:
|
|
`railiance-platform` (20), `vergabe-teilnahme` (16), `railiance-apps` (8), then
|
|
single figures elsewhere.
|
|
|
|
All 9 remaining errors are private repositories the pod cannot clone
|
|
anonymously — unchanged, and not a code problem.
|
|
|
|
|
|
## T05 outcome — every hub record derives from a file (2026-08-26)
|
|
|
|
| | |
|
|
|---|---|
|
|
| repositories | 121 |
|
|
| noop | **106** |
|
|
| refused | 6 — all identifier collisions, no retirements |
|
|
| errored | 9 — private repositories |
|
|
| **retired** | **51** |
|
|
| progress events | 21522, untouched throughout |
|
|
|
|
Zero repositories hold a record that no longer derives.
|
|
|
|
**26 real workplans were rescued from wrongful retirement**, more than half the
|
|
number actually retired:
|
|
|
|
- `vergabe-teilnahme` (17) — a repository predating the convention entirely: no
|
|
`type:` field anywhere, bare `WP-NNNN` identifiers, no hub identifiers, no
|
|
registered prefix. The diff showed 16 records to retire; the truth was 17
|
|
workplans no projection could see.
|
|
- `glas-harness` (3), `binky-control` (2), `rein-aharness` (2), `ops-mason` (1),
|
|
`rein-openweights` (1) — files carrying no `type:` field at all.
|
|
|
|
Three of those nine were created *by this session's own renames*, which never
|
|
added a type the files had never had. The pre-flight check — does the forge
|
|
derivation actually see workplans in this repository — is the only reason they
|
|
survived, and it has now caught wrongful retirement three times: `kont-wp-0015`,
|
|
all of `vergabe-teilnahme`, and these nine.
|
|
|
|
**A collision this session created.** Retiring `railiance-platform` first
|
|
required repairing a fault introduced earlier the same day: the `RAILIANCE-WP`
|
|
migration numbered from `RPF-WP-0001` without checking whether the target prefix
|
|
was in use. It was, and the migration collided at 0018, 0019 and 0020. The fleet
|
|
reset then ran over the colliding files and left mixed records on central —
|
|
`rpf-wp-0018` carried one file's status and another's backing path. The
|
|
displaced files moved to 0025-0027. The source numbering was checked before
|
|
renaming; the target was not.
|
|
|
|
**Remaining, and not solvable by retirement:** 9 private repositories central
|
|
cannot clone, and 9 identifier collisions that are identity decisions.
|