state-hub/workplans/STATE-WP-0083-forge-derived-projection-reset.md

---
id: STATE-WP-0083
type: workplan
title: "Forge-derived projection reset, per repository"
domain: infotech
repo: state-hub
status: active
owner: codex
topic_slug: infotech
created: "2026-08-25"
updated: "2026-08-25"
related:
  - CUST-ADR-012
  - CUST-WP-0068
---

# Forge-derived projection reset, per repository

## Goal

Implement `ADR-012` decision 7: discard a repository's projection and rebuild it
from what Forgejo holds. Per repository, routine, idempotent, and verifiable.

This is the only sanctioned way to remove a hub record. The hub has no
hard-delete for work records — `DELETE /tasks/{id}` is `cancel_task`,
`DELETE /workstreams/{id}` is `410 Gone`, and no `session.delete` exists for
either. Re-derivation is not one option among several; it is the mechanism.

## Why here and not in Repo Manager

`ADR-012` decision 1 says central derives from the forge, and the pod can reach
it: `git ls-remote https://forgejo.coulomb.social/...` succeeds anonymously from
inside the cluster, and `git` is already in the image.

Driving the reset from Repo Manager would mean computing a projection on a
workstation and pushing it into central — precisely the injection of derived
state `ADR-010` decision 5 forbids. Central must do its own reading. The cost is
that workplan parsing exists in two places; `scripts/consistency_check.py`
already parses frontmatter and task blocks here, so the capability is present
and should be extracted rather than rewritten.

## What it must not do

- **Not restore preliminary overlay records.** They exist because the forge does
  not hold them; a rebuild from the forge cannot reproduce them and must not
  pretend to.
- **Not destroy hub-native records.** Progress events, decisions and inbox
  messages originate in the hub (`ADR-010` decision 4). They are not
  forge-derived and must survive a rebuild of forge-derived state.
- **Not proceed silently when records would be lost.** Stop, name them, and
  require an explicit acknowledgement.

## Derive a repository's projection from the forge

```task
id: STATE-WP-0083-T01
status: done
priority: high
```

Given a repository slug, clone or fetch its default branch from Forgejo into a
temporary location, parse the workplan files, and compute the projection that
state would produce: workplans, tasks, their identifiers, statuses and backing
paths.

Read-only and side-effect free. Record the commit the derivation came from —
`ADR-012` decision 2 requires provenance, and the existing `git_fingerprint` is
worthless precisely because nothing ever wrote it correctly.

Acceptance: deriving `the-custodian` twice from the same commit yields identical
output, and the commit is reported.

**Done (2026-08-25).** `api/services/forge_projection.py`. Central clones the
default branch from Forgejo and derives 69 workplans and 459 tasks from
`the-custodian` at commit `d5013ae`, identically across runs. Identifiers are
derived from the canonical record id in the `ADR-007` namespace, verified against
live records — so a forge-derived projection and a preliminary overlay agree on
identity without reconciliation. Eight tests, including that identifiers are
derived rather than inherited from whatever a file happens to carry.

A fresh shallow clone each time is deliberate: reusing a working copy is how a
projection ends up reflecting someone's local state instead of the forge.

## Report the difference before changing anything

```task
id: STATE-WP-0083-T02
status: progress
priority: high
```

Compare the derived projection against what the hub currently holds for that
repository, and report: records the forge has that the hub lacks, records the
hub has that the forge does not, and records whose fields differ.

This is the half that is immediately useful without any destructive capability,
and it is what makes decision 7's "verifiable" real. It also answers the
question `CUST-WP-0068-T09` is waiting on — which stale rows would actually
clear — before anyone commits to clearing them.

Acceptance: a dry-run diff for a repository with known drift matches what
manual inspection shows.

**Workplan-level diff done; task-level blocked (2026-08-25).**

The first implementation matched hub records to derived ones *by UUID* and was
badly wrong: most hub records still carry pre-`ADR-007` random identifiers, so
nearly every record appeared simultaneously missing and stale. A reset built on
that comparison would have destroyed and recreated the whole projection. It now
matches on canonical record id, falling back to the backing file.

Workplan-level results are trustworthy and inspectable:

| Repository | missing | stale | differing |
|---|---|---|---|
| `whitehat-security` | 0 | 0 | 0 |
| `the-custodian` | 3 | 4 | 52 |
| `railiance-platform` | 18 | 25 | 2 |

`whitehat-security` reporting clean is the control: it was bootstrapped directly
from its files, so a forge derivation must agree with it exactly. The four stale
workplans on `the-custodian` were checked by hand and are real —
`CUST-WP-0023`/`0024` have no file in the repository at all, and
`state-hub-v0.1`/`v0.2` carry slugs that were never canonical ids. All four are
genuine hub-first records, the class `ADR-010` says to disposition.

**Blocked: hub tasks carry no canonical record id.** The task schema is
`id, workplan_id, title, status, priority, …` with nothing holding
`CUST-WP-0067-T01`. A file task and a hub task can therefore only be matched by
title, which is why the task diff reports 149 missing and 131 stale for
`the-custodian` where the workplan diff reports 3 and 4. That is the matching
failing, not drift.

Task-level reset must not be built on title matching — renaming a task heading
would silently destroy and recreate its record. `T03` is limited to workplans
until tasks carry their canonical id, which is `T06`.

## Apply the reset transactionally

```task
id: STATE-WP-0083-T03
status: done
priority: high
```

Replace the repository's forge-derived records with the derived projection in
one transaction: create what is missing, update what differs, remove what no
longer derives. Hub-native records are untouched.

Refuse by default when removal would destroy a record with no counterpart in the
forge; report what would be lost and require an explicit acknowledgement to
proceed. That refusal is evaluated per repository, so one unresolved repository
never blocks the rest.

Acceptance: reset twice produces the same projection; a repository holding
records the forge lacks is refused with those records named; hub-native record
counts are unchanged across a reset.

**Done (2026-08-26).** `reset_repository_projection()` in
`api/services/forge_projection.py`. Verified against live data, rolled back:
`whitehat-security` applied 5 updates and 0 retirements; `the-custodian`
**refused**, naming `cust-wp-0023`, `cust-wp-0024`, `state-hub-v0.1` and
`state-hub-v0.2` — the four records confirmed by hand as genuine hub-first
records with no file — and changed nothing.

Refusal is the default because a record that stops deriving is ambiguous: the
file may have been deleted deliberately, or the caller may have pointed at the
wrong branch. Only the caller can say which, so only the caller may authorise it.

Retirement never deletes, with a test asserting the row survives and
`session.delete` is never called. A record that derives again is un-retired
rather than left contradicting the forge.

The first execution of this write path was run against the **cache** database
rather than central, and rolled back. A write path's first run belongs on the
discardable copy.

**Scope limit, enforced in the code rather than documented beside it.** Tasks of
existing workplans are untouched; tasks are created only alongside a *new*
workplan, where nothing exists to mis-match. The outcome carries that as a note
so partial convergence cannot be mistaken for full. 678 tests pass.

## Fleet form as a loop over the repository form

```task
id: STATE-WP-0083-T04
status: todo
priority: medium
```

The fleet-wide reset iterates the per-repository reset and shares its
implementation, so the rarely-used dangerous path is exercised by the frequently
used safe one. Repositories that refuse are skipped and reported, never aborting
the pass.

Acceptance: a fleet dry-run reports per-repository outcomes including refusals,
and completes despite them.

## Retire what the reset replaces

```task
id: STATE-WP-0083-T05
status: wait
priority: medium
```

Once reset is trusted, `CUST-WP-0068-T09` can clear its stale rows — 4 workplan
rows from the prefix migration and 305 task rows across 44 workplans.

`ADR-003` decision 2's `mtime`-based fingerprints should also be replaced by the
source commit at this point; `ADR-012` invalidated that composition and the
replacement belongs with the provenance work in `T01`.

Acceptance: `CUST-WP-0068-T09` closes; no fingerprint input depends on a local
filesystem.


## Give hub tasks their canonical record id

```task
id: STATE-WP-0083-T06
status: done
priority: high
```

Hub task rows hold no canonical identifier — only a title — so nothing reliably
connects `CUST-WP-0067-T01` in a file to its row. Every other record type has a
stable identity; tasks do not, and that gap is what stops the reset from covering
them.

Add the canonical id to the task record and populate it during derivation and
registration. Once present, task matching becomes identity-based like workplans,
and `T03` can extend to tasks safely.

Until then a task's identity is its title, which changes whenever someone edits a
heading. That is not a foundation for deletion.

Acceptance: task rows carry their canonical record id; the task diff for
`whitehat-security` reports clean; `the-custodian`'s task diff falls to something
manual inspection confirms.

**Built (2026-08-26).** Migration `e2b3c4d5f6a7` adds `tasks.record_id`,
nullable because no migration can invent an identity for an existing row — only
the repository files hold the mapping.

The backfill *reads* that mapping rather than inferring it: a file task declares
both its canonical id and the projection UUID it was registered under, so the
pairing is stated, not guessed. Across 121 repositories and 1104 files it
recovered **5516 pairs with zero conflicts**, and identified **4456 of 6073**
task rows in the cache database.

Diff and reset now key on `record_id` where present, falling back to a
`title:`-prefixed key so an unidentified row is *visibly* unidentified rather
than silently title-matched.

Two refusals are tested rather than documented: a row the files do not claim
keeps no identity, and a row that already has one is never overwritten — a
mismatch is recorded as a conflict, not resolved. Title matching would have
"worked" and destroyed a record every time someone edited a heading.

**Remaining, in order.** Deploy the migration to central; then run the backfill
against central as an explicit operation — deliberately *not* inside the Helm
hook, where a partial failure would silently leave half the tasks identified.
The cache matched 4456 of 6073 and central has a different history, so the
numbers will differ and a dry run should be compared before applying.

Only then may `T03` be extended to tasks of existing workplans, which is a
further change with its own verification. `CUST-WP-0068-T09` is therefore two
moves away, not one.

**Backfill run against central (2026-08-26), and a defect it exposed.**

The first implementation read local filesystem paths. Central has no workstation
checkout and must not depend on one — `ADR-012` decision 1 makes the forge the
projection source — and the point became concrete rather than theoretical:
central's Postgres is unreachable from the workstation, since the tunnel carries
HTTP only. A forge-sourced variant now clones each repository in-cluster and
reads the pairing from what the forge holds.

Dry run across 121 repositories: 1094 files, 5620 pairs, 5248 to update, 372
file-declared UUIDs central does not hold, **zero conflicts**. The count
differing from the cache's 4456 of 6073 was the check that mattered — central
holds the re-keyed records the cache never had, so a *matching* number would have
meant cache state was leaking in. The forge scan reading 1094 files against the
workstation's 1104 is the preliminary-overlay distinction showing up in the
numbers.

**Then applied, and wrong.** 5248 rows were identified but only 5100 identities
were distinct. A task id written as a bare `T01` is unique only inside its own
workplan; stored as canonical it gave every workplan's first task one identity.
51 such ids landed on 148 rows.

The backfill's own conflict detection could not see it: it checks one UUID
claimed by two ids, and this was the inverse. It surfaced only because the
identified count and the distinct count did not reconcile — a check that could
easily have been skipped after a dry run reporting zero conflicts.

Short ids are now qualified as `WORKPLAN-ID-T01`, and a short id in a file with
no workplan id gets no identity at all. The 136 affected rows on central were
cleared for reassignment, since the backfill never overwrites an existing
identity.

**Central now: 5112 of 5974 identified, 5077 distinct.** The remaining 34
duplicates are not from this work. They are unqualified ad-hoc ids reused across
repositories — `ADHOC-2026-07-02-T01` exists in three workplans — which is the
problem `CUST-WP-0066` closed for new records while existing ones kept their
unqualified ids. They need their own disposition.

**Done (2026-08-26).** Corrected backfill applied to central: **5248 of 5974
tasks identified, 5213 distinct, 35 duplicate identities.**

The check before applying was uniqueness, not volume — the lesson from the first
attempt, where a clean dry run reporting zero conflicts concealed 51 non-unique
ids. All 136 pending updates were verified to take identities held by nothing
else before a single row was written.

The 35 remaining duplicates predate this work and fall into three classes:

| Count | Class | Origin |
|---|---|---|
| 18 | `KONT-WP-0013-T00N` and similar | a workplan registered twice, so both copies claim the same task ids |
| 12 | `RAILIANCE-WP-00NN-T0N` | archived RAILIANCE-WP files, left unrenamed by the owner decision of 2026-08-25 — the same numbers exist in three repositories, so their task ids collide |
| 4 | `ADHOC-*-T0N` | unqualified ad-hoc ids reused across repositories, the class `CUST-WP-0066` closed for new records only |

The `RAILIANCE-WP` twelve are worth noting as a consequence rather than a
surprise: migrating only the *active* workplans was the right call for blast
radius, and this is the price — the archived copies keep colliding identities
until they are migrated too, and task-level reset cannot act on those workplans
in the meantime.

**Task-level reset remains blocked**, now on a narrower and better-understood
problem: 35 identities, not 5974 unidentified rows.


## Restore a migration mechanism for central

```task
id: STATE-WP-0083-T07
status: done
priority: high
```

**Central's schema is behind the code it is running, and nothing detects it.**
Discovered 2026-08-25 while preparing the retirement columns.

Central is at revision `b8d4f0a2c6e1`; the chain is
`b8d4f0a2c6e1 → c9e5a1b3d7f2 → d1a2b3c4e5f6`. The `review_contracts` table does
not exist on central even though the migration creating it ships inside the image
currently serving traffic. The code and the database disagree silently.

There is no migration mechanism. The image `CMD` is bare `uvicorn`, the Helm
chart declares no migration job, and the only evidence migrations ever ran was an
ad-hoc `state-hub-alembic-init` Job created outside the chart — deleted earlier
the same day as a retired-forge leftover. Deleting a completed Job removed a
historical record rather than a working mechanism, since Jobs do not re-run, but
it also removed the last visible sign that this gap existed.

This blocks `T03`: the retirement columns cannot reach central without it.

Needed: migrations as a declared part of the release — a chart-managed job or
init container that runs `alembic upgrade head` before the API serves — and a
check that surfaces a schema/code mismatch instead of leaving it silent. A
deployment that can run against a schema it was not built for is the same class
of defect as a projection that cannot name its source commit.

Acceptance: central reaches `head`; `review_contracts` exists; the mechanism is
declared in the chart; a mismatch between code and schema is reported rather than
tolerated.

**Done (2026-08-26).** Central reports
`schema: {status: ok, applied: d1a2b3c4e5f6, expected: d1a2b3c4e5f6}`.
`review_contracts` exists. Both pods run `main-97c8762`.

The first execution is worth recording, because it failed in a way that proved
the design. The `pre-upgrade` hook ran, applied both migrations and deleted
itself on success — then the k3s API tunnel dropped (`unexpected EOF` →
`connection refused`) and Helm could not read the deployment to roll the API.
The release wedged at `pending-upgrade`; no pod rolled.

The schema was therefore briefly *ahead* of the running code, which is the safe
direction and the reason additive migrations were the right shape: the old image
served correctly against the new schema throughout. A destructive migration in
the same circumstance would have taken the service down.

Recovery was `helm rollback` to the last deployed revision — which clears the
wedge without touching the schema, there being nothing to undo — then re-running
the upgrade. The hook is idempotent, so the second run's `alembic upgrade head`
was a no-op.

Two things this exposed, neither yet addressed:

- The hook's `hook-succeeded` delete policy removes the Job on success, so a
  *successful* migration leaves no trace. That made it look as though nothing had
  run when in fact everything had. Retaining a short-lived record of successful
  migrations, not only failed ones, would have answered the question immediately.
- `bridge status` reported `k3s-api-railiance01` as `connected` while holding a
  stale pid, so its health signal did not reflect the dropped connection. A
  tunnel monitor that cannot detect the failure it exists to detect is the same
  defect class as a projection that cannot name its source commit.


## Repair workplan files whose frontmatter never terminates

```task
id: STATE-WP-0083-T08
status: todo
priority: high
```

**Blocks the first production reset.** Found 2026-08-26 by reviewing the reset's
diff before applying it — the diff proposed retiring `kont-wp-0015`, a live and
correctly registered workplan, which is what prompted the investigation.

Eighteen workplan files across the fleet have a closing `---` fused onto the last
frontmatter value:

```yaml
depends_on_workplans:
  - KONT-WP-0016---
```

There is no delimiter line, so the frontmatter never terminates. The file
declares a dependency on a workplan literally named `KONT-WP-0016---` and the
entire body is swallowed as frontmatter. Any parser selecting on `type: workplan`
sees nothing, so the file is not merely unparsed but *invisible* — it appears as
neither a workplan nor an error.

Affected: `citation-evidence` (5), `infospace-bench` (2), `kontextual-engine`,
`net-kingdom` (2 incl. archived), and others — 18 in total.

The consequence for reset is severe and must not be worked around: a repository
containing one of these files derives a projection missing that workplan, so the
reset proposes retiring a record that is entirely correct. Running it before this
is repaired would retire live work.

That the refusal-by-default caught this is the design working, but the diff would
have looked plausible to anyone not checking a specific record — `created=0,
would_retire=N` reads like a tidy-up.

Repair the frontmatter, then re-derive and confirm each affected repository's
diff no longer proposes retiring live workplans.

Acceptance: no workplan file yields empty frontmatter while starting with `---`;
the reset diff for each affected repository proposes retiring only records whose
backing file genuinely no longer exists.