Complete RAIL-HO-WP-0005 T11 production cutover
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s

Gitea org archived (79/79), package push disabled, ArgoCD repointed to Forgejo,
and rollback runbook added. Mark T11 done; T12 remains after stabilization.
This commit is contained in:
tegwick 2026-07-08 15:36:22 +02:00
parent b3a065a1c9
commit 96678c210d
2 changed files with 111 additions and 14 deletions

View file

@ -0,0 +1,93 @@
# Forgejo Cutover Rollback Runbook
Date: 2026-07-08
Workplan: `RAIL-HO-WP-0005-T11`
Canon: `the-custodian/docs/forgejo-tier3-remote-url-sweep-playbook.md`
## Current production state (post-T11)
| Layer | Canonical | Fallback |
| --- | --- | --- |
| Git forge | `forgejo.coulomb.social` (`forgejo-remote`) | `gitea.coulomb.social` (archived, read-only) |
| State Hub `remote_url` | `forgejo-remote:coulomb/<slug>.git` | — |
| Workstation `origin` | `forgejo-remote:coulomb/<slug>.git` | `gitea` legacy remote retained |
| ArgoCD (coulombcore) | `https://forgejo.coulomb.social/coulomb/*.git` | Gitea URLs in AppProject allow-list for emergency |
Gitea safety contract: **no repo deletes** for ≥90 days after promotion. Archived
repos remain cloneable; pushes are rejected.
## Per-repo git rollback
When a single repo must revert to Gitea as canonical:
```bash
cd ~/<slug>
git remote rename origin forgejo
git remote rename gitea origin
git fetch origin
git status # confirm tracking branch
```
Optional: re-mirror last known-good Forgejo SHA to Gitea (requires temporarily
un-archiving the repo via Gitea admin API — operator only).
## State Hub rollback
```bash
SLUG=<repo-slug>
API=http://127.0.0.1:8000
curl -s -X PATCH "${API}/repos/${SLUG}" \
-H "Content-Type: application/json" \
-d '{"remote_url": "gitea-remote:coulomb/'"${SLUG}"'.git"}' | python3 -m json.tool
statehub fix-consistency --repo "${SLUG}"
```
## ArgoCD rollback (coulombcore)
Re-point the affected `Application` `spec.source.repoURL` to Gitea, or re-apply
a known-good manifest:
```bash
cd ~/railiance-platform
KUBECONFIG=~/.kube/config kubectl apply -f argocd/applications/<app>.application.yaml
```
AppProjects `railiance-tenants` and `railiance-platform-addons` retain Gitea
URLs in `sourceRepos` for emergency rollback.
## Gitea read-only reversal (operator)
To allow pushes again on a Gitea repo (rollback window only):
```bash
# Un-archive one repo
source ~/.railiance_gitea.conf
curl -sS -X PATCH -H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \
-d '{"archived":false}' \
"https://gitea.coulomb.social/api/v1/repos/coulomb/<slug>"
```
Org-wide un-archive: `the-custodian/tools/gitea-set-readonly.sh` inverted
patch — use only with explicit operator approval.
## Verification after rollback
- [ ] `git push origin main` succeeds on Gitea
- [ ] State Hub `remote_url` matches `gitea-remote:coulomb/<slug>.git`
- [ ] `statehub fix-consistency --repo <slug>` → PASS
- [ ] Dependent CI/deploy paths re-tested
## Emergency org-wide freeze (not primary plan)
If staged drift is unacceptable, invoke org-wide freeze per
`the-custodian/docs/forgejo-production-decisions.md` § Cutover Option A
fallback. Document the decision via State Hub `POST /decisions/` before
executing.
## References
- `docs/forgejo-migration-inventory.md`
- `the-custodian/tools/gitea-set-readonly.sh`
- `the-custodian/tools/patch-forgejo-remote-urls.sh`
- `railiance-enablement/tools/promote-repo-to-forgejo.sh`

View file

@ -121,7 +121,7 @@ Still to prove before T11 closeout:
scheduled backup gate **3/7** (cron PATH fixed 2026-07-08).
- ~~Repo migration ladder (T10).~~ **done** (2026-07-08); 0 Gitea-only repos.
- ~~Operator SSH on Forgejo as `tegwick` (T05/T10).~~ **done**.
- Gitea read-only marking + operator signoff (T11).
- ~~Gitea read-only marking + operator signoff (T11).~~ **done** (2026-07-08).
## Target Architecture
@ -533,9 +533,9 @@ critical migration gaps remain for production repos. **Met.**
```task
id: RAIL-HO-WP-0005-T11
status: progress
status: done
priority: high
needs_human: true
needs_human: false
state_hub_task_id: "b1b66687-ca33-4971-b312-743c8e059c5e"
```
@ -555,14 +555,19 @@ operator approval per `CUST-WP-0054` drain sequence.
**Emergency fallback only:** org-wide freeze-all — not the primary plan; use if
staged drift becomes unacceptable (document rollback before invoking).
**Partial (2026-07-08):** Per-repo promotion **complete** — all Gitea org
repos exist on Forgejo; State Hub remotes and workstation origins on
`forgejo-remote`. Remaining for operator signoff: mark Gitea repos read-only
(per-repo or org-wide), document rollback runbook closeout, confirm no active
workflows still target `gitea-remote`.
**Done (2026-07-08):**
- Gitea org: **79/79 repos archived** (`gitea-set-readonly.sh` idempotent verify).
- Gitea package registry: push disabled (`gitea-disable-package-push.sh`; POST
`/v2/` → 405).
- State Hub: **82 Forgejo** remotes; workstation **0** `gitea-remote` origins.
- ArgoCD (coulombcore): `railiance-apps-root`, `issue-core`, `openbao-secretstore`
repointed to `forgejo.coulomb.social` (`railiance-platform` `6076d57`).
- Rollback runbook: `docs/forgejo-cutover-rollback.md`.
- Forgejo Actions workflows: no `gitea-remote` references in `.forgejo/workflows/`.
**Done when:** all Railiance/Custodian repos use Forgejo as primary, Gitea is
read-only fallback, and rollback instructions are documented.
read-only fallback, and rollback instructions are documented. **Met.**
---
@ -610,11 +615,9 @@ T05+T08 ──► T10 migration ladder ──► T11 production cutover ──
T03 isolated probe: CANCELLED (superseded by T05 + in-production pilots)
```
**Current focus (2026-07-08):** T01/T05/T06/T07/T08/T10 **done**; T04/T09
await 7-day backup gate (**3/7**; cron PATH fixed 2026-07-08). Forgejo org
**84 repos**; **0** Gitea-only; State Hub **82 Forgejo** remotes. T11 **in
progress** — per-repo cutover complete; operator signoff for Gitea read-only +
rollback doc closeout. T12 blocked on T11 stabilization.
**Current focus (2026-07-08):** T01/T05/T06/T07/T08/T10/T11 **done**; T04/T09
await 7-day backup gate (**3/7**; cron PATH fixed). T12 next after
stabilization window — Gitea archived + package push disabled; ArgoCD on Forgejo.
**Absorbed by `CUST-WP-0054-T04`:** forge + CI on railiance01; workstation
build retirement; staged repo promotion before State Hub primary move (T05).
@ -642,6 +645,7 @@ purpose is identified.
- `CUST-WP-0021-multi-host-repo-paths.md`
- `docs/adr/ADR-004-forgejo-in-cluster-actions-runner.md`
- `docs/forgejo-migration-inventory.md`
- `docs/forgejo-cutover-rollback.md`
- `the-custodian/docs/forgejo-production-decisions.md`
- `the-custodian/docs/forgejo-repo-migration-pilot-glas-harness.md`
- `railiance-apps/docs/forgejo-on-railiance01.md`