Mark package registry task complete after npm probe; document hub drift resolution for six tier-2.5 repos and current backup gate status.
635 lines
26 KiB
Markdown
635 lines
26 KiB
Markdown
---
|
||
id: RAIL-HO-WP-0005
|
||
type: workplan
|
||
title: "Forgejo Production Migration on railiance01"
|
||
domain: financials
|
||
repo: railiance-infra
|
||
status: active
|
||
owner: railiance
|
||
topic_slug: railiance
|
||
created: "2026-05-03"
|
||
updated: "2026-07-08"
|
||
state_hub_workstream_id: "84e17675-0d15-4268-a8bd-540124d37018"
|
||
---
|
||
|
||
# Forgejo Production Migration on railiance01
|
||
|
||
## Goal
|
||
|
||
Establish Forgejo as the production-grade source forge and package base for
|
||
Railiance, then migrate all repositories and workflows currently relying on
|
||
Gitea to the new Forgejo installation.
|
||
|
||
Forgejo will become the heart of Railiance infrastructure. The work must be
|
||
fully automated, backup-backed, recovery-drilled, and suitable for long-lived
|
||
operation on railiance01 before any production cutover happens.
|
||
|
||
**Sequencing update (2026-07-07):** Production Forgejo is live on railiance01.
|
||
Repo cutover proceeds **staged per-repo** using a migration ladder (disposable
|
||
probes → non-production pilots → image-capable pilots → production repos).
|
||
`state-hub` tier-3 cutover to railiance01 **completed 2026-07-06** under
|
||
`CUST-WP-0054-T05` (phases 1–5 in progress; stabilization window through
|
||
2026-07-09). Gitea remains read-only fallback for unmigrated repos. See
|
||
`CUST-WP-0054-T04`, `the-custodian/docs/forgejo-repo-migration-pilot-glas-harness.md`,
|
||
and `the-custodian/docs/state-hub-railiance01-cutover-plan.md`.
|
||
|
||
## Placement in the Railiance Tooling Set
|
||
|
||
This workplan lives in `railiance-infra` because it is the cross-layer
|
||
production infrastructure coordination plan and belongs next to
|
||
`RAIL-HO-WP-0004-production-readiness.md`.
|
||
|
||
Implementation must respect the OAS repo boundaries:
|
||
|
||
| Concern | Repo | Layer |
|
||
|---------|------|-------|
|
||
| Server prerequisites, inventory, OS packages, SSH/system users | `railiance-infra` | S1 |
|
||
| k3s runtime prerequisites, namespaces, ingress class, cluster backup hooks | `railiance-cluster` | S2 |
|
||
| PostgreSQL, object storage, backup targets, registry storage dependencies | `railiance-platform` | S3 |
|
||
| Forgejo Actions runner templates, CI conventions, migration automation | `railiance-enablement` | S4 |
|
||
| Forgejo Helm release, app config, mail config, package registry, app backups | `railiance-apps` | S5 |
|
||
|
||
This file is the umbrella plan. If an implementation step requires files in a
|
||
different repo, that repo should receive its own workplan or task before the
|
||
change is made there.
|
||
|
||
## Key Decisions to Confirm
|
||
|
||
1. ~~Public/private hostname for Forgejo~~ **DECIDED 2026-07-03:**
|
||
`forgejo.coulomb.social` → railiance01 (`92.205.62.239`). DNS active;
|
||
Traefik edge live; Forgejo workload deployed and serving HTTPS. Gitea remains
|
||
canonical until migration drills pass. Record:
|
||
`the-custodian/docs/forgejo-production-decisions.md`.
|
||
2. ~~Mail delivery path~~ **DECIDED 2026-07-07:** IONOS SMTP for `@coulomb.social`
|
||
mailboxes (operator-provisioned domain/subdomains). Sender mailbox TBD
|
||
(`forgejo@coulomb.social`); SPF/DKIM via IONOS zone. T06 SMTP delivery
|
||
verified 2026-07-08.
|
||
3. ~~Package registry scope~~ **DECIDED 2026-07-08 (Option C):** OCI + **npm** +
|
||
generic at launch; PyPI, Go, Maven, Helm as inventory or consumer need proves.
|
||
Record: `the-custodian/docs/forgejo-production-decisions.md` § Package scope.
|
||
4. ~~Actions runner model~~ **DECIDED 2026-07-03:** in-cluster long-lived runner
|
||
Deployment with DinD sidecar on railiance01 (`ADR-004`). Interim coulombcore
|
||
host runner retired after cutover.
|
||
5. ~~Backup destination and retention~~ **DECIDED 2026-07-09 (Option A):**
|
||
daily age-encrypted `forgejo dump` + `pg_dump` → Nextcloud WebDAV; 14 daily +
|
||
4 weekly retention; RPO 24h / RTO 4h. Record:
|
||
`the-custodian/docs/forgejo-production-decisions.md` § Backup.
|
||
6. ~~Cutover mode~~ **DECIDED 2026-07-09 (Option A):** staged per-repo promotion
|
||
(no org-wide freeze). Freeze-all documented as emergency fallback only. Record:
|
||
`the-custodian/docs/forgejo-production-decisions.md` § Cutover.
|
||
|
||
## Safety Contract
|
||
|
||
- Gitea remains the production source of truth until Forgejo restore and
|
||
migration drills pass.
|
||
- No repository is deleted from Gitea during this workplan.
|
||
- A fresh Gitea backup must be taken before every migration drill and before
|
||
final cutover.
|
||
- Forgejo backups must be restored into an isolated namespace before accepting
|
||
production use.
|
||
- Password reset and email recovery must be verified with a real controlled
|
||
account before onboarding users.
|
||
- Forgejo Actions may not receive broad cluster credentials by default; runner
|
||
permissions must be least-privilege and repo-scoped where practical.
|
||
- Secrets stay in SOPS/age or Kubernetes Secrets managed by the appropriate
|
||
repo. No plaintext SMTP passwords, admin tokens, runner tokens, or registry
|
||
credentials in Git.
|
||
|
||
## Probe and pilot strategy (revised 2026-07-04)
|
||
|
||
Original T03 planned a **disposable isolated-namespace probe** before any
|
||
production install. That path was **superseded**: production Forgejo deployed on
|
||
railiance01 under the safety contract (Gitea remains canonical; no Gitea deletes).
|
||
|
||
Integration evidence now comes from **in-production probes and repo pilots**:
|
||
|
||
| Tier | Repo | Purpose | Status |
|
||
| --- | --- | --- | --- |
|
||
| 0 | `coulomb/forgejo-actions-probe` | Runner scheduling, DinD, OCI image-build | **done** |
|
||
| 1 | `coulomb/glas-harness` | Non-production git+SSH+CI routing drill | **done** |
|
||
| 2 | `coulomb/key-cape` | Image-build workflow + registry pull on railiance01 | **done** |
|
||
| 2.5 | `railiance-enablement`, `railiance-infra`, `railiance-apps`, `railiance-platform`, `railiance-cluster` | Stack repos on Forgejo + S4 templates | **done** |
|
||
| 3 | Production set (`state-hub`, `issue-core`, …) | Canonical remotes, sweep paths, deploy loops | **in progress** (`state-hub` cutover 2026-07-06) |
|
||
|
||
Each tier must pass before the next. T03 (isolated probe namespace) is cancelled;
|
||
acceptance criteria below are tracked across T05, T07, T08, and T10 instead.
|
||
|
||
Still to prove before T11:
|
||
|
||
- ~~SMTP/password reset end-to-end (T06).~~ **done** (2026-07-08).
|
||
- Backup and restore in isolated namespace (T09) — **drill passed 2026-07-04**;
|
||
scheduled automation pending.
|
||
- Issues/releases/wiki/LFS per inventory classification (T10 matrix).
|
||
- Operator SSH identity on Forgejo beyond interim `forgejo_admin` keys (T02/T10).
|
||
|
||
## Target Architecture
|
||
|
||
```
|
||
operator / agents / developers
|
||
-> https://forgejo.coulomb.social (railiance01 Traefik ingress)
|
||
-> forgejo Service in forgejo namespace
|
||
-> Forgejo Deployment/StatefulSet
|
||
-> forgejo-db CloudNative PG Cluster in databases namespace
|
||
-> Valkey/cache if required
|
||
-> persistent storage for repositories, attachments, LFS, packages
|
||
-> Actions runner(s) with restricted execution scope
|
||
-> backup jobs to the approved backup target
|
||
```
|
||
|
||
## Tasks
|
||
|
||
### T01 — Inventory current Gitea functionality and migration requirements
|
||
|
||
```task
|
||
id: RAIL-HO-WP-0005-T01
|
||
status: progress
|
||
priority: high
|
||
state_hub_task_id: "cf59d171-5629-45c9-9d44-8d6499827ffc"
|
||
```
|
||
|
||
Create a source-of-truth inventory of current Gitea usage.
|
||
|
||
First-pass inventory artifact: `docs/forgejo-migration-inventory.md`.
|
||
|
||
Minimum inventory:
|
||
|
||
- All repositories in the `coulomb` organization.
|
||
- Registered vs unregistered State Hub repos.
|
||
- Users, organizations, teams, deploy keys, SSH keys, access tokens.
|
||
- Issues, labels, milestones, releases, wiki, packages, LFS, attachments.
|
||
- Existing webhook usage and automation assumptions.
|
||
- Current Gitea package registry status and the missing `[packages]` config
|
||
that is blocking container image publication.
|
||
|
||
**Done when:** the inventory identifies every feature that must work in
|
||
Forgejo before cutover and classifies each migration item as automatic,
|
||
manual, unsupported, or explicitly out of scope.
|
||
|
||
**Gap (2026-07-07):** Forgejo org snapshot refreshed (14 repos; `ops-warden` tier-3).
|
||
Forgejo admin inventory captured (2 users, 1 team, 0 org hooks). Authenticated
|
||
**Gitea** admin inventory (users, teams, deploy keys, tokens, webhooks, LFS)
|
||
still blocked — no Gitea admin PAT on workstation. Hub drift: 7 Forgejo repos
|
||
still register Gitea `remote_url` in State Hub.
|
||
|
||
---
|
||
|
||
### T02 — Resolve Forgejo production design decisions
|
||
|
||
```task
|
||
id: RAIL-HO-WP-0005-T02
|
||
status: done
|
||
priority: high
|
||
needs_human: false
|
||
state_hub_task_id: "f88115bf-4f99-49ef-a415-0b23750141b3"
|
||
```
|
||
|
||
Decide the production choices listed in "Key Decisions to Confirm".
|
||
|
||
**Done (2026-07-09):** All Key Decisions recorded in
|
||
`the-custodian/docs/forgejo-production-decisions.md`:
|
||
|
||
- Hostname, exposure, deployment pattern, live deploy (2026-07-03)
|
||
- SMTP IONOS + OpenBao/ESO (T06 done 2026-07-08)
|
||
- Package scope **Option C** (OCI + npm + generic; others as needed)
|
||
- Actions runner in-cluster (`ADR-004`)
|
||
- Backup **Option A** (Nextcloud + age; balanced retention)
|
||
- Cutover **Option A** (staged per-repo; freeze-all emergency fallback only)
|
||
|
||
Expected output:
|
||
|
||
- A short decision record in this workplan or a dedicated ADR. ✓
|
||
- Hostname and exposure model. ✓
|
||
- SMTP provider and sender identity. ✓
|
||
- Package registry scope. ✓
|
||
- Actions runner isolation model. ✓
|
||
- Backup target, retention, encryption, and restore cadence. ✓
|
||
- Cutover strategy and rollback window. ✓ staged per-repo + per-repo rollback
|
||
|
||
**Done when:** implementation tasks are no longer blocked by open production
|
||
choices. **Met.**
|
||
|
||
---
|
||
|
||
### T03 — Build forgejo-railiance-probe (isolated namespace)
|
||
|
||
```task
|
||
id: RAIL-HO-WP-0005-T03
|
||
status: cancel
|
||
priority: high
|
||
state_hub_task_id: "b516018a-415e-4a58-8c62-07c14ece9353"
|
||
```
|
||
|
||
**Cancelled 2026-07-04:** superseded by production Forgejo on railiance01 (T05)
|
||
plus in-production integration probes (`forgejo-actions-probe`, `glas-harness`).
|
||
Isolated-namespace probe added latency without reducing risk given the safety
|
||
contract (Gitea canonical, no deletes). Remaining T03 acceptance items map to:
|
||
T05 (deploy), T06 (mail), T07 (packages), T08 (Actions), T09 (backup restore),
|
||
T10 (repo migration drill).
|
||
|
||
---
|
||
|
||
### T04 — Define Forgejo platform services
|
||
|
||
```task
|
||
id: RAIL-HO-WP-0005-T04
|
||
status: progress
|
||
priority: high
|
||
state_hub_task_id: "28b351fe-bfbe-4a8b-bbfa-1b148e69f8e0"
|
||
```
|
||
|
||
In `railiance-platform`, define production platform services for Forgejo.
|
||
|
||
Minimum scope:
|
||
|
||
- `forgejo-db` CloudNative PG cluster.
|
||
- Database credentials via SOPS-managed Secret or approved secret flow.
|
||
- Backup configuration: daily logical `pg_dump` + off-node Nextcloud target
|
||
(T02 Option A; WAL/S3 deferred).
|
||
- Object storage or persistent volume plan for repositories, attachments, LFS,
|
||
packages, Actions artifacts, and logs.
|
||
- Restore runbook for database and blob/package data.
|
||
|
||
**Partial (2026-07-07):** `forgejo-db` CNPG cluster healthy on railiance01.
|
||
SOPS secret path and network policies in `railiance-platform`. **Backup target
|
||
decided (Option A, Nextcloud).** **Automation:** `railiance-platform`
|
||
`make forgejo-backup` / `docs/forgejo-backup.md` — daily `forgejo dump` +
|
||
`pg_dump` → age → Nextcloud `forgejo/`. **First production backup succeeded
|
||
2026-07-07** (~686M dump + 2.1M pg_dump uploaded). Daily cron installed
|
||
(02:15 UTC). Remaining: 7-day success gate (1/7), restore drill from Nextcloud
|
||
artifact (not `/tmp/forgejo-drill/`).
|
||
|
||
**Done when:** platform dependencies can be deployed and restored without the
|
||
Forgejo app running.
|
||
|
||
---
|
||
|
||
### T05 — Define production Forgejo application deployment
|
||
|
||
```task
|
||
id: RAIL-HO-WP-0005-T05
|
||
status: done
|
||
priority: high
|
||
state_hub_task_id: "11540ba4-d31c-4f64-836b-c6de69107aa4"
|
||
```
|
||
|
||
In `railiance-apps`, create the production Forgejo deployment.
|
||
|
||
Minimum scope:
|
||
|
||
- Forgejo Helm release or manifests in the S5 boundary.
|
||
- App configuration for database, SSH, HTTPS, mailer, packages, LFS, and
|
||
security settings.
|
||
- Initial admin/user bootstrap that is automated but does not commit secrets.
|
||
- Health/status targets in the Makefile.
|
||
- Migration-safe configuration for coexistence with Gitea during the cutover.
|
||
|
||
**Partial (2026-07-07):** `railiance-apps` deploy live — HTTPS smoke pass,
|
||
ingress + TLS, SSH NodePort `30022`, Actions enabled, `coulomb` org,
|
||
in-cluster runner (ADR-004). Git push/pull via HTTPS and `forgejo-remote` SSH
|
||
proven as operator `tegwick` (not `forgejo_admin`). Mailer via OpenBao/ESO.
|
||
`make forgejo-verify` T05 acceptance: **10 pass / 0 fail** (2026-07-07).
|
||
`make forgejo-operator-bootstrap` idempotent. Security: `INSTALL_LOCK`,
|
||
`COOKIE_SECURE`, `DISABLE_REGISTRATION`. Runner registration SOPS captured
|
||
2026-07-08 (`helm/forgejo-runner-registration.sops.yaml`; `make
|
||
forgejo-secrets-check` pass).
|
||
|
||
**Done when:** Forgejo runs on railiance01 against production platform
|
||
services and can serve login, git clone/push, package registry, and admin
|
||
operations.
|
||
|
||
---
|
||
|
||
### T06 — Implement usable email recovery cycle
|
||
|
||
```task
|
||
id: RAIL-HO-WP-0005-T06
|
||
status: done
|
||
priority: high
|
||
needs_human: false
|
||
state_hub_task_id: "417faa4d-eab8-4247-9485-4f80e5d5b7ff"
|
||
```
|
||
|
||
Configure and test mail delivery for account recovery.
|
||
|
||
**Provider decided (2026-07-07):** IONOS SMTP for `@coulomb.social` mailboxes.
|
||
See `the-custodian/docs/forgejo-production-decisions.md` and
|
||
`railiance-apps/docs/forgejo-on-railiance01.md` (SMTP section).
|
||
|
||
**Verified (2026-07-08):** OpenBao path `platform/workloads/forgejo/forgejo-mailer`
|
||
(field `PASSWD`) → ESO → Helm `GITEA__mailer__PASSWD`. Mailer uses
|
||
`smtp.ionos.de:587` with `PROTOCOL=smtp+starttls` (EU relay; `.com` auth fails).
|
||
Operator test email and forgot-password flow delivered to `bernd.worsch@gmail.com`.
|
||
|
||
Minimum scope:
|
||
|
||
- SMTP credentials stored through the approved secret path (OpenBao + ESO).
|
||
- Sender address and domain alignment documented.
|
||
- Password reset email works for a controlled non-admin account.
|
||
- Account recovery runbook covers lost password, lost MFA, disabled account,
|
||
and emergency admin access (emergency path: `forgejo admin user change-password`).
|
||
- Mail failure is observable through logs or a health check.
|
||
|
||
**Done when:** a user can complete password recovery without operator database
|
||
edits, and the operator has a documented emergency path.
|
||
|
||
---
|
||
|
||
### T07 — Enable and harden package registry base
|
||
|
||
```task
|
||
id: RAIL-HO-WP-0005-T07
|
||
status: done
|
||
priority: high
|
||
state_hub_task_id: "9578f672-e2b8-43a3-8419-5f86f8871326"
|
||
```
|
||
|
||
Enable Forgejo packages for Railiance's near-term build and deployment needs.
|
||
|
||
**Scope decided (T02, 2026-07-08):** Option C — multi-ecosystem registry.
|
||
|
||
Initial package types:
|
||
|
||
- **OCI** container registry (proven; tier 0–3).
|
||
- **npm** — launch requirement; publish/pull + CI template (T07).
|
||
- **Generic** packages for release artifacts.
|
||
- **PyPI, Go, Maven, Helm** — enable per repo when T01 inventory or consumer
|
||
need proves it; not a default big-bang.
|
||
|
||
Acceptance:
|
||
|
||
- Authenticated push and pull works from operator workstation and railiance01.
|
||
- Container image pull works from k3s deployments.
|
||
- Retention and cleanup expectations are documented.
|
||
- Package data is included in backup and restore drills.
|
||
|
||
**Partial (2026-07-08):** OCI registry live (`/v2/` auth challenge). Tier-0/2
|
||
images built and pulled on railiance01. Tier-3 `state-hub` images proven.
|
||
**npm (2026-07-08):** `@coulomb/forgejo-npm-probe@0.0.1` publish + install
|
||
verified; `make forgejo-npm-smoke`; docs `railiance-apps/docs/forgejo-package-registry.md`;
|
||
CI template `railiance-enablement/workflows/npm-publish.yaml`. Retention documented;
|
||
package tree included in scheduled `forgejo dump` (T09).
|
||
|
||
**Done when:** tier-2 gate is fully satisfied (✓) and tier-3 production images
|
||
follow the same pattern after explicit approval. **Tier-2 gate met; tier-3
|
||
pattern proven by `state-hub`.**
|
||
|
||
---
|
||
|
||
### T08 — Enable Forgejo Actions
|
||
|
||
```task
|
||
id: RAIL-HO-WP-0005-T08
|
||
status: done
|
||
priority: high
|
||
state_hub_task_id: "f45f98c9-2f02-4224-bbfd-c2e1ec38581e"
|
||
```
|
||
|
||
Enable Forgejo Actions with a least-privilege runner model.
|
||
|
||
Minimum scope:
|
||
|
||
- Runner registration automated without committing runner tokens.
|
||
- Runner isolation model documented.
|
||
- Minimal workflows for lint/test/build on representative repositories.
|
||
- Workflow to build and publish a probe container image to Forgejo packages.
|
||
- Secret handling policy for Actions.
|
||
- Resource limits to avoid repeating previous single-node overload patterns.
|
||
|
||
**Partial (2026-07-04):** in-cluster runner live (`railiance-apps/manifests/
|
||
forgejo-runner.yaml`, ADR-004). Proven workflows: `forgejo-actions-probe`
|
||
(image-build), `glas-harness` (host+container CI smoke). Org secrets
|
||
`REGISTRY_USER`/`REGISTRY_TOKEN` set. Documented constraints: host runner is
|
||
non-root (static docker-cli, no `apk add`); `actions/checkout@v4` fails — use
|
||
`git clone` in job. Remaining: reusable workflow templates in
|
||
`railiance-enablement` (S4); resource limits review; no cluster-admin on runner.
|
||
|
||
**Partial (2026-07-04):** tier-2 satisfied by `key-cape` (`container-build`,
|
||
archive checkout, static docker-cli). **Templates (2026-07-04):** S4
|
||
`railiance-enablement/workflows/` (`ci-smoke`, `container-build-push`,
|
||
`container-build-push-multirepo` for `hub-core` contexts) +
|
||
`docs/forgejo-actions-workflow-templates.md`. Tier-3 `state-hub` image build
|
||
and deploy proven 2026-07-06 (`main-f9f0091` on railiance01).
|
||
|
||
**Done when:** tier-2 pilot repo runs Forgejo Actions end-to-end and publishes
|
||
a pullable image without privileged cluster-wide credentials. **Met 2026-07-04.**
|
||
Resource limits review deferred to ongoing ops (non-blocking).
|
||
|
||
---
|
||
|
||
### T09 — Implement Forgejo backup and restore automation
|
||
|
||
```task
|
||
id: RAIL-HO-WP-0005-T09
|
||
status: progress
|
||
priority: high
|
||
state_hub_task_id: "25892007-36ca-4bd9-8adf-84d505465d7d"
|
||
```
|
||
|
||
Create backup automation for all Forgejo state.
|
||
|
||
Must cover:
|
||
|
||
- PostgreSQL database.
|
||
- Git repositories.
|
||
- Attachments.
|
||
- LFS.
|
||
- Packages.
|
||
- Avatars and app data.
|
||
- Actions logs/artifacts if retained.
|
||
- App configuration required for restore.
|
||
|
||
Acceptance:
|
||
|
||
- Scheduled backups run without manual intervention.
|
||
- Backups are encrypted or stored in an approved protected target.
|
||
- Restore into an isolated namespace is drilled and documented.
|
||
- RPO/RTO expectations are recorded.
|
||
|
||
**Partial (2026-07-04):** isolated restore drill **passed**. Production
|
||
`forgejo dump` (~11.7 MiB) restored into `forgejo-restore-drill` namespace;
|
||
post-restore API checks: health 200, `coulomb/glas-harness` and
|
||
`coulomb/key-cape` on `main`, 3 org repos visible. Evidence:
|
||
`docs/forgejo-restore-drill-evidence.md`. Assets: `infra/forgejo-restore-drill/`,
|
||
`tools/forgejo-restore-drill.sh`. **Target decided (T02 Option A).** **Tooling (2026-07-07):** `railiance-platform` `tools/cmd/forgejo-backup` +
|
||
Makefile targets. Fixed websocket EOF on large dumps (use `kubectl cp` after
|
||
in-pod dump). **First scheduled-path backup:** 20260707T180844Z (~686M, 14 repos).
|
||
Daily cron installed (02:15 UTC). Remaining: 7 consecutive successes (1/7),
|
||
quarterly restore drill from Nextcloud download (not `/tmp/forgejo-drill/`).
|
||
|
||
**Done when:** a fresh backup restores to a working isolated Forgejo instance
|
||
with repository, package, and user recovery checks passing **and** scheduled
|
||
backups run without manual intervention.
|
||
|
||
---
|
||
|
||
### T10 — Drill Gitea to Forgejo migration (staged ladder)
|
||
|
||
```task
|
||
id: RAIL-HO-WP-0005-T10
|
||
status: progress
|
||
priority: high
|
||
state_hub_task_id: "6befde73-00bc-4643-be0b-a7ce7944e75f"
|
||
```
|
||
|
||
Run staged migration drills from Gitea to Forgejo before production repos move.
|
||
|
||
**Tier 1 complete (2026-07-04):** `glas-harness` — git history preserved,
|
||
`origin` on Forgejo, `gitea` legacy remote retained, SSH+HTTPS push, CI smoke
|
||
green. Result matrix:
|
||
`the-custodian/docs/forgejo-repo-migration-pilot-glas-harness.md`.
|
||
|
||
Minimum checks (per tier):
|
||
|
||
- Git history and default branches preserved.
|
||
- Issues, labels, milestones, releases, wiki, and attachments handled per
|
||
inventory classification (N/A for tier-1 git-only repos).
|
||
- SSH/HTTPS clone and push paths work (`forgejo-remote` in `~/.ssh/config`).
|
||
- Existing local remotes can be transformed predictably (`origin`/`gitea` split).
|
||
- State Hub registered repo remotes can be updated safely (deferred for tier-1).
|
||
- Rollback plan is rehearsed (Gitea copy unchanged).
|
||
|
||
**Tier 2 complete (2026-07-04):** `key-cape` — multi-stage Dockerfile built and
|
||
pushed via archive-checkout workflow; `crictl pull` on railiance01 succeeded.
|
||
Evidence in `the-custodian/docs/forgejo-repo-migration-pilot-glas-harness.md`
|
||
(tier 2 section).
|
||
|
||
**Tier 2.5 complete (2026-07-04):** railiance stack repos on Forgejo with
|
||
`origin=forgejo-remote`, `gitea` legacy remote, `ci-smoke` workflows; operator
|
||
`tegwick` SSH on Forgejo; enablement templates published. Evidence: pilot doc
|
||
tier 2.5 section.
|
||
|
||
**Tier 3 in progress (2026-07-06):** `state-hub` cutover executed under
|
||
`CUST-WP-0054-T05` — phases 1–4 complete (empty stack, data restore, access
|
||
rewire, sweep checkout migration on railiance01); phase 5 stabilization window
|
||
open through 2026-07-09. Evidence:
|
||
`the-custodian/docs/state-hub-railiance01-cutover-phase{1,2,4,5}-2026-07-06.md`.
|
||
Forgejo also hosts `issue-core`, `activity-core`, `hub-core`, `core-hub`,
|
||
`ops-warden` (14 org repos as of 2026-07-07).
|
||
|
||
**Hub drift (2026-07-08):** tier-2.5 `remote_url` patched for 6 registered repos
|
||
(`patch-forgejo-remote-urls.sh --tier-25`). Hub: 11 Forgejo / 52 Gitea remotes.
|
||
`ops-warden` tier-3 on Forgejo — hub patch deferred pending operator approval.
|
||
|
||
**Remaining:** tier-3 promotion for remaining production repos; scheduled
|
||
backups 7-day gate (1/7); stabilization closeout + operator signoff before
|
||
coulombcore teardown.
|
||
|
||
**Done when:** tiers 0–2 pass with written result matrices and no unknown
|
||
critical migration gaps remain for production repos. **Tiers 0–2.5: met.**
|
||
|
||
---
|
||
|
||
### T11 — Production cutover from Gitea to Forgejo
|
||
|
||
```task
|
||
id: RAIL-HO-WP-0005-T11
|
||
status: todo
|
||
priority: high
|
||
needs_human: true
|
||
state_hub_task_id: "b1b66687-ca33-4971-b312-743c8e059c5e"
|
||
```
|
||
|
||
Execute production migration only after T06, T07, T08, T09, and T10 tier 0–2
|
||
gates pass. `state-hub` and other Wave-1 production repos require explicit
|
||
operator approval per `CUST-WP-0054` drain sequence.
|
||
|
||
**Cutover mode (T02 Option A, 2026-07-09):** staged per-repo — no org-wide freeze.
|
||
|
||
1. Per repo: Gitea backup snapshot before promotion.
|
||
2. Mirror git to Forgejo; switch workstation `origin` to `forgejo-remote`.
|
||
3. Port/verify Actions workflows on Forgejo runner.
|
||
4. Update State Hub `remote_url` and railiance01 sweep checkouts when promoted.
|
||
5. Mark **that** Gitea repo read-only; do not delete.
|
||
6. Stabilization window; repeat until production set complete.
|
||
|
||
**Emergency fallback only:** org-wide freeze-all — not the primary plan; use if
|
||
staged drift becomes unacceptable (document rollback before invoking).
|
||
|
||
**Done when:** all Railiance/Custodian repos use Forgejo as primary, Gitea is
|
||
read-only fallback, and rollback instructions are documented.
|
||
|
||
---
|
||
|
||
### T12 — Retire or archive legacy Gitea
|
||
|
||
```task
|
||
id: RAIL-HO-WP-0005-T12
|
||
status: todo
|
||
priority: medium
|
||
needs_human: true
|
||
state_hub_task_id: "a63147b0-31d5-4705-89ea-40c10faf779f"
|
||
```
|
||
|
||
Retire legacy Gitea only after a stabilization period and explicit approval.
|
||
|
||
Minimum scope:
|
||
|
||
- Confirm no active remotes, webhooks, packages, or dashboards depend on Gitea.
|
||
- Preserve final Gitea backup.
|
||
- Update runbooks and dashboards from Gitea to Forgejo.
|
||
- Remove or archive Gitea Helm release according to the rollback decision.
|
||
- Close stale State Hub references to `railiance-bootstrap` if confirmed as
|
||
an alias rather than a real repo.
|
||
|
||
**Done when:** Forgejo is the only active source forge and package base, with
|
||
legacy Gitea either archived or intentionally retained as documented fallback.
|
||
|
||
## Phasing and Dependencies
|
||
|
||
```
|
||
T01 inventory ──► T02 decisions ──┬──► T04 platform (forgejo-db ✓ partial)
|
||
├──► T05 app (live ✓ partial)
|
||
├──► T06 mail recovery
|
||
├──► T07 packages (OCI probe ✓ partial)
|
||
├──► T08 actions (runner ✓ partial)
|
||
└──► T09 backups
|
||
|
||
T05+T08 ──► T10 migration ladder ──► T11 production cutover ──► T12 Gitea retire
|
||
tier0 probe ✓
|
||
tier1 glas-harness ✓
|
||
tier2 key-cape ✓
|
||
tier2.5 railiance-stack ✓
|
||
tier3 production (gated)
|
||
|
||
T03 isolated probe: CANCELLED (superseded by T05 + in-production pilots)
|
||
```
|
||
|
||
**Current focus (2026-07-08):** T05/T06/T07/T08 **done**; T10 tiers 0–2.5 **complete**
|
||
(tier-2.5 hub `remote_url` patched). Tier-3 `state-hub` stabilization; backup
|
||
7-day gate **1/7**. Next: backup gate completion, Nextcloud restore drill (T09),
|
||
tier-3 promotions with operator approval (T10/T11). `ops-warden` hub patch
|
||
pending approval. Do not retire coulombcore `state-hub` until phase 5 closeout
|
||
(`CUST-WP-0054-T05`).
|
||
|
||
**Absorbed by `CUST-WP-0054-T04`:** forge + CI on railiance01; workstation
|
||
build retirement; staged repo promotion before State Hub primary move (T05).
|
||
|
||
## railiance-bootstrap Note
|
||
|
||
State Hub currently registers both `railiance-bootstrap` and
|
||
`railiance-cluster`, but they point to the same local path
|
||
(`/home/worsch/railiance-cluster`) and the same git fingerprint. The
|
||
`railiance-bootstrap` entry has no remote URL. The earlier restructure workplan
|
||
(`RAIL-HO-WP-0003-T03`) says `railiance-bootstrap` was renamed to
|
||
`railiance-cluster`.
|
||
|
||
Working assumption: `railiance-bootstrap` is a stale logical alias or leftover
|
||
repo goal, not a separate Gitea repository. This workplan should not create a
|
||
new Forgejo repository named `railiance-bootstrap` unless a concrete remaining
|
||
purpose is identified.
|
||
|
||
## References
|
||
|
||
- `RAIL-HO-WP-0004-production-readiness.md`
|
||
- `RAIL-HO-WP-0003-5repo-stack-restructure.md`
|
||
- `CUST-WP-0054-workstation-independence-and-fleet-realignment.md` (T04 forge+CI)
|
||
- `CUST-WP-0014-repo-sync-automation.md`
|
||
- `CUST-WP-0021-multi-host-repo-paths.md`
|
||
- `docs/adr/ADR-004-forgejo-in-cluster-actions-runner.md`
|
||
- `docs/forgejo-migration-inventory.md`
|
||
- `the-custodian/docs/forgejo-production-decisions.md`
|
||
- `the-custodian/docs/forgejo-repo-migration-pilot-glas-harness.md`
|
||
- `railiance-apps/docs/forgejo-on-railiance01.md`
|
||
- `railiance-forge/docs/forgejo-actions-runner-substrate.md`
|
||
- `ops/incidents/2026-03-25-gitea-pgpool-crashloop.md`
|
||
- `ops/incidents/2026-03-26-coulombcore-runaway-agent-overload.md`
|