railiance-platform/workplans/RPF-WP-0044-argocd-phase-b-adopt-existing-applications.md
codex 4ca88a14f0
Some checks failed
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Has been cancelled
RPF-WP-0044: T01 done; exact T02/T03 commands pinned to c3ebd6d.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 63291@bnt-lap001
Assistant-Session: 8bd77868-ca68-4f49-bb1e-d539ecc0d703
2026-09-21 19:05:56 +02:00

16 KiB
Raw Blame History

id type title domain repo status flavor owner topic_slug created updated related state_hub_workstream_id
RPF-WP-0044 workplan ArgoCD phase B: adopt the four existing Applications on railiance01 financials railiance-platform active planning railiance-platform railiance 2026-09-21 2026-09-21
RPF-WP-0043
RPF-WP-0022
98140775-3b9a-5cf9-9af6-722502d487dc

The founder, Bernd Worsch, decided ArchitectureBlueprint §5.6 on 2026-09-21, exercising GOVERN @ estate: option 1, adopt properly. Phase A installed Argo CD Core v3.5.3 on railiance01 (record: the-custodian/docs/kubernetes-change-gate-decision.md). This plan is phase B: bring the four Applications in argocd/applications/ under that ArgoCD, one at a time, each with its own founder go-ahead. Terms follow SecurityCanon Mode of Authority v0.2.0 (draft).

This plan changes nothing live. Every task that would change railiance01 is wait on the founder. Each live step is ADMINISTER @ realm:kubernetes/railiance01 (bootstrap) or CONSTRUCT @ railiance-platform with the merge as the gate, both activation=APPROVED.

Related, kept separate: RPF-WP-0043 onboards rapp-policy-nexus, which is not one of these four. It needs T02 of this plan (a working root app on railiance01) before its own T04. Phase C, retiring coulombcore's ArgoCD, is not planned here (T08).

Read-only evidence, 2026-09-21

Full record: docs/evidence/argocd-phase-b-adoption-diff-2026-09-21.json. Rendered locally (helm template, kubectl kustomize), then kubectl diff client-side and --server-side dry run over ssh railiance01. Nothing was applied.

App Live spec diff Would prune Repo credential Blocker
openbao-secretstore none (1 ClusterSecretStore) nothing none, public none
issue-core none after repointing nothing new: rapp-issue-core is private declared path issue-core.git k8s/railiance no longer exists
target-revenue none on Deployment/Service/Ingress/ExternalSecret/Cluster nothing none, public CNPG Cluster not in railiance-tenants whitelist; two Sync-hook Jobs
external-secrets none (39 objects, 20 CRDs) nothing now none, public Helm repo cluster-wide blast radius; CRDs must never be pruned

Nothing is pruned at adoption because no live object carries an ArgoCD tracking annotation; ArgoCD only prunes what it tracks. The one change every adoption makes is that tracking annotation, on metadata only, so no rollout.

Sync settings found: railiance-apps-root and all four children declare automated: {prune: true, selfHeal: true}; none has a finalizer. Applying argocd/bootstrap/ as committed would therefore adopt all four at once. T01 removes that before anything is applied.

Also found: the external-secrets Helm release is stuck at revision 7, pending-upgrade, since 2026-07-07 (values identical to the Application); issue-core is now server-side applied by rapp-issue-core; target-revenue runs a mutable image tag (0.1.4). ArgoCD on railiance01 has 0 AppProjects, 0 Applications, 0 repository Secrets, and there is no argocd CLI on the node, so the commands below use kubectl against the Application resource. Not verified: repo-server egress to Forgejo and charts.external-secrets.io; T02 proves it.

Neutralise automated sync and fix the declared state

id: RPF-WP-0044-T01
status: done
priority: high
state_hub_task_id: "2d7230f5-50c8-5eee-a050-1501ce5af62a"

Done 2026-09-21, commit c3ebd6d. The founder chose Option A (GOVERN @ estate): a railiance01-only source path. Repository-only; nothing applied. argocd/applications/ and argocd/bootstrap/ are untouched, because coulombcore's ArgoCD still reads argocd/applications/ on main and its bootstrap may be re-applied from argocd/bootstrap/. coulombcore could not be read in this session, so the railiance01 bootstrap has its own path.

Declared (all under argocd/railiance01/):

  1. bootstrap/: the three AppProjects and railiance-apps-root with no automated block, path argocd/railiance01/applications, targetRevision: main (each sync pins its revision in the operation).
  2. bootstrap/01-railiance-tenants-project.yaml: postgresql.cnpg.io/Cluster added to namespaceResourceWhitelist. The other two AppProjects are copies of argocd/bootstrap/.
  3. applications/ holds only README.md. A placeholder is needed: git does not keep an empty directory and a missing path is a ComparisonError. ArgoCD's directory source reads only *.yaml/*.yml/*.json, so it renders zero children (confirmed live by T02).
  4. drafts/: the four children, no automated, no finalizer, pinned: openbao-secretstore d2dbc19 (railiance-platform), issue-core a78f38c (rapp-issue-core.git, path manifests), target-revenue f1109d5, external-secrets chart 0.16.1. No root syncs drafts/.
  5. argocd/repositories/rapp-issue-core.repository.sops.yaml.template (no value); issue-core.repository.sops.yaml.template marked obsolete.

Offline proof: kubectl kustomize argocd/railiance01/bootstrap renders exactly 3 AppProjects and 1 Application with no automated field; ssh railiance01 'kubectl apply --dry-run=server -f -' of that render returned rc=0 (all four "created (server dry run)").

Hazard kept as is: make argocd-bootstrap-deploy still defaults to argocd/bootstrap (automated root). Do not run it on railiance01.

Apply the AppProjects and the root app, automated sync off (live)

id: RPF-WP-0044-T02
status: wait
priority: high
state_hub_task_id: "7d97ee9c-b2fb-520f-8468-3ecf7eea6c01"

Waits on the founder's go-ahead. T01 is done. Applies the render of argocd/railiance01/bootstrap at commit c3ebd6dddc830373c75144da178f829b47c3e5de, from a clean export (not the working tree), then syncs the root by hand at that commit.

cd ~/railiance-platform && git fetch origin && git merge-base --is-ancestor c3ebd6dddc830373c75144da178f829b47c3e5de origin/main && echo pinned-commit-on-main
R=$(mktemp -d) && git -C ~/railiance-platform archive c3ebd6dddc830373c75144da178f829b47c3e5de argocd/railiance01/bootstrap | tar -x -C "$R"
kubectl kustomize "$R/argocd/railiance01/bootstrap" > "$R/bootstrap.yaml" && grep -c '^kind: AppProject' "$R/bootstrap.yaml" && grep -c automated "$R/bootstrap.yaml"   # expect 3 and 0
ssh railiance01 'kubectl apply --dry-run=server -f -' < "$R/bootstrap.yaml"
ssh railiance01 'kubectl apply -f -' < "$R/bootstrap.yaml"
ssh railiance01 'kubectl -n argocd patch application railiance-apps-root --type merge -p "{\"operation\":{\"initiatedBy\":{\"username\":\"founder-approved\"},\"sync\":{\"revision\":\"c3ebd6dddc830373c75144da178f829b47c3e5de\",\"prune\":false}}}"'
ssh railiance01 'kubectl -n argocd get appprojects,applications'
ssh railiance01 'kubectl -n argocd get application railiance-apps-root -o jsonpath="{.status.sync.status} {.status.operationState.phase} {.status.operationState.syncResult.revision} resources={.status.resources}{\"\n\"}"'

Expect three AppProjects and railiance-apps-root Synced, operation Succeeded at c3ebd6d, and no resources (zero children). A ComparisonError means the repo-server cannot reach Forgejo: stop and fix egress before any app task. No repository Secret is needed; coulomb/railiance-platform is public.

Rollback: kubectl -n argocd delete application railiance-apps-root (no finalizer, no children), then delete the three AppProjects. Nothing else is touched.

Per-app procedure (T03T06)

Every adoption task follows the same steps. <app> and <sha> vary.

  1. Diff. Re-run the evidence render and kubectl diff --server-side at the commit or chart version being pinned. Any spec change, any missing whitelist kind, or a different image: stop.
  2. Merge the child Application (no automated) from argocd/railiance01/drafts/ into argocd/railiance01/applications/, then sync the root by hand so it creates the child object only:
    ssh railiance01 'kubectl -n argocd patch application railiance-apps-root --type merge -p "{\"operation\":{\"initiatedBy\":{\"username\":\"founder-approved\"},\"sync\":{\"revision\":\"<sha>\",\"prune\":false}}}"'
    
  3. ArgoCD's own diff: read kubectl -n argocd get application <app> -o jsonpath='{.status.sync.status} {.status.health.status}' and .status.resources. Only metadata differences are acceptable.
  4. Manual sync, prune off, hooks skipped (apply strategy):
    ssh railiance01 'kubectl -n argocd patch application <app> --type merge -p "{\"operation\":{\"initiatedBy\":{\"username\":\"founder-approved\"},\"sync\":{\"revision\":\"<sha>\",\"prune\":false,\"syncStrategy\":{\"apply\":{}}}}}"'
    
    Then verify Synced/Healthy and run the owner's live check.
  5. Prove it: Synced and Healthy for at least 24 hours (7 days for external-secrets). Only then, with a second go-ahead, a merge adds automated: {selfHeal: true}; prune: true is a third step once the app's tracked set is confirmed complete.
  6. Record docs/evidence/<date>-<app>-argocd-adoption.json.

Rollback:

  • R1, before automated sync: revert the merge, then kubectl -n argocd delete application <app>. No finalizer, so nothing cascades; the workload keeps running. Tracking annotations stay and are harmless.
  • R2, after automated sync: revert the automated-sync commit first and sync the root by hand, then R1. Never hand-patch a child while the root self-heals; that is BREAK_GLASS, recorded and reconciled into git.
  • Workload restore: only once ArgoCD no longer manages the app, with the previous deploy method named in the task.

Adopt openbao-secretstore (lowest risk)

id: RPF-WP-0044-T03
status: wait
priority: high
state_hub_task_id: "6d5fc801-e361-579b-bcd8-6fe719a82e94"

Waits on the founder's go-ahead. Depends on T02. One object, ClusterSecretStore/openbao, zero diff, public source. It is the store issue-core's ExternalSecret reads, so adopt it before issue-core. The 24 other ClusterSecretStores on railiance01 are not in the kustomization and stay untracked (declared gap, RPF-WP-0043-T05). Live check: kubectl get clustersecretstore openbao stays Valid, and externalsecret/issue-core-runtime stays SecretSynced. Workload restore: kubectl apply -f argocd/platform-addons/openbao-secretstore/openbao.clustersecretstore.yaml.

Exact commands (the root sync pins the merge commit $SHA; the child is pinned to d2dbc19, the last commit touching its source path):

# 1. Diff at the pinned source commit (expect no output, rc=0)
cd ~/railiance-platform && git fetch origin && git checkout main && git pull --ff-only
R=$(mktemp -d) && git archive d2dbc19c254247652c49fda8721c80d53bca206a argocd/platform-addons/openbao-secretstore | tar -x -C "$R"
ssh railiance01 'kubectl diff --server-side -f -' < <(kubectl kustomize "$R/argocd/platform-addons/openbao-secretstore"); echo rc=$?
# 2. Merge the child (drop the 3-line DRAFT header), push, sync the root at that commit
sed '1,3d' argocd/railiance01/drafts/openbao-secretstore.application.yaml > argocd/railiance01/applications/openbao-secretstore.application.yaml
git rm -q argocd/railiance01/drafts/openbao-secretstore.application.yaml
git add argocd/railiance01/applications/openbao-secretstore.application.yaml
git commit -m "Adopt openbao-secretstore on railiance01 (RPF-WP-0044-T03)" && git push origin main
SHA=$(git rev-parse HEAD) && echo "$SHA"
ssh railiance01 "kubectl -n argocd patch application railiance-apps-root --type merge -p '{\"operation\":{\"initiatedBy\":{\"username\":\"founder-approved\"},\"sync\":{\"revision\":\"$SHA\",\"prune\":false}}}'"
# 3. ArgoCD's own diff (expect OutOfSync or Synced, one resource ClusterSecretStore/openbao, metadata-only)
ssh railiance01 'kubectl -n argocd get application openbao-secretstore -o jsonpath="{.status.sync.status} {.status.health.status} {.status.resources}{\"\n\"}"'
# 4. Manual sync, prune off, apply strategy (hooks skipped)
ssh railiance01 'kubectl -n argocd patch application openbao-secretstore --type merge -p "{\"operation\":{\"initiatedBy\":{\"username\":\"founder-approved\"},\"sync\":{\"revision\":\"d2dbc19c254247652c49fda8721c80d53bca206a\",\"prune\":false,\"syncStrategy\":{\"apply\":{}}}}}"'
ssh railiance01 'kubectl -n argocd get application openbao-secretstore -o jsonpath="{.status.sync.status} {.status.health.status} {.status.operationState.phase}{\"\n\"}"'
ssh railiance01 'kubectl get clustersecretstore openbao; kubectl -n issue-core get externalsecret issue-core-runtime'

Rollback (R1): git revert the merge commit and push, then ssh railiance01 'kubectl -n argocd delete application openbao-secretstore' (no finalizer; the ClusterSecretStore keeps running).

Adopt issue-core (from rapp-issue-core)

id: RPF-WP-0044-T04
status: wait
priority: high
state_hub_task_id: "0e619658-0453-51ed-b9a0-bed28b5544fd"

Waits on the founder's go-ahead, and on two preconditions:

  • rapp-issue-core agrees (hub message) that ArgoCD owns production and its make deploy stops being a production path; it would fight self-heal. Image releases become commits to manifests/30-runtime.yaml.
  • A CCR for the ArgoCD repository credential, source at platform/operators/argocd/repositories/rapp-issue-core, read-only deploy token. Applied with make argocd-repo-apply; no value is handled in this plan.

Zero diff against live for both the raw manifests/ and the rendered output (digest sha256:a56c80cc…b92cf). All kinds are in the railiance-tenants whitelist. Live check: rapp-issue-core make verify-live. Workload restore: rapp-issue-core make rollback.

Adopt target-revenue

id: RPF-WP-0044-T05
status: wait
priority: medium
state_hub_task_id: "d418068a-6fb0-5416-aac5-d23c93924d9c"

Waits on the founder's go-ahead. Preconditions: T01 item 2 (CNPG Cluster whitelisted) applied to the AppProject by hand, and target-revenue confirms the two Sync-hook Jobs (target-revenue-migrate, target-revenue-bootstrap-binky) are safe to re-run against production. They are absent live and would run on every sync with the default hook strategy. The adoption sync uses the apply strategy, which skips them. Once automated sync is enabled they run on each sync, so the second go-ahead depends on that confirmation. Ask target-revenue to pin an image digest instead of 0.1.4. Live check: https://revenue.coulomb.social health and the CNPG cluster healthy. Workload restore: kubectl apply -k k8s/railiance from the previous target-revenue commit.

Adopt external-secrets (highest risk)

id: RPF-WP-0044-T06
status: wait
priority: medium
state_hub_task_id: "4ca54868-c31f-5e3d-b7d3-a15b84c15f80"

Waits on the founder's go-ahead. Zero diff, but ESO serves every ExternalSecret on the cluster (25 ClusterSecretStores). Before the merge, add crds.annotations: {argocd.argoproj.io/sync-options: "Prune=false,Delete=false"} to the values and re-run the diff; pruning a CRD would delete every ExternalSecret and store. Adopt with ServerSideApply=true as declared. Watch for drift on caBundle fields injected by the cert-controller; if ArgoCD shows it, add ignoreDifferences before enabling self-heal. After adoption, retire the stale pending-upgrade Helm release record (its sh.helm.release.v1.external-secrets.v* Secrets) under a separate go-ahead; ArgoCD does not track them. Live check: all ClusterSecretStores keep their current Valid state (two, openbao-activity-core and openbao-email-connect, are already ValidationFailed). Workload restore: helm template 0.16.1 with the same values, kubectl apply --server-side.

Turn the root back to the production lane

id: RPF-WP-0044-T07
status: wait
priority: medium
state_hub_task_id: "844ff115-def2-55ca-ab01-413ba7e6857d"

Waits on the founder's go-ahead. After T03T06 are proven, restore automated: {prune: true, selfHeal: true} on railiance-apps-root, as the change-gate decision's production row expects. From then on a merge to the railiance01 path is a live production change, and RPF-WP-0043-T04 (policy-nexus) can use it.

Plan phase C: retire coulombcore's ArgoCD

id: RPF-WP-0044-T08
status: todo
priority: low
state_hub_task_id: "55d1382f-5862-5321-a1c9-96767764ba43"

Planning only. Needs a read-only check of coulombcore's ArgoCD, outside this session's scope. Under Option A, retiring it also removes argocd/applications/. Also hand back to the cluster layer: the phase A install is not declared in any repository and its pods have no resource requests (BestEffort).