railiance-platform/docs/argocd-gitops.md

# ArgoCD GitOps Contract

Railiance ArgoCD is the cluster sync engine. This repo owns only the shared
platform contract around GitOps trust, guardrails, and OpenBao-backed secret
delivery. Application workload manifests remain in the owning application
repos.

## Ownership Boundary

`railiance-platform` owns:

- ArgoCD AppProject guardrails for Railiance tenant workloads.
- The root app-of-apps entrypoint.
- Repository credential registration templates.
- Secret delivery conventions through OpenBao and External Secrets Operator.

Tenant repos own:

- Container images.
- Workload manifests under `k8s/railiance/`.
- The proposed ArgoCD `Application` manifest for platform review.
- Application config that contains no secret values.

Cluster/runtime ownership remains outside this repo: installing or upgrading
ArgoCD itself belongs with the cluster layer.

## Production Change Path (founder decision, 2026-09-21)

Source: `the-custodian/docs/kubernetes-change-gate-decision.md`. Bernd Worsch,
the founder, decided it exercising `GOVERN @ estate`. Terms follow SecurityCanon
Mode of Authority v0.2.0, which is still a draft.

The `railiance-apps-root` lane in this repository is the change path for
workloads whose ADR-0006 `readiness_state` is `production-approved`. The gate is
a quality gate, not an authorization decision. No PDP sits in this path.

| Target readiness state | Path for a change | `Activation` |
| --- | --- | --- |
| `declared`, `installed`, `verified` | Direct `ADMINISTER @ realm:kubernetes` (ops-mason) | `APPROVED`: founder approval of the construction plan |
| `production-approved` | `CONSTRUCT @` the tenant manifest repository, plus a reviewed Application here; ArgoCD reconciles | `APPROVED` for the change; the merge is the gate |
| `production-approved`, emergency | Direct `ADMINISTER` | `BREAK_GLASS`, recorded, then reconciled back into the repository |

Platform objects that carry no readiness state default to the production tier.
A change through git leaves `external-audited` evidence. A direct apply leaves
only `target-audited` evidence. That difference is why production changes go
through git.

What this means for platform review in this repository:

- A merge to `argocd/applications/` is a live production change. Once the
  commit is on `main`, `railiance-apps-root` syncs it with prune and self-heal.
  Treat the merge as the `APPROVED` gate, not as a draft.
- A new Application for a workload that is already running is an **adoption**.
  Follow the adoption steps in `workplans/RPF-WP-0043-policy-nexus-argocd-onboarding.md`:
  manual sync first, a diff, no resources finalizer, and the founder's go-ahead.
- Many platform objects are still changed by direct Make targets:
  ClusterSecretStores on railiance01, OpenBao configuration and the bootstrap
  itself. Under this decision they default to the production tier. That is a
  declared gap (RPF-WP-0043-T05), not conformance.

**Transition.** `rapp-policy-nexus` is `production-approved` but not yet an
Application here. Until 2026-12-21 its changes keep `activation=APPROVED` by
founder plan approval, and each one is recorded as production-tier.
Onboarding is due by 2026-12-21 (RPF-WP-0043).

**Not verified: that ArgoCD syncs on railiance01.** This repository does not
show it, and some records point the other way:

- The only record of ArgoCD's health is the RPF-WP-0022 inbox evidence from
  2026-06: "installed and healthy on `railiance01`", with zero Applications.
  That is from before the 2026-07-02 naming correction. Until then, "railiance01
  cluster" often meant coulombcore.
- The later records in this file and in
  `argocd/platform-addons/openbao-secretstore/kustomization.yaml` name the
  ArgoCD that runs these Applications "coulombcore ArgoCD". They say the
  railiance01 stores are "not managed by coulombcore ArgoCD".
- Every Application uses `destination.server: https://kubernetes.default.svc`,
  so it reconciles whichever cluster ArgoCD itself runs in.
- No evidence file in this repository records a sync of `railiance-apps-root`
  on railiance01 (92.205.62.239).

Confirming it takes an attended **read-only** check against railiance01, not
coulombcore: check the node IP, the `argocd` namespace, and the `railiance-apps-root`
sync and health status, for example `make argocd-status` with a kubeconfig whose
server is verified to be 92.205.62.239. Record the result as evidence. If ArgoCD
is not running there, the production row has no working path on railiance01
yet. Installing ArgoCD belongs to the cluster layer and needs its own go-ahead
from the founder. This is RPF-WP-0043-T01.

## Bootstrap Layout

```text
argocd/bootstrap/
  00-railiance-bootstrap-project.yaml
  01-railiance-tenants-project.yaml
  10-railiance-apps-root.application.yaml

argocd/applications/
  *.application.yaml

argocd/repositories/
  *.repository.sops.yaml
```

The bootstrap is applied once by an operator. If the Git source is private,
apply the encrypted `railiance-platform` repository Secret first so the root
Application can sync this repo:

```bash
ARGOCD_REPOSITORY_SECRET=argocd/repositories/railiance-platform.repository.sops.yaml \
  make argocd-repo-apply
make argocd-bootstrap-dry-run
make argocd-bootstrap-deploy
make argocd-status
```

After that, `railiance-apps-root` syncs tenant Application manifests from
`argocd/applications/`.

## Repository Registration

Every Git source repo used by ArgoCD must be registered in the `argocd`
namespace with an ArgoCD repository Secret. Use one read-only deploy token or
deploy key per repo unless an operator approves a narrower shared credential
model.

Repository credentials are operator credentials, not workload secrets. Store
their source material in OpenBao under:

```text
platform/operators/argocd/repositories/<repo-name>
```

Create an encrypted repository Secret from the matching template:

```bash
cp argocd/repositories/issue-core.repository.sops.yaml.template \
  argocd/repositories/issue-core.repository.sops.yaml
sops -e -i argocd/repositories/issue-core.repository.sops.yaml
```

Apply only encrypted files:

```bash
ARGOCD_REPOSITORY_SECRET=argocd/repositories/issue-core.repository.sops.yaml \
  make argocd-repo-apply
```

Do not commit plaintext deploy tokens, passwords, SSH private keys, OpenBao
tokens, or ArgoCD API tokens.

## Tenant Application Contract

Tenant Applications are thin routing manifests reviewed into
`argocd/applications/`. The workload source remains in the tenant repo,
normally:

```text
k8s/railiance/
```

Default Application shape:

```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: example-service
  namespace: argocd
spec:
  project: railiance-tenants
  source:
    repoURL: https://forgejo.coulomb.social/coulomb/example-service.git
    targetRevision: main
    path: k8s/railiance
  destination:
    server: https://kubernetes.default.svc
    namespace: example-service
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
      - ApplyOutOfSyncOnly=true
      - PruneLast=true
```

Use sync waves only for real dependency ordering. Secret delivery resources
should sync before Deployments that consume them.

## Secret Delivery

OpenBao is the canonical runtime secret custody authority.

Default pattern:

1. Store workload secret material in OpenBao.
2. Use External Secrets Operator to materialize a Kubernetes Secret in the
   workload namespace.
3. Reference that Kubernetes Secret from the Deployment, Job, or CronJob.

Path convention for workload credential custody:

```text
platform/workloads/<tenant-or-org>/<workload>/<secret-purpose>
```

Kubernetes namespace and service-account bounds belong in the OpenBao auth role
or External Secrets binding, not in the tenant segment unless the namespace is
itself the approved workload identity.

Use CSI-mounted files only for workloads that need file references, sharper
mount boundaries, or refresh behavior that should not rewrite application
manifests. Do not use the OpenBao injector in the current deployment.

For `issue-core`, the expected custody shape is:

```text
platform/workloads/issue-core/issue-core/issue-core-runtime
```

with properties such as:

```text
ISSUE_CORE_API_KEY
GITEA_BACKEND_TOKEN
```

The exact ExternalSecret manifest belongs with `issue-core` workload
manifests, because it is part of that service's runtime deployment.

## AppProject Guardrails

`railiance-bootstrap` allows the root app to manage ArgoCD `Application`
objects in the `argocd` namespace.

`railiance-tenants` allows ordinary namespaced workload resources and namespace
creation. It does not allow tenant Applications to create CRDs, ClusterRoles,
ClusterRoleBindings, or other cluster-admin resources.

If a tenant needs a cluster-scoped platform resource, create a new
platform-owned workplan instead of broadening the tenant project by default.

## Platform Add-ons

External Secrets Operator is a platform-owned add-on because it installs CRDs,
webhooks, and cluster RBAC. Tenant Applications must not install or upgrade it.

The GitOps contract uses:

- `railiance-platform-addons` AppProject for cluster add-ons.
- `external-secrets` ArgoCD Application for the public Helm chart.
- `openbao-secretstore` ArgoCD Application for the in-cluster OpenBao
  `ClusterSecretStore` on coulombcore (`openbao` only).
- OpenBao Kubernetes auth role `external-secrets-issue-core` for the
  issue-core pilot.

`ClusterSecretStore/openbao` is limited to the `issue-core` namespace on
coulombcore.

**Railiance01 stores** use the private railiance01 OpenBao service and are
**not** managed by coulombcore ArgoCD. Apply per workload:

| Store | Bootstrap |
| --- | --- |
| `openbao-forgejo` | Dedicated Kubernetes auth; `docs/eso-auth-recovery.md` |
| `openbao-activity-core` | `activity-core` `make openbao-eso-token-apply` + `kubectl apply -f …/openbao-activity-core.clustersecretstore.yaml` |
| `openbao-reuse` | Dedicated Kubernetes auth; `docs/eso-auth-recovery.md` |
| `openbao-target-revenue` | Dedicated Kubernetes auth; `docs/eso-auth-recovery.md` |

The three recovered stores no longer use the old `*-eso-token-apply` bootstrap
targets. Do not recreate static delivery tokens for them.

Manifests live under `argocd/platform-addons/openbao-secretstore/` but are
excluded from the coulombcore kustomization. Broaden or add stores only with
platform review.

Configure the OpenBao side without printing token values:

```bash
OPENBAO_TOKEN_FILE=~/.local/openbao/platform-admin.token \
  make openbao-configure-external-secrets-issue-core

OPENBAO_TOKEN_FILE=~/.local/openbao/platform-admin.token \
  make openbao-configure-external-secrets-activity-core
```

The helper keeps Kubernetes auth in local-reviewer mode: OpenBao rereads its
own mounted service-account token and CA file instead of storing an expiring
reviewer JWT.

Then sync ArgoCD and verify:

```bash
make argocd-bootstrap-deploy
make argocd-status
kubectl -n external-secrets get deploy,pod
kubectl get clustersecretstore.external-secrets.io openbao
```