coulomb/net-kingdom
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
net-kingdom/docs/railiance01-coulombcore-cutover-inventory-2026-07-28.md
tegwick f0bbd0bfce
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
Details
CI Smoke / container-smoke (push) Successful in 1s
Details
Complete CoulombCore identity backup gate
2026-07-28 09:26:16 +02:00

6.6 KiB
Raw Blame History

Railiance01 / CoulombCore cutover inventory

Captured read-only on 2026-07-28. This inventory intentionally contains no Secret values, private keys, password hashes, tokens, or database contents.

Public routing

Name Current target Cutover owner/state
kc.coulomb.social railiance01 (92.205.62.239) KeyCape active
auth.coulomb.social railiance01 (92.205.62.239) Authelia active
login.coulomb.social railiance01 (92.205.62.239) Authelia alias active
lldap.coulomb.social CoulombCore (92.205.130.254) identity cutover pending
pink.coulomb.social CoulombCore (92.205.130.254) identity cutover pending
pink-account.coulomb.social CoulombCore (92.205.130.254) identity cutover pending
bao.coulomb.social CoulombCore (92.205.130.254) OpenBao-owned cutover
gitea.coulomb.social CoulombCore (92.205.130.254) Forgejo/Gitea-owned cutover
hub.coulomb.social CoulombCore (92.205.130.254) State Hub-owned cutover

Public recursive resolvers return railiance01 for both Authelia names. The certificate at railiance01 is trusted and covers both names. The portal → KeyCape → Authelia redirect chain terminates on railiance01.

During cutover, cert-manager's HTTP-01 self-check inherited stale node DNS. The live controller now uses 1.1.1.1:53,8.8.8.8:53 for ACME self-checks and the expanded certificate was renewed through cmctl. The cluster add-on configuration must codify that resolver flag so a future Helm reconciliation does not remove it.

Identity workload comparison

Component CoulombCore railiance01 Finding
KeyCape main-nonce-0601 key-cape:909bb32 New runtime is ahead; both healthy
Authelia authelia:4.38 authelia:4.38 Same declared image
LLDAP lldap/lldap:stable lldap/lldap:stable Independent PVCs; content differs
privacyIDEA 3.12.2 3.12.2 Independent data/log PVCs
NetKingdom PostgreSQL 10 Gi PVC 10 Gi PVC Independent clusters require semantic comparison
Identity provisioner absent identity-provisioner:dbf7cfd railiance01-only lifecycle adapter
User engine absent user-engine:portal-e23674d railiance01-only control surface

Both KeyCape endpoints serve the same public JWKS fingerprint: 8e3237da6030c6af91c5005d0112a149204bc549f67d2234939c5581fdb95a32. This proves current public signing-key continuity, not full configuration equivalence.

Persistent state and backup observations

  • Both LLDAP databases are 139264 bytes, but their SHA-256 fingerprints differ: CoulombCore b3be5eed...b91278; railiance01 fc538cc8...03dab. Equal size is not equivalence. Railiance01 also has newer writes from the provisioning conformance path.
  • railiance01 runs daily LLDAP, Authelia, and privacyIDEA backup CronJobs and exposes an LLDAP backup directory on its PVC.
  • The queried CoulombCore workload inventory exposed daily logical PostgreSQL backup CronJobs, but no matching LLDAP, Authelia, or privacyIDEA backup CronJobs. A fresh source-side protected backup is therefore a hard gate.
  • Both servers retain independent LLDAP, Authelia, privacyIDEA, and NetKingdom PostgreSQL PVCs. No DNS move for LLDAP/privacyIDEA is evidence of data migration.
  • CoulombCore still runs KeyCape and Authelia ingress even though their public DNS moved. Keep these intact only for the bounded rollback window; prevent new authoritative writes once state migration begins.

Encrypted source backup

A fresh source backup set was created on CoulombCore at ~/identity-cutover-backups/2026-07-28T0100Z, encrypted directly to the existing SOPS/age custody recipient. The directory is mode 0700; encrypted artifacts and SHA256SUMS are mode 0600.

The set contains:

  • hot SQLite backups for LLDAP and Authelia;
  • a complete pg_dumpall stream for net-kingdom-pg;
  • privacyIDEA encryption/signing custody files;
  • encrypted Kubernetes identity Secrets and ConfigMaps;
  • encrypted workload/recovery manifests.

All encrypted checksums pass. Every artifact decrypts with the custody key; both restored SQLite databases return pragma integrity_check = ok. Plaintext verification used /dev/shm or streams and was removed, and the temporary export pod was deleted.

The encrypted set was copied to independent custody on railiance01 at ~/identity-cutover-backups/coulombcore/2026-07-28T0100Z. The destination directory is mode 0700, every file is mode 0600, and all source hashes match. The original checksum manifest contains absolute CoulombCore paths, so the independent-copy check preserved the manifest and resolved each entry by basename.

An isolated restore drill started a temporary PostgreSQL 16 pod with no service, ingress, or persistent volume. The decrypted dump recreated interhub and privacyidea_db; the restored privacyIDEA database contained the expected one token, one token owner, two realms, two resolvers, and two policies. The pod and transient restore log were deleted automatically.

Semantic comparison

State CoulombCore railiance01 Result
LLDAP users 4 3 divergent
LLDAP groups 7 7 stable-identifier digest divergent
LLDAP memberships 7 3 divergent
privacyIDEA tokens 1 0 critical migration required
privacyIDEA token owners 1 0 critical migration required
privacyIDEA realms 2 0 configuration migration required
privacyIDEA resolvers 2 0 configuration migration required
privacyIDEA policies 2 0 configuration migration required

Authelia schemas match. Session, consent, access-token, authorization-code, and audit counts differ because both instances have been independently active; those ephemeral sessions should not be merged as authoritative identity data.

Every compared privacyIDEA custody fingerprint (enckey, signing keypair, and instance UUID) differs. Migrating only database rows would make the enrolled token unusable. The CoulombCore privacyIDEA database plus its encryption and signing custody must move as one rollback-tested unit.

Required next evidence

  1. Copy the encrypted backup set to independent custody and perform an isolated PostgreSQL/privacyIDEA restore drill.
  2. Decide the LLDAP merge rule for the one source-only user and four source-only memberships while preserving railiance01-only lifecycle groups.
  3. Freeze writes, migrate privacyIDEA database and custody as one unit, and migrate/merge authoritative LLDAP state.
  4. Rerun semantic counts/digests and restart recovery probes.
  5. Exercise platform-root and Binky login/MFA.
  6. Move LLDAP and privacyIDEA DNS only after those gates pass.