net-kingdom/canon/standards/credential-management_v0.2.md

Credential Management Standard — net-kingdom

Version: 0.2 Status: current with OpenBao runtime and secret-free handoff refinements Supersedes: v0.1 (retired with NK-WP-0004)

---

1. Purpose

Define how service credentials are generated, stored, rotated, handed off to runtime secret authority, and recovered in the net-kingdom SSO/MFA platform. This standard governs operational security of all secrets used by the Authelia + LLDAP + KeyCape + privacyIDEA stack, its PostgreSQL backend, and the OpenBao runtime secret authority used by the platform control plane.

---

2. Trust Hierarchy

age public recipient ────────►  encrypts secrets.enc/ in git
      │
      └──► ops bundle (encrypted snapshot of all secrets)
                └──► stored offsite; decrypt with age key

break-glass passwords  ────────►  direct service access if cluster/auth is down
      │
      └──► stored in human's personal password manager

K8s Secrets  ──────────────────►  bootstrap/delivery state for running services
      │
      └──► bootstrap/delivery mechanism; created by create-secrets.sh scripts

OpenBao runtime authority ─────►  scoped workload secrets and dynamic credentials
      │
      ├──► leases, revocation, and audit records
      └──► direct client, External Secrets Operator, or CSI delivery

KeePassXC is NOT in the operational path. If you choose to import the emergency bundle into KeePassXC for personal use, that is your business — it is not required or assumed by any tooling in this repo.

The age private key and SOPS/age-encrypted git files are the bootstrap credential store. The ops bundle is the bootstrap backup. The emergency bundle is the human's break-glass key ring. OpenBao is the runtime secret authority once the platform control plane is alive.

---

3. Credential Lifecycle

Phase 0 — Bootstrap (attended, public-recipient encryption)

make creds-agent-init retains its compatibility name but runs in the operator's attended terminal. It requires an independently registered keys/age.pub; it never generates a root key or searches HOME for one. The operator separately establishes custody of the matching private key before bootstrap.

The workflow generates service files with private permissions, encrypts with the public recipient, applies and verifies the scoped cluster setup, then creates a fresh encrypted ops bundle. The recovery handoff displays its location and digest. The operator stores that exact artifact off-host and verifies decryption in an isolated recovery workspace before typing STORED AND VERIFIED. Only then does bootstrap remove plaintext and mark completion. This acknowledgment is a human statement; a separate native drill receipt is required to establish recovery.

For an encrypted resume with no plaintext service directory, invoke bootstrap through sops-custody-unlock.sh from the operator terminal. Only the explicitly supplied temporary NETKINGDOM_CUSTODY_AGE_KEY_FILE is used, and it must match the registered public recipient before any cluster call. There is no ambient key fallback. Use the owner-approved target and change scope; the helper supplies custody material, not authorization.

bash sso-mfa/bootstrap/creds-bootstrap-agent.sh --dry-run emits a non-mutating plan without reading keys or state, contacting the cluster, or creating files. A terminal check prevents accidental transcript-driven execution; it is not a credential-isolation boundary or proof of the caller's identity.

Historical emergency_bundle_delivered fields remain historical. New receipts use recovery_handoff_confirmed, recovery_handoff_bundle_sha256 and a timestamp; an old true value cannot skip acknowledgment of a newly generated bundle.

Phase 1 — Normal operation

No human credential management is needed after bootstrap. All secrets live in:

• secrets.enc/ — encrypted in git (decrypt with age key)
• K8s Secrets — bootstrap or delivery state (updated by creds-apply.sh, External Secrets Operator, CSI, or other approved delivery paths)
• OpenBao — runtime secret authority for platform services, workload secrets, dynamic credentials, leases, revocation, and audit

Once OpenBao is ready, new long-lived workload secret authority should be introduced there rather than by expanding the bootstrap SOPS/age surface.

Phase 2 — Rotation

make creds-rotate SECRET=<name>            # guided interactive mode
SECRET=<name> bash sso-mfa/bootstrap/creds-rotate.sh --non-interactive  # agent mode

Rotation is handled per-secret with appropriate atomicity guarantees. See Section 6 for details.

Phase 3 — Recovery

Use the emergency bundle stored in your personal password manager. See Section 4 — Emergency Recovery Handoff.

---

4. Emergency Recovery Handoff

The custodian age private key and platform-root account/factors stay in the independently held password safe/offline custody packet. OpenBao init/unseal material follows its separately selected custody ceremony; the lab SOPS-held automation branch is outside the new workflow tests and retains its own gate. Service recovery files travel in the age-encrypted ops bundle. The bundle must not contain its own private decryption key. Current workload authority remains in OpenBao.

make creds-emergency-reprint is now a non-secret recovery guide. It reads no private key or service secret, writes no plaintext copy and cannot reconstruct lost custody material. Legacy secret-export arguments are refused. Clearing a terminal or shredding a temporary file cannot retract an existing transcript.

Recovery sequence:

1. Retrieve the separately held key and exact encrypted snapshot from their independent custody locations. Check the non-secret digest/receipt.
2. Use the custody unlock helper in the approved isolated workspace. Decrypt to private files, never stdout; verify required service files without printing their contents. Consult docs/security-bootstrap-age-custody.md.
3. Execute the scoped service owner's recovery procedure with the approved cluster/target. A historical snapshot can contain superseded credentials; validate current authority and consumer acceptance before reuse.
4. Remove temporary keys and decrypted files, and retain only safe outcomes, artifact identities and timestamps. Loss or suspected disclosure requires the separate owner rotation/revocation process; guide output rotates nothing.

The 2026-10-12 source correction removes the old stdout handoff. It does not retroactively revoke any root or password disclosed by earlier runs. Existing founder-deferred exposure rotation remains in CUST-WP-0073-T05. Native acceptance of the corrected workflow remains NK-WP-0049; source tests do not certify it.

---

5. OpenBao Runtime Authority

OpenBao is the canonical runtime secret authority for the platform control plane once deployed and verified. It stores, issues, leases, audits, and revokes runtime secret material; it does not replace identity or authorization.

Required boundaries:

• SOPS/age remains the bootstrap and Git-at-rest protection mechanism.
• OpenBao root tokens, unseal keys, recovery keys, platform mounts, and global auth methods are platform-root material.
• Tenant administrators may manage tenant-scoped secret paths only through approved policies; they must not receive OpenBao platform-root authority.
• Workloads should use scoped OpenBao auth roles, External Secrets Operator, CSI-mounted secrets, or another approved delivery mechanism.
• flex-auth decides whether a secret or dynamic credential request is allowed when the request is authorization-sensitive; OpenBao performs storage, issuance, lease, revocation, and audit.
• OpenBao audit logs must be shipped to durable storage and included in restore and break-glass drills.

OpenBao may issue dynamic credentials for databases, object storage, or other systems where provider support and policy make that safer than static secret distribution. Provider-native STS remains valid where it gives better-scoped temporary credentials; OpenBao can still broker, store, or audit the handoff where appropriate.

---

6. Secret Rotation

Rotatable secrets

Secret	Blast radius	Notes
OpenBao workload secret	Variable	Prefer lease expiry or dynamic regeneration
OpenBao auth role/policy	Variable	Requires policy review and audit check
PI_SECRET_KEY	Low — invalidates PI sessions	Safe to rotate anytime
PI_DB_PASSWORD	Medium — DB + pod	Atomic update required
LLDAP_JWT_SECRET	Low — invalidates LLDAP sessions	Safe to rotate anytime
LLDAP_LDAP_USER_PASS	High — 3-way coordinated	Authelia + KeyCape + LLDAP web UI
AUTHELIA_SESSION_SECRET	Low — invalidates sessions	Safe to rotate anytime
AUTHELIA_KEYCAPE_CLIENT_SECRET	Medium — 2-way atomic	Authelia + KeyCape together
KEYCAPE_RSA_KEY	High — invalidates all JWTs	Causes brief auth outage
BREAKGLASS_PASSWORD	Minimal	Rotate freely

Non-rotatable

Secret	Reason
PI_PEPPER	Rotating requires re-hashing all PI user passwords

Age key rotation

Rotating the age private key is a special case:

1. Generate a new age key
2. Re-encrypt all secrets.enc/ files with the new public key (make sops-rotate)
3. Commit
4. A new emergency bundle must be delivered before the old key is revoked (see Section 4)

OpenBao root and recovery material

OpenBao root tokens, unseal keys, and recovery keys are not routine rotation targets. Treat them as platform-root break-glass material:

1. Prefer short-lived or revoked root tokens after initial setup.
2. Use scoped policies and auth methods for normal operations.
3. Rotate recovery or unseal material only with an explicit maintenance window, backup verification, and post-rotation emergency-bundle update.

---

7. Ops Bundle

The ops bundle is an age-encrypted tar archive of all plaintext secrets at a point in time. It is created automatically during bootstrap and can be refreshed with:

make creds-bundle

Store the bundle offsite (cloud, external drive, second location). Decrypt:

# In the approved isolated recovery workspace, with the custody helper:
age -d -i "$NETKINGDOM_CUSTODY_AGE_KEY_FILE" -o secrets.tar ops-bundle-<date>.tar.age
# Inspect/extract privately under the owner procedure; never print file contents.

---

8. Prohibited Patterns

The following are permanently prohibited:

1. Committing secrets/ (plaintext) to git — the pre-commit hook blocks this; do not bypass with --no-verify

2. Using a personal safe as runtime workload secret authority — runtime workload custody belongs to OpenBao. Independently held bootstrap/root custody and offline recovery remain distinct from runtime workload delivery.

3. Skipping the current encrypted-recovery handoff acknowledgment — a historical emergency_bundle_delivered value cannot certify a new bundle. Completion requires the attended acknowledgment bound to its SHA256; native recovery readiness still requires the separate drill receipt.

4. Hardcoding secrets in manifests or ConfigMaps — all service secrets must flow through K8s Secrets created by create-secrets.sh scripts

5. Storing or printing the root key through automation — the private key belongs in independent custody. Only the explicit attended unlock supplies a temporary key; no permanent HOME key discovery or generation in bootstrap.

6. Using OpenBao as a policy decision point — OpenBao stores, leases, audits, and revokes secret material; identity comes from the IAM Profile and authorization decisions come from flex-auth where a decision boundary is needed

7. Giving tenants OpenBao platform-root authority — tenants may only receive scoped access paths approved for their tenant resources

---

Appendix A — KeePassXC Group Structure (optional)

If you choose to import the emergency bundle into KeePassXC, a suggested group structure for the break-glass passwords is:

net-kingdom/
  Break-glass/
    privacyIDEA admin    username=pi-admin        password=<PI_ADMIN_PASSWORD>
    LLDAP admin          username=admin           password=<LLDAP_LDAP_USER_PASS>
    PostgreSQL root      username=postgres        password=<PG_ROOT_PASSWORD>
    break-glass user     username=break-glass     password=<BREAKGLASS_PASSWORD>
  age key               (attach keys.txt as binary attachment)
  ops bundle decrypt    (store path + decrypt command as a note)

This is entirely optional — the agent does not read from or write to KeePassXC.