net-kingdom/docs/identity-service-restore-packet.md
tegwick 1764f40b43 Prepare independent outage recovery and Storage Share documentation
Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a13eab-1e6a-7b33-9c37-372e90ada3a2
2026-10-16 17:59:18 +02:00

6.8 KiB
Raw Blame History

Isolated identity service restore packet

NK-WP-0049-T15. Prepared 2026-10-16; draft, not admitted or executed. Accountable preparer: codex. Execution/acceptance: Railiance Platform and the LLDAP, Authelia, KeyCape and privacyIDEA service owners.

Fixed starting point

Use the actual recovered artifact and receipt in docs/evidence/2026-10-16-identity-backup-live/actual-offsite.json, with matching original keys and manifest. The completed archive recovery verified 36 file hashes, SQLite integrity, RSA pairs and native PostgreSQL restore. It did not prove service authentication or audit-signature continuity. A newer generation needs equivalent provenance and archive checks before replacing this input.

Targets and image admission still required

No execution target has been admitted. Select a disposable host/context and record its immutable identity, operator, private endpoints, maximum lifetime, resource limits and exact owned cleanup inventory. Never select the production context or reuse live PVCs. Use fresh private storage; no production host mounts, service-account tokens, ingress controller or external DNS update credentials.

The October 16 read-only pod observation is recorded in docs/evidence/2026-10-16-identity-outage-recovery/image-observation.json. These are candidate inputs, not confirmation of the images in the selected archive:

Component Candidate reference Admission gap
PostgreSQL postgres@sha256:ff23cdce56cac62ada6f66013e1a50864c0abbe79d132d40b6e05bd80f378a70 Passed archive DB restore; stage independently of failed registry
LLDAP docker.io/lldap/lldap@sha256:9e605a66c02514bfcffd1b67cafb1e98d50992216bb2871d7ae44622047dd09d Actual running digest resolved; compare selected archive/runtime lineage and stage it
Authelia docker.io/authelia/authelia@sha256:bd97cff4fcbf715b5ff1f9ae286afbe6033afce385302520b0368122d43a6f54 Compare selected archive deployment and stage image
KeyCape forgejo.coulomb.social/coulomb/key-cape@sha256:cdec9dd7c515f7e0a6169fc3fa560ee2cc47bf40d899755dec808c362f5ce010 Compare selected archive; require independent image availability
privacyIDEA ghcr.io/gpappsoft/privacyidea-docker@sha256:af7841adad262f129e0c1d4f553af13f21cb2f4dc713533f316cfe43ed0b4473 Actual main and sidecar digest observed separately; compare archive/runtime lineage and stage it

Record image archive digests and offline load/inspect results. Resolve mutable references from trusted runtime/image metadata rather than pulling today's tag. The encrypted archive deployment records must be compared privately; do not export literal environment values, Secret payloads or arbitrary configs.

Restore order and containment

  1. Freeze the selected artifact/manifest/image/target references. The owner supplies existing-user credentials and an existing retained MFA token through private inputs. Agree time synchronization and test identities without exposing passwords, seed material, client secrets or user records.
  2. Establish an isolated network with default-deny external egress, no published ports, no host networking and no production DNS/ingress. Permit only named internal service links. The test browser, if needed, must use a separately reviewed local route and hosts mapping. Preserve issuer/client identity while keeping all test traffic private. Verify denied external connection attempts.
  3. Import manifest-checked data and original keys into fresh private storage. Do not apply archived Kubernetes objects wholesale: metadata UIDs, mounts, service accounts, endpoints and sidecars require explicit translation. Keep notification/provider credentials sealed and unused; disable sending paths.
  4. Restore the linked privacyIDEA database with native PostgreSQL first; restore LLDAP data/config and verify directory readiness. Run directory authentication before dependent provider checks. Restore privacyIDEA with its original enckey and audit pair against the isolated database/directory. Restore Authelia with original storage/OIDC keys and private directory routes. Restore KeyCape with original signing/config/policies and private MFA/directory dependencies.
  5. Run the individual acceptance checks below; do not start provisioning, enrollment, rotation, scheduled jobs or production consumers automatically. An unsupported dependency is a named gap, never an implicit production route.
  6. Remove only the named disposable containers/network/storage and temporary plaintext/key files. Confirm absence and production resources unchanged; retain sanitized receipts, original ciphertext and independently held keys.

Acceptance matrix

Task / owner lane Positive check Negative check Safe evidence
T16 / LLDAP Preserved user/group/membership identities; existing-user bind/login Wrong password rejected Image/target IDs, aggregate comparisons and booleans
T17 / Authelia Native storage decryption; same signing identity/client configs; isolated authentication Wrong credentials or unregistered client refused Public key thumbprint, test outcome, notification suppression
T19 / privacyIDEA MFA Existing retained token decrypts internally and actual existing-token MFA succeeds Wrong enckey and invalid code refused in disposable copies Token material never exported; bounded outcomes only
T20 / privacyIDEA audit Verify historical signatures; create and verify one isolated entry with original key Mismatched public key/tampered disposable record rejected Signature counts, continuity result, cleanup
T18 / KeyCape Same issuer/JWKS, registry/policy/receipt continuity; named current consumer authenticates Wrong secret and denied scope rejected Public fingerprints, owner-selected test IDs and outcomes

Do not silently reset credentials or re-enroll tokens to pass. Historical audit verification failures must be reported, not hidden with replacement keys.

Unresolved inputs before execution

  • Named immutable disposable target, approved private browser route if required, resource limits, cleanup inventory and operator window.
  • Confirm archive/runtime lineage for the observed image digests and stage all images independently of production and the failed registry.
  • A retained user and MFA token available in private owner custody; exact native verification commands reviewed against the chosen service versions.
  • T02 current-caller ledger and T21 per-caller owner admission, including legacy token, diagnostic, resolver and LDAP-group boundaries.
  • Service-owner acceptance of network/config translations and all required grants. New grants or production effects require their own owner decisions.

Until resolved, T15 stays in progress and T16–T20 remain waiting. Production cutover, full-host/OpenBao restoration and application database recovery are separate owner procedures.