diff --git a/AGENTS.md b/AGENTS.md index b13f7ec..d080bee 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -229,6 +229,36 @@ To create a new workplan: the session close protocol. Ask the operator only if the CLI or API is unavailable. +### Task refinement after substantial work (required) + +At each substantive checkpoint and before session close, compare the completed +work and new knowledge with the current task structure. If substantial work was +completed while the workplan's task granularity stayed unchanged, **add completed +task records for those evidenced outcomes and refine the remaining tasks**. +A long progress note inside the same broad open task does not satisfy this rule. + +- Give each substantial completed deliverable a task with its actual outcome, + completion date, evidence and limits. Retrospective recording is allowed; do + not invent completion dates, deliveries, native acceptance or approvals. +- Split remaining work into independently manageable deliverables with an owner, + next action, completion criteria and real dependencies. Use `todo` for work + that can start; use `wait` only with a concrete prerequisite and unblock condition. +- Preserve existing task IDs, Hub UUIDs and history. When narrowing an umbrella + task, explain the split and map moved scope to the new task IDs. Do not mark + the umbrella done unless its retained scope is complete; preserve externally + referenced acceptance gates until their dependencies are satisfied. +- Keep task counts honest: extracting completed scope must not count the same + outcome twice or imply an effort-based completion percentage. Distinguish + source/fixture proof, actual operational proof and owner acceptance. +- Record new risks and out-of-scope findings as explicit disposition/handoff + tasks or linked live work records; do not silently expand authorization. + +Make these changes in the repository source, commit/push, run the repository's +required projection sync, and verify the actual projected task counts, statuses +and dependencies. Progress events supplement the task structure; they do not +replace it. A checkpoint report must show both substantive outcomes and newly +refined remaining work when this rule is triggered. + ## NetKingdom situational knowledge pilot diff --git a/workplans/NK-WP-0049-bootstrap-secret-free-recovery-handoff.md b/workplans/NK-WP-0049-bootstrap-secret-free-recovery-handoff.md index 5c72006..2d444ae 100644 --- a/workplans/NK-WP-0049-bootstrap-secret-free-recovery-handoff.md +++ b/workplans/NK-WP-0049-bootstrap-secret-free-recovery-handoff.md @@ -14,10 +14,34 @@ depends_on: [] state_hub_workstream_id: "17d315f0-829e-57c7-85cc-3a7914152804" --- -Native owner work for CUST-WP-0080 G08. NK-WP-0045 proves recovery of the -non-secret root-grant record, not custody of the bootstrap private key or -secret-bearing emergency bundle. Existing founder-deferred exposure rotation -remains CUST-WP-0073-T05; this work does not override it or rotate any root. +Native owner work for CUST-WP-0080 G08. This workplan remains active until the +remaining caller, service-recovery and custody acceptance gates are satisfied. +NK-WP-0045 covers the non-secret root-grant record. Founder-deferred exposure +rotation remains in CUST-WP-0073-T05; this plan does not silently reopen it. + +**Replanned 2026-10-16 at the operator's request:** substantial completed work is +now represented by completed tasks, and remaining work has separate deliverables, +dependencies and acceptance criteria. T01–T03 retain their historical IDs and Hub +UUIDs. Completed parts of the old T02/T03 umbrellas are extracted into T04–T14; +T02 now covers the current-caller ledger and T03 retains final owner acceptance. +These are retrospective task records of evidenced work, not additional delivery +or newly granted authority. The earlier dated narrative remains in Git at +`cc2dd01`; its evidence references are carried into the tasks below. + +**Current position:** 12 tasks done; 1 in progress; 3 ready to start (`todo`); +8 waiting on explicit prerequisites. These are task counts, not an effort-based +completion percentage. No remaining task is being executed by this replan. + +**Next actions:** finish the read-only caller ledger (T02); prepare the isolated +restore packet (T15), independent offline retrieval (T23), and residual-scope +handoffs (T24). After owner admission, execute the individual native service and +caller checks. A task's owner is accountable for the result; another repository's +approval or credential grant still requires its own owner path. + +**Closure:** T03 assembles owner acceptance after its named dependencies pass. +Passing archive recovery does not prove existing-user authentication, retained +MFA-token decryption, historical audit signatures or current caller admission. +The verified backup job stays active while those separate tasks are completed. ## Remove secret-bearing output and ambient root discovery @@ -45,583 +69,511 @@ Make/help/status/console and current credential guidance reflect that distinctio Evidence and exact source manifest: Custodian `docs/evidence/2026-10-12-netkingdom-security/bootstrap-handoff.json`. -## Reconcile other legacy custody callers and root-output boundaries +## Harden custody, SOPS and rotation source entry points + +```task +id: NK-WP-0049-T04 +status: done +priority: high +owner: codex +depends_on: [NK-WP-0049-T01] +``` + +Completed 2026-10-12–13. Public-recipient-only encryption, explicit temporary +custody, private no-overwrite output, sanitized diagnostics, stdin-only secret +transport and failed-operation cleanup replace ambient-key and output paths. +SOPS read/edit/data-key rotation preserve recipient/MAC/source-version guards; +component rotations preserve unrelated fields and retain recovery candidates. +This records source and disposable-native proof, not live rotation or revocation. + +Evidence: Custodian `docs/evidence/2026-10-12-netkingdom-security/` receipts +`custody-entrypoints.json`, `sops-make-entrypoints.json`, `rotation-output.json`, +and `admin-bootstrap.json`; October13 `sops-write-callers.json`. + +## Repair component, diagnostic, LDAP and resolver callers + +```task +id: NK-WP-0049-T05 +status: done +priority: high +owner: codex +depends_on: [NK-WP-0049-T04] +``` + +Completed 2026-10-13–14. Repaired KeyCape configuration and component Secret +publication, original privacyIDEA key export/restore transport, scoped MFA +counts, LDAP group maintenance and configuration-preserving resolver cutover. +Retired unsafe personal-token mint/refresh and direct LDAP lifecycle shortcuts. +Disposable native PI/LLDAP tests and source regressions passed; estate provider +admission, historical credential validity and service authentication remain T21/T22. + +Evidence: Custodian October13 `keycape-caller-reconciliation.json`, +`service-secret-callers.json`, `privacyidea-key-custody.json`, +`mfa-state-diagnostic.json`, `realm-bootstrap-repair.json`; October14 +`legacy-lldap-retirement.json`, `platform-group-boundary.json`, +`lldap-group-maintenance.json`, `resolver-cutover.json` under the corresponding +`docs/evidence/YYYY-MM-DD-netkingdom-security/` directories. + +## Inventory source callers and retire remaining unsafe delivery paths + +```task +id: NK-WP-0049-T06 +status: done +priority: high +owner: codex +depends_on: [NK-WP-0049-T04, NK-WP-0049-T05] +``` + +Completed 2026-10-15. Enumerated all 87 current non-test shell/Python callers +in the declared bootstrap/k8s scope plus Make/console/delivery adapters. Repaired +streamed encrypted bundle publication and fail-closed bootstrap/lab paths; +retired unsafe delivery copiers, root prompts, registry patchers and stale +rollback recipes. The attended inventory records source dispositions and their +native-proof limits. T02 separately maps these to current live users/owners. + +Evidence: [caller inventory](../docs/evidence/2026-10-15-bootstrap-recovery/caller-source-inventory.json), +[attended procedures](../docs/attended-procedure-inventory.md) and the October15 +recovery evidence directory. No live production bootstrap or lab-init acceptance +is claimed. + +## Discover off-host coverage and recover the existing PI database + +```task +id: NK-WP-0049-T07 +status: done +priority: high +owner: codex +``` + +Completed 2026-10-15–16. Bounded Storage Share/Scaleway inventories identified +existing database coverage and the missing full identity bundle. The operator +independently matched the backup recipient, then recovered the exact existing +PI dump: correct-key decryption, wrong-key refusal, isolated native PostgreSQL +restore/readback and cleanup passed (5 tokens, 2 resolvers, 2 realms). + +Evidence: October15 recovery `offsite-discovery-summary.json`, +`privacyidea-transfer-r1.json`, `privacyidea-database-drill-r1.json`, +`database-drill-review.json`. This database-only proof led to T08–T14. + +## Implement the full identity capture and archive recovery toolchain + +```task +id: NK-WP-0049-T08 +status: done +priority: high +owner: codex +depends_on: [NK-WP-0049-T06, NK-WP-0049-T07] +``` + +Completed 2026-10-16. Defined exact LLDAP/Authelia/KeyCape/privacyIDEA assets, +Deployment/PVC/reference bindings, original keys and snapshot consistency limits. +Implemented native SQLite/PG capture, streamed age sealing, strict data-bearing +manifest checks, provider readback, independent retrieval, bounded retention, +root runtime and attended archive verifier. All 56 focused checks passed, +including actual synthetic encrypted/native PostgreSQL recovery. + +Evidence: [profile](../tools/bootstrap-recovery/identity-backup-profile.json), +[implementation review](../docs/evidence/2026-10-16-identity-backup-implementation/review.json), +and the live evidence's tested ESO-default/host-binding corrections. Production +archive recovery is separately evidenced in T13. + +## Admit and verify the two scoped backup credential deliveries + +```task +id: NK-WP-0049-T09 +status: done +priority: high +owner: codex +depends_on: [NK-WP-0049-T08] +``` + +Completed 2026-10-16. Platform CCR-2026-0048/0049 are verified. The operator +approved folder-only provider-normalized publisher21/retention25 rights, including +re-sharing, in addition to the original rollout. Coding-agent denials precede KV +publication. Exact 900-second single-policy readers, wrong-SA/namespace/audience +rejection, metadata/sibling/old-escrow/token-create denials, ESO equality and +reader/session revocation passed. Operator passwords and age private keys are +not job credentials. + +Evidence: live `publisher-setup-verified.json`, `retention-setup-verified.json`, +`publisher-native-delivery-verified.json`, `retention-native-delivery-verified.json`; +approval decisions `40bfae40-6108-4e51-8613-073339f5345f` and +`5edd5203-fae7-461b-8775-5d22b7873971`; Platform CCR files. + +## Apply capture isolation and install the guarded node runtime + +```task +id: NK-WP-0049-T10 +status: done +priority: high +owner: codex +depends_on: [NK-WP-0049-T08, NK-WP-0049-T09] +``` + +Completed 2026-10-16. Applied exact UID/resource-version/spec-guarded DNS +selector exclusions, preserving ordinary service matches. Capture refuses any +remaining helper allow policy. Installed immutable runtime sources bound to the +actual hostname, machine identity and cluster UID. Disabled-unit replacement +checked previous contents and retained rollback bytes. Health unit validation +passed. Activation remained gated until T13. + +Evidence: live `isolation-applied.json`, `capture-ready-plan.json`, +`installed-runtime-packet-r2.json`, `upgrade-disabled-runtime.py`. Actual complete +hardened-job execution is recorded in T14. + +## Connect recovery to Railiance bootstrap and reuse discovery + +```task +id: NK-WP-0049-T11 +status: done +priority: high +owner: codex +depends_on: [NK-WP-0049-T08] +``` + +Completed 2026-10-16. Added the Railiance bootstrap `identity-recovery` +dispatcher and independent-custody → isolated verification → layer/service-owner +handoff. Updated `capability.security.bootstrap-recovery`, linked bootstrap +sequencing and composed reuse-surface federation. Keyword discovery for identity +backup, restore, privacyIDEA and disaster recovery was verified. The operator +actually used this handoff in T13. No unattended production importer is claimed. + +Evidence: [ceremony](../docs/bootstrap-recovery-ceremony.md), owning capability +entry, `railiance-bootstrap/docs/identity-recovery.md`, bootstrap commit `d1dcfde` +and reuse-surface federation commit `662782e`. + +## Create and verify the first full off-host identity bundle + +```task +id: NK-WP-0049-T12 +status: done +priority: high +owner: codex +depends_on: [NK-WP-0049-T09, NK-WP-0049-T10] +``` + +Completed 2026-10-16. Captured original service keys/current data plus the +linked native PI database dump, sealed and published a unique Storage Share +artifact, then downloaded it and verified size/digest before publishing its index. +Helper and plaintext cleanup passed. Artifact: +`net-kingdom-identity-20261016T063159Z-5882771aa2ce.tar.age` (109738344 bytes). + +Evidence: live `actual-capture.json`, `actual-offsite.json`; ciphertext SHA256 +`7117b3fe1e4840c86b301c94c85e6b16709e5626bb0884bd0580bde630e30e30`. +No age private key was needed by the capture job. + +## Independently retrieve and recover the complete actual archive + +```task +id: NK-WP-0049-T13 +status: done +priority: high +owner: codex +depends_on: [NK-WP-0049-T11, NK-WP-0049-T12] +``` + +Completed 2026-10-16. Independently downloaded the exact first generation +through attended Backup-account custody, rehashed it locally and revoked the +session. The operator used the independently held age key through the bootstrap +handoff. Correct/wrong-key cases, all 36 manifest files/hashes, original RSA pairs, +SQLite integrity and isolated native PG16.14 restore/readback passed; plaintext +and containers were removed. No key or restored secret values reached the agent. + +Evidence: live `independent-owner-transfer.json`, `actual-full-archive-recovery.json`, +`actual-full-recovery-review.json`. Existing-user/MFA authentication and old audit +signatures are deliberately separate T16–T20 outcomes. + +## Activate and prove the complete recurring backup and retention job + +```task +id: NK-WP-0049-T14 +status: done +priority: high +owner: codex +depends_on: [NK-WP-0049-T10, NK-WP-0049-T13] +``` + +Completed 2026-10-16 after T13. Both daily backup/retention and hourly health +timers are enabled/active. The actual hardened systemd service passed retention +before/after a second capture/upload/download (Result=success, ExecMainStatus=0). +Both verified generations are retained, the independently recovered point is +pinned, and nothing was deleted. Helpers/plaintext were cleaned; failure markers +and reporting outbox are empty. Health check succeeded. + +Evidence: live `activation-result.json`, `activation-review.json`, +`recurring-job-review.json`. Cadence: daily 03:45 UTC plus up to 5 minutes jitter, +hourly health, 25-hour freshness bound. Second-provider/host-loss gaps go to T24; +this completed task is not a full-estate recovery claim. + +## Map current callers, owners and native acceptance gaps ```task id: NK-WP-0049-T02 status: progress priority: high +owner: codex +depends_on: [NK-WP-0049-T06] state_hub_task_id: "479d6970-6516-50f5-9436-bb5b56d58aad" ``` -Inspect remaining direct encrypt/decrypt defaults, permanent-key setup commands, -custody helper tracing/error handling and delegated apply/init helpers against -the accepted single-founder attended custody decision. Public-recipient-only -encryption and explicit temporary unlock must hold at their actual call sites. -Do not treat a tty check as identity, authority or agent isolation. Reuse -CUST-WP-0073/GLAS-WP-0012 for runtime isolation and Platform for root custody; -record any further exact source defects and repair them before native acceptance. +The source inventory and known legacy objects exist; a complete current-use +ledger is not yet accepted. Next action is read-only reconciliation of the 87 +source entries against active Make/bootstrap/operator routes and owner evidence. -2026-10-12 caller audit: direct encrypt/decrypt helpers and custody unlock repaired. -Eight initial regressions failed against prior source. Encryption defaults to the -registered public file and refuses private-key inputs; decryption has no HOME -fallback and publishes only complete private output from staging, preserving -existing outputs and removing partial failures. Unlock suppresses xtrace before -hidden input, rejects ambiguous modes, removes competing inline/key-command -credentials and preserves signal failure status with EXIT cleanup. Disposable -crypto and prompt/child tests cover these boundaries. Bash foreground-child -signal deferral and arbitrary-child copy retention are explicitly not solved by -this helper. Evidence: Custodian `custody-entrypoints.json`. +**Done when:** each entry is identified as current, retired or owner-deferred, +with its owner, credential route, current consumer(s), native evidence or exact +missing proof, and next responsible task. Explicitly cover the historical +`sso/keycape-pi-token` and `mfa/privacyidea-trigger-admin` objects, lab init, +factor diagnostic delivery, LDAP group/provider boundary, resolver cutover and +root-output/isolation boundaries. Secret presence is not credential validity. +Send current native checks to T21 and historical revocation/rotation decisions to +T22. Do not reopen completed source repairs without a new finding. -Concrete uncovered callers remain (T02 stays progress): -- The KeyCape personal-admin token mint/refresh paths are retired by the October - 13 repair below. Historical token copies still need owner inventory/revocation; - the old `privacyidea/check-user-mfa-state.sh` also consumes that legacy Secret - with insecure curl/argv handling and still requires reconciliation. -- Legacy trigger account/Secret inventory and owner-approved revocation are still - required; removing automatic creation does not revoke historical credentials. -- RSA and Authelia-to-KeyCape client-secret rotation need configuration-preserving - updates, retained rollback custody and native multi-consumer acceptance. The - legacy replacement paths now refuse before mutation; this is not rotation closure. -- PostgreSQL/privacyIDEA/LLDAP/Authelia generators and the batch apply caller - and direct SOPS edit/encrypt/rotate are repaired below. Other diagnostic, - bootstrap and key-export callers remain open; native effect/recovery evidence - is mandatory. +## Prepare the isolated full-service restore and acceptance packet -2026-10-12 Make caller repair: sops-setup now validates only the public recipient; -creds-init no longer requires or installs a permanent private key. sops-decrypt -uses the new age-only SOPS recovery adapter: explicit matching private custody -file, isolated HOME/XDG/environment, pinned file descriptor, sanitized diagnostics, -private no-overwrite output outside Git, atomic publication and failure cleanup. -FILE/OUTPUT are passed as environment data instead of interpolated shell code. -The old targets failed all 11 initial real-SOPS regressions. Additional format, -parser, environment-command and credential-mode tests cover the final adapter. -Existing HOME copies were not deleted; native recovery and the remaining direct -editor/rotate/delegated paths are unverified. Evidence is Custodian -`sops-make-entrypoints.json`; T02 remains progress, not source-surface closure. +```task +id: NK-WP-0049-T15 +status: todo +priority: high +owner: codex +depends_on: [NK-WP-0049-T08, NK-WP-0049-T13] +``` -2026-10-12 rotation caller repair: 20 initial regressions failed against prior -source. New values and delegated diagnostics are suppressed, xtrace disabled, -private owned no-symlink/single-link service files validated and replaced -atomically, and Make passes SECRET as data. Database discovery fails closed -before local mutation; SQL travels on stdin with ON_ERROR_STOP. Unattended LDAP -password rotation is refused while the server update remains manual. Break-glass -uses file input/server-side apply rather than argv literals/client annotations. -Additional refusal/failure/cancel regressions cover final source. Successful -script timestamps are not native consumer/revocation evidence; partial updates -retain the local candidate for recovery. No live rotation occurred. Evidence: -Custodian `rotation-output.json`. T02 and the founder-deferred rotation stay open. +**Next action:** turn the recovered manifest and current caller gaps into a +reviewable procedure with selected immutable images, restore order, isolated +endpoints/network, data/key ownership, cleanup and per-service positive/negative +checks. Preparation can start with the existing archive; incorporate T02's ledger +before submitting current-caller acceptance. -2026-10-12 admin-bootstrap reconciliation: current SCOPE marks Keycloak backlog; -KeyCape uses the separate pi-admin token path. Removed automatic legacy trigger -account/policy/Secret creation and corresponding current readiness expectations. -Fourteen old-source tests failed (two passed). The new entry point parses owned -private input without executing shell code, sends credentials only on kubectl -stdin, sanitizes failures and requires one pod plus a strict value-free receipt. +**Done when:** service owners accept the exact targets/procedure, existing-user +and MFA test identities/input custody are available without posting secrets, +outbound notification/provider writes are disabled, and every required new +credential or production effect has its own owner decision. The verifier deletes +its plaintext; it is not a production importer. This task prepares authority and +inputs for T16–T20, not an automatic production restore. -Upstream 3.12.2 inspection corrects the earlier mismatch hypothesis: `admin add` -uses an upsert and may reset an existing password. The new version-scoped in-pod -adapter verifies existing credentials and refuses mismatches or conflicting -policies; new rows use insert-only transactional writes and a policy scoped to -pi-admin. Disposable actual privacyIDEA 3.12.2/SQLite tests verify repeat stability, -wrong-password refusal, legacy-policy/other-admin refusal, rollback and conflict -handling. These do not prove the deployed image/PostgreSQL or effective MFA. -Existing objects are not deleted or rotated. Current callers and the native -acceptance gate remain open. Evidence: Custodian `admin-bootstrap.json`. +## Prove restored LLDAP directory and existing-user authentication -2026-10-13 KeyCape caller reconciliation: RPF-WP-0040 is finished and its -owner contract/deployment use the scoped renewable factor-reader credential with -ESO and `adminTokenFile`. Retired `create-pi-token.sh` and -`refresh-pi-token-live.sh` before input/credential/cluster access; neither personal -admin JWT delivery nor automatic realm/MFA-policy replacement is a current -consumer requirement. Existing provider identities/Secrets were not revoked. +```task +id: NK-WP-0049-T16 +status: wait +priority: high +owner: codex +depends_on: [NK-WP-0049-T15] +blocking_reason: "Await the accepted isolated restore packet and independently supplied existing-user test credentials; no production restore is admitted by task refinement." +``` -The first-bootstrap config generator now uses that mounted-file source, private -literal input parsing, captured child diagnostics, stdin-only Secret data and -atomic private signing-key publication without replacement. It requires Ready -factor delivery and refuses existing config; API creation cannot overwrite a -concurrent creator and adds no client-side apply annotation. A failed API call -leaves its remote outcome unverified and retains the private key for recovery. -Existing installations require a scoped owner update preserving clients/policy. +Owner boundary: NetKingdom directory/User Engine owners. Restore the captured +original directory/configuration in the admitted isolated environment. -Bootstrap callers and recovery guidance no longer invoke retired token helpers. -Legacy RSA/client-secret replacement rotations refuse before local or remote -mutation: the former destroyed rollback custody before attempting publication, -and both would overwrite newer live configuration. The remaining coordinated -rotation work stays here, not silently completed by refusal. Native recovery, -current deployed consumer acceptance and other create/apply/SOPS callers remain -open. Source verification and its limits: Custodian -`docs/evidence/2026-10-13-netkingdom-security/keycape-caller-reconciliation.json`. +**Done when:** user, group and membership identities are preserved; an independently +supplied existing-user credential authenticates; a wrong credential is refused; +provider writes/outbound effects stay confined; cleanup is verified. Native +fixtures from T05 and SQLite integrity from T13 do not satisfy this current-data +acceptance. Record only safe counts, identities and outcomes. -2026-10-13 delegated service caller repair: the PostgreSQL, privacyIDEA, -LLDAP and Authelia generators now share private literal-input parsing, captured -child diagnostics and stdin-only Secret transfer. Ordinary invocation creates -an absent object or verifies matching fields; it cannot silently rotate an -existing object. Named rotation updates only its allowed field. UID/resource -version/data tests guard patches; unknown data and unrelated metadata survive, -and last-applied copies are removed under the same guard. Owner references, -ESO management labels, immutable objects and wrong target identity are refused. -The returned API object is checked before success; write failure remains an -unknown remote outcome requiring reconciliation, never blind retry. +## Prove restored Authelia storage and OIDC continuity -Authelia first creation retains a private signing key atomically; API failure -keeps it for recovery. Existing issuer keys and matching bcrypt hashes survive -repeat apply and session rotation. Explicit unsafe/differing keys are refused. -The batch caller suppresses child output and records only successful components. -Rotation callers pass exact field selectors; database rotation now updates the -CNPG application Secret as well as the privacyIDEA URI. The attended LDAP path -also depended on retired full KeyCape replacement, so now refuses before any -change. SQL, Secrets and restarts remain non-atomic; coordinated recovery and -native database/consumer acceptance are still required. +```task +id: NK-WP-0049-T17 +status: wait +priority: high +owner: codex +depends_on: [NK-WP-0049-T15, NK-WP-0049-T16] +blocking_reason: "Await the accepted isolated packet and restored-directory authentication proof from T16." +``` -All 63 initial regression cases failed on the old helpers; the replacement and -expanded caller tests pass, including real OpenSSL/bcrypt and actual kubectl -local stdin patch/stale-version refusal with no cluster. Final source-bound -suite results, dependency skips and test limitations are in Custodian -`docs/evidence/2026-10-13-netkingdom-security/service-secret-callers.json`. -No live Secret, signing key, credential or deployment was changed. T02 remains -progress for the explicit callers/lifecycle work above; T03 remains waiting for -its owner-scoped native recovery ceremony. +Owner boundary: SSO/Authelia and directory owners. Restore the original storage, +encryption/session/signing material and configuration in isolation. -2026-10-13 SOPS write caller repair: Make passes paths and custody arguments -literally, removing shell/Make execution before handoff. Public-only encryption -refuses tracked plaintext. Edit consumes private replacement JSON/YAML outside -Git and verifies the original MAC; rotate changes the data key only. Both require -explicit matching custody and exactly the independently registered recipient, -refusing metadata-injected recipients. Child diagnostics/plaintext output are -suppressed, policy is preserved, and ciphertext publication checks source version -before atomic replacement. Shared private-key validation rejects links/FIFOs. -A stat/rename race against noncooperating same-UID writers remains; no OS isolation -or live recipient migration is claimed. Validation and exact source binding: -Custodian `docs/evidence/2026-10-13-netkingdom-security/sops-write-callers.json`. +**Done when:** native storage decryption succeeds, issuer/signing identity and +client configuration are preserved, and an isolated authentication flow has +positive/negative results using the restored directory. Disable outbound +notifications and verify cleanup. Archive integrity/RSA parsing alone are not +native Authelia acceptance. -Remaining closure checklist (source review, not proof of current live use): -- `privacyidea/enckey-bootstrap.sh` and its bootstrap/documentation callers are - repaired below, including private no-overwrite export and a real PVC recovery - transport. Actual off-host backup lineage, reviewed image/cluster execution and - native service/database/token/audit recovery acceptance remain T03 requirements. -- `privacyidea/check-user-mfa-state.sh` and its guide/dry-run callers are repaired - below: explicit private input, scoped counts and reader-visibility proof replace - retired Secret retrieval. Live diagnostic delivery/execution still requires - separate owner admission; KeyCape's workload lane is not silently shared. -- LDAP user-lifecycle shortcuts are retired and group callers are repaired below - through the existing User Engine/service-group boundary. Inventory completeness - and live provider admission still need owner evidence. -- General realm repair and both incident resolver helpers are repaired below. - Full caller/root-output inventory and live owner-scoped acceptance remain open. -- Reconcile the complete `docs/attended-procedure-inventory.md` against current - scripts and Make/bootstrap/root-output callers. The selected findings above - are not an exhaustive closure audit. Retain historical revocation and - configuration-preserving rotation requirements already recorded here. -- After source coverage, execute T03's exact owner-approved off-host recovery and - current-consumer checks. Source regressions do not satisfy that gate. +## Prove retained privacyIDEA token decryption and existing-token MFA -2026-10-13 privacyIDEA key export/recovery source repair: `key-custody.py` -and its captured in-pod adapter verify the Ready deployment-owned source pod and -Bound PVC, validate the 96-byte encryption key and matching RSA audit pair, and -retain private local copies without overwrite. DR Secrets are create-or-verify; -existing extra fields/metadata survive and annotation removal uses conditional -UID/version/data tests. The shared Secret publisher now accepts binary values. -Missing encryption keys are never regenerated. Audit-pair creation is a separate -explicit first-bootstrap flag, refused when any local/cluster recovery copy or -partial pair already exists. The bootstrap caller forwards that flag only when -explicitly selected, and script errors/key bytes are not forwarded to the terminal. +```task +id: NK-WP-0049-T19 +status: wait +priority: high +owner: codex +depends_on: [NK-WP-0049-T15, NK-WP-0049-T16] +blocking_reason: "Await the accepted isolated packet, restored directory and owner-selected retained-token/MFA test inputs." +``` -Restore now has an executable captured transport to the actual `privacyidea-data` -PVC. A rendered helper uses an independently reviewed immutable image digest, -no application entrypoint or service-account mount, and the real data volume. -The wrapper requires a stopped Deployment, exact PVC UID and the sole Ready -mounting helper pod, refuses differing files and retains partial writes for -reconciliation. Files are published without replacement and checked afterward; -this is not a multi-file transaction or UID-atomic Kubernetes exec. The admitted -new-file ownership is root:999 with mode 0640 for private files, matching the -source fsGroup. Current runtime ownership/consumer acceptance still needs proof. +Owner boundary: MFA/privacyIDEA and directory owners. Use the exact linked dump +and original enckey from the independently recovered generation. -`privacyidea/KEY-RECOVERY.md` replaces the broken unmounted-helper and Secret-only -restart examples, resolves `enckey` versus `PI_ENCFILE`, and keeps stopping, -creating/removing the reviewed helper and resuming the service explicit in the -owner ceremony. Disposable native privacyIDEA 3.12.2 checks prove decryption of -pre-recovery values and audit signature continuity after the local restore. -Failure, partial-publication, wrong-target and mismatch tests plus broader results -are retained in Custodian -`docs/evidence/2026-10-13-netkingdom-security/privacyidea-key-custody.json`. -No production key, PVC, workload or Secret was read or changed for these tests. -T02 remains progress for the remaining diagnostic/LDAP/realm callers and full -inventory/lifecycle reconciliation. T03 remains the native recovery gate. +**Done when:** retained token material decrypts inside the isolated process, +a selected existing token completes a native MFA check, a wrong key/code is +refused, and cleanup is verified. Obtain any token/user input through attended +private custody; never emit token plaintext, seeds or credentials. Native database +row counts and disposable synthetic-token fixtures do not prove this outcome. -2026-10-13 scoped factor diagnostic repair: replaced retired admin-token Secret -retrieval with `mfa_state.py` and explicit private JWT input. The read-only helper -uses only `/token/` in `coulomb`, checks the existing disabled/unassigned realm -probe before and after lookup, refuses incomplete pages, and distinguishes -active-confirmed, active-pending and inactive counts. Empty results are not proof -of user existence or login. TLS verification, origin restriction, redirect/proxy -refusal, bounded responses and fixed errors prevent the original credential -argv/provider-error output paths. No automatic retrieval, issuance or permission -expansion is introduced. Guide and non-root dry-run now require explicit admitted -input; the latter checks its selected user rather than hard-coded platform-root. -Source/native disposable verification and limits are recorded in Custodian -`docs/evidence/2026-10-13-netkingdom-security/mfa-state-diagnostic.json`. -KeyCape-only RPF-WP-0040 delivery is not treated as authorization for this helper. -LDAP/realm callers, full inventory and T03 native recovery remain open. +## Prove historical and new privacyIDEA audit-signature continuity -2026-10-13 realm/bootstrap source repair: replaced shell-sourced/%q credential -transport with literal private files or attended in-memory prompts. Preview is -read-only after authentication; `--apply` executes missing-object/tuning repair. -Existing bind credentials, valid custom tuning, realm bindings/priorities and -defaults are preserved; target/profile drift refuses before mutation. Existing -policies are never replayed. The unsafe broad self-service/passthrough bootstrap -writes are withdrawn in favor of the established onboarding-policy owner flow. -The attended wrapper retains functional lookup/fresh-MFA verification and fails -on partial/unknown outcomes without retry. Shared `pi_api.py` now pins allowed -origins, verifies TLS, refuses redirects/proxies, bounds responses and sanitizes -errors. The console and guides reflect these boundaries. -Actual privacyIDEA 3.12.2 proves minimal resolver POST preserves omitted fields; -this corrects the historical general replacement claim without rewriting its -incident receipts. Disposable and broader test outcomes are in Custodian -`docs/evidence/2026-10-13-netkingdom-security/realm-bootstrap-repair.json`. -No live credential, resolver, realm, policy or token was touched. LDAP callers, -incident-specific rotation/cutover review, full inventory and native recovery -acceptance remain open; T02 stays progress and T03 wait. +```task +id: NK-WP-0049-T20 +status: wait +priority: high +owner: codex +depends_on: [NK-WP-0049-T15] +blocking_reason: "Await the accepted isolated target/procedure and owner-admitted historical/new audit verification from T15." +``` -2026-10-14 legacy lifecycle retirement: `create-user.sh`, -`dry-run-nonroot-user.sh` (including pattern cleanup), and `break-glass.sh` now -refuse with exit 2 before arguments, credential discovery or external calls. -The Make targets no longer interpolate user inputs. Console execution/claims -commands refuse before metadata loading or delegation; the inferred-claims helper -cannot report success. Lifecycle worksheets start blank with false assurances, -and their validators reject them until populated. Existing historical records -remain unchanged. Current guidance routes ordinary onboarding/lifecycle through -User Engine and the existing identity-provisioner; platform-root and independent -recovery retain their separate IAM owner boundaries. +Owner boundary: MFA/privacyIDEA audit owner. Use the original recovered audit +pair and linked database in the admitted isolated target. -Eleven corrected-baseline regressions failed against the old source; the first -draft also contained a wrong guide-function name, corrected before that baseline. -Final focused console/retirement tests cover refusal under tracing, no credential -file execution/external calls, Make shell-expansion refusal, no metadata reads, -blank-template rejection and no inferred claims. Active callers and operator -instructions no longer recommend the retired scripts; old architecture and -assessment snapshots carry explicit superseding notices. New reference: -`docs/legacy-lldap-lifecycle-retirement.md`. Evidence is retained by Custodian in -`docs/evidence/2026-10-14-netkingdom-security/legacy-lldap-retirement.json`. +**Done when:** existing pre-backup audit signatures are verified natively, a new +isolated audit entry is signed and verified with the preserved identity, a wrong +key fails, and cleanup is recorded. Report any historical verification failure +for explicit owner disposition; do not regenerate keys to hide it. Matching RSA +pairs from T13 are a prerequisite, not audit-history verification. -No account/password/token or deployed workload changed; retirement is not -revocation or proof of independent recovery. `bootstrap-users.sh`, -`manage-group-members.sh`, and `verify-t07.sh` still need credential-path/native -verification repair (only stale lifecycle guidance changed in two of them). -Incident cutover helpers, the full caller/root-output inventory and T03 remain -open. T02 stays progress; no workplan closure is claimed. +## Prove restored KeyCape issuer, policy and consumer continuity -2026-10-14 group-provider boundary follow-up: source review before reusing -service-group APIs reproduced acceptance of `net-kingdom-admins` and -`net-kingdom-users`. The reserved prefix covered `netkingdom-` (suspension) but -omitted the actual hyphenated platform namespace. Both ensure and revoke now -reject `net-kingdom-` before directory login, including mixed service/platform -requests and delete-if-empty. Six new regressions fail on prior source; provider -suite and source hash evidence: Custodian -`docs/evidence/2026-10-14-netkingdom-security/platform-group-boundary.json`. -This is a provider source repair, not deployed enforcement or evidence of past -exploitation. The existing service-group behavior and tenant lifecycle remain -covered by their tests. Deploy/admit this source through the existing owner path -before treating the live boundary as fixed; native enforcement remains in T03. +```task +id: NK-WP-0049-T18 +status: wait +priority: high +owner: codex +depends_on: [NK-WP-0049-T15, NK-WP-0049-T16, NK-WP-0049-T19] +blocking_reason: "Await the accepted isolated packet, restored-directory proof and retained-token MFA proof for the selected authentication flow." +``` -2026-10-14 LDAP group callers reconciled: `bootstrap-users.sh`, -`manage-group-members.sh` and `verify-t07.sh` now delegate to `lldap_admin.py`. -The adapter reuses identity-provisioner operations/service-group validation and -the existing literal private-custody reader. No shell sourcing, automatic -credential discovery, curl/kubectl invocation, credential argv or KeyCape config -Secret read remains in these entry points. Fixed-origin/system-trust transport -refuses redirects/ambient proxies, duplicate/oversized/malformed responses and -raw diagnostics. Bootstrap creates only the three missing empty baseline groups; -service membership uses an existing exact subject/group and excludes reserved -namespaces. No new account/password lifecycle or credential broker is introduced. +Owner boundary: KeyCape and the selected current consumer owners. Restore the +original signing/configuration, registry, authentication policies and receipts. -Default preview binds the origin/operation/target and observed directory state. -Apply needs that exact digest, a fresh prewrite observation and exact final -readback preserving unrelated state. LLDAP has no atomic cross-query CAS here: -owner-controlled exclusive maintenance remains required, and partial writes need -reconciliation rather than automatic rollback. A digest grants no authority. -T07 emits structural observations with native acceptance unverified and exit 2; -authentication/protocol/input failure exits 1. Object presence cannot close -lifecycle/login/MFA/revocation/recovery requirements. +**Done when:** issuer/JWKS signing identity, registry/client configuration and +policy/receipt continuity are preserved; selected current consumers pass native +positive/negative authentication using the restored dependencies; and isolation +and cleanup hold. Confirm the actual dependency graph in T15. Do not route a +production consumer to the isolated issuer or mint new client grants implicitly. -Three regressions reproduced executable credential sourcing in the old callers. -All 200 affected tests pass with no skips. An actual local immutable LLDAP image -passed preview, fixed bootstrap/preservation/idempotence, exact add/remove, -reserved-group and stale-preview refusal, and T07's incomplete exit without -tracing disclosure. Its synthetic credentials/storage/container were removed. -Current procedure: `sso-mfa/k8s/lldap/GROUP-MAINTENANCE.md`; evidence: Custodian -`docs/evidence/2026-10-14-netkingdom-security/lldap-group-maintenance.json`. -No live estate credentials or directory state were read or changed. Incident -cutover helpers, full caller/root-output inventory and owner-approved native -recovery/estate acceptance keep T02 progress and T03 wait. +## Verify current caller and provider admission at their actual boundaries -2026-10-14 incident resolver helpers reconciled: both legacy shell bodies now -invoke `resolver_cutover.py` with tracing disabled and attended memory-only -inputs. The shared JSON transport has separate destination/route profiles for -LLDAP authentication and KeyCape discovery; no privacyIDEA token crosses those -profiles. Existing resolver/realm configuration is mandatory. Preview binds the -observed public state and exact targets; apply rechecks it, performs one minimal -BINDPW/type write, and verifies all other fields and realms are preserved. -Native candidate and stored LDAP bind tests prevent cached-lookup false proof. +```task +id: NK-WP-0049-T21 +status: wait +priority: high +owner: codex +depends_on: [NK-WP-0049-T02] +blocking_reason: "Await the current-caller ledger and exact per-caller owner admission/test inputs; the backup credential grants do not admit unrelated callers." +``` -Full reconcile pins kubeconfig/context/server/cluster UID, checks current single -replica generations, validates discovery issuer/endpoints, requires one exact -user and token-backed MFA, and tests supplied predecessor HTTP denial. Missing -predecessor returns incomplete/exit 2; an ambiguous write remains unverified and -is never retried. Owner input provenance and actual KeyCape flow acceptance stay -external. No secret files, ambient endpoint overrides or raw --note receipts. -The runbook, historical merge-semantics correction and inventory are updated. +Owner boundary: each current caller's service/provider and credential owner. +Use T02's ledger to select the exact current procedures and required owner lanes. -Validation: 171 affected tests pass (including terminal tracing tests), plus -actual disposable privacyIDEA 3.12.2 and LLDAP prove old-bind refusal, candidate -and stored-bind success, exact password-only update with custom settings/realms/ -policies preserved, and real user/HOTP validation. Cluster/KeyCape gates are -substituted only in that native fixture; production acceptance is unverified. -The initial run had one test-fixture error (missing-resolver digest before the -refusal call), corrected and retained in the evidence receipt. No live change -or credential retrieval. Custodian evidence: -`docs/evidence/2026-10-14-netkingdom-security/resolver-cutover.json`. -This completes the bounded two-helper source repair; the full caller/root-output -inventory and owner-native recovery/consumer acceptance still keep T02 progress -and T03 wait. Founder-deferred exposure rotation remains deferred. +**Done when:** each current caller has scoped delivery and native positive/negative +acceptance or an explicit accepted retirement/disposition. Include dedicated +factor-diagnostic admission (do not borrow KeyCape's reader), deployed reserved +LDAP group denial, resolver/cutover input provenance and functional proof, and +bootstrap/root-output/isolation callers. Fixtures, an existing tty and historical +bootstrap receipts are not current authority. Propose exact write/cutover actions +before execution; record deferred actions separately rather than faking a pass. -2026-10-15 continuation: expanded the source inventory to all 87 current -non-test shell/Python callers under bootstrap/k8s plus Make, console, delivery -copier and the recovery transport checker. Reconciled the attended inventory; -source enumeration is not native acceptance. Six new regressions failed against -the old bundle/delivery/preview paths, then passed after repair. Bundle packing -now streams directly to age, refuses links/private age keys/non-private inputs, -publishes a private new output without overwriting an existing artifact, and -creates no plaintext tar intermediate. Bootstrap no longer erases retained -service custody to regenerate or continues through failed OpenBao verification. +## Resolve historical credential and configuration-preserving rotation dispositions -The lab OpenBao adapter now requires an explicit lab profile, admitted kubeconfig -and exact cluster UID, contains child diagnostics, retains private no-overwrite -init output and sends shares only on stdin. Production and automatic post-init -root configuration are refused. Existing model selection is not authorization; -new transport tests use only a fake cluster. The historical greenfield receipt -does not prove this revised adapter was exercised natively. +```task +id: NK-WP-0049-T22 +status: wait +priority: high +owner: codex +depends_on: [NK-WP-0049-T02, NK-WP-0049-T21] +blocking_reason: "Await the caller/consumer ledger, current replacement-lane acceptance and explicit credential-owner dispositions; existing exposure-rotation deferral is unchanged." +``` -Retired the unsafe User Engine shared-token copier, root-token pod prompt, -unguarded legacy registry patchers/raw Python registry output and stale managed -workload rollback. Current package/provider procedures remain authoritative. -Presence diagnostics now check the scoped factor reader and preserve historical -applied/native state. Dependency/client diagnostics suppress decoded-config -tracing and provider response dumps. Withdrew old runbook recipes that generated -replacement PI keys, wrote predictable plaintext backups, used a HOME private key -for public metadata, encrypted only one export, or piped a pod dump into local -SQLite. No existing credential/Secret/workload was rotated, deleted or reapplied. +Owner boundary: Platform/custody and affected consumer owners. Review historical +personal-admin/trigger credential copies and unsafe predecessor paths against +known current consumers and rollback custody. -Read-only primary metadata: all six identity-related Deployments are Ready; -local backup jobs ran October 15. Personal-token Secret `sso/keycape-pi-token` -and `mfa/privacyidea-trigger-admin` still exist; this is not proof of validity -or provider revocation. Current job commands target PVCs, and the PI job does not -mention the audit pair; off-host coverage must be established from actual -artifacts. Founder identified Scaleway and Storage Share as backup destinations; -exact artifact/manifest and independently held matching-key availability remain -pending. No recovery key or backup content was retrieved for this review. +**Done when:** every affected credential has an owner-accepted disposition: +verified revocation/replacement, continued bounded use with acceptance, or an +explicit linked deferral. Signing/client-secret changes require preservation and +native multi-consumer proof if actually approved. CUST-WP-0073-T05's founder +rotation deferral and decision `1591daa1-6e85-4dfd-aadd-5a9fe1ef47fa` remain in force; +record their exact applicability rather than automatically rotating or treating +source retirement as provider revocation. -Prepared `tools/bootstrap-recovery/verify-bundle.py` and -`docs/bootstrap-recovery-ceremony.md`: digest-bound ciphertext, explicit private -key/registered recipient, strict file coverage and archive checks, unrelated -wrong-key refusal, isolated file restore/readback and cleanup, with a closed -non-secret receipt. Nine actual age/archive tests use disposable material. -A successful transport check explicitly does not prove independent custody, -current credential validity or native service recovery. This is preparation for -the owner ceremony, not a completed production restore. +## Provide retrieval that works when OpenBao and SSO are unavailable -T02 remains progress for current owner/runtime acceptance, legacy provider -credential disposition and retained coordinated-rotation requirements. The -founder-deferred exposure rotation stays CUST-WP-0073-T05; this request does not -silently reopen it. T03 remains wait for the exact off-host artifact/key lineage, -reviewed isolated target/procedure and actual owner/native acceptance. Evidence: -`docs/evidence/2026-10-15-bootstrap-recovery/`; full verification results are -recorded there. No workplan completion is asserted. +```task +id: NK-WP-0049-T23 +status: todo +priority: high +owner: codex +depends_on: [NK-WP-0049-T13] +``` -2026-10-15 off-host discovery follow-through: operator reports holding the -`railiance-backup` age key independently; no private key was received/read. -The read-only Storage Share inventory completed through the existing attended -Platform Backup-account custody procedure; Warden exited 0, confirming session -self-revocation/cleanup. The inspected account has a current privacyIDEA database -ciphertext (`r01-net-kingdom-pg/net-kingdom-pg-privacyidea_db-20261015T023000Z.dump.age`, -89,129,309 bytes) and the October14 OpenBao snapshot. The inspected folders do -not contain a bootstrap/identity-file bundle. Bounded Scaleway metadata inventory -under the existing scoped backup delivery also completed; it found database -base/WAL trees and an application archive subtree, with no identity-file bundle -in the inspected scope. No bootstrap API key, age key or object content was read -by these inventories; other accounts/buckets/prefixes remain unassessed. +The existing independent retrieval succeeded through a healthy attended OpenBao +path. A full custody outage still lacks a reviewed offline provider adapter. -Thirteen fixture checks pass for contained caller scope, metadata/path bounds, -pagination and malformed/foreign responses. Requested the human's hidden-terminal -key check against the backup owner's declared public recipient. Matching that -recipient remains distinct from actual decryption and full recovery; the -bootstrap `keys/age.pub` is not assumed to be the same recipient. T02/T03 remain -open. Evidence: `offsite-discovery-summary.json` and the two exact inventory -receipts in the existing October15 evidence folder. +**Next action:** prepare bounded read-only download/receipt tooling using +independently held provider access in private operator input, with fixed provider, +artifact/ETag/size/digest guards, sanitized output and no standing agent secret. -2026-10-16 custody/retrieval checkpoint: operator reports the hidden check -matched the backup recipient and removed the temporary key. No private key was -provided to the agent. The exact existing PI database ciphertext was fetched -through the attended Platform Backup-account procedure, with strong ETag binding, -89,129,309 bytes and SHA256 -`cc81da442755928171d2c51bce93eda7de4392a0a1aa9d0ffb83a25ba34e5ead`. -The session exited0 and self-revoked. Ciphertext is private temporary storage; -no plaintext or age key was read by the fetch. Prepared the exact human custody -command in the recovery ceremony for an isolated, pinned PostgreSQL restore -with no network, ports or host mounts; fixed aggregate readback and verified -container/plaintext cleanup.23 focused tests passed including actual local -PostgreSQL restore using synthetic data and a fresh fixture age key. Actual -backup decryption remains pending the operator command. This is partial database -recovery preparation; full matching identity key/config coverage, service -continuity, T02 owner acceptance and legacy credential disposition remain open. -Evidence: `privacyidea-transfer-r1.json`, `database-drill-preparation.json`. +**Done when:** the owner can retrieve the exact off-host bundle without relying +on the failed OpenBao/SSO instance, run the existing hidden-key checker, and +verify refusal/cleanup. Use the credential-owner route before obtaining inputs; +a new provider grant or broader scope needs its own approval. Source fixtures +alone do not close the actual outage-path proof. -2026-10-16 actual recovery and next coverage: the operator ran the prepared -hidden-key command successfully. Reviewed the actual receipt against exact -source/image/transfer digest and rehashed retained ciphertext. Original off-host -PI ciphertext decrypted, unrelated wrong key refused, native PostgreSQL restore -and aggregate readback passed (5 tokens, 2 resolvers, 2 realms), container and -plaintext cleanup passed; no NK49 Docker containers remain. The agent received -no private key or restored values. Full identity/token decryption/audit continuity -remain unproven; this is actual database recovery, not full service recovery. +## Record owner disposition for second-provider and external monitoring gaps -The operator confirmed no known full identity bundle and requested preparation -of missing coverage (decision `ed20023f-0168-421d-be86-460391339876`). Prepared -`identity-backup-profile.json` and a metadata-only preparer. Current exact four -Deployment and five Bound PVC identities and all Secret/ConfigMap references -were checked on the primary cluster without Secret-value reads or pod exec. -Fourteen source tests pass for drift/unreviewed dependencies/private-data scope. -The profile specifies consistent data snapshots, original key/config capture, -credential recovery/reissue disposition, proposed primary/secondary archive -paths, per-generation lineage, immutable readback, measured size/quota, eventual -schedule/retention and native service acceptance. It is `prepared_not_admitted`: -new capture/sealing/transfer adapters and scoped publisher owner admission remain -necessary. No capture/upload/grant/schedule/rotation occurred in preparation. -T02 remains progress and T03 wait. Evidence: actual database drill/review and -`identity-backup-preparation-r1.json` under the October15 recovery evidence. +```task +id: NK-WP-0049-T24 +status: todo +priority: medium +owner: codex +depends_on: [NK-WP-0049-T14] +``` -2026-10-16 implementation and bootstrap/discovery checkpoint: implemented the -complete new-profile capture/seal, unique Storage Share publication/readback, -independent owner retrieval, attended archive/native database verifier, separate -retention and source-bound node installer. Prepared exact CCR-2026-0048/0049 -for folder-only publisher(read/create) and retention(read/delete), exact ESO -readers and native positive/negative admission checks. Both CCRs validate with -no errors; claim confirmation and all owner approvals remain pending. Decision -`40bfae40-6108-4e51-8613-073339f5345f` is pending exact live-packet admission. +**Next action:** record the current single-provider and host-local-health limits, +then obtain explicit owner scope decisions for a second encrypted provider copy, +external host-loss/freshness detection and recurring recovery-drill cadence. -51 focused checks passed including actual synthetic age/full archive and native -PostgreSQL restoration, SQLite WAL snapshots, hostile archive/helper boundaries, -provider permission/retention gates and bootstrap dispatch. Live cluster -server-dry-run accepted ten new helper/delivery manifests plus both guarded DNS -selector patches, and systemd unit validation passed. No live secret capture, -new grant, upload, policy change, node installation or timer enable occurred. -Initial provider is Storage Share; the earlier proposed Scaleway identity prefix -is deferred. Runtime quota/headroom is checked; host-local health cannot detect -host loss. Full native identity/token/audit/caller recovery remains open. +**Done when:** each gap has either an accepted bounded residual risk or a linked +live intake/workplan with owner and trigger. Existing daily RPO/25-hour health +settings remain the approved baseline. Scaleway full-identity publication and +an external watchdog are not implemented; do not add their grants/jobs implicitly. +This task triages the new insights instead of silently expanding NK49's required +implementation to an entire disaster-recovery programme. -At the operator's request reviewed railiance-bootstrap scope and added its -`identity-recovery` dispatcher and documented independent-custody → isolated -verification → S1/S2/S3 replacement → native identity acceptance handoff. This is -not an automated production importer; total-loss offline provider retrieval is -still an explicit gap. Updated the existing owner capability -`capability.security.bootstrap-recovery`, related it to bootstrap sequencing and -validated both registries; reuse-surface federation already includes both owner -indexes. No maturity promotion implies live acceptance. Reviewable rollout and -hidden-key steps: `docs/bootstrap-recovery-ceremony.md`; evidence in -`docs/evidence/2026-10-16-identity-backup-implementation/`. T02 progress/T03 wait. - -2026-10-16 approved live admission in progress: user approved the exact packet -(decision40bfae40) and then explicitly approved Nextcloud-added re-sharing for -publisher21/retention25 (decision5edd5203). The initial5/9 request failed closed -before KV publication when provider returned21. Scoped share26 exists under -Backup's exact new folder; it remains undelivered until native checks pass. -Owner sessions with executed children were self-revoked; a later callback timeout -ran no child and did not confirm possible-session revocation. No root fallback. - -Fixed runtime guard to use railiance01's actual verified hostname, machine-ID -hash and cluster UID, with root required.52 focused checks including actual -synthetic encrypted archive/native PG restore passed. Both guarded DNS helper -exclusions applied with UID/RV/spec tests. Installed the source-bound runtime -with unit validation; timers remain disabled pending actual off-site recovery. -Provider boundary testing and ESO/native login delivery remain in progress. -No identity capture or scheduled retention has occurred. T02 progress/T03 wait. -Evidence: `docs/evidence/2026-10-16-identity-backup-live/`. - -2026-10-16 native publisher checkpoint: actual fixture read/create worked and -publisher overwrite/delete were denied. Both exact fixture objects were cleaned; -the remaining parent-path check returned400 for a malformed path. Corrected the -checker to distinguish that rejection from a separate valid outside-folder -request that must fail authentication.16 provider checks pass after correction. -The next attended login timed out before handoff, so corrected native checks, -KV/ESO delivery and first full capture remain pending the human identity act. -Fresh capture metadata reports zero helper allow policies. Installed timers are -disabled. No new coding-agent credential or root fallback; partial owner share26 -is undistributed. Pending browser status from operator, without duplicate logins. - -2026-10-16 first actual full identity backup: provider/reader admission completed. -Both exact native roles have one policy/900-second TTL and deny sibling, -metadata, old escrow and token-create; wrong-SA/namespace/audience negative cases -passed, ESO values matched KV version1 and temporary readers/session were revoked. -Fixed strict ESO comparison to accept only its known no-transformation defaults. -56 focused checks passed, including real synthetic encrypted/native PG recovery. -The disabled runtime was updated through exact previous-unit/source guards. - -First full original-key/current-data capture and Storage Share publication plus -fresh downloaded hash readback succeeded: artifact -`net-kingdom-identity-20261016T063159Z-5882771aa2ce.tar.age`,109738344bytes, -SHA256`7117b3fe1e4840c86b301c94c85e6b16709e5626bb0884bd0580bde630e30e30`. -Helper and plaintext cleanup verified. No age private key was used by the agent. -Independent owner retrieval/human-key recovery is next; both timers remain -disabled. Full native service/token/audit/caller and legacy credential -acceptance remain open; T02 progress/T03 wait. - -2026-10-16 full archive proof and backup activation complete: independently -retrieved the exact 109738344-byte ciphertext through attended Backup custody and -rehashed it. The operator ran the hidden-key command via railiance-bootstrap's -identity-recovery handoff. Reviewed the actual receipt against exact checker -source/artifact: recipient match, wrong-key denial, all 36 manifest files/hashes, -original key pairs, SQLite integrity and native disposable PostgreSQL restore -passed (5 tokens/2 resolvers/2 realms). Plaintext/container cleanup verified; no NK49 -Docker containers remain. The agent received no age key or restored values. - -Activated only after this actual proof, under the already approved packet and -root-private artifact-bound accepted receipt. Daily backup/retention and hourly -health timers are enabled/active; actual health service succeeded. Next observed -backup 03:47:39 UTC Oct17; health 08:00 UTC Oct16. Live retention preview protects the -recovered generation, candidates/deletions empty; no failure marker/outbox. -Backup coverage setup is complete, but whole NK49 is not: full native directory/ -issuer/MFA, retained token decryption, audit continuity, current callers and T02 -legacy credential dispositions remain open. T02 progress/T03 wait retained. -Evidence: actual archive recovery/review, independent transfer and activation -receipts in `docs/evidence/2026-10-16-identity-backup-live/`. - -2026-10-16 final recurring-job acceptance: the actual installed hardened systemd -service passed all three phases (retention → capture/upload/readback → retention), -Result=success/ExecMainStatus=0. Second actual generation -`net-kingdom-identity-20261016T073927Z-1224cb9362f8.tar.age` verified off-host. -Retention kept the independently recovered first point and newest point, with -no candidates/deletions/local pruning; helper/plaintext cleanup passed, failure -markers and outbox empty. `recurring-job-review.json` records this final native -path acceptance. Backup/recovery setup is complete; wider NK49 closure remains -as stated in T02/T03, without falsely marking full service recovery complete. - -## Verify owner-approved recovery and current native callers +## Accept remaining recovery and caller evidence and close the G08 handoff ```task id: NK-WP-0049-T03 status: wait priority: high -depends_on: [NK-WP-0049-T02] -blocking_reason: "Actual full off-host archive recovery passed with the independently held key, complete manifest, original key pairs, SQLite integrity and disposable native PG restore; cleanup verified. Daily backup/retention and hourly health timers are active with the recovered generation pinned. Full native directory/issuer/MFA authentication, retained token decryption, audit continuity and current/T02 legacy caller acceptance remain open. No root fallback or deferred exposure rotation is authorized." +owner: codex +depends_on: [NK-WP-0049-T02, NK-WP-0049-T16, NK-WP-0049-T17, NK-WP-0049-T18, NK-WP-0049-T19, NK-WP-0049-T20, NK-WP-0049-T21, NK-WP-0049-T22, NK-WP-0049-T23, NK-WP-0049-T24] +blocking_reason: "Await the separately tracked native service proofs, current/legacy caller acceptance and independent outage retrieval; full archive recovery and backup activation are already complete." state_hub_task_id: "d68cd4d3-1ba3-530e-88f9-26ebb8dd6eac" ``` -Review the final scoped procedure, current consumer/caller usage and independent -key/snapshot availability. Under an exact approved ceremony, verify off-host -ciphertext retrieval, matching independently held key, isolated decryption and -required current service-file coverage, failed-key refusal and cleanup. Record -only safe identities/digests/statuses. Do not run the whole live bootstrap merely -to test this patch, or infer current credential validity from a historical bundle. -Historical disclosure is separate incident/rotation work; terminal clearing and -source repair do not revoke any earlier exposed value. G08 stays open until -caller coverage, owner review and native recovery evidence are adequate. +Owner boundary: NetKingdom, Platform and affected service owners, with Custodian +G08 receiving the final evidence. The completed archive/job milestones remain +accepted independently of this final gate. + +**Done when:** T02 and T15–T24 have their stated completed or explicitly accepted +handoff outcomes; service authentication/token/audit proof and current/legacy +caller dispositions are accepted; the source workplan and projected tasks agree; +and G08 has an exact evidence/handoff record. Record actionable residuals as live +work records before workplan closure. Preserve historical IDs and receipts; do +not call a deferred rotation or an untested full-estate restore complete. + +## Evidence locations and refinement history + +“Live” above means `docs/evidence/2026-10-16-identity-backup-live/` in NetKingdom. +Implementation preparation is in the October16 implementation directory; earlier +backup discovery/database recovery is in `docs/evidence/2026-10-15-bootstrap-recovery/`. +The source-repair receipts explicitly labelled Custodian belong to that owning +repository. Use the [recovery ceremony](../docs/bootstrap-recovery-ceremony.md) +for exact procedure and approval history. No secret values belong in task evidence. + +Refinement rule: substantial work and newly understood remaining scope must be +visible in task blocks at a checkpoint, not only in progress prose. This replan +preserves T01–T03 identity and original evidence while making the completed +milestones and each remaining acceptance gate separately manageable.