Prepare independent outage recovery and Storage Share documentation

Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a13eab-1e6a-7b33-9c37-372e90ada3a2
This commit is contained in:
tegwick 2026-10-16 17:59:18 +02:00
parent 2a3800e7f1
commit 1764f40b43
12 changed files with 1050 additions and 22 deletions

View file

@ -28,10 +28,11 @@ These are retrospective task records of evidenced work, not additional delivery
or newly granted authority. The earlier dated narrative remains in Git at
`cc2dd01`; its evidence references are carried into the tasks below.
**Current position:** 12 tasks done; 1 in progress; 3 ready to start (`todo`);
8 waiting on explicit prerequisites. These are task counts, not an effort-based
**Current position:** 14 tasks done; 2 in progress; 1 ready to start (`todo`);
10 waiting on explicit prerequisites. These are task counts, not an effort-based
completion percentage. Assignees are also written in task descriptions because
the current Forge projector does not populate the dedicated Hub assignee field. No remaining task is being executed by this replan.
the current Forge projector does not populate the dedicated Hub assignee field. The outage-retrieval tool and restore-packet draft are now prepared;
owner execution/publication and service admission remain explicit gates.
**Next actions:** finish the read-only caller ledger (T02); prepare the isolated
restore packet (T15), independent offline retrieval (T23), and residual-scope
@ -369,8 +370,8 @@ state_hub_task_id: "479d6970-6516-50f5-9436-bb5b56d58aad"
the named owner boundary.
The source inventory and known legacy objects exist; a complete current-use
ledger is not yet accepted. Next action is read-only reconciliation of the 87
source entries against active Make/bootstrap/operator routes and owner evidence.
ledger is not yet accepted. Next action is read-only reconciliation of the 93
entries in the latest October15 source inventory (the earlier snapshot had 87) against active Make/bootstrap/operator routes and owner evidence.
**Done when:** each entry is identified as current, retired or owner-deferred,
with its owner, credential route, current consumer(s), native evidence or exact
@ -385,21 +386,26 @@ T22. Do not reopen completed source repairs without a new finding.
```task
id: NK-WP-0049-T15
status: todo
status: progress
priority: high
assignee: codex
depends_on: [NK-WP-0049-T08, NK-WP-0049-T13]
depends_on: [NK-WP-0049-T08, NK-WP-0049-T13, NK-WP-0049-T27]
state_hub_task_id: "1d1eb903-a4b0-5bc4-a421-0b842d7f68ae"
```
**Accountable worker:** codex. Service/credential-owner acceptance remains with
the named owner boundary.
**Next action:** turn the recovered manifest and current caller gaps into a
reviewable procedure with selected immutable images, restore order, isolated
endpoints/network, data/key ownership, cleanup and per-service positive/negative
checks. Preparation can start with the existing archive; incorporate T02's ledger
before submitting current-caller acceptance.
**Prepared draft scope moved to T27.** See
`docs/identity-service-restore-packet.md`: restore order, containment, safe evidence
and per-service positive/negative checks are drafted. Read-only current runtime
image digests were observed for all four services; correspondence to the selected
archive and independent image availability are not yet proved.
**Next action:** admit a named disposable target/private route, verify and stage
matching images independently, obtain private existing-user/MFA test inputs and
review version-specific native commands with service owners. Incorporate T02's
ledger before submitting current-caller acceptance.
**Done when:** service owners accept the exact targets/procedure, existing-user
and MFA test identities/input custody are available without posting secrets,
@ -585,22 +591,29 @@ source retirement as provider revocation.
```task
id: NK-WP-0049-T23
status: todo
status: wait
priority: high
assignee: codex
depends_on: [NK-WP-0049-T13]
depends_on: [NK-WP-0049-T13, NK-WP-0049-T25]
blocking_reason: "Await the operator-run private independent-provider download and hidden-key recovery drill; bounded tooling and self-contained kit are prepared in T25."
state_hub_task_id: "1ec537ee-dfee-5576-be36-a81bdba417d1"
```
**Accountable worker:** codex. Service/credential-owner acceptance remains with
the named owner boundary.
The existing independent retrieval succeeded through a healthy attended OpenBao
path. A full custody outage still lacks a reviewed offline provider adapter.
The previous independent retrieval succeeded through a healthy attended OpenBao
path. Source/fixture preparation of a separate offline adapter is now complete
in T25. Actual independently held provider access has not yet been exercised
for this outage path; no completed drill is claimed.
**Next action:** prepare bounded read-only download/receipt tooling using
independently held provider access in private operator input, with fixed provider,
artifact/ETag/size/digest guards, sanitized output and no standing agent secret.
**Unblock action:** operator runs the prepared private terminal handoff
`bash tools/bootstrap-recovery/run-offline-recovery.sh` (combined guide
publication and known-generation retrieval) or the standalone read-only
`retrieve-identity-offline.py` command in the guide. Independently held provider
access and age key go only into the respective hidden prompts; inspect safe
receipts and perform the real archive recovery check. Never use a captured agent
PTY for secret entry. T26 separately tracks actual guide publication.
**Done when:** the owner can retrieve the exact off-host bundle without relying
on the failed OpenBao/SSO instance, run the existing hidden-key checker, and
@ -633,6 +646,91 @@ an external watchdog are not implemented; do not add their grants/jobs implicitl
This task triages the new insights instead of silently expanding NK49's required
implementation to an entire disaster-recovery programme.
## Deliver and verify the offline retrieval adapter and recovery tools kit
```task
id: NK-WP-0049-T25
status: done
priority: high
assignee: codex
depends_on: [NK-WP-0049-T13]
```
**Accountable worker:** codex.
Completed 2026-10-16. `retrieve-identity-offline.py` uses privately entered
independent Backup-account access or an exact read-capable identity-folder share.
It performs only fixed-provider HEAD/GET, binds the reviewed generation by strong
ETag/length/SHA256, refuses redirects/proxies/unreviewed scope and removes failed
downloads. No OpenBao/SSO, age key or standing agent credential is required.
`recovery-kit.py` builds a reproducible allowlisted ZIP containing the tools,
public profile, known-good ciphertext receipt and recovery guides, with file
checksums. Its publication path creates only three versioned guide/ZIP/checksum
objects and verifies readback without overwriting existing conflicting objects.
`run-offline-recovery.sh` provides the owner's two-hidden-prompt handoff.
Evidence: `tests/test_identity_offline_recovery.py` and the bounded test/build
receipt in `docs/evidence/2026-10-16-identity-outage-recovery/`. The focused run
passed 65 tests including real disposable age/PostgreSQL fixtures. ZIP-contained
entry points import without sibling repositories. This is source/fixture proof,
not actual independent-provider retrieval (T23) or publication (T26).
## Publish and verify the disaster recovery guide on Storage Share
```task
id: NK-WP-0049-T26
status: wait
priority: high
assignee: codex
depends_on: [NK-WP-0049-T25, NK-WP-0049-T27]
blocking_reason: "Await private operator provider-access execution of the prepared publication command and actual three-object readback receipts; guide and kit are built."
```
**Accountable worker:** codex; operator holds provider access privately.
Requested by the operator on 2026-10-16. Guide source:
`docs/disaster-recovery-storage-share.md`. Document scope, independently held
provider/key requirements, exact known-good fallback, copy/download verification,
archive-check commands, service restore order, recurring schedule and remaining
coverage limits. Include the offline tools so failed Forgejo does not prevent
access to the recovery scripts; retain an independent copy and trusted digest.
**Unblock action:** operator runs `run-offline-recovery.sh`, or the reviewed
`recovery-kit.py --publish` command through the existing attended Backup-account
owner conduit. Existing publisher/retention workload grants are not repurposed.
The user authorized this documentation upload; no new grant is required by the
prepared command. Credentials remain private; no provider permissions are widened.
**Done when:** the versioned START-HERE guide, self-contained ZIP and ZIP SHA256
file are present beside the identity backups with exact conditional-create and
ETag/length/content readback proof, and their folder/name/digests are recorded for
future discovery. Local build or attempted upload alone does not close this task.
## Draft the isolated service restore packet and observe candidate images
```task
id: NK-WP-0049-T27
status: done
priority: high
assignee: codex
depends_on: [NK-WP-0049-T08, NK-WP-0049-T13]
```
**Accountable worker:** codex.
Completed 2026-10-16. Draft in `docs/identity-service-restore-packet.md` names the
known recovered input, restore order, original key/data ownership, default-deny
containment, blocked production/provider/notification effects, cleanup and five
native acceptance lanes with positive/negative checks. Read-only metadata from
the pinned primary cluster resolved all four current service image digests,
including the separately observed privacyIDEA main container and LLDAP's mutable
source tag. Evidence: `docs/evidence/2026-10-16-identity-outage-recovery/image-observation.json`.
This task extracts completed preparation from T15. Named target admission,
archive/runtime image-lineage matching and independent image availability,
private existing-user/MFA inputs and owner-reviewed native commands remain T15;
no service restore, production change or owner acceptance is claimed.
## Accept remaining recovery and caller evidence and close the G08 handoff
```task
@ -640,7 +738,7 @@ id: NK-WP-0049-T03
status: wait
priority: high
assignee: codex
depends_on: [NK-WP-0049-T02, NK-WP-0049-T16, NK-WP-0049-T17, NK-WP-0049-T18, NK-WP-0049-T19, NK-WP-0049-T20, NK-WP-0049-T21, NK-WP-0049-T22, NK-WP-0049-T23, NK-WP-0049-T24]
depends_on: [NK-WP-0049-T02, NK-WP-0049-T16, NK-WP-0049-T17, NK-WP-0049-T18, NK-WP-0049-T19, NK-WP-0049-T20, NK-WP-0049-T21, NK-WP-0049-T22, NK-WP-0049-T23, NK-WP-0049-T24, NK-WP-0049-T26]
blocking_reason: "Await the separately tracked native service proofs, current/legacy caller acceptance and independent outage retrieval; full archive recovery and backup activation are already complete."
state_hub_task_id: "d68cd4d3-1ba3-530e-88f9-26ebb8dd6eac"
```
@ -652,7 +750,7 @@ Owner boundary: NetKingdom, Platform and affected service owners, with Custodian
G08 receiving the final evidence. The completed archive/job milestones remain
accepted independently of this final gate.
**Done when:** T02 and T15–T24 have their stated completed or explicitly accepted
**Done when:** T02 and T15–T27 have their stated completed or explicitly accepted
handoff outcomes; service authentication/token/audit proof and current/legacy
caller dispositions are accepted; the source workplan and projected tasks agree;
and G08 has an exact evidence/handoff record. Record actionable residuals as live