Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a13eab-1e6a-7b33-9c37-372e90ada3a2
422 lines
29 KiB
Markdown
422 lines
29 KiB
Markdown
# NK-WP-0049 recovery ceremony and closure packet
|
|
|
|
Status: actual off-host privacyIDEA database drill passed; full identity backup
|
|
coverage remains in preparation. This document grants no credential export,
|
|
provider access, revocation, rotation, root retrieval or production bootstrap.
|
|
|
|
## What has been established
|
|
|
|
On 2026-10-15 the operator identified Scaleway and Storage Share as the off-host
|
|
backup providers. This identifies destinations, not the exact recovery artifact,
|
|
its freshness, its file coverage or independently held decryption key. Platform's
|
|
current backup contracts distinguish Scaleway database/full archives from the
|
|
Nextcloud secondary copy. An upload-only share is not a retrieval identity.
|
|
|
|
Read-only primary-cluster metadata is retained under
|
|
`docs/evidence/2026-10-15-bootstrap-recovery/`. LLDAP, Authelia, KeyCape,
|
|
privacyIDEA, identity-provisioner and User Engine each have one Ready replica.
|
|
The three SQLite/key backup CronJobs report successful October 15 invocations.
|
|
Their current commands target local PVCs; neither an external export nor complete
|
|
off-host key/database coverage is established by those observations. The
|
|
privacyIDEA command mentions its encryption key, but not its audit key pair.
|
|
This command inspection is a coverage question, not proof that no other backup
|
|
exists. The personal-token and trigger-admin Secrets still exist. Neither their
|
|
validity nor provider-side revocation has been tested. A scan of 34 current
|
|
Deployment/StatefulSet/DaemonSet/CronJob/Pod resources in sso, mfa and user-engine
|
|
found no references to those two old Secrets; it still found the expected audit
|
|
delivery reference in User Engine. Other namespaces, historic jobs, external
|
|
consumers and issued provider credentials are outside that scan.
|
|
|
|
## Assemble the exact owner packet before execution
|
|
|
|
Record only safe references and checksums. Keep credentials in their current
|
|
owner custody and pass the decryption key through attended temporary unlock.
|
|
|
|
| Required owner input | Acceptance |
|
|
| --- | --- |
|
|
| Off-host artifact | Provider, account/folder or object/version, backup timestamp, ciphertext SHA256 and independent retrieval receipt; do not substitute a new local export for the existing off-host backup. |
|
|
| Independent key | Human confirms matching custody outside the servers and OpenBao. A successful decrypt using only the OpenBao escrow key does not prove independent recovery. Never send the key in chat. |
|
|
| Required assets | Exact manifest of matching service env/config files, LLDAP and Authelia data, privacyIDEA database plus original encryption/audit keys, and KeyCape registry/signing material. Resolve assets held through other current owner lanes explicitly. |
|
|
| Recovery environment | Named disposable target, pinned images available independently of the failed production registry, no production ingress, no production identity/client execution, bounded egress, private storage and exact cleanup scope. |
|
|
| Native proof | Owner-specific database integrity, identity/group counts, decryption of retained token material, audit signature continuity, issuer/client/signing configuration preservation and admitted login/MFA checks. Distinguish an isolated restore from current production credential validity. |
|
|
| Current consumer source | Reconcile the deployed revisions with the repaired caller/provider source. The October 14 reserved platform-group fix must be deployed and independently checked before claiming live enforcement. |
|
|
| Legacy credential disposition | IAM owner identifies the exact historical personal-token/trigger principals, actual consumers and revocation method before any deletion. Deleting a Secret does not revoke an issued provider credential. |
|
|
|
|
The operator supplied the independently held `railiance-backup` age key only to
|
|
the attended local custody helper. The actual off-host PI ciphertext decrypted
|
|
and restored successfully; the agent received no private key material. The full
|
|
identity asset manifest remains unfulfilled. The backup lane recipient and
|
|
bootstrap `keys/age.pub` must not be assumed to be the same.
|
|
Do not draft a blanket restore or grant approval while full coverage is unknown.
|
|
Once fixed, bind the source revisions, target identities, allowed operations,
|
|
recovery/cleanup sequence and maintenance window to the reviewed live packet.
|
|
Existing eligibility or another client's approval is not this ceremony's grant.
|
|
|
|
## Execute in stages
|
|
|
|
1. Use the existing Platform/provider recovery lane to retrieve the identified
|
|
ciphertext, verify its object/version and digest, and retain a safe transfer
|
|
receipt. Do not request or expose the provider credential to the agent.
|
|
2. The human unlocks the independently held age key in the existing custody
|
|
helper. Validate it against the registered public recipient. No permanent
|
|
key is installed in HOME, generated as a replacement, or copied into a bundle.
|
|
3. Run the local transport/coverage check below. It contacts no cluster, keeps
|
|
plaintext in a private temporary directory outside Git, refuses unsafe
|
|
archives/private age keys, tests an unrelated disposable wrong key, reads the
|
|
restored files back and removes that staging. It emits a closed non-secret
|
|
receipt. This is one prerequisite, not whole-service recovery acceptance.
|
|
4. Under the exact separately reviewed owner ceremony, restore the matching
|
|
service data/key/config sets into the disposable environment and perform the
|
|
native checks above. Never run the full production bootstrap to test source
|
|
repairs. Unknown writes stop for reconciliation rather than automatic retry.
|
|
5. Verify cleanup, preserve the original ciphertext and independent custody,
|
|
and reconcile current callers/legacy credentials through their owners.
|
|
Retain exact source/object/image/actor/time and positive/negative receipts.
|
|
6. Close T02/T03 only after the recorded remaining requirements are satisfied.
|
|
Update the workplan first, run `statehub fix-consistency --repo net-kingdom`,
|
|
and verify the Hub projection. A transport fixture or generated evidence
|
|
worksheet cannot mark this plan finished.
|
|
|
|
## Prepared local transport check
|
|
|
|
The human invokes this through the existing attended custody helper after the
|
|
exact artifact and required-file list are agreed. `NETKINGDOM_CUSTODY_AGE_KEY_FILE`
|
|
is the temporary private file provided inside that envelope. The tool never
|
|
reads an ambient HOME key. The receipt destination must be new.
|
|
|
|
```bash
|
|
python3 tools/bootstrap-recovery/verify-bundle.py \
|
|
--bundle /private/recovery/fetched-ops-bundle.tar.age \
|
|
--bundle-sha256 <independently-recorded-ciphertext-sha256> \
|
|
--required-file lldap/secrets.env \
|
|
--required-file privacyidea/secrets.env \
|
|
--required-file privacyidea/pi.enc \
|
|
--required-file privacyidea/private.pem \
|
|
--required-file privacyidea/public.pem \
|
|
--receipt /private/recovery/new-transport-receipt.json
|
|
```
|
|
|
|
This example file list is not an accepted whole-stack manifest. Add every
|
|
agreed service file supported by the closed transport profile; database copies
|
|
and other archive formats use their native owner restore tools. A successful
|
|
receipt explicitly leaves independent custody, current service validity and
|
|
native consumer recovery unproven. Off-host retrieval lineage and human custody
|
|
confirmation must accompany it; the tool cannot infer where a local input came
|
|
from. A failed check publishes no acceptance receipt.
|
|
|
|
## Rotation boundaries
|
|
|
|
CUST-WP-0073-T05 remains founder-deferred under decision
|
|
`1591daa1-6e85-4dfd-aadd-5a9fe1ef47fa`. This work does not reopen that rotation,
|
|
replace signing keys, change directory passwords or erase old backup keys.
|
|
Existing historical exposure, provider revocation and coordinated rotation
|
|
requirements stay explicit in their live owner records. If completion needs a
|
|
protected native operation, prepare that exact operation and obtain its scoped
|
|
disposition; never treat a source repair or successful backup job as permission.
|
|
|
|
## Knowledge/source scope
|
|
|
|
Emergent Security 0.2.6, package digest
|
|
`8d1d6a9d4d07fec7b619618de7dfe2be2254c1790f85f4198c7fe19db9751c02`:
|
|
start/root cards withheld, orientation only. Current SECURITY-COMPANION,
|
|
Platform backup/custody procedures, owner runtime contracts and the workplan
|
|
were reviewed as sources. No execution authority is derived from the cards.
|
|
|
|
|
|
## Off-host inventory return — 2026-10-15
|
|
|
|
Both bounded metadata inventories completed. Storage Share's inspected
|
|
`Backup/railiance-backups` scope includes the current privacyIDEA database
|
|
ciphertext and the October14 OpenBao snapshot. It did not reveal a matching
|
|
bootstrap/identity-file bundle in the inspected folders. Scaleway's inspected
|
|
`railiance-platform-pg-backup/platform-pg/` tree contains database backup trees
|
|
and an application archive subtree, with no identity-file bundle in this bounded
|
|
scope. Other accounts/buckets/prefixes are unassessed. See
|
|
[the discovery summary](evidence/2026-10-15-bootstrap-recovery/offsite-discovery-summary.json).
|
|
No object contents or age keys were read; the operator envelope exited0 and
|
|
self-revoked. Existing scoped backup credentials were reused without changes;
|
|
the Scaleway bootstrap API key was not used.
|
|
|
|
The operator subsequently reported the hidden-terminal check passed and the
|
|
temporary key file was removed. The later actual database drill also passed;
|
|
the private key was not provided to the agent.
|
|
|
|
## Prepared bounded privacyIDEA database drill — 2026-10-16
|
|
|
|
The identified existing off-host ciphertext was downloaded through the same
|
|
attended Platform Backup-account procedure. The before/GET/after strong-ETag
|
|
binding and exact byte count passed. Ciphertext SHA256:
|
|
`cc81da442755928171d2c51bce93eda7de4392a0a1aa9d0ffb83a25ba34e5ead`.
|
|
See `privacyidea-transfer-r1.json` in the October15 evidence folder. The owner
|
|
envelope exited0 and self-revoked; no age key or plaintext was read by the fetch.
|
|
Ciphertext is retained in the fresh private `/tmp/nk49-offhost-j9dzl0ge/` directory.
|
|
|
|
The following exact human command performs only an isolated local database
|
|
drill. It checks the transfer receipt/digest before opening the temporary key,
|
|
checks the backup recipient, refuses an unrelated wrong key, decrypts to private
|
|
temporary storage and restores into disposable PostgreSQL. Image
|
|
`postgres@sha256:ff23cdce56cac62ada6f66013e1a50864c0abbe79d132d40b6e05bd80f378a70`
|
|
is already cached locally (pg_restore16.14). There is no image pull, network,
|
|
published port, host data mount, production connection or IAM/client execution.
|
|
Database storage is a bounded temporary memory filesystem. Only fixed aggregate
|
|
counts for `token`, `resolver`, and `realm` leave the database. The container,
|
|
volumes and temporary plaintext are removed and cleanup is checked. The original
|
|
ciphertext remains for reproducibility. This proves database recovery only;
|
|
token decryption, audit signature continuity and full identity service recovery
|
|
require matching key/config files and further native owner checks.
|
|
|
|
Executed successfully on 2026-10-16 (local time). The command is retained for
|
|
provenance; its r1 receipt now exists and must not be overwritten or rerun merely
|
|
to repeat evidence. Run future reviewed drills with a new receipt. Paste the key
|
|
only into the hidden prompt, then share the success/error line or safe JSON
|
|
receipt. The receipt destination must be new; do not overwrite a failed receipt.
|
|
|
|
```bash
|
|
cd /home/worsch/net-kingdom
|
|
bash sso-mfa/bootstrap/sops-custody-unlock.sh \
|
|
--expected-recipient age1zvryunvjhvpkmasskauga2heeg0ztnte9ymgppvjge36ekumk50syr3tsz \
|
|
--prompt -- python3 tools/bootstrap-recovery/check-privacyidea-database.py \
|
|
--transfer-receipt docs/evidence/2026-10-15-bootstrap-recovery/privacyidea-transfer-r1.json \
|
|
--receipt docs/evidence/2026-10-15-bootstrap-recovery/privacyidea-database-drill-r1.json
|
|
```
|
|
|
|
Source verification:23 focused checks passed, including a real local Docker
|
|
custom-format dump, age encryption, temporary fixture-key unlock, wrong-key
|
|
refusal, native restore/readback and cleanup. Those tests use synthetic data and
|
|
fresh fixture keys. They are not evidence of actual backup decryption. If the
|
|
real drill fails, review its closed receipt; do not expose PostgreSQL/age stderr
|
|
or dump contents in chat. T02/T03 remain open.
|
|
|
|
## Actual database recovery result — 2026-10-16
|
|
|
|
The operator reported success and the actual receipt was reviewed against the
|
|
exact checker source, pinned image and fetched ciphertext SHA256. The retained
|
|
ciphertext digest was independently rechecked. Decryption, unrelated wrong-key
|
|
refusal, native PostgreSQL restore, fixed aggregate readback and container/plaintext
|
|
cleanup all passed. The restored database has five tokens, two resolvers and
|
|
two realms. The local Docker inventory contains no remaining NK49 containers.
|
|
Evidence: `privacyidea-database-drill-r1.json` and `database-drill-review.json`
|
|
in the October15 evidence directory. No production connection occurred.
|
|
|
|
This proves recovery of the selected database using the operator's independently
|
|
held key. It does not prove the recoverability of token plaintext, old audit
|
|
signatures, directory accounts, issuer keys or a full identity environment.
|
|
|
|
## Missing coverage prepared at the operator's request
|
|
|
|
The operator confirmed there is no known full identity bundle and requested
|
|
preparation of the missing coverage. The machine-readable proposal is
|
|
[`identity-backup-profile.json`](../tools/bootstrap-recovery/identity-backup-profile.json).
|
|
The read-only preparer checked the primary cluster UID, four current Deployment
|
|
UIDs/readiness/observed generations, five Bound PVC UIDs and the complete current
|
|
Secret/ConfigMap reference sets. It captured names and identities only; it did
|
|
not read Secret values or execute in pods. Its exact result is
|
|
`identity-backup-preparation-r1.json`. Fourteen tests cover foreign/stale targets,
|
|
new unreviewed dependencies, projected Secrets and private-value suppression.
|
|
|
|
| Component | Required new coverage | Required recovery proof |
|
|
| --- | --- | --- |
|
|
| LLDAP | Consistent `users.db` snapshot, remaining durable data and current credentials/configuration | User, group and membership preservation; existing-user authentication |
|
|
| Authelia | Consistent storage DB, original encryption/OIDC/session keys and current configuration | Storage decryption, signing identity and isolated authentication |
|
|
| privacyIDEA | Original enckey plus matching audit pair, configuration/scripts and a linked native database dump | Existing token decryption/MFA, old audit signature verification and new isolated audit entry |
|
|
| KeyCape | Current registry/config and signing key, policy/receipts, every referenced credential's recovery or independent reissue disposition | Issuer/JWKS identity, registry/policy continuity and positive/negative consumer acceptance |
|
|
|
|
The proposed primary prefix is
|
|
`platform-pg/application-archives/net-kingdom/`; the proposed secondary folder is
|
|
`Backup/railiance-backups/net-kingdom-identity/`. These are **not admitted**.
|
|
Platform's current generic application archive adapter only admits
|
|
`coulomb-social` and `email-connect`, and their publisher custody profiles are
|
|
application-bound. A new NetKingdom profile and scoped publisher require owner
|
|
admission; existing credentials must not be borrowed. The operator's Scaleway
|
|
bootstrap API key is unnecessary for preparation.
|
|
|
|
The initial goal is one verified generation, with a measured encrypted size and
|
|
fresh provider quota/upload-overlap checks. A proposed 512 MiB ceiling is a
|
|
refusal bound, not evidence that the data fits the secondary budget. Seven daily
|
|
and two weekly points remain a later measured proposal. No retention deletion
|
|
or recurring schedule was installed. Capture must preserve original keys and
|
|
use SQLite online-backup/PostgreSQL logical snapshots, record each capture
|
|
interval and refuse changed source identities/configuration. It must not claim
|
|
global atomicity or substitute a fresh export for the already recovered old dump.
|
|
|
|
The source implementation below supersedes the earlier adapter-preparation gap.
|
|
The Scaleway primary/Storage Share secondary proposal above is deferred: the
|
|
initial rollout implements Storage Share only. Neither destination has yet
|
|
received a full identity bundle. Owner admission and actual recovery remain open.
|
|
|
|
|
|
## Implemented identity backup packet — 2026-10-16
|
|
|
|
The new capture, sealing, upload/readback, independent retrieval, archive/native
|
|
PostgreSQL verification, separate retention and node installer are implemented
|
|
under `tools/bootstrap-recovery/identity-*.py`, `capture-identity.py`,
|
|
`retrieve-identity.py`, `verify-identity*.py` and `install-identity-backup.py`.
|
|
Source profile: `identity-backup-profile.json`. No production keys were collected,
|
|
no new credential grants were made and no timers were installed in this source
|
|
checkpoint. Decision `40bfae40-6108-4e51-8613-073339f5345f` remains pending.
|
|
|
|
The exact new owner requests are:
|
|
|
|
- [CCR-2026-0048 publisher](../../railiance-platform/credential-change-requests/CCR-2026-0048-net-kingdom-backup-publisher.yaml): a read/create share for only `Backup/railiance-backups/net-kingdom-identity/`, delivered through a dedicated 15-minute, audience/namespace/SA-bound ESO reader.
|
|
- [CCR-2026-0049 retention](../../railiance-platform/credential-change-requests/CCR-2026-0049-net-kingdom-backup-retention.yaml): a separate read/delete share for that same folder and separate process. Compromise of this credential can delete points in the folder; it must not be given to the publisher process.
|
|
|
|
Both require Platform, NetKingdom and Backup-share owner review. Only the scoped
|
|
share tokens enter exact new workload KV paths and ESO-managed Secrets. Neither
|
|
the operator account password nor the age private key is delivered to the job.
|
|
The setup helper refuses proposed/unapproved or hash-mismatched CCRs. Native
|
|
positive and wrong-SA/namespace/audience probes must prove the admitted reader
|
|
boundary, followed by equality to the exact ESO delivery; temporary readers are
|
|
revoked. These are new grants, so prior approval to prepare the missing coverage
|
|
does not count as completed credential-owner admission.
|
|
|
|
The initial provider is Storage Share only. Ciphertext and safe receipt are
|
|
created once under a unique generation name, read back using strong ETags and
|
|
checked by size/SHA256. A 512 MiB refusal ceiling and two GiB fresh provider
|
|
headroom apply; unsupported public-share quota fails closed. Public DAV behavior
|
|
and permission denials are checked with disposable public-recipient fixture
|
|
ciphertext before accepting the grants. No Scaleway credential or another
|
|
workload publisher is reused. Nextcloud documents the public-share DAV interface
|
|
in its [official WebDAV guide](https://docs.nextcloud.com/server/25/user_manual/en/files/access_webdav.html).
|
|
|
|
Two temporary helper Pods mount only reviewed PVCs read-only, have no service
|
|
account token, use pinned images, private tmpfs scratch and deny-all network
|
|
policies, then are deleted with exact UID preconditions. Kubernetes network
|
|
allows combine, so `identity-isolation.py` first prepares two guarded patches to
|
|
exclude the capture label from the existing namespace-wide DNS allows. Every
|
|
peer/port and ordinary service selector match is preserved. Plans bind the
|
|
current cluster/policy UIDs, resource versions and full original specs, and
|
|
refuse an existing pod using the capture label. Capture refuses any remaining
|
|
matching allow policy, source drift or unknown helper cleanup outcome.
|
|
|
|
Capture uses native SQLite online backups and a fresh native PostgreSQL dump;
|
|
all required current Secrets/configuration and original key pairs are sealed
|
|
with the public recipient. The manifest and plaintext hashes are inside the
|
|
ciphertext. privacyIDEA service logs, historical LLDAP/Authelia/PI `backups/`
|
|
subtrees and Authelia notification.txt are omitted; the audit database is in the
|
|
linked PostgreSQL dump, and the full KeyCape policy/receipt tree is preserved.
|
|
There is no claim of a globally atomic multi-service snapshot. Drift/partial
|
|
cleanup stops future runs and requires exact owner reconciliation.
|
|
|
|
### Ordered live rollout after exact packet approval
|
|
|
|
1. Record all required owner reviews in CCR-2026-0048/0049 and bind their exact file hashes. Run `identity-credential-setup.py publisher|retention --ccr-sha256 HASH --receipt PRIVATE_NEW_PATH` through the attended Platform-admin owner conduit. Never inspect private helper output or copy operator credentials into the runtime. Verify actual provider permissions, initial CAS-only KV publication, exact policy/role, agent high-risk denials and revoked fixture readers.
|
|
2. Apply only `identity-custody.json`; wait for its two SecretStores/ExternalSecrets to be Ready. Run `verify-identity-delivery.py publisher|retention --ccr-sha256 HASH --kubeconfig REVIEWED_PATH --receipt PRIVATE_NEW_PATH` through that same owner conduit. Resolve any native denial before accepting delivery; do not set confirmed-claim flags from fixtures.
|
|
3. Generate a fresh `identity-isolation.py plan --kubeconfig REVIEWED_PATH --receipt PRIVATE_PLAN`; review it, server-dry-run it, then `apply --plan PRIVATE_PLAN --plan-sha256 HASH` using the same explicit kubeconfig and a fresh receipt. Each patch tests UID/resourceVersion/spec. Regenerate stale plans rather than weakening the guards. Confirm the cluster's network policy enforcement through its owning layer.
|
|
4. Render `install-identity-backup.py render --decision-id 40bfae40-6108-4e51-8613-073339f5345f --receipt PRIVATE_PACKET`. Transfer only the reviewed runtime files in that packet plus the packet to the admitted node. As root on railiance01, run installer `install --packet PRIVATE_PACKET --packet-sha256 HASH --receipt PRIVATE_NEW_PATH`. Installation verifies exact cluster/source hashes and unit syntax; timers remain disabled. Execute the packet's exact `first_capture_command`. All output receipts/state and encrypted copies stay under root-private `/var/lib/net-kingdom-backup`.
|
|
5. Retrieve the successful generation's safe off-site receipt from that private state, then run attended `retrieve-identity.py --source-receipt PRIVATE_NODE_RECEIPT --receipt PRIVATE_INDEPENDENT_TRANSFER` with the independently authorized Backup account via Platform custody. It re-downloads and binds the ciphertext by ETag/size/hash. The receipt names the new private local ciphertext path. A published index alone is not independent retrieval proof.
|
|
6. The human independently verifies the downloaded bundle using the hidden-key command below. Review the actual receipt and exact source/artifact/hash. Copy only this safe receipt to private node state; activate with `install-identity-backup.py activate --accepted-receipt PRIVATE_ACTUAL_RECOVERY --receipt PRIVATE_NEW_PATH`. Activation binds the actual artifact to its local successful off-site receipt, pins that generation, checks health and enables the timers.
|
|
7. Confirm both active timers, last success, retained protection and safe Hub progress. Full NK49 native identity/caller and legacy-credential acceptance remains separately open.
|
|
|
|
All actions accept source/target guards; flags do not create authority. The
|
|
installer runs as the already admitted node owner. If installation or a grant
|
|
partly fails, stop and reconcile its safe receipt/exact resources before retry.
|
|
Do not rotate keys, change existing service credentials or remove older backups
|
|
as an implicit repair. For rollback, disable the two exact new timers first,
|
|
retain the verified ciphertext and restore receipt, then have the owners revoke
|
|
only these newly issued shares/roles and remove their delivery resources after
|
|
confirming no consumer remains. Never delete existing unrelated backup jobs.
|
|
|
|
### Independent full archive verification and bootstrap connection
|
|
|
|
The key is entered only in the operator's hidden terminal, using the independently
|
|
held backup key already checked against this public recipient:
|
|
|
|
```bash
|
|
sso-mfa/bootstrap/sops-custody-unlock.sh \
|
|
--expected-recipient age1zvryunvjhvpkmasskauga2heeg0ztnte9ymgppvjge36ekumk50syr3tsz \
|
|
--prompt -- python3 tools/bootstrap-recovery/verify-identity.py \
|
|
--transfer-receipt /PRIVATE/independent-transfer.json \
|
|
--receipt /PRIVATE/new-full-archive-recovery.json
|
|
```
|
|
|
|
With the reviewed sibling `railiance-bootstrap` checkout, the equivalent handoff
|
|
is `bash ../railiance-bootstrap/bin/railiance identity-recovery` with the same
|
|
transfer/receipt arguments, run inside that hidden-key wrapper. Its
|
|
[bootstrap sequence](../../railiance-bootstrap/docs/identity-recovery.md) links
|
|
independent custody and isolated verification to replacement-host/layer handoff.
|
|
The wrapper delegates to this verifier and never imports into production.
|
|
|
|
The complete manifest, SQLite integrity, original RSA pairs, wrong-key refusal,
|
|
native isolated PostgreSQL restore and verified cleanup are the archive gate for
|
|
recurring backups. A successful archive gate does not prove existing-token
|
|
plaintext/MFA, historical/new audit signatures, existing-user login or issuer
|
|
consumer flows. Those require separately reviewed isolated service acceptance
|
|
before T03 and NK49 can close. The verifier deletes its plaintext; it is not a
|
|
production importer.
|
|
|
|
Today's attended retrieval helper requires healthy OpenBao. For total custody
|
|
loss, independently held provider access and a separately reviewed offline
|
|
retrieval receipt procedure are still required; that offline adapter is not
|
|
implemented. No bare-VM unattended restore is claimed. The backup public key is
|
|
not interchangeable with the separate bootstrap encryption recipient.
|
|
|
|
### Recurrence, retention and health
|
|
|
|
After the first actual archive gate, the prepared root unit runs daily at
|
|
03:45 UTC with up to five minutes jitter. Retention runs in separate processes
|
|
before/after capture, protecting newest, seven distinct daily points, two
|
|
distinct weekly points and the independently recovered pinned generation.
|
|
Unknown/unindexed objects are never candidates. Both pinned and newest points
|
|
are re-downloaded and hashed before any expiry; each candidate is checked and
|
|
deleted conditionally by ETag. Partial/unknown expiry stops subsequent expiry
|
|
until reconciled. A matching guarded local-cache prune preserves protected
|
|
points and leaves unknown/failed runs for owner inspection. There is no claim
|
|
that 10 GiB is sufficient indefinitely: measured quota/headroom is checked each
|
|
run and an excess refuses new capture rather than deleting a protected point.
|
|
|
|
An hourly node health check reports missing/older-than-25-hour backups or partial
|
|
failure to State Hub, with durable local nonsecret outbox/replay. This host-local
|
|
check cannot detect or report a dead host; external outage monitoring remains a
|
|
separate gap. The daily backup RPO is explicit and distinct from the one-hour
|
|
ecosystem coordination frame. Timers are currently disabled/uninstalled.
|
|
|
|
Find this functionality through the owning registry entry
|
|
`capability.security.bootstrap-recovery` (keywords backup, restore,
|
|
disaster-recovery, age) federated by reuse-surface, related to
|
|
`capability.railiance.bootstrap-sequencing`. Maturity remains D3/A2/C1/R0;
|
|
source fixtures and discoverability do not assert live production acceptance.
|
|
|
|
|
|
## Live admission update — 2026-10-16
|
|
|
|
The operator approved the exact packet in decision
|
|
`40bfae40-6108-4e51-8613-073339f5345f`. The first owner login succeeded but
|
|
Nextcloud returned publisher permission21 for requested5. Setup refused before
|
|
publishing any new KV value. Official Nextcloud link-share normalization adds
|
|
permission16 (re-sharing) to read-enabled public links when outgoing federation
|
|
is enabled. The operator then explicitly approved folder-only publisher21
|
|
(read/create/re-share) and retention25 (read/delete/re-share), recorded in
|
|
`5edd5203-fae7-461b-8775-5d22b7873971`. The two CCRs and setup guards now require
|
|
that additional explicit approval. Neither credential gains update rights,
|
|
other-folder or account administration. New-share metadata and failed-attempt
|
|
receipts are retained in the October16 live evidence folder. This supersedes
|
|
the initial no-re-sharing provider assumption; it does not infer a native test
|
|
pass from approval.
|
|
|
|
Node preflight also verified that `railiance01` is an SSH alias whose actual
|
|
hostname is `239.62.205.92.host.secureserver.net`. Runtime/installer now require
|
|
that exact hostname, the reviewed machine-ID hash and the original cluster UID,
|
|
plus root. No server hostname or machine identity was changed.52 focused
|
|
checks passed including the real synthetic encrypted/native PG fixture after
|
|
this guard repair. Unapproved-CCR negative tests use proposed fixtures even
|
|
when the actual owner requests have been approved.
|
|
|
|
Both exact guarded DNS-selector patches passed server-dry-run and were applied.
|
|
The node has active k3s with network-policy rules present. The source-bound
|
|
runtime release was installed and unit validation completed; both timers remain
|
|
disabled. Provider admission/readback and actual full capture remain pending;
|
|
no periodic capture/expiry is enabled by installation alone. One subsequent
|
|
OIDC attempt timed out before owner-command handoff; it did not run mutations,
|
|
and its possible session revocation was unconfirmed. Only the contained owner
|
|
login lane is retried, without switching to root or inspecting captured bytes.
|
|
|
|
|
|
The native publisher fixture subsequently proved read/create, denied overwrite
|
|
and delete, and exact fixture cleanup. Its malformed parent path was rejected
|
|
withHTTP400. The checker now records that separately and also requires a valid
|
|
outside-account-folder request to fail authentication; accepting a malformed
|
|
path rejection alone would not establish authorization confinement.16 provider
|
|
checks pass after this correction. A fresh metadata capture plan confirms zero
|
|
allow policies matching helper Pods. The next owner login timed out before
|
|
handoff; no corrected provider probe or KV/ESO publication ran in that attempt.
|
|
The installed runtime's timers remain disabled while attended admission resumes.
|