net-kingdom/docs/evidence/2026-10-03-root-grant-delivery-progress.md

# Root-grant delivery progress — 2026-10-03

Work: NK-WP-0045. Scope: approved OpenBao custody-record delivery. This receipt
records source preparation, safe preflight and verified attended admission/
publication receipts. It is not custody-grant or recovery-readiness evidence.

## Human disposition and initial empty record

Bernd accepted ADR-0017 and the proposed OpenBao design (scoped made decision
`c2cff929-91b9-4018-bfd6-fa04f7da8280`), then approved going forward.
Asked which existing signed ceremony grounds the first grant, he replied:
“i dont think there is one”. No current ceremony reference or exact custodian
issuer/subject was supplied. Historical local bootstrap metadata and the
planned signed escrow roster cannot substitute for a real grant.

Prepared [initial empty record](../../examples/custody-record/initial-empty-record.json)
for review and attended publication: stable record ID `nk-custody:platform-root`,
revision 1, owner `net-kingdom-custody`, 900-second maximum and `entries: []`.
This is an intentionally empty authoritative candidate, not a synthetic grant
fixture. It represents no certified root grants. Schema and semantic validation
passed. It was subsequently published through the attended lane, as recorded
below. Initial publication must use CAS=0 and fail if an
existing document or unresolved retained history is discovered; do not replace
an existing record on the assumption that no ceremony was supplied.

A future real grant requires a human-controlled ceremony naming exact issuer,
subject, actor, action time/reason and evidence reference, introduced in a
strictly newer revision. No identity is inferred from JWT roles or LLDAP groups.

## Source preparation and checks

- NetKingdom validator now emits `expires_at` so consumers can honor cache bounds
  tighter than 900 seconds. All 18 focused tests pass, including zero/60-second
  bounds. Canon source pin `db319335b27b61b855382ee5af5d690f06eeb010`.
- railiance-platform implemented exact reader admission, whole-document CAS,
  independent receipt history, interrupted-publication recovery, replay and
  stale-receipt refusal, and restore checks; 14 tests passed. Source baseline
  `fa958cc`, attended empty-publication packet `35d326f`, exact-commit primary
  synchronization applied.
- rapp-user-engine prepared dedicated openbao-audience JWT projection, scoped
  egress and explicit-image release rendering; 7 packaging tests passed. Source
  baseline `576f5e7`, pushed with clean working tree; empty-source rollout is
  explicitly supported. PostgreSQL baseline uses
  the existing DB; no new persistent volume is needed.
- user-engine implements pinned canon, authenticated bounded reads, atomic
  PostgreSQL baseline, controlled receipt-bound bootstrap and optional
  AccountAuthority mapping. Isolated PostgreSQL checks verified competing CAS
  updates, cross-connection persistence and reconnect persistence. Final source
  `0d0751d`: suite ran 296 tests with eight skips; layer checks, wheel build and
  embedded-resource integrity passed. Published image verified by registry manifest and disposable container:
  `sha256:030fe90232e20bb02b372e3d16b8ff659a5407d54f6405eb85754262b9ad7dc6` (source `0d0751d`). Embedded reader, baseline,
  bootstrap, canon and AccountAuthority field were checked; owner sync `f85c968`.

## Safe live preflight

OpenBao workstation tunnel `http://127.0.0.1:18200`: initialized true, sealed
false, standby false. Anonymous read of the exact custody data path returned
403; response bodies were discarded. This proves only anonymous denial, not
reader admission, document existence or positive live custody delivery.

## Remaining execution and acceptance

The founder completed OpenBao provisioning and publication in the attended
OIDC/MFA lane; fresh success receipts are recorded below. Existing approval was
sufficient for the reviewed delivery work. The concrete producer packet is in
`railiance-platform/openbao/root-grant/README.md`.
Keep independently retained latest publication history through restore. Its
location and backup receipt must be recorded before recovery readiness closes.

The custody-capable application image is published/pinned and its deployment
passed preflight. Deploy through rapp-user-engine and initialize the
baseline using the authenticated document plus publisher digest. Empty source
acceptance proves no-grant delivery; it cannot prove a real positive grant or
revocation. Positive grant/revocation exercises use isolated synthetic test
sources unless a real human ceremony explicitly introduces a live grant.

No live admission, publication, workload rollout or actual root grant occurred
in this receipt. Source/fixture passes are distinct from live acceptance.


## Exact deployment preflight

rapp-user-engine rendered the published image into
`/tmp/user-engine-custody-release.yaml`, manifest SHA256
`5e21548fc2d23926d167f17795f098f8db5fffafe9bcd3bee43875f7bb51a50e`
(rapp-user-engine release packet commit `13a9c0b`, committed and synced).
Live node `92.205.62.239` Ready; requested CPU 3445m/4000m (86%), memory
9384Mi (58%). Staged Deployment retains single-replica Recreate and requests
50m/64Mi. Server dry-run passed eight objects. Client diff returned expected
changes (exit 1) to Deployment and runtime NetworkPolicy; raw diff output
suppressed. Existing live factor-recovery URL, mounts, volumes and egress
semantics are preserved. This final artifact supersedes the earlier preflight
hash after preserving that live setting. No apply or workload mutation.

Image publication and attended reader/source admission/publication are resolved.
Reviewed runtime rollout, controlled baseline initialization and live
no-grant/denial/outage/restart acceptance remain. Off-host publication history
retention/recovery needs its real receipts before recovery readiness closes.

## Attended execution receipts

Founder executed the reviewed commands and reported success. Both fresh receipts
were independently read and their non-secret fields retained in
[admission receipt](2026-10-03-platform-root-admission-receipt.json) and
[publication receipt](2026-10-03-platform-root-publish-receipt.json), with source
file SHA256 checksums. Both report `ok: true`; admission confirms exact role,
namespace/ServiceAccount, audience, CAS and audit. Publication confirms revision
1, KV version 1 and the reviewed empty-document digest. The independently
persisted publisher baseline matches these fields. This supersedes the earlier
pending-admission/publication state, without claiming runtime or recovery
acceptance. The operator's older OIDC role configuration is not an execution
receipt. Owner rollout and authenticated baseline/read acceptance resumed.

## Live runtime and consumer acceptance

rapp-user-engine applied only the reviewed Deployment and runtime NetworkPolicy.
Bounded apply artifact SHA256
`804400134a705651d12ada05a068543d2f1665a14eec5c3b65898c7560a63a8f`.
Rollout succeeded; Ready/available 1, health/readiness HTTP 200. One initial
restart was safely traced to a transient original runtime PostgreSQL connection
refusal; subsequent restart count remained stable. This is distinct from the
post-baseline restart acceptance exercise.

The consumer's [live acceptance receipt](2026-10-03-user-custody-live-acceptance.json)
reports `ok: true`: baseline initially absent, controlled initialization from a
fresh authenticated owner read matched the publisher digest, empty-source
mapping returns no root grant, wrong tenant returns no grant, and a real
wrong-audience JWT login is denied. Database reconnect preserves baseline.
Injected read failure and same-revision mutation preserve the trusted baseline;
these are injection checks, not claims of an actual OpenBao backend outage.
Post-initialization pod restart, exact-reader negative capabilities/audit and
independent backup/recovery evidence are still being collected. No actual grant
was introduced.

The [final consumer receipt](2026-10-03-user-custody-live-final-acceptance.json)
subsequently verified baseline persistence after an actual controlled pod
restart, a fresh matching owner read and HTTP 403 for the actual reader's
metadata LIST, sibling GET and exact-path POST. The internal fixture service
serialized `root_grant: null` against the actual live empty source; it does not
claim acceptance of the separate external Hub authenticated route. Safe read
and login request IDs were retained for owner audit correlation. USER-WP-0037-T02
is accepted for this delivered no-grant integration; the separate Hub route
task is outside this acceptance. The restarted workload is Ready; its initial
DB connection refusal also occurred on the controlled restart, recovered, and
remained stable. No ongoing CrashLoop or zero-restart claim is made.

The [isolated Raft recovery receipt](2026-10-03-platform-root-isolated-raft.json)
reports actual disposable OpenBao v2.5.5 snapshot/restore checks: an older
snapshot missing the custody source was refused against the independent
baseline, and the latest snapshot restored the matching revision/digest.
Scratch copy/restore of baseline and immutable history verified content and
permissions. Temporary daemon/storage were removed. No production snapshot
or rollback was used. This proves the isolated recovery guard, not current
provider backup coverage or the attended live-source restore check.

## Final audit and recovery handoff

The [safe audit receipt](2026-10-03-root-custody-audit-correlation.json)
independently correlates both consumer read/login request IDs with request and
response audit events and no errors. The bounded lookup inspected only the
latest 64 MiB of the owner audit file; raw audit records were never emitted.
This resolves audit correlation.

USER-WP-0037-T02 is done, synced HEAD `d955cd7`; runtime workplan is finished,
synced HEAD `f108c65`. The remaining T05 gate is the attended producer recovery
finish helper: current retained-baseline/source match plus encrypted history
provider upload/download/decryption through the existing governed Nextcloud
Backup lane. Final producer/finish suite: 19 passed, including actual age crypto
roundtrip against an in-memory disposable provider. It preserves all backups and
uses a unique receipt/object. The surfaced historical backup-share URL is not
repeated or stored here; the helper compares it with the current lane only as a
boolean and stops for owner incident handling if they match. This unresolved
comparison is not represented as a harmless or rotated credential.

Producer synced `b8f8a7ada65f0a86b90a54812365757838102991`, exact commit
applied on primary/railiance01, clean with ahead/behind 0. Non-authoritative
baseline/history recovery copy in owner source `a252447058105eed0931d34f1a906f39ce7c2e5e`
was retrieved through a fresh remote clone and matched byte-for-byte; the owner
retains its safe remote readback receipt. Provider archive generation is
explicitly unverified; the prepared attended direct encrypted history roundtrip
is still required. All authorized source work is implemented and synced.

## Failed recovery finish attempt — 2026-10-03

Founder ran the prepared helper with fresh receipt path
`/tmp/platform-root-finish-20261003T185421Z-333977.json`. Warden reported child
nonzero and session revoked; no receipt was created. Recovery/provider readiness
is not established. The helper's catch-all exit without a safe failure receipt
was an implementation gap and is being repaired before another attended retry.

Independent [bounded audit lookup](2026-10-03-root-custody-finish-failure-audit.json)
confirms successful custody, governed backup-lane and operator backup-bundle
reads at 18:54:37–38 UTC, with no errors. The failure is therefore after those
reads, within the backup roundtrip preparation/execution. Its exact stage cannot
be inferred from suppressed exceptions. No IAM widening is justified. Local
imports, validator/schema pins, retained-state validation and installed age
binaries pass offline preflight. Owner repair adds allowlisted stage/failure
receipts and full-command failure tests while keeping all raw diagnostics private.

Producer repair committed `33ff822`: stdlib entry reserves a fresh mode-0600
receipt before imports, with fixed stage/error codes for expected failures and
safe numeric HTTP/artifact metadata. Foreign exception text is never emitted.
The installed wrapper was independently smoke-checked outside the attended
envelope: it refused before owner reads and created the expected safe failure
receipt, with no raw output. Producer suite passed 32 tests, including full
private attended-HOME command execution with the real validator and age crypto
against disposable owner/provider fixtures, startup/dependency, stale receipt,
auth, quota, crypto, upload/download and incident/escrow failures.

Recovery decryption now verifies the existing independent local escrow recipient
against the governed lane and decrypts downloaded history with that escrow key;
no key is created, reset or printed. A mismatch is a specific recovery blocker.
Corrected attended retry with a new receipt is necessary to prove the actual
provider roundtrip or expose its precise safe failure stage. The original
failure's exact cause is still unknown, and provider readiness remains unproven.

## Provider upload rejection diagnosed — 2026-10-03

Founder retried with fresh receipt
`/tmp/platform-root-finish-20261003T191229Z-333977.json`; the
[safe failure receipt](2026-10-03-root-custody-finish-upload-rejected.json)
reports `stage: provider_upload`, `error_code: new_artifact_upload_failed` and
HTTP 401. Current retained baseline matches revision 1/KV1/digest; existing
independent escrow recipient matches the governed lane. Historical local share
does not match the current lane; its revocation remains separately unverified.
No provider backup success is claimed.

Source review found the custody upload omits `X-Requested-With: XMLHttpRequest`.
[Official Nextcloud WebDAV documentation](https://docs.nextcloud.com/server/stable/developer_manual/client_apis/WebDAV/basic.html)
requires this header on public DAV non-GET requests unless outgoing federation
is enabled; without it, the documented response is 401 Not Authenticated.
This is a concrete client defect and a likely explanation of the observed 401,
not proof that the current share credential is invalid or that header correction
already resolves this live instance. Producer is fixing/testing the protocol
request while retaining existing endpoint, credentials and permissions. A new
attended live result is still required.

Producer protocol repair committed `50da37c`: custody public-DAV upload now sends
the required header, with a real local HTTP wire test verifying its presence and
conditional-create semantics. Shared migration request helper and common upload
client have the same scoped source correction. Forty-one tests passed (19 finish,
14 producer, eight backup credential checks); shell syntax and diff checks pass.
Failure receipts retain the backup lane's KV version. No token, endpoint, policy,
key or account-fallback change was made. Source synchronization does not establish
rollout of older vendored backup clients; owner records that refresh requirement
separately. The corrected custody finish command needs a new attended run and
fresh receipt to prove this live provider path.

## Header-corrected live retry still rejected — 2026-10-03

Founder ran the corrected helper with fresh receipt
`/tmp/platform-root-finish-20261003T192330Z-290055.json`. The
[retained retry receipt](2026-10-03-root-custody-finish-header-retry-rejected.json)
again reports provider upload HTTP 401, now tied to backup lane KV version 3.
Current custody baseline and independent escrow checks pass, and the historical
share remains different from the current lane. The header correction does not
resolve this live rejection; an invalid, protected, expired or mismapped share
cannot be distinguished from client endpoint/auth semantics by 401 alone.

Producer is preparing a bounded read-only attended provider/share inspection:
compare the current protected token with the operator's exact reviewed share,
report owner/path/create-only permissions, password/expiry indicators and safe
endpoint probes. No upload, rotation, provider share mutation or IAM widening
is authorized by diagnosis alone. Another unchanged finish retry is not requested.
The remaining task stays blocked on diagnosed provider delivery and its live
encrypted backup/readback proof; source/consumer/audit readiness is preserved.

Read-only inspection implementation synced at producer
`4dd9646ec21b63a7743dae0ad3273da2f73adefe`. The existing attended wrapper now
accepts `--inspect-share`: current governed OCS share inventory is compared in
protected memory, and fixed provider public routes receive depth-zero PROPFIND
only. Output contains binding booleans and safe status classifications. Full-main
inspection tests forbid entering the backup roundtrip. Twenty-three finish tests
pass. Final review correction synced at
`fee1d339f98d0fd7ec2336d946eea2340d557f8a`: aware expiry offsets are converted
correctly, naive expiry and omitted password remain unknown, and clearly
protected/expired shares skip public authentication probes. Twenty-five finish
tests pass. Exact expected/derived remote projection applied, working tree clean.
Inspection still requires the founder's own attended credential route;
no provider conclusion or actual backup success is inferred from source tests.

## Successful share inspection; recovery remains blocked

The attended read-only receipt at 19:36 UTC is retained in
`2026-10-03-root-custody-share-inspection.json`. The baseline matches live KV1;
backup lane KV3 has neither its token nor the expected labelled share in the
reviewed owner folder inventory. Both authenticated route probes returned 401;
the unauthenticated documented route returned 503. No provider mutation or
encrypted-history roundtrip was performed. Inventory absence is scoped to that
folder; these results do not prove global deletion or successful recovery.

The platform owner is preparing a tested attended repair that proves replacement
share upload/download/decryption before CAS of only the lane's upload coordinates,
preserving all other fields, keys, backups and unknown shares. Activity-core owns
the subsequent consumer refresh: worker environment snapshots and the frozen
Forgejo backup ConfigMap require explicit loaded-client evidence.

Security orientation reloaded after compaction: package 0.2.4, digest
`025aab4f38246219c924bc94954e70ea0e6e7f72bcaf8d4344cd599602d345c3`,
start card withheld/orientation-only because a source changed. Reviewed the local
start card and operative NetKingdom SECURITY-COMPANION directly; this retrieval
does not grant authority or establish live readiness.

Bounded repair source `railiance-platform/f1a780d` passed 33 finish tests, with
55 relevant producer/backup tests total. Exact public DAV and OCS wire shape,
real encryption/decryption, stale-version refusal and partial CAS receipts are
covered. Actual attended provider success is still outstanding.
Activity-core owns the live stale-client refresh in `ACTIVITY-WP-0042-T08`,
task `020d19be-0576-5052-ad35-14b1d5a48fe1`; T07 holds the read-only inspection.

Activity-core final source `42d71fa` is clean and Git-pushed; local consistency
passes and duplicate task blocks were corrected without changing UUIDs.
Its repo-manager remote projection preflight rejects grandfathered historical
`custodian-WP-*` identifiers. Exact remote projection is therefore unproved;
this compatibility blocker is separate from the provider repair and consumer
refresh. Historical identifiers must not be renumbered to bypass the rejection.

## Final attended recovery acceptance — 19:56 UTC

Verified actual local receipt
`2026-10-03-root-custody-recovery-verified.json` against the acceptance fields.
Current custody record revision1/KV1 and digest match the independent baseline.
Encrypted history upload returned201 and owner download200; ciphertext digest
`577498be3c6566e9622fe0d87ea3ad55063941acfab3aeb0accfc4a05dd21200` matched,
and independent escrow decryption produced the exact retained history.
Provider proof preceded backup lane CAS3->4. Only upload coordinates changed;
escrow and unrelated fields, unknown shares and old backups were preserved.
Replacement share23 was created within the reviewed Backup scope.

NK-WP-0045-T05 is done, plan finished (6/6). Actual live consumer, denial/restart,
audit and isolated Raft restore evidence recorded above remain applicable.
Runtime backup consumers were not refreshed; activity-core T08 continues to own
that requirement after its provider prerequisite was met. Historical credential
owner/revocation remains unverified in the separate platform incident. No real
custodian was certified: authoritative entries remain empty.

Root knowledge card reloaded for closure: package0.2.4, digest
`025aab4f38246219c924bc94954e70ea0e6e7f72bcaf8d4344cd599602d345c3`,
verified-files/orientation-only. Actual authority came from the approved design
and founder-attended execution, not the knowledge card.