Validate OpenBao snapshot evidence and record assurance closure gates
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s

Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a06ecb-456a-71c2-b41e-0755d336e883
This commit is contained in:
codex 2026-09-06 15:24:58 +02:00
parent a867ec269a
commit 31386c8e63
7 changed files with 115 additions and 3 deletions

View file

@ -10,6 +10,11 @@
"signal": "forgejo-db.restore",
"path": "docs/evidence/forgejo-scaleway-restore-2026-09-06.json",
"sha256": "071a732318a55a8ca152d90b7a4cd70644728f94ce2753ab73d1c94ba2a26114"
},
{
"signal": "openbao.snapshot",
"path": "reviews/WARDEN-WP-0027-T02-DRILL-20260822-01-openbao-snapshot-receipt.json",
"sha256": "20e7f5daada384937fb5c1cde8366cb472d552006581a6966f336a6e39836e6f"
}
]
}

View file

@ -129,3 +129,11 @@ overwrite bug. The restore tool explicitly accepts both forms, with verified
hash/decryption flags. Existing historical receipts are unchanged. These producer
fixes enable future dated archive evidence; automatic archive adapters, fresh
end-to-end receipts and recurring execution are still pending.
The OpenBao snapshot adapter also accepts the reviewed, hash-pinned receipt in
`reviews/`. It requires the expected source cluster identity, encrypted off-host
custody, verified hashes and creation time. That time is a conservative age
anchor for snapshot creation, not proof of a later restore or renewed custody.
The August 22 receipt evaluates stale under the 36-hour budget. Updating the
index or reading the file cannot renew it; an isolated OpenBao restore remains
a separate obligation.

View file

@ -0,0 +1,37 @@
# WP-0036 closure review — 2026-09-06
Result: not eligible for completion. T01/T02/T05/T07 are done; T03/T04/T06
retain unmet acceptance criteria. No task criteria were relaxed or moved into
new plans to create an apparent closure.
| Task | Verified local result | Remaining acceptance |
| --- | --- | --- |
| T03 | Native apps-pg/forgejo-db adapters; archive receipt producers; encrypted off-host OpenBao snapshot adapter | Fresh isolated recovery for all supported services, independently available recovery custody, approved installed cadence with execution receipts and operator delivery |
| T04 | Bounded metadata capture and local validation | Q2 receiving contract, named recipient, controlled failure delivery and missing-emission detection |
| T06 | Exact compatibility inventory and dated retention through October 5 | Scoped legacy alias repair and generated brief matching source; retain compatibility until owner acceptance or the recorded retention decision is reviewed |
New source checks:
- `railiance-telemetry/README.md` explicitly says seeded, no implementation;
its only local workplan is proposed RTELE-WP-0001. A receiving contract/runtime
is not available in that checkout. S3 must not build a replacement Q2 plane
merely to satisfy this plan. Owner response is needed for any newer service.
- `rapp-postgres/docs/evidence/backup-restore-20260813T111651Z-remote.json`
explicitly identifies same-node scratch MinIO, not the governed off-host
target. Its successful drill cannot satisfy the current production recovery
acceptance criterion.
- The August 22 OpenBao snapshot receipt is encrypted, hash-verified and copied
off-host. Added its exact SHA-256 to the recovery index and a narrow adapter
that validates source identity and custody flags. It reports the original
creation time and therefore stale, never healthy by rereading. It does not
establish isolated restore or current independent quorum access.
The prepared requests in `docs/platform-ownership-handoffs.md` remain ready for
telemetry, repo-manager/State Hub and compatibility owners. Authorization to
send them was requested separately; no message was sent during this review.
Sending a request alone would not fulfill the receiving owner's acceptance.
Next closure sequence: obtain the Q2 contract and accepted execution/recipient
binding; run fresh owner-authorized recovery and cadence proofs; verify failure
and missing-emission delivery; reconcile aliases and regenerate orientation.
Existing WP-0015 outage exercises remain separate, with their own prerequisites.

View file

@ -1,6 +1,7 @@
"""Hash-pinned native recovery receipts, with original completion timestamps."""
import hashlib
import json
import re
from pathlib import Path
from service_assurance import timestamp
@ -14,17 +15,39 @@ def recovery_signals(now, root=ROOT):
signals = {}
for entry in index['receipts']:
signal = entry['signal']
if signal in signals or signal not in ('apps-pg.restore', 'forgejo-db.restore'):
if signal in signals or signal not in ('apps-pg.restore', 'forgejo-db.restore', 'openbao.snapshot'):
raise ValueError('unexpected recovery signal')
sample = {'result': 'unavailable', 'observed_at': now.isoformat()}
try:
path = (root / entry['path']).resolve()
if not path.is_relative_to((root / 'docs/evidence').resolve()):
allowed = [root / 'docs/evidence']
if signal == 'openbao.snapshot':
allowed.append(root / 'reviews')
if not any(path.is_relative_to(directory.resolve()) for directory in allowed):
raise ValueError('receipt outside evidence directory')
raw = path.read_bytes()
if hashlib.sha256(raw).hexdigest() != entry['sha256']:
raise ValueError('receipt drift')
receipt = json.loads(raw)
if signal == 'openbao.snapshot':
required = ('snapshot_created', 'source_initialized', 'source_unsealed',
'snapshot_encrypted', 'encrypted_copy_off_host',
'encryption_verified', 'hash_verified', 'no_secret_material_recorded')
if (receipt.get('receipt_version') != 1
or receipt.get('source_cluster') != 'railiance01'
or receipt.get('source_namespace') != 'openbao'
or receipt.get('cluster_id') != 'fd28df5d-98ec-57dd-42ec-9b3e4f4e53bf'
or not all(receipt.get(key) is True for key in required)
or not receipt.get('encrypted_location_ref', '').startswith('offhost-custody:')):
raise ValueError('snapshot not accepted')
for key in ('snapshot_sha256', 'encrypted_snapshot_sha256'):
value = receipt.get(key, '')
if not re.fullmatch(r'sha256:[0-9a-f]{64}', value):
raise ValueError('snapshot hash missing')
if timestamp(receipt['created_at']) > now:
raise ValueError('future snapshot')
signals[signal] = {'result': 'pass', 'observed_at': receipt['created_at']}
continue
cell = signal.removesuffix('.restore')
if (receipt['schema'] != 'platform.scaleway-primary-restore.v1'
or receipt['primary_destination'] != f's3://railiance-platform-pg-backup/platform-pg/{cell}/'

View file

@ -41,3 +41,32 @@ def test_invalid_receipt_is_unavailable(tmp_path, change):
(tmp_path / 'assurance').mkdir()
(tmp_path / 'assurance/recovery-evidence.json').write_text(json.dumps(index))
assert recovery_signals(NOW, tmp_path)['apps-pg.restore']['result'] == 'unavailable'
def test_snapshot_is_stale_and_never_substitutes_for_restore():
signals = recovery_signals(NOW)
assert signals['openbao.snapshot']['observed_at'] == '2026-08-22T22:29:21Z'
assert 'openbao.restore' not in signals
contract = {'cluster_uid': 'test', 'capture_max_age_seconds': 900,
'signals': {'openbao.snapshot': {'owner': 'platform', 'max_age_seconds': 129600}}}
result = evaluate(contract, {'schema': 'railiance-platform.observation.v1',
'cluster_uid': 'test', 'captured_at': NOW.isoformat(),
'signals': {'openbao.snapshot': signals['openbao.snapshot']}}, NOW)
assert result['signals']['openbao.snapshot']['state'] == 'stale'
@pytest.mark.parametrize('key,value', [('encrypted_copy_off_host', False),
('hash_verified', False), ('cluster_id', 'other'), ('snapshot_sha256', 'invalid'),
('created_at', '2027-01-01T00:00:00Z')])
def test_snapshot_rejects_unverified_or_wrong_scope(tmp_path, key, value):
index = json.loads((ROOT / 'assurance/recovery-evidence.json').read_text())
entry = next(e for e in index['receipts'] if e['signal'] == 'openbao.snapshot')
receipt = json.loads((ROOT / entry['path']).read_text())
receipt[key] = value
path = tmp_path / entry['path']; path.parent.mkdir(parents=True)
path.write_text(json.dumps(receipt))
entry['sha256'] = hashlib.sha256(path.read_bytes()).hexdigest()
(tmp_path / 'assurance').mkdir()
index['receipts'] = [entry]
(tmp_path / 'assurance/recovery-evidence.json').write_text(json.dumps(index))
assert recovery_signals(NOW, tmp_path)['openbao.snapshot']['result'] == 'unavailable'

View file

@ -148,7 +148,7 @@ class CollectorTests(unittest.TestCase):
with patch.object(self.collector, 'query', side_effect=query), patch.object(self.collector, 'admission', return_value=baseline):
observation = self.collector.capture()
for name, sample in observation['signals'].items():
if name not in ('apps-pg.restore', 'forgejo-db.restore'):
if name not in ('apps-pg.restore', 'forgejo-db.restore', 'openbao.snapshot'):
self.assertEqual(sample['result'], 'unavailable')
# Recorded recovery evidence is independent of failed live status reads.
self.assertEqual(observation['signals']['apps-pg.restore']['result'], 'pass')

View file

@ -291,3 +291,13 @@ automatic recovery adapters pending reviewed fresh archive receipts and adapter
implementation. Validation and residual gates are recorded in
`history/2026-09-06-archive-receipt-review.md`; T03 remains waiting on its wider
cadence/custody/acceptance gates.
## Closure review — 2026-09-06
User requested completion. Source review still finds T03/T04/T06 acceptance
unmet; see `history/2026-09-06-WP-0036-closure-gates.md`. Telemetry's checkout has
no receiver implementation, and the older platform-pg scratch MinIO drill does
not establish production off-host recovery. Added the verified August 22
OpenBao snapshot to the hash-pinned evidence adapter: it reports stale using
its original creation time, separately from the still-missing isolated restore.
No criteria were weakened, external acceptance inferred or live window reused.