Close verified incident task and finish local workplan loose ends
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 3s

Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a0e3c3-621b-7350-9f77-50a8d3ee7657
This commit is contained in:
codex 2026-09-27 19:04:54 +02:00
parent 5781d34b3b
commit debf981097
22 changed files with 1210 additions and 62 deletions

View file

@ -1,6 +1,6 @@
# DRAFT for railiance01 (RPF-WP-0044). Not synced by any root: move to # DRAFT for railiance01 (RPF-WP-0044). Not synced by any root: move to
# ../applications/ only in this app's adoption task, with the founder's go-ahead. # ../applications/ only in this app's adoption task, with the founder's go-ahead.
# No automated sync, no finalizer. targetRevision: chart version; T06 adds CRD Prune=false,Delete=false before merge. # No automated sync, no finalizer. CRDs survive Application pruning/deletion.
apiVersion: argoproj.io/v1alpha1 apiVersion: argoproj.io/v1alpha1
kind: Application kind: Application
metadata: metadata:
@ -21,6 +21,9 @@ spec:
releaseName: external-secrets releaseName: external-secrets
values: | values: |
installCRDs: true installCRDs: true
crds:
annotations:
argocd.argoproj.io/sync-options: Prune=false,Delete=false
serviceAccount: serviceAccount:
create: true create: true
name: external-secrets name: external-secrets

View file

@ -0,0 +1,62 @@
# Coulombcore ArgoCD retirement preparation
Existing owner task: RPF-WP-0044-T08. Prepared September 27, 2026; blocked
pending a readable live inventory. No controller, application or workload was
changed. This document is the phase C preparation within the existing workplan.
The host SSH lane works. Kubernetes returned Unauthorized for both the normal
kubectl context and `sudo -n k3s kubectl --kubeconfig
/etc/rancher/k3s/k3s.yaml get nodes`. The infrastructure/cluster owner must
restore accepted read access; do not rotate cluster credentials or restart k3s
as an incidental inventory fix. Current workload ownership cannot be inferred
from the old manifests.
## Inventory and decision inputs
Capture node/cluster identity, ArgoCD deployments/statefulsets and installed
version, Applications and AppProjects, each Application's destination, pinned
revision, tracked resource set, hooks, automated sync and finalizers. Read only
repository Secret metadata, never data. Identify external cluster destinations:
an old controller can still manage a remote cluster. Search platform and tenant
source for references to `argocd/applications/` and `argocd/bootstrap/`, including
Make entry points, and match each live application to its accepting owner.
The railiance01 root uses `argocd/railiance01/applications`; the legacy root
uses `argocd/applications`. Do not remove the legacy tree while the old controller
can reconcile it with prune enabled. Keep evidence of each workload's current
replicas, readiness and image before any retirement execution.
## Ordered execution after inventory and owner acceptance
1. Have the cluster owner pin the installed railiance01 ArgoCD version and
reviewed resource requests in its canonical source. Inspect current requests;
the original phase A BestEffort observation is historical, not a fresh check.
2. Classify each old tracked resource: already adopted on railiance01, retained
on coulombcore with another owner, or separately approved for retirement.
An application name match does not prove matching cluster/resource identity.
3. Freeze the old root and child reconciliation through the cluster owner's
reviewed procedure. Confirm no running sync operation and no second writer.
Preserve the old Application specs and controller configuration in protected
recovery storage; repository credentials stay under existing custody.
4. Detach only inventory-approved Application objects without cascading workload
deletion. Review resource finalizers first; never delete the ArgoCD namespace
or CRDs as a shortcut. Verify every retained workload is still healthy and
each replacement owner can reconcile its accepted resource set.
5. Disable the old controller through its actual installation owner. Prove it
no longer reconciles and that no unrelated system uses its repository/auth
resources. Revoke retired credentials only through their custody owners.
6. Once the old controller is inert, remove the legacy source directories and
retire or repoint their callers in one reviewed platform change. Render the
railiance01 bootstrap and children and verify no legacy reference remains.
Stop for missing inventory, ambiguous tracking, active operations, unknown
finalizers, missing acceptance or degraded retained workloads. Before detachment,
rollback restores the recorded sync configuration. After replacement ownership,
keep the old controller stopped until the replacement writer is explicitly
suspended; rollback must never run two reconcilers against one workload. Package
or data deletion needs its own exact approved disposition.
Completion evidence must include the inventory, accepting owners, source/caller
changes, controller shutdown and retained-workload checks. T08's planning closure
still requires the live read-only inventory. Execution remains with the existing
cluster/infra owners; no new task or workplan is created here.

View file

@ -0,0 +1,492 @@
{
"observed_at": "2026-09-27T16:53:28.268968+00:00",
"applications": [
{
"name": "activity-core",
"sync": {
"comparedTo": {
"destination": {
"namespace": "activity-core",
"server": "https://kubernetes.default.svc"
},
"source": {
"path": "k8s/gitops",
"repoURL": "https://forgejo.coulomb.social/coulomb/activity-core.git",
"targetRevision": "a12f1169f9d130058ce767f5b26de0606997916c"
}
},
"revision": "a12f1169f9d130058ce767f5b26de0606997916c",
"status": "Synced"
},
"health": {
"lastTransitionTime": "2026-09-27T14:06:22Z",
"status": "Healthy"
},
"reconciledAt": "2026-09-27T16:50:21Z",
"operation": {
"phase": "Succeeded",
"startedAt": "2026-09-27T14:06:19Z",
"finishedAt": "2026-09-27T14:06:20Z"
},
"automated": null,
"conditions": [],
"resources": [
{
"kind": "ConfigMap",
"name": "actcore-external-activity-definitions",
"namespace": "activity-core",
"status": "Synced",
"version": "v1"
},
{
"kind": "ConfigMap",
"name": "actcore-ops-service-inventory",
"namespace": "activity-core",
"status": "Synced",
"version": "v1"
},
{
"kind": "ConfigMap",
"name": "actcore-report-schemas",
"namespace": "activity-core",
"status": "Synced",
"version": "v1"
},
{
"kind": "ConfigMap",
"name": "actcore-runtime-config",
"namespace": "activity-core",
"status": "Synced",
"version": "v1"
},
{
"kind": "Service",
"name": "actcore-api",
"namespace": "activity-core",
"status": "Synced",
"version": "v1"
},
{
"kind": "Service",
"name": "actcore-worker-metrics",
"namespace": "activity-core",
"status": "Synced",
"version": "v1"
},
{
"group": "apps",
"kind": "Deployment",
"name": "actcore-api",
"namespace": "activity-core",
"status": "Synced",
"version": "v1"
},
{
"group": "apps",
"kind": "Deployment",
"name": "actcore-event-router",
"namespace": "activity-core",
"status": "Synced",
"version": "v1"
},
{
"group": "apps",
"kind": "Deployment",
"name": "actcore-worker",
"namespace": "activity-core",
"status": "Synced",
"version": "v1"
}
]
},
{
"name": "bao-notice",
"sync": {
"comparedTo": {
"destination": {
"namespace": "bao-notice",
"server": "https://kubernetes.default.svc"
},
"source": {
"path": "argocd/platform-addons/bao-notice",
"repoURL": "https://forgejo.coulomb.social/coulomb/railiance-platform.git",
"targetRevision": "9ee20967ca7b50a8926c99ae86fa027f8bfb0fa3"
}
},
"revision": "9ee20967ca7b50a8926c99ae86fa027f8bfb0fa3",
"status": "OutOfSync"
},
"health": {
"lastTransitionTime": "2026-09-23T22:56:40Z",
"status": "Healthy"
},
"reconciledAt": "2026-09-27T16:52:12Z",
"operation": {
"phase": "Succeeded",
"startedAt": "2026-09-23T22:56:34Z",
"finishedAt": "2026-09-23T22:56:36Z"
},
"automated": null,
"conditions": [
{
"lastTransitionTime": "2026-09-23T22:55:19Z",
"message": "Application has 1 orphaned resources",
"type": "OrphanedResourceWarning"
}
],
"resources": [
{
"kind": "ConfigMap",
"name": "bao-notice-conf-4f5g9kc7c2",
"namespace": "bao-notice",
"status": "Synced",
"version": "v1"
},
{
"kind": "ConfigMap",
"name": "bao-notice-conf-dg4hmf2dg4",
"namespace": "bao-notice",
"requiresPruning": true,
"status": "OutOfSync",
"version": "v1"
},
{
"kind": "ConfigMap",
"name": "bao-notice-html-6mm6h6mgkf",
"namespace": "bao-notice",
"status": "Synced",
"version": "v1"
},
{
"kind": "Namespace",
"name": "bao-notice",
"status": "Synced",
"version": "v1"
},
{
"kind": "Service",
"name": "bao-notice",
"namespace": "bao-notice",
"status": "Synced",
"version": "v1"
},
{
"group": "apps",
"kind": "Deployment",
"name": "bao-notice",
"namespace": "bao-notice",
"status": "Synced",
"version": "v1"
},
{
"group": "networking.k8s.io",
"kind": "Ingress",
"name": "bao-notice",
"namespace": "bao-notice",
"status": "Synced",
"version": "v1"
},
{
"group": "networking.k8s.io",
"kind": "Ingress",
"name": "bao-notice-http-redirect",
"namespace": "bao-notice",
"status": "Synced",
"version": "v1"
},
{
"group": "networking.k8s.io",
"kind": "NetworkPolicy",
"name": "bao-notice-acme-solver",
"namespace": "bao-notice",
"status": "Synced",
"version": "v1"
},
{
"group": "networking.k8s.io",
"kind": "NetworkPolicy",
"name": "bao-notice-isolation",
"namespace": "bao-notice",
"status": "Synced",
"version": "v1"
},
{
"group": "traefik.io",
"kind": "Middleware",
"name": "redirect-https",
"namespace": "bao-notice",
"status": "Synced",
"version": "v1alpha1"
}
]
},
{
"name": "eso-token-renewer",
"sync": {
"comparedTo": {
"destination": {
"namespace": "external-secrets",
"server": "https://kubernetes.default.svc"
},
"source": {
"path": "argocd/platform-addons/eso-token-renewer",
"repoURL": "https://forgejo.coulomb.social/coulomb/railiance-platform.git",
"targetRevision": "b2ebe108495c4d06ebb6ca33893ea4a2a23c9725"
}
},
"revision": "b2ebe108495c4d06ebb6ca33893ea4a2a23c9725",
"status": "Synced"
},
"health": {
"lastTransitionTime": "2026-09-23T22:38:19Z",
"status": "Healthy"
},
"reconciledAt": "2026-09-27T16:50:17Z",
"operation": {
"phase": "Succeeded",
"startedAt": "2026-09-23T22:38:19Z",
"finishedAt": "2026-09-23T22:38:19Z"
},
"automated": null,
"conditions": [
{
"lastTransitionTime": "2026-09-23T22:37:58Z",
"message": "Application has 28 orphaned resources",
"type": "OrphanedResourceWarning"
}
],
"resources": [
{
"kind": "ConfigMap",
"name": "eso-token-renewer-worker-5c86dt7fm7",
"namespace": "external-secrets",
"status": "Synced",
"version": "v1"
},
{
"kind": "ServiceAccount",
"name": "eso-token-renewer",
"namespace": "external-secrets",
"status": "Synced",
"version": "v1"
},
{
"group": "batch",
"kind": "CronJob",
"name": "eso-token-renewer",
"namespace": "external-secrets",
"status": "Synced",
"version": "v1"
}
]
},
{
"name": "openbao-secretstore",
"sync": {
"comparedTo": {
"destination": {
"namespace": "external-secrets",
"server": "https://kubernetes.default.svc"
},
"source": {
"path": "argocd/platform-addons/openbao-secretstore",
"repoURL": "https://forgejo.coulomb.social/coulomb/railiance-platform.git",
"targetRevision": "d2dbc19c254247652c49fda8721c80d53bca206a"
}
},
"revision": "d2dbc19c254247652c49fda8721c80d53bca206a",
"status": "Synced"
},
"health": {
"lastTransitionTime": "2026-09-21T17:09:00Z",
"status": "Healthy"
},
"reconciledAt": "2026-09-27T16:52:14Z",
"operation": {
"phase": "Succeeded",
"startedAt": "2026-09-21T17:09:16Z",
"finishedAt": "2026-09-21T17:09:18Z"
},
"automated": null,
"conditions": [
{
"lastTransitionTime": "2026-09-23T18:14:58Z",
"message": "Application has 28 orphaned resources",
"type": "OrphanedResourceWarning"
}
],
"resources": [
{
"group": "external-secrets.io",
"kind": "ClusterSecretStore",
"name": "openbao",
"status": "Synced",
"version": "v1beta1"
}
]
},
{
"name": "railiance-apps-root",
"sync": {
"comparedTo": {
"destination": {
"namespace": "argocd",
"server": "https://kubernetes.default.svc"
},
"source": {
"path": "argocd/railiance01/applications",
"repoURL": "https://forgejo.coulomb.social/coulomb/railiance-platform.git",
"targetRevision": "main"
}
},
"revision": "5781d34b3b9a6e3779b913de200f0eaf63cabacd",
"status": "Synced"
},
"health": {
"lastTransitionTime": "2026-09-21T17:08:13Z",
"status": "Healthy"
},
"reconciledAt": "2026-09-27T16:51:31Z",
"operation": {
"phase": "Succeeded",
"startedAt": "2026-09-27T14:05:47Z",
"finishedAt": "2026-09-27T14:05:49Z"
},
"automated": null,
"conditions": [
{
"lastTransitionTime": "2026-09-21T17:08:13Z",
"message": "Application has 1 orphaned resources",
"type": "OrphanedResourceWarning"
}
],
"resources": [
{
"group": "argoproj.io",
"kind": "Application",
"name": "activity-core",
"namespace": "argocd",
"status": "Synced",
"version": "v1alpha1"
},
{
"group": "argoproj.io",
"kind": "Application",
"name": "bao-notice",
"namespace": "argocd",
"status": "Synced",
"syncWave": 10,
"version": "v1alpha1"
},
{
"group": "argoproj.io",
"kind": "Application",
"name": "eso-token-renewer",
"namespace": "argocd",
"status": "Synced",
"syncWave": 2,
"version": "v1alpha1"
},
{
"group": "argoproj.io",
"kind": "Application",
"name": "openbao-secretstore",
"namespace": "argocd",
"status": "Synced",
"syncWave": 1,
"version": "v1alpha1"
},
{
"group": "argoproj.io",
"kind": "Application",
"name": "target-revenue",
"namespace": "argocd",
"status": "Synced",
"syncWave": 10,
"version": "v1alpha1"
}
]
},
{
"name": "target-revenue",
"sync": {
"comparedTo": {
"destination": {
"namespace": "target-revenue",
"server": "https://kubernetes.default.svc"
},
"source": {
"path": "k8s/railiance",
"repoURL": "https://forgejo.coulomb.social/coulomb/target-revenue.git",
"targetRevision": "f1109d54eeda9f187daa215cf1c7163610d35d0a"
}
},
"revision": "f1109d54eeda9f187daa215cf1c7163610d35d0a",
"status": "Synced"
},
"health": {
"lastTransitionTime": "2026-09-21T17:13:49Z",
"status": "Healthy"
},
"reconciledAt": "2026-09-27T16:50:17Z",
"operation": {
"phase": "Succeeded",
"startedAt": "2026-09-21T17:14:01Z",
"finishedAt": "2026-09-21T17:14:10Z"
},
"automated": null,
"conditions": [
{
"lastTransitionTime": "2026-09-21T17:13:49Z",
"message": "Application has 5 orphaned resources",
"type": "OrphanedResourceWarning"
}
],
"resources": [
{
"kind": "Service",
"name": "target-revenue",
"namespace": "target-revenue",
"status": "Synced",
"version": "v1"
},
{
"group": "apps",
"kind": "Deployment",
"name": "target-revenue",
"namespace": "target-revenue",
"status": "Synced",
"syncWave": 3,
"version": "v1"
},
{
"group": "external-secrets.io",
"kind": "ExternalSecret",
"name": "target-revenue-runtime",
"namespace": "target-revenue",
"status": "Synced",
"version": "v1beta1"
},
{
"group": "networking.k8s.io",
"kind": "Ingress",
"name": "target-revenue",
"namespace": "target-revenue",
"status": "Synced",
"syncWave": 4,
"version": "v1"
},
{
"group": "postgresql.cnpg.io",
"kind": "Cluster",
"name": "target-revenue-pg",
"namespace": "target-revenue",
"status": "Synced",
"syncWave": -2,
"version": "v1"
}
]
}
],
"coulombcore_inventory": "unavailable: Kubernetes Unauthorized, including explicit local k3s kubeconfig; no credential or controller mutation"
}

View file

@ -0,0 +1,12 @@
{
"observed_at": "2026-09-27T16:59:10.112128+00:00",
"chart": "0.16.1",
"draft_sha256": "a0b349cddd90aa707f080efd8556c8379d5b178d93b4c9e408f3b7f439bbf064",
"render_sha256": "55236645afc9f2d9d376c28e73aed70bd31727d0e6f166664b3ebd568d5f136f",
"objects": 39,
"protected_crds": 20,
"server_diff_exit_code": 1,
"applied": false,
"diff_summary": "Exactly 20 CRD metadata annotation additions; no spec, workload or other object changes",
"diff_sha256": "3c34ad673fc374442adc13eeaf17c5216fce7b904103e63014d10e4ba6272965"
}

View file

@ -0,0 +1,33 @@
{
"schema": "platform.incident-owner-return.v1",
"reviewed_at": "2026-09-27T17:01:20.402024+00:00",
"task": "RPF-WP-0027-T05",
"sources": [
{
"repository": "net-kingdom",
"path": "workplans/NK-WP-0033-keycape-secret-exposure-rotation.md",
"commit": "1da6e5457ad86fff1173bd2ff6174d70719b487d",
"sha256": "8268aa86f7fa9f2a3848adfe2dc3022bcf2e9dc9197328993445f4138a40e513"
},
{
"repository": "key-cape",
"path": "workplans/KEY-WP-0011-live-secret-exposure-recovery.md",
"commit": "6a996bd71e5e36d7483e7a79b3301bd895907644",
"sha256": "3c5458180fe81a04e357f8db737e4287f79640b5ab02682bcadc595b73f846cd"
}
],
"operator_ruling_date": "2026-09-23",
"operator": "Bernd Worsch",
"owner_check": "reconcile-lldap-resolver-live.sh --check --predecessor-unavailable",
"owner_receipt": {
"resolver_lookup": "PASS",
"privacyidea_mfa": "PASS",
"predecessor_denial": "NOT-PROVEN",
"readiness": "PASS",
"health": "PASS",
"cleanup": "PASS"
},
"disposition": "Operator explicitly accepted the August 27 observations; unrecoverable predecessor is not a required new test. NetKingdom marks incident closed.",
"live_action_performed_in_this_review": false,
"custody_handoff_complete": false
}

View file

@ -0,0 +1,192 @@
{
"observation": {
"schema": "railiance-platform.observation.v1",
"cluster_uid": "a553c742-0115-43d4-99a4-a5ca56fe0786",
"captured_at": "2026-09-27T16:53:44.173824+00:00",
"signals": {
"apps-pg.ready": {
"result": "pass",
"observed_at": "2026-09-27T16:53:28.888062+00:00"
},
"apps-pg.backup": {
"result": "pass",
"observed_at": "2026-09-27T02:15:16Z"
},
"apps-pg.wal": {
"result": "pass",
"observed_at": "2026-09-27T16:53:28.888098+00:00"
},
"apps-pg.headroom": {
"result": "pass",
"observed_at": "2026-09-27T16:53:16Z"
},
"platform-pg.ready": {
"result": "pass",
"observed_at": "2026-09-27T16:53:32.163508+00:00"
},
"platform-pg.backup": {
"result": "pass",
"observed_at": "2026-09-27T02:15:18Z"
},
"platform-pg.wal": {
"result": "pass",
"observed_at": "2026-09-27T16:53:32.163550+00:00"
},
"platform-pg.headroom": {
"result": "pass",
"observed_at": "2026-09-27T16:53:32Z"
},
"platform-pg-2.ready": {
"result": "pass",
"observed_at": "2026-09-27T16:53:35.169803+00:00"
},
"platform-pg-2.backup": {
"result": "pass",
"observed_at": "2026-09-27T02:15:14Z"
},
"platform-pg-2.wal": {
"result": "pass",
"observed_at": "2026-09-27T16:53:35.169827+00:00"
},
"platform-pg-2.headroom": {
"result": "pass",
"observed_at": "2026-09-27T16:53:22Z"
},
"openbao.seal": {
"result": "pass",
"observed_at": "2026-09-27T16:53:42.333471+00:00"
},
"eso.ready": {
"result": "pass",
"observed_at": "2026-09-27T16:53:43.311315+00:00"
},
"eso.refresh": {
"result": "pass",
"observed_at": "2026-09-27T16:06:45Z"
},
"eso.token-renewal": {
"result": "pass",
"observed_at": "2026-09-27T02:40:09Z"
},
"apps-pg.restore": {
"result": "pass",
"observed_at": "2026-09-05T22:30:45.208512+00:00"
},
"forgejo-db.restore": {
"result": "pass",
"observed_at": "2026-09-05T22:55:54.893587+00:00"
},
"openbao.snapshot": {
"result": "pass",
"observed_at": "2026-08-22T22:29:21Z"
}
}
},
"evaluation": {
"schema": "railiance-platform.assurance-signal.v1",
"cluster_uid": "a553c742-0115-43d4-99a4-a5ca56fe0786",
"evaluated_at": "2026-09-27T16:53:44.173824+00:00",
"signals": {
"apps-pg.ready": {
"state": "healthy",
"owner": "railiance-platform"
},
"apps-pg.backup": {
"state": "healthy",
"owner": "railiance-platform"
},
"apps-pg.wal": {
"state": "healthy",
"owner": "railiance-platform"
},
"apps-pg.restore": {
"state": "healthy",
"owner": "railiance-platform"
},
"apps-pg.headroom": {
"state": "healthy",
"owner": "railiance-platform"
},
"platform-pg.ready": {
"state": "healthy",
"owner": "rapp-postgres"
},
"platform-pg.backup": {
"state": "healthy",
"owner": "rapp-postgres"
},
"platform-pg.wal": {
"state": "healthy",
"owner": "rapp-postgres"
},
"platform-pg.restore": {
"state": "missing",
"owner": "rapp-postgres"
},
"platform-pg.headroom": {
"state": "healthy",
"owner": "rapp-postgres"
},
"platform-pg-2.ready": {
"state": "healthy",
"owner": "rapp-postgres"
},
"platform-pg-2.backup": {
"state": "healthy",
"owner": "rapp-postgres"
},
"platform-pg-2.wal": {
"state": "healthy",
"owner": "rapp-postgres"
},
"platform-pg-2.restore": {
"state": "missing",
"owner": "rapp-postgres"
},
"platform-pg-2.headroom": {
"state": "healthy",
"owner": "rapp-postgres"
},
"openbao.seal": {
"state": "healthy",
"owner": "railiance-platform"
},
"openbao.snapshot": {
"state": "stale",
"owner": "railiance-platform"
},
"openbao.restore": {
"state": "missing",
"owner": "railiance-platform"
},
"offsite.upload": {
"state": "missing",
"owner": "railiance-platform"
},
"offsite.restore": {
"state": "missing",
"owner": "railiance-platform"
},
"eso.ready": {
"state": "healthy",
"owner": "railiance-platform"
},
"eso.refresh": {
"state": "healthy",
"owner": "railiance-platform"
},
"forgejo-db.restore": {
"state": "healthy",
"owner": "railiance-platform"
},
"eso.token-renewal": {
"state": "healthy",
"owner": "railiance-platform"
}
},
"transport": "unmonitored",
"guarantees": "unsupported",
"threshold_status": "local-diagnostic-only",
"healthy": false
}
}

View file

@ -3,8 +3,8 @@
Historical resolver lanes: **draft / blocked**. Historical resolver lanes: **draft / blocked**.
KeyCape factor service lane: **active**, established and verified 2026-09-13 (below). KeyCape factor service lane: **active**, established and verified 2026-09-13 (below).
Incident: `KEYCAPE-EXPOSURE-20260823-01` Incident: `KEYCAPE-EXPOSURE-20260823-01`
Consumer procedure: NetKingdom `NK-WP-0033`, resolver reconciliation revision Consumer procedure: NetKingdom `NK-WP-0033`; latest attended check used
`eec7007` / checkout `f2e578c` checkout `6096c395` (script `4a38511`) on 2026-09-23.
This document defines the Railiance-side contract without containing or This document defines the Railiance-side contract without containing or
deriving any credential value. It is not an authorization to fetch, export, deriving any credential value. It is not an authorization to fetch, export,
@ -49,6 +49,17 @@ Operator decision 2026-09-15: leave both historical resolver lanes blocked.
Do not invent mount, path, or field names. The KeyCape factor service lane Do not invent mount, path, or field names. The KeyCape factor service lane
below is separate and does not close this gate. below is separate and does not close this gate.
September 27 evidence review: NetKingdom's September 23 operator ruling and
green attended `--check --predecessor-unavailable` receipt close the incident
verification obligation (RPF-WP-0027-T05). The predecessor remains NOT-PROVEN;
the operator explicitly accepted its unavailable disposition. The owner records
human custody at `operators/lldap/admin` and `operators/privacyidea/pi-admin`,
including KV v2 LLDAP version 1 and withheld delete under `operator-custody`.
These are confirmed owner coordinates, but they do not by themselves establish
either historical route's complete field, auth, expiry and handoff contract.
T03/T06 and the non-resolvable historical routes therefore remain blocked.
See `docs/evidence/2026-09-27-keycape-incident-owner-return.json`.
## KeyCape factor service lane — authorized setup, 2026-09-13 ## KeyCape factor service lane — authorized setup, 2026-09-13
RPF-WP-0040 / CCR-2026-0023 establish a new dedicated service lane. The user, RPF-WP-0040 / CCR-2026-0023 establish a new dedicated service lane. The user,

View file

@ -61,13 +61,12 @@ CronJob status timestamps only. It fails when a newer scheduled run has not
succeeded within an hour, and it goes stale after 36h. The tokens lapse after succeeded within an hour, and it goes stale after 36h. The tokens lapse after
7 days without renewal. 7 days without renewal.
The following remain missing until a native value-safe adapter and acceptance Current gaps include accepted platform-pg/platform-pg-2 and OpenBao isolated
exist: validated isolated restore receipts, restore samples, fresh OpenBao snapshots, and dated full-archive receipts.
OpenBao snapshot/restore proof, and offsite upload/restore receipts. Missing Missing evidence is not inferred healthy from pod readiness. RTEL-WP-0002
adapters are not inferred healthy from pod readiness. The local producer is implements the Q2 reference contract and local delivery tests; its T04 still
not the Q2 standard; railiance-telemetry has no implemented receiving contract owns production mapping, recipient and controlled failure/absence acceptance.
in the reviewed checkout. Integration, routing and scheduled delivery remain Integration, routing and scheduled delivery remain T04 here.
T04, and no notification was sent during implementation.
## Service records and evidence inventory ## Service records and evidence inventory
@ -87,7 +86,7 @@ provider invalidation/replacement recovery for the shared offsite lane.
| OpenBao snapshot | WARDEN-WP-0027 preparation receipt, 2026-08-23 | Snapshot/encrypted off-host preparation, not isolated restore | | OpenBao snapshot | WARDEN-WP-0027 preparation receipt, 2026-08-23 | Snapshot/encrypted off-host preparation, not isolated restore |
| OpenBao restore | Existing `openbao-validate-restore-evidence.sh` and package procedure | Example receipt cannot pass as a fresh execution | | OpenBao restore | Existing `openbao-validate-restore-evidence.sh` and package procedure | Example receipt cannot pass as a fresh execution |
| Recovery exercise | RPF-WP-0015-T02/T03 | Separate windows, synthetic driver/quorum and abort operator required | | Recovery exercise | RPF-WP-0015-T02/T03 | Separate windows, synthetic driver/quorum and abort operator required |
| Logical/Forgejo offsite | activity-core backup definitions, existing helper/runbooks | No new upload/restore performed; RPF-WP-0029 remains open | | Logical/Forgejo offsite | activity-core backup definitions, existing helper/runbooks | WP-0029 is finished; WP-0038 retains recurring primary/secondary activation |
## Admission and disclosure drift ## Admission and disclosure drift
@ -134,8 +133,26 @@ Decryption now retains `platform.forgejo-primary-decryption.v1` and its own
operation times; older decryption receipts used the transfer schema through an operation times; older decryption receipts used the transfer schema through an
overwrite bug. The restore tool explicitly accepts both forms, with verified overwrite bug. The restore tool explicitly accepts both forms, with verified
hash/decryption flags. Existing historical receipts are unchanged. These producer hash/decryption flags. Existing historical receipts are unchanged. These producer
fixes enable future dated archive evidence; automatic archive adapters, fresh fixes enable dated archive evidence. The full primary archive adapter is now
end-to-end receipts and recurring execution are still pending. implemented; fresh end-to-end receipts and recurring execution remain pending.
For `offsite.upload`, add a reviewed entry with `signal`, `path` and `sha256`
to the recovery index. It must name a full-profile Scaleway archive transfer
with completed multipart upload, version-pinned GET, matching byte counts/hash,
the application-archive destination and ordered timezone-aware timestamps.
For `offsite.restore`, the entry additionally names `decryption` and `transfer`,
each with `path` and `sha256`. The restore must attest database import, application
health, repository verification, all package blobs and successful scratch cleanup.
The receipt hashes, ciphertext identity, destination, profile and operation order
must match across all three receipts. Only the distinct decryption schema is
accepted for automatic assurance. Essentials and Nextcloud-only proofs cannot
substitute for this primary full-application recovery signal. Other supported
services still need their own evidence; one Forgejo receipt does not close T03.
No historical index entry was added: the September 6 archive receipts lack
operation timestamps. They remain manual evidence. Tests use synthetic receipt
chains and prove expiry, provenance rejection and rejection of the real undated
receipt; those fixtures do not assert a new live recovery.
The OpenBao snapshot adapter also accepts the reviewed, hash-pinned receipt in The OpenBao snapshot adapter also accepts the reviewed, hash-pinned receipt in
`reviews/`. It requires the expected source cluster identity, encrypted off-host `reviews/`. It requires the expected source cluster identity, encrypted off-host

View file

@ -0,0 +1,73 @@
# Loose-end review — September 27, 2026
Reviewed every unfinished source workplan, terminal/archived task states,
the empty unread inbox and human-needed Hub records, with current owner sources
and read-only cluster evidence. No new workplan or task was created. Historical
`cancelled` task statuses in the superseded baseline are terminal, not open work.
Retired Hub aliases are not additional source obligations.
## Completed work
- Closed RPF-WP-0027-T05 from the newer NK-WP-0033 operator ruling and attended
September 23 receipt. KEY-WP-0011 already records the other credential classes.
Predecessor denial remains NOT-PROVEN; the operator accepted the unavailable
predecessor's disposition explicitly. Source commits and hashes are in
`docs/evidence/2026-09-27-keycape-incident-owner-return.json`. No rotation was
repeated. Historical custody routing still needs its complete accepted contract.
- Implemented the WP-0036-T03 full-primary archive assurance adapter. It requires
an ordered, hash-bound transfer/decryption/recovery chain, verified versioned
download, full profile, application/database/repository/package proof and cleanup.
New primary receipts preserve the archive profile. Undated historical receipts,
essentials-only restores, failures and mismatched chains cannot pass. No fresh
live archive receipt or recurring cadence is claimed by these fixture tests.
- Completed WP-0044-T06 repository preparation: all 20 ESO CRDs receive
`Prune=false,Delete=false` in the inactive draft. Chart 0.16.1 renders 39 objects;
server-side diff contains exactly those 20 annotation additions and no spec
changes. Evidence: `docs/evidence/2026-09-27-eso-adoption-preparation.json`.
- Prepared the concrete phase C retirement sequence under WP-0044-T08 at
`docs/argocd-coulombcore-retirement.md`. Live inventory still cannot finish:
coulombcore SSH succeeds but Kubernetes returns Unauthorized, including with
the explicit host-local k3s kubeconfig. No credential rotation or restart was
attempted as a workaround.
- Corrected the current index, custody incident status and stale assurance
documentation. Platform accepts policy-nexus's proposed chart/values ownership
split in principle; railiance-apps still must accept and publish its values
path/revision. No cross-owner agreement is fabricated.
## Live observations and remaining gates
`docs/evidence/2026-09-27-argocd-loose-end-status.json` records six Applications.
Openbao-secretstore and target-revenue are Synced/Healthy, with September 21
healthy transition times; their original observation periods have elapsed.
Their remaining promotion/owner gates now show `wait` instead of `progress`.
All 25 ClusterSecretStores are Valid and issue-core-runtime is SecretSynced.
Activity-core is Synced/Healthy; its required observation cannot end before
September 28 at 16:06:22 Berlin. Its scoped broker/admission remains separate.
Bao-notice is Healthy but OutOfSync; this review did not sync unrelated apps.
`docs/evidence/2026-09-27-service-assurance.json` preserves the current observation
and evaluation: 18 healthy, five missing and one stale signal. The stale receipt
is the old OpenBao snapshot. Missing signals are platform-pg/platform-pg-2 restore,
OpenBao restore and offsite upload/restore. The archive adapter is ready for fresh
reviewed receipts; the existing undated September 6 evidence is not re-dated.
Transport remains unmonitored. Q2 has a local contract and selected rapp-telemetry
package; RTEL-WP-0002-T04 still owns actual mapping/delivery/absence acceptance.
All eight unfinished workplans now explicitly say `blocked`; their 23 remaining
tasks say `wait`. They retain the concrete unblocks in their September 27 entries.
The open work is dominated by native owner acceptance, protected attended actions,
fresh recovery evidence, inventory access and the time-bound observation gate.
No blanket approval request, duplicate task, new schedule, credential read,
backup expiration, production rollout or owner message was introduced.
## Validation
The repository suite passed 418 tests with one skip before the additional three
profile-propagation cases. The final focused archive suite passed all 70 tests,
including those three cases. Admission matches the reviewed baseline; live metadata capture succeeds.
The assurance evaluator correctly exits nonzero for the disclosed missing/stale
evidence. ESO render assertions and the exact metadata-only diff passed.
Commit and Repo Manager synchronization receipts are recorded in the session
close progress event. The generated legacy aliases remain WP-0036-T06's scoped
projection-owner dependency and are not silently repaired by changing UUIDs.

View file

@ -8,6 +8,82 @@ from service_assurance import timestamp
ROOT = Path(__file__).resolve().parents[1] ROOT = Path(__file__).resolve().parents[1]
def pinned_receipt(entry, root):
path = (root / entry['path']).resolve()
if not path.is_relative_to((root / 'docs/evidence').resolve()):
raise ValueError('receipt outside evidence directory')
raw = path.read_bytes()
if hashlib.sha256(raw).hexdigest() != entry['sha256']:
raise ValueError('receipt drift')
return json.loads(raw)
def operation_times(receipt, now):
start, finish = (timestamp(receipt[key]) for key in ('started_at', 'finished_at'))
if not start <= finish <= now:
raise ValueError('invalid receipt chronology')
return start, finish
def primary_archive(receipt, now):
"""Accept only explicit full primary transfers, including a verified versioned GET."""
operation_times(receipt, now)
if (receipt['schema'] != 'platform.forgejo-primary-archive.v1'
or receipt['status'] != 'primary_fetched_pending_application_restore'
or receipt['stage'] != 'transfer_verified'
or receipt['archive_profile'] != 'full'
or not receipt['destination'].startswith(
's3://railiance-platform-pg-backup/platform-pg/application-archives/forgejo/')
or not re.fullmatch(r'[0-9a-f]{64}', receipt['ciphertext_sha256'])
or not all(receipt.get(key) is True for key in
('multipart_completed', 'version_pinned', 'download_hash_matches'))
or type(receipt['ciphertext_bytes']) is not int
or receipt['ciphertext_bytes'] <= 0
or any(type(receipt[key]) is not int or receipt[key] != receipt['ciphertext_bytes']
for key in ('uploaded_bytes', 'downloaded_bytes'))):
raise ValueError('primary archive not accepted')
def archive_signal(entry, receipt, now, root):
if entry['signal'] == 'offsite.upload':
primary_archive(receipt, now)
else:
start, _ = operation_times(receipt, now)
if (receipt['schema'] != 'platform.forgejo-isolated-restore.v1'
or receipt['status'] != 'restored'
or receipt['archive_profile'] != 'full'
or receipt['source_provider'] != 'Scaleway'
or receipt['stage'] != 'package_blob_recovery'
or not all(receipt.get(key) is True for key in
('database_import', 'application_health', 'cleanup'))
or not receipt['repositories_verified']
or type(receipt['package_blobs_verified']) is not int
or receipt['package_blobs_verified'] < 0
or type(receipt['database_counts']['package_blobs']) is not int
or receipt['package_blobs_verified'] != receipt['database_counts']['package_blobs']):
raise ValueError('full application recovery not accepted')
decryption = pinned_receipt(entry['decryption'], root)
transfer = pinned_receipt(entry['transfer'], root)
primary_archive(transfer, now)
decrypt_start, decrypt_finish = operation_times(decryption, now)
if (decryption['schema'] != 'platform.forgejo-primary-decryption.v1'
or decryption['status'] != 'primary_fetched_pending_application_restore'
or decryption['archive_profile'] != 'full'
or decryption['decrypted'] is not True
or decryption['download_hash_matches'] is not True
or not re.fullmatch(r'[0-9a-f]{64}', decryption['plaintext_sha256'])
or receipt['transfer_receipt_sha256'] != entry['decryption']['sha256']
or decryption['transfer_receipt_sha256'] != entry['transfer']['sha256']
or not timestamp(transfer['finished_at']) <= decrypt_start <= decrypt_finish <= start
or receipt['offsite_artifact'] != transfer['destination']
or decryption['destination'] != transfer['destination']
or decryption['ciphertext_bytes'] != transfer['ciphertext_bytes']
or any(r['ciphertext_sha256'] != transfer['ciphertext_sha256']
for r in (receipt, decryption))):
raise ValueError('recovery provenance mismatch')
return {'result': 'pass', 'observed_at': receipt['finished_at']}
def recovery_signals(now, root=ROOT): def recovery_signals(now, root=ROOT):
index = json.loads((root / 'assurance/recovery-evidence.json').read_text()) index = json.loads((root / 'assurance/recovery-evidence.json').read_text())
if index['schema'] != 'railiance-platform.recovery-evidence.v1': if index['schema'] != 'railiance-platform.recovery-evidence.v1':
@ -15,10 +91,14 @@ def recovery_signals(now, root=ROOT):
signals = {} signals = {}
for entry in index['receipts']: for entry in index['receipts']:
signal = entry['signal'] signal = entry['signal']
if signal in signals or signal not in ('apps-pg.restore', 'forgejo-db.restore', 'openbao.snapshot'): if signal in signals or signal not in ('apps-pg.restore', 'forgejo-db.restore',
'openbao.snapshot', 'offsite.upload', 'offsite.restore'):
raise ValueError('unexpected recovery signal') raise ValueError('unexpected recovery signal')
sample = {'result': 'unavailable', 'observed_at': now.isoformat()} sample = {'result': 'unavailable', 'observed_at': now.isoformat()}
try: try:
if signal.startswith('offsite.'):
signals[signal] = archive_signal(entry, pinned_receipt(entry, root), now, root)
continue
path = (root / entry['path']).resolve() path = (root / entry['path']).resolve()
allowed = [root / 'docs/evidence'] allowed = [root / 'docs/evidence']
if signal == 'openbao.snapshot': if signal == 'openbao.snapshot':
@ -61,7 +141,7 @@ def recovery_signals(now, root=ROOT):
if not timestamp(receipt['started_at']) <= completed <= now: if not timestamp(receipt['started_at']) <= completed <= now:
raise ValueError('invalid receipt chronology') raise ValueError('invalid receipt chronology')
sample = {'result': 'pass', 'observed_at': receipt['finished_at']} sample = {'result': 'pass', 'observed_at': receipt['finished_at']}
except (OSError, ValueError, KeyError, TypeError): except (OSError, ValueError, KeyError, TypeError, AttributeError):
pass pass
signals[signal] = sample signals[signal] = sample
return signals return signals

View file

@ -77,10 +77,12 @@ def main():
receipt['stage']='source_validation';checkpoint() receipt['stage']='source_validation';checkpoint()
source=json.loads(a.source_receipt.read_text()) source=json.loads(a.source_receipt.read_text())
if (source.get('status')!='offsite_fetched_pending_isolated_restore' if (source.get('status')!='offsite_fetched_pending_isolated_restore'
or source.get('archive_profile', 'full') not in ('full', 'essentials')
or not a.source.name.endswith('.zip.age') or a.output.exists() or not a.source.name.endswith('.zip.age') or a.output.exists()
or not 0<a.source.stat().st_size<=20*1024**3 or not 0<a.source.stat().st_size<=20*1024**3
or source.get('ciphertext_sha256')!=digest(a.source)): or source.get('ciphertext_sha256')!=digest(a.source)):
raise ValueError('verified_source_required') raise ValueError('verified_source_required')
receipt['archive_profile'] = source.get('archive_profile', 'full')
receipt['stage']='cluster_identity';checkpoint() receipt['stage']='cluster_identity';checkpoint()
k=['kubectl','--kubeconfig',a.kubeconfig];assert_cluster(k) k=['kubectl','--kubeconfig',a.kubeconfig];assert_cluster(k)
# Existing governed databases delivery, captured only in process memory. # Existing governed databases delivery, captured only in process memory.

View file

@ -67,6 +67,27 @@ def test_primary_validation_failure_has_terminal_time_without_credentials(tmp_pa
times(json.loads(output.read_text())) times(json.loads(output.read_text()))
@pytest.mark.parametrize('profile', ['full', 'essentials', 'unsupported'])
def test_primary_preserves_profile_before_credential_access(tmp_path, profile):
encrypted = tmp_path/'source.zip.age'; encrypted.write_bytes(b'fixture')
source = tmp_path/'source.json'; output = tmp_path/'receipt.json'
source.write_text(json.dumps(dict(status='offsite_fetched_pending_isolated_restore',
archive_profile=profile, ciphertext_sha256=hashlib.sha256(b'fixture').hexdigest())))
with patch.object(sys, 'argv', ['primary', '--source', str(encrypted),
'--source-receipt', str(source), '--output', str(tmp_path/'download'),
'--receipt', str(output), '--kubeconfig', 'unused']), \
patch.object(primary, 'assert_cluster', side_effect=RuntimeError('stop before credentials')) as cluster:
assert primary.main() == 1
receipt = json.loads(output.read_text())
if profile == 'unsupported':
cluster.assert_not_called()
assert 'archive_profile' not in receipt
else:
cluster.assert_called_once()
assert receipt['archive_profile'] == profile
times(receipt)
def test_decryption_keeps_own_schema_and_times(tmp_path): def test_decryption_keeps_own_schema_and_times(tmp_path):
encrypted=tmp_path/'encrypted';encrypted.write_bytes(b'fixture') encrypted=tmp_path/'encrypted';encrypted.write_bytes(b'fixture')
source=tmp_path/'source.json';source.write_text(json.dumps(dict(schema='platform.forgejo-primary-archive.v1',status='primary_fetched_pending_application_restore',download_hash_matches=True,ciphertext_sha256=hashlib.sha256(b'fixture').hexdigest(),started_at='2000-01-01T00:00:00Z',finished_at='2000-01-01T00:00:01Z'))) source=tmp_path/'source.json';source.write_text(json.dumps(dict(schema='platform.forgejo-primary-archive.v1',status='primary_fetched_pending_application_restore',download_hash_matches=True,ciphertext_sha256=hashlib.sha256(b'fixture').hexdigest(),started_at='2000-01-01T00:00:00Z',finished_at='2000-01-01T00:00:01Z')))

View file

@ -0,0 +1,130 @@
"""Full primary recovery requires a complete, ordered, hash-bound receipt chain."""
import copy
from datetime import datetime, timezone, timedelta
import hashlib
import json
from pathlib import Path
import sys
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'scripts'))
from recovery_evidence import recovery_signals, ROOT
from service_assurance import evaluate
NOW = datetime(2026, 9, 27, 12, tzinfo=timezone.utc)
DESTINATION = 's3://railiance-platform-pg-backup/platform-pg/application-archives/forgejo/fixture.zip.age'
def chain():
transfer = dict(schema='platform.forgejo-primary-archive.v1',
status='primary_fetched_pending_application_restore', stage='transfer_verified',
started_at='2026-09-27T10:00:00Z', finished_at='2026-09-27T10:01:00Z',
archive_profile='full', destination=DESTINATION, ciphertext_sha256='a'*64,
ciphertext_bytes=123, uploaded_bytes=123, downloaded_bytes=123,
multipart_completed=True, version_pinned=True, download_hash_matches=True)
decryption = dict(schema='platform.forgejo-primary-decryption.v1',
status='primary_fetched_pending_application_restore',
started_at='2026-09-27T10:02:00Z', finished_at='2026-09-27T10:03:00Z',
archive_profile='full', destination=DESTINATION, ciphertext_sha256='a'*64,
ciphertext_bytes=123, decrypted=True, download_hash_matches=True,
plaintext_sha256='b'*64)
restore = dict(schema='platform.forgejo-isolated-restore.v1', status='restored',
started_at='2026-09-27T10:04:00Z', finished_at='2026-09-27T10:05:00Z',
archive_profile='full', source_provider='Scaleway', stage='package_blob_recovery',
offsite_artifact=DESTINATION, ciphertext_sha256='a'*64, database_import=True,
application_health=True, cleanup=True, repositories_verified=['fixture/repo'],
package_blobs_verified=3, database_counts={'package_blobs': 3})
return transfer, decryption, restore
def write_chain(root, receipts, broken_link=None):
def write(name, receipt):
relative = f'docs/evidence/{name}.json'
path = root / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(receipt))
return {'path': relative, 'sha256': hashlib.sha256(path.read_bytes()).hexdigest()}
transfer, decryption, restore = copy.deepcopy(receipts)
source = write('transfer', transfer)
decryption['transfer_receipt_sha256'] = source['sha256'] if broken_link != 'transfer' else '0'*64
decrypted = write('decryption', decryption)
restore['transfer_receipt_sha256'] = decrypted['sha256'] if broken_link != 'decryption' else '0'*64
recovered = write('restore', restore)
index = {'schema': 'railiance-platform.recovery-evidence.v1', 'receipts': [
{'signal': 'offsite.upload', **source},
{'signal': 'offsite.restore', **recovered, 'decryption': decrypted, 'transfer': source}]}
(root / 'assurance').mkdir(exist_ok=True)
(root / 'assurance/recovery-evidence.json').write_text(json.dumps(index))
return index
def test_complete_chain_preserves_times_and_expires(tmp_path):
write_chain(tmp_path, chain())
signals = recovery_signals(NOW, tmp_path)
assert signals['offsite.upload'] == {'result': 'pass', 'observed_at': '2026-09-27T10:01:00Z'}
assert signals['offsite.restore'] == {'result': 'pass', 'observed_at': '2026-09-27T10:05:00Z'}
later = NOW + timedelta(days=31)
contract = {'cluster_uid': 'fixture', 'capture_max_age_seconds': 900,
'signals': {s: {'owner': 'platform', 'max_age_seconds': 2592000} for s in signals}}
report = evaluate(contract, {'schema': 'railiance-platform.observation.v1',
'cluster_uid': 'fixture', 'captured_at': later.isoformat(),
'signals': recovery_signals(later, tmp_path)}, later)
assert all(s['state'] == 'stale' for s in report['signals'].values())
@pytest.mark.parametrize('index,key,value', [
(0, 'version_pinned', False), (0, 'download_hash_matches', False),
(0, 'downloaded_bytes', 122), (0, 'ciphertext_bytes', True),
(0, 'destination', 's3://other/fixture'), (0, 'archive_profile', 'essentials'),
(0, 'finished_at', '2027-01-01T00:00:00Z'),
(1, 'archive_profile', 'essentials'), (1, 'decrypted', False),
(1, 'destination', DESTINATION + 'other'), (1, 'ciphertext_sha256', 'c'*64),
(1, 'started_at', '2026-09-27T10:00:30Z'),
(1, 'schema', 'platform.forgejo-primary-archive.v1'),
(2, 'status', 'failed'), (2, 'cleanup', False), (2, 'application_health', False),
(2, 'database_import', False), (2, 'archive_profile', 'essentials'),
(2, 'source_provider', 'Nextcloud'), (2, 'offsite_artifact', DESTINATION + 'other'),
(2, 'package_blobs_verified', 2), (2, 'package_blobs_verified', True),
(2, 'repositories_verified', []), (2, 'finished_at', '2026-09-27T10:05:00'),
(2, 'started_at', '2026-09-27T10:02:30Z'),
])
def test_incomplete_or_wrong_recovery_never_passes(tmp_path, index, key, value):
receipts = chain()
receipts[index][key] = value
write_chain(tmp_path, receipts)
signals = recovery_signals(NOW, tmp_path)
assert signals['offsite.restore']['result'] == 'unavailable'
if index == 0:
assert signals['offsite.upload']['result'] == 'unavailable'
@pytest.mark.parametrize('link', ['transfer', 'decryption'])
def test_provenance_links_must_match_exact_bytes(tmp_path, link):
write_chain(tmp_path, chain(), broken_link=link)
assert recovery_signals(NOW, tmp_path)['offsite.restore']['result'] == 'unavailable'
@pytest.mark.parametrize('case', ['drift', 'missing', 'escape', 'missing_time', 'malformed'])
def test_untrusted_files_are_unavailable(tmp_path, case):
index = write_chain(tmp_path, chain())
entry = index['receipts'][1]
path = tmp_path / entry['path']
if case == 'drift': path.write_text('{}')
if case == 'missing': path.unlink()
if case == 'escape': entry['path'] = '/tmp/outside-recovery-evidence.json'
if case in ('missing_time', 'malformed'):
receipt = json.loads(path.read_text())
if case == 'missing_time': del receipt['finished_at']
else: receipt = []
path.write_text(json.dumps(receipt))
entry['sha256'] = hashlib.sha256(path.read_bytes()).hexdigest()
(tmp_path / 'assurance/recovery-evidence.json').write_text(json.dumps(index))
assert recovery_signals(NOW, tmp_path)['offsite.restore']['result'] == 'unavailable'
def test_historical_undated_receipt_is_not_freshened(tmp_path):
receipts = list(chain())
receipts[2] = json.loads((ROOT / 'docs/evidence/RPF-WP-0038-primary-archive-restore-2026-09-06.json').read_text())
write_chain(tmp_path, receipts)
assert recovery_signals(NOW, tmp_path)['offsite.restore']['result'] == 'unavailable'

View file

@ -1,37 +1,25 @@
# Current platform work # Current platform work
Reviewed 2026-09-15. Open workplans below; RPF-WP-0029 finished on predecessor Reviewed September 27, 2026. Eight workplans remain blocked with 23 waiting
unshare. Completed designs and implementations are under `archived/`; their IDs tasks. No active, ready or proposed source workplan remains. Completed work
and UUIDs are preserved. The number of blocked plans is not a count of missing retains its IDs and managed UUIDs, including plans under `archived/`.
implementations or independent incidents.
| Workplan | Purpose and next gate | S3 boundary | | Workplan | Remaining tasks | Next dependency |
| --- | --- | --- | | --- | --- | --- |
| [RPF-WP-0027](RPF-WP-0027-keycape-live-secret-exposure-recovery.md) | Incident custody and final evidence; accept NetKingdom's residual disposition and publish exact custody handoff | The bundle was already rotated. Provider/MFA reconciliation belongs to NetKingdom. | | [RPF-WP-0015](RPF-WP-0015-audit-core-custody-and-recovery-coordination.md) | T02/T03 | Fresh recovery windows, synthetic sender/abort owner and snapshot/quorum/access evidence |
| [RPF-WP-0027](RPF-WP-0027-keycape-live-secret-exposure-recovery.md) | T03/T06 | Complete historical resolver custody/routing contract; T05 closed from September 23 operator disposition and owner receipt |
| [RPF-WP-0035](RPF-WP-0035-credential-lane-implementation.md) | T02/T03/T06/T07/T08 | Exact service registration/custody, human memo acceptance, reader/disablement returns, governed npm migration and formerly-valid audit-bearer revocation proof |
| [RPF-WP-0036](RPF-WP-0036-platform-service-assurance.md) | T03/T04/T06 | Current recovery/cadence/custody evidence, Q2 production delivery and owner/projection acceptance; local archive adapter implemented |
| [RPF-WP-0038](RPF-WP-0038-forgejo-scaleway-primary-coverage.md) | T04 | Durable scheduled caller, canonical verified inventory and quota/retention gates; September 15 cutover/expiration hold persists |
| [RPF-WP-0043](RPF-WP-0043-policy-nexus-argocd-onboarding.md) | T02/T03/T04 | Cross-owner chart/values agreement, committed values and source access, zero-diff proof and adoption |
| [RPF-WP-0044](RPF-WP-0044-argocd-phase-b-adopt-existing-applications.md) | T03–T08 | Per-app promotion gates, issue-core credentials/ownership, target-revenue hook acceptance, ESO adoption/observation and readable old-controller inventory |
| [RPF-WP-0048](RPF-WP-0048-activity-core-gitops-adoption.md) | T02 | Healthy observation through September 28 at 16:06:22 Berlin plus bounded authenticated release broker/admission and rollback proof |
| [RPF-WP-0025](RPF-WP-0025-openbao-operator-only-access.md) | Public Ingress retracted 2026-09-15; private tunnel remains | DNS withdrawal with railiance-infra; rollback phrase still available. | Latest [review and evidence](../history/2026-09-27-loose-end-review.md).
| [RPF-WP-0015](RPF-WP-0015-audit-core-custody-and-recovery-coordination.md) | Two prepared recovery exercises; registered load driver exists; fresh sender/window/abort approvals and custody readiness remain | S3 contributes lease/ESO and snapshot/unseal proof; S1/S2 and audit-core execute their parts. | No new task or workplan was created. Existing completed credential reviews,
| [RPF-WP-0035](RPF-WP-0035-credential-lane-implementation.md) | Three remaining lanes: secrets-engine JWT, Fluid operator KV, KeyCape approval clients | Signing T04 is complete; T05 admission answered and awaiting owner approval plus a founder-attended window. | delivery, retention projection and rotations are not reopened by these waits.
| [RPF-WP-0036](RPF-WP-0036-platform-service-assurance.md) | Implemented local assurance/admission; waits for recurring restore evidence, Q2 reception and owner handoff | Run the assurance commands; live acceptance and external ownership remain gated. |
| [RPF-WP-0038](RPF-WP-0038-forgejo-scaleway-primary-coverage.md) | Native backup, full Scaleway archive recovery and 273 MiB Nextcloud essentials recovery verified; scheduled tier cutover remains | Bind recurring caller/dependencies, verified inventory, quota checks and separate owner retention. |
| [RPF-WP-0043](RPF-WP-0043-policy-nexus-argocd-onboarding.md) | Onboard production-approved rapp-policy-nexus to the ArgoCD lane by 2026-12-21; T01 confirms ArgoCD reconciles on railiance01 (unverified) | Plan only; adoption waits on the founder's go-ahead and rapp-policy-nexus's manifest decision. |
Source files are authoritative. The generated brief still contains retired
RPF-WP-0036-T02/T05/T07 are complete; T03/T04/T06 retain the remaining aliases and stale statuses; do not recreate work from it. Routine synchronization
acceptance gates. Treat credential exposure closure as the highest-priority attended uses the exact-commit Repo Manager receipt described in AGENTS.md. Legacy alias
work; task order does not combine or waive approvals. repair remains the scoped owner dependency in WP-0036-T06.
[Assessment and disposition of every plan](../history/2026-09-05-platform-intent-workplan-assessment.md)
and [generated current record index](../WORK-RECORDS.md).
Do not recreate completed workplans because an old Hub alias or generated brief
still shows them active. Use source IDs, and follow AGENTS.md for verified sync.
## Latest closure review
[2026-09-05 blocker review](../history/2026-09-05-blocked-workplan-closure-review.md):
At that review: 12 unfinished tasks across six genuine blocked plans.
The September 6 follow-up adds WP-0038 with one remaining full-archive task. All terminal plans have
only done/cancel tasks. Completed ESO recovery RPF-WP-0037 is archived.
Three retired Hub aliases still appear open; they are a derived-view defect,
not three more workplans. Use this file before the dated generated brief.

View file

@ -9,7 +9,7 @@ flavor: implementation
owner: codex owner: codex
topic_slug: railiance topic_slug: railiance
created: "2026-08-22" created: "2026-08-22"
updated: "2026-09-05" updated: "2026-09-27"
related: related:
- AUDIT-WP-0008 - AUDIT-WP-0008
- WH-ENG-20260822-AUDIT-E2-01 - WH-ENG-20260822-AUDIT-E2-01
@ -348,3 +348,7 @@ is no longer missing. T02 waits on its separately approved sender identity,
fresh bounded window, abort operator and live recovery receipt. Local driver fresh bounded window, abort operator and live recovery receipt. Local driver
success does not prove lease revocation/ESO recovery. T03 remains a separate success does not prove lease revocation/ESO recovery. T03 remains a separate
outage exercise; no historical window or terminal NO-GO may be reused. outage exercise; no historical window or terminal NO-GO may be reused.
## Loose-end review — 2026-09-27
T02/T03 remain waiting on fresh attended execution windows, named abort operators, current custody/quorum evidence and owner execution. The implemented load driver and prior procedure approvals do not supply a fresh live recovery receipt. No expired window was reused.

View file

@ -9,7 +9,7 @@ flavor: implementation
owner: codex owner: codex
topic_slug: railiance topic_slug: railiance
created: "2026-08-23" created: "2026-08-23"
updated: "2026-09-15" updated: "2026-09-27"
related: related:
- KEY-WP-0011 - KEY-WP-0011
origin: routed origin: routed
@ -110,7 +110,7 @@ and all T03 acknowledgements. No value may enter captured output.
```task ```task
id: RPF-WP-0027-T05 id: RPF-WP-0027-T05
status: wait status: done
priority: high priority: high
state_hub_task_id: "9cb5fa67-012a-58cf-bafb-e7c7d4f9782d" state_hub_task_id: "9cb5fa67-012a-58cf-bafb-e7c7d4f9782d"
``` ```
@ -172,3 +172,7 @@ and acceptable disposition. Overall incident closure remains open.
No new owner acceptance or coordination message is asserted by this review. No new owner acceptance or coordination message is asserted by this review.
Keep this incident separate from the new-lane queue; broad lane approval cannot Keep this incident separate from the new-lane queue; broad lane approval cannot
close an exposure. close an exposure.
## Loose-end review — 2026-09-27
T05 is now done by the newer owner record: NK-WP-0033 accepted the operator ruling on 2026-09-23 and recorded the green attended read-only resolver/MFA/health/cleanup receipt. Predecessor denial remains explicitly NOT-PROVEN, with the unavailable-predecessor disposition accepted by Bernd Worsch; it is not relabelled as a successful negative test. KEY-WP-0011 already closes the other credential-class rotation/recovery evidence. See `docs/evidence/2026-09-27-keycape-incident-owner-return.json`. T03/T06 still wait for the complete platform custody/routing handoff. The owner names operators/lldap/admin and operators/privacyidea/pi-admin, but does not supply the complete field/auth/expiry/handoff contract required by T06. The September 15 instruction to leave historical routing lanes blocked remains in force. No repeat rotation or predecessor recovery was attempted.

View file

@ -415,7 +415,7 @@ into this client-identity grant.
```task ```task
id: RPF-WP-0035-T08 id: RPF-WP-0035-T08
status: progress status: wait
priority: high priority: high
assignee: railiance-platform assignee: railiance-platform
needs_human: false needs_human: false
@ -696,3 +696,7 @@ T07 consumed the confirmed legacy consumer return. The stale value-comparison
ask is removed. WARDEN-WP-0037-T03's no-rotation hold stays in force until the ask is removed. WARDEN-WP-0037-T03's no-rotation hold stays in force until the
governed native migration is evidenced. Legacy destruction follows migration; governed native migration is evidenced. Legacy destruction follows migration;
neither source reconciliation nor the historical pilot is that evidence. neither source reconciliation nor the historical pilot is that evidence.
## Loose-end review — 2026-09-27
T08 now explicitly waits for formerly-valid audit-bearer revocation acceptance from AUDIT-WP-0009-T09/T11. Native delivery, producer checks and independent readback already passed; do not reopen login, seed or review gates. T02/T03/T06/T07 retain their latest exact service registration, signed-in human acceptance, owner delivery/disposition and governed-consumer migration dependencies. No credential value was read or changed.

View file

@ -8,7 +8,7 @@ status: blocked
flavor: implementation flavor: implementation
owner: codex owner: codex
created: "2026-09-05" created: "2026-09-05"
updated: "2026-09-06" updated: "2026-09-27"
state_hub_workstream_id: "ca639c3d-3a87-5fa4-ad13-6f2e014b0c84" state_hub_workstream_id: "ca639c3d-3a87-5fa4-ad13-6f2e014b0c84"
--- ---
@ -314,3 +314,7 @@ RTEL-WP-0002-T04: accepted package/runtime, authenticated execution, actual
operator recipient, cadence/storage/retention and independent receiver watchdog, operator recipient, cadence/storage/retention and independent receiver watchdog,
then acknowledged controlled delivery. Local test-inbox visibility is not that then acknowledged controlled delivery. Local test-inbox visibility is not that
live receipt; S3 remains unmonitored until acceptance. live receipt; S3 remains unmonitored until acceptance.
## Loose-end review — 2026-09-27
Completed the local full-primary archive assurance adapter under T03. It validates ordered transfer/decryption/restore receipts, exact provenance hashes, full profile, version-pinned verified download and application/database/repository/package/cleanup proof; historical undated receipts remain ineligible. Updated producer profile propagation and assurance documentation. Fresh live metadata reports 18 healthy signals, five missing recovery/upload signals and one stale OpenBao snapshot; transport remains unmonitored. T03/T04/T06 remain waiting on current recovery/cadence/custody proof, Q2 production delivery acceptance and owner/derived-record acceptance respectively. RTEL-WP-0002-T04 already selects rapp-telemetry; the stale claim that no receiver implementation/package exists is superseded. See `history/2026-09-27-loose-end-review.md`.

View file

@ -4,11 +4,11 @@ type: workplan
title: "Close Forgejo primary backup coverage on Scaleway" title: "Close Forgejo primary backup coverage on Scaleway"
domain: financials domain: financials
repo: railiance-platform repo: railiance-platform
status: active status: blocked
flavor: implementation flavor: implementation
owner: codex owner: codex
created: "2026-09-06" created: "2026-09-06"
updated: "2026-09-15" updated: "2026-09-27"
state_hub_workstream_id: "7beec1a7-aa82-5a36-9a66-6b60008a2455" state_hub_workstream_id: "7beec1a7-aa82-5a36-9a66-6b60008a2455"
--- ---
@ -72,7 +72,7 @@ removed. Evidence: `docs/evidence/forgejo-scaleway-restore-2026-09-06.json`.
```task ```task
id: RPF-WP-0038-T04 id: RPF-WP-0038-T04
status: progress status: wait
priority: high priority: high
state_hub_task_id: "a4807df0-ec96-58f1-9cbd-42b1dcde0d6f" state_hub_task_id: "a4807df0-ec96-58f1-9cbd-42b1dcde0d6f"
``` ```
@ -118,3 +118,7 @@ the existing durable caller, inventory, quota and retention gates.
**Operator decision, 2026-09-15:** do not expire retained backups or cut over **Operator decision, 2026-09-15:** do not expire retained backups or cut over
scheduled secondary delivery. The planner and attended executor stay idle until scheduled secondary delivery. The planner and attended executor stay idle until
durable caller/inventory/quota gates are closed. durable caller/inventory/quota gates are closed.
## Loose-end review — 2026-09-27
T04 now explicitly waits. Primary and essentials manual recovery are already proven. Durable scheduled caller/dependency binding, canonical verified inventory, quota/retention gates and owner activation are still absent. September 15 prohibits expiration and scheduled secondary cutover until those gates close; no retained backups or schedules were changed. New primary transfer receipts preserve archive_profile for the assurance adapter in WP-0036-T03.

View file

@ -4,12 +4,12 @@ type: workplan
title: "Onboard rapp-policy-nexus to the ArgoCD production lane" title: "Onboard rapp-policy-nexus to the ArgoCD production lane"
domain: financials domain: financials
repo: railiance-platform repo: railiance-platform
status: active status: blocked
flavor: planning flavor: planning
owner: railiance-platform owner: railiance-platform
topic_slug: railiance topic_slug: railiance
created: "2026-09-21" created: "2026-09-21"
updated: "2026-09-22" updated: "2026-09-27"
due: "2026-12-21" due: "2026-12-21"
related: [RPF-WP-0022] related: [RPF-WP-0022]
state_hub_workstream_id: "ec41a4bd-df18-5b07-9b63-ccb80d9f001c" state_hub_workstream_id: "ec41a4bd-df18-5b07-9b63-ccb80d9f001c"
@ -231,3 +231,7 @@ the inventory.
- **T05 done.** Gap declared and inventoried in - **T05 done.** Gap declared and inventoried in
`docs/direct-apply-gap-inventory.md`, with a per-group proposal for the `docs/direct-apply-gap-inventory.md`, with a per-group proposal for the
founder. No target changed. founder. No target changed.
## Loose-end review — 2026-09-27
The workplan is blocked on the source-owner agreement and its downstream adoption gates. RAPP-POLICY-NEXUS-WP-0002 proposes a multi-source Application: chart in rapp-policy-nexus and release values in railiance-apps. This supersedes the earlier platform-only suggestion to put production values in the package repository. The platform accepts that shape and the helm/policy-nexus path exception in principle; it still needs railiance-apps acceptance, an actual committed values path/revision, source access and a zero-diff render before T02/T03 can close. No owner acceptance or cross-repository change is inferred. T01 remains proven by current railiance01 Synced/Healthy evidence.

View file

@ -4,12 +4,12 @@ type: workplan
title: "ArgoCD phase B: adopt the four existing Applications on railiance01" title: "ArgoCD phase B: adopt the four existing Applications on railiance01"
domain: financials domain: financials
repo: railiance-platform repo: railiance-platform
status: active status: blocked
flavor: planning flavor: planning
owner: railiance-platform owner: railiance-platform
topic_slug: railiance topic_slug: railiance
created: "2026-09-21" created: "2026-09-21"
updated: "2026-09-21" updated: "2026-09-27"
related: [RPF-WP-0043, RPF-WP-0022] related: [RPF-WP-0043, RPF-WP-0022]
state_hub_workstream_id: "98140775-3b9a-5cf9-9af6-722502d487dc" state_hub_workstream_id: "98140775-3b9a-5cf9-9af6-722502d487dc"
--- ---
@ -185,7 +185,7 @@ Rollback:
```task ```task
id: RPF-WP-0044-T03 id: RPF-WP-0044-T03
status: progress status: wait
priority: high priority: high
state_hub_task_id: "6d5fc801-e361-579b-bcd8-6fe719a82e94" state_hub_task_id: "6d5fc801-e361-579b-bcd8-6fe719a82e94"
``` ```
@ -256,7 +256,7 @@ whitelist. Live check: `rapp-issue-core make verify-live`. Workload restore:
```task ```task
id: RPF-WP-0044-T05 id: RPF-WP-0044-T05
status: progress status: wait
priority: medium priority: medium
state_hub_task_id: "d418068a-6fb0-5416-aac5-d23c93924d9c" state_hub_task_id: "d418068a-6fb0-5416-aac5-d23c93924d9c"
``` ```
@ -317,7 +317,7 @@ railiance01 path is a live production change, and `RPF-WP-0043-T04`
```task ```task
id: RPF-WP-0044-T08 id: RPF-WP-0044-T08
status: todo status: wait
priority: low priority: low
state_hub_task_id: "55d1382f-5862-5321-a1c9-96767764ba43" state_hub_task_id: "55d1382f-5862-5321-a1c9-96767764ba43"
``` ```
@ -326,3 +326,7 @@ Planning only. Needs a read-only check of coulombcore's ArgoCD, outside this
session's scope. Under Option A, retiring it also removes `argocd/applications/`. session's scope. Under Option A, retiring it also removes `argocd/applications/`.
Also hand back to the cluster layer: the phase A install is not declared in Also hand back to the cluster layer: the phase A install is not declared in
any repository and its pods have no resource requests (BestEffort). any repository and its pods have no resource requests (BestEffort).
## Loose-end review — 2026-09-27
T03/T05 passed their elapsed healthy observation gate: current ArgoCD status is Synced/Healthy with last healthy transitions on September 21; openbao is Valid and issue-core-runtime is SecretSynced. They now wait on the remaining automation go-aheads and, for target-revenue, hook-owner acceptance. T06 local preparation is complete: the inactive ESO draft protects all 20 CRDs from prune/delete. The pinned 0.16.1 chart renders 39 objects; server-side diff is exactly the 20 protective metadata annotations, with no spec changes. All 25 ClusterSecretStores are Valid. T04 still needs the repository credential/production-owner contract; T07 depends on the per-app gates. T08 has a concrete retirement procedure at `docs/argocd-coulombcore-retirement.md`, but its required live inventory is blocked: SSH works and Kubernetes rejects both default and explicit local k3s kubeconfigs. No controller or workload was changed. Evidence: `docs/evidence/2026-09-27-argocd-loose-end-status.json` and `docs/evidence/2026-09-27-eso-adoption-preparation.json`.

View file

@ -4,7 +4,7 @@ type: workplan
title: "Adopt activity-core application runtime into railiance01 GitOps" title: "Adopt activity-core application runtime into railiance01 GitOps"
domain: financials domain: financials
repo: railiance-platform repo: railiance-platform
status: active status: blocked
owner: codex owner: codex
topic_slug: railiance topic_slug: railiance
created: "2026-09-27" created: "2026-09-27"
@ -130,3 +130,7 @@ were made by this probe. See docs/evidence/2026-09-27-digest-retention-release.j
New healthy observation start is 2026-09-27T14:06:22Z after worker rollout; New healthy observation start is 2026-09-27T14:06:22Z after worker rollout;
earliest eligibility is September 28 at 16:06:22 Europe/Berlin. Automation remains earliest eligibility is September 28 at 16:06:22 Europe/Berlin. Automation remains
off. T02 still owns the scoped broker/admission and observation requirements. off. T02 still owns the scoped broker/admission and observation requirements.
## Loose-end review — 2026-09-27
The workplan is blocked on T02. Current activity-core remains Synced/Healthy at a12f1169f9d130058ce767f5b26de0606997916c, with health transition 2026-09-27T14:06:22Z. Earliest observation eligibility remains September 28 at 16:06:22 Europe/Berlin. The authenticated bounded broker/admission and rollback proof remain owned jointly with ACTIVITY-WP-0041-T03. Elapsed time is not release-identity admission. T01/T03 remain done; no root automation, pruning or credential delegation was enabled.