activity-core/docs/platform-recovery-scheduling-plan.md

Platform recovery scheduling — scoped delivery plan

Reviewed 2026-10-03 for ACTIVITY-WP-0042. Offline implementation is authorized within the existing scheduling role by the user's task and explicit Emergent Security guidance. The runner contract remains a candidate for platform review; no new operation or live execution is admitted.

Scheduling boundary

Continue within activity-core's existing scheduling-only ownership: Temporal triggers, bounded dispatch, run correlation, missed-run visibility, and normalized result evidence. Railiance Platform retains recovery implementations, acceptance validators, cadence/freshness policy, destinations, retention, and credential custody. Existing ACT-ADR-007 operations remain unchanged.

Proceed with offline design and tests under C4/A2/A6. Admit each new mutating operation separately after its owner supplies the contract and ACT-ADR-007 is amended or superseded. No new generic shell executor, restore schedule, retention action, or production rollout follows from role acceptance.

What the investigation established

• The current definition activity-definitions/daily-cnpg-option-a-backup.md selects three logical Nextcloud targets at 02:30 UTC. It does not produce Scaleway physical recovery or Forgejo archive recovery evidence. Its old host kubeconfig names and RPO wording are not proof of current cluster routing or an accepted guarantee; review those before reusing the wiring.
• src/activity_core/bounded_operations.py permits exactly three operations. CNPG backup has one Temporal attempt because no durable remote idempotency receipt is available. New restore dispatch must not inherit retry defaults.
• The stale-token blocker was resolved under RPF-WP-0045. Read-only inspection on 2026-10-03 confirmed node IP 92.205.62.239, ClusterSecretStore openbao-activity-core: Ready=True/Valid, and ExternalSecret activity-core/actcore-backup-offsite: Ready=True/SecretSynced, last refresh 2026-10-03T09:29:02Z. No Secret object or value was read. This verifies the existing Nextcloud lane, not permission or credentials for a new restore.
• warden route show railiance-backup-offsite-lane --json names railiance-platform as credential owner and reports workload resolution unknown. Recovery escrow must stay outside the ordinary backup worker.
• Platform's scripts/verify_scaleway_primary_restore.py supports apps-pg and forgejo-db, checks the cluster/destination, restores into a scratch namespace, validates database contents, and attempts cleanup. Its CLI has no dry-run option, approval-reference input, or durable idempotency key. Do not register it directly as an unattended bounded operation.
• Platform's scripts/restore_forgejo_offsite_locally.py validates a transfer receipt and supports full/essentials profiles. Its archive/decryption input and local container execution need a separately scoped owner contract.
• RPF-WP-0038 retains the September 15 prohibition on backup expiration and scheduled secondary cutover pending caller/inventory/quota gates. It is not lifted by this proposal. RPF-WP-0036 retains cadence and evidence acceptance.
• Ops Warden's registry/flex-auth/production_registry_snapshot.json calls atm-backup-daily an "Example nightly automation actor" and associates it with workload backup-daily, but records resolution as absent. That is not evidence that activity-core owns or uses it. T03 remains independent.

Platform message 0638c13b-cf6f-4342-9714-dfb1b6c67fc7 already consents to the scheduling-only boundary and commits to supplying the credential route once the operation contract exists. It specifies per-run approval as the interim; there is no standing grant. This proposal resolves its obsolete token-expiry prerequisite and makes the remaining contract explicit.

First delivery slice

Start with a single manually triggered, disabled-by-default apps-pg physical recovery evidence run against Scaleway, using a platform-owned isolated runner. This is a proposal for the first target, not an expansion of the existing logical-backup job. Add recurring triggers only after one admitted run and cleanup receipt have passed. Forgejo archive tiers and retention remain later slices with separate input/receipt contracts.

Before coding a production adapter, the platform owner must supply:

Contract item	Required evidence
Fixed operation and implementation	Owner-reviewed version/digest; no caller-provided command or import path
Targets	Exact cluster identity, source apps-pg, scratch namespace policy; production never a restore destination
Admission	Explicit dry-run semantics and per-run authority checked by the executing enforcement point; an approval reference alone is insufficient
Time and capacity	Agreed timeout and CPU/memory/storage ceilings including cleanup; no new numbers inferred from historical tests
Credentials	Scoped workload route and recovery custody; no borrowed host-admin kubeconfig or escrow added to activity-core
Replay and overlap	Stable run/operation ID, durable owner receipt lookup and single-flight behavior; one dispatch attempt until retry safety is proven
Cleanup	Confirmed scratch removal; timeout or ambiguous result requires owner reconciliation before another mutation
Receipt	Schema/version, run ID, implementation revision, target identity, start/finish times, terminal result, validation and cleanup booleans, authoritative receipt hash/reference

The scheduler retains only allowlisted receipt fields. Raw subprocess output, SQL/application contents, archive locations, credentials and provider payloads stay out of State Hub evidence. Failed, missing, stale and unverified receipts must remain distinguishable. A scheduled fire or successful upload is not a successful restore.

Implementation and verification sequence

1. T01 is complete: the explicit user task and Emergent Security guidance authorize development within the existing scheduling boundary. The other decisions bundled in D2 remain unchanged.
2. Agree the fixed runner contract above with railiance-platform (T05).
3. Build offline adapter/admission tests within T02. Cover unknown operation, target drift, absent/expired/wrong-scope authority, duplicate dispatch, timeouts, malformed/mismatched/stale receipts, failed cleanup and secret canaries in child output. Use a fake owner runner; no live restore required.
4. Review the ADR change and exact credential/evidence contract before registering the operation. Keep definitions disabled pending release.
5. Platform owner performs an approved isolated canary. Record evidence and cleanup, then agree cadence/freshness/retention policy and rollout separately.
6. Enable the agreed Temporal schedule through the governed release path (ACTIVITY-WP-0041), and verify missed-run and failed-evidence reporting.

Independent identity question

Do not assign atm-backup-daily to activity-core from its name. Obtain an issuance/deployment record identifying the actual execution unit, or a founder ruling that it is an unused example to retire. Scheduling development does not need to claim this historical actor. No owner message has been sent by this review; the contract and identity questions are recorded as workplan tasks.

Candidate contract implementation

src/activity_core/recovery_contract.py provides a pure request parser and receipt validator; tests/test_recovery_contract.py exercises it offline. It deliberately has no executor, transport, credential access, or registration in the bounded-operation allowlist. It is not imported by the runtime workflow.

An apply request must name a per-run authority UUID, but UUID presence never proves permission. The platform enforcement point must verify scope, expiry and consumption before an effect. The receipt envelope checks only the owner's assertions and correlation; platform validators remain authoritative for backup contents and recovery correctness. Digests require verification against the owner evidence store by the future authenticated transport, not trust by format. The supplied freshness window must come from accepted owner policy.

A successful apply receipt requires validated recovery and completed cleanup; a dry-run may only describe a plan without recovery effects. Failed/rejected/ unknown results remain visible for reconciliation and do not trigger retries. Malformed input errors suppress raw validation values. This contract is an implementation candidate, not an assertion of platform-owner acceptance.