activity-core/workplans/ACTIVITY-WP-0042-platform-backup-restore-evidence-scheduling.md

id	type	title	domain	repo	status	owner	topic_slug	created	updated	state_hub_workstream_id
ACTIVITY-WP-0042	workplan	Scheduling role for recurring platform backup/restore evidence runs	infotech	activity-core	blocked	codex	activity-core	2026-10-03	2026-10-03	e1bfb5a4-e647-592d-883f-fe8728ad2f5f

Origin: State Hub message e3668cd9 from railiance-platform (RPF-WP-0036-T03, RPF-WP-0038-T04) proposing activity-core as execution owner for recurring backup/restore evidence runs. Scheduling/dispatch remains within the existing activity-core intent and the platform owner's scheduling-only consent. The user directed continuation under Emergent Security on 2026-10-03; offline contract development is complete; owner acceptance is pending. New recovery execution and live activation remain gated.

Facts from the repo (2026-10-03):

• daily-cnpg-option-a-backup already runs a cron-scheduled bounded operation (cnpg_option_a_backup, ACT-ADR-007) with state-hub-progress evidence. Its definition targets the Nextcloud lane and covers forgejo-db, net-kingdom-pg, state-hub-db; it does not cover apps-pg/platform-pg/platform-pg-2 or the Scaleway primary, and it never performs restores.
• ACT-ADR-007 allowlist is closed. A restore-evidence or Scaleway-backup run needs a new code-registered operation plus an ADR successor, tests, a credential route and an evidence contract. A definition string cannot add one.
• The old expired-token finding is superseded: RPF-WP-0045 replaced static tokens with Kubernetes auth. On 2026-10-03, read-only railiance01 inspection confirmed openbao-activity-core Ready/Valid and actcore-backup-offsite Ready/SecretSynced (refresh 09:29:02Z). This verifies the existing Nextcloud lane only; new recovery authority and credential scope remain unapproved.

Decide whether activity-core accepts the scheduling role

id: ACTIVITY-WP-0042-T01
status: done
needs_human: false
decision_id: 5ec92a02-cb0b-466d-9aaf-9da5ffc6c92d
blocking_reason: null
priority: high
state_hub_task_id: "2a368988-2611-596a-a847-82c048bd179e"

Founder decision. Options: accept scheduling only (when/what/where; S3 keeps validators, cadence, expiry and evidence), accept scheduling plus a bounded operation, or decline. Needs: restore execution stays with an S3-owned tool invoked as a fixed bounded operation; approval-engine authority supplied per run; no general executor behaviour.

2026-10-03 (CUST-WP-0075 inbox sweep): railiance-platform replied (message 0638c13b) asking activity-core to proceed under its conditions, because "roles tie to the source repo". That is the S3 owner's consent. It does not resolve this founder gate, so T01 stays waiting on D2 and the plan stays proposed. It also asks activity-core to declare atm-backup-daily, which remains T03's founder confirmation. activity-core replied with the same position.

2026-10-03: human gate routed to hub decision 89f1058f-33bc-47a8-8a49-0b44d426c67e (D2: ArgoCD owns production: go-ahead sequence for railiance-apps-root, issue-core, policy-nexus and activity-core automation) by CUST-WP-0075; resolving that decision raises C-40 here.

2026-10-03 superseding scope clarification: the user explicitly directed continuation under emergent-security/emergent-security-agent.md. Apply C4, A2 and A6 to the already-owned scheduling/dispatch role in INTENT.md, with platform owner consent in message 0638c13b. No new responsibility or runtime capability is being claimed (C3/A5). The redundant blanket role gate is closed; proceed with offline contract code/tests. Per-run authority remains an executing platform enforcement point's responsibility (C2/C9). This is an autonomous scope clarification, not a human answer to D2; D2's other gates and T03 remain.

Admit a bounded operation and credential lane

id: ACTIVITY-WP-0042-T02
status: wait
blocking_reason: "Offline contract is implemented; operation admission, credential scope and live proof wait on platform acceptance in T05."
depends_on: [ACTIVITY-WP-0042-T01, ACTIVITY-WP-0042-T05]
priority: medium
state_hub_task_id: "250c6e8e-7281-5dec-9a92-06081b8c96bb"

Offline contract development is authorized by the explicit task and existing scheduling scope. Actual operation admission waits on T05. Any new local bounded operation still requires a successor to ACT-ADR-007, registry entry (target bounds, timeout, idempotency, dry_run explicit), tests, credential route via OpenBao (owner railiance-platform; existing store health verified), and non-secret evidence mode. Scheduled caller satisfies RPF-WP-0038-T04 only once recurring tiered scheduling and retention are accepted by their owners.

Offline candidate implemented in src/activity_core/recovery_contract.py: closed request/receipt envelopes, explicit mode, run/authority-reference correlation, owner-supplied freshness window, cleanup proof and safe errors. No runner call, credential acquisition or allowlist change. Owner contract acceptance and live proof remain incomplete; T02 is not done.

Answer ops-warden on atm-backup-daily

id: ACTIVITY-WP-0042-T03
status: done
needs_human: false
blocking_reason: null
priority: low
state_hub_task_id: "e1c313dc-9a94-5b7c-b406-604b700bf3eb"

WARDEN-WP-0040 asks which owner declares workload resolution backup-daily for atm-backup-daily. activity-core has no atm-* identity of its own in this repo and no evidence that the actor was minted for its worker. Declare only if the founder confirms activity-core's backup worker is that unit; otherwise the declaring owner is whoever holds the actor's issuance (railiance-platform was also asked). Never infer membership.

2026-10-03: human gate routed to hub decision 89f1058f-33bc-47a8-8a49-0b44d426c67e (D2: ArgoCD owns production: go-ahead sequence for railiance-apps-root, issue-core, policy-nexus and activity-core automation) by CUST-WP-0075; resolving that decision raises C-40 here.

2026-10-09: done. The founder confirmed on 2026-10-08, in an ops-warden session (hub msg 49292aca), that the daily-cnpg-option-a-backup worker is the atm-backup-daily unit. Classification: ordinary, not z2-continuity. The confirmation covers T03 only and does not resolve D2 (89f1058f). The decision link is removed here because D2 still gates other tasks. Published declarations/backup-daily/tenancy.yaml (workload backup-daily; bindings ssh-cert atm-backup-daily and SA activity-core:default). The founder chose membership z2-protected with context confidential/high/M1, so admission is expected to resolve below-floor (effective unknown) until M2 evidence exists. This is a recorded gap, not a claim of admission. Validated with the net-kingdom tenancy-posture validator. ops-warden compiles it under WARDEN-WP-0040-T02.

Prepare the concrete scheduling and runner proposal

id: ACTIVITY-WP-0042-T04
status: done
priority: high
state_hub_task_id: "859e8857-ea51-5147-bd3a-70acac1426dc"

User requested investigation and forward progress on 2026-10-03. Reviewed the existing definition, bounded-operation registry, platform recovery entry points, credential route and actor declaration. Removed the stale ESO blocker after a fresh metadata-only check. Concrete recommendation and staged acceptance: docs/platform-recovery-scheduling-plan.md.

Recommend scheduling-only ownership; platform retains the recovery runner and policy. First proposed slice is one disabled/manual apps-pg recovery-evidence operation, with offline contract tests before any live admission. T01 was subsequently accepted under the explicit user task and Emergent Security guidance; see its decision record. The broader D2 decision and actor ownership question remain separate.

Agree the platform runner contract

id: ACTIVITY-WP-0042-T05
status: wait
priority: high
blocked_on: message-from:railiance-platform
blocking_reason: "Platform owner must accept the candidate request/receipt contract, execution authority checks, limits and credential scope before runtime wiring."
state_hub_task_id: "a2d95c80-bfc5-54b7-9987-ad3571d9c75f"

Use the contract table in docs/platform-recovery-scheduling-plan.md for an owner review: authority verification, exact target, implementation version, durable run receipts, overlap/retry behavior, cleanup, resource ceilings and credential scope. Existing restore CLIs are not an admitted unattended API. This task records the dependency; no external acceptance or message is claimed. T02 implements/adopts the agreed contract; T03 does not gate offline design.

Decision record — Emergent Security application

• Decision class: task-scoped code/tests and scheduling-boundary clarification.
• Delegator: the user who requested ACTIVITY-WP-0042 progress and explicitly directed use of Emergent Security agent guidance in this conversation.
• Scope: activity-core repository development in the build estate; no tenant effect, production mutation, identity assignment, or recovery credential use.
• Rules: C4 lowest sufficient scope; A2 proceed within the task; A6 code/tests; C3/A5 no new capability or unowned responsibility; C2/C9 authority enforcement stays with the platform executor; A8 this record and commit evidence.
• Evidence: INTENT.md scheduling boundary, ACT-ADR-007 existing allowlist, platform consent 0638c13b, offline adversarial contract tests.
• Outcome: active offline implementation, T01 done; T05 and T03 retain their specific unresolved requirements. The broader D2 decision is unchanged.

Validation (2026-10-03): full activity-core suite 653 passed, 1 skipped; focused recovery-contract/bounded-operation suite 43 passed; Ruff and git diff --check clean. No runtime wiring, new operation registration, credential fetch, or live restore was performed. Decision receipt: 5ec92a02-cb0b-466d-9aaf-9da5ffc6c92d (codex scope clarification; D2 unchanged).

Harden the existing backup evidence boundary

id: ACTIVITY-WP-0042-T06
status: done
priority: high
state_hub_task_id: "97ef0499-3edd-5458-b093-184de0bac01c"

User requested the follow-up backup error-handling fix. Enforce ACT-ADR-007's existing non-secret evidence constraint and Emergent Security C2/C6/C9 without changing execution scope: discard raw subprocess stderr/targets/errors at the adapter boundary, suppress unsafe exception causes, and allowlist status values. Exercise malformed output, timeout/launch/decode failures, failure exit codes, and Temporal failure serialization with synthetic secret canaries. Preserve normal/degraded backup summaries and surface negative signal exit codes.

Decision class: task-scoped code/tests; delegation: explicit user task and Emergent Security A6; no credentials, live effects, or policy widening.

T06 result: the resolver returns the existing allowlisted evidence shape directly; raw output never escapes in returned results or serialized Temporal failures. Known status values are preserved; unknown status content becomes unknown. Unexpected exit codes cannot claim success, and negative signal codes survive. Validation: 676 passed, 1 skipped in the full suite; final focused backup, bounded-operation and evidence-sink checks 48 passed. Ruff baseline comparison shows no introduced findings (five pre-existing findings remain across the changed files); git diff --check passes. This is source hardening only, not a worker rollout or execution-authority change.

Loose-end closure review — 2026-10-03

T01/T04/T06 are complete. T02's offline request/receipt implementation is complete, but its admission acceptance cannot close until T05 supplies the platform-owned execution contract, authority enforcement, limits and credential lane. T02 is therefore wait and the workplan is blocked. Existing owner restore CLIs do not supply the required durable idempotency or per-run authority enforcement; adding a registered unattended operation now would invent those guarantees.

T03 still requires evidence identifying the actual issued actor or founder confirmation; the ops-warden example seed does not establish worker ownership. T05 still requires platform acceptance, not another local candidate implementation. No new workplan/task, identity claim, credential fetch, scheduled restore or owner acceptance was invented. The source-only backup hardening remains committed; this review does not claim it was rolled out to production.

Inspect downstream shared backup-lane delivery

id: ACTIVITY-WP-0042-T07
status: done
priority: high
state_hub_task_id: "436ed744-ae7a-564a-8e9b-e66f6ce0b098"

2026-10-03 NK-WP-0045 recovery follow-up: inspect actual backup consumer delivery without reading Secret objects. scripts/inspect_backup_delivery.py checks the railiance01 node, ESO readiness/refresh metadata, fixed source/library digests and mounted Forgejo/CNPG library digests. Captures and discards raw command failures; receipts contain only safe metadata, digests and booleans.

Forgejo uses immutable ConfigMap backup-verified-0220ca56520c; CNPG uses the railiance01 /home/tegwick/railiance-platform hostPath. They are separate source delivery paths. ExternalSecret actcore-backup-offsite refreshes each hour into actcore-runtime-secret with Merge/Retain; worker envFrom credentials remain in existing processes after a Secret refresh. Ready/SecretSynced is delivery health, not valid provider credentials or backup proof. Platform common source 50da37cf50016b03a7e6a537e87772a72979aba1 now includes public DAV X-Requested-With: XMLHttpRequest; vendored/mounted versions need verification.

Knowledge orientation: package 0.2.4 digest 025aab4f38246219c924bc94954e70ea0e6e7f72bcaf8d4344cd599602d345c3, start withheld because informed-decision source changed; reviewed that README and current custodian environment orientation directly. No orientation claim establishes runtime authority.

Refresh the existing backup consumer after the owner repairs the lane

id: ACTIVITY-WP-0042-T08
status: wait
needs_human: true
decision_id: "6110ead5-1831-4e20-aebd-ac1f978b503b"
priority: high
blocking_reason: "KV4 provider repair is delivered; repo-manager historical-prefix acceptance is delivered (receipt 39f2c067, 2026-10-06). Separate approved CNPG hostPath/Forgejo GitOps bundle delivery, ESO/env refresh and native consumer receipt remain. Founder per-change go is required before live promotion."
state_hub_task_id: "020d19be-0576-5052-ad35-14b1d5a48fe1"

The reviewed-folder inspection for NK-WP-0045 found the current token and expected labelled share absent; upload and authenticated route probes return 401. Lane KV v3 being synced does not resolve this provider mismatch.

Owner handoff: railiance-platform must first publish its non-secret receipt identifying the replacement lane KV version, exact two updated field names (NC_WEBDAV_TOKEN, NC_WEBDAV_URL), preserved AGE recipient, owner-reviewed share binding, and encrypted retained-history upload/readback proof. No token or URL goes into this repo or receipts. Root custody recovery can finish independently of this larger recurring backup consumer acceptance.

Prepare the bounded worker change only after that owner receipt: refresh ESO and verify status/resource metadata; replace the Forgejo immutable bundle using the platform-owned reviewed bundle builder; synchronize the CNPG hostPath source separately through repo-manager; reconcile the new bundle reference in k8s/gitops/runtime.yaml before ArgoCD synchronization. Preserve unrelated live mounts/environment. Do not use the old direct deployment patch helper against this adopted ArgoCD application. Perform one reviewed worker rollout to refresh envFrom only after source and ESO prerequisites; compare new pod start to ESO refresh and run this inspector again. Secret object bodies are never diagnostic output.

Require safe non-secret post-rollout source digests and readiness plus an owner-approved existing bounded backup run/provider receipt before claiming activity delivery acceptance. No automatic scheduled backup invocation, new operation admission, restore capability or customer recovery readiness is claimed. T02/T03/T05 keep their separate existing dependencies.

Read-only inspection receipt (2026-10-03T19:47:12Z): ESO Ready, last refresh 19:29:02Z. Source common SHA256 09f349ae815339732db505d6896fb1ca384be7c66b40574750a3ba30d4d922e0; immutable bundle common and both mounted Forgejo/CNPG libraries differ. No secret object read, provider probe or live mutation performed. Three injected inspection checks passed: current/stale source, delivery versus credential proof, and wrong-cluster refusal. git diff --check passed.

Producer handoff receipt fields agreed with platform owner: require published_lane_kv_version, coordinates_only_updated, escrow_and_unrelated_fields_preserved, and provider_roundtrip_verified_before_cas. The repair packet initially expects CAS from lane version 3 to 4; accept only the actual verified publication receipt, not that forecast. runtime_consumers_refreshed: false correctly leaves this T08 pending. Shared header source is 50da37c; no broader backup readiness follows from initial root-history provider proof.

Repo-manager receipt (2026-10-06, message 39f2c067 on thread 6bf8ca98): RMGR-WP-0019-T03 is done. rmgr sync --path ~/activity-core --push at e809fb2 applied against primary/railiance01 with 0 created, 45 updated (ACTIVITY-WP-* only), 0 retired, 0 refused; custodian-WP-0001..0004 were admitted with their historical row and task UUIDs preserved. This clears the exact-primary projection prerequisite only. The CNPG hostPath/Forgejo bundle delivery, ESO/env refresh, native consumer receipt and founder per-change go (decision 6110ead5) still gate T08.

2026-10-03T19:56:59Z supersedes the missing replacement-lane prerequisite: producer repair succeeded, lane CAS3->4 changed only token/URL, existing escrow/unrelated fields preserved, encrypted upload201/readback200 and independent decryption verified. Non-secret receipt retained in docs/evidence/ACTIVITY-WP-0042-backup-lane-repair-2026-10-03.json. runtime_consumers_refreshed: false; T08 remains waiting solely on reviewed consumer delivery and its acceptance. Root-history provider proof is complete.

Concrete review packet k8s/backup-release/README.md and its source-owned immutable Forgejo bundle/worker patch are staged outside ordinary GitOps rendering. No runtime apply/restart occurred. CNPG requires separate primary checkout delivery. rmgr sync --path . --push currently fails preflight on existing historic custodian-WP-* identifiers, before projection; source Git and local Hub sync succeed. Do not renumber grandfathered IDs or claim exact primary projection. This existing repo-manager compatibility issue must be resolved by its owner before that projection-based rollout path.

2026-10-04 CUST-WP-0075 reconciliation: ACTIVITY-WP-0042-T05: T01 scheduling-scope commitment is complete, so its terminal predecessor edge is removed. Exact platform runner request/receipt/authority/limits/custody acceptance remains; source-review request afb8004e-ed55-42a2-b0c7-5b1163d05bd7 names that act, with no external acceptance invented.

2026-10-04 CUST-WP-0075 reconciliation: ACTIVITY-WP-0042-T08: Verified retained repair receipt supplies KV4 provider upload/readback/decryption with runtime_consumers_refreshed=false; the done producer predecessor RPF-WP-0049-T05 is removed. RMGR-WP-0014/0015 historical compatibility does not supply the current activity-core exact-primary receipt. Owner request 6bf8ca98-ada1-43ab-83e7-207d4ab195fe asks for the actual receiving work record. Remaining runtime promotion is human-gated by 6110ead5-1831-4e20-aebd-ac1f978b503b; no source/env rollout or backup run occurred.