the-custodian/docs/weekly-allocation-report.md

Retained weekly allocation assessment

scripts/weekly_allocation_report.py implements the offline evidence join for CUST-WP-0071-T05. It reads the existing cluster collector, Custodian allocation reconcile, Prometheus query-response capture and resource-control inventory. It performs no collection, deployment, network calls, task writes or delivery. It is suitable for an accepted durable executor; it does not install one.

Inputs and retained evidence

The required inputs are:

• --cluster: railiance.cluster-resource-observation.v1, produced by railiance-cluster's make cluster-observe REMOTE=railiance01.
• --allocation: custodian.allocation-reconcile.v1 from scripts/reconcile_railiance_allocation.py, matching the same capture and workload inventory. Run both against the same retained observation.
• --telemetry: the existing September 28 query-capture shape: captured_at, window: 7d, step: 5m, and queries mapping each metric name to its exact promql and Prometheus response. Preserve original sample evaluation times; do not restamp historical query results. These describe the query window, not independently verified raw scrape completeness.
• --inventory: resource-control's data/resources/reef-railiance-k3s.json. Dated portfolio capacity is retained as portfolio context, never summed with the live cluster or treated as current scheduler evidence.
• --provenance: exact source collector revisions and source_observation matching the cluster capture, as in the September 28 allocation provenance.
• --as-of: explicit evaluation timestamp; identical retries use the same value.
• --output: a new retained JSON path. Exact identical retries succeed; changed content at that path fails and requires an explicit revision path. Writes use a complete temporary file and atomic create-without-replacement publication.

Optional --previous selects the previous retained report. Deltas require the same resource, exact query texts, sample step, seven-day window and consecutive capture endpoints within one sample step. Other comparisons stay unknown; the report still records disappeared workloads. Recommendation identity depends on resource/namespace/workload/review purpose, not the date or recommendation text. Existing receiving-task links survive retries and subsequent reports.

Metric names are cpu_p95_m, cpu_peak_m, memory_p95_bytes, memory_peak_bytes, cpu_throttle_ratio, latency_p95_seconds, errors, request_count, restarts, oom_events, cpu_sample_count and memory_sample_count. Metrics are nonnegative, finite instant-vector aggregates. Workload queries must carry exactly namespace and workload identity labels (an optional Prometheus __name__ is allowed). Extra labels or duplicate series are not silently aggregated. Namespace-only results are retained separately; they never become per-workload values, even for a currently single-workload namespace. Namespace peaks and percentiles are not additive.

Missing series, failed queries and non-finite values remain null with explicit reasons. Observed zero remains zero. Evaluation counts never prove raw scrape coverage. CPU/memory limits and instantaneous observations come from the cluster snapshot; historical latency, request/errors, OOM or throttling are not inferred from that snapshot. Unknown operating margin remains null; reservation residual does not establish processing capacity or release acceptance.

Owner decisions

Without a complete current owner review the result is investigate, with low confidence and explicit evidence gaps. --reviews accepts a JSON mapping keyed by the stable recommendation ID. A review provides decision (keep, increase, decrease or investigate), owner, period_end matching telemetry capture, reviewed_at, evidence_refs, receiving_task, configuration, rollback_ref, coverage_ref, service_acceptance_ref, and margin_ref. Optional outcome records what happened to the preceding recommendation. These references are owner assertions to be checked during acceptance, not proof generated by this adapter. Wrong-owner/window reviews, missing workload metrics, zero CPU requests, unknown owners and stale/future evidence keep the result at investigate and retain the proposal for review. No idle sample automatically recommends a cut.

Only new observation evidence and verified owner disposition can complete T02–T04. In particular, the historical Vergabe sample does not establish the attended two-user p95/error acceptance target or incremental database attribution.

Disabled execution contract

The owner-source candidate is activity-core/activity-definitions/weekly-railiance-allocation-review.md. Its fixed command is:

python3 scripts/weekly_allocation_report.py --request config/weekly-allocation-request.json

The request is intentionally unbound and the command fails until the accepted executor supplies real input paths, a fixed as_of, and a durable output path. Paths resolve relative to the request file. The executor must snapshot this request per run, keep it stable for retries, bind a real previous report, and retain reports beyond the seven-day raw-metric retention. The committed null template is not a production storage location or collection implementation.

Activity Core source 4ae00fc selects its existing ADR-007 bounded-operation executor with fixed owner commands; both definitions remain disabled. The prior coding profile cannot run Python or reach the Hub. Runtime owners must still bind accepted source/fleet mounts, separate private retained state and scoped network access. Activity Core owns the Monday 08:00 Europe/Berlin schedule and bounded-activity/context/progress-sink evidence, with one Temporal attempt. The full binding contract is activity-core/docs/custodian-bounded-operations.md. The adapter's matching recommendation IDs are the downstream upsert keys for existing owner work records; it does not itself create those records. Delivery, explicit owner acknowledgment, independent missed/failed-run detection and recovery remain unbound. A report file or completed bounded activity alone proves none of them. T05 needs the first unattended schedule receipt and exercised recovery.

Verification

python3 -m unittest discover -s tests -p test_weekly_allocation_report.py -v

The tests cover historical namespace-only input, missing/zero/NaN, invalid aggregation, query failure, compatible/incompatible periods, stable identities and owner-task links, stale/future evidence, owner review, resource/capture mismatch and retained retry/collision behavior.

In-cluster runner (CUST-WP-0071-T05, founder decision 2026-10-21)

docs/decisions/2026-10-21-cust-wp-0071-delivery-interview.md selects an in-cluster Kubernetes CronJob for T05 and supersedes the Activity Core placement above. scripts/weekly_allocation_incluster.py is the job entry point. It is standard-library only and reuses the existing owner code unchanged: railiance-cluster build_observation, reconcile.join (PyYAML is now imported only by the reconcile CLI) and this adapter's build_report and retain.

• Reads: the Kubernetes API through the pod ServiceAccount, with get/list on nodes, pods, persistentvolumeclaims, persistentvolumes, storageclasses and metrics.k8s.io nodes/pods only. Prometheus is reached through PROMETHEUS_URL with config/weekly-allocation-queries.json.
• Week identity: the ISO week of (run time - 6h). The telemetry window ends at Monday 06:00 UTC, so consecutive windows are exactly seven days apart in CET and CEST. Prometheus keeps seven days of data, so the run must happen on the Monday. A 2026-10-21 (Wednesday) read-only dry run against railiance01 retained the report, but per-namespace CPU sample counts reached at most 1512 of 2016, because the window start had already expired.
• Retention: <data>/reports/<YYYY>-W<ww>.json (create-without-replacement), per-attempt inputs under <data>/runs/<week>/<attempt>/, and an append-only <data>/receipts.jsonl with status retained, already-retained or failed, previous week, missed_weeks, comparability and failed queries.
• Idempotency: a retained week is a no-op success. A failed attempt leaves no report, so the Job retry or a manual run collects again.
• Delivery (founder decision 2026-10-21, "Hub message + repo tasks"): with STATE_HUB_URL set, each newly retained week produces one State Hub message from custodian-allocation-review to the-custodian. The message carries a summary, the report path and its sha256, and its id is recorded in receipts.jsonl. A week that already has a message id is never posted again. If a post fails, the receipt says delivery: failed, the Job exits 1, and its retry posts for the already-retained week. The next Custodian session reviews the report and records recommendations as tasks in the owner repositories (ADR-001). The job itself creates no tasks.
• Missed/failed-run check: scripts/weekly_allocation_check.py, which scripts/security_flow_daily.py calls. It flags failed when the latest receipt failed, and missing when the due ISO week has no retained report (the current week falls due on Monday 12:00 Europe/Berlin). The agent is notified once per distinct flag set. It is inactive until CUSTODIAN_ALLOCATION_RECEIPTS (or --allocation-receipts) is set to hub (delivery messages, where a failure shows as missing) or to a path holding a copy of receipts.jsonl. Set it in infra/security-flow/ only when the CronJob is activated.
• Not done: owner acknowledgment. Workload rows stay investigate because the queries are namespace-level. kube-state-metrics does not export kube_pod_labels, so no workload-labelled series exist yet.

Executor manifests (suspended, not registered with ArgoCD) live in railiance-platform argocd/platform-addons/custodian-allocation-review/, with the draft Application in argocd/railiance01/drafts/.

python3 -m pytest tests/test_weekly_allocation_incluster.py tests/test_weekly_allocation_report.py -q