the-custodian/docs/weekly-allocation-report.md
codex 02e69269c8
Some checks failed
CI Smoke / host-smoke (push) Successful in 1s
CI Smoke / container-smoke (push) Successful in 2s
Reject generated-only commits / generated-commit-guard (push) Successful in 11s
Python Tests / pytest (push) Failing after 28s
Adopt staged generation and prepare weekly sizing and credential isolation
2026-10-19 12:42:11 +02:00

6.2 KiB
Raw Blame History

Retained weekly allocation assessment

scripts/weekly_allocation_report.py implements the offline evidence join for CUST-WP-0071-T05. It reads the existing cluster collector, Custodian allocation reconcile, Prometheus query-response capture and resource-control inventory. It performs no collection, deployment, network calls, task writes or delivery. It is suitable for an accepted durable executor; it does not install one.

Inputs and retained evidence

The required inputs are:

  • --cluster: railiance.cluster-resource-observation.v1, produced by railiance-cluster's make cluster-observe REMOTE=railiance01.
  • --allocation: custodian.allocation-reconcile.v1 from scripts/reconcile_railiance_allocation.py, matching the same capture and workload inventory. Run both against the same retained observation.
  • --telemetry: the existing September 28 query-capture shape: captured_at, window: 7d, step: 5m, and queries mapping each metric name to its exact promql and Prometheus response. Preserve original sample evaluation times; do not restamp historical query results. These describe the query window, not independently verified raw scrape completeness.
  • --inventory: resource-control's data/resources/reef-railiance-k3s.json. Dated portfolio capacity is retained as portfolio context, never summed with the live cluster or treated as current scheduler evidence.
  • --provenance: exact source collector revisions and source_observation matching the cluster capture, as in the September 28 allocation provenance.
  • --as-of: explicit evaluation timestamp; identical retries use the same value.
  • --output: a new retained JSON path. Exact identical retries succeed; changed content at that path fails and requires an explicit revision path. Writes use a complete temporary file and atomic create-without-replacement publication.

Optional --previous selects the previous retained report. Deltas require the same resource, exact query texts, sample step, seven-day window and consecutive capture endpoints within one sample step. Other comparisons stay unknown; the report still records disappeared workloads. Recommendation identity depends on resource/namespace/workload/review purpose, not the date or recommendation text. Existing receiving-task links survive retries and subsequent reports.

Metric names are cpu_p95_m, cpu_peak_m, memory_p95_bytes, memory_peak_bytes, cpu_throttle_ratio, latency_p95_seconds, errors, request_count, restarts, oom_events, cpu_sample_count and memory_sample_count. Metrics are nonnegative, finite instant-vector aggregates. Workload queries must carry exactly namespace and workload identity labels (an optional Prometheus __name__ is allowed). Extra labels or duplicate series are not silently aggregated. Namespace-only results are retained separately; they never become per-workload values, even for a currently single-workload namespace. Namespace peaks and percentiles are not additive.

Missing series, failed queries and non-finite values remain null with explicit reasons. Observed zero remains zero. Evaluation counts never prove raw scrape coverage. CPU/memory limits and instantaneous observations come from the cluster snapshot; historical latency, request/errors, OOM or throttling are not inferred from that snapshot. Unknown operating margin remains null; reservation residual does not establish processing capacity or release acceptance.

Owner decisions

Without a complete current owner review the result is investigate, with low confidence and explicit evidence gaps. --reviews accepts a JSON mapping keyed by the stable recommendation ID. A review provides decision (keep, increase, decrease or investigate), owner, period_end matching telemetry capture, reviewed_at, evidence_refs, receiving_task, configuration, rollback_ref, coverage_ref, service_acceptance_ref, and margin_ref. Optional outcome records what happened to the preceding recommendation. These references are owner assertions to be checked during acceptance, not proof generated by this adapter. Wrong-owner/window reviews, missing workload metrics, zero CPU requests, unknown owners and stale/future evidence keep the result at investigate and retain the proposal for review. No idle sample automatically recommends a cut.

Only new observation evidence and verified owner disposition can complete T02–T04. In particular, the historical Vergabe sample does not establish the attended two-user p95/error acceptance target or incremental database attribution.

Disabled execution contract

The owner-source candidate is activity-core/activity-definitions/weekly-railiance-allocation-review.md. Its fixed command is:

python3 scripts/weekly_allocation_report.py --request config/weekly-allocation-request.json

The request is intentionally unbound and the command fails until the accepted executor supplies real input paths, a fixed as_of, and a durable output path. Paths resolve relative to the request file. The executor must snapshot this request per run, keep it stable for retries, bind a real previous report, and retain reports beyond the seven-day raw-metric retention. The committed null template is not a production storage location or collection implementation.

Activity-core must select a scoped durable Railiance executor/profile before projection or activation; no workstation profile is asserted here. It owns the Monday 08:00 Europe/Berlin schedule and operations-run retry/lease evidence. The adapter's matching recommendation IDs are the downstream upsert keys for existing owner work records; it does not itself create those records. Delivery, explicit owner acknowledgment, independent missed/failed-run detection and recovery remain unbound. A report file or successful ops_run alone proves none of them. T05 needs the first unattended schedule receipt and exercised recovery.

Verification

python3 -m unittest discover -s tests -p test_weekly_allocation_report.py -v

The tests cover historical namespace-only input, missing/zero/NaN, invalid aggregation, query failure, compatible/incompatible periods, stable identities and owner-task links, stale/future evidence, owner review, resource/capture mismatch and retained retry/collision behavior.