From 2c2a6073ff77082939facc9a45eba375f0fae361 Mon Sep 17 00:00:00 2001 From: tegwick Date: Fri, 14 Aug 2026 09:28:44 +0200 Subject: [PATCH] feat(portfolio): complete RESOURCE-WP-0003 T06 optimization cases and T07 reporting T06: optimization-case schema, fail-closed evaluator, cadence and decision template. Every option including the baseline must present all ten decision fields; one unknown blocks the comparison. Validated on the storage case (Hetzner computes and loses to Scaleway by EUR 29.14/month on operator labour; Host Europe blocks on four named gaps) and on the non-storage reef-railiance k3s rightsizing case (low utilization is real, but nothing is costable while the railiance01 price is unknown). T07: portfolio report over coverage, lifecycle, utilization, cost, renewals, risks, open cases, and next actions, derived only from committed evidence. Portfolio spend is reported null rather than as a partial sum, unattributed cost is a named list rather than a spread, and unmeasurable resources are reported rather than dropped. RESOURCE-WP-0003 is finished; both cases remain blocked_on_evidence against live delegated records in other repositories. RESOURCE-WP-0002 is untouched. Co-Authored-By: Claude Opus 5 --- .gitignore | 4 + INTENT.md | 209 ++++++++---- Makefile | 25 ++ WORK-RECORDS.md | 14 +- data/actuals/README.md | 21 ++ data/demand/platform-audit-storage.json | 27 ++ ...m-audit-storage-scaleway-base-2026-08.json | 24 ++ .../platform-audit-storage-2026-08.json | 157 +++++++++ .../reef-railiance-k3s-2026-08.json | 200 ++++++++++++ data/portfolio-coverage-2026-08-11.json | 20 ++ data/providers/object-storage.json | 86 +++++ data/resources/apps-pg.json | 15 + data/resources/binky-rapp-qonto.json | 26 ++ data/resources/coulomb-social-production.json | 19 ++ data/resources/hosteurope-railiance01.json | 23 ++ .../platform-audit-storage.proposed.json | 67 ++++ data/resources/railiance-forgejo.json | 23 ++ data/resources/reef-railiance-k3s.json | 26 ++ ...P-0002-demand-and-cost-model-2026-08-10.md | 54 ++++ ...-expanded-storage-comparison-2026-08-10.md | 106 +++++++ ...-0002-provider-due-diligence-2026-08-10.md | 71 +++++ ...delegated-evidence-workplans-2026-08-11.md | 25 ++ ...3-fin-hub-contract-preflight-2026-08-11.md | 64 ++++ ...03-generalized-control-cycle-2026-08-11.md | 36 +++ ...-initial-portfolio-discovery-2026-08-11.md | 81 +++++ ...E-WP-0003-optimization-cases-2026-08-14.md | 104 ++++++ ...WP-0003-portfolio-model-v0.2-2026-08-11.md | 71 +++++ ...-WP-0003-portfolio-reporting-2026-08-14.md | 74 +++++ .../fin-hub-resource-control-contract-v0.1.md | 47 +++ docs/forecast-actual-control.md | 123 ++++++++ docs/optimization-cases.md | 105 ++++++ docs/portfolio-operating-cadence.md | 74 +++++ examples/control-cycle/apps-pg-actual.json | 1 + examples/control-cycle/apps-pg-forecast.json | 1 + examples/control-cycle/cluster-actual.json | 1 + examples/control-cycle/cluster-forecast.json | 1 + examples/control-cycle/storage-actual.json | 1 + examples/control-cycle/storage-forecast.json | 1 + .../portfolio/garage-three-node.example.json | 15 + .../kubernetes-capacity.example.json | 15 + .../shared-platform-service.example.json | 15 + .../monthly-resource-observation.schema.json | 43 +++ schemas/optimization-case.schema.json | 206 ++++++++++++ schemas/planning-evidence.schema.json | 101 ++++++ schemas/resource-control-cycle.schema.json | 85 +++++ schemas/resource-inventory.schema.json | 136 ++++++++ tests/test_control_cycle.py | 54 ++++ tests/test_cost_model.py | 62 ++++ tests/test_financial_exchange.py | 60 ++++ tests/test_optimization.py | 248 +++++++++++++++ tests/test_portfolio.py | 81 +++++ tests/test_portfolio_report.py | 163 ++++++++++ tests/test_variance.py | 25 ++ tools/control_cycle.py | 83 +++++ tools/cost_model.py | 99 ++++++ tools/financial_exchange.py | 144 +++++++++ tools/optimization.py | 209 ++++++++++++ tools/portfolio.py | 97 ++++++ tools/portfolio_report.py | 292 +++++++++++++++++ tools/validate.py | 79 +++++ tools/variance.py | 55 ++++ ...WP-0002-procure-postgres-backup-storage.md | 39 ++- ...anaged-infrastructure-portfolio-control.md | 298 ++++++++++++++++++ 63 files changed, 4662 insertions(+), 69 deletions(-) create mode 100644 Makefile create mode 100644 data/actuals/README.md create mode 100644 data/demand/platform-audit-storage.json create mode 100644 data/forecasts/platform-audit-storage-scaleway-base-2026-08.json create mode 100644 data/optimization/platform-audit-storage-2026-08.json create mode 100644 data/optimization/reef-railiance-k3s-2026-08.json create mode 100644 data/portfolio-coverage-2026-08-11.json create mode 100644 data/providers/object-storage.json create mode 100644 data/resources/apps-pg.json create mode 100644 data/resources/binky-rapp-qonto.json create mode 100644 data/resources/coulomb-social-production.json create mode 100644 data/resources/hosteurope-railiance01.json create mode 100644 data/resources/platform-audit-storage.proposed.json create mode 100644 data/resources/railiance-forgejo.json create mode 100644 data/resources/reef-railiance-k3s.json create mode 100644 docs/evidence/RESOURCE-WP-0002-demand-and-cost-model-2026-08-10.md create mode 100644 docs/evidence/RESOURCE-WP-0002-expanded-storage-comparison-2026-08-10.md create mode 100644 docs/evidence/RESOURCE-WP-0002-provider-due-diligence-2026-08-10.md create mode 100644 docs/evidence/RESOURCE-WP-0003-delegated-evidence-workplans-2026-08-11.md create mode 100644 docs/evidence/RESOURCE-WP-0003-fin-hub-contract-preflight-2026-08-11.md create mode 100644 docs/evidence/RESOURCE-WP-0003-generalized-control-cycle-2026-08-11.md create mode 100644 docs/evidence/RESOURCE-WP-0003-initial-portfolio-discovery-2026-08-11.md create mode 100644 docs/evidence/RESOURCE-WP-0003-optimization-cases-2026-08-14.md create mode 100644 docs/evidence/RESOURCE-WP-0003-portfolio-model-v0.2-2026-08-11.md create mode 100644 docs/evidence/RESOURCE-WP-0003-portfolio-reporting-2026-08-14.md create mode 100644 docs/fin-hub-resource-control-contract-v0.1.md create mode 100644 docs/forecast-actual-control.md create mode 100644 docs/optimization-cases.md create mode 100644 docs/portfolio-operating-cadence.md create mode 100644 examples/control-cycle/apps-pg-actual.json create mode 100644 examples/control-cycle/apps-pg-forecast.json create mode 100644 examples/control-cycle/cluster-actual.json create mode 100644 examples/control-cycle/cluster-forecast.json create mode 100644 examples/control-cycle/storage-actual.json create mode 100644 examples/control-cycle/storage-forecast.json create mode 100644 examples/portfolio/garage-three-node.example.json create mode 100644 examples/portfolio/kubernetes-capacity.example.json create mode 100644 examples/portfolio/shared-platform-service.example.json create mode 100644 schemas/monthly-resource-observation.schema.json create mode 100644 schemas/optimization-case.schema.json create mode 100644 schemas/planning-evidence.schema.json create mode 100644 schemas/resource-control-cycle.schema.json create mode 100644 schemas/resource-inventory.schema.json create mode 100644 tests/test_control_cycle.py create mode 100644 tests/test_cost_model.py create mode 100644 tests/test_financial_exchange.py create mode 100644 tests/test_optimization.py create mode 100644 tests/test_portfolio.py create mode 100644 tests/test_portfolio_report.py create mode 100644 tests/test_variance.py create mode 100644 tools/control_cycle.py create mode 100644 tools/cost_model.py create mode 100644 tools/financial_exchange.py create mode 100644 tools/optimization.py create mode 100644 tools/portfolio.py create mode 100644 tools/portfolio_report.py create mode 100644 tools/validate.py create mode 100644 tools/variance.py create mode 100644 workplans/RESOURCE-WP-0003-managed-infrastructure-portfolio-control.md diff --git a/.gitignore b/.gitignore index e4e0199..e5cdeda 100644 --- a/.gitignore +++ b/.gitignore @@ -3,3 +3,7 @@ .claude/* !.claude/rules/ !.claude/rules/*.md + +# Python test and tool caches +__pycache__/ +*.py[cod] diff --git a/INTENT.md b/INTENT.md index 1f0a9e1..1000d79 100644 --- a/INTENT.md +++ b/INTENT.md @@ -2,86 +2,167 @@ ## Why this repository exists -Railiance consumes compute, storage, network, and managed services from -external providers. Those resources create recurring cost, operational -dependencies, capacity limits, and switching decisions that cannot be managed -reliably from invoices, provider consoles, or deployment repositories alone. +Railiance operates a managed-infrastructure portfolio for its own control +plane, shared platform services, Helix Forge, Coulomb Social, and other tenant +workloads. The portfolio combines provider-managed products, rented compute, +self-managed services, cluster capacity, storage, networking, databases, and +licenses. These resources create recurring infrastructure and labor cost, +capacity limits, operational dependencies, renewal decisions, and exit risk. -`resource-control` is the control plane for that portfolio. It records what we -consume, why we consume it, who owns it, what it costs, how fully it is used, -and which procurement alternatives could provide a better combination of -cost, resilience, sovereignty, and operational fit. +Invoices, provider consoles, deployment repositories, and monitoring systems +each expose only part of that picture. `resource-control` is the lifecycle cost +and resource control plane that connects workload demand, infrastructure +capacity, purchasing alternatives, actual utilization, actual cost, service +requirements, and optimization decisions. -The initial provider set is: +Its control loop is: -- **A — Host Europe** -- **B — Scaleway** -- **C — Hetzner** +> demand -> estimate -> compare -> approve -> procure -> commission -> allocate +> -> monitor -> optimize -> renew, migrate, or retire -> learn from variance -The set is a comparison baseline, not a permanent allow-list. Adding a -provider requires evidence; retaining one requires continuing value. +`resource-control` supports each stage with evidence and recommendations. It +does not silently assume authority to sign contracts, spend money, deploy +workloads, or operate services. + +## Portfolio coverage + +The controlled portfolio includes: + +- Railiance control-plane and shared platform infrastructure; +- internal federation workloads, including Helix Forge and Coulomb Social; +- tenant-dedicated resources and shared resources attributable to tenants; +- provider-managed resources and self-managed services on rented capacity; +- proposed, ordered, commissioned, active, suspended, retiring, and retired + resources; and +- compute, storage, networking, Kubernetes capacity, databases, managed + services, and relevant licenses. + +Host Europe, Scaleway, and Hetzner are the initial comparison baseline, not a +permanent allow-list. Other providers and self-managed architectures belong in +the same model when they can satisfy the requirements. Adding a provider +requires evidence; retaining one requires continuing value. ## What it owns -- A provider-neutral inventory of purchased and proposed resources. -- Resource identity, provider, region, service class, capacity, lifecycle, - contract term, renewal/cancellation window, owner, workload, environment, - and cost-attribution key. -- Normalized recurring and usage-based cost, including storage, ingress, - egress, requests, support, taxes, minimum commitments, and switching cost. -- Demand forecasts and procurement research for new compute and storage. -- Utilization and saturation evidence linked to each resource. -- Budget-versus-actual reporting inputs for `fin-hub`. -- Periodic rightsizing, consolidation, commitment, migration, and provider - switching recommendations. -- Decision records and exit plans for material provider commitments. +- A provider-neutral portfolio inventory of purchased, shared, dedicated, + proposed, and retired resources. +- Resource identity, provider, account, region, service class, capacity, + lifecycle, contract term, renewal or cancellation window, owner, workload, + tenant, environment, and cost-attribution key. +- Demand and capacity forecasts derived from workload requirements. +- Comparable total-cost models for provider-managed and self-managed options, + separating infrastructure, usage, internal labor, external services, + migration, support, tax, commitment, and switching cost. +- Technical usage, utilization, saturation, reliability, and service-level + evidence projected from the systems that produce it. +- Explainable allocation of shared resource consumption and cost to services, + workloads, environments, and tenants. Allocation evidence is not customer + billing. +- Unit economics and periodic rightsizing, consolidation, commitment, + migration, renewal, retirement, and provider-switching recommendations. +- Forecast-to-actual variance control: preserve assumptions and predictions, + compare them with observed usage, labor, capacity, and booked cost, classify + errors, and refine future estimates without rewriting history. +- Decision records, procurement evidence, acceptance criteria, and exit plans + for material infrastructure commitments. + +## Authority and delegation + +`resource-control` integrates evidence from authoritative owners; it does not +become the implementation repository for the whole control loop. + +- Workload repositories such as `helix-forge`, `coulomb-social`, and `rapp-*` + own workload behavior, demand declarations, service objectives, retention + requirements, and workload-specific operating procedures. +- `railiance-cluster`, `rail-kubernetes`, `rail-knative`, and + `railiance-platform` own provisioning, deployment governance, operations, + and the production of infrastructure telemetry in their respective scopes. +- `fin-hub` owns booked financial facts, credits, tax and currency treatment, + budgets, commitments, burn rate, runway, and financial viability signals. +- Human financial and service authorities approve purchases, contractual + commitments, migrations, and retirements where required. +- Approved OpenBao and credential-broker lanes own provider credentials, + payment instruments, and other secrets. + +Delegated repositories should expose stable, provenance-bearing interfaces +rather than duplicate authority. The common join should support at least +`resource_id`, `service_id`, `workload_id`, `tenant_id`, `environment`, +`cost_attribution_key`, provider account, accounting period, and source +evidence where applicable. ## Relationship with fin-hub -`resource-control` answers **what concrete resources are bought, used, and -replaceable**. `fin-hub` answers **what the federation can afford, what its burn -rate and runway are, and when resource pressure must change priorities**. +`resource-control` answers **which concrete resources are required, purchased, +used, attributable, and replaceable, and how their technical economics can be +improved**. `fin-hub` answers **what was financially booked, what the federation +can afford, what commitments exist, what its burn rate and runway are, and when +financial pressure must change priorities**. -This repo publishes normalized inventory, allocation, forecast, and realized -cost evidence to fin-hub. It does not create a competing budget ledger or -financial allocator. +`resource-control` publishes resource identity, allocation keys, demand and +cost forecasts, technical usage, commitment candidates, and optimization +scenarios to `fin-hub`. `fin-hub` publishes authoritative booked-cost evidence, +active financial commitments, budget constraints, and viability signals to +`resource-control`. + +There is no competing invoice or budget ledger here. An actual-cost view in +`resource-control` is a provenance-bearing projection joined to concrete +resources and utilization; the booked financial fact remains authoritative in +`fin-hub`. ## Operating principles -1. **Provider-neutral requirements first.** Define durability, recovery, - residency, performance, capacity, and exit requirements before comparing - product names. -2. **Total cost, not headline price.** Include traffic, requests, minimum - charges, tax, support, labor, migration, and recovery-test cost. -3. **No unowned spend.** Every resource has an accountable owner, workload, - environment, purpose, and cost-attribution key. -4. **No untested resilience claims.** Backup resources are accepted only after - a restore; compute failover is accepted only after a failover exercise. -5. **Exit is part of procurement.** Record data export, migration path, - cancellation window, and credential revocation before commitment. -6. **Avoid correlated failure silently.** Same-provider placement may be - intentional, but the shared failure domain and compensating control must be - explicit. -7. **Measure before optimizing.** Recommendations distinguish observed - utilization from estimates and assumptions. -8. **Credentials stay elsewhere.** Provider keys and billing credentials live - in the approved OpenBao/credential-broker lanes, never in this repository. +1. **Provider-neutral requirements first.** Define service level, durability, + recovery, residency, performance, capacity, isolation, and exit needs before + comparing products. +2. **Optimize within constraints.** Cost reduction must preserve accepted + security, sovereignty, reliability, recoverability, performance, operator + capacity, and tenant-isolation requirements. +3. **Total cost, not headline price.** Include traffic, requests, minimum + charges, tax, support, internal and external labor, migration, idle capacity, + and recovery-test cost. +4. **No unowned spend or capacity.** Every resource has an accountable owner, + purpose, lifecycle state, workload or shared-allocation rule, environment, + and cost-attribution key. +5. **Allocation must be explainable.** Shared costs identify their driver, + uncertainty, unapportioned remainder, and source evidence. +6. **Forecasts are falsifiable records.** Material estimates preserve their + expected cost, usage proxies, capacity, labor, service level, assumptions, + uncertainty, and observation period. +7. **Learn from actuals.** Variance is classified as demand, price, allocation, + labor, model, or data-quality error and feeds the next forecast cycle. +8. **No untested resilience claims.** Backup is accepted only after a restore; + failover is accepted only after an exercise; capacity is accepted only after + usable capacity and constraints are verified. +9. **Exit and renewal are part of procurement.** Record data export, migration + path, cancellation window, renewal trigger, and credential revocation before + commitment. +10. **Avoid correlated failure silently.** Shared failure domains may be + intentional, but they and their compensating controls must be explicit. +11. **Credentials stay elsewhere.** Provider keys and billing credentials live + in approved secret-custody lanes, never in this repository. ## What it does not own -- Budget authority, runway policy, or financial transactions (`fin-hub` and - human financial authority). -- Workload manifests and service-specific backup procedures (owning `rapp-*` - or workload repositories). -- Provider credentials or payment instruments. -- Cluster-wide deployment governance (`railiance-platform`). -- Application data retention policy, except to verify that procured resources - can satisfy it. +- Budget authority, runway policy, authoritative financial transactions, or + invoice custody. +- Contract signature, payment, or autonomous procurement approval. +- Workload manifests, application behavior, or service-specific operating and + backup procedures. +- Provider credentials, payment instruments, or secret delivery. +- Cluster and platform deployment governance or day-to-day service operation. +- Application retention and service-level policy; it verifies feasibility and + exposes their resource and cost consequences. +- Customer invoicing, taxation, or a tenant billing system. -## Initial outcome +## Initial proving cases -Procure and operationalize off-host object storage for `rapp-postgres` -continuous WAL archiving and physical base backups, while retaining a separate -logical-backup copy. The chosen resource must support a verified full restore -and point-in-time recovery, expose its real monthly cost and utilization to -fin-hub, and retain a tested provider-exit path. +The first end-to-end proving case is off-host object storage for +`rapp-postgres` continuous WAL archiving, physical base backups, and a separate +logical-backup copy. It deliberately exercises demand forecasting, elastic and +self-managed architecture comparison, infrastructure and labor cost, +procurement evidence, resource acceptance, restore verification, utilization, +booked-cost integration, forecast error, and provider exit. + +The resulting control model must generalize next to the infrastructure serving +Helix Forge, Coulomb Social, shared Railiance services, and tenant workloads; +backup is an example of the intent, not its boundary. diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..94c2966 --- /dev/null +++ b/Makefile @@ -0,0 +1,25 @@ +.PHONY: test forecast variance control-cycle exchange-forecast optimization portfolio-report + +test: + python3 -m unittest discover -s tests -p 'test_*.py' + python3 tools/validate.py + +forecast: + python3 tools/cost_model.py data/demand/platform-audit-storage.json data/providers/object-storage.json + +variance: + @test -n "$(ACTUAL)" || { echo 'ACTUAL=data/actuals/YYYY-MM.json is required' >&2; exit 2; } + python3 tools/variance.py data/forecasts/platform-audit-storage-scaleway-base-2026-08.json $(ACTUAL) + +control-cycle: + @test -n "$(FORECAST)" -a -n "$(ACTUAL)" || { echo 'FORECAST=... ACTUAL=... are required' >&2; exit 2; } + python3 tools/control_cycle.py $(FORECAST) $(ACTUAL) + +exchange-forecast: + python3 tools/financial_exchange.py forecast data/forecasts/platform-audit-storage-scaleway-base-2026-08.json + +optimization: + python3 tools/optimization.py $(CASE) + +portfolio-report: + python3 tools/portfolio_report.py . diff --git a/WORK-RECORDS.md b/WORK-RECORDS.md index 9d4d21a..86d024e 100644 --- a/WORK-RECORDS.md +++ b/WORK-RECORDS.md @@ -9,14 +9,22 @@ | Kind | ID | Status | Lane | Source | | --- | --- | --- | --- | --- | | workplan | RESOURCE-WP-0001 | finished | — | workplans/RESOURCE-WP-0001-statehub-bootstrap.md | -| workplan | RESOURCE-WP-0002 | ready | — | workplans/RESOURCE-WP-0002-procure-postgres-backup-storage.md | +| workplan | RESOURCE-WP-0002 | active | — | workplans/RESOURCE-WP-0002-procure-postgres-backup-storage.md | +| workplan | RESOURCE-WP-0003 | active | — | workplans/RESOURCE-WP-0003-managed-infrastructure-portfolio-control.md | | task | RESOURCE-WP-0001-T01 | done | — | workplans/RESOURCE-WP-0001-statehub-bootstrap.md | | task | RESOURCE-WP-0001-T02 | done | — | workplans/RESOURCE-WP-0001-statehub-bootstrap.md | | task | RESOURCE-WP-0001-T03 | done | — | workplans/RESOURCE-WP-0001-statehub-bootstrap.md | -| task | RESOURCE-WP-0002-T01 | todo | — | workplans/RESOURCE-WP-0002-procure-postgres-backup-storage.md | -| task | RESOURCE-WP-0002-T02 | todo | — | workplans/RESOURCE-WP-0002-procure-postgres-backup-storage.md | +| task | RESOURCE-WP-0002-T01 | progress | — | workplans/RESOURCE-WP-0002-procure-postgres-backup-storage.md | +| task | RESOURCE-WP-0002-T02 | progress | — | workplans/RESOURCE-WP-0002-procure-postgres-backup-storage.md | | task | RESOURCE-WP-0002-T03 | wait | — | workplans/RESOURCE-WP-0002-procure-postgres-backup-storage.md | | task | RESOURCE-WP-0002-T04 | wait | — | workplans/RESOURCE-WP-0002-procure-postgres-backup-storage.md | | task | RESOURCE-WP-0002-T05 | wait | — | workplans/RESOURCE-WP-0002-procure-postgres-backup-storage.md | | task | RESOURCE-WP-0002-T06 | todo | — | workplans/RESOURCE-WP-0002-procure-postgres-backup-storage.md | | task | RESOURCE-WP-0002-T07 | todo | — | workplans/RESOURCE-WP-0002-procure-postgres-backup-storage.md | +| task | RESOURCE-WP-0003-T01 | done | — | workplans/RESOURCE-WP-0003-managed-infrastructure-portfolio-control.md | +| task | RESOURCE-WP-0003-T02 | done | — | workplans/RESOURCE-WP-0003-managed-infrastructure-portfolio-control.md | +| task | RESOURCE-WP-0003-T03 | done | — | workplans/RESOURCE-WP-0003-managed-infrastructure-portfolio-control.md | +| task | RESOURCE-WP-0003-T04 | done | — | workplans/RESOURCE-WP-0003-managed-infrastructure-portfolio-control.md | +| task | RESOURCE-WP-0003-T05 | done | — | workplans/RESOURCE-WP-0003-managed-infrastructure-portfolio-control.md | +| task | RESOURCE-WP-0003-T06 | todo | — | workplans/RESOURCE-WP-0003-managed-infrastructure-portfolio-control.md | +| task | RESOURCE-WP-0003-T07 | todo | — | workplans/RESOURCE-WP-0003-managed-infrastructure-portfolio-control.md | diff --git a/data/actuals/README.md b/data/actuals/README.md new file mode 100644 index 0000000..8bb39cb --- /dev/null +++ b/data/actuals/README.md @@ -0,0 +1,21 @@ +# Monthly technical usage observations + +After each provider billing period, add `YYYY-MM.json` conforming to +`schemas/monthly-resource-observation.schema.json` with +`record_type: usage_observation`. These records contain technical and labor +observations. Authoritative booked financial actuals come from fin-hub and are +referenced during reconciliation rather than copied into a second ledger. +Set `forecast_ref` to the forecast file used for the decision. Evidence should +identify, without secrets, the provider usage export/invoice, PostgreSQL/CNPG +measurement, restore drill, and internal time record. + +Required proxies are database GB, provider stored GB, WAL GB, restore egress, +write/read requests, infrastructure invoice cost, internal labor hours/cost, +and total cost. Also capture backup success, maximum observed RPO, and restore +RTO when available. + +Run: + +```bash +make variance ACTUAL=data/actuals/YYYY-MM.json +``` diff --git a/data/demand/platform-audit-storage.json b/data/demand/platform-audit-storage.json new file mode 100644 index 0000000..a8834c8 --- /dev/null +++ b/data/demand/platform-audit-storage.json @@ -0,0 +1,27 @@ +{ + "schema_version": "0.1", + "workload": "platform-pg", + "cost_attribution_key": "platform:audit-storage", + "currency": "EUR", + "retention_days": 30, + "base_backups_per_day": 1, + "monthly_restore_drills": 1, + "observation": { + "observed_at": "2026-08-10T15:54:12Z", + "database_bytes": 39573871, + "wal_bytes": 19744568, + "wal_stats_reset": "2026-08-10T13:37:53Z", + "extrapolated_wal_gb_per_day": 0.208, + "source": "read-only pg_database_size and pg_stat_wal query on platform-pg-1" + }, + "scenarios": { + "low": {"initial_database_gb": 1, "monthly_database_growth_pct": 5, "wal_gb_per_day": 0.25, "restore_egress_gb": 1, "write_requests_per_month": 1000, "read_requests_per_month": 500, "operator_hours_per_month": 0.5, "operator_hourly_eur": 60}, + "base": {"initial_database_gb": 5, "monthly_database_growth_pct": 10, "wal_gb_per_day": 1, "restore_egress_gb": 5, "write_requests_per_month": 2500, "read_requests_per_month": 1000, "operator_hours_per_month": 1, "operator_hourly_eur": 60}, + "high": {"initial_database_gb": 20, "monthly_database_growth_pct": 20, "wal_gb_per_day": 5, "restore_egress_gb": 20, "write_requests_per_month": 12000, "read_requests_per_month": 5000, "operator_hours_per_month": 2, "operator_hourly_eur": 60} + }, + "notes": [ + "Observed workload is less than one day old; scenario floors intentionally dominate the raw sample.", + "Storage forecast assumes 30 retained daily base backups plus 30 days of WAL and is conservative because it does not model compression or Barman's exact point-of-recoverability deletion timing.", + "Provider exit is modeled as one full stored-byte egress event and is reported separately from recurring monthly cost." + ] +} diff --git a/data/forecasts/platform-audit-storage-scaleway-base-2026-08.json b/data/forecasts/platform-audit-storage-scaleway-base-2026-08.json new file mode 100644 index 0000000..6909752 --- /dev/null +++ b/data/forecasts/platform-audit-storage-scaleway-base-2026-08.json @@ -0,0 +1,24 @@ +{ + "schema_version": "0.1", + "record_type": "forecast", + "workload": "platform-pg", + "cost_attribution_key": "platform:audit-storage", + "provider_id": "scaleway-standard-multi-az", + "created_at": "2026-08-10T17:10:00Z", + "scenario": "base", + "forecast_ref": null, + "rows": [ + {"period":"2026-09","database_gb":5,"stored_gb":180,"wal_gb":30,"restore_egress_gb":5,"write_requests":2500,"read_requests":1000,"infrastructure_eur":2.89,"internal_labor_hours":1,"internal_labor_eur":60,"total_eur":62.89}, + {"period":"2026-10","database_gb":5.5,"stored_gb":195,"wal_gb":30,"restore_egress_gb":5,"write_requests":2500,"read_requests":1000,"infrastructure_eur":3.13,"internal_labor_hours":1,"internal_labor_eur":60,"total_eur":63.13}, + {"period":"2026-11","database_gb":6.05,"stored_gb":211.5,"wal_gb":30,"restore_egress_gb":5,"write_requests":2500,"read_requests":1000,"infrastructure_eur":3.4,"internal_labor_hours":1,"internal_labor_eur":60,"total_eur":63.4}, + {"period":"2026-12","database_gb":6.655,"stored_gb":229.65,"wal_gb":30,"restore_egress_gb":5,"write_requests":2500,"read_requests":1000,"infrastructure_eur":3.69,"internal_labor_hours":1,"internal_labor_eur":60,"total_eur":63.69}, + {"period":"2027-01","database_gb":7.321,"stored_gb":249.615,"wal_gb":30,"restore_egress_gb":5,"write_requests":2500,"read_requests":1000,"infrastructure_eur":4.01,"internal_labor_hours":1,"internal_labor_eur":60,"total_eur":64.01}, + {"period":"2027-02","database_gb":8.053,"stored_gb":271.577,"wal_gb":30,"restore_egress_gb":5,"write_requests":2500,"read_requests":1000,"infrastructure_eur":4.36,"internal_labor_hours":1,"internal_labor_eur":60,"total_eur":64.36}, + {"period":"2027-03","database_gb":8.858,"stored_gb":295.734,"wal_gb":30,"restore_egress_gb":5,"write_requests":2500,"read_requests":1000,"infrastructure_eur":4.75,"internal_labor_hours":1,"internal_labor_eur":60,"total_eur":64.75}, + {"period":"2027-04","database_gb":9.744,"stored_gb":322.308,"wal_gb":30,"restore_egress_gb":5,"write_requests":2500,"read_requests":1000,"infrastructure_eur":5.18,"internal_labor_hours":1,"internal_labor_eur":60,"total_eur":65.18}, + {"period":"2027-05","database_gb":10.718,"stored_gb":351.538,"wal_gb":30,"restore_egress_gb":5,"write_requests":2500,"read_requests":1000,"infrastructure_eur":5.65,"internal_labor_hours":1,"internal_labor_eur":60,"total_eur":65.65}, + {"period":"2027-06","database_gb":11.79,"stored_gb":383.692,"wal_gb":30,"restore_egress_gb":5,"write_requests":2500,"read_requests":1000,"infrastructure_eur":6.16,"internal_labor_hours":1,"internal_labor_eur":60,"total_eur":66.16}, + {"period":"2027-07","database_gb":12.969,"stored_gb":419.061,"wal_gb":30,"restore_egress_gb":5,"write_requests":2500,"read_requests":1000,"infrastructure_eur":6.73,"internal_labor_hours":1,"internal_labor_eur":60,"total_eur":66.73}, + {"period":"2027-08","database_gb":14.266,"stored_gb":457.968,"wal_gb":30,"restore_egress_gb":5,"write_requests":2500,"read_requests":1000,"infrastructure_eur":7.35,"internal_labor_hours":1,"internal_labor_eur":60,"total_eur":67.35} + ] +} diff --git a/data/optimization/platform-audit-storage-2026-08.json b/data/optimization/platform-audit-storage-2026-08.json new file mode 100644 index 0000000..6f53b02 --- /dev/null +++ b/data/optimization/platform-audit-storage-2026-08.json @@ -0,0 +1,157 @@ +{ + "schema_version": "0.1", + "record_scope": "operational", + "case_id": "opt:platform-audit-storage:2026-08", + "case_type": "provider_switch", + "trigger": "procurement", + "review_period": "2026-08", + "created_at": "2026-08-14T00:00:00Z", + "resource_ids": [ + "resource:platform:audit-storage" + ], + "baseline": { + "option_id": "scaleway-standard-multi-az", + "label": "Scaleway Standard Multi-AZ object storage (nl-ams), provisional primary from due diligence", + "one_time_eur": 0, + "recurring_infrastructure_eur_month": 7.35, + "recurring_internal_labor_eur_month": 60, + "recurring_external_labor_eur_month": 0, + "utilization": { + "stored_gb": { + "provisioned": { "value": 457.968, "unit": "GB" }, + "used": { "value": 457.968, "unit": "GB" } + } + }, + "uncertainty": { + "level": "medium", + "notes": [ + "Month-12 base-scenario figures from tools/cost_model.py, not a booked invoice", + "Elastic capacity: provisioned equals used by construction, so the ratio does not signal waste", + "Published list price; no account quote or committed discount" + ] + }, + "service_constraints": { + "retention_days": { "value": 30, "unit": "days" }, + "target_rpo_minutes": { "value": 5, "unit": "minutes" } + }, + "failure_domains": [ + "provider:scaleway", + "region:nl-ams" + ], + "exit_path": "Delete objects and buckets, revoke the scoped access key, close the project, and confirm removal; modelled exit cost EUR 243.83.", + "unknowns": [] + }, + "alternatives": [ + { + "option_id": "hetzner-object-storage", + "label": "Hetzner Object Storage (nbg1), monthly base price including 1 TB storage and egress", + "one_time_eur": 0, + "recurring_infrastructure_eur_month": 6.49, + "recurring_internal_labor_eur_month": 90.0, + "recurring_external_labor_eur_month": 0, + "utilization": { + "stored_gb": { + "provisioned": { "value": 1000, "unit": "GB" }, + "used": { "value": 457.968, "unit": "GB" } + } + }, + "uncertainty": { + "level": "medium", + "notes": [ + "Priced only within the included 1 TB quota; excess storage and egress rates are not published in the recorded source", + "The base scenario stays inside the quota, so the high scenario cannot be evaluated from this option" + ] + }, + "service_constraints": { + "retention_days": { "value": 30, "unit": "days" }, + "target_rpo_minutes": { "value": 5, "unit": "minutes" } + }, + "failure_domains": [ + "provider:hetzner", + "region:nbg1" + ], + "exit_path": "Delete objects and buckets, revoke the S3 credential, cancel the subscription; modelled exit cost EUR 240.00.", + "unknowns": [] + }, + { + "option_id": "host-europe-cloud-storage", + "label": "Host Europe Cloud Storage, same provider as the reef-railiance compute host", + "one_time_eur": null, + "recurring_infrastructure_eur_month": null, + "recurring_internal_labor_eur_month": 120, + "recurring_external_labor_eur_month": 0, + "utilization": { + "stored_gb": { + "provisioned": { "value": null, "unit": "GB" }, + "used": { "value": 457.968, "unit": "GB" } + } + }, + "uncertainty": { + "level": "high", + "notes": [ + "Only a historical published price and service specification exists; current orderability for this account is unconfirmed", + "Co-located with railiance01, so a provider-level failure would remove the primary host and the backup copy together" + ] + }, + "service_constraints": { + "retention_days": { "value": 30, "unit": "days" }, + "target_rpo_minutes": { "value": 5, "unit": "minutes" } + }, + "failure_domains": [ + "provider:host-europe" + ], + "exit_path": null, + "unknowns": [ + "current S3 orderability for this account (owner: human account authority via Host Europe support)", + "storage_eur_per_gb_month, monthly_minimum_eur, egress_eur_per_gb, operations_eur_per_month, support_eur_per_month", + "cancellation terms and minimum contract period", + "correlated-failure acceptance for same-provider placement (owner: human financial and risk authority)" + ] + } + ], + "decision": { + "state": "blocked_on_evidence", + "recommended_option_id": null, + "rationale": "This case exercises the optimization process against real cost-model output; it is not the procurement decision, which RESOURCE-WP-0002-T03 owns and human financial authority approves. Hetzner computes fully and is EUR 29.14 per month more expensive than the Scaleway baseline at month-12 base demand, driven by operator labour rather than storage price. Host Europe cannot be compared at all: its price, cancellation terms, and current orderability are unknown, and it shares a failure domain with the compute host it is meant to protect. No provider is recommended until Host Europe returns written account evidence.", + "approver": null, + "approved_on": null, + "delegated_to": [] + }, + "financial_handoff": { + "cost_attribution_key": "platform:audit-storage", + "sent": false, + "reference": null + }, + "outcome": { + "feeds_forecast": [ + "data/forecasts/platform-audit-storage-scaleway-base-2026-08.json" + ], + "actual_refs": [] + }, + "evidence": [ + { + "kind": "forecast", + "ref": "tools/cost_model.py base scenario month 12 over data/demand/platform-audit-storage.json", + "authority": "resource-control", + "observed_at": "2026-08-14" + }, + { + "kind": "document", + "ref": "docs/evidence/RESOURCE-WP-0002-provider-due-diligence-2026-08-10.md", + "authority": "resource-control", + "observed_at": "2026-08-10" + }, + { + "kind": "document", + "ref": "docs/evidence/RESOURCE-WP-0002-expanded-storage-comparison-2026-08-10.md", + "authority": "resource-control", + "observed_at": "2026-08-10" + }, + { + "kind": "workplan", + "ref": "RESOURCE-WP-0002-T03", + "authority": "resource-control", + "observed_at": null + } + ] +} diff --git a/data/optimization/reef-railiance-k3s-2026-08.json b/data/optimization/reef-railiance-k3s-2026-08.json new file mode 100644 index 0000000..494789d --- /dev/null +++ b/data/optimization/reef-railiance-k3s-2026-08.json @@ -0,0 +1,200 @@ +{ + "schema_version": "0.1", + "record_scope": "operational", + "case_id": "opt:reef-railiance-k3s:2026-08", + "case_type": "rightsizing", + "trigger": "cadence", + "review_period": "2026-08", + "created_at": "2026-08-14T00:00:00Z", + "resource_ids": [ + "resource:railiance:reef-railiance:k3s", + "resource:hosteurope:railiance01" + ], + "baseline": { + "option_id": "railiance01-single-node", + "label": "Current single-node k3s on the Host Europe railiance01 server (4 vCPU, 15.62 GiB)", + "one_time_eur": 0, + "recurring_infrastructure_eur_month": null, + "recurring_internal_labor_eur_month": null, + "recurring_external_labor_eur_month": 0, + "utilization": { + "cpu": { + "provisioned": { "value": 4, "unit": "vCPU" }, + "used": { "value": 0.564, "unit": "vCPU" } + }, + "memory": { + "provisioned": { "value": 15.62, "unit": "GiB" }, + "used": { "value": 5.83, "unit": "GiB" } + }, + "root_filesystem": { + "provisioned": { "value": 192.69, "unit": "GiB" }, + "used": { "value": 66.77, "unit": "GiB" } + } + }, + "uncertainty": { + "level": "high", + "notes": [ + "Capacity and usage are a single live observation on 2026-08-11, not a utilization history", + "A one-off sample cannot distinguish steady-state headroom from a quiet moment; peak and growth are unknown", + "The server carries Helix Forge, Coulomb Social, rapp-qonto, and shared platform services, so downsizing headroom is shared, not per-workload" + ] + }, + "service_constraints": { + "nodes": { "value": 1, "unit": "count" }, + "pod_limit": { "value": 110, "unit": "count" } + }, + "failure_domains": [ + "provider:host-europe", + "host:railiance01", + "cluster:reef-railiance" + ], + "exit_path": "Provision replacement capacity, restore or redeploy the cluster and stateful services, switch DNS and ingress, verify workloads, then cancel and erase the server.", + "unknowns": [ + "railiance01 product, booked monthly price, tax treatment, and renewal or cancellation dates (owner: railiance-infra, RAIL-HO-WP-0008)", + "cluster operations labour hours per month (owner: railiance-cluster, RAIL-BS-WP-0014)", + "requested-versus-used capacity history and peak load (owner: railiance-cluster, RAIL-BS-WP-0014)", + "declared service objectives for the workloads sharing the node (owner: workload repositories)" + ] + }, + "alternatives": [ + { + "option_id": "smaller-single-server", + "label": "Consolidate onto a smaller Host Europe server sized to observed usage plus headroom", + "one_time_eur": null, + "recurring_infrastructure_eur_month": null, + "recurring_internal_labor_eur_month": null, + "recurring_external_labor_eur_month": 0, + "utilization": { + "cpu": { + "provisioned": { "value": null, "unit": "vCPU" }, + "used": { "value": 0.564, "unit": "vCPU" } + }, + "memory": { + "provisioned": { "value": null, "unit": "GiB" }, + "used": { "value": 5.83, "unit": "GiB" } + } + }, + "uncertainty": { + "level": "high", + "notes": [ + "No target instance can be sized without peak-load history; observed mean usage is not a sizing basis", + "Migration requires a full cluster rebuild and stateful restore, so one-time labour is material and unmeasured", + "Retains the single-host failure domain the baseline already carries" + ] + }, + "service_constraints": { + "nodes": { "value": 1, "unit": "count" } + }, + "failure_domains": [ + "provider:host-europe", + "host:railiance01", + "cluster:reef-railiance" + ], + "exit_path": null, + "unknowns": [ + "Host Europe server catalogue, sizes, and prices available to this account (owner: railiance-infra, RAIL-HO-WP-0008)", + "peak CPU and memory over a full billing period, required to size the target (owner: railiance-cluster, RAIL-BS-WP-0014)", + "one-time migration labour and workload downtime budget (owner: railiance-cluster)" + ] + }, + { + "option_id": "threephoenix-ha-cluster", + "label": "Three-node HA cluster per RAIL-BS-WP-0007, removing the single-host failure domain", + "one_time_eur": null, + "recurring_infrastructure_eur_month": null, + "recurring_internal_labor_eur_month": null, + "recurring_external_labor_eur_month": 0, + "utilization": { + "cpu": { + "provisioned": { "value": null, "unit": "vCPU" }, + "used": { "value": 0.564, "unit": "vCPU" } + }, + "memory": { + "provisioned": { "value": null, "unit": "GiB" }, + "used": { "value": 5.83, "unit": "GiB" } + } + }, + "uncertainty": { + "level": "high", + "notes": [ + "This option raises recurring cost by design; it is justified by availability, not by saving, and payback is the wrong test for it", + "Node count and placement are owned by RAIL-BS-WP-0007, not by this case" + ] + }, + "service_constraints": { + "nodes": { "value": 3, "unit": "count" } + }, + "failure_domains": [ + "provider:host-europe", + "cluster:reef-railiance" + ], + "exit_path": null, + "unknowns": [ + "per-node infrastructure price and whether nodes span independent failure domains (owner: railiance-cluster, RAIL-BS-WP-0014)", + "operations labour for a multi-node cluster versus a single node (owner: railiance-cluster)", + "declared availability objective that would justify the added recurring cost (owner: workload repositories)", + "replicated-storage requirement created by removing local-path storage (owner: railiance-cluster)" + ] + } + ], + "decision": { + "state": "blocked_on_evidence", + "recommended_option_id": null, + "rationale": "Observed utilization is genuinely low: 14 percent of CPU, 37 percent of memory, and 35 percent of the root filesystem on a single sample. That is a real rightsizing signal, and it is not sufficient to act on. No option can be costed because the railiance01 booked price is unknown, so no saving, no payback, and no comparison between downsizing and the HA alternative can be computed. The case stays open and blocked against the named delegated workplans rather than producing a rightsizing recommendation from a mean-usage sample.", + "approver": null, + "approved_on": null, + "delegated_to": [ + "railiance-infra", + "railiance-cluster" + ] + }, + "financial_handoff": { + "cost_attribution_key": null, + "sent": false, + "reference": null + }, + "outcome": { + "feeds_forecast": [ + "examples/control-cycle/cluster-forecast.json" + ], + "actual_refs": [] + }, + "evidence": [ + { + "kind": "telemetry", + "ref": "kubernetes:reef-railiance/node/239.62.205.92.host.secureserver.net@2026-08-11T09:05:25Z", + "authority": "reef-railiance Kubernetes API", + "observed_at": "2026-08-11" + }, + { + "kind": "telemetry", + "ref": "ssh:railiance01 host capacity observation 2026-08-11T09:05:25Z", + "authority": "railiance01 operating system", + "observed_at": "2026-08-11" + }, + { + "kind": "document", + "ref": "docs/evidence/RESOURCE-WP-0003-initial-portfolio-discovery-2026-08-11.md", + "authority": "resource-control", + "observed_at": "2026-08-11" + }, + { + "kind": "workplan", + "ref": "RAIL-HO-WP-0008", + "authority": "railiance-infra", + "observed_at": null + }, + { + "kind": "workplan", + "ref": "RAIL-BS-WP-0014", + "authority": "railiance-cluster", + "observed_at": null + }, + { + "kind": "workplan", + "ref": "RAIL-BS-WP-0007", + "authority": "railiance-cluster", + "observed_at": null + } + ] +} diff --git a/data/portfolio-coverage-2026-08-11.json b/data/portfolio-coverage-2026-08-11.json new file mode 100644 index 0000000..30401c2 --- /dev/null +++ b/data/portfolio-coverage-2026-08-11.json @@ -0,0 +1,20 @@ +{ + "schema_version": "0.1", + "observed_at": "2026-08-11T09:05:25Z", + "inventory_records": 7, + "coverage": [ + {"group": "helix-forge", "status": "covered-by-shared-forge", "resource_ids": ["resource:railiance:forgejo", "resource:railiance:reef-railiance:k3s", "resource:hosteurope:railiance01"]}, + {"group": "coulomb-social", "status": "covered", "resource_ids": ["resource:tenant:coulomb:coulomb-social", "resource:railiance:apps-pg", "resource:railiance:reef-railiance:k3s", "resource:hosteurope:railiance01"]}, + {"group": "shared-railiance", "status": "covered-substrate-and-selected-services", "resource_ids": ["resource:railiance:reef-railiance:k3s", "resource:railiance:forgejo", "resource:railiance:apps-pg", "resource:hosteurope:railiance01", "resource:platform:audit-storage"]}, + {"group": "representative-tenant", "status": "covered", "tenant_id": "tenant:friendly:binky", "resource_ids": ["resource:tenant:friendly:binky:rapp-qonto", "resource:railiance:reef-railiance:k3s", "resource:hosteurope:railiance01"]} + ], + "owned_gaps": [ + {"owner": "railiance-infra", "gap": "Host Europe product, provider resource ID, account reference, country/region evidence, contract price, tax treatment, order date, renewal and cancellation dates", "delegated_workplan": "RAIL-HO-WP-0008", "state_hub_workplan_id": "7122657f-87c8-46b5-a725-a1af1ba0af12"}, + {"owner": "railiance-cluster", "gap": "cluster-wide requested/used capacity history, allocation driver, generic persistent-storage contract, and independent failure domains", "delegated_workplan": "RAIL-BS-WP-0014", "state_hub_workplan_id": "ea6ec98a-0d65-4afa-b6c5-6e7ed34011ac"}, + {"owner": "railiance-forge", "gap": "Forgejo resource requests/utilization, storage growth, operations labor, and cost-allocation driver", "delegated_workplan": "RAILIANCE-WP-0002", "state_hub_workplan_id": "72f935a5-d921-48a4-96e4-2365556c7374"}, + {"owner": "railiance-platform", "gap": "apps-pg utilization, backup/restore evidence, operations labor, and consumer allocation driver", "delegated_workplan": "RAILIANCE-WP-0016", "state_hub_workplan_id": "49084fb8-de63-4f32-a4a9-3a42d4e708ac"}, + {"owner": "coulomb-social", "gap": "declared CPU/memory demand, usage proxies, service objectives, and application operations labor", "delegated_workplan": "CSOC-WP-0005", "state_hub_workplan_id": "00de1578-6150-40d3-b193-e852666ed945"}, + {"owner": "rapp-qonto", "gap": "request/cold-start frequency, external API costs, runtime utilization, and recurring operator labor", "delegated_workplan": "RAPP-QONTO-WP-0002", "state_hub_workplan_id": "25b3b715-9a56-499e-a85c-a09ebd9bbb79"}, + {"owner": "fin-hub", "gap": "booked Host Europe financial facts joined to resource:hosteurope:railiance01", "delegated_workplan": "FIN-WP-0004", "state_hub_workplan_id": "67b6de6c-4820-4478-9789-f50260204c27", "delegated_tasks": ["FIN-WP-0004-T04", "FIN-WP-0004-T05", "FIN-WP-0004-T06"]} + ] +} diff --git a/data/providers/object-storage.json b/data/providers/object-storage.json new file mode 100644 index 0000000..68a9ebb --- /dev/null +++ b/data/providers/object-storage.json @@ -0,0 +1,86 @@ +{ + "schema_version": "0.1", + "observed_at": "2026-08-10", + "prices_exclude_vat": true, + "fx": {"usd_per_eur": 1.1555, "observed_at": "2026-08-10", "source": "https://www.ecb.europa.eu/stats/eurofxref/eurofxref-daily.xml"}, + "providers": [ + { + "id": "host-europe-cloud-storage", + "provider": "Host Europe", + "service_class": "Cloud Storage (historical S3 specification)", + "region": null, + "storage_eur_per_gb_month": null, + "monthly_minimum_eur": null, + "included_storage_gb": 0, + "included_egress_gb": 0, + "egress_eur_per_gb": null, + "operations_eur_per_month": null, + "support_eur_per_month": null, + "operator_hours_per_month": 2, + "status": "blocked-account-confirmation", + "source": "https://www.hosteurope.de/download/PVL/Preis_und_Leistungsverzeichnis_CloudStorage_1-0-1.pdf" + }, + { + "id": "scaleway-standard-multi-az", + "provider": "Scaleway", + "service_class": "Standard Multi-AZ", + "region": "nl-ams", + "storage_eur_per_gb_month": 0.01606, + "monthly_minimum_eur": 0, + "included_storage_gb": 0, + "included_egress_gb": 75, + "egress_eur_per_gb": 0.01, + "operations_eur_per_month": 0, + "support_eur_per_month": 0, + "operator_hours_per_month": 1, + "status": "priced", + "source": "https://www.scaleway.com/en/pricing/storage/" + }, + { + "id": "hetzner-object-storage", + "provider": "Hetzner", + "service_class": "Object Storage", + "region": "nbg1", + "storage_eur_per_gb_month": null, + "monthly_minimum_eur": 6.49, + "included_storage_gb": 1000, + "included_egress_gb": 1000, + "egress_eur_per_gb": null, + "operations_eur_per_month": 0, + "support_eur_per_month": 0, + "operator_hours_per_month": 1.5, + "status": "priced-within-included-quota", + "source": "https://www.hetzner.com/storage/object-storage/" + }, + { + "id": "host-europe-garage-1", "provider": "Host Europe", "service_class": "Self-managed Garage, 1 VM", "region": "Germany", "storage_eur_per_gb_month": null, "monthly_minimum_eur": 31.92, "included_storage_gb": 400, "included_egress_gb": 100000, "egress_eur_per_gb": 0, "operations_eur_per_month": 0, "support_eur_per_month": 0, "operator_hours_per_month": 2, "setup_operator_hours": 8, "topology": "1 x 8 vCPU / 16 GB / 400 GB NVMe; replication factor 1; no redundancy", "status": "estimate-not-production", "source": "https://www.hosteurope.de/Server/Virtual-Server/" + }, + { + "id": "host-europe-garage-2", "provider": "Host Europe", "service_class": "Self-managed Garage, 2 VMs", "region": "Germany", "storage_eur_per_gb_month": null, "monthly_minimum_eur": 63.85, "included_storage_gb": 400, "included_egress_gb": 100000, "egress_eur_per_gb": 0, "operations_eur_per_month": 0, "support_eur_per_month": 0, "operator_hours_per_month": 3, "setup_operator_hours": 12, "topology": "2 x 8 vCPU / 16 GB / 400 GB NVMe; replication factor 2; one provider/location", "status": "estimate-degraded-quorum", "source": "https://www.hosteurope.de/Server/Virtual-Server/" + }, + { + "id": "host-europe-garage-3", "provider": "Host Europe", "service_class": "Self-managed Garage, 3 VMs", "region": "Germany", "storage_eur_per_gb_month": null, "monthly_minimum_eur": 95.77, "included_storage_gb": 400, "included_egress_gb": 100000, "egress_eur_per_gb": 0, "operations_eur_per_month": 0, "support_eur_per_month": 0, "operator_hours_per_month": 4, "setup_operator_hours": 16, "topology": "3 x 8 vCPU / 16 GB / 400 GB NVMe; replication factor 3; one provider/location", "status": "estimate-production-candidate", "source": "https://www.hosteurope.de/Server/Virtual-Server/" + }, + { + "id": "hetzner-garage-1", "provider": "Hetzner", "service_class": "Self-managed Garage, 1 VM", "region": "nbg1", "storage_eur_per_gb_month": null, "monthly_minimum_eur": 30.09, "included_storage_gb": 320, "included_egress_gb": 20000, "egress_eur_per_gb": 0, "operations_eur_per_month": 0, "support_eur_per_month": 0, "operator_hours_per_month": 2, "setup_operator_hours": 8, "topology": "1 x CX53 / 16 vCPU / 32 GB / 320 GB local SSD plus IPv4; replication factor 1", "status": "estimate-not-production", "source": "https://docs.hetzner.com/general/infrastructure-and-availability/price-adjustment/" + }, + { + "id": "hetzner-garage-2", "provider": "Hetzner", "service_class": "Self-managed Garage, 2 VMs", "region": "nbg1", "storage_eur_per_gb_month": null, "monthly_minimum_eur": 60.18, "included_storage_gb": 320, "included_egress_gb": 20000, "egress_eur_per_gb": 0, "operations_eur_per_month": 0, "support_eur_per_month": 0, "operator_hours_per_month": 3, "setup_operator_hours": 12, "topology": "2 x CX53 / 16 vCPU / 32 GB / 320 GB local SSD plus IPv4; replication factor 2", "status": "estimate-degraded-quorum", "source": "https://docs.hetzner.com/general/infrastructure-and-availability/price-adjustment/" + }, + { + "id": "hetzner-garage-3", "provider": "Hetzner", "service_class": "Self-managed Garage, 3 VMs", "region": "nbg1", "storage_eur_per_gb_month": null, "monthly_minimum_eur": 90.27, "included_storage_gb": 320, "included_egress_gb": 20000, "egress_eur_per_gb": 0, "operations_eur_per_month": 0, "support_eur_per_month": 0, "operator_hours_per_month": 4, "setup_operator_hours": 16, "topology": "3 x CX53 / 16 vCPU / 32 GB / 320 GB local SSD plus IPv4; replication factor 3 in one location", "status": "estimate-production-candidate", "source": "https://docs.hetzner.com/general/infrastructure-and-availability/price-adjustment/" + }, + { + "id": "aws-s3-standard", "provider": "AWS", "service_class": "S3 Standard", "region": "eu-central-1", "storage_eur_per_gb_month": 0.02120, "monthly_minimum_eur": 0, "included_storage_gb": 0, "included_egress_gb": 100, "egress_eur_per_gb": 0.07789, "write_eur_per_1000": 0.00467, "read_eur_per_1000": 0.00037, "operations_eur_per_month": 0, "support_eur_per_month": 0, "operator_hours_per_month": 1.5, "setup_operator_hours": 4, "status": "priced", "source": "https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonS3/current/eu-central-1/index.json" + }, + { + "id": "azure-blob-hot-zrs", "provider": "Microsoft Azure", "service_class": "Blob Storage Hot ZRS (not S3 API)", "region": "germanywestcentral", "storage_eur_per_gb_month": 0.0197, "monthly_minimum_eur": 0, "included_storage_gb": 0, "included_egress_gb": 0, "egress_eur_per_gb": 0.08, "write_eur_per_1000": 0.00474, "read_eur_per_1000": 0.00038, "operations_eur_per_month": 0, "support_eur_per_month": 0, "operator_hours_per_month": 2, "setup_operator_hours": 6, "status": "priced-egress-estimate", "source": "https://prices.azure.com/api/retail/prices" + }, + { + "id": "gcp-cloud-storage-standard", "provider": "Google Cloud", "service_class": "Cloud Storage Standard regional (not S3 API)", "region": "europe-west3", "storage_eur_per_gb_month": 0.01731, "monthly_minimum_eur": 0, "included_storage_gb": 0, "included_egress_gb": 0, "egress_eur_per_gb": 0.10, "write_eur_per_1000": 0.00433, "read_eur_per_1000": 0.00035, "operations_eur_per_month": 0, "support_eur_per_month": 0, "operator_hours_per_month": 2, "setup_operator_hours": 6, "status": "priced-egress-estimate", "source": "https://cloud.google.com/storage/pricing" + }, + { + "id": "stackit-object-storage", "provider": "STACKIT", "service_class": "Object Storage Premium-EU01", "region": "Germany South", "storage_eur_per_gb_month": 0.02662, "monthly_minimum_eur": 0, "included_storage_gb": 0, "included_egress_gb": 100000, "egress_eur_per_gb": 0, "write_eur_per_1000": 0, "read_eur_per_1000": 0, "operations_eur_per_month": 0, "support_eur_per_month": 0, "operator_hours_per_month": 1.5, "setup_operator_hours": 4, "status": "priced", "source": "https://www.stackit.de/de/preise__trashed/cloud-services/iaas/stackit-storage/" + } + ] +} diff --git a/data/resources/apps-pg.json b/data/resources/apps-pg.json new file mode 100644 index 0000000..b8ddfed --- /dev/null +++ b/data/resources/apps-pg.json @@ -0,0 +1,15 @@ +{ + "schema_version": "0.2", "record_scope": "inventory", + "id": "resource:railiance:apps-pg", "resource_class": "database", + "status": "active", "management_model": "self_managed", + "provider": {"name": "Railiance", "account_ref": null, "product_ref": "cloudnative-pg", "provider_resource_id": "kubernetes:databases/cluster/apps-pg"}, + "service": {"name": "Shared applications PostgreSQL", "service_id": "apps-pg", "class": "CloudNativePG database service", "capacity_model": "shared"}, + "location": {"region": null, "country": null, "failure_domains": ["cluster:reef-railiance", "host:railiance01", "storage:local-path"], "residency": null}, + "capacity": [{"metric": "instances", "value": 1, "unit": "count", "kind": "observed", "observed_at": "2026-08-11"}, {"metric": "storage", "value": 10, "unit": "GiB", "kind": "provisioned", "observed_at": "2026-08-11"}], + "ownership": {"owner": "railiance-platform", "environment": "production", "workload_ids": ["coulomb-social"], "tenant_id": null, "allocation": {"mode": "unattributed", "cost_attribution_key": null, "driver": null, "method_version": null}}, + "cost": {"currency": "EUR", "tax_status": "not_applicable", "billing_model": "shared cluster and operations cost; allocation method unknown", "commitment_ref": null, "price_evidence": null}, + "lifecycle": {"proposed_on": null, "ordered_on": null, "commissioned_on": "2026-08-09", "renews_on": null, "cancel_by": null, "retired_on": null, "exit_path": "Restore databases to a replacement PostgreSQL service, rotate application connections, verify consumers, then retire the cluster and volume."}, + "relationships": [{"type": "hosted_on", "resource_id": "resource:railiance:reef-railiance:k3s"}], + "requirements": [{"kind": "recovery", "ref": "/home/worsch/railiance-platform/docs/apps-pg.md"}], + "evidence": [{"kind": "telemetry", "ref": "kubernetes:reef-railiance/databases/cluster/apps-pg@2026-08-11", "observed_at": "2026-08-11", "authority": "reef-railiance Kubernetes API"}, {"kind": "workload", "ref": "/home/worsch/railiance-platform/helm/apps-pg-cluster.yaml", "observed_at": "2026-08-11", "authority": "railiance-platform"}] +} diff --git a/data/resources/binky-rapp-qonto.json b/data/resources/binky-rapp-qonto.json new file mode 100644 index 0000000..482425c --- /dev/null +++ b/data/resources/binky-rapp-qonto.json @@ -0,0 +1,26 @@ +{ + "schema_version": "0.2", "record_scope": "inventory", + "id": "resource:tenant:friendly:binky:rapp-qonto", "resource_class": "self_managed_service", + "status": "active", "management_model": "shared_capacity", + "provider": {"name": "Railiance", "account_ref": null, "product_ref": "rail-knative", "provider_resource_id": "knative:rapp-qonto/rapp-qonto"}, + "service": {"name": "Binky Qonto assistant", "service_id": "rapp-qonto", "class": "tenant Knative application with dedicated egress proxy", "capacity_model": "elastic"}, + "location": {"region": null, "country": null, "failure_domains": ["cluster:reef-railiance", "host:railiance01"], "residency": null}, + "capacity": [ + {"metric": "minimum_replicas", "value": 0, "unit": "count", "kind": "limit", "observed_at": "2026-08-11"}, + {"metric": "maximum_replicas", "value": 1, "unit": "count", "kind": "limit", "observed_at": "2026-08-11"}, + {"metric": "workload_cpu_request", "value": 0.05, "unit": "vCPU", "kind": "allocated", "observed_at": "2026-08-11"}, + {"metric": "workload_memory_request", "value": 128, "unit": "MiB", "kind": "allocated", "observed_at": "2026-08-11"}, + {"metric": "egress_cpu_request", "value": 0.02, "unit": "vCPU", "kind": "allocated", "observed_at": "2026-08-11"}, + {"metric": "egress_memory_request", "value": 64, "unit": "MiB", "kind": "allocated", "observed_at": "2026-08-11"} + ], + "ownership": {"owner": "rapp-qonto", "environment": "production", "workload_ids": ["rapp-qonto"], "tenant_id": "tenant:friendly:binky", "allocation": {"mode": "dedicated", "cost_attribution_key": "resource:tenant:friendly:binky:rapp-qonto", "driver": "direct-workload", "method_version": "direct-v1"}}, + "cost": {"currency": "EUR", "tax_status": "not_applicable", "billing_model": "allocated cluster and operator cost plus external Qonto/API costs; values unknown", "commitment_ref": null, "price_evidence": null}, + "lifecycle": {"proposed_on": null, "ordered_on": null, "commissioned_on": "2026-07-27", "renews_on": null, "cancel_by": null, "retired_on": null, "exit_path": "Deploy the verified Knative revision and egress policy on replacement capacity, verify tenant authorization and Qonto access, then retire old revisions and credentials."}, + "relationships": [{"type": "hosted_on", "resource_id": "resource:railiance:reef-railiance:k3s"}], + "requirements": [{"kind": "security", "ref": "/home/worsch/rapp-qonto/docs/security-and-reliability-gates.md"}], + "evidence": [ + {"kind": "telemetry", "ref": "kubernetes:reef-railiance/rapp-qonto/ksvc/rapp-qonto@2026-08-11", "observed_at": "2026-08-11", "authority": "reef-railiance Kubernetes API"}, + {"kind": "workload", "ref": "/home/worsch/rapp-qonto/bindings/rail-knative.yaml", "observed_at": "2026-08-11", "authority": "rapp-qonto"}, + {"kind": "test", "ref": "/home/worsch/rapp-qonto/evidence/live/2026-07-29-railiance01.json", "observed_at": "2026-07-29", "authority": "rapp-qonto"} + ] +} diff --git a/data/resources/coulomb-social-production.json b/data/resources/coulomb-social-production.json new file mode 100644 index 0000000..8d22c6c --- /dev/null +++ b/data/resources/coulomb-social-production.json @@ -0,0 +1,19 @@ +{ + "schema_version": "0.2", "record_scope": "inventory", + "id": "resource:tenant:coulomb:coulomb-social", "resource_class": "self_managed_service", + "status": "active", "management_model": "shared_capacity", + "provider": {"name": "Railiance", "account_ref": null, "product_ref": "coulomb-social", "provider_resource_id": "kubernetes:coulomb-social/deployment/coulomb-social"}, + "service": {"name": "Coulomb Social production", "service_id": "coulomb-social", "class": "tenant application", "capacity_model": "shared"}, + "location": {"region": null, "country": null, "failure_domains": ["cluster:reef-railiance", "host:railiance01"], "residency": null}, + "capacity": [{"metric": "replicas", "value": 1, "unit": "count", "kind": "observed", "observed_at": "2026-08-11"}], + "ownership": {"owner": "coulomb-social", "environment": "production", "workload_ids": ["coulomb-social"], "tenant_id": "tenant:coulomb", "allocation": {"mode": "dedicated", "cost_attribution_key": "resource:tenant:coulomb:coulomb-social", "driver": "direct-workload", "method_version": "direct-v1"}}, + "cost": {"currency": "EUR", "tax_status": "not_applicable", "billing_model": "allocated share of cluster, database, ingress, identity dependencies, and operations labor; values unknown", "commitment_ref": null, "price_evidence": null}, + "lifecycle": {"proposed_on": null, "ordered_on": null, "commissioned_on": "2026-08-09", "renews_on": null, "cancel_by": null, "retired_on": null, "exit_path": "Deploy the application and restore tenant data on replacement capacity, verify identity and health paths, switch app.coulomb.social, then retire the old release."}, + "relationships": [{"type": "hosted_on", "resource_id": "resource:railiance:reef-railiance:k3s"}, {"type": "depends_on", "resource_id": "resource:railiance:apps-pg"}], + "requirements": [{"kind": "security", "ref": "/home/worsch/coulomb-social/INTENT.md"}], + "evidence": [ + {"kind": "telemetry", "ref": "kubernetes:reef-railiance/coulomb-social/deployment/coulomb-social@2026-08-11", "observed_at": "2026-08-11", "authority": "reef-railiance Kubernetes API"}, + {"kind": "workload", "ref": "/home/worsch/railiance-apps/helm/coulomb-social-values.yaml", "observed_at": "2026-08-11", "authority": "railiance-apps"}, + {"kind": "workload", "ref": "/home/worsch/coulomb-social/INTENT.md", "observed_at": "2026-08-11", "authority": "coulomb-social"} + ] +} diff --git a/data/resources/hosteurope-railiance01.json b/data/resources/hosteurope-railiance01.json new file mode 100644 index 0000000..01b6d55 --- /dev/null +++ b/data/resources/hosteurope-railiance01.json @@ -0,0 +1,23 @@ +{ + "schema_version": "0.2", "record_scope": "inventory", + "id": "resource:hosteurope:railiance01", "resource_class": "compute_instance", + "status": "active", "management_model": "provider_managed", + "provider": {"name": "Host Europe", "account_ref": null, "product_ref": null, "provider_resource_id": null}, + "service": {"name": "railiance01", "service_id": "railiance01", "class": "virtual server", "capacity_model": "fixed"}, + "location": {"region": null, "country": null, "failure_domains": ["provider:host-europe", "host:railiance01"], "residency": null}, + "capacity": [ + {"metric": "cpu", "value": 4, "unit": "vCPU", "kind": "usable", "observed_at": "2026-08-11"}, + {"metric": "memory", "value": 15.62, "unit": "GiB", "kind": "usable", "observed_at": "2026-08-11"}, + {"metric": "root_filesystem", "value": 192.69, "unit": "GiB", "kind": "provisioned", "observed_at": "2026-08-11"}, + {"metric": "root_filesystem_used", "value": 66.77, "unit": "GiB", "kind": "observed", "observed_at": "2026-08-11"} + ], + "ownership": {"owner": "railiance-infra", "environment": "production", "workload_ids": ["reef-railiance"], "tenant_id": null, "allocation": {"mode": "unattributed", "cost_attribution_key": null, "driver": null, "method_version": null}}, + "cost": {"currency": "EUR", "tax_status": "unknown", "billing_model": "fixed recurring provider server; exact product and booked price unknown", "commitment_ref": null, "price_evidence": null}, + "lifecycle": {"proposed_on": null, "ordered_on": null, "commissioned_on": null, "renews_on": null, "cancel_by": null, "retired_on": null, "exit_path": "Provision replacement capacity, restore or redeploy cluster and stateful services, switch DNS and ingress, verify workloads, then cancel and erase the server."}, + "relationships": [{"type": "provides_capacity_to", "resource_id": "resource:railiance:reef-railiance:k3s"}], + "requirements": [{"kind": "other", "ref": "/home/worsch/railiance-cluster/workplans/RAIL-BS-WP-0007-threephoenix-ha-cluster.md"}], + "evidence": [ + {"kind": "telemetry", "ref": "ssh:railiance01 host capacity observation 2026-08-11T09:05:25Z", "observed_at": "2026-08-11", "authority": "railiance01 operating system"}, + {"kind": "workload", "ref": "/home/worsch/railiance-apps/docs/operator-setup.md", "observed_at": "2026-08-11", "authority": "railiance-apps"} + ] +} diff --git a/data/resources/platform-audit-storage.proposed.json b/data/resources/platform-audit-storage.proposed.json new file mode 100644 index 0000000..f863ce0 --- /dev/null +++ b/data/resources/platform-audit-storage.proposed.json @@ -0,0 +1,67 @@ +{ + "schema_version": "0.2", + "record_scope": "inventory", + "id": "resource:platform:audit-storage", + "resource_class": "storage", + "status": "proposed", + "management_model": "provider_managed", + "provider": { + "name": "Scaleway", + "account_ref": null, + "product_ref": "scaleway-standard-multi-az", + "provider_resource_id": null + }, + "service": { + "name": "platform audit storage", + "service_id": "object-storage", + "class": "Standard Multi-AZ object storage", + "capacity_model": "elastic" + }, + "location": { + "region": "nl-ams", + "country": "NL", + "failure_domains": ["provider:scaleway", "region:nl-ams"], + "residency": "European Union" + }, + "capacity": [ + {"metric": "stored_data", "value": null, "unit": "GB", "kind": "unknown", "observed_at": null} + ], + "ownership": { + "owner": "resource-control", + "environment": "production", + "workload_ids": ["rapp-postgres/platform-pg"], + "tenant_id": null, + "allocation": { + "mode": "dedicated", + "cost_attribution_key": "platform:audit-storage", + "driver": "direct-resource", + "method_version": "direct-v1" + } + }, + "cost": { + "currency": "EUR", + "tax_status": "excluded", + "billing_model": "usage-based storage and egress; no commitment", + "commitment_ref": null, + "price_evidence": "data/providers/object-storage.json#scaleway-standard-multi-az" + }, + "lifecycle": { + "proposed_on": "2026-08-10", + "ordered_on": null, + "commissioned_on": null, + "renews_on": null, + "cancel_by": null, + "retired_on": null, + "exit_path": "Copy and verify all Barman objects in an independent S3-compatible target, run full and PITR restore, revoke the scoped key, then delete source objects and bucket after evidence is durable." + }, + "relationships": [], + "requirements": [ + {"kind": "recovery", "ref": "workplans/RESOURCE-WP-0002-procure-postgres-backup-storage.md#acceptance-requirements"}, + {"kind": "retention", "ref": "data/demand/platform-audit-storage.json"} + ], + "evidence": [ + {"kind": "provider", "ref": "https://www.scaleway.com/en/pricing/storage/", "observed_at": "2026-08-10", "authority": "Scaleway"}, + {"kind": "provider", "ref": "https://www.scaleway.com/en/object-storage/", "observed_at": "2026-08-10", "authority": "Scaleway"}, + {"kind": "decision", "ref": "docs/evidence/RESOURCE-WP-0002-provider-due-diligence-2026-08-10.md", "observed_at": "2026-08-10", "authority": "resource-control"} + ] +} diff --git a/data/resources/railiance-forgejo.json b/data/resources/railiance-forgejo.json new file mode 100644 index 0000000..5db12cd --- /dev/null +++ b/data/resources/railiance-forgejo.json @@ -0,0 +1,23 @@ +{ + "schema_version": "0.2", "record_scope": "inventory", + "id": "resource:railiance:forgejo", "resource_class": "shared_platform_service", + "status": "active", "management_model": "self_managed", + "provider": {"name": "Railiance", "account_ref": null, "product_ref": "forgejo-11.0.3", "provider_resource_id": "kubernetes:forgejo/deployment/forgejo-gitea"}, + "service": {"name": "Forgejo source and package forge", "service_id": "forgejo", "class": "shared source forge and package registry", "capacity_model": "shared"}, + "location": {"region": null, "country": null, "failure_domains": ["cluster:reef-railiance", "host:railiance01", "storage:local-path"], "residency": null}, + "capacity": [ + {"metric": "replicas", "value": 1, "unit": "count", "kind": "observed", "observed_at": "2026-08-11"}, + {"metric": "forge_storage", "value": 10, "unit": "GiB", "kind": "provisioned", "observed_at": "2026-08-11"}, + {"metric": "database_storage", "value": 10, "unit": "GiB", "kind": "provisioned", "observed_at": "2026-08-11"} + ], + "ownership": {"owner": "railiance-forge", "environment": "production", "workload_ids": ["helix-forge", "coulomb-social", "railiance"], "tenant_id": null, "allocation": {"mode": "unattributed", "cost_attribution_key": null, "driver": null, "method_version": null}}, + "cost": {"currency": "EUR", "tax_status": "not_applicable", "billing_model": "shared share of cluster, persistent storage, database, backup, and operations labor; allocation unknown", "commitment_ref": null, "price_evidence": null}, + "lifecycle": {"proposed_on": null, "ordered_on": null, "commissioned_on": "2026-07-03", "renews_on": null, "cancel_by": null, "retired_on": null, "exit_path": "Restore repositories, packages, configuration, and database to a replacement forge, verify clone/push/package workflows, switch DNS, then retire the deployment."}, + "relationships": [{"type": "hosted_on", "resource_id": "resource:railiance:reef-railiance:k3s"}], + "requirements": [{"kind": "recovery", "ref": "/home/worsch/railiance-apps/docs/forgejo-package-registry.md"}], + "evidence": [ + {"kind": "telemetry", "ref": "kubernetes:reef-railiance/forgejo/deployment/forgejo-gitea@2026-08-11", "observed_at": "2026-08-11", "authority": "reef-railiance Kubernetes API"}, + {"kind": "workload", "ref": "/home/worsch/railiance-apps/helm/forgejo-values.yaml", "observed_at": "2026-08-11", "authority": "railiance-apps"}, + {"kind": "decision", "ref": "/home/worsch/railiance-apps/docs/forge-source-of-truth-decision.md", "observed_at": "2026-08-11", "authority": "railiance-apps"} + ] +} diff --git a/data/resources/reef-railiance-k3s.json b/data/resources/reef-railiance-k3s.json new file mode 100644 index 0000000..d1729a3 --- /dev/null +++ b/data/resources/reef-railiance-k3s.json @@ -0,0 +1,26 @@ +{ + "schema_version": "0.2", "record_scope": "inventory", + "id": "resource:railiance:reef-railiance:k3s", "resource_class": "kubernetes_capacity", + "status": "active", "management_model": "self_managed", + "provider": {"name": "Railiance", "account_ref": null, "product_ref": "k3s", "provider_resource_id": "k3s://239.62.205.92.host.secureserver.net"}, + "service": {"name": "reef-railiance k3s", "service_id": "reef-railiance", "class": "single-node Kubernetes runtime", "capacity_model": "shared"}, + "location": {"region": null, "country": null, "failure_domains": ["provider:host-europe", "host:railiance01", "cluster:reef-railiance"], "residency": null}, + "capacity": [ + {"metric": "cpu", "value": 4, "unit": "vCPU", "kind": "usable", "observed_at": "2026-08-11"}, + {"metric": "memory", "value": 15.62, "unit": "GiB", "kind": "usable", "observed_at": "2026-08-11"}, + {"metric": "ephemeral_storage", "value": 183.06, "unit": "GiB", "kind": "usable", "observed_at": "2026-08-11"}, + {"metric": "pods", "value": 110, "unit": "count", "kind": "limit", "observed_at": "2026-08-11"}, + {"metric": "nodes", "value": 1, "unit": "count", "kind": "observed", "observed_at": "2026-08-11"}, + {"metric": "cpu_usage", "value": 0.564, "unit": "vCPU", "kind": "observed", "observed_at": "2026-08-11"}, + {"metric": "memory_usage", "value": 5.83, "unit": "GiB", "kind": "observed", "observed_at": "2026-08-11"} + ], + "ownership": {"owner": "railiance-cluster", "environment": "production", "workload_ids": ["helix-forge", "coulomb-social", "rapp-qonto", "railiance-platform"], "tenant_id": null, "allocation": {"mode": "unattributed", "cost_attribution_key": null, "driver": null, "method_version": null}}, + "cost": {"currency": "EUR", "tax_status": "unknown", "billing_model": "allocated share of railiance01 plus cluster operations labor; method not established", "commitment_ref": null, "price_evidence": null}, + "lifecycle": {"proposed_on": null, "ordered_on": null, "commissioned_on": "2026-03-10", "renews_on": null, "cancel_by": null, "retired_on": null, "exit_path": "Deploy a replacement cluster, restore stateful services, promote workloads and ingress, verify health, then remove the old node and credentials."}, + "relationships": [{"type": "hosted_on", "resource_id": "resource:hosteurope:railiance01"}], + "requirements": [{"kind": "service_objective", "ref": "/home/worsch/railiance-cluster/INTENT.md"}, {"kind": "recovery", "ref": "/home/worsch/railiance-cluster/workplans/RAIL-BS-WP-0007-threephoenix-ha-cluster.md"}], + "evidence": [ + {"kind": "telemetry", "ref": "kubernetes:reef-railiance/node/239.62.205.92.host.secureserver.net@2026-08-11T09:05:25Z", "observed_at": "2026-08-11", "authority": "reef-railiance Kubernetes API"}, + {"kind": "workload", "ref": "/home/worsch/railiance-cluster/docs/rail-kubernetes-substrate-profile.md", "observed_at": "2026-08-11", "authority": "railiance-cluster"} + ] +} diff --git a/docs/evidence/RESOURCE-WP-0002-demand-and-cost-model-2026-08-10.md b/docs/evidence/RESOURCE-WP-0002-demand-and-cost-model-2026-08-10.md new file mode 100644 index 0000000..df05c1f --- /dev/null +++ b/docs/evidence/RESOURCE-WP-0002-demand-and-cost-model-2026-08-10.md @@ -0,0 +1,54 @@ +# PostgreSQL backup demand and cost model — 2026-08-10 + +## Observed baseline + +A read-only query against `platform-pg-1` at `2026-08-10T15:54:12Z` measured +39,573,871 bytes across connectable databases. `pg_stat_wal` reported +19,744,568 bytes since `2026-08-10T13:37:53Z`, an extrapolated 0.208 GB/day. +The cluster is less than one day old, so this is a seed observation rather than +a representative trend. No credential value was read or recorded. + +The governed requirements are a 30-day recovery window, daily physical base +backup, continuous WAL archive, one monthly restore drill, and a complete exit +calculation. The scenario inputs live in +`data/demand/platform-audit-storage.json` and deliberately floor demand above +the tiny initial observation: + +| Scenario | Initial DB | Monthly growth | WAL/day | Restore egress | Labor/month | +| --- | ---: | ---: | ---: | ---: | ---: | +| low | 1 GB | 5% | 0.25 GB | 1 GB | 0.5 h | +| base | 5 GB | 10% | 1 GB | 5 GB | 1 h | +| high | 20 GB | 20% | 5 GB | 20 GB | 2 h | + +The conservative stored-byte formula is `30 × current database size + 30 × +daily WAL`. It does not claim compression savings. Operator labor is valued at +€60/hour; provider-specific labor floors capture relative operational effort. +The exit event adds full stored-byte egress plus four operator hours. + +Run `make forecast` for all 12 months, scenarios, and providers. A `null` total +is intentional: it means a required price is unknown and prevents an unknown +from silently becoming free. + +The catalog also includes self-managed Garage on 1–3 Host Europe/Hetzner VMs +and managed AWS, Azure, Google Cloud, and STACKIT price points. See +`RESOURCE-WP-0002-expanded-storage-comparison-2026-08-10.md` for assumptions +and topology warnings. + +Calculator rows split `monthly_infrastructure_eur` from +`monthly_internal_labor_eur` and split internal setup labor from external setup +services. The top-level `comparison_320gb` array provides a normalized running +cost at 320 GB using the base request and restore pattern. + +## Initial comparison + +- Scaleway Standard Multi-AZ is fully calculable at €0.01606/GB-month, with + 75 GB/month free egress then €0.01/GB. Prices exclude tax. +- Hetzner is calculable while usage remains inside its €6.49 monthly minimum, + which includes 1 TB storage and 1 TB egress. Excess unit prices still need to + be captured before high-growth or exit usage crosses the included quota. +- Host Europe cannot be calculated until it confirms that the historical S3 + Cloud Storage product is currently orderable and supplies a current quote. + +Because labor dominates this very small workload, selection should emphasize +restore compatibility, independent failure domain, encryption, and exit proof +before sub-euro storage differences. diff --git a/docs/evidence/RESOURCE-WP-0002-expanded-storage-comparison-2026-08-10.md b/docs/evidence/RESOURCE-WP-0002-expanded-storage-comparison-2026-08-10.md new file mode 100644 index 0000000..5f11d54 --- /dev/null +++ b/docs/evidence/RESOURCE-WP-0002-expanded-storage-comparison-2026-08-10.md @@ -0,0 +1,106 @@ +# Expanded object-storage comparison — 2026-08-10 + +This adds self-managed S3-compatible storage and four large-cloud price points +to the managed-provider comparison. All recurring totals include operator labor +at €60/hour. Taxes are excluded. USD prices use the 2026-08-10 ECB reference +rate of 1 EUR = 1.1555 USD. + +## Self-managed reference architecture + +Garage is the reference open-source implementation. It has an S3-compatible +API and is designed for distributed deployments. Garage explicitly says its +single-node quick-start has no redundancy and must not be used in production. +CNPG/Barman compatibility remains a live preflight requirement. + +The initial fixed-size estimates use locally attached storage: + +- Host Europe: 8 vCPU, 16 GB RAM, 400 GB NVMe at €37.99/month including 19% + VAT, normalized to €31.92 net per node. Network traffic is flat-rate. +- Hetzner: CX53, 16 vCPU, 32 GB RAM, 320 GB local SSD at €29.49/month plus + €0.50 IPv4, net per node. The June 2026 price adjustment is used. + +Replication consumes the extra nodes; it does not multiply usable capacity. +All nodes at one provider/location still share provider and site failure risk. + +| Topology | Infrastructure/month | Internal labor/month | Running total at 320 GB | Internal setup | External setup | Usable ceiling | Assessment | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | --- | +| Host Europe, 1 node | €31.92 | 2 h / €120 | €151.92 | 8 h / €480 | not quoted | 400 GB | no redundancy; not production | +| Host Europe, 2 nodes | €63.85 | 3 h / €180 | €243.85 | 12 h / €720 | not quoted | 400 GB | replicated, but poor failure/quorum shape | +| Host Europe, 3 nodes | €95.77 | 4 h / €240 | €335.77 | 16 h / €960 | not quoted | 400 GB | production candidate; single provider/site risk | +| Hetzner, 1 node | €30.09 | 2 h / €120 | €150.09 | 8 h / €480 | not quoted | 320 GB | no redundancy; not production | +| Hetzner, 2 nodes | €60.18 | 3 h / €180 | €240.18 | 12 h / €720 | not quoted | 320 GB | replicated, but poor failure/quorum shape | +| Hetzner, 3 nodes | €90.27 | 4 h / €240 | €330.27 | 16 h / €960 | not quoted | 320 GB | production candidate; single location risk | + +These sizes cover the base scenario's first month (180 GB), but not its month +12 estimate (458 GB). The calculator therefore returns `null` once the fixed +capacity is exceeded. A larger Host Europe 800 GB node tier or Hetzner Volumes +would be required; obtain account quotes before treating that expansion as a +decision price. Hetzner also documents that its Volumes are already replicated +across three physical servers, which means combining three Garage replicas with +three replicated Volumes would buy nested redundancy at additional cost. + +## Managed object-storage price points + +Azure Blob and Google Cloud Storage are Barman-supported object stores but are +not S3-compatible APIs. AWS and STACKIT are direct S3 API comparisons. The +following table fixes stored volume at **320 GB**, uses the base request pattern +(2,500 writes and 1,000 reads/month), and includes one 5 GB restore drill. This +puts elastic per-GB products, Hetzner's bundled minimum, and Garage's fixed +capacity on the same running-cost basis. + +| Service | Infrastructure at 320 GB | Internal labor/month | Running total at 320 GB | Internal setup | External setup | +| --- | ---: | ---: | ---: | ---: | ---: | +| Scaleway Standard Multi-AZ | €5.14 | 1 h / €60 | **€65.14** | none | €0 assumed | +| Hetzner Object Storage | €6.49 | 1.5 h / €90 | **€96.49** | none | €0 assumed | +| AWS S3 Standard, Frankfurt | €6.80 | 1.5 h / €90 | **€96.80** | 4 h / €240 | €0 assumed | +| STACKIT Object Storage Premium-EU01 | €8.52 | 1.5 h / €90 | **€98.52** | 4 h / €240 | €0 assumed | +| Google Cloud Storage Standard, Frankfurt | €6.05 | 2 h / €120 | **€126.05** | 6 h / €360 | €0 assumed | +| Azure Blob Hot ZRS, Germany West Central | €6.72 | 2 h / €120 | **€126.72** | 6 h / €360 | €0 assumed | + +Cloud request costs are negligible here; operator effort and egress assumptions +matter more. Azure/GCP egress figures are explicit planning estimates rather +than captured retail meters, so they must be refreshed in an account calculator +before decision approval. + +"Internal setup" values the federation's own implementation time at €60/hour. +"External setup" means paid consulting, vendor professional services, or a +contractor. No such service is assumed for managed self-service onboarding. It +is **not quoted**, rather than zero, for Garage because an external security or +operations review may be prudent and no scope or rate has been obtained. + +The apparent difference between the earlier €62.89 Scaleway total and the +normalized €5.14 infrastructure figure is reconciled in +`docs/forecast-actual-control.md`: €62.89 is 180 GB infrastructure (€2.89) plus +€60 labor, while €5.14 is infrastructure alone at 320 GB. The comparable 320 GB +total is €65.14. + +## Interpretation + +Self-management does not win at this workload once even two hours/month of +patching, monitoring, certificate handling, upgrades, disk replacement, +capacity management, and restore support are valued. Three-node Garage costs +roughly €330–€336/month plus €960 setup labor and still shares a provider/site +failure domain. It may become attractive if the platform wants the capability +for broader workloads, can amortize operations across them, or deliberately +values open-source control above the cost difference. + +For this backup alone, Scaleway remains the lowest modeled managed option. +STACKIT is especially interesting as a sovereign three-AZ S3-compatible +alternative: its documentation now states automatic replication across all +three availability zones, versioning, lifecycle rules, server-side encryption, +bucket policies, and Object Lock. + +## Primary sources + +- Host Europe VM pricing: https://www.hosteurope.de/Server/Virtual-Server/ +- Hetzner June 2026 pricing: https://docs.hetzner.com/general/infrastructure-and-availability/price-adjustment/ +- Hetzner Volume replication: https://docs.hetzner.com/cloud/volumes/overview/ +- Garage single-node warning: https://garagehq.deuxfleurs.fr/documentation/ +- AWS S3 price list: https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonS3/current/eu-central-1/index.json +- AWS egress/free allowance: https://aws.amazon.com/ec2/pricing/on-demand/ +- Azure retail pricing API: https://learn.microsoft.com/en-us/rest/api/cost-management/retail-prices/azure-retail-prices +- Google Cloud Storage pricing: https://cloud.google.com/storage/pricing +- STACKIT storage pricing: https://www.stackit.de/de/preise__trashed/cloud-services/iaas/stackit-storage/ +- STACKIT object-storage capabilities: https://docs.stackit.cloud/de/products/storage/object-storage/basics/introduction/ +- STACKIT network pricing: https://www.stackit.de/de/preise__trashed/cloud-services/iaas/stackit-network/ +- ECB daily FX rate: https://www.ecb.europa.eu/stats/eurofxref/eurofxref-daily.xml diff --git a/docs/evidence/RESOURCE-WP-0002-provider-due-diligence-2026-08-10.md b/docs/evidence/RESOURCE-WP-0002-provider-due-diligence-2026-08-10.md new file mode 100644 index 0000000..8758125 --- /dev/null +++ b/docs/evidence/RESOURCE-WP-0002-provider-due-diligence-2026-08-10.md @@ -0,0 +1,71 @@ +# Object-storage provider due diligence — 2026-08-10 + +Status vocabulary: `yes`, `no`, `unknown`, or `test`. `Test` means documentation +is promising but the live CNPG/Barman preflight remains mandatory. + +| Requirement | Host Europe Cloud Storage | Scaleway Standard Multi-AZ | Hetzner Object Storage | +| --- | --- | --- | --- | +| Currently orderable for account | unknown — written confirmation required | yes | yes | +| S3-compatible HTTPS / SigV4 | unknown | yes / test | yes / test | +| EU region considered | unknown | Amsterdam `nl-ams` | Nuremberg `nbg1` | +| Different provider from reef-railiance | no | yes | yes | +| Multi-AZ / availability | unknown | yes; 99.9% service availability | unknown; docs describe Ceph and infrastructure redundancy, not a quantified service SLA | +| Published durability | unknown | 99.999999999% | unknown | +| Scoped access key | unknown | yes; bucket policy test required | yes; key defaults to all project buckets, bucket policy required | +| Private bucket | unknown | yes | yes | +| Multipart/list/get/put/delete | unknown | yes / test | yes / test | +| Versioning | unknown | yes | yes | +| Lifecycle expiration | unknown | yes | yes | +| Object lock | unknown | yes; cannot later disable | yes; enable at bucket creation | +| TLS encryption | unknown | yes; enforceable by bucket policy | yes / test | +| Encryption at rest | unknown | yes: SSE-ONE, SSE-C, or SSE-KMS | **no default**; SSE-C is documented | +| Initial 100 GiB without long commitment | unknown | yes, usage-based | yes, hourly base billing | +| Operations pricing | unknown | no separate request charge found; confirm invoice model | PUT/GET/DELETE free | +| Storage / egress pricing | unknown | €0.01606/GB-month; 75 GB egress free, then €0.01/GB | €6.49 minimum includes 1 TB storage and 1 TB egress; excess prices still to capture | +| Cancellation / support terms | unknown | contract review required | contract review required | +| Provider status visibility | unknown | yes | yes | +| CNPG 1.25/Barman compatibility | unknown | test | test | + +## Evidence + +- Host Europe Backup Storage supports FTP, SCP, SFTP, and rsync, so it is a + secondary logical-copy candidate, not the primary Barman object store: + https://www.hosteurope.de/faq/server/virtual-server/backup-storage/ +- Host Europe's S3 claim is currently only backed by an older specification: + https://www.hosteurope.de/download/PVL/Preis_und_Leistungsverzeichnis_CloudStorage_1-0-1.pdf +- Scaleway pricing: https://www.scaleway.com/en/pricing/storage/ +- Scaleway capabilities and durability: + https://www.scaleway.com/en/object-storage/ +- Scaleway regions/lifecycle/multipart: + https://www.scaleway.com/en/docs/object-storage/concepts/ +- Scaleway object lock: + https://www.scaleway.com/en/docs/object-storage/how-to/use-object-lock/ +- Hetzner overview, endpoints, quotas, operations, and limits: + https://docs.hetzner.com/storage/object-storage/overview/ +- Hetzner supported S3 actions and encryption limitation: + https://docs.hetzner.com/storage/object-storage/supported-actions/ +- Hetzner encryption FAQ: + https://docs.hetzner.com/storage/object-storage/faq/general/ +- CNPG 1.25 S3-compatible endpoint and lifecycle guidance: + https://cloudnative-pg.io/docs/1.25/appendixes/object_stores/ + +## Blocking evidence requests + +1. Host Europe: ask account support to confirm current orderability, endpoint, + region, SigV4, supported API, versioning/lifecycle, encryption, durability, + availability, price, support, and cancellation terms in writing. +2. Scaleway and Hetzner: obtain account-visible contractual/support terms and + confirm tax treatment. Capture Hetzner excess prices. +3. Run a disposable live preflight for each finalist: private bucket, scoped + positive and negative credentials, multipart CRUD, versioning/lifecycle, + Barman empty-WAL-archive check, and deletion/exit behavior. + +## Provisional conclusion + +Host Europe is not selectable as primary while current S3 orderability is +unknown. Hetzner fails the current acceptance requirement for provider-managed +encryption at rest unless client-side/SSE-C custody is explicitly accepted. +Scaleway Standard Multi-AZ is therefore the provisional primary, subject to a +live Barman preflight, contract review, and human purchase approval. Hetzner +remains a price comparator; Host Europe Backup Storage or the governed +Nextcloud lane remains the independent logical-copy candidate. diff --git a/docs/evidence/RESOURCE-WP-0003-delegated-evidence-workplans-2026-08-11.md b/docs/evidence/RESOURCE-WP-0003-delegated-evidence-workplans-2026-08-11.md new file mode 100644 index 0000000..5d58cb8 --- /dev/null +++ b/docs/evidence/RESOURCE-WP-0003-delegated-evidence-workplans-2026-08-11.md @@ -0,0 +1,25 @@ +# RESOURCE-WP-0003 delegated evidence workplans — 2026-08-11 + +## Result + +Every owned gap from the initial portfolio discovery now points to a live, +repository-local work record. Resource-control retains the inventory, +forecast, actual-observation, allocation, and optimization contract; source +repositories retain authority for infrastructure and workload evidence; and +fin-hub retains booked financial facts. + +| Evidence owner | Workplan | State Hub workplan ID | Interface outcome | +|---|---|---|---| +| railiance-infra | `RAIL-HO-WP-0008` | `7122657f-87c8-46b5-a725-a1af1ba0af12` | Host identity, lifecycle, commercial and host-capacity evidence | +| railiance-cluster | `RAIL-BS-WP-0014` | `ea6ec98a-0d65-4afa-b6c5-6e7ed34011ac` | Cluster capacity, utilization, storage/failure-domain and allocation evidence | +| railiance-forge | `RAILIANCE-WP-0002` | `72f935a5-d921-48a4-96e4-2365556c7374` | Forgejo demand, utilization, labor/SLO and allocation evidence | +| railiance-platform | `RAILIANCE-WP-0016` | `49084fb8-de63-4f32-a4a9-3a42d4e708ac` | apps-pg utilization, recovery, labor and consumer evidence | +| coulomb-social | `CSOC-WP-0005` | `00de1578-6150-40d3-b193-e852666ed945` | Workload forecasts, observations, objectives and labor evidence | +| rapp-qonto | `RAPP-QONTO-WP-0002` | `25b3b715-9a56-499e-a85c-a09ebd9bbb79` | Scale-to-zero usage, external service cost inputs and labor evidence | +| fin-hub | `FIN-WP-0004-T04..T06` | `67b6de6c-4820-4478-9789-f50260204c27` | Budgets/constraints, forecast-to-actual loop and generalized attribution | + +The authoritative machine-readable mapping is +`data/portfolio-coverage-2026-08-11.json`. State Hub synchronization completed +without failures. Its warnings concern pre-existing fleet prefix collisions, +unregistered legacy task prefixes, and missing optional DoR metadata; none +prevented registration of these plans. diff --git a/docs/evidence/RESOURCE-WP-0003-fin-hub-contract-preflight-2026-08-11.md b/docs/evidence/RESOURCE-WP-0003-fin-hub-contract-preflight-2026-08-11.md new file mode 100644 index 0000000..525bbed --- /dev/null +++ b/docs/evidence/RESOURCE-WP-0003-fin-hub-contract-preflight-2026-08-11.md @@ -0,0 +1,64 @@ +# RESOURCE-WP-0003 fin-hub contract preflight + +Date: 2026-08-11 +Resource-control task: `RESOURCE-WP-0003-T02` +Fin-hub tasks: `FIN-WP-0004-T01`, `T02`, `T03`, `T07`, `T08`, `T09` +Fin-hub commit: `0034330` + +## Result + +Resource-control accepts the authority and schema boundary in the joint v0.1 +contract. Fin-hub's remediation was reviewed against the timestamped +assessment and its full suite passed: + +```text +46 passed, 1 third-party deprecation warning +``` + +Resource-control's suite passed after adding its producer-owned schema, +forecast exporter, booked-cost consumer validation, and reconciliation seam: + +```text +13 passed +resource-control declarations: valid +``` + +## Cross-repository check + +The live immutable backup forecast +`data/forecasts/platform-audit-storage-scaleway-base-2026-08.json` was processed +through both implementations using: + +- resource-control `tools/financial_exchange.py::forecast_records`; and +- fin-hub `fin_hub.services.exchange::ingest_resource_forecast`. + +Observed result: + +```json +{ + "producer_consumer_records": 12, + "idempotent_store_rows": 12, + "payloads_equal": true, + "booked_cost_gate": "awaits attributable provider fact" +} +``` + +Replaying the same forecast did not add rows. All canonical record payloads +matched, including IDs, periods, cost breakdown, attribution, assumptions, and +provenance reference. + +## Residual acceptance gate + +This preflight proves the planning direction; it is not evidence of purchased +storage or actual spend. `FIN-WP-0004-T05` remains waiting until +`RESOURCE-WP-0002` produces a real provider financial fact attributable to +`platform:audit-storage`. That round trip must then demonstrate: + +1. exactly one authoritative booked fact in fin-hub; +2. a non-ledger projection consumed by resource-control; +3. a join to the immutable forecast and technical observation; +4. infrastructure-cost variance with the financial fact ID preserved; and +5. duplicate delivery and any correction leaving effective totals correct. + +Synthetic records remain suitable for automated contract tests but cannot +satisfy this operational gate. diff --git a/docs/evidence/RESOURCE-WP-0003-generalized-control-cycle-2026-08-11.md b/docs/evidence/RESOURCE-WP-0003-generalized-control-cycle-2026-08-11.md new file mode 100644 index 0000000..72d17bd --- /dev/null +++ b/docs/evidence/RESOURCE-WP-0003-generalized-control-cycle-2026-08-11.md @@ -0,0 +1,36 @@ +# RESOURCE-WP-0003 generalized control cycle — 2026-08-11 + +## Result + +`RESOURCE-WP-0003-T05` generalized the backup-specific forecast-to-actual +method into a provider-neutral portfolio contract. + +The implementation consists of: + +- `schemas/resource-control-cycle.schema.json` for immutable forecast and + actual records; +- `tools/control_cycle.py` for resource-neutral comparison and controlled + variance attribution; +- six paired examples covering storage, fixed cluster compute, and a hybrid + shared PostgreSQL service; +- tests for the three resource classes, immutable forecast binding, explicit + cost attribution, and fail-closed data-quality handling; +- updated operating guidance in `docs/forecast-actual-control.md`. + +## Boundary with fin-hub + +Actual records carry `booked_cost_refs`; resource-control does not reproduce +the ledger fact. Fin-hub message `d33f2452-8dc6-4249-a3d2-84fda938103a` +confirmed explicit `financial_fact_id -> resource_id` binding, budget and +constraint signals, and generalized exchange health. The first operational +backup round trip remains correctly gated on `RESOURCE-WP-0002` procurement; +that does not invalidate these illustrative contract fixtures. + +## Verification + +- `make test`: 25 tests passed and declarations validated. +- All three example pairs completed through `make control-cycle`. +- `git diff --check`: clean. + +The examples are intentionally marked illustrative. Their values must not be +reported as actual consumption or booked cost. diff --git a/docs/evidence/RESOURCE-WP-0003-initial-portfolio-discovery-2026-08-11.md b/docs/evidence/RESOURCE-WP-0003-initial-portfolio-discovery-2026-08-11.md new file mode 100644 index 0000000..668fc4b --- /dev/null +++ b/docs/evidence/RESOURCE-WP-0003-initial-portfolio-discovery-2026-08-11.md @@ -0,0 +1,81 @@ +# RESOURCE-WP-0003 initial portfolio discovery + +Date: 2026-08-11 +Task: `RESOURCE-WP-0003-T03` +Coverage data: `data/portfolio-coverage-2026-08-11.json` + +## Result + +The first real portfolio slice contains seven inventory records: + +| Resource | Owner | Role | +| --- | --- | --- | +| `resource:hosteurope:railiance01` | `railiance-infra` | provider compute host | +| `resource:railiance:reef-railiance:k3s` | `railiance-cluster` | shared single-node Kubernetes runtime | +| `resource:railiance:forgejo` | `railiance-forge` | Helix Forge source/package infrastructure | +| `resource:railiance:apps-pg` | `railiance-platform` | shared application database service | +| `resource:tenant:coulomb:coulomb-social` | `coulomb-social` | Coulomb reference-tenant application | +| `resource:tenant:friendly:binky:rapp-qonto` | `rapp-qonto` | representative tenant workload | +| `resource:platform:audit-storage` | `resource-control` | proposed backup storage | + +Helix Forge does not itself operate production infrastructure; its repository +scope explicitly delegates runtime operation. Its current infrastructure +coverage is therefore the shared Forgejo service and its underlying cluster +and host, not a fictional Helix Forge server. + +## Live substrate observation + +A read-only observation of `railiance01` and the reef-railiance Kubernetes API +at `2026-08-11T09:05:25Z` confirmed: + +- one Ready k3s control-plane/etcd node on `railiance01`; +- 4 allocatable vCPU, approximately 15.62 GiB allocatable memory, 183.06 GiB + allocatable ephemeral storage, and a 110-pod limit; +- node usage of 564m CPU and 5,975 MiB memory at observation time; +- a 192.69 GiB root filesystem, 66.77 GiB used; +- active Forgejo and Coulomb Social deployments with one ready replica each; +- active `apps-pg` with one ready CNPG instance and a 10 GiB volume; +- a healthy `rapp-qonto` Knative Service with min-scale 0 and max-scale 1; +- a continuously running Qonto egress proxy; and +- all discovered persistent volumes using `local-path`, therefore sharing the + host failure domain. + +No credential values or secret-bearing resources were read or recorded. + +## Source authority + +- `railiance-cluster` owns the k3s runtime and publishes the substrate profile. +- `railiance-apps` supplies current Forgejo and Coulomb Social deployment + values; `railiance-forge` is the declared forge-operation owner. +- `railiance-platform` owns the shared applications PostgreSQL definition. +- `coulomb-social` owns tenant and application behavior. +- `rapp-qonto` owns the Binky tenant binding, resource requests, Knative + package, and runtime requirements. +- the live operating system and Kubernetes API supplied timestamped capacity + and workload-state evidence. + +## Material findings + +The portfolio currently has one physical/virtual host and one Kubernetes node. +Compute, local persistent storage, databases, Forgejo, Coulomb Social, and the +tenant workload therefore share one provider/host failure domain. The planned +ThreePhoenix HA work remains backlog; the inventory does not describe its +target three nodes as current capacity. + +The active cluster has significant point-in-time headroom, but a single CPU and +memory sample is not a utilization history and cannot support rightsizing. +Commercial identifiers, booked server price, renewal dates, operations labor, +and shared allocation methods remain unknown. + +Each gap has an authoritative repository owner in the coverage data. Creating +live delegated work records for those gaps is `RESOURCE-WP-0003-T04`; this +discovery does not silently assign implementation through prose. + +## Verification + +```text +20 tests passed +resource-control declarations: valid +all inventory relationships resolve +git diff --check: pass +``` diff --git a/docs/evidence/RESOURCE-WP-0003-optimization-cases-2026-08-14.md b/docs/evidence/RESOURCE-WP-0003-optimization-cases-2026-08-14.md new file mode 100644 index 0000000..97c1577 --- /dev/null +++ b/docs/evidence/RESOURCE-WP-0003-optimization-cases-2026-08-14.md @@ -0,0 +1,104 @@ +# RESOURCE-WP-0003 optimization cases — 2026-08-14 + +## Result + +`RESOURCE-WP-0003-T06` established the review cadence, the decision template, +and a fail-closed evaluator for optimization cases, and validated the process +against the backup case and one non-storage portfolio candidate. + +The implementation consists of: + +- `schemas/optimization-case.schema.json` — baseline and alternatives carry + identical decision fields, so a comparison against an undescribed status quo + is structurally impossible; +- `tools/optimization.py` — comparison, payback, failure-domain delta, verdicts, + and decision-state validation; +- `data/optimization/platform-audit-storage-2026-08.json` — the storage case; +- `data/optimization/reef-railiance-k3s-2026-08.json` — the non-storage case; +- `docs/optimization-cases.md` — case types, decision template, cadence, and the + loop that returns outcomes to the next forecast; +- 25 tests in `tests/test_optimization.py`, plus schema and case checks in + `tools/validate.py`. + +## Decision fields + +T06 requires every recommendation to show baseline, alternative, one-time cost, +recurring infrastructure and labour cost, utilization, uncertainty, +service-level constraints, failure domains, exit path, and expected payback. +The schema requires all ten on every option, and the evaluator refuses to +recommend while any is `null`. Payback is computed from one-time cost and +realised monthly saving; it is never asserted. + +Recurring cost is split into infrastructure, internal labour, and external +labour. This is not cosmetic: in the storage case Hetzner has the lower +infrastructure price (EUR 6.49 against EUR 7.35) and is still EUR 29.14 per +month more expensive, entirely on operator hours. A comparison on unit price +alone would have inverted the result. + +## Fail-closed behaviour + +One unknown anywhere in the compared pair makes the comparison +`blocked_on_evidence`. Unknowns are `null` values plus a named `unknowns` list +that attributes each gap to an owning repository or authority. The evaluator +also refuses invalid decision states: a blocked case cannot be `proposed` or +`approved`, a decided case must name `approver` and `approved_on`, and a case +with no blockers cannot claim to be blocked. + +`reject` is a cost verdict only. The three-node cluster option raises recurring +cost by design and buys availability instead; that argument belongs to the +deciding authority against a declared service objective, not to the calculator. + +## Case 1 — `opt:platform-audit-storage:2026-08` (storage) + +Baseline is Scaleway Standard Multi-AZ, the provisional primary from T02 due +diligence, at month-12 base-scenario demand from `tools/cost_model.py`. + +| Option | Recurring EUR/month | Verdict | +|---|---|---| +| Scaleway Standard Multi-AZ (baseline) | 67.35 | — | +| Hetzner Object Storage | 96.49 | `reject` (+29.14) | +| Host Europe Cloud Storage | unknown | `blocked_on_evidence` | + +Host Europe is blocked on four named gaps: current S3 orderability for this +account, five price fields, cancellation terms, and correlated-failure +acceptance. It has no known exit path, and it shares `provider:host-europe` +with the compute host the backup is meant to protect. + +This case exercises the process against real cost-model output. It is +deliberately **not** the procurement decision, which `RESOURCE-WP-0002-T03` +owns and human financial authority approves. The case state is +`blocked_on_evidence` and recommends nothing. + +## Case 2 — `opt:reef-railiance-k3s:2026-08` (non-storage) + +Baseline is the current single-node k3s on railiance01, with the live capacity +observation from T03 discovery: 0.564 of 4 vCPU, 5.83 of 15.62 GiB, 66.77 of +192.69 GiB root filesystem. + +Utilization is genuinely low — 14 percent CPU, 37 percent memory — and that is +a real rightsizing signal. Every comparison is nevertheless blocked, for two +independent reasons the case records rather than works around: + +1. The railiance01 booked price is unknown, so no option has a recurring cost, + no saving exists, and no payback can be computed. +2. The utilization figure is a single sample taken on 2026-08-11. Mean usage is + not a sizing basis; peak load over a billing period is owed by + `RAIL-BS-WP-0014`. + +Both alternatives — a smaller single server and the RAIL-BS-WP-0007 three-node +cluster — are recorded with their unknowns attributed to `railiance-infra` +(`RAIL-HO-WP-0008`) and `railiance-cluster` (`RAIL-BS-WP-0014`). + +A case that refuses to conclude is the correct output here, and it is a useful +one: it names exactly which evidence would unblock a rightsizing decision, and +it is visible in the portfolio report until that evidence arrives. + +## Boundaries held + +`resource-control` produced both cases and decided neither. Financial +implications reach `fin-hub` through `financial_handoff` under +`docs/fin-hub-resource-control-contract-v0.1.md`; neither case has sent one, +because neither is approved. Implementation is delegated through +`decision.delegated_to`, which names `railiance-infra` and `railiance-cluster` +for the cluster case. `outcome.feeds_forecast` binds each case to the +control-cycle records that will later show whether it was right. diff --git a/docs/evidence/RESOURCE-WP-0003-portfolio-model-v0.2-2026-08-11.md b/docs/evidence/RESOURCE-WP-0003-portfolio-model-v0.2-2026-08-11.md new file mode 100644 index 0000000..d8ddc5d --- /dev/null +++ b/docs/evidence/RESOURCE-WP-0003-portfolio-model-v0.2-2026-08-11.md @@ -0,0 +1,71 @@ +# RESOURCE-WP-0003 portfolio model v0.2 + +Date: 2026-08-11 +Task: `RESOURCE-WP-0003-T01` +Schema: `schemas/resource-inventory.schema.json` + +## Result + +The inventory model is generalized from a storage purchase record into a +managed-infrastructure portfolio record. Version 0.2 represents: + +- provider-managed elastic storage; +- self-managed services hosted on rented capacity; +- Kubernetes or other shared capacity; +- shared platform services; and +- future compute, network, database, managed-service, and license records. + +The existing proposed Scaleway backup resource was migrated as the only +authoritative inventory record. The Garage, Kubernetes-capacity, and ingress +records live under `examples/portfolio/`, carry `record_scope: example`, and +contain explicit example evidence. They are schema proofs, not claims about +live infrastructure. + +## Model decisions + +- Resource identity uses the stable `resource:` namespace independently of + provider-native identifiers. +- Lifecycle separates proposed, ordered, commissioning, active, suspended, + retiring, retired, and rejected states. +- Provider-managed, self-managed, and shared-capacity management models are + explicit. +- Capacity is a set of typed dimensions and distinguishes provisioned, usable, + allocated, observed, limiting, and unknown values. +- Ownership supports multiple workloads, optional tenant identity, and a + dedicated, shared, or unattributed allocation mode. +- Shared allocations require a driver and versioned method. Unknown allocation + remains explicit rather than receiving an invented key. +- Relationships express hosting, composition, dependency, replacement, + capacity consumers, and backup placement without embedding another + repository's graph authority. +- Requirements and evidence carry their kind, reference, observation date, + and authoritative source. +- Exit path is mandatory for every resource class. + +## Semantic controls + +`tools/portfolio.py` adds controls not conveniently expressed as shape alone: + +- valid lifecycle transitions; +- ordered lifecycle dates; +- commissioned dates for operational resources; +- order evidence for operational provider-managed resources; +- allocation-key and shared-driver consistency; +- unique capacity metric/kind dimensions; +- prevention of self-relationships; and +- unmistakable example evidence. + +Verification: + +```text +18 tests passed +resource-control declarations: valid +git diff --check: pass +``` + +## Next use + +`RESOURCE-WP-0003-T03` should now discover real portfolio records for Helix +Forge, Coulomb Social, shared Railiance services, and one representative tenant +workload. It must replace example assumptions with source evidence and preserve +unknown values where authority is not yet available. diff --git a/docs/evidence/RESOURCE-WP-0003-portfolio-reporting-2026-08-14.md b/docs/evidence/RESOURCE-WP-0003-portfolio-reporting-2026-08-14.md new file mode 100644 index 0000000..774946b --- /dev/null +++ b/docs/evidence/RESOURCE-WP-0003-portfolio-reporting-2026-08-14.md @@ -0,0 +1,74 @@ +# RESOURCE-WP-0003 portfolio reporting and cadence — 2026-08-14 + +## Result + +`RESOURCE-WP-0003-T07` established the portfolio view and the operating cadence. + +The implementation consists of: + +- `tools/portfolio_report.py` — coverage, lifecycle, utilization, cost, + renewals, risks, open optimization cases, and next actions, derived entirely + from committed evidence; +- `docs/portfolio-operating-cadence.md` — monthly observation, monthly report, + quarterly calibration, pre-renewal review, and event-driven triggers, each + with named inputs, consumers, and outputs; +- `make portfolio-report`; +- 22 tests in `tests/test_portfolio_report.py`, including a suite that renders + the real committed portfolio rather than fixtures. + +Nothing in the report is typed by hand. It reads `data/resources/`, the latest +`data/portfolio-coverage-*.json`, and `data/optimization/`, and it re-validates +every record it reads. + +## Not forcing false precision + +T07 requires reporting missing evidence and unapportioned cost rather than +forcing false precision. Two decisions carry that requirement: + +- **`known_monthly_spend_eur` is `null`, never `0`.** Six of seven resources + carry no price evidence and no booked cost has arrived from `fin-hub`. + Summing the one priced resource would report a portfolio spend roughly an + order of magnitude below reality while looking authoritative. The field stays + unknown, accompanied by a note stating how many resources are unpriced. +- **Unattributed cost is a list of resources, not a spread.** Four resources + have allocation mode `unattributed`. They are named, with the repository that + owes the allocation driver; their cost is not divided across consumers by a + plausible default. + +Resources whose capacity cannot be measured appear in `utilization.unmeasured` +with the reason, rather than being dropped from the ratios and thereby +flattering the portfolio average. + +## What the current report tells an operator + +T07 is done when an operator can identify material spend, idle or saturated +capacity, forecast error, approaching commitments, and the next evidence-backed +action. Against the portfolio as committed on 2026-08-14: + +| Question | Answer | +|---|---| +| Material spend | **Unknown, explicitly.** Only the proposed backup storage carries price evidence; the rest waits on `RAIL-HO-WP-0008` and `FIN-WP-0004`. | +| Idle capacity | `resource:hosteurope:railiance01` and `resource:railiance:reef-railiance:k3s`, idle on every measured metric at 14 percent CPU and 37 percent memory. Five resources are unmeasurable. | +| Saturated capacity | None. | +| Forecast error | **Not yet computable.** The control-cycle mechanism is proven on paired examples, but no operational actual observation exists, so no variance is reported and none is fabricated. | +| Approaching commitments | None visible within 90 days — and the finding is that six active resources have **no** renewal or cancellation date at all, so no cancellation window can be respected. | +| Next action | The `next_actions` list: seven delegated workplans, the booked-cost dependency on `fin-hub`, and six missing contract-date records, each addressed to its owner. | + +## Structural risk surfaced + +The risk section reports concentration mechanically rather than in prose: six of +seven resources share `host:railiance01` and five share `cluster:reef-railiance`. +That includes every workload the proposed backup storage exists to protect, +which is the standing argument against same-provider placement recorded in +`opt:platform-audit-storage:2026-08`. Thresholds are explicit in the tool — +idle at or below 35 percent, saturated at or above 85 percent, renewal horizon +90 days, failure-domain concentration above two resources. + +## Boundaries held + +The report reads; it does not write to any other repository. Every unknown it +prints is attributed to the repository or authority that owns it, and the +`next_actions` list is the message sent back to those owners on the monthly +cadence. No booked financial fact is reproduced here; spend remains +`fin-hub`'s authority under +`docs/fin-hub-resource-control-contract-v0.1.md`. diff --git a/docs/fin-hub-resource-control-contract-v0.1.md b/docs/fin-hub-resource-control-contract-v0.1.md new file mode 100644 index 0000000..a7fa2a5 --- /dev/null +++ b/docs/fin-hub-resource-control-contract-v0.1.md @@ -0,0 +1,47 @@ +# fin-hub ↔ resource-control contract v0.1 — resource-control ratification + +Status: authority and schema boundary accepted on 2026-08-11 +Fin-hub implementation reviewed: commit `0034330` +Workplans: `RESOURCE-WP-0003-T02`, `FIN-WP-0004` + +The jointly reviewed contract is maintained in fin-hub at +`docs/fin-resource-authority-contract-v0.1.md`. This record is the +resource-control-side ratification and identifies the artifacts for which this +repository is authoritative. + +## Authority + +- fin-hub owns booked financial facts and corrections, tax and currency + treatment, budgets, commitments, burn, runway, and viability signals. +- resource-control owns resource identity and lifecycle, technical usage, + allocation evidence, demand and cost forecasts, and optimization evidence. +- workload and platform repositories remain authoritative for native workload + identity, service requirements, provisioning, operation, and source + telemetry. +- neither repository approves contracts, executes payment, or turns planning + evidence into booked spend. + +## Artifacts + +- `schemas/planning-evidence.schema.json` is the authoritative + resource-control outbound schema. +- `tools/financial_exchange.py forecast` converts the existing immutable + monthly backup forecast into canonical `ForecastEvidence` records. +- `tools/financial_exchange.py reconcile` validates fin-hub booked-cost + projections and compares attributable infrastructure cost by accounting + period. Booked facts are referenced projections here, never a second ledger. +- `schemas/monthly-resource-observation.schema.json` uses + `usage_observation` for technical observations. “Booked actual” is reserved + for fin-hub facts. + +Fin-hub owns its outbound `BookedCostEvidence` executable schema. This +repository validates the contract fields and arithmetic it consumes, but does +not redefine financial authority. + +## Acceptance boundary + +The 12-row `platform:audit-storage` forecast can complete an idempotent +planning-evidence preflight before procurement. `FIN-WP-0004-T05` and the +operational round trip remain open until an attributable provider charge or +credit exists. Fixture or synthetic money may test the seam but cannot satisfy +that evidence gate. diff --git a/docs/forecast-actual-control.md b/docs/forecast-actual-control.md new file mode 100644 index 0000000..3879964 --- /dev/null +++ b/docs/forecast-actual-control.md @@ -0,0 +1,123 @@ +# Forecast-to-actual resource control + +## Why the €2.89, €5.14, €62.89, and €65.14 figures differ + +They describe two volumes and two cost scopes: + +| Figure | Stored volume | Scope | Calculation | +| --- | ---: | --- | --- | +| €2.89 | 180 GB | infrastructure only | 180 × €0.01606 | +| €62.89 | 180 GB | infrastructure + internal labor | €2.89 + 1 h × €60 | +| €5.14 | 320 GB | infrastructure only | 320 × €0.01606 | +| €65.14 | 320 GB | infrastructure + internal labor | €5.14 + 1 h × €60 | + +The older €62.89 estimate was not “storage without elastic cost”; it already +included the €60 monthly operator allowance. The normalized table introduced a +different stored volume and showed infrastructure separately. Future reports +must always label both `stored_gb` and cost scope. + +## Control records + +A forecast is a timestamped, immutable record of what was believed at decision +time. The initial base forecast is +`data/forecasts/platform-audit-storage-scaleway-base-2026-08.json`. Do not edit +it after actual evidence exists. If assumptions change, create a new forecast +with a new creation date and retain the prior one so forecast accuracy remains +auditable. + +After every billing month, record a technical `usage_observation` under +`data/actuals/`. Authoritative booked financial facts come from fin-hub and +are joined by reference; resource-control does not originate a second booked +actual. +The required cost and usage proxies are: + +- database size and provider stored bytes; +- WAL volume; +- write and read requests; +- restore-test egress; +- infrastructure invoice cost; +- internal operations hours and valued labor cost; +- total attributable cost; +- backup success, maximum observed RPO, and restore RTO when measured. + +Provider stored bytes are intentionally distinct from logical database size. +They expose base-backup retention, WAL, compression, versioning, incomplete +multipart uploads, and lifecycle behavior. Invoice cost is distinct from the +provider's usage estimator and must reconcile to non-secret billing evidence. + +## Error calculation and review + +`make variance ACTUAL=data/actuals/YYYY-MM.json` reports signed error and +absolute percentage error for each numeric proxy. Initially review rather than +automatically rewrite assumptions: + +- investigate infrastructure cost error above 10%; +- investigate stored-byte or WAL error above 20%; +- investigate any unplanned egress or request-class charge; +- investigate labor error above 1 hour or 25%; +- investigate every backup failure, RPO breach, or restore-RTO regression. + +After three comparable months, calculate mean absolute percentage error by +proxy and recalibrate the next forecast. Avoid MAPE where the forecast is zero; +use absolute error and explain the new activity instead. Separate forecast +error from price variance: a bill can differ because usage was wrong, the rate +changed, tax/discount treatment differed, or an unmodeled SKU appeared. + +Quarterly provider comparison should use the latest actual trailing three +months, the next 12-month forecast, one monthly restore, an exit event, and +observed—not aspirational—operator labor. + +## General portfolio control record + +The storage-specific v0.1 observation remains valid for the backup procurement +case. New portfolio controls use +`schemas/resource-control-cycle.schema.json`, whose resource-specific +`usage_proxies` allow the same comparison mechanism to cover storage, VMs or +cluster compute, shared platform services, managed services, and future +resource types without pretending they share the same utilization unit. + +Each record separates: + +- provisioned and used capacity, with a fixed, elastic, or hybrid model; +- infrastructure, internal labor, and external labor cost; +- booked financial facts, referenced rather than copied from fin-hub; +- allocation method, driver, version, and unattributed residual; +- service constraints and their units; +- low/base/high forecast scenario or observed actual; +- evidence and uncertainty. + +Forecast records are append-only. A changed forecast receives a new +`record_id`, creation time, and `revision_of` reference. Actuals must name the +exact original `forecast_ref`; a later revision must never replace the +decision-time baseline when measuring forecast error. + +Run a generic comparison with: + +```sh +make control-cycle FORECAST=examples/control-cycle/storage-forecast.json \ + ACTUAL=examples/control-cycle/storage-actual.json +``` + +The examples cover storage, `reef-railiance` cluster compute, and the shared +`apps-pg` service. Their numbers are illustrative contract fixtures, not booked +facts. Operational records replace their evidence references after the owner +workplans publish observations. + +## Variance attribution + +Every material variance is assigned to one of six controlled categories: + +- `demand`: the amount of consumed service differed; +- `provider_price`: rate, discount, tax, currency, or billed SKU differed; +- `allocation`: a shared-cost driver or attribution changed; +- `labor`: internal or external effort differed; +- `model`: a formula, assumption, capacity behavior, or omitted component was + wrong; +- `data_quality`: evidence is missing, late, inconsistent, or uses a different + unit. + +The comparator defaults usage differences to demand, infrastructure cost to +provider price, and labor fields to labor, while preserving explicit +attribution supplied with the actual. Missing proxies and unit mismatches fail +closed as data-quality errors. Attribution explains an error; it does not +rewrite the forecast. diff --git a/docs/optimization-cases.md b/docs/optimization-cases.md new file mode 100644 index 0000000..c8e787f --- /dev/null +++ b/docs/optimization-cases.md @@ -0,0 +1,105 @@ +# Optimization cases — review cadence and decision template + +`resource-control` produces optimization cases. It does not approve them and it +does not implement them. A case turns portfolio evidence into a reproducible +comparison; a human or repository authority decides; the owning platform or +workload repository implements; `fin-hub` records the money. + +Schema: `schemas/optimization-case.schema.json`. +Evaluator: `tools/optimization.py`. Cases live in `data/optimization/`. + +## Case types + +| Type | Question it answers | +|---|---| +| `rightsizing` | Is provisioned capacity materially larger or smaller than sustained demand? | +| `consolidation` | Can separate resources share one substrate without breaking failure domains? | +| `commitment` | Does a term or reserved commitment beat on-demand pricing at forecast demand? | +| `renewal` | Should this contract renew, renegotiate, or lapse? | +| `migration` | Should the workload move to different capacity at the same provider? | +| `retirement` | Is this resource still serving anything? | +| `provider_switch` | Should the same service come from a different provider? | + +## Decision template + +Every option — the baseline included — must present the same fields. The +baseline is a full option, not a footnote, because a comparison against an +undescribed status quo is not a comparison. + +1. **Baseline** — what is in place now, on the same terms as every alternative. +2. **Alternatives** — at least one, each independently costed. +3. **One-time cost** — migration, setup, dual-running, and the labour to do it. +4. **Recurring cost** — split into infrastructure, internal labour, and external + labour. A cheaper unit price that doubles operator hours is not a saving. +5. **Utilization** — provisioned and used per capacity metric, with the + observation that supports it. +6. **Uncertainty** — `low`, `medium`, or `high`, with the reasons written out. +7. **Service-level constraints** — what the option must still satisfy. +8. **Failure domains** — the evaluator reports which the alternative removes and + which it newly introduces. +9. **Exit path** — how to leave the option. An option with no known exit path is + blocked, not merely riskier. +10. **Expected payback** — computed, never asserted. + +### Unknowns are first-class + +Any unknown cost, utilization, or exit path is `null`, and each blocking gap is +named in the option's `unknowns` list with the repository or authority that owns +it. The evaluator is fail-closed: one unknown makes the affected comparison +`blocked_on_evidence` rather than optimistic. A case that cannot conclude is a +valid, useful output — it names exactly what evidence would unblock it. + +### Verdicts + +| Verdict | Meaning | +|---|---| +| `blocked_on_evidence` | A required decision field is unknown. No recommendation. | +| `recommend` | Fully costed and saves more than EUR 5 per month. | +| `no_material_change` | Fully costed, difference within EUR 5 per month. | +| `reject` | Fully costed and more expensive than the baseline. | + +`reject` is a cost verdict only. An option that costs more may still be right — +the three-node cluster case buys availability, not savings — and that argument +belongs in `decision.rationale`, made by the deciding authority against a +declared service objective, not by the calculator. + +### Decision states + +`blocked_on_evidence` → `proposed` → `approved` | `rejected` → `superseded`. + +The evaluator enforces the transitions it can check: a case cannot be `proposed` +while any comparison is blocked, cannot be `approved` while blocked, and cannot +be decided without a named `approver` and `approved_on`. + +## Cadence + +| Rhythm | Trigger | Scope | +|---|---|---| +| Monthly | `cadence` | Review variance from `tools/control_cycle.py`. Open a case where forecast error is attributed to demand, price, or allocation rather than data quality. | +| Quarterly | `cadence` | Re-run every open case against refreshed evidence. Recompute all provider comparisons; a stale price is not evidence. | +| Pre-renewal | `renewal` | Open at least 60 days before `lifecycle.renews_on` or `cancel_by`, whichever is earlier, so cancellation stays possible. | +| On variance | `variance` | Sustained variance beyond the thresholds in `docs/forecast-actual-control.md`. | +| On demand | `request`, `procurement`, `incident` | A workload repository, a procurement decision, or an incident that exposed a failure domain. | + +## Closing the loop + +An approved case is not finished when it is approved. + +1. `financial_handoff` carries the cost attribution key and the reference to + `fin-hub` under the contract in `docs/fin-hub-resource-control-contract-v0.1.md`. +2. `decision.delegated_to` names the repositories that implement. Implementation + detail is theirs; this repository states the interface and the acceptance + evidence only. +3. `outcome.feeds_forecast` names the control-cycle forecast records the decision + changes. The original forecast is never overwritten — a decision produces a + revision, per `docs/forecast-actual-control.md`. +4. `outcome.actual_refs` collects the monthly actuals that later show whether the + predicted saving materialised. A case whose actuals never arrive is an + unverified case, and the next quarterly review should say so. + +## Current cases + +| Case | Type | State | Why | +|---|---|---|---| +| `opt:platform-audit-storage:2026-08` | `provider_switch` | `blocked_on_evidence` | Hetzner computes and is EUR 29.14 per month dearer than Scaleway at month-12 base demand, on labour rather than storage price. Host Europe has no usable price, no cancellation terms, and shares a failure domain with the host it would protect. The procurement decision itself belongs to `RESOURCE-WP-0002-T03`. | +| `opt:reef-railiance-k3s:2026-08` | `rightsizing` | `blocked_on_evidence` | Observed utilization is 14 percent CPU and 37 percent memory — a real signal from one sample. Nothing can be costed while the railiance01 booked price is unknown, so no payback exists. Blocked against `RAIL-HO-WP-0008` and `RAIL-BS-WP-0014`. | diff --git a/docs/portfolio-operating-cadence.md b/docs/portfolio-operating-cadence.md new file mode 100644 index 0000000..df2b354 --- /dev/null +++ b/docs/portfolio-operating-cadence.md @@ -0,0 +1,74 @@ +# Portfolio reporting and operating cadence + +`make portfolio-report` renders the current portfolio view from +`tools/portfolio_report.py`. The report is derived entirely from committed +evidence: resource records in `data/resources/`, the latest +`data/portfolio-coverage-*.json`, and the optimization cases in +`data/optimization/`. Nothing in it is typed by hand. + +## What the report answers + +| Section | Question | +|---|---| +| `coverage` | Which service groups are represented, and which evidence gaps are still delegated and to whom. | +| `lifecycle` | How many resources sit in each lifecycle state, and whether any lacks an owner. | +| `utilization` | Provisioned versus observed capacity per metric, flagged `idle`, `normal`, or `saturated`; and which resources cannot be measured at all. | +| `cost` | Which resources carry price evidence, which do not, and which have no allocation method so their cost reaches no consumer. | +| `renewals` | Contract dates inside the 90-day horizon, and resources with no recorded dates at all. | +| `risks` | Concentrated failure domains, unpriced resources, unattributed cost, idle and saturated capacity. | +| `optimization` | Open cases, their per-option verdicts, and the named evidence blocking each one. | +| `next_actions` | The smallest set of evidence that would unblock the most decisions, addressed to the repository that owns it. | + +## Not forcing false precision + +Two rules the report holds to, because the alternative is a number that looks +authoritative and is not: + +- **`known_monthly_spend_eur` is `null`, not `0`.** No booked cost has reached + this repository, and six of seven resources carry no price evidence. Summing + what happens to be known would report a portfolio spend an order of magnitude + below reality. The figure stays unknown until `fin-hub` delivers booked facts + under `docs/fin-hub-resource-control-contract-v0.1.md`. +- **Unattributed cost is a list, not a spread.** Four resources have allocation + mode `unattributed`. Their cost is not divided across consumers by a plausible + default; they are named, and the allocation driver is owed by the repository + that owns the resource. + +Resources that cannot be measured appear in `utilization.unmeasured` with the +reason, rather than being silently omitted from the ratios. + +## Cadence + +| Rhythm | When | Inputs | Consumers | Output | +|---|---|---|---|---| +| **Monthly observation** | First week, for the closed month | Monthly actual records from delegated telemetry; booked costs from `fin-hub` | resource-control | `tools/control_cycle.py` variance per resource; a new optimization case where variance is attributed to demand, price, or allocation rather than data quality | +| **Monthly portfolio report** | With the observation | Resource records, coverage, optimization cases | Human operator; owning repositories | `make portfolio-report`; the `next_actions` list is re-sent to the repositories named in it | +| **Quarterly calibration** | End of quarter | Three months of variance | resource-control; `fin-hub` | Forecast revisions per `docs/forecast-actual-control.md` — revisions, never overwrites; refreshed provider prices in every open case | +| **Pre-renewal review** | At least 60 days before the earliest `renews_on` or `cancel_by` | Contract evidence, utilization, open cases | Human financial authority | A `renewal` case, decided while cancellation is still possible | +| **Event-driven** | Sustained variance beyond threshold, an incident exposing a failure domain, or a workload request | The triggering evidence | Owning repository | A case of the matching type per `docs/optimization-cases.md` | + +## Reading the current report + +As of 2026-08-14 an operator asking the four questions this cadence exists to +answer gets these answers, and the honest ones are the useful ones: + +- **Material spend?** Unknown, and explicitly so. Only the proposed backup + storage carries price evidence. Every other resource is waiting on + `RAIL-HO-WP-0008` and `FIN-WP-0004`. +- **Idle or saturated capacity?** `railiance01` and the `reef-railiance` k3s + cluster are idle on every measured metric — 14 percent CPU, 37 percent memory. + Nothing is saturated. Five resources cannot be measured at all. +- **Forecast error?** Not yet computable. The control-cycle mechanism is proven + on paired examples, but no operational actual observation exists, so there is + no variance to report and none is fabricated. +- **Approaching commitments?** None visible — and that is itself the finding: + six active resources have no renewal or cancellation date recorded, so the + cancellation window cannot be respected for any of them. +- **Next evidence-backed action?** The `next_actions` list, led by the seven + delegated workplans that own the missing evidence. + +The single largest structural risk the report surfaces is concentration: six of +seven resources share `host:railiance01`, including the backup storage's own +intended protection target. That is the standing argument for placing backup +storage outside the Host Europe failure domain, and it is visible in the report +rather than only in prose. diff --git a/examples/control-cycle/apps-pg-actual.json b/examples/control-cycle/apps-pg-actual.json new file mode 100644 index 0000000..fd14ce2 --- /dev/null +++ b/examples/control-cycle/apps-pg-actual.json @@ -0,0 +1 @@ +{"schema_version":"0.1","record_id":"actual:apps-pg:2026-09","record_type":"actual","resource_id":"resource:railiance:apps-pg","resource_class":"shared_platform_service","period":"2026-09","created_at":"2026-10-02T00:00:00Z","scenario":"observed","forecast_ref":"forecast:apps-pg:2026-09:base:v1","revision_of":null,"usage_proxies":{"database_gb":{"value":6,"unit":"GB"},"connections":{"value":18,"unit":"peak"}},"capacity":{"model":"hybrid","provisioned":{"storage":{"value":20,"unit":"GiB"}},"used":{"storage":{"value":6,"unit":"GB"}}},"costs":{"currency":"EUR","infrastructure":15,"internal_labor":90,"external_labor":0,"total":105,"booked_cost_refs":["financial-fact:example-shared-allocation-2026-09"]},"allocation":{"method":"proportional","driver":"database_gb","method_version":"1","unattributed_eur":5},"service_constraints":{"rpo_minutes":{"value":5,"unit":"minutes"},"rto_minutes":{"value":60,"unit":"minutes"}},"variance_attribution":{"database_gb":"demand","connections":"model","costs.internal_labor":"labor"},"evidence":["illustrative actual; replace with RAILIANCE-WP-0016 evidence"]} diff --git a/examples/control-cycle/apps-pg-forecast.json b/examples/control-cycle/apps-pg-forecast.json new file mode 100644 index 0000000..60ff678 --- /dev/null +++ b/examples/control-cycle/apps-pg-forecast.json @@ -0,0 +1 @@ +{"schema_version":"0.1","record_id":"forecast:apps-pg:2026-09:base:v1","record_type":"forecast","resource_id":"resource:railiance:apps-pg","resource_class":"shared_platform_service","period":"2026-09","created_at":"2026-08-11T00:00:00Z","scenario":"base","forecast_ref":null,"revision_of":null,"usage_proxies":{"database_gb":{"value":5,"unit":"GB"},"connections":{"value":20,"unit":"peak"}},"capacity":{"model":"hybrid","provisioned":{"storage":{"value":20,"unit":"GiB"}},"used":{"storage":{"value":5,"unit":"GB"}}},"costs":{"currency":"EUR","infrastructure":15,"internal_labor":60,"external_labor":0,"total":75,"booked_cost_refs":[]},"allocation":{"method":"proportional","driver":"database_gb","method_version":"1","unattributed_eur":5},"service_constraints":{"rpo_minutes":{"value":5,"unit":"minutes"},"rto_minutes":{"value":60,"unit":"minutes"}},"uncertainty":{"level":"high","notes":["Illustrative until platform evidence arrives"]},"evidence":["RAILIANCE-WP-0016"]} diff --git a/examples/control-cycle/cluster-actual.json b/examples/control-cycle/cluster-actual.json new file mode 100644 index 0000000..ecb52da --- /dev/null +++ b/examples/control-cycle/cluster-actual.json @@ -0,0 +1 @@ +{"schema_version":"0.1","record_id":"actual:reef-railiance:2026-09","record_type":"actual","resource_id":"resource:railiance:reef-railiance:k3s","resource_class":"cluster_compute","period":"2026-09","created_at":"2026-10-02T00:00:00Z","scenario":"observed","forecast_ref":"forecast:reef-railiance:2026-09:base:v1","revision_of":null,"usage_proxies":{"cpu_requested":{"value":3,"unit":"vCPU"},"memory_requested":{"value":7,"unit":"GiB"}},"capacity":{"model":"fixed","provisioned":{"cpu":{"value":4,"unit":"vCPU"},"memory":{"value":16,"unit":"GiB"}},"used":{"cpu":{"value":3,"unit":"vCPU"},"memory":{"value":7,"unit":"GiB"}}},"costs":{"currency":"EUR","infrastructure":40,"internal_labor":150,"external_labor":0,"total":190,"booked_cost_refs":["financial-fact:example-host-2026-09"]},"allocation":{"method":"proportional","driver":"cpu_and_memory_requests","method_version":"1","unattributed_eur":10},"service_constraints":{"availability_pct":{"value":99,"unit":"percent"}},"variance_attribution":{"cpu_requested":"demand","memory_requested":"demand","costs.internal_labor":"labor"},"evidence":["illustrative actual; replace with RAIL-BS-WP-0014 evidence"]} diff --git a/examples/control-cycle/cluster-forecast.json b/examples/control-cycle/cluster-forecast.json new file mode 100644 index 0000000..3003c77 --- /dev/null +++ b/examples/control-cycle/cluster-forecast.json @@ -0,0 +1 @@ +{"schema_version":"0.1","record_id":"forecast:reef-railiance:2026-09:base:v1","record_type":"forecast","resource_id":"resource:railiance:reef-railiance:k3s","resource_class":"cluster_compute","period":"2026-09","created_at":"2026-08-11T00:00:00Z","scenario":"base","forecast_ref":null,"revision_of":null,"usage_proxies":{"cpu_requested":{"value":2.5,"unit":"vCPU"},"memory_requested":{"value":6,"unit":"GiB"}},"capacity":{"model":"fixed","provisioned":{"cpu":{"value":4,"unit":"vCPU"},"memory":{"value":16,"unit":"GiB"}},"used":{"cpu":{"value":2.5,"unit":"vCPU"},"memory":{"value":6,"unit":"GiB"}}},"costs":{"currency":"EUR","infrastructure":40,"internal_labor":120,"external_labor":0,"total":160,"booked_cost_refs":[]},"allocation":{"method":"proportional","driver":"cpu_and_memory_requests","method_version":"1","unattributed_eur":10},"service_constraints":{"availability_pct":{"value":99,"unit":"percent"}},"uncertainty":{"level":"high","notes":["Illustrative until delegated cluster observations arrive"]},"evidence":["RAIL-BS-WP-0014"]} diff --git a/examples/control-cycle/storage-actual.json b/examples/control-cycle/storage-actual.json new file mode 100644 index 0000000..9cf1bc1 --- /dev/null +++ b/examples/control-cycle/storage-actual.json @@ -0,0 +1 @@ +{"schema_version":"0.1","record_id":"actual:platform-audit-storage:2026-09","record_type":"actual","resource_id":"resource:platform:audit-storage","resource_class":"storage","period":"2026-09","created_at":"2026-10-02T00:00:00Z","scenario":"observed","forecast_ref":"forecast:platform-audit-storage:2026-09:base:v1","revision_of":null,"usage_proxies":{"stored_gb":{"value":350,"unit":"GB-month"},"egress_gb":{"value":5,"unit":"GB"}},"capacity":{"model":"elastic","provisioned":{},"used":{"stored_gb":{"value":350,"unit":"GB-month"}}},"costs":{"currency":"EUR","infrastructure":5.62,"internal_labor":60,"external_labor":0,"total":65.62,"booked_cost_refs":["financial-fact:example-storage-2026-09"]},"allocation":{"method":"direct","driver":"stored_gb","method_version":"1","unattributed_eur":0},"service_constraints":{"rpo_minutes":{"value":5,"unit":"minutes"},"rto_minutes":{"value":55,"unit":"minutes"}},"variance_attribution":{"stored_gb":"demand","costs.infrastructure":"provider_price"},"evidence":["illustrative actual; replace with provider and platform evidence"]} diff --git a/examples/control-cycle/storage-forecast.json b/examples/control-cycle/storage-forecast.json new file mode 100644 index 0000000..643faff --- /dev/null +++ b/examples/control-cycle/storage-forecast.json @@ -0,0 +1 @@ +{"schema_version":"0.1","record_id":"forecast:platform-audit-storage:2026-09:base:v1","record_type":"forecast","resource_id":"resource:platform:audit-storage","resource_class":"storage","period":"2026-09","created_at":"2026-08-11T00:00:00Z","scenario":"base","forecast_ref":null,"revision_of":null,"usage_proxies":{"stored_gb":{"value":320,"unit":"GB-month"},"egress_gb":{"value":5,"unit":"GB"}},"capacity":{"model":"elastic","provisioned":{},"used":{"stored_gb":{"value":320,"unit":"GB-month"}}},"costs":{"currency":"EUR","infrastructure":5.14,"internal_labor":60,"external_labor":0,"total":65.14,"booked_cost_refs":[]},"allocation":{"method":"direct","driver":"stored_gb","method_version":"1","unattributed_eur":0},"service_constraints":{"rpo_minutes":{"value":5,"unit":"minutes"},"rto_minutes":{"value":60,"unit":"minutes"}},"uncertainty":{"level":"high","notes":["No booked provider fact exists yet"]},"evidence":["data/demand/platform-audit-storage.json"]} diff --git a/examples/portfolio/garage-three-node.example.json b/examples/portfolio/garage-three-node.example.json new file mode 100644 index 0000000..da2064c --- /dev/null +++ b/examples/portfolio/garage-three-node.example.json @@ -0,0 +1,15 @@ +{ + "schema_version": "0.2", "record_scope": "example", + "id": "resource:example:garage:cluster", "resource_class": "self_managed_service", + "status": "proposed", "management_model": "self_managed", + "provider": {"name": "Example IaaS provider", "account_ref": null, "product_ref": "three-vm-topology", "provider_resource_id": null}, + "service": {"name": "Garage object storage", "service_id": "object-storage", "class": "S3-compatible replicated service", "capacity_model": "fixed"}, + "location": {"region": "example-eu", "country": "DE", "failure_domains": ["provider:example", "region:example-eu"], "residency": "European Union"}, + "capacity": [{"metric": "usable_storage", "value": 320, "unit": "GB", "kind": "usable", "observed_at": null}, {"metric": "nodes", "value": 3, "unit": "count", "kind": "provisioned", "observed_at": null}], + "ownership": {"owner": "railiance-platform", "environment": "production", "workload_ids": [], "tenant_id": null, "allocation": {"mode": "shared", "cost_attribution_key": "platform:object-storage", "driver": "stored-gb-month", "method_version": "storage-share-v1"}}, + "cost": {"currency": "EUR", "tax_status": "unknown", "billing_model": "fixed VM capacity plus internal operations labor", "commitment_ref": null, "price_evidence": null}, + "lifecycle": {"proposed_on": null, "ordered_on": null, "commissioned_on": null, "renews_on": null, "cancel_by": null, "retired_on": null, "exit_path": "Export objects to a verified S3-compatible target, revoke credentials, and retire nodes after consumers move."}, + "relationships": [{"type": "hosted_on", "resource_id": "resource:example:garage:nodes"}], + "requirements": [{"kind": "recovery", "ref": "example:restore-test-required"}], + "evidence": [{"kind": "example", "ref": "RESOURCE-WP-0003-T01", "observed_at": "2026-08-11", "authority": "resource-control"}] +} diff --git a/examples/portfolio/kubernetes-capacity.example.json b/examples/portfolio/kubernetes-capacity.example.json new file mode 100644 index 0000000..8c66c2e --- /dev/null +++ b/examples/portfolio/kubernetes-capacity.example.json @@ -0,0 +1,15 @@ +{ + "schema_version": "0.2", "record_scope": "example", + "id": "resource:example:railiance:kubernetes-capacity", "resource_class": "kubernetes_capacity", + "status": "active", "management_model": "shared_capacity", + "provider": {"name": "Example infrastructure provider", "account_ref": "account:example", "product_ref": "virtual-machines", "provider_resource_id": null}, + "service": {"name": "Railiance Kubernetes worker capacity", "service_id": "kubernetes", "class": "shared worker pool", "capacity_model": "shared"}, + "location": {"region": "example-region", "country": "DE", "failure_domains": ["cluster:example", "region:example-region"], "residency": "Germany"}, + "capacity": [{"metric": "cpu", "value": 24, "unit": "vCPU", "kind": "usable", "observed_at": "2026-08-11"}, {"metric": "memory", "value": 64, "unit": "GiB", "kind": "usable", "observed_at": "2026-08-11"}], + "ownership": {"owner": "railiance-cluster", "environment": "production", "workload_ids": ["helix-forge", "coulomb-social"], "tenant_id": null, "allocation": {"mode": "shared", "cost_attribution_key": "platform:kubernetes", "driver": "requested-cpu-memory-hours", "method_version": "k8s-requests-v1"}}, + "cost": {"currency": "EUR", "tax_status": "unknown", "billing_model": "fixed node cost allocated by resource requests", "commitment_ref": null, "price_evidence": null}, + "lifecycle": {"proposed_on": null, "ordered_on": null, "commissioned_on": "2026-01-01", "renews_on": null, "cancel_by": null, "retired_on": null, "exit_path": "Drain workloads to replacement capacity, verify persistent data and ingress, then terminate nodes."}, + "relationships": [{"type": "provides_capacity_to", "resource_id": "resource:example:shared:ingress"}], + "requirements": [{"kind": "service_objective", "ref": "example:cluster-capacity-slo"}], + "evidence": [{"kind": "example", "ref": "RESOURCE-WP-0003-T01", "observed_at": "2026-08-11", "authority": "resource-control"}] +} diff --git a/examples/portfolio/shared-platform-service.example.json b/examples/portfolio/shared-platform-service.example.json new file mode 100644 index 0000000..6e55acf --- /dev/null +++ b/examples/portfolio/shared-platform-service.example.json @@ -0,0 +1,15 @@ +{ + "schema_version": "0.2", "record_scope": "example", + "id": "resource:example:shared:ingress", "resource_class": "shared_platform_service", + "status": "active", "management_model": "self_managed", + "provider": {"name": "Railiance", "account_ref": null, "product_ref": "platform-ingress", "provider_resource_id": null}, + "service": {"name": "Shared application ingress", "service_id": "platform-ingress", "class": "cluster platform service", "capacity_model": "shared"}, + "location": {"region": null, "country": "DE", "failure_domains": ["cluster:example"], "residency": "Germany"}, + "capacity": [{"metric": "requests", "value": null, "unit": "requests/month", "kind": "unknown", "observed_at": null}], + "ownership": {"owner": "railiance-platform", "environment": "production", "workload_ids": ["helix-forge", "coulomb-social"], "tenant_id": null, "allocation": {"mode": "shared", "cost_attribution_key": "platform:shared-ingress", "driver": "request-count", "method_version": "ingress-requests-v1"}}, + "cost": {"currency": "EUR", "tax_status": "not_applicable", "billing_model": "allocated share of hosting capacity and operator labor", "commitment_ref": null, "price_evidence": null}, + "lifecycle": {"proposed_on": null, "ordered_on": null, "commissioned_on": "2026-01-01", "renews_on": null, "cancel_by": null, "retired_on": null, "exit_path": "Move routes and certificates to replacement ingress, verify traffic, then remove the service."}, + "relationships": [{"type": "hosted_on", "resource_id": "resource:example:railiance:kubernetes-capacity"}], + "requirements": [{"kind": "service_objective", "ref": "example:ingress-availability-slo"}], + "evidence": [{"kind": "example", "ref": "RESOURCE-WP-0003-T01", "observed_at": "2026-08-11", "authority": "resource-control"}] +} diff --git a/schemas/monthly-resource-observation.schema.json b/schemas/monthly-resource-observation.schema.json new file mode 100644 index 0000000..511f91d --- /dev/null +++ b/schemas/monthly-resource-observation.schema.json @@ -0,0 +1,43 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://coulomb.social/resource-control/monthly-resource-observation.schema.json", + "title": "Monthly resource forecast or technical usage observation", + "type": "object", + "additionalProperties": false, + "required": ["schema_version", "record_type", "workload", "cost_attribution_key", "provider_id", "created_at", "rows"], + "properties": { + "schema_version": {"const": "0.1"}, + "record_type": {"enum": ["forecast", "usage_observation"]}, + "workload": {"type": "string"}, + "cost_attribution_key": {"type": "string"}, + "provider_id": {"type": "string"}, + "created_at": {"type": "string", "format": "date-time"}, + "scenario": {"type": ["string", "null"]}, + "forecast_ref": {"type": ["string", "null"]}, + "rows": { + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "required": ["period", "database_gb", "stored_gb", "wal_gb", "restore_egress_gb", "write_requests", "read_requests", "infrastructure_eur", "internal_labor_hours", "internal_labor_eur", "total_eur"], + "properties": { + "period": {"type": "string", "pattern": "^[0-9]{4}-(0[1-9]|1[0-2])$"}, + "database_gb": {"type": "number", "minimum": 0}, + "stored_gb": {"type": "number", "minimum": 0}, + "wal_gb": {"type": "number", "minimum": 0}, + "restore_egress_gb": {"type": "number", "minimum": 0}, + "write_requests": {"type": "integer", "minimum": 0}, + "read_requests": {"type": "integer", "minimum": 0}, + "infrastructure_eur": {"type": "number", "minimum": 0}, + "internal_labor_hours": {"type": "number", "minimum": 0}, + "internal_labor_eur": {"type": "number", "minimum": 0}, + "total_eur": {"type": "number", "minimum": 0}, + "backup_success_pct": {"type": ["number", "null"], "minimum": 0, "maximum": 100}, + "restore_rto_minutes": {"type": ["number", "null"], "minimum": 0}, + "max_rpo_minutes": {"type": ["number", "null"], "minimum": 0}, + "evidence": {"type": "array", "items": {"type": "string"}} + } + } + } + } +} diff --git a/schemas/optimization-case.schema.json b/schemas/optimization-case.schema.json new file mode 100644 index 0000000..3f566b8 --- /dev/null +++ b/schemas/optimization-case.schema.json @@ -0,0 +1,206 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://coulomb.social/resource-control/optimization-case.schema.json", + "title": "Resource optimization case", + "description": "An evidence-backed comparison between a baseline resource arrangement and one or more alternatives. Every option must present the same decision fields; unknown values are null and are reported as blocking evidence gaps rather than assumed.", + "type": "object", + "additionalProperties": false, + "required": [ + "schema_version", + "record_scope", + "case_id", + "case_type", + "trigger", + "review_period", + "created_at", + "resource_ids", + "baseline", + "alternatives", + "decision", + "evidence" + ], + "properties": { + "schema_version": { "const": "0.1" }, + "record_scope": { + "description": "operational cases assert real facts; illustrative cases exist only to exercise the mechanism.", + "enum": ["operational", "illustrative"] + }, + "case_id": { "type": "string", "pattern": "^opt:[a-z0-9][a-z0-9:._-]+$" }, + "case_type": { + "enum": [ + "rightsizing", + "consolidation", + "commitment", + "renewal", + "migration", + "retirement", + "provider_switch" + ] + }, + "trigger": { + "enum": ["cadence", "renewal", "variance", "incident", "request", "procurement"] + }, + "review_period": { "type": "string", "pattern": "^[0-9]{4}-(0[1-9]|1[0-2])$" }, + "created_at": { "type": "string", "format": "date-time" }, + "resource_ids": { + "type": "array", + "minItems": 1, + "uniqueItems": true, + "items": { "type": "string", "pattern": "^resource:" } + }, + "baseline": { "$ref": "#/$defs/option" }, + "alternatives": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/$defs/option" } + }, + "decision": { + "type": "object", + "additionalProperties": false, + "required": ["state", "recommended_option_id", "rationale", "approver", "approved_on", "delegated_to"], + "properties": { + "state": { + "description": "blocked_on_evidence until every decision field of the compared options is known; proposed once the case computes; then approved or rejected by the named authority.", + "enum": ["blocked_on_evidence", "proposed", "approved", "rejected", "superseded"] + }, + "recommended_option_id": { "type": ["string", "null"] }, + "rationale": { "type": "string", "minLength": 1 }, + "approver": { + "description": "Named human or repository authority. Null while the case is not yet decided.", + "type": ["string", "null"] + }, + "approved_on": { "type": ["string", "null"], "format": "date" }, + "delegated_to": { + "description": "Repositories that own any approved implementation. resource-control never implements.", + "type": "array", + "items": { "type": "string" }, + "uniqueItems": true + } + } + }, + "financial_handoff": { + "description": "What is sent to fin-hub when the case is approved.", + "type": "object", + "additionalProperties": false, + "required": ["cost_attribution_key", "sent"], + "properties": { + "cost_attribution_key": { "type": ["string", "null"] }, + "sent": { "type": "boolean" }, + "reference": { "type": ["string", "null"] } + } + }, + "outcome": { + "description": "Closes the loop: which control-cycle records will show whether the case was right.", + "type": "object", + "additionalProperties": false, + "required": ["feeds_forecast", "actual_refs"], + "properties": { + "feeds_forecast": { + "type": "array", + "items": { "type": "string" }, + "uniqueItems": true + }, + "actual_refs": { + "type": "array", + "items": { "type": "string" }, + "uniqueItems": true + } + } + }, + "evidence": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/$defs/evidence_ref" } + } + }, + "$defs": { + "measurement": { + "type": "object", + "additionalProperties": false, + "required": ["value", "unit"], + "properties": { + "value": { "type": ["number", "null"] }, + "unit": { "type": "string", "minLength": 1 } + } + }, + "utilization": { + "type": "object", + "additionalProperties": false, + "required": ["provisioned", "used"], + "properties": { + "provisioned": { "$ref": "#/$defs/measurement" }, + "used": { "$ref": "#/$defs/measurement" } + } + }, + "evidence_ref": { + "type": "object", + "additionalProperties": false, + "required": ["kind", "ref", "authority"], + "properties": { + "kind": { + "enum": ["telemetry", "quote", "contract", "invoice", "forecast", "workplan", "document", "decision"] + }, + "ref": { "type": "string", "minLength": 1 }, + "authority": { "type": "string", "minLength": 1 }, + "observed_at": { "type": ["string", "null"], "format": "date" } + } + }, + "option": { + "description": "Baseline and alternatives carry identical decision fields so they compare on equal terms. A null cost or utilization value means unknown, never zero.", + "type": "object", + "additionalProperties": false, + "required": [ + "option_id", + "label", + "one_time_eur", + "recurring_infrastructure_eur_month", + "recurring_internal_labor_eur_month", + "recurring_external_labor_eur_month", + "utilization", + "uncertainty", + "service_constraints", + "failure_domains", + "exit_path", + "unknowns" + ], + "properties": { + "option_id": { "type": "string", "minLength": 1 }, + "label": { "type": "string", "minLength": 1 }, + "one_time_eur": { "type": ["number", "null"], "minimum": 0 }, + "recurring_infrastructure_eur_month": { "type": ["number", "null"], "minimum": 0 }, + "recurring_internal_labor_eur_month": { "type": ["number", "null"], "minimum": 0 }, + "recurring_external_labor_eur_month": { "type": ["number", "null"], "minimum": 0 }, + "utilization": { + "type": "object", + "additionalProperties": { "$ref": "#/$defs/utilization" } + }, + "uncertainty": { + "type": "object", + "additionalProperties": false, + "required": ["level", "notes"], + "properties": { + "level": { "enum": ["low", "medium", "high"] }, + "notes": { "type": "array", "items": { "type": "string" } } + } + }, + "service_constraints": { + "type": "object", + "additionalProperties": { "$ref": "#/$defs/measurement" } + }, + "failure_domains": { + "type": "array", + "minItems": 1, + "items": { "type": "string" }, + "uniqueItems": true + }, + "exit_path": { "type": ["string", "null"] }, + "unknowns": { + "description": "Named blocking evidence gaps, each attributable to an owning repository or authority.", + "type": "array", + "items": { "type": "string" }, + "uniqueItems": true + } + } + } + } +} diff --git a/schemas/planning-evidence.schema.json b/schemas/planning-evidence.schema.json new file mode 100644 index 0000000..469e14c --- /dev/null +++ b/schemas/planning-evidence.schema.json @@ -0,0 +1,101 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://coulomb.social/resource-control/planning-evidence.schema.json", + "title": "Resource-control planning evidence v0.1", + "oneOf": [ + {"$ref": "#/$defs/forecast"}, + {"$ref": "#/$defs/usage_observation"}, + {"$ref": "#/$defs/allocation"}, + {"$ref": "#/$defs/optimization"}, + {"$ref": "#/$defs/commitment_candidate"} + ], + "$defs": { + "money": {"type": "string", "pattern": "^(0|[1-9][0-9]*)\\.[0-9]{2}$"}, + "common": { + "type": "object", + "required": ["schema_version", "record_type", "record_id", "resource_id", "period_start", "period_end", "source_evidence", "created_at"], + "properties": { + "schema_version": {"const": "0.1"}, + "record_type": {"enum": ["forecast", "usage_observation", "allocation", "optimization", "commitment_candidate"]}, + "record_id": {"type": "string", "minLength": 1}, + "revision_of": {"type": ["string", "null"]}, + "resource_id": {"type": "string", "pattern": "^resource:"}, + "service_id": {"type": ["string", "null"]}, + "workload_id": {"type": ["string", "null"]}, + "tenant_id": {"type": ["string", "null"]}, + "environment": {"type": ["string", "null"]}, + "cost_attribution_key": {"type": ["string", "null"]}, + "period_start": {"type": "string", "format": "date"}, + "period_end": {"type": "string", "format": "date"}, + "source_evidence": {"type": "array", "items": {"type": "string"}}, + "created_at": {"type": "string", "format": "date-time"} + } + }, + "cost_breakdown": { + "type": "object", + "additionalProperties": false, + "required": ["infrastructure", "internal_labor", "external_services", "setup", "other"], + "properties": { + "infrastructure": {"$ref": "#/$defs/money"}, + "internal_labor": {"$ref": "#/$defs/money"}, + "external_services": {"$ref": "#/$defs/money"}, + "setup": {"$ref": "#/$defs/money"}, + "other": {"$ref": "#/$defs/money"} + } + }, + "forecast": { + "allOf": [ + {"$ref": "#/$defs/common"}, + { + "type": "object", + "required": ["currency", "scenario", "forecast_version", "costs", "assumptions"], + "properties": { + "record_type": {"const": "forecast"}, + "currency": {"type": "string", "pattern": "^[A-Z]{3}$"}, + "scenario": {"enum": ["low", "base", "high"]}, + "forecast_version": {"type": "string", "minLength": 1}, + "costs": {"$ref": "#/$defs/cost_breakdown"}, + "uncertainty": {"type": ["string", "null"]}, + "assumptions": {"type": "array", "items": {"type": "string"}} + } + } + ], + "unevaluatedProperties": false + }, + "usage_observation": { + "allOf": [ + {"$ref": "#/$defs/common"}, + { + "type": "object", + "required": ["measures"], + "properties": { + "record_type": {"const": "usage_observation"}, + "measures": {"type": "array", "items": {"type": "object", "additionalProperties": false, "required": ["name", "value", "unit"], "properties": {"name": {"type": "string"}, "value": {"type": "string"}, "unit": {"type": "string"}}}} + } + } + ], + "unevaluatedProperties": false + }, + "allocation": { + "allOf": [ + {"$ref": "#/$defs/common"}, + {"type": "object", "required": ["currency", "financial_fact_ids", "method", "allocated_amount", "shares", "residual_share"], "properties": {"record_type": {"const": "allocation"}, "currency": {"type": "string", "pattern": "^[A-Z]{3}$"}, "financial_fact_ids": {"type": "array", "minItems": 1, "items": {"type": "string"}}, "method": {"type": "string"}, "allocated_amount": {"$ref": "#/$defs/money"}, "shares": {"type": "array"}, "residual_share": {"type": "string"}}} + ], + "unevaluatedProperties": false + }, + "optimization": { + "allOf": [ + {"$ref": "#/$defs/common"}, + {"type": "object", "required": ["currency", "baseline", "alternative", "one_time_cost", "expected_period_savings", "assumptions"], "properties": {"record_type": {"const": "optimization"}, "currency": {"type": "string", "pattern": "^[A-Z]{3}$"}, "baseline": {"$ref": "#/$defs/cost_breakdown"}, "alternative": {"$ref": "#/$defs/cost_breakdown"}, "one_time_cost": {"$ref": "#/$defs/money"}, "expected_period_savings": {"$ref": "#/$defs/money"}, "assumptions": {"type": "array", "items": {"type": "string"}}}} + ], + "unevaluatedProperties": false + }, + "commitment_candidate": { + "allOf": [ + {"$ref": "#/$defs/common"}, + {"type": "object", "required": ["currency", "setup_cost", "recurring_cost", "cadence", "term_start", "approval_status"], "properties": {"record_type": {"const": "commitment_candidate"}, "currency": {"type": "string", "pattern": "^[A-Z]{3}$"}, "setup_cost": {"$ref": "#/$defs/money"}, "recurring_cost": {"$ref": "#/$defs/money"}, "cadence": {"enum": ["monthly", "quarterly", "annual", "one_time"]}, "term_start": {"type": "string", "format": "date"}, "term_end": {"type": ["string", "null"], "format": "date"}, "approval_status": {"const": "candidate"}}} + ], + "unevaluatedProperties": false + } + } +} diff --git a/schemas/resource-control-cycle.schema.json b/schemas/resource-control-cycle.schema.json new file mode 100644 index 0000000..94be5aa --- /dev/null +++ b/schemas/resource-control-cycle.schema.json @@ -0,0 +1,85 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://coulomb.social/resource-control/resource-control-cycle.schema.json", + "title": "Immutable resource forecast or actual observation", + "type": "object", + "additionalProperties": false, + "required": ["schema_version", "record_id", "record_type", "resource_id", "resource_class", "period", "created_at", "usage_proxies", "capacity", "costs", "allocation", "service_constraints", "evidence"], + "properties": { + "schema_version": {"const": "0.1"}, + "record_id": {"type": "string", "minLength": 1}, + "record_type": {"enum": ["forecast", "actual"]}, + "resource_id": {"type": "string", "pattern": "^resource:"}, + "resource_class": {"enum": ["storage", "vm", "cluster_compute", "shared_platform_service", "managed_service", "other"]}, + "period": {"type": "string", "pattern": "^[0-9]{4}-(0[1-9]|1[0-2])$"}, + "created_at": {"type": "string", "format": "date-time"}, + "scenario": {"enum": ["low", "base", "high", "observed"]}, + "forecast_ref": {"type": ["string", "null"]}, + "revision_of": {"type": ["string", "null"]}, + "usage_proxies": { + "type": "object", + "minProperties": 1, + "additionalProperties": {"$ref": "#/$defs/measurement"} + }, + "capacity": { + "type": "object", + "additionalProperties": false, + "required": ["model", "provisioned", "used"], + "properties": { + "model": {"enum": ["fixed", "elastic", "hybrid"]}, + "provisioned": {"type": "object", "additionalProperties": {"$ref": "#/$defs/measurement"}}, + "used": {"type": "object", "additionalProperties": {"$ref": "#/$defs/measurement"}} + } + }, + "costs": { + "type": "object", + "additionalProperties": false, + "required": ["currency", "infrastructure", "internal_labor", "external_labor", "total"], + "properties": { + "currency": {"const": "EUR"}, + "infrastructure": {"type": "number", "minimum": 0}, + "internal_labor": {"type": "number", "minimum": 0}, + "external_labor": {"type": "number", "minimum": 0}, + "total": {"type": "number", "minimum": 0}, + "booked_cost_refs": {"type": "array", "items": {"type": "string"}, "uniqueItems": true} + } + }, + "allocation": { + "type": "object", + "additionalProperties": false, + "required": ["method", "driver", "method_version", "unattributed_eur"], + "properties": { + "method": {"type": "string", "minLength": 1}, + "driver": {"type": "string", "minLength": 1}, + "method_version": {"type": "string", "minLength": 1}, + "unattributed_eur": {"type": "number", "minimum": 0} + } + }, + "service_constraints": {"type": "object", "additionalProperties": {"$ref": "#/$defs/measurement"}}, + "uncertainty": { + "type": "object", + "additionalProperties": false, + "required": ["level", "notes"], + "properties": { + "level": {"enum": ["low", "medium", "high"]}, + "notes": {"type": "array", "items": {"type": "string"}} + } + }, + "variance_attribution": { + "type": "object", + "additionalProperties": {"enum": ["demand", "provider_price", "allocation", "labor", "model", "data_quality"]} + }, + "evidence": {"type": "array", "minItems": 1, "items": {"type": "string"}} + }, + "$defs": { + "measurement": { + "type": "object", + "additionalProperties": false, + "required": ["value", "unit"], + "properties": { + "value": {"type": "number", "minimum": 0}, + "unit": {"type": "string", "minLength": 1} + } + } + } +} diff --git a/schemas/resource-inventory.schema.json b/schemas/resource-inventory.schema.json new file mode 100644 index 0000000..3782c57 --- /dev/null +++ b/schemas/resource-inventory.schema.json @@ -0,0 +1,136 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://coulomb.social/resource-control/resource-inventory.schema.json", + "title": "Railiance managed-infrastructure portfolio resource v0.2", + "type": "object", + "additionalProperties": false, + "required": ["schema_version", "record_scope", "id", "resource_class", "status", "management_model", "provider", "service", "location", "capacity", "ownership", "cost", "lifecycle", "relationships", "requirements", "evidence"], + "properties": { + "schema_version": {"const": "0.2"}, + "record_scope": {"enum": ["inventory", "example"]}, + "id": {"type": "string", "pattern": "^resource:[a-z0-9][a-z0-9:_-]+$"}, + "resource_class": {"enum": ["compute_instance", "storage", "network", "kubernetes_capacity", "database", "managed_service", "self_managed_service", "shared_platform_service", "license"]}, + "status": {"enum": ["proposed", "ordered", "commissioning", "active", "suspended", "retiring", "retired", "rejected"]}, + "management_model": {"enum": ["provider_managed", "self_managed", "shared_capacity"]}, + "provider": { + "type": "object", "additionalProperties": false, + "required": ["name", "account_ref", "product_ref", "provider_resource_id"], + "properties": { + "name": {"type": "string", "minLength": 1}, + "account_ref": {"type": ["string", "null"]}, + "product_ref": {"type": ["string", "null"]}, + "provider_resource_id": {"type": ["string", "null"]} + } + }, + "service": { + "type": "object", "additionalProperties": false, + "required": ["name", "service_id", "class", "capacity_model"], + "properties": { + "name": {"type": "string", "minLength": 1}, + "service_id": {"type": ["string", "null"]}, + "class": {"type": "string", "minLength": 1}, + "capacity_model": {"enum": ["elastic", "fixed", "shared", "licensed", "unknown"]} + } + }, + "location": { + "type": "object", "additionalProperties": false, + "required": ["region", "country", "failure_domains", "residency"], + "properties": { + "region": {"type": ["string", "null"]}, + "country": {"type": ["string", "null"], "pattern": "^[A-Z]{2}$"}, + "failure_domains": {"type": "array", "items": {"type": "string"}}, + "residency": {"type": ["string", "null"]} + } + }, + "capacity": { + "type": "array", + "items": { + "type": "object", "additionalProperties": false, + "required": ["metric", "value", "unit", "kind", "observed_at"], + "properties": { + "metric": {"type": "string", "minLength": 1}, + "value": {"type": ["number", "null"], "minimum": 0}, + "unit": {"type": "string", "minLength": 1}, + "kind": {"enum": ["provisioned", "usable", "limit", "allocated", "observed", "unknown"]}, + "observed_at": {"type": ["string", "null"], "format": "date"} + } + } + }, + "ownership": { + "type": "object", "additionalProperties": false, + "required": ["owner", "environment", "workload_ids", "tenant_id", "allocation"], + "properties": { + "owner": {"type": "string", "minLength": 1}, + "environment": {"type": "string", "minLength": 1}, + "workload_ids": {"type": "array", "items": {"type": "string"}}, + "tenant_id": {"type": ["string", "null"]}, + "allocation": { + "type": "object", "additionalProperties": false, + "required": ["mode", "cost_attribution_key", "driver", "method_version"], + "properties": { + "mode": {"enum": ["dedicated", "shared", "unattributed"]}, + "cost_attribution_key": {"type": ["string", "null"]}, + "driver": {"type": ["string", "null"]}, + "method_version": {"type": ["string", "null"]} + } + } + } + }, + "cost": { + "type": "object", "additionalProperties": false, + "required": ["currency", "tax_status", "billing_model", "commitment_ref", "price_evidence"], + "properties": { + "currency": {"type": ["string", "null"], "pattern": "^[A-Z]{3}$"}, + "tax_status": {"enum": ["included", "excluded", "unknown", "not_applicable"]}, + "billing_model": {"type": "string", "minLength": 1}, + "commitment_ref": {"type": ["string", "null"]}, + "price_evidence": {"type": ["string", "null"]} + } + }, + "lifecycle": { + "type": "object", "additionalProperties": false, + "required": ["proposed_on", "ordered_on", "commissioned_on", "renews_on", "cancel_by", "retired_on", "exit_path"], + "properties": { + "proposed_on": {"type": ["string", "null"], "format": "date"}, + "ordered_on": {"type": ["string", "null"], "format": "date"}, + "commissioned_on": {"type": ["string", "null"], "format": "date"}, + "renews_on": {"type": ["string", "null"], "format": "date"}, + "cancel_by": {"type": ["string", "null"], "format": "date"}, + "retired_on": {"type": ["string", "null"], "format": "date"}, + "exit_path": {"type": "string", "minLength": 1} + } + }, + "relationships": { + "type": "array", + "items": { + "type": "object", "additionalProperties": false, + "required": ["type", "resource_id"], + "properties": { + "type": {"enum": ["depends_on", "hosted_on", "part_of", "provides_capacity_to", "replaces", "backed_up_to"]}, + "resource_id": {"type": "string", "pattern": "^resource:"} + } + } + }, + "requirements": { + "type": "array", + "items": { + "type": "object", "additionalProperties": false, + "required": ["kind", "ref"], + "properties": {"kind": {"enum": ["service_objective", "security", "residency", "recovery", "retention", "performance", "other"]}, "ref": {"type": "string", "minLength": 1}} + } + }, + "evidence": { + "type": "array", + "items": { + "type": "object", "additionalProperties": false, + "required": ["kind", "ref", "observed_at", "authority"], + "properties": { + "kind": {"enum": ["provider", "contract", "telemetry", "workload", "decision", "test", "example"]}, + "ref": {"type": "string", "minLength": 1}, + "observed_at": {"type": ["string", "null"], "format": "date"}, + "authority": {"type": "string", "minLength": 1} + } + } + } + } +} diff --git a/tests/test_control_cycle.py b/tests/test_control_cycle.py new file mode 100644 index 0000000..0150474 --- /dev/null +++ b/tests/test_control_cycle.py @@ -0,0 +1,54 @@ +import sys +import unittest +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).parents[1] / "tools")) +from control_cycle import compare + + +def record(kind="forecast", resource_class="storage"): + result = { + "record_id": f"test:{kind}", "record_type": kind, + "resource_id": "resource:test", "resource_class": resource_class, + "period": "2026-09", "usage_proxies": {"stored_gb": {"value": 100, "unit": "GB-month"}}, + "costs": {"currency": "EUR", "infrastructure": 10, "internal_labor": 60, "external_labor": 0, "total": 70}, + "allocation": {"method": "direct", "driver": "stored_gb", "method_version": "1", "unattributed_eur": 0}, + } + if kind == "actual": + result["forecast_ref"] = "test:forecast" + return result + + +class ControlCycleTest(unittest.TestCase): + def test_same_mechanism_supports_required_resource_classes(self): + for resource_class in ("storage", "cluster_compute", "shared_platform_service"): + forecast = record(resource_class=resource_class) + actual = record("actual", resource_class) + actual["usage_proxies"]["stored_gb"]["value"] = 120 + result = compare(forecast, actual) + self.assertEqual(resource_class, result["resource_class"]) + self.assertEqual("demand", result["usage_proxies"]["stored_gb"]["category"]) + + def test_explicit_attribution_and_cost_split_are_preserved(self): + forecast, actual = record(), record("actual") + actual["costs"].update(infrastructure=12, internal_labor=90, total=102) + actual["variance_attribution"] = {"costs.infrastructure": "model", "costs.internal_labor": "labor"} + result = compare(forecast, actual) + self.assertEqual("model", result["costs"]["infrastructure"]["category"]) + self.assertEqual(30, result["costs"]["internal_labor"]["error"]) + + def test_rejects_wrong_forecast_reference(self): + forecast, actual = record(), record("actual") + actual["forecast_ref"] = "test:other" + with self.assertRaisesRegex(ValueError, "immutable forecast"): + compare(forecast, actual) + + def test_missing_proxy_is_data_quality_error(self): + forecast, actual = record(), record("actual") + actual["usage_proxies"] = {"cpu_hours": {"value": 4, "unit": "vCPU-hour"}} + result = compare(forecast, actual) + self.assertEqual("data_quality", result["usage_proxies"]["stored_gb"]["category"]) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_cost_model.py b/tests/test_cost_model.py new file mode 100644 index 0000000..b389865 --- /dev/null +++ b/tests/test_cost_model.py @@ -0,0 +1,62 @@ +import json +import sys +import unittest +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).parents[1] / "tools")) +from cost_model import forecast + + +class CostModelTest(unittest.TestCase): + def setUp(self): + root = Path(__file__).parents[1] + self.demand = json.loads((root / "data/demand/platform-audit-storage.json").read_text()) + self.providers = json.loads((root / "data/providers/object-storage.json").read_text()) + + def test_all_scenarios_and_providers_are_projected_for_12_months(self): + result = forecast(self.demand, self.providers) + for rows in result["scenarios"].values(): + self.assertEqual(12 * len(self.providers["providers"]), len(rows)) + + def test_unknown_prices_never_become_zero(self): + result = forecast(self.demand, self.providers) + host_europe = [r for r in result["scenarios"]["base"] if r["provider_id"] == "host-europe-cloud-storage"] + self.assertTrue(all(r["recurring_total_eur"] is None for r in host_europe)) + self.assertTrue(all(r["missing_price_fields"] for r in host_europe)) + + def test_storage_grows_month_over_month(self): + result = forecast(self.demand, self.providers) + rows = [r for r in result["scenarios"]["high"] if r["provider_id"] == "scaleway-standard-multi-az"] + self.assertGreater(rows[-1]["stored_gb"], rows[0]["stored_gb"]) + + def test_fixed_capacity_self_hosted_option_fails_closed_when_full(self): + result = forecast(self.demand, self.providers) + rows = [r for r in result["scenarios"]["base"] if r["provider_id"] == "hetzner-garage-3"] + self.assertIsNotNone(rows[0]["recurring_total_eur"]) + self.assertIsNone(rows[-1]["recurring_total_eur"]) + + def test_managed_cloud_comparators_calculate(self): + result = forecast(self.demand, self.providers) + ids = {"aws-s3-standard", "azure-blob-hot-zrs", "gcp-cloud-storage-standard", "stackit-object-storage"} + rows = [r for r in result["scenarios"]["base"] if r["month"] == 1 and r["provider_id"] in ids] + self.assertEqual(ids, {r["provider_id"] for r in rows}) + self.assertTrue(all(r["recurring_total_eur"] is not None for r in rows)) + + def test_recurring_total_is_infrastructure_plus_labor(self): + result = forecast(self.demand, self.providers) + row = next(r for r in result["comparison_320gb"] if r["provider_id"] == "stackit-object-storage") + self.assertAlmostEqual(row["recurring_total_eur"], row["monthly_infrastructure_eur"] + row["monthly_internal_labor_eur"]) + + def test_normalized_comparison_uses_320gb_for_every_provider(self): + result = forecast(self.demand, self.providers) + self.assertEqual(len(self.providers["providers"]), len(result["comparison_320gb"])) + self.assertTrue(all(r["stored_gb"] == 320 for r in result["comparison_320gb"])) + + def test_garage_external_setup_services_are_unquoted(self): + result = forecast(self.demand, self.providers) + rows = [r for r in result["comparison_320gb"] if "garage" in r["provider_id"]] + self.assertTrue(all(r["setup_external_services_eur"] is None for r in rows)) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_financial_exchange.py b/tests/test_financial_exchange.py new file mode 100644 index 0000000..0ddc201 --- /dev/null +++ b/tests/test_financial_exchange.py @@ -0,0 +1,60 @@ +import sys +import unittest +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).parents[1] / "tools")) +from financial_exchange import forecast_records, reconcile, validate_booked_cost + + +class FinancialExchangeTest(unittest.TestCase): + def setUp(self): + self.forecast = { + "schema_version": "0.1", "record_type": "forecast", + "workload": "platform-pg", "cost_attribution_key": "platform:audit-storage", + "provider_id": "scaleway-standard-multi-az", "created_at": "2026-08-10T17:10:00Z", + "scenario": "base", "forecast_ref": None, + "rows": [{"period": "2026-09", "database_gb": 5, "stored_gb": 180, + "wal_gb": 30, "restore_egress_gb": 5, "write_requests": 2500, + "read_requests": 1000, "infrastructure_eur": 2.89, + "internal_labor_hours": 1, "internal_labor_eur": 60, + "total_eur": 62.89}] + } + + def test_forecast_export_separates_costs_and_is_stable(self): + first = forecast_records(self.forecast, resource_id="resource:platform_audit_storage", source_ref="forecast.json") + second = forecast_records(self.forecast, resource_id="resource:platform_audit_storage", source_ref="forecast.json") + self.assertEqual(first, second) + self.assertEqual("2.89", first[0]["costs"]["infrastructure"]) + self.assertEqual("60.00", first[0]["costs"]["internal_labor"]) + self.assertEqual("usage_observation" not in first[0]["record_type"], True) + + def test_reconciles_only_attributable_booked_cost(self): + forecast = forecast_records(self.forecast, resource_id="resource:platform_audit_storage", source_ref="forecast.json") + booked = [{ + "schema_version": "0.1", "record_type": "booked_cost", "financial_fact_id": "fact:1", + "correction_of": None, "adjustment_kind": "charge", "source_type": "provider_invoice", + "source_document_id": "invoice:1", "source_line_id": "invoice:1:1", + "content_fingerprint": "sha256:x", "provider": "scaleway", + "accounting_period": "2026-09", "currency": "EUR", "gross_amount": "3.10", + "adjustment_amount": "0.00", "effective_amount": "3.10", "tax_status": "unknown", + "tax_amount": None, "cost_attribution_key": "platform:audit-storage", + "source_evidence_ref": "invoice:1", "recorded_at": "2026-10-01T00:00:00Z" + }] + rows = reconcile(forecast, booked) + self.assertEqual("reconciled", rows[0]["status"]) + self.assertEqual("0.21", rows[0]["variance"]) + self.assertEqual(["fact:1"], rows[0]["financial_fact_ids"]) + + def test_rejects_invalid_booked_arithmetic(self): + with self.assertRaisesRegex(ValueError, "effective_amount"): + validate_booked_cost({ + "schema_version": "0.1", "record_type": "booked_cost", "financial_fact_id": "fact:1", + "adjustment_kind": "charge", "source_document_id": "doc", "source_line_id": "line", + "content_fingerprint": "hash", "provider": "provider", "accounting_period": "2026-09", + "currency": "EUR", "gross_amount": "1.00", "adjustment_amount": "0.00", + "effective_amount": "2.00", "source_evidence_ref": "doc", "recorded_at": "2026-10-01T00:00:00Z" + }) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_optimization.py b/tests/test_optimization.py new file mode 100644 index 0000000..ab29751 --- /dev/null +++ b/tests/test_optimization.py @@ -0,0 +1,248 @@ +import json +import sys +import unittest +from copy import deepcopy +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).parents[1] / "tools")) +from optimization import ( + compare_option, + evaluate, + recurring_total, + utilization_ratios, + validate_case, +) + +CASE_DIR = Path(__file__).parents[1] / "data" / "optimization" + + +def option(option_id="alt", **overrides): + result = { + "option_id": option_id, + "label": f"option {option_id}", + "one_time_eur": 0, + "recurring_infrastructure_eur_month": 10, + "recurring_internal_labor_eur_month": 60, + "recurring_external_labor_eur_month": 0, + "utilization": { + "cpu": { + "provisioned": {"value": 4, "unit": "vCPU"}, + "used": {"value": 1, "unit": "vCPU"}, + } + }, + "uncertainty": {"level": "low", "notes": []}, + "service_constraints": {"nodes": {"value": 1, "unit": "count"}}, + "failure_domains": ["provider:test"], + "exit_path": "cancel and erase", + "unknowns": [], + } + result.update(overrides) + return result + + +def case(**overrides): + result = { + "schema_version": "0.1", + "record_scope": "illustrative", + "case_id": "opt:test:2026-08", + "case_type": "rightsizing", + "trigger": "cadence", + "review_period": "2026-08", + "created_at": "2026-08-14T00:00:00Z", + "resource_ids": ["resource:test"], + "baseline": option("baseline"), + "alternatives": [option("alt", recurring_infrastructure_eur_month=0)], + "decision": { + "state": "proposed", + "recommended_option_id": "alt", + "rationale": "cheaper", + "approver": None, + "approved_on": None, + "delegated_to": [], + }, + "evidence": [ + {"kind": "telemetry", "ref": "test", "authority": "test", "observed_at": None} + ], + } + result.update(overrides) + return result + + +class CostArithmeticTest(unittest.TestCase): + def test_recurring_total_sums_all_three_cost_components(self): + self.assertEqual(70, recurring_total(option())) + + def test_recurring_total_is_unknown_when_any_component_is_unknown(self): + for field in ( + "recurring_infrastructure_eur_month", + "recurring_internal_labor_eur_month", + "recurring_external_labor_eur_month", + ): + self.assertIsNone(recurring_total(option(**{field: None}))) + + def test_utilization_ratio_is_unknown_rather_than_zero_when_unmeasured(self): + unknown = option( + utilization={ + "cpu": { + "provisioned": {"value": None, "unit": "vCPU"}, + "used": {"value": 1, "unit": "vCPU"}, + } + } + ) + self.assertIsNone(utilization_ratios(unknown)["cpu"]) + self.assertEqual(0.25, utilization_ratios(option())["cpu"]) + + +class ComparisonTest(unittest.TestCase): + def test_material_saving_is_recommended_with_computed_payback(self): + baseline = option("baseline") + alternative = option("alt", recurring_infrastructure_eur_month=0, one_time_eur=100) + result = compare_option(baseline, alternative) + self.assertEqual("recommend", result["verdict"]) + self.assertEqual(10, result["monthly_saving_eur"]) + self.assertEqual(10.0, result["payback_months"]) + + def test_more_expensive_alternative_is_rejected_and_never_pays_back(self): + result = compare_option(option("baseline"), option("alt", recurring_infrastructure_eur_month=100)) + self.assertEqual("reject", result["verdict"]) + self.assertIsNone(result["payback_months"]) + self.assertIn("never", result["payback_note"]) + + def test_difference_below_materiality_threshold_is_not_a_change(self): + result = compare_option(option("baseline"), option("alt", recurring_infrastructure_eur_month=8)) + self.assertEqual("no_material_change", result["verdict"]) + + def test_labor_increase_can_cancel_an_infrastructure_saving(self): + alternative = option( + "alt", recurring_infrastructure_eur_month=0, recurring_internal_labor_eur_month=90 + ) + result = compare_option(option("baseline"), alternative) + self.assertEqual(20, result["monthly_delta_eur"]) + self.assertEqual("reject", result["verdict"]) + + def test_unknown_cost_blocks_the_comparison_instead_of_assuming_zero(self): + result = compare_option(option("baseline"), option("alt", recurring_infrastructure_eur_month=None)) + self.assertEqual("blocked_on_evidence", result["verdict"]) + self.assertIsNone(result["monthly_saving_eur"]) + self.assertIn("alt.recurring_infrastructure_eur_month", result["blocking_evidence"]) + + def test_missing_exit_path_blocks_an_otherwise_cheaper_option(self): + alternative = option("alt", recurring_infrastructure_eur_month=0, exit_path=None) + result = compare_option(option("baseline"), alternative) + self.assertEqual("blocked_on_evidence", result["verdict"]) + self.assertFalse(result["exit_path_known"]) + + def test_named_unknown_blocks_even_when_every_number_is_present(self): + alternative = option("alt", recurring_infrastructure_eur_month=0, unknowns=["price not confirmed"]) + result = compare_option(option("baseline"), alternative) + self.assertEqual("blocked_on_evidence", result["verdict"]) + self.assertIn("alt.unknown:price not confirmed", result["blocking_evidence"]) + + def test_unknown_baseline_blocks_every_alternative(self): + baseline = option("baseline", recurring_infrastructure_eur_month=None) + result = compare_option(baseline, option("alt", recurring_infrastructure_eur_month=0)) + self.assertEqual("blocked_on_evidence", result["verdict"]) + self.assertIn("baseline.recurring_infrastructure_eur_month", result["blocking_evidence"]) + + def test_failure_domain_changes_are_reported_in_both_directions(self): + baseline = option("baseline", failure_domains=["provider:test", "host:one"]) + alternative = option("alt", failure_domains=["provider:test", "region:two"]) + result = compare_option(baseline, alternative) + self.assertEqual(["host:one"], result["failure_domains_removed"]) + self.assertEqual(["region:two"], result["failure_domains_added"]) + + +class CaseValidationTest(unittest.TestCase): + def test_valid_case_evaluates_and_picks_the_best_option(self): + record = case() + validate_case(record) + self.assertEqual("alt", evaluate(record)["best_option_id"]) + + def test_blocked_case_cannot_be_proposed(self): + record = case(alternatives=[option("alt", one_time_eur=None)]) + with self.assertRaises(ValueError): + validate_case(record) + + def test_unblocked_case_cannot_claim_to_be_blocked(self): + record = case() + record["decision"]["state"] = "blocked_on_evidence" + with self.assertRaises(ValueError): + validate_case(record) + + def test_approval_requires_a_named_authority_and_date(self): + record = case() + record["decision"]["state"] = "approved" + with self.assertRaises(ValueError): + validate_case(record) + record["decision"]["approver"] = "human financial authority" + record["decision"]["approved_on"] = "2026-08-14" + validate_case(record) + + def test_blocked_case_cannot_be_approved(self): + record = case(alternatives=[option("alt", exit_path=None)]) + record["decision"].update( + {"state": "approved", "approver": "human", "approved_on": "2026-08-14"} + ) + with self.assertRaises(ValueError): + validate_case(record) + + def test_duplicate_option_identifiers_are_rejected(self): + record = case(alternatives=[option("baseline", recurring_infrastructure_eur_month=0)]) + with self.assertRaises(ValueError): + validate_case(record) + + def test_resource_ids_must_be_portfolio_identifiers(self): + with self.assertRaises(ValueError): + validate_case(case(resource_ids=["railiance01"])) + + +class RegisteredCasesTest(unittest.TestCase): + """The two cases required by RESOURCE-WP-0003-T06: storage and non-storage.""" + + def setUp(self): + self.cases = { + path.stem: json.loads(path.read_text()) for path in sorted(CASE_DIR.glob("*.json")) + } + + def test_every_registered_case_validates(self): + self.assertTrue(self.cases) + for record in self.cases.values(): + validate_case(record) + + def test_process_is_validated_on_a_storage_and_a_non_storage_candidate(self): + types = {record["case_id"] for record in self.cases.values()} + self.assertIn("opt:platform-audit-storage:2026-08", types) + self.assertIn("opt:reef-railiance-k3s:2026-08", types) + + def test_storage_case_computes_hetzner_and_blocks_host_europe(self): + report = evaluate(self.cases["platform-audit-storage-2026-08"]) + verdicts = {c["option_id"]: c["verdict"] for c in report["comparisons"]} + self.assertEqual("reject", verdicts["hetzner-object-storage"]) + self.assertEqual("blocked_on_evidence", verdicts["host-europe-cloud-storage"]) + hetzner = next(c for c in report["comparisons"] if c["option_id"] == "hetzner-object-storage") + self.assertEqual(29.14, hetzner["monthly_delta_eur"]) + + def test_cluster_case_reports_low_utilization_but_refuses_to_recommend(self): + report = evaluate(self.cases["reef-railiance-k3s-2026-08"]) + self.assertIsNone(report["best_option_id"]) + self.assertEqual(0.141, report["baseline"]["utilization"]["cpu"]) + self.assertIsNone(report["baseline"]["recurring_eur_month"]) + for comparison in report["comparisons"]: + self.assertEqual("blocked_on_evidence", comparison["verdict"]) + self.assertTrue(comparison["blocking_evidence"]) + + def test_blocking_evidence_names_an_owner_for_every_cluster_unknown(self): + record = self.cases["reef-railiance-k3s-2026-08"] + for option_record in [record["baseline"]] + record["alternatives"]: + for unknown in option_record["unknowns"]: + self.assertIn("owner:", unknown) + + def test_registered_cases_are_not_silently_mutated_by_evaluation(self): + for name, record in self.cases.items(): + before = deepcopy(record) + evaluate(record) + self.assertEqual(before, record, name) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_portfolio.py b/tests/test_portfolio.py new file mode 100644 index 0000000..b91f7d9 --- /dev/null +++ b/tests/test_portfolio.py @@ -0,0 +1,81 @@ +import json +import sys +import unittest +from copy import deepcopy +from pathlib import Path + +ROOT = Path(__file__).parents[1] +sys.path.insert(0, str(ROOT / "tools")) +from portfolio import validate_record, validate_transition + + +class PortfolioTest(unittest.TestCase): + def records(self): + paths = list((ROOT / "data/resources").glob("*.json")) + paths += list((ROOT / "examples/portfolio").glob("*.json")) + return [(path, json.loads(path.read_text())) for path in paths] + + def test_inventory_and_examples_are_semantically_valid(self): + records = self.records() + self.assertGreaterEqual(len(records), 4) + for path, record in records: + with self.subTest(path=path): + validate_record(record) + + def test_examples_cover_required_portfolio_shapes(self): + classes = {record["resource_class"] for _, record in self.records()} + self.assertTrue({"storage", "self_managed_service", "kubernetes_capacity", "shared_platform_service"} <= classes) + models = {record["management_model"] for _, record in self.records()} + self.assertEqual({"provider_managed", "self_managed", "shared_capacity"}, models) + + def test_shared_resource_requires_allocation_driver(self): + record = deepcopy(next(r for _, r in self.records() if r["ownership"]["allocation"]["mode"] == "shared")) + record["ownership"]["allocation"]["driver"] = None + with self.assertRaisesRegex(ValueError, "shared resources"): + validate_record(record) + + def test_unknown_commission_date_is_preserved(self): + record = deepcopy(next(r for _, r in self.records() if r["status"] == "active")) + record["lifecycle"]["commissioned_on"] = None + validate_record(record) + + def test_lifecycle_dates_cannot_be_out_of_order(self): + record = deepcopy(self.records()[0][1]) + record["lifecycle"]["ordered_on"] = "2026-08-12" + record["lifecycle"]["commissioned_on"] = "2026-08-11" + with self.assertRaisesRegex(ValueError, "out of order"): + validate_record(record) + + def test_lifecycle_transition_rules(self): + validate_transition("proposed", "ordered") + validate_transition("active", "retiring") + validate_transition("suspended", "active") + with self.assertRaisesRegex(ValueError, "invalid lifecycle transition"): + validate_transition("proposed", "active") + with self.assertRaisesRegex(ValueError, "invalid lifecycle transition"): + validate_transition("retired", "active") + + def test_inventory_relationships_resolve_to_inventory_records(self): + inventory = {r["id"]: r for _, r in self.records() if r["record_scope"] == "inventory"} + for record in inventory.values(): + for relationship in record["relationships"]: + self.assertIn(relationship["resource_id"], inventory) + + def test_initial_coverage_references_the_complete_inventory(self): + inventory = {r["id"] for _, r in self.records() if r["record_scope"] == "inventory"} + coverage = json.loads((ROOT / "data/portfolio-coverage-2026-08-11.json").read_text()) + referenced = { + resource_id + for group in coverage["coverage"] + for resource_id in group["resource_ids"] + } + self.assertEqual(len(inventory), coverage["inventory_records"]) + self.assertEqual(inventory, referenced) + self.assertEqual( + {"helix-forge", "coulomb-social", "shared-railiance", "representative-tenant"}, + {group["group"] for group in coverage["coverage"]}, + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_portfolio_report.py b/tests/test_portfolio_report.py new file mode 100644 index 0000000..efe3563 --- /dev/null +++ b/tests/test_portfolio_report.py @@ -0,0 +1,163 @@ +import sys +import unittest +from datetime import date +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).parents[1] / "tools")) +from portfolio_report import ( + build, + cost_section, + lifecycle_section, + renewals_section, + risk_section, + utilization_section, +) + +ROOT = Path(__file__).parents[1] +TODAY = date(2026, 8, 14) + + +def resource(rid="resource:test", **overrides): + result = { + "id": rid, + "status": "active", + "location": {"failure_domains": ["provider:test"]}, + "capacity": [ + {"metric": "cpu", "value": 4, "unit": "vCPU", "kind": "usable", "observed_at": "2026-08-11"}, + {"metric": "cpu_usage", "value": 1, "unit": "vCPU", "kind": "observed", "observed_at": "2026-08-11"}, + ], + "ownership": {"owner": "test-owner", "allocation": {"mode": "dedicated"}}, + "cost": {"price_evidence": "quote#1", "billing_model": "fixed"}, + "lifecycle": {"renews_on": None, "cancel_by": None}, + } + result.update(overrides) + return result + + +class UtilizationTest(unittest.TestCase): + def test_paired_metrics_produce_a_ratio_and_a_signal(self): + section = utilization_section([resource()]) + metric = section["measured"][0]["metrics"][0] + self.assertEqual(0.25, metric["ratio"]) + self.assertEqual("idle", metric["signal"]) + self.assertEqual(["resource:test"], section["idle"]) + + def test_saturated_capacity_is_flagged_separately(self): + record = resource() + record["capacity"][1]["value"] = 3.8 + section = utilization_section([record]) + self.assertEqual(["resource:test"], section["saturated"]) + self.assertEqual([], section["idle"]) + + def test_unpaired_capacity_is_reported_as_unmeasured_not_omitted(self): + record = resource(capacity=[{"metric": "cpu", "value": 4, "unit": "vCPU", "kind": "usable"}]) + section = utilization_section([record]) + self.assertEqual([], section["measured"]) + self.assertEqual("resource:test", section["unmeasured"][0]["resource_id"]) + + def test_zero_provisioned_capacity_does_not_divide_by_zero(self): + record = resource() + record["capacity"][0]["value"] = 0 + self.assertEqual([], utilization_section([record])["measured"]) + + +class CostTest(unittest.TestCase): + def test_portfolio_spend_is_unknown_rather_than_a_partial_sum(self): + section = cost_section([resource(), resource("resource:b", cost={"price_evidence": None, "billing_model": "unknown"})]) + self.assertIsNone(section["known_monthly_spend_eur"]) + self.assertEqual(1, len(section["unpriced"])) + self.assertIn("not computable", section["spend_note"]) + + def test_unattributed_resources_are_named_not_spread(self): + record = resource("resource:shared", ownership={"owner": "o", "allocation": {"mode": "unattributed"}}) + section = cost_section([record]) + self.assertEqual(["resource:shared"], [r["resource_id"] for r in section["unattributed_allocation"]]) + + +class RenewalTest(unittest.TestCase): + def test_dates_inside_the_horizon_are_reported_with_days_remaining(self): + record = resource(lifecycle={"renews_on": "2026-09-01", "cancel_by": None}) + section = renewals_section([record], TODAY) + self.assertEqual(18, section["approaching"][0]["days_remaining"]) + + def test_dates_beyond_the_horizon_are_not_reported(self): + record = resource(lifecycle={"renews_on": "2027-09-01", "cancel_by": None}) + self.assertEqual([], renewals_section([record], TODAY)["approaching"]) + + def test_active_resource_without_dates_is_a_named_gap(self): + section = renewals_section([resource()], TODAY) + self.assertEqual("resource:test", section["undated"][0]["resource_id"]) + + def test_retired_resource_without_dates_is_not_a_gap(self): + self.assertEqual([], renewals_section([resource(status="retired")], TODAY)["undated"]) + + +class RiskTest(unittest.TestCase): + def test_shared_failure_domain_is_flagged_above_two_resources(self): + records = [resource(f"resource:{i}") for i in range(3)] + risks = risk_section(records, utilization_section(records), cost_section(records)) + kinds = {risk["kind"] for risk in risks} + self.assertIn("concentrated_failure_domain", kinds) + + def test_two_resources_sharing_a_domain_is_not_yet_a_concentration_risk(self): + records = [resource(f"resource:{i}") for i in range(2)] + risks = risk_section(records, utilization_section(records), cost_section(records)) + self.assertNotIn("concentrated_failure_domain", {risk["kind"] for risk in risks}) + + +class LifecycleTest(unittest.TestCase): + def test_states_are_counted_and_unowned_resources_named(self): + records = [resource(), resource("resource:b", status="proposed", + ownership={"owner": None, "allocation": {"mode": "dedicated"}})] + section = lifecycle_section(records) + self.assertEqual({"active": 1, "proposed": 1}, section["by_status"]) + self.assertEqual(["resource:b"], section["without_owner"]) + + +class LivePortfolioTest(unittest.TestCase): + """The report must render from the real committed portfolio, not fixtures.""" + + @classmethod + def setUpClass(cls): + cls.report = build(ROOT, TODAY) + + def test_report_covers_every_registered_resource(self): + self.assertEqual(7, self.report["resource_count"]) + self.assertEqual([], self.report["lifecycle"]["without_owner"]) + + def test_every_service_group_from_discovery_is_present(self): + groups = {row["group"] for row in self.report["coverage"]["groups"]} + self.assertEqual( + {"helix-forge", "coulomb-social", "shared-railiance", "representative-tenant"}, groups + ) + + def test_spend_is_reported_unknown_while_no_booked_cost_exists(self): + self.assertIsNone(self.report["cost"]["known_monthly_spend_eur"]) + self.assertTrue(self.report["cost"]["unpriced"]) + + def test_host_concentration_is_surfaced_as_a_risk(self): + concentrations = [r for r in self.report["risks"] if r["kind"] == "concentrated_failure_domain"] + self.assertIn("host:railiance01", " ".join(r["detail"] for r in concentrations)) + + def test_idle_cluster_capacity_is_surfaced(self): + self.assertIn("resource:railiance:reef-railiance:k3s", self.report["utilization"]["idle"]) + + def test_missing_contract_dates_are_surfaced_rather_than_read_as_no_commitments(self): + self.assertTrue(self.report["renewals"]["undated"]) + + def test_open_optimization_cases_are_listed_with_their_blockers(self): + self.assertEqual(2, len(self.report["optimization"]["cases"])) + self.assertEqual(2, len(self.report["optimization"]["undecided"])) + self.assertTrue(self.report["optimization"]["blocked_on_evidence"]) + + def test_next_actions_name_an_owning_repository_for_each_gap(self): + self.assertTrue(self.report["next_actions"]) + for action in self.report["next_actions"]: + self.assertIn(":", action) + + def test_report_is_deterministic_for_a_fixed_date(self): + self.assertEqual(self.report, build(ROOT, TODAY)) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_variance.py b/tests/test_variance.py new file mode 100644 index 0000000..de15cd6 --- /dev/null +++ b/tests/test_variance.py @@ -0,0 +1,25 @@ +import sys +import unittest +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).parents[1] / "tools")) +from variance import compare + + +class VarianceTest(unittest.TestCase): + def test_reports_signed_error_and_absolute_percentage_error(self): + base = {"period":"2026-09","database_gb":5,"stored_gb":100,"wal_gb":10,"restore_egress_gb":5,"write_requests":100,"read_requests":50,"infrastructure_eur":10,"internal_labor_hours":1,"internal_labor_eur":60,"total_eur":70} + actual_row = dict(base, stored_gb=120, infrastructure_eur=12, total_eur=72) + result = compare({"created_at":"2026-08-10T00:00:00Z","rows":[base]}, {"provider_id":"test","rows":[actual_row]}) + stored = result["rows"][0]["metrics"]["stored_gb"] + self.assertEqual(20, stored["error"]) + self.assertEqual(20, stored["absolute_percentage_error"]) + + def test_zero_forecast_has_no_percentage_error(self): + base = {"period":"2026-09","database_gb":0,"stored_gb":0,"wal_gb":0,"restore_egress_gb":0,"write_requests":0,"read_requests":0,"infrastructure_eur":0,"internal_labor_hours":0,"internal_labor_eur":0,"total_eur":0} + result = compare({"created_at":"x","rows":[base]}, {"provider_id":"test","rows":[dict(base, wal_gb=1)]}) + self.assertIsNone(result["rows"][0]["metrics"]["wal_gb"]["absolute_percentage_error"]) + + +if __name__ == "__main__": + unittest.main() diff --git a/tools/control_cycle.py b/tools/control_cycle.py new file mode 100644 index 0000000..53e4a1e --- /dev/null +++ b/tools/control_cycle.py @@ -0,0 +1,83 @@ +#!/usr/bin/env python3 +"""Compare a generic immutable resource forecast with an actual observation.""" + +from __future__ import annotations + +import json +import sys +from pathlib import Path + +COST_FIELDS = ("infrastructure", "internal_labor", "external_labor", "total") +ATTRIBUTIONS = {"demand", "provider_price", "allocation", "labor", "model", "data_quality"} + + +def delta(forecast: float, actual: float) -> dict: + error = actual - forecast + return { + "forecast": forecast, + "actual": actual, + "error": round(error, 4), + "absolute_percentage_error": None if forecast == 0 else round(abs(error) / forecast * 100, 2), + } + + +def compare(forecast: dict, actual: dict) -> dict: + if forecast["record_type"] != "forecast" or actual["record_type"] != "actual": + raise ValueError("expected forecast and actual records") + for field in ("resource_id", "resource_class", "period"): + if forecast[field] != actual[field]: + raise ValueError(f"{field} mismatch") + if actual.get("forecast_ref") != forecast["record_id"]: + raise ValueError("actual forecast_ref must identify the immutable forecast") + + attribution = actual.get("variance_attribution", {}) + unknown = set(attribution.values()) - ATTRIBUTIONS + if unknown: + raise ValueError(f"unknown variance attribution: {sorted(unknown)}") + + proxies = {} + all_proxies = sorted(set(forecast["usage_proxies"]) | set(actual["usage_proxies"])) + for name in all_proxies: + planned = forecast["usage_proxies"].get(name) + observed = actual["usage_proxies"].get(name) + if planned is None or observed is None: + proxies[name] = {"status": "missing", "category": "data_quality"} + elif planned["unit"] != observed["unit"]: + proxies[name] = {"status": "unit-mismatch", "category": "data_quality"} + else: + proxies[name] = {**delta(planned["value"], observed["value"]), "unit": planned["unit"], "category": attribution.get(name, "demand")} + + costs = {} + for name in COST_FIELDS: + default_category = "labor" if "labor" in name else "provider_price" + costs[name] = {**delta(forecast["costs"][name], actual["costs"][name]), "currency": "EUR", "category": attribution.get(f"costs.{name}", default_category)} + + if forecast["allocation"] != actual["allocation"]: + costs["allocation_method"] = {"status": "changed", "category": attribution.get("allocation", "allocation")} + + return { + "forecast_ref": forecast["record_id"], + "actual_ref": actual["record_id"], + "resource_id": forecast["resource_id"], + "resource_class": forecast["resource_class"], + "period": forecast["period"], + "usage_proxies": proxies, + "costs": costs, + } + + +def main() -> int: + if len(sys.argv) != 3: + print(f"usage: {sys.argv[0]} FORECAST.json ACTUAL.json", file=sys.stderr) + return 2 + try: + result = compare(json.loads(Path(sys.argv[1]).read_text()), json.loads(Path(sys.argv[2]).read_text())) + except (KeyError, ValueError) as exc: + print(f"control-cycle error: {exc}", file=sys.stderr) + return 1 + print(json.dumps(result, indent=2)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/cost_model.py b/tools/cost_model.py new file mode 100644 index 0000000..88e4902 --- /dev/null +++ b/tools/cost_model.py @@ -0,0 +1,99 @@ +#!/usr/bin/env python3 +"""Calculate comparable 12-month object-storage forecasts from JSON evidence.""" + +from __future__ import annotations + +import json +import sys +from pathlib import Path + + +def quote_provider(provider: dict, scenario: dict, stored_gb: float, restore_gb: float) -> dict: + excess_storage = max(0, stored_gb - provider["included_storage_gb"]) + excess_egress = max(0, restore_gb - provider["included_egress_gb"]) + missing = [] + if excess_storage and provider["storage_eur_per_gb_month"] is None: + missing.append("storage_eur_per_gb_month") + if excess_egress and provider["egress_eur_per_gb"] is None: + missing.append("egress_eur_per_gb") + for field in ("monthly_minimum_eur", "operations_eur_per_month", "support_eur_per_month"): + if provider[field] is None: + missing.append(field) + + infrastructure = None + labor_hours = max(scenario["operator_hours_per_month"], provider["operator_hours_per_month"]) + labor = labor_hours * scenario["operator_hourly_eur"] + recurring = None + exit_cost = None + if not missing: + usage = excess_storage * (provider["storage_eur_per_gb_month"] or 0) + usage += excess_egress * (provider["egress_eur_per_gb"] or 0) + usage += scenario.get("write_requests_per_month", 0) / 1000 * provider.get("write_eur_per_1000", 0) + usage += scenario.get("read_requests_per_month", 0) / 1000 * provider.get("read_eur_per_1000", 0) + service = max(provider["monthly_minimum_eur"], usage) + infrastructure = service + provider["operations_eur_per_month"] + provider["support_eur_per_month"] + recurring = infrastructure + labor + exit_excess = max(0, stored_gb - provider["included_egress_gb"]) + if exit_excess and provider["egress_eur_per_gb"] is None: + missing.append("egress_eur_per_gb_for_exit") + else: + exit_cost = exit_excess * (provider["egress_eur_per_gb"] or 0) + 4 * scenario["operator_hourly_eur"] + + setup_internal_hours = provider.get("setup_operator_hours", 0) + setup_internal = setup_internal_hours * scenario["operator_hourly_eur"] + setup_external = provider.get("setup_external_eur", None if "garage" in provider["id"] else 0) + return { + "monthly_infrastructure_eur": None if infrastructure is None else round(infrastructure, 2), + "monthly_internal_labor_hours": labor_hours, + "monthly_internal_labor_eur": round(labor, 2), + "recurring_total_eur": None if recurring is None else round(recurring, 2), + "setup_internal_labor_hours": setup_internal_hours, + "setup_internal_labor_eur": round(setup_internal, 2), + "setup_external_services_eur": setup_external, + "setup_known_total_eur": None if setup_external is None else round(setup_internal + setup_external, 2), + "exit_cost_eur": None if exit_cost is None else round(exit_cost, 2), + "missing_price_fields": sorted(set(missing)), + } + + +def forecast(demand: dict, catalog: dict) -> dict: + result = {"currency": demand["currency"], "months": 12, "scenarios": {}, "comparison_320gb": []} + retention = demand["retention_days"] + backups_per_day = demand["base_backups_per_day"] + for scenario_name, scenario in demand["scenarios"].items(): + rows = [] + db_gb = scenario["initial_database_gb"] + for month in range(1, 13): + stored_gb = db_gb * retention * backups_per_day + scenario["wal_gb_per_day"] * retention + restore_gb = scenario["restore_egress_gb"] + for provider in catalog["providers"]: + rows.append({ + "month": month, "provider_id": provider["id"], + "database_gb": round(db_gb, 3), "stored_gb": round(stored_gb, 3), + "restore_egress_gb": restore_gb, + **quote_provider(provider, scenario, stored_gb, restore_gb), + }) + db_gb *= 1 + scenario["monthly_database_growth_pct"] / 100 + result["scenarios"][scenario_name] = rows + normalized = demand["scenarios"]["base"] + for provider in catalog["providers"]: + result["comparison_320gb"].append({ + "provider_id": provider["id"], "stored_gb": 320, + "restore_egress_gb": normalized["restore_egress_gb"], + **quote_provider(provider, normalized, 320, normalized["restore_egress_gb"]), + }) + return result + + +def main() -> int: + if len(sys.argv) != 3: + print(f"usage: {sys.argv[0]} DEMAND.json PROVIDERS.json", file=sys.stderr) + return 2 + demand = json.loads(Path(sys.argv[1]).read_text()) + catalog = json.loads(Path(sys.argv[2]).read_text()) + print(json.dumps(forecast(demand, catalog), indent=2)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/financial_exchange.py b/tools/financial_exchange.py new file mode 100644 index 0000000..0609346 --- /dev/null +++ b/tools/financial_exchange.py @@ -0,0 +1,144 @@ +#!/usr/bin/env python3 +"""Produce planning evidence and consume fin-hub booked-cost projections.""" + +from __future__ import annotations + +import json +import sys +from calendar import monthrange +from datetime import date +from decimal import Decimal, ROUND_HALF_EVEN +from pathlib import Path + +CENT = Decimal("0.01") + + +def money(value: object) -> Decimal: + return Decimal(str(value)).quantize(CENT, rounding=ROUND_HALF_EVEN) + + +def money_text(value: object) -> str: + return format(money(value), ".2f") + + +def forecast_records(payload: dict, *, resource_id: str, source_ref: str) -> list[dict]: + if payload.get("schema_version") != "0.1" or payload.get("record_type") != "forecast": + raise ValueError("expected a resource-control forecast v0.1") + if not resource_id.startswith("resource:"): + raise ValueError("resource_id must use the resource: namespace") + created_at = payload["created_at"] + version = f"{payload['provider_id']}:{created_at}" + records = [] + for row in payload["rows"]: + infrastructure = money(row["infrastructure_eur"]) + labor = money(row["internal_labor_eur"]) + if money(row["total_eur"]) != infrastructure + labor: + raise ValueError("total_eur does not match infrastructure plus internal labor") + year, month = (int(part) for part in row["period"].split("-")) + period_start = date(year, month, 1) + period_end = date(year, month, monthrange(year, month)[1]) + records.append({ + "schema_version": "0.1", + "record_type": "forecast", + "record_id": f"forecast:{payload['provider_id']}:{payload['cost_attribution_key']}:{row['period']}:{created_at}", + "revision_of": payload.get("forecast_ref"), + "resource_id": resource_id, + "service_id": payload["provider_id"], + "workload_id": payload["workload"], + "tenant_id": None, + "environment": "production", + "cost_attribution_key": payload["cost_attribution_key"], + "period_start": period_start.isoformat(), + "period_end": period_end.isoformat(), + "currency": "EUR", + "source_evidence": [source_ref, *row.get("evidence", [])], + "created_at": created_at, + "scenario": payload.get("scenario") or "base", + "forecast_version": version, + "costs": { + "infrastructure": money_text(infrastructure), + "internal_labor": money_text(labor), + "external_services": "0.00", + "setup": "0.00", + "other": "0.00" + }, + "uncertainty": None, + "assumptions": [ + f"database_gb={row['database_gb']}", + f"stored_gb={row['stored_gb']}", + f"wal_gb={row['wal_gb']}", + f"restore_egress_gb={row['restore_egress_gb']}", + f"write_requests={row['write_requests']}", + f"read_requests={row['read_requests']}", + f"internal_labor_hours={row['internal_labor_hours']}" + ] + }) + return records + + +def validate_booked_cost(record: dict) -> dict: + required = { + "schema_version", "record_type", "financial_fact_id", "adjustment_kind", + "source_document_id", "source_line_id", "content_fingerprint", "provider", + "accounting_period", "currency", "gross_amount", "adjustment_amount", + "effective_amount", "source_evidence_ref", "recorded_at" + } + missing = required - record.keys() + if missing: + raise ValueError(f"booked-cost evidence missing {sorted(missing)}") + if record["schema_version"] != "0.1" or record["record_type"] != "booked_cost": + raise ValueError("unsupported booked-cost schema") + if len(record["currency"]) != 3 or record["currency"] != record["currency"].upper(): + raise ValueError("currency must be an uppercase three-letter code") + if money(record["effective_amount"]) != money(record["gross_amount"]) + money(record["adjustment_amount"]): + raise ValueError("effective_amount must equal gross plus adjustment") + if record.get("tax_status") == "unknown" and record.get("tax_amount") is not None: + raise ValueError("unknown tax must not have a tax amount") + return record + + +def reconcile(forecasts: list[dict], booked_costs: list[dict]) -> list[dict]: + planned = { + (record["period_start"][:7], record["currency"], record.get("cost_attribution_key")): + money(record["costs"]["infrastructure"]) + for record in forecasts if record["record_type"] == "forecast" + } + observed: dict[tuple[str, str, str | None], Decimal] = {} + fact_ids: dict[tuple[str, str, str | None], list[str]] = {} + for raw in booked_costs: + record = validate_booked_cost(raw) + key = (record["accounting_period"], record["currency"], record.get("cost_attribution_key")) + observed[key] = observed.get(key, Decimal("0")) + money(record["effective_amount"]) + fact_ids.setdefault(key, []).append(record["financial_fact_id"]) + rows = [] + for key in sorted(set(planned) | set(observed), key=lambda item: (item[0], item[1], item[2] or "")): + forecast = planned.get(key) + actual = observed.get(key) + rows.append({ + "period": key[0], "currency": key[1], "cost_attribution_key": key[2], + "forecast_infrastructure": None if forecast is None else money_text(forecast), + "booked_effective": None if actual is None else money_text(actual), + "variance": None if forecast is None or actual is None else money_text(actual - forecast), + "financial_fact_ids": fact_ids.get(key, []), + "status": "reconciled" if forecast is not None and actual is not None else "missing-booked-cost" if forecast is not None else "missing-forecast" + }) + return rows + + +def main() -> int: + if len(sys.argv) < 3 or sys.argv[1] not in {"forecast", "reconcile"}: + print(f"usage: {sys.argv[0]} forecast FORECAST.json [RESOURCE_ID] | reconcile FORECAST_EVIDENCE.json BOOKED_COST.json", file=sys.stderr) + return 2 + if sys.argv[1] == "forecast": + payload = json.loads(Path(sys.argv[2]).read_text()) + resource_id = sys.argv[3] if len(sys.argv) > 3 else "resource:platform_audit_storage" + print(json.dumps(forecast_records(payload, resource_id=resource_id, source_ref=sys.argv[2]), indent=2)) + return 0 + forecasts = json.loads(Path(sys.argv[2]).read_text()) + booked = json.loads(Path(sys.argv[3]).read_text()) + print(json.dumps(reconcile(forecasts, booked), indent=2)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/optimization.py b/tools/optimization.py new file mode 100644 index 0000000..7514103 --- /dev/null +++ b/tools/optimization.py @@ -0,0 +1,209 @@ +#!/usr/bin/env python3 +"""Evaluate resource optimization cases. + +The evaluator is fail-closed: any decision field that is unknown makes the +affected comparison unknown rather than optimistic. A case only reaches a +recommendation when every field the decision template requires is present for +the baseline and for the alternative being compared. +""" + +from __future__ import annotations + +import json +import sys +from pathlib import Path + +CASE_TYPES = { + "rightsizing", "consolidation", "commitment", "renewal", + "migration", "retirement", "provider_switch", +} +DECISION_STATES = { + "blocked_on_evidence", "proposed", "approved", "rejected", "superseded", +} +COST_FIELDS = ( + "recurring_infrastructure_eur_month", + "recurring_internal_labor_eur_month", + "recurring_external_labor_eur_month", +) +# Fields the decision template requires before any recommendation is made. +REQUIRED_FOR_DECISION = COST_FIELDS + ("one_time_eur", "exit_path") + +# A saving below this monthly threshold is not material enough to act on. +MATERIALITY_EUR_MONTH = 5.0 + + +def recurring_total(option: dict) -> float | None: + """Monthly recurring cost, or None when any component is unknown.""" + values = [option[field] for field in COST_FIELDS] + if any(value is None for value in values): + return None + return round(sum(values), 2) + + +def missing_fields(option: dict) -> list[str]: + missing = [field for field in REQUIRED_FOR_DECISION if option[field] is None] + if not option["utilization"]: + missing.append("utilization") + if not option["service_constraints"]: + missing.append("service_constraints") + return sorted(missing) + + +def utilization_ratios(option: dict) -> dict: + """Used-over-provisioned per metric; None where either side is unknown.""" + ratios = {} + for metric, pair in option["utilization"].items(): + provisioned = pair["provisioned"]["value"] + used = pair["used"]["value"] + if provisioned in (None, 0) or used is None: + ratios[metric] = None + else: + ratios[metric] = round(used / provisioned, 4) + return ratios + + +def compare_option(baseline: dict, alternative: dict) -> dict: + """Compare one alternative against the baseline on the decision fields.""" + blocking = sorted(set( + [f"baseline.{field}" for field in missing_fields(baseline)] + + [f"{alternative['option_id']}.{field}" for field in missing_fields(alternative)] + + [f"baseline.unknown:{item}" for item in baseline["unknowns"]] + + [f"{alternative['option_id']}.unknown:{item}" for item in alternative["unknowns"]] + )) + + base_recurring = recurring_total(baseline) + alt_recurring = recurring_total(alternative) + monthly_delta = None + monthly_saving = None + if base_recurring is not None and alt_recurring is not None: + monthly_delta = round(alt_recurring - base_recurring, 2) + monthly_saving = round(-monthly_delta, 2) + + one_time = alternative["one_time_eur"] + payback_months = None + payback_note = None + if monthly_saving is None or one_time is None: + payback_note = "unknown: incomplete cost evidence" + elif monthly_saving <= 0: + payback_note = "never: the alternative does not reduce recurring cost" + elif one_time == 0: + payback_months = 0.0 + payback_note = "immediate: no one-time cost" + else: + payback_months = round(one_time / monthly_saving, 1) + + # Failure domains the alternative removes, and ones it newly introduces. + base_domains = set(baseline["failure_domains"]) + alt_domains = set(alternative["failure_domains"]) + + if blocking: + verdict = "blocked_on_evidence" + elif monthly_saving is not None and monthly_saving > MATERIALITY_EUR_MONTH: + verdict = "recommend" + elif monthly_delta is not None and abs(monthly_delta) <= MATERIALITY_EUR_MONTH: + verdict = "no_material_change" + else: + verdict = "reject" + + return { + "option_id": alternative["option_id"], + "label": alternative["label"], + "verdict": verdict, + "baseline_recurring_eur_month": base_recurring, + "alternative_recurring_eur_month": alt_recurring, + "monthly_delta_eur": monthly_delta, + "monthly_saving_eur": monthly_saving, + "one_time_eur": one_time, + "payback_months": payback_months, + "payback_note": payback_note, + "baseline_utilization": utilization_ratios(baseline), + "alternative_utilization": utilization_ratios(alternative), + "uncertainty": alternative["uncertainty"]["level"], + "failure_domains_removed": sorted(base_domains - alt_domains), + "failure_domains_added": sorted(alt_domains - base_domains), + "exit_path_known": alternative["exit_path"] is not None, + "blocking_evidence": blocking, + } + + +def validate_case(case: dict) -> None: + if case["schema_version"] != "0.1": + raise ValueError("unsupported optimization-case schema version") + if case["record_scope"] not in {"operational", "illustrative"}: + raise ValueError("record_scope must be operational or illustrative") + if case["case_type"] not in CASE_TYPES: + raise ValueError(f"unknown case_type {case['case_type']}") + if not case["case_id"].startswith("opt:"): + raise ValueError("case_id must start with 'opt:'") + if not case["resource_ids"]: + raise ValueError("a case must name at least one resource") + if any(not rid.startswith("resource:") for rid in case["resource_ids"]): + raise ValueError("resource_ids must be portfolio resource identifiers") + if not case["evidence"]: + raise ValueError("a case must cite evidence") + + decision = case["decision"] + if decision["state"] not in DECISION_STATES: + raise ValueError(f"unknown decision state {decision['state']}") + + option_ids = [case["baseline"]["option_id"]] + [a["option_id"] for a in case["alternatives"]] + if len(set(option_ids)) != len(option_ids): + raise ValueError("option identifiers must be unique within a case") + + report = evaluate(case) + blocked = any(r["verdict"] == "blocked_on_evidence" for r in report["comparisons"]) + + # A case cannot be approved while its own evidence is incomplete, and an + # approval must name the authority that gave it. + if decision["state"] in {"approved", "rejected"}: + if decision["approver"] is None or decision["approved_on"] is None: + raise ValueError("a decided case must record approver and approved_on") + if decision["state"] == "approved": + if blocked: + raise ValueError("cannot approve a case with blocking evidence gaps") + if decision["recommended_option_id"] not in option_ids: + raise ValueError("approved case must recommend a known option") + if decision["state"] == "blocked_on_evidence" and not blocked: + raise ValueError("case is marked blocked but every decision field is known") + if decision["state"] == "proposed" and blocked: + raise ValueError("case has blocking evidence gaps and cannot be proposed") + + +def evaluate(case: dict) -> dict: + baseline = case["baseline"] + comparisons = [compare_option(baseline, alt) for alt in case["alternatives"]] + recommended = [c for c in comparisons if c["verdict"] == "recommend"] + recommended.sort(key=lambda c: (c["payback_months"] is None, c["payback_months"])) + return { + "case_id": case["case_id"], + "case_type": case["case_type"], + "record_scope": case["record_scope"], + "review_period": case["review_period"], + "baseline": { + "option_id": baseline["option_id"], + "label": baseline["label"], + "recurring_eur_month": recurring_total(baseline), + "utilization": utilization_ratios(baseline), + }, + "comparisons": comparisons, + "best_option_id": recommended[0]["option_id"] if recommended else None, + "decision_state": case["decision"]["state"], + } + + +def main() -> int: + paths = sys.argv[1:] or sorted(str(p) for p in Path("data/optimization").glob("*.json")) + if not paths: + print("no optimization cases found", file=sys.stderr) + return 2 + reports = [] + for path in paths: + case = json.loads(Path(path).read_text()) + validate_case(case) + reports.append(evaluate(case)) + print(json.dumps(reports, indent=2)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/portfolio.py b/tools/portfolio.py new file mode 100644 index 0000000..ea81c44 --- /dev/null +++ b/tools/portfolio.py @@ -0,0 +1,97 @@ +#!/usr/bin/env python3 +"""Semantic validation for managed-infrastructure portfolio records.""" + +from __future__ import annotations + +import json +import re +import sys +from datetime import date +from pathlib import Path + +RESOURCE_ID = re.compile(r"^resource:[a-z0-9][a-z0-9:_-]+$") +STATUSES = { + "proposed", "ordered", "commissioning", "active", "suspended", + "retiring", "retired", "rejected", +} +TRANSITIONS = { + "proposed": {"ordered", "rejected"}, + "ordered": {"commissioning", "rejected"}, + "commissioning": {"active", "rejected"}, + "active": {"suspended", "retiring"}, + "suspended": {"active", "retiring"}, + "retiring": {"retired", "active"}, + "retired": set(), + "rejected": {"proposed"}, +} + + +def _day(value: str | None) -> date | None: + return date.fromisoformat(value) if value else None + + +def validate_transition(current: str, target: str) -> None: + if current not in STATUSES or target not in STATUSES: + raise ValueError("unknown lifecycle status") + if target not in TRANSITIONS[current]: + raise ValueError(f"invalid lifecycle transition {current} -> {target}") + + +def validate_record(record: dict) -> None: + if record.get("schema_version") != "0.2": + raise ValueError("portfolio records must use schema_version 0.2") + if not RESOURCE_ID.fullmatch(record.get("id", "")): + raise ValueError("invalid resource id") + if record.get("status") not in STATUSES: + raise ValueError("unknown lifecycle status") + if record.get("record_scope") not in {"inventory", "example"}: + raise ValueError("record_scope must be inventory or example") + + allocation = record["ownership"]["allocation"] + if allocation["mode"] == "unattributed": + if allocation["cost_attribution_key"] is not None: + raise ValueError("unattributed resources cannot have an attribution key") + elif not allocation["cost_attribution_key"]: + raise ValueError("dedicated/shared resources require an attribution key") + if allocation["mode"] == "shared" and ( + not allocation["driver"] or not allocation["method_version"] + ): + raise ValueError("shared resources require a driver and method version") + + dimensions = [(item["metric"], item["kind"]) for item in record["capacity"]] + if len(dimensions) != len(set(dimensions)): + raise ValueError("capacity metric/kind pairs must be unique") + if any(rel["resource_id"] == record["id"] for rel in record["relationships"]): + raise ValueError("a resource cannot relate to itself") + + lifecycle = record["lifecycle"] + proposed = _day(lifecycle["proposed_on"]) + ordered = _day(lifecycle["ordered_on"]) + commissioned = _day(lifecycle["commissioned_on"]) + retired = _day(lifecycle["retired_on"]) + dated = [value for value in (proposed, ordered, commissioned, retired) if value] + if dated != sorted(dated): + raise ValueError("lifecycle dates are out of order") + if record["status"] == "retired" and retired is None: + raise ValueError("retired resources require retired_on") + # Discovery must preserve unknown commercial and commissioning dates as + # null instead of manufacturing precision. Date ordering is enforced when + # the authoritative repositories or provider evidence supply the values. + if record["record_scope"] == "example" and not any( + item["kind"] == "example" for item in record["evidence"] + ): + raise ValueError("examples require explicit example evidence") + + +def main() -> int: + if len(sys.argv) < 2: + print(f"usage: {sys.argv[0]} RECORD.json ...", file=sys.stderr) + return 2 + for name in sys.argv[1:]: + validate_record(json.loads(Path(name).read_text())) + print(f"portfolio records valid: {len(sys.argv) - 1}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/portfolio_report.py b/tools/portfolio_report.py new file mode 100644 index 0000000..8d8ac50 --- /dev/null +++ b/tools/portfolio_report.py @@ -0,0 +1,292 @@ +#!/usr/bin/env python3 +"""Portfolio view: coverage, lifecycle, capacity, cost, renewals, risk, and openings. + +The report never fills a gap with a guess. Every section separates what is known +from what is unknown, names the owner of each unknown, and reports unattributed +cost as its own figure rather than spreading it across resources. +""" + +from __future__ import annotations + +import json +import sys +from datetime import date +from pathlib import Path + +from optimization import evaluate, validate_case +from portfolio import validate_record + +# How far ahead a contract date counts as "approaching". +RENEWAL_HORIZON_DAYS = 90 +# Utilization at or below this fraction is flagged as idle capacity. +IDLE_THRESHOLD = 0.35 +# Utilization at or above this fraction is flagged as saturated. +SATURATED_THRESHOLD = 0.85 + +USAGE_PAIRS = { + "cpu": "cpu_usage", + "memory": "memory_usage", + "root_filesystem": "root_filesystem_used", +} + + +def _load_all(pattern: Path) -> list[dict]: + return [json.loads(path.read_text()) for path in sorted(pattern.parent.glob(pattern.name))] + + +def load_portfolio(root: Path) -> dict: + resources = _load_all(root / "data" / "resources" / "*.json") + for resource in resources: + validate_record(resource) + + coverage_files = sorted((root / "data").glob("portfolio-coverage-*.json")) + coverage = json.loads(coverage_files[-1].read_text()) if coverage_files else None + + cases = _load_all(root / "data" / "optimization" / "*.json") + for case in cases: + validate_case(case) + + return {"resources": resources, "coverage": coverage, "cases": cases} + + +def coverage_section(coverage: dict | None) -> dict: + if coverage is None: + return {"observed_at": None, "groups": [], "unresolved_gaps": [], "note": "no coverage observation recorded"} + return { + "observed_at": coverage["observed_at"], + "inventory_records": coverage["inventory_records"], + "groups": [ + {"group": group["group"], "status": group["status"], "resources": len(group["resource_ids"])} + for group in coverage["coverage"] + ], + "unresolved_gaps": [ + {"owner": gap["owner"], "gap": gap["gap"], "delegated_workplan": gap["delegated_workplan"]} + for gap in coverage["owned_gaps"] + ], + } + + +def lifecycle_section(resources: list[dict]) -> dict: + counts: dict[str, int] = {} + for resource in resources: + counts[resource["status"]] = counts.get(resource["status"], 0) + 1 + unowned = [r["id"] for r in resources if not r["ownership"]["owner"]] + return {"by_status": dict(sorted(counts.items())), "without_owner": unowned} + + +def _capacity_index(resource: dict) -> dict: + return {entry["metric"]: entry for entry in resource["capacity"]} + + +def utilization_section(resources: list[dict]) -> dict: + measured, unmeasured = [], [] + for resource in resources: + metrics = _capacity_index(resource) + rows = [] + for metric, usage_metric in USAGE_PAIRS.items(): + if metric not in metrics or usage_metric not in metrics: + continue + provisioned = metrics[metric]["value"] + used = metrics[usage_metric]["value"] + if not provisioned: + continue + ratio = round(used / provisioned, 4) + rows.append({ + "metric": metric, + "provisioned": provisioned, + "used": used, + "unit": metrics[metric]["unit"], + "ratio": ratio, + "observed_at": metrics[metric].get("observed_at"), + "signal": ( + "idle" if ratio <= IDLE_THRESHOLD + else "saturated" if ratio >= SATURATED_THRESHOLD + else "normal" + ), + }) + if rows: + measured.append({"resource_id": resource["id"], "metrics": rows}) + else: + unmeasured.append({ + "resource_id": resource["id"], + "owner": resource["ownership"]["owner"], + "reason": "no paired provisioned and observed capacity metric", + }) + return { + "measured": measured, + "unmeasured": unmeasured, + "idle": sorted({ + row["resource_id"] for row in measured + for metric in row["metrics"] if metric["signal"] == "idle" + }), + "saturated": sorted({ + row["resource_id"] for row in measured + for metric in row["metrics"] if metric["signal"] == "saturated" + }), + } + + +def cost_section(resources: list[dict]) -> dict: + priced, unpriced, unattributed = [], [], [] + for resource in resources: + entry = { + "resource_id": resource["id"], + "owner": resource["ownership"]["owner"], + "billing_model": resource["cost"]["billing_model"], + } + if resource["cost"]["price_evidence"]: + priced.append({**entry, "price_evidence": resource["cost"]["price_evidence"]}) + else: + unpriced.append(entry) + if resource["ownership"]["allocation"]["mode"] == "unattributed": + unattributed.append(entry) + return { + "priced": priced, + "unpriced": unpriced, + "unattributed_allocation": unattributed, + # Deliberately not a number: no booked cost has reached this repository, + # so any portfolio spend total would be invented rather than measured. + "known_monthly_spend_eur": None, + "spend_note": ( + f"{len(priced)} of {len(resources)} resources carry price evidence and none carries a booked " + "cost from fin-hub, so total portfolio spend is not computable and is reported as unknown " + "rather than as zero." + ), + } + + +def renewals_section(resources: list[dict], today: date) -> dict: + approaching, undated = [], [] + for resource in resources: + lifecycle = resource["lifecycle"] + dates = {k: lifecycle[k] for k in ("renews_on", "cancel_by") if lifecycle[k]} + if not dates: + if resource["status"] in {"active", "ordered", "commissioning"}: + undated.append({ + "resource_id": resource["id"], + "owner": resource["ownership"]["owner"], + "risk": "no renewal or cancellation date recorded; the cancellation window cannot be respected", + }) + continue + for field, value in dates.items(): + days = (date.fromisoformat(value) - today).days + if days <= RENEWAL_HORIZON_DAYS: + approaching.append({ + "resource_id": resource["id"], + "field": field, + "date": value, + "days_remaining": days, + }) + approaching.sort(key=lambda row: row["days_remaining"]) + return {"horizon_days": RENEWAL_HORIZON_DAYS, "approaching": approaching, "undated": undated} + + +def risk_section(resources: list[dict], utilization: dict, cost: dict) -> list[dict]: + risks = [] + + domains: dict[str, list[str]] = {} + for resource in resources: + for domain in resource["location"]["failure_domains"]: + domains.setdefault(domain, []).append(resource["id"]) + for domain, members in sorted(domains.items()): + if len(members) > 2: + risks.append({ + "kind": "concentrated_failure_domain", + "detail": f"{len(members)} resources share {domain}", + "resource_ids": sorted(members), + }) + + if cost["unpriced"]: + risks.append({ + "kind": "unpriced_resources", + "detail": f"{len(cost['unpriced'])} resources have no price evidence, so spend cannot be measured", + "resource_ids": sorted(row["resource_id"] for row in cost["unpriced"]), + }) + if cost["unattributed_allocation"]: + risks.append({ + "kind": "unattributed_cost", + "detail": f"{len(cost['unattributed_allocation'])} resources have no allocation method, so their cost reaches no consumer", + "resource_ids": sorted(row["resource_id"] for row in cost["unattributed_allocation"]), + }) + if utilization["idle"]: + risks.append({ + "kind": "idle_capacity", + "detail": f"utilization at or below {IDLE_THRESHOLD:.0%} of provisioned capacity", + "resource_ids": utilization["idle"], + }) + if utilization["saturated"]: + risks.append({ + "kind": "saturated_capacity", + "detail": f"utilization at or above {SATURATED_THRESHOLD:.0%} of provisioned capacity", + "resource_ids": utilization["saturated"], + }) + return risks + + +def optimization_section(cases: list[dict]) -> dict: + open_cases, blocked_on = [], [] + for case in cases: + report = evaluate(case) + open_cases.append({ + "case_id": report["case_id"], + "case_type": report["case_type"], + "state": report["decision_state"], + "best_option_id": report["best_option_id"], + "verdicts": {c["option_id"]: c["verdict"] for c in report["comparisons"]}, + }) + for comparison in report["comparisons"]: + for item in comparison["blocking_evidence"]: + if ".unknown:" in item: + blocked_on.append({"case_id": report["case_id"], "missing": item.split(".unknown:", 1)[1]}) + return { + "cases": open_cases, + "undecided": [c["case_id"] for c in open_cases if c["state"] in {"blocked_on_evidence", "proposed"}], + "blocked_on_evidence": blocked_on, + } + + +def next_actions(report: dict) -> list[str]: + """The smallest set of evidence that would unblock the most decisions.""" + actions = [] + for gap in report["coverage"]["unresolved_gaps"]: + actions.append(f"{gap['owner']}: deliver {gap['delegated_workplan']} — {gap['gap']}") + if report["cost"]["unpriced"]: + actions.append( + "resource-control: no portfolio spend figure exists until at least one booked cost arrives " + "from fin-hub under the exchange contract" + ) + for row in report["renewals"]["undated"]: + actions.append(f"{row['owner']}: record renewal and cancellation dates for {row['resource_id']}") + return actions + + +def build(root: Path, today: date | None = None) -> dict: + today = today or date.today() + data = load_portfolio(root) + resources = data["resources"] + utilization = utilization_section(resources) + cost = cost_section(resources) + report = { + "schema_version": "0.1", + "generated_on": today.isoformat(), + "resource_count": len(resources), + "coverage": coverage_section(data["coverage"]), + "lifecycle": lifecycle_section(resources), + "utilization": utilization, + "cost": cost, + "renewals": renewals_section(resources, today), + "optimization": optimization_section(data["cases"]), + } + report["risks"] = risk_section(resources, utilization, cost) + report["next_actions"] = next_actions(report) + return report + + +def main() -> int: + root = Path(sys.argv[1]) if len(sys.argv) > 1 else Path.cwd() + print(json.dumps(build(root), indent=2)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/validate.py b/tools/validate.py new file mode 100644 index 0000000..e013a4f --- /dev/null +++ b/tools/validate.py @@ -0,0 +1,79 @@ +#!/usr/bin/env python3 +"""Dependency-free validation for resource-control JSON declarations.""" + +import json +from pathlib import Path + +from optimization import validate_case +from portfolio import validate_record +from portfolio_report import build as build_portfolio_report + + +def load(path: str) -> dict: + return json.loads(Path(path).read_text()) + + +def main() -> int: + demand = load("data/demand/platform-audit-storage.json") + providers = load("data/providers/object-storage.json") + schema = load("schemas/resource-inventory.schema.json") + observation_schema = load("schemas/monthly-resource-observation.schema.json") + planning_schema = load("schemas/planning-evidence.schema.json") + control_schema = load("schemas/resource-control-cycle.schema.json") + forecasts = [load(str(path)) for path in Path("data/forecasts").glob("*.json")] + resource_paths = list(Path("data/resources").glob("*.json")) + resource_paths += list(Path("examples/portfolio").glob("*.json")) + resources = [load(str(path)) for path in resource_paths] + control_records = [load(str(path)) for path in Path("examples/control-cycle").glob("*.json")] + assert demand["schema_version"] == providers["schema_version"] == "0.1" + assert demand["retention_days"] >= 30 + assert set(demand["scenarios"]) == {"low", "base", "high"} + assert len({p["id"] for p in providers["providers"]}) == len(providers["providers"]) + assert {"Host Europe", "Scaleway", "Hetzner", "AWS", "Microsoft Azure", "Google Cloud", "STACKIT"} <= {p["provider"] for p in providers["providers"]} + assert schema["$schema"].endswith("2020-12/schema") + assert observation_schema["$schema"].endswith("2020-12/schema") + assert planning_schema["$schema"].endswith("2020-12/schema") + assert control_schema["$schema"].endswith("2020-12/schema") + assert len(planning_schema["oneOf"]) == 5 + for forecast in forecasts: + assert forecast["record_type"] == "forecast" + assert len({row["period"] for row in forecast["rows"]}) == len(forecast["rows"]) + assert all(row["total_eur"] == round(row["infrastructure_eur"] + row["internal_labor_eur"], 2) for row in forecast["rows"]) + assert schema["properties"]["schema_version"]["const"] == "0.2" + required = set(schema["required"]) + for resource in resources: + assert not required - resource.keys(), f"missing fields: {required - resource.keys()}" + validate_record(resource) + record_ids = {record["record_id"] for record in control_records} + assert len(record_ids) == len(control_records) + assert {record["resource_class"] for record in control_records} == {"storage", "cluster_compute", "shared_platform_service"} + for record in control_records: + assert record["schema_version"] == "0.1" + assert record["resource_id"].startswith("resource:") + costs = record["costs"] + assert costs["total"] == round(costs["infrastructure"] + costs["internal_labor"] + costs["external_labor"], 2) + if record["record_type"] == "actual": + assert record["forecast_ref"] in record_ids + case_schema = load("schemas/optimization-case.schema.json") + assert case_schema["$schema"].endswith("2020-12/schema") + assert case_schema["properties"]["schema_version"]["const"] == "0.1" + cases = [load(str(path)) for path in Path("data/optimization").glob("*.json")] + assert len({case["case_id"] for case in cases}) == len(cases) + for case in cases: + assert not set(case_schema["required"]) - case.keys() + validate_case(case) + # The optimization process must be validated on the backup case and on at + # least one non-storage portfolio candidate (RESOURCE-WP-0003-T06). + case_resources = {rid for case in cases for rid in case["resource_ids"]} + assert "resource:platform:audit-storage" in case_resources + assert case_resources - {"resource:platform:audit-storage"} + report = build_portfolio_report(Path(".")) + assert report["resource_count"] == len(resource_paths) - len(list(Path("examples/portfolio").glob("*.json"))) + assert report["cost"]["known_monthly_spend_eur"] is None + assert report["next_actions"] + print("resource-control declarations: valid") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/variance.py b/tools/variance.py new file mode 100644 index 0000000..809f56a --- /dev/null +++ b/tools/variance.py @@ -0,0 +1,55 @@ +#!/usr/bin/env python3 +"""Compare monthly resource actuals with the immutable decision forecast.""" + +from __future__ import annotations + +import json +import sys +from pathlib import Path + +METRICS = ( + "database_gb", "stored_gb", "wal_gb", "restore_egress_gb", + "write_requests", "read_requests", "infrastructure_eur", + "internal_labor_hours", "internal_labor_eur", "total_eur", +) + + +def compare(forecast: dict, actual: dict) -> dict: + expected = {row["period"]: row for row in forecast["rows"]} + rows = [] + for observed in actual["rows"]: + period = observed["period"] + if period not in expected: + rows.append({"period": period, "status": "no-forecast", "metrics": {}}) + continue + metrics = {} + for metric in METRICS: + planned = expected[period][metric] + measured = observed[metric] + error = measured - planned + metrics[metric] = { + "forecast": planned, + "actual": measured, + "error": round(error, 4), + "absolute_percentage_error": None if planned == 0 else round(abs(error) / planned * 100, 2), + } + rows.append({"period": period, "status": "compared", "metrics": metrics}) + return { + "forecast_created_at": forecast["created_at"], + "provider_id": actual["provider_id"], + "rows": rows, + } + + +def main() -> int: + if len(sys.argv) != 3: + print(f"usage: {sys.argv[0]} FORECAST.json ACTUAL.json", file=sys.stderr) + return 2 + forecast = json.loads(Path(sys.argv[1]).read_text()) + actual = json.loads(Path(sys.argv[2]).read_text()) + print(json.dumps(compare(forecast, actual), indent=2)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/workplans/RESOURCE-WP-0002-procure-postgres-backup-storage.md b/workplans/RESOURCE-WP-0002-procure-postgres-backup-storage.md index 8a19e14..cbcf5ad 100644 --- a/workplans/RESOURCE-WP-0002-procure-postgres-backup-storage.md +++ b/workplans/RESOURCE-WP-0002-procure-postgres-backup-storage.md @@ -104,7 +104,7 @@ with an explicit correlated-failure decision and an independent second copy. ```task id: RESOURCE-WP-0002-T01 -status: todo +status: progress priority: high state_hub_task_id: "f578a9ec-dbdb-4b26-93bc-e53f7bc87ad1" ``` @@ -121,11 +121,38 @@ Do not compare only advertised price per GB. Done when the demand forecast and cost model can calculate effective monthly cost for A, B, and C under the same scenarios. +Progress 2026-08-10: added a timestamped live database/WAL observation, +low/base/high 12-month inputs, and a tested fail-closed calculator in +`data/`, `tools/cost_model.py`, and +`docs/evidence/RESOURCE-WP-0002-demand-and-cost-model-2026-08-10.md`. +Scaleway and in-quota Hetzner costs calculate. Host Europe remains deliberately +`null` until a current account quote exists; Hetzner excess pricing is also +required before its high scenario and full exit can calculate. + +Expanded 2026-08-10: added 1/2/3-node self-managed Garage estimates for Host +Europe and Hetzner plus managed AWS S3, Azure Blob, Google Cloud Storage, and +STACKIT Object Storage price points. Fixed-capacity VM options fail closed when +forecast demand exceeds usable replicated capacity. Currency conversion, +request charges, setup labor, recurring operations, and topology limitations +are explicit in +`docs/evidence/RESOURCE-WP-0002-expanded-storage-comparison-2026-08-10.md`. + +Refined 2026-08-10: every quote now separates monthly infrastructure from +internal operations labor and internal setup from external services. Added a +normalized 320 GB running-cost view so bundled, fixed-capacity, and elastic +products compare on the same stored volume and request/restore pattern. + +Control refinement 2026-08-10: added an immutable 12-month forecast record, +monthly actual-observation schema, and variance calculator covering stored +bytes, database size, WAL, requests, restore egress, invoice cost, and internal +labor. `docs/forecast-actual-control.md` defines error thresholds, evidence, +revision discipline, and the three-month recalibration loop. + ## T02 — Complete provider due diligence ```task id: RESOURCE-WP-0002-T02 -status: todo +status: progress priority: high state_hub_task_id: "6dedb8db-c08f-4b58-90f2-32b1ea832d98" ``` @@ -144,6 +171,14 @@ object-lock behavior, and provider status visibility. Done when every acceptance requirement has evidence, `unknown`, or a blocking answer for each provider—no blank cells and no marketing inference. +Progress 2026-08-10: completed the public-primary-source evidence matrix in +`docs/evidence/RESOURCE-WP-0002-provider-due-diligence-2026-08-10.md` and added +a provisional Scaleway inventory record. Host Europe current S3 orderability +and commercial terms require written account-support evidence. Scaleway is the +provisional primary because it documents Multi-AZ service, durability, S3 +features, and managed at-rest encryption; selection still requires a live +Barman preflight, contract review, and human approval. + ## T03 — Select and procure the primary resource ```task diff --git a/workplans/RESOURCE-WP-0003-managed-infrastructure-portfolio-control.md b/workplans/RESOURCE-WP-0003-managed-infrastructure-portfolio-control.md new file mode 100644 index 0000000..68faea9 --- /dev/null +++ b/workplans/RESOURCE-WP-0003-managed-infrastructure-portfolio-control.md @@ -0,0 +1,298 @@ +--- +id: RESOURCE-WP-0003 +type: workplan +title: "Establish managed-infrastructure portfolio control" +domain: financials +repo: resource-control +status: finished +owner: codex +topic_slug: railiance +created: "2026-08-10" +updated: "2026-08-14" +related: + - RESOURCE-WP-0002 + - FIN-WP-0004 +state_hub_workstream_id: "b4d541d2-c114-4189-b431-e301c4dcef4e" +--- + +# RESOURCE-WP-0003 — managed-infrastructure portfolio control + +## Goal + +Turn the revised repository intent into an operational, provider-neutral +control loop for infrastructure serving Railiance itself, Helix Forge, Coulomb +Social, shared platform services, and tenant workloads. Establish the portfolio +model, authority contracts, delegated evidence interfaces, and a repeatable +forecast-to-actual optimization cycle without duplicating financial, workload, +or platform authority. + +The PostgreSQL backup work in `RESOURCE-WP-0002` is the first proving case and +input to this workplan, not the limit of its scope. + +## Boundaries + +- `resource-control` owns portfolio identity, lifecycle, technical economics, + allocation evidence, forecasts, variance analysis, and optimization cases. +- `fin-hub` owns authoritative booked financial facts, budgets, commitments, + burn, runway, and viability signals. +- Workload repositories own demand meaning and service requirements. +- Cluster and platform repositories own provisioning, operations, and source + telemetry. +- Human authorities approve contractual commitments and material lifecycle + changes. + +This workplan defines and validates interfaces. Cross-repository implementation +must be handed off as work records in the repository that owns the source. + +## T01 — Define the portfolio and lifecycle model + +```task +id: RESOURCE-WP-0003-T01 +status: done +priority: high +state_hub_task_id: "9c396da2-9cae-4681-8d7e-3ebcbf9db29f" +``` + +Define the minimum resource classes, lifecycle states, ownership fields, +capacity forms, provider/account identity, shared-versus-dedicated placement, +tenant and workload attribution, contract dates, and evidence provenance. + +Reuse the current backup inventory and forecast schemas where they generalize; +identify and migrate assumptions that are storage-specific. Include validation +rules for stable identifiers and lifecycle transitions. + +Done when a versioned schema and examples can represent provider-managed +object storage, self-managed Garage on VMs, cluster capacity, and a shared +platform service without ambiguous ownership or cost attribution. + +Completed 2026-08-11: introduced portfolio schema v0.2 with resource classes, +management models, capacity dimensions, lifecycle states, shared and dedicated +allocation, relationships, requirements, and provenance-bearing evidence. +Migrated the real proposed backup record and added explicitly non-authoritative +Garage, Kubernetes-capacity, and shared-ingress examples. Semantic validation +covers identifier, lifecycle, allocation, capacity, and example-scope rules. +Evidence: `docs/evidence/RESOURCE-WP-0003-portfolio-model-v0.2-2026-08-11.md`. + +## T02 — Record the authority and fin-hub exchange contract + +```task +id: RESOURCE-WP-0003-T02 +status: done +priority: high +state_hub_task_id: "efa64dce-7b5e-435e-b010-7aee9e0a410e" +``` + +Create an authority matrix and a versioned exchange contract with `fin-hub`. +Define the two directions separately: + +- resource-control to fin-hub: resource references, allocation keys, forecasts, + technical usage, commitment candidates, and optimization scenarios; +- fin-hub to resource-control: booked costs, credits, tax and currency treatment, + financial commitments, budget constraints, and viability signals. + +Specify period, currency, net/gross semantics, provenance, corrections, +idempotency, uncertainty, and identifiers including `resource_id`, `service_id`, +`workload_id`, `tenant_id`, `environment`, and `cost_attribution_key`. + +Done when both repositories link the same reviewed contract and no field has +two authoritative writers. Coordinate with `FIN-WP-0004`. + +Progress 2026-08-11: verified fin-hub commit `0034330` and its 46-test suite; +the six review remediations satisfy the authority, exact-money, typed-evidence, +idempotency, dimensional-reporting, and model-parity conditions. Added the +resource-control ratification, producer-owned planning schema, canonical +forecast exporter, booked-cost consumer validation, and reconciliation seam. +The 12-row backup forecast preflight and cross-repository compatibility check +remain before this task is complete. A real booked backup cost is deliberately +reserved for `FIN-WP-0004-T05` after procurement. + +Completed 2026-08-11: all 12 resource-control producer records matched +fin-hub's executable schema and adapter output, and duplicate ingestion retained +exactly 12 planning rows. Evidence is in +`docs/evidence/RESOURCE-WP-0003-fin-hub-contract-preflight-2026-08-11.md`. +The authority contract is accepted from resource-control. The separate real +booked-cost round trip remains gated on `RESOURCE-WP-0002` procurement and is +not simulated as operational evidence. + +## T03 — Inventory the first managed-infrastructure portfolio + +```task +id: RESOURCE-WP-0003-T03 +status: done +priority: high +state_hub_task_id: "4afbb2ea-8021-4dd7-968b-50c113a8c108" +``` + +Discover and register the material infrastructure serving Helix Forge, +Coulomb Social, shared Railiance services, and a representative tenant +workload. Record unknowns explicitly and separate observed resources from +inferred relationships. + +For each resource, identify the authoritative workload and platform repository, +owner, lifecycle state, capacity, service objective or requirement reference, +provider/account, failure domain, contract or renewal evidence, and allocation +key. Do not retrieve or record credentials. + +Done when portfolio coverage and evidence gaps are measurable and every +material discovered resource has an owner or a human-review record. + +Completed 2026-08-11: registered seven evidence-backed resources covering the +Host Europe host, reef-railiance k3s capacity, Forgejo/Helix Forge, shared +applications PostgreSQL, Coulomb Social, Binky's `rapp-qonto`, and proposed +backup storage. Live read-only observation confirmed capacity, current +workloads, PVCs, and the shared single-host/local-storage failure domain. +Unknown commercial, utilization-history, labor, and allocation facts remain +explicit and are assigned to authoritative repository owners in +`data/portfolio-coverage-2026-08-11.json`. Evidence: +`docs/evidence/RESOURCE-WP-0003-initial-portfolio-discovery-2026-08-11.md`. + +## T04 — Delegate demand and telemetry evidence + +```task +id: RESOURCE-WP-0003-T04 +status: done +priority: high +state_hub_task_id: "b4053889-69fd-4e3c-888b-0764eee2e9e4" +``` + +Create scoped work records in the authoritative workload and platform +repositories for missing demand declarations, service objectives, capacity, +utilization, reliability, and allocation-driver evidence. Initial targets are +`helix-forge`, `coulomb-social`, `railiance-cluster`, `rail-kubernetes`, +`rail-knative`, and `railiance-platform`; add only repositories supported by +the inventory evidence. + +Each handoff must define a stable non-secret interface and acceptance evidence, +not prescribe unrelated internal implementation. + +Done when all required source evidence is either available through a versioned +interface or represented by a live delegated work record with an owner. + +Completed 2026-08-11: created and registered repository-local evidence plans +`RAIL-HO-WP-0008`, `RAIL-BS-WP-0014`, railiance-forge +`RAILIANCE-WP-0002`, railiance-platform `RAILIANCE-WP-0016`, +`CSOC-WP-0005`, and `RAPP-QONTO-WP-0002`. The booked-cost boundary remains in +fin-hub and is linked to existing `FIN-WP-0004-T04..T06`. The exact State Hub +workplan identifiers are recorded in +`data/portfolio-coverage-2026-08-11.json`. No duplicate plan was created in +`helix-forge`, `rail-kubernetes`, or `rail-knative`: their relevant evidence is +owned respectively by railiance-forge, railiance-cluster, and the workload plus +cluster layers. + +## T05 — Generalize forecast-to-actual control + +```task +id: RESOURCE-WP-0003-T05 +status: done +priority: high +state_hub_task_id: "76986764-5e23-4e51-aaed-6bbb92d08915" +``` + +Generalize the immutable forecast and monthly actual-observation controls from +`RESOURCE-WP-0002`. Support resource-specific usage proxies, fixed and elastic +capacity, internal and external labor, infrastructure cost, booked cost +references, allocation methods, service-level constraints, and low/base/high +uncertainty. + +Classify variance as demand, provider price, allocation, labor, model, or +data-quality error. Preserve the original forecast and create explicit +revisions rather than overwriting it. + +Done when the same mechanism can evaluate at least storage, VM or cluster +compute, and one shared platform service. + +Completed 2026-08-11: added the generic immutable control-cycle schema and +comparator, controlled six-way variance attribution, revision and booked-fact +references, and paired contract examples for elastic storage, fixed cluster +compute, and the hybrid shared apps-pg service. The examples are illustrative, +not operational facts. Evidence: +`docs/evidence/RESOURCE-WP-0003-generalized-control-cycle-2026-08-11.md`. + +## T06 — Produce and govern optimization cases + +```task +id: RESOURCE-WP-0003-T06 +status: done +priority: medium +state_hub_task_id: "08d5a687-6ce4-4bbf-950a-9682b1f7b003" +``` + +Define the review cadence and decision template for rightsizing, +consolidation, commitment, renewal, migration, retirement, and provider +switching. Every recommendation must show baseline, alternative, one-time +cost, recurring infrastructure and labor cost, utilization, uncertainty, +service-level constraints, failure domains, exit path, and expected payback. + +Validate the process using the backup case and at least one non-storage +portfolio candidate. Send financial implications to `fin-hub`; delegate any +approved implementation to the owning platform or workload repository. + +Done when recommendations are reproducible from evidence and their eventual +outcomes feed the next forecast cycle. + +Completed 2026-08-14: added the optimization-case schema, a fail-closed +evaluator, and the cadence and decision template in `docs/optimization-cases.md`. +Every option, baseline included, must present all ten decision fields; one +unknown blocks the comparison. Validated on the storage case, where Hetzner +computes fully and loses to Scaleway by EUR 29.14 per month on operator labour +rather than storage price while Host Europe blocks on four named gaps, and on +the non-storage reef-railiance k3s rightsizing case, which reports genuinely low +utilization and still refuses to recommend because the railiance01 booked price +is unknown and the utilization figure is a single sample. Neither case decides: +procurement remains `RESOURCE-WP-0002-T03`, and the cluster unknowns are +delegated to `RAIL-HO-WP-0008` and `RAIL-BS-WP-0014`. Evidence: +`docs/evidence/RESOURCE-WP-0003-optimization-cases-2026-08-14.md`. + +## T07 — Establish portfolio reporting and operating cadence + +```task +id: RESOURCE-WP-0003-T07 +status: done +priority: medium +state_hub_task_id: "3927e658-6807-4737-ae15-ea1997dc210f" +``` + +Publish a portfolio view covering coverage, lifecycle, capacity, utilization, +forecast and actual variance, unattributed cost, renewals, risks, and open +optimization opportunities. Report missing evidence and unapportioned cost +rather than forcing false precision. + +Define monthly observation, quarterly model calibration, and pre-renewal review +cadences with named inputs and consumers. + +Done when an operator can identify material spend, idle or saturated capacity, +forecast error, approaching commitments, and the next evidence-backed action. + +Completed 2026-08-14: added `tools/portfolio_report.py` and +`make portfolio-report`, rendering coverage, lifecycle, utilization, cost, +renewals, risks, open cases, and next actions from committed evidence only, plus +the cadence in `docs/portfolio-operating-cadence.md`. The report refuses false +precision: portfolio spend is `null` rather than a partial sum, unattributed +cost is a named list rather than a spread, and unmeasurable resources are +reported instead of dropped. Against the current portfolio it reports spend as +unknown, railiance01 and the k3s cluster as idle at 14 percent CPU and 37 percent +memory, no computable forecast error, no dated commitments but six active +resources with no contract dates at all, and six of seven resources sharing +`host:railiance01`. Evidence: +`docs/evidence/RESOURCE-WP-0003-portfolio-reporting-2026-08-14.md`. + +## Acceptance + +- [x] Portfolio schema and lifecycle rules cover the initial resource classes. +- [x] The fin-hub authority and exchange contract is jointly reviewed. +- [x] Helix Forge, Coulomb Social, shared Railiance, and tenant coverage is + measured. +- [x] Missing source evidence has live delegated work records. +- [x] Forecast-to-actual control works for storage and non-storage resources. +- [x] At least two optimization cases complete the evidence-to-decision loop. +- [x] Portfolio reporting exposes unknown and unattributed values explicitly. + +## Standing gate + +Both optimization cases are `blocked_on_evidence` by design: the mechanism is +complete and proven, and the facts it needs are owed by other repositories. +The first *decided* case requires either the Host Europe account evidence that +`RESOURCE-WP-0002-T03` and `RAIL-HO-WP-0008` are waiting on, or the cluster +utilization history in `RAIL-BS-WP-0014`. The first real forecast-to-actual +variance requires a booked cost from `fin-hub` under `FIN-WP-0004`. None of +these is a gap in this workplan; each is a live delegated record with an owner.