feat: join audit-storage usage without inventing a booked Scaleway fact
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s

Ingest resource-control's August technical usage, omit nulls, and
reconcile forecast + usage + booked facts. Commissioning month reports
missing_booked_fact instead of 0.00. T05 still waits on the first invoice.
This commit is contained in:
tegwick 2026-08-15 20:32:55 +02:00
parent 75356ea19e
commit ac0949afd0
7 changed files with 426 additions and 1 deletions

View file

@ -26,6 +26,8 @@ uv run finhub import-ai-plan tests/fixtures/ai-plans.csv
uv run finhub ledger set-balance 12000 uv run finhub ledger set-balance 12000
uv run finhub ledger import cloud tests/fixtures/cloud-costs.csv uv run finhub ledger import cloud tests/fixtures/cloud-costs.csv
uv run finhub ledger import ai-plan tests/fixtures/ai-plans.csv uv run finhub ledger import ai-plan tests/fixtures/ai-plans.csv
uv run finhub ledger ingest-usage --resource resource:platform:audit-storage --source resource-control:data/actuals/2026-08.json tests/fixtures/platform-audit-storage-usage-2026-08.json
uv run finhub ledger reconcile-resource --resource resource:platform:audit-storage --period 2026-08
uv run finhub ledger commitments uv run finhub ledger commitments
uv run finhub ledger set-entitlement --provider anthropic --plan claude-max --period 2026-08 --unit plan --plan-label "Max 20x" --source vendor-plan uv run finhub ledger set-entitlement --provider anthropic --plan claude-max --period 2026-08 --unit plan --plan-label "Max 20x" --source vendor-plan
uv run finhub ledger plan-month --period 2026-08 uv run finhub ledger plan-month --period 2026-08

View file

@ -0,0 +1,45 @@
# FIN-WP-0004-T05 — commissioning-period join, 2026-08
Date: 2026-08-15
Resource: `resource:platform:audit-storage`
Attribution key: `platform:audit-storage`
## What exists
| Record | Count | Authority |
| --- | --- | --- |
| Forecast (base, 2026-09 … 2027-08) | 12 rows | resource-control |
| Technical usage 2026-08 | 1 row | resource-control |
| Booked Scaleway fact 2026-08 | **0** | fin-hub |
The object store was commissioned 2026-08-14. resource-controls usage
record carries `infrastructure_eur: null` and names fin-hub as the
owner of that gap. There is no invoice. A null is not booked as 0.00.
## Join result
`ingest_resource_usage` stores only measured quantities (`database_gb`,
`stored_gb`, `backup_success_pct`, `restore_rto_minutes`). Null usage
fields and the missing infrastructure amount remain gaps in
`source_evidence`.
`reconcile_resource_period` for 2026-08 reports:
- usage present
- `missing_booked_fact: true`
- `booked_effective_amount: null`
- `invented_zero: false`
`exchange_health` raises `usage_without_booked_fact` for this period.
## Why T05 stays open
The done criterion is: the same cost booked once, projected,
joined to technical evidence, and returned without a second ledger.
That cost does not exist yet. The first comparable Scaleway invoice
period is 2026-09. This note is the commissioning baseline, not
closure.
When a provider charge or credit lands, book it once, bind
`financial_fact_id``resource:platform:audit-storage`, and rerun
`ledger reconcile-resource --period 2026-09`.

View file

@ -20,6 +20,11 @@ from fin_hub.services.allocation import shared_cost_allocations
from fin_hub.services.billing import build_billing_basis from fin_hub.services.billing import build_billing_basis
from fin_hub.services.effectiveness import effectiveness_report from fin_hub.services.effectiveness import effectiveness_report
from fin_hub.services.evaluate import evaluate_runway from fin_hub.services.evaluate import evaluate_runway
from fin_hub.services.exchange import (
ingest_resource_forecast,
ingest_resource_usage,
reconcile_resource_period,
)
from fin_hub.services.reporting_allocation import ( from fin_hub.services.reporting_allocation import (
allocate_ai_plan_report, allocate_ai_plan_report,
list_ai_plan_allocations, list_ai_plan_allocations,
@ -264,6 +269,40 @@ def _cmd_ledger_effectiveness(args: argparse.Namespace) -> int:
return 0 return 0
def _cmd_ledger_ingest_forecast(args: argparse.Namespace) -> int:
payload = json.loads(Path(args.path).read_text(encoding="utf-8"))
records = ingest_resource_forecast(
payload,
resource_id=args.resource,
source_evidence_ref=args.source,
ledger_path=_ledger_path(args),
)
print(json.dumps({"ingested": len(records)}, indent=2))
return 0
def _cmd_ledger_ingest_usage(args: argparse.Namespace) -> int:
payload = json.loads(Path(args.path).read_text(encoding="utf-8"))
records = ingest_resource_usage(
payload,
resource_id=args.resource,
source_evidence_ref=args.source,
ledger_path=_ledger_path(args),
)
print(json.dumps({"ingested": len(records)}, indent=2))
return 0
def _cmd_ledger_reconcile_resource(args: argparse.Namespace) -> int:
report = reconcile_resource_period(
resource_id=args.resource,
period=args.period,
ledger_path=_ledger_path(args),
)
print(json.dumps(report.as_dict(), indent=2))
return 0
def _cmd_ledger_allocations(args: argparse.Namespace) -> int: def _cmd_ledger_allocations(args: argparse.Namespace) -> int:
reports = shared_cost_allocations(ledger_path=_ledger_path(args)) reports = shared_cost_allocations(ledger_path=_ledger_path(args))
print(json.dumps([report.as_dict() for report in reports], indent=2, default=str)) print(json.dumps([report.as_dict() for report in reports], indent=2, default=str))
@ -364,6 +403,35 @@ def build_parser() -> argparse.ArgumentParser:
ledger_import.add_argument("--force", action="store_true", help="Re-import even if file unchanged") ledger_import.add_argument("--force", action="store_true", help="Re-import even if file unchanged")
ledger_import.set_defaults(func=_cmd_ledger_import) ledger_import.set_defaults(func=_cmd_ledger_import)
ledger_ingest_forecast = ledger_sub.add_parser(
"ingest-forecast",
help="Ingest a resource-control monthly forecast as planning evidence",
)
ledger_ingest_forecast.add_argument("path")
ledger_ingest_forecast.add_argument("--resource", required=True)
ledger_ingest_forecast.add_argument("--source", required=True)
ledger_ingest_forecast.add_argument("--ledger", help="Ledger database path")
ledger_ingest_forecast.set_defaults(func=_cmd_ledger_ingest_forecast)
ledger_ingest_usage = ledger_sub.add_parser(
"ingest-usage",
help="Ingest a resource-control usage observation (nulls stay omitted)",
)
ledger_ingest_usage.add_argument("path")
ledger_ingest_usage.add_argument("--resource", required=True)
ledger_ingest_usage.add_argument("--source", required=True)
ledger_ingest_usage.add_argument("--ledger", help="Ledger database path")
ledger_ingest_usage.set_defaults(func=_cmd_ledger_ingest_usage)
ledger_reconcile = ledger_sub.add_parser(
"reconcile-resource",
help="Join forecast, usage, and booked facts for one resource period",
)
ledger_reconcile.add_argument("--resource", required=True)
ledger_reconcile.add_argument("--period", required=True, help="YYYY-MM")
ledger_reconcile.add_argument("--ledger", help="Ledger database path")
ledger_reconcile.set_defaults(func=_cmd_ledger_reconcile_resource)
ledger_summary = ledger_sub.add_parser("summary", help="Monthly rollup summary from ledger") ledger_summary = ledger_sub.add_parser("summary", help="Monthly rollup summary from ledger")
ledger_summary.add_argument("--ledger", help="Ledger database path (default: .fin-hub/ledger.db)") ledger_summary.add_argument("--ledger", help="Ledger database path (default: .fin-hub/ledger.db)")
ledger_summary.set_defaults(func=_cmd_ledger_summary) ledger_summary.set_defaults(func=_cmd_ledger_summary)

View file

@ -438,6 +438,31 @@ def exchange_health(
) )
) )
for row in planning_rows:
if row["record_type"] != "usage_observation":
continue
payload = json.loads(row["payload_json"])
usage_resource = payload.get("resource_id")
if not usage_resource:
continue
period = payload["period_start"][:7]
if any(
fact.resource_id == usage_resource and fact.accounting_period == period
for fact in booked
):
continue
issues.append(
ExchangeQualityIssue(
code="usage_without_booked_fact",
severity="warning",
record_id=row["record_id"],
detail=(
f"{usage_resource} {period} has technical usage and no booked "
"fact; absence is not zero spend"
),
)
)
for row in rejection_rows: for row in rejection_rows:
issues.append( issues.append(
ExchangeQualityIssue( ExchangeQualityIssue(
@ -525,3 +550,174 @@ def ingest_resource_forecast(
) )
records.append(record) records.append(record)
return records return records
_USAGE_MEASURES = (
("database_gb", "GB"),
("stored_gb", "GB"),
("wal_gb", "GB"),
("restore_egress_gb", "GB"),
("write_requests", "count"),
("read_requests", "count"),
("backup_success_pct", "percent"),
("restore_rto_minutes", "minutes"),
)
def ingest_resource_usage(
payload: dict,
*,
resource_id: str,
source_evidence_ref: str,
ledger_path: Path | None = None,
) -> list[PlanningEvidence]:
"""Adapt resource-control monthly usage. Nulls stay omitted, never zero."""
if payload.get("record_type") != "usage_observation":
raise ValueError("resource usage payload must have record_type=usage_observation")
if payload.get("schema_version") not in {"0.1", "0.2"}:
raise ValueError("resource usage payload must be schema 0.1 or 0.2")
if not resource_id.startswith("resource:"):
raise ValueError("resource_id must use the resource: namespace")
created_at = payload["created_at"]
records: list[PlanningEvidence] = []
for row in payload["rows"]:
year, month = (int(part) for part in row["period"].split("-"))
period_start = date(year, month, 1)
period_end = date(year, month, monthrange(year, month)[1])
measures = []
for name, unit in _USAGE_MEASURES:
value = row.get(name)
if value is None:
continue
measures.append({"name": name, "value": value, "unit": unit})
gaps = [f"gap:{gap}" for gap in row.get("measurement_gaps", [])]
if row.get("infrastructure_eur") is None:
gaps.append("gap:infrastructure_eur:no booked fact (owner: fin-hub)")
record = ingest_planning_evidence(
{
"schema_version": "0.1",
"record_type": "usage_observation",
"record_id": (
f"usage:{payload['provider_id']}:"
f"{payload['cost_attribution_key']}:{row['period']}"
),
"revision_of": None,
"resource_id": resource_id,
"service_id": payload["provider_id"],
"workload_id": payload["workload"],
"tenant_id": None,
"environment": "production",
"cost_attribution_key": payload["cost_attribution_key"],
"period_start": period_start.isoformat(),
"period_end": period_end.isoformat(),
"source_evidence": [
source_evidence_ref,
*payload.get("evidence", []),
*gaps,
],
"created_at": created_at,
"measures": measures,
},
ledger_path=ledger_path,
)
records.append(record)
return records
@dataclass(frozen=True)
class ResourcePeriodReconciliation:
resource_id: str
period: str
forecast_record_ids: tuple[str, ...]
usage_record_ids: tuple[str, ...]
booked_financial_fact_ids: tuple[str, ...]
booked_effective_amount: Decimal | None
booked_currency: str | None
usage_measures: tuple[tuple[str, str, str], ...]
missing_booked_fact: bool
invented_zero: bool
def as_dict(self) -> dict:
payload = asdict(self)
payload["booked_effective_amount"] = (
None
if self.booked_effective_amount is None
else str(self.booked_effective_amount)
)
return payload
def reconcile_resource_period(
*,
resource_id: str,
period: str,
ledger_path: Path | None = None,
fact_resource_ids: Mapping[str, str] | None = None,
) -> ResourcePeriodReconciliation:
"""Join forecast, usage, and booked facts. Absence of a booked amount is not zero."""
if not resource_id.startswith("resource:"):
raise ValueError("resource_id must use the resource: namespace")
year, month = (int(part) for part in period.split("-"))
period_start = date(year, month, 1)
period_end = date(year, month, monthrange(year, month)[1])
ledger = ledger_path or default_ledger_path()
with _connect(ledger) as conn:
_ensure_planning_schema(conn)
planning_rows = conn.execute(
"SELECT record_id, record_type, payload_json FROM planning_evidence "
"WHERE is_current = 1 ORDER BY record_id"
).fetchall()
forecast_ids: list[str] = []
usage_ids: list[str] = []
usage_measures: list[tuple[str, str, str]] = []
for row in planning_rows:
payload = json.loads(row["payload_json"])
if payload.get("resource_id") != resource_id:
continue
start = date.fromisoformat(payload["period_start"])
end = date.fromisoformat(payload["period_end"])
if end < period_start or start > period_end:
continue
if row["record_type"] == "forecast":
forecast_ids.append(row["record_id"])
if row["record_type"] == "usage_observation":
usage_ids.append(row["record_id"])
for measure in payload.get("measures", []):
usage_measures.append(
(measure["name"], str(measure["value"]), measure["unit"])
)
booked = booked_cost_projection(
ledger_path=ledger, fact_resource_ids=fact_resource_ids
)
matching = [
fact
for fact in booked
if fact.resource_id == resource_id and fact.accounting_period == period
]
amount = None
currency = None
if matching:
currencies = {fact.currency for fact in matching}
if len(currencies) != 1:
raise ValueError("booked facts for this resource period mix currencies")
currency = next(iter(currencies))
amount = sum((fact.effective_amount for fact in matching), Decimal("0.00"))
invented_zero = amount == Decimal("0.00") and all(
fact.adjustment_kind == "charge" for fact in matching
)
else:
invented_zero = False
return ResourcePeriodReconciliation(
resource_id=resource_id,
period=period,
forecast_record_ids=tuple(forecast_ids),
usage_record_ids=tuple(usage_ids),
booked_financial_fact_ids=tuple(fact.financial_fact_id for fact in matching),
booked_effective_amount=amount,
booked_currency=currency,
usage_measures=tuple(usage_measures),
missing_booked_fact=not matching,
invented_zero=invented_zero,
)

View file

@ -0,0 +1,33 @@
{
"schema_version": "0.2",
"record_type": "usage_observation",
"workload": "rapp-postgres/platform-pg",
"cost_attribution_key": "platform:audit-storage",
"provider_id": "scaleway-standard-multi-az",
"created_at": "2026-08-14T18:24:00Z",
"scenario": "observed",
"forecast_ref": "data/forecasts/platform-audit-storage-scaleway-base-2026-08.json",
"rows": [
{
"period": "2026-08",
"database_gb": 0.6365,
"stored_gb": 0.0066,
"wal_gb": null,
"restore_egress_gb": null,
"write_requests": null,
"read_requests": null,
"infrastructure_eur": null,
"internal_labor_hours": null,
"internal_labor_eur": null,
"total_eur": null,
"backup_success_pct": 100,
"restore_rto_minutes": 1.08,
"measurement_gaps": [
"infrastructure_eur: no invoice for a resource commissioned 2026-08-14 (owner: fin-hub, FIN-WP-0004)"
]
}
],
"evidence": [
"resource-control:data/actuals/2026-08.json"
]
}

View file

@ -1,3 +1,4 @@
import json
from datetime import date, datetime, timezone from datetime import date, datetime, timezone
from decimal import Decimal from decimal import Decimal
from pathlib import Path from pathlib import Path
@ -17,6 +18,8 @@ from fin_hub.services.exchange import (
financial_constraint_projection, financial_constraint_projection,
ingest_planning_evidence, ingest_planning_evidence,
ingest_resource_forecast, ingest_resource_forecast,
ingest_resource_usage,
reconcile_resource_period,
) )
from fin_hub.services.ledger import import_csv from fin_hub.services.ledger import import_csv
@ -341,3 +344,72 @@ def test_exchange_health_exposes_rejection_stale_forecast_and_unattributed_cost(
"stale_forecast", "stale_forecast",
"unattributed_booked_cost", "unattributed_booked_cost",
} }
FIXTURES = Path(__file__).parent / "fixtures"
AUDIT_STORAGE = "resource:platform:audit-storage"
def test_audit_storage_usage_omits_nulls_and_does_not_invent_zero_spend(tmp_path: Path):
ledger = tmp_path / "ledger.db"
payload = json.loads(
(FIXTURES / "platform-audit-storage-usage-2026-08.json").read_text()
)
records = ingest_resource_usage(
payload,
resource_id=AUDIT_STORAGE,
source_evidence_ref="resource-control:data/actuals/2026-08.json",
ledger_path=ledger,
)
assert len(records) == 1
names = {measure.name: measure for measure in records[0].measures}
assert set(names) == {
"database_gb",
"stored_gb",
"backup_success_pct",
"restore_rto_minutes",
}
assert "infrastructure_eur" not in names
assert names["stored_gb"].value == Decimal("0.0066")
assert any("gap:infrastructure_eur" in item for item in records[0].source_evidence)
forecast = {
"schema_version": "0.1",
"record_type": "forecast",
"workload": "platform-pg",
"cost_attribution_key": "platform:audit-storage",
"provider_id": "scaleway-standard-multi-az",
"created_at": "2026-08-10T17:10:00Z",
"scenario": "base",
"forecast_ref": None,
"rows": [
{
"period": "2026-09",
"database_gb": 5,
"stored_gb": 180,
"wal_gb": 30,
"restore_egress_gb": 5,
"write_requests": 2500,
"read_requests": 1000,
"infrastructure_eur": 2.89,
"internal_labor_hours": 1,
"internal_labor_eur": 60,
"total_eur": 62.89,
}
],
}
ingest_resource_forecast(
forecast,
resource_id=AUDIT_STORAGE,
source_evidence_ref="resource-control:data/forecasts/platform-audit-storage",
ledger_path=ledger,
)
august = reconcile_resource_period(
resource_id=AUDIT_STORAGE, period="2026-08", ledger_path=ledger
)
assert august.usage_record_ids
assert august.missing_booked_fact is True
assert august.booked_effective_amount is None
assert august.invented_zero is False
health = exchange_health(ledger_path=ledger)
assert "usage_without_booked_fact" in {issue.code for issue in health.issues}

View file

@ -8,7 +8,7 @@ status: active
owner: codex owner: codex
topic_slug: financials topic_slug: financials
created: "2026-08-10" created: "2026-08-10"
updated: "2026-08-11" updated: "2026-08-15"
related: related:
- FIN-WP-0001 - FIN-WP-0001
- RESOURCE-WP-0002 - RESOURCE-WP-0002
@ -198,6 +198,15 @@ The task now waits on `RESOURCE-WP-0002` procurement and the first real
provider fact attributable to `platform:audit-storage`. Synthetic records test provider fact attributable to `platform:audit-storage`. Synthetic records test
the contract but do not satisfy the operational booked-cost round trip. the contract but do not satisfy the operational booked-cost round trip.
Progress 2026-08-15: resource-control commissioned the store on 2026-08-14
and published technical usage for August (`data/actuals/2026-08.json`).
`ingest_resource_usage` and `reconcile_resource_period` now join that
usage to the 12-row forecast without inventing a booked amount. August
reconciles as usage present, `missing_booked_fact: true`,
`invented_zero: false`. Evidence:
`docs/evidence/fin-wp-0004-t05-commissioning-2026-08.md`. T05 still
waits on the first Scaleway charge or credit (expected 2026-09).
## T06 — Generalize and operate the contract ## T06 — Generalize and operate the contract
```task ```task