diff --git a/.custodian-brief.md b/.custodian-brief.md index 38264d5..63c43d0 100644 --- a/.custodian-brief.md +++ b/.custodian-brief.md @@ -2,31 +2,26 @@ # Custodian Brief — fin-hub **Domain:** financials -**Last synced:** 2026-08-11 00:20 UTC +**Last synced:** 2026-08-11 08:20 UTC **State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)* ## Active Workstreams ### Establish the resource cost evidence contract -Progress: 0/9 done | workplan_id: `67b6de6c-4820-4478-9789-f50260204c27` +Progress: 6/9 done | workplan_id: `67b6de6c-4820-4478-9789-f50260204c27` **Open tasks:** -- ► T01 — Review and record the authority boundary `6616e6a0` -- · T02 — Specify fin-hub to resource-control booked-cost evidence `0a90dd6f` -- · T03 — Specify resource-control to fin-hub planning evidence `ad9dccb4` - · T04 — Expose budgets, commitments, and viability constraints `3620fc2a` - · T05 — Implement and reconcile the first round trip `9a411734` - · T06 — Generalize and operate the contract `a1309d51` -- · T07 — Correct cost-report dimensions and currency safety `406ce484` -- … and 2 more open tasks ### Client attribution and billing basis Progress: 3/6 done | workplan_id: `ebc1d2de-ae11-4cde-b860-047922fc74b9` **Open tasks:** -- ! Define shared-infrastructure allocation `899a5c29` - ! Export a per-client billing basis `b5886131` - ! Select the external invoicing system `d30b606f` +- · Define shared-infrastructure allocation `899a5c29` --- ## MCP Orientation (when available) diff --git a/INTENT.md b/INTENT.md index 1f39644..33516dd 100644 --- a/INTENT.md +++ b/INTENT.md @@ -1,8 +1,17 @@ # Fin Hub Intent -Provide the **Financial Allocator** surface for the FOS federation: make -resource pressure, burn rate, and runway visible so dev-hub can deprioritize -work and canon can receive viability alerts when thresholds are breached. +Provide the **Financial Allocator** and authoritative booked-cost surface for +the FOS federation. Fin-hub owns booked financial facts and their corrections, +budgets, financial commitments, burn, runway, viability signals, and +client-level price, cost-attribution, and margin reporting. -Fin-hub does not execute financial transactions. It tracks, projects, and -signals. +Fin-hub consumes resource identity, technical usage, allocation evidence, +forecasts, commitment candidates, and optimization scenarios from +`resource-control` without becoming authoritative for those concepts. A +forecast or allocation never becomes booked spend merely because fin-hub +stores or reports it. + +Fin-hub does not generate legal invoices, perform bookkeeping, execute or +track payments, approve contractual commitments, own client/resource identity, +or provision infrastructure. It records, reconciles, reports, projects, and +signals within that boundary. diff --git a/README.md b/README.md index cbdaf2c..7f85afe 100644 --- a/README.md +++ b/README.md @@ -54,6 +54,20 @@ month, and currency. Corrections are append-only: pass the current record ID to `ledger set-price --revision-of`; margin reports use the newest revision and preserve the earlier price for auditability. +Ledger imports identify financial facts independently of filename and mtime. +Duplicate, renamed, touched, and forced unchanged deliveries do not change +totals. A changed row is rejected unless `force=True` explicitly appends a +correction; reversals use `reverse_financial_fact` and retain their predecessor. +All effective calculations use integer minor units with decimal +round-half-even normalization. + +The executable resource-control exchange contract lives in +`fin_hub.schemas.exchange`. `booked_cost_projection` emits current booked facts; +`ingest_planning_evidence` stores typed forecasts, usage observations, +allocations, optimization cases, and commitment candidates outside booked +spend. `ingest_resource_forecast` adapts resource-control's v0.1 monthly backup +forecast without inventing resource identity. + ## Related Workplans - `the-custodian/workplans/CUST-WP-0025-fos-hub-bootstrap.md` — umbrella diff --git a/WORK-RECORDS.md b/WORK-RECORDS.md index 7b018d9..ffa9fc7 100644 --- a/WORK-RECORDS.md +++ b/WORK-RECORDS.md @@ -26,15 +26,18 @@ | task | FIN-WP-0002-T00 | done | — | workplans/FIN-WP-0002-client-attribution-and-billing-basis.md | | task | FIN-WP-0002-T01 | done | — | workplans/FIN-WP-0002-client-attribution-and-billing-basis.md | | task | FIN-WP-0002-T02 | done | — | workplans/FIN-WP-0002-client-attribution-and-billing-basis.md | -| task | FIN-WP-0002-T03 | wait | — | workplans/FIN-WP-0002-client-attribution-and-billing-basis.md | +| task | FIN-WP-0002-T03 | todo | — | workplans/FIN-WP-0002-client-attribution-and-billing-basis.md | | task | FIN-WP-0002-T04 | wait | — | workplans/FIN-WP-0002-client-attribution-and-billing-basis.md | | task | FIN-WP-0002-T05 | wait | — | workplans/FIN-WP-0002-client-attribution-and-billing-basis.md | | task | FIN-WP-0003-T01 | todo | — | workplans/FIN-WP-0003-fabric-authority-boundary.md | | task | FIN-WP-0003-T02 | todo | — | workplans/FIN-WP-0003-fabric-authority-boundary.md | | task | FIN-WP-0003-T03 | todo | — | workplans/FIN-WP-0003-fabric-authority-boundary.md | -| task | FIN-WP-0004-T01 | progress | — | workplans/FIN-WP-0004-resource-cost-evidence-contract.md | -| task | FIN-WP-0004-T02 | todo | — | workplans/FIN-WP-0004-resource-cost-evidence-contract.md | -| task | FIN-WP-0004-T03 | todo | — | workplans/FIN-WP-0004-resource-cost-evidence-contract.md | +| task | FIN-WP-0004-T01 | done | — | workplans/FIN-WP-0004-resource-cost-evidence-contract.md | +| task | FIN-WP-0004-T02 | done | — | workplans/FIN-WP-0004-resource-cost-evidence-contract.md | +| task | FIN-WP-0004-T03 | done | — | workplans/FIN-WP-0004-resource-cost-evidence-contract.md | | task | FIN-WP-0004-T04 | todo | — | workplans/FIN-WP-0004-resource-cost-evidence-contract.md | | task | FIN-WP-0004-T05 | todo | — | workplans/FIN-WP-0004-resource-cost-evidence-contract.md | | task | FIN-WP-0004-T06 | todo | — | workplans/FIN-WP-0004-resource-cost-evidence-contract.md | +| task | FIN-WP-0004-T07 | done | — | workplans/FIN-WP-0004-resource-cost-evidence-contract.md | +| task | FIN-WP-0004-T08 | done | — | workplans/FIN-WP-0004-resource-cost-evidence-contract.md | +| task | FIN-WP-0004-T09 | done | — | workplans/FIN-WP-0004-resource-cost-evidence-contract.md | diff --git a/docs/fin-resource-authority-contract-v0.1.md b/docs/fin-resource-authority-contract-v0.1.md index 99f56c3..be54917 100644 --- a/docs/fin-resource-authority-contract-v0.1.md +++ b/docs/fin-resource-authority-contract-v0.1.md @@ -1,7 +1,9 @@ # Fin-hub ↔ resource-control authority contract v0.1 -Status: draft for joint review -Owners: `fin-hub` / `resource-control` +Status: authority boundary reviewed; v0.1 implementation acceptance pending + +Owners: `fin-hub` / `resource-control` + Workplans: `FIN-WP-0004-T01`, `RESOURCE-WP-0003-T02` ## Boundary @@ -51,6 +53,19 @@ identifier and consumers preserve it byte-for-byte. exchange also carries `financial_fact_id`, period, currency, and provenance; every technical exchange carries its record ID and resource/workload context. +## Identifier decisions + +- Client and infrastructure attribution identifiers remain distinct, + versioned namespaces under this common envelope. +- Native service/workload identity stays with its owning repository; + resource-control may issue a clearly scoped portfolio surrogate only when no + native authority exists. +- `usage_observation` is the technical observation record. “Booked actual” is + reserved for fin-hub financial facts. +- Each producer owns its outbound schema. This contract pins compatible + versions and reconciliation; transport is selected after the backup round + trip. + ## Fin-hub → resource-control: booked-cost evidence Minimum envelope: @@ -61,17 +76,24 @@ Minimum envelope: "record_type": "booked_cost", "financial_fact_id": "opaque", "correction_of": null, + "adjustment_kind": "charge", "source_type": "provider_invoice", + "source_document_id": "opaque", + "source_line_id": "opaque", + "content_fingerprint": "sha256", "provider": "opaque-provider-id", "provider_account_ref": null, "accounting_period": "YYYY-MM", "service_period_start": "YYYY-MM-DD", "service_period_end": "YYYY-MM-DD", "currency": "EUR", - "net_amount": 0, - "tax_amount": 0, - "gross_amount": 0, - "credit_amount": 0, + "net_amount": "10.00", + "discount_amount": "0.00", + "tax_status": "known", + "tax_amount": "1.90", + "gross_amount": "11.90", + "adjustment_amount": "0.00", + "effective_amount": "11.90", "resource_id": null, "service_id": null, "workload_id": null, @@ -87,6 +109,15 @@ Delivery is idempotent on `financial_fact_id`. Corrections append a new record with `correction_of`; consumers retain both and calculate the effective value. Unknown attribution stays null and remains visible in reconciliation. +Money uses decimal strings at two fractional digits, ISO 4217 currency, and +round-half-even. `net_amount`, `discount_amount`, and `gross_amount` are +non-negative. When tax is known, `gross = net - discount + tax`; unknown tax is +represented by `tax_status: unknown` and `tax_amount: null` without inventing a +net/tax split. `adjustment_amount` is signed and `effective = gross + +adjustment`; credits, refunds, and reversals require a negative adjustment and +an explicit predecessor. The executable schema is +`fin_hub.schemas.exchange.BookedCostEvidence`. + ## Resource-control → fin-hub: planning and allocation evidence Minimum common envelope: @@ -94,7 +125,7 @@ Minimum common envelope: ```json { "schema_version": "0.1", - "record_type": "forecast|allocation|usage|optimization|commitment_candidate", + "record_type": "forecast|allocation|usage_observation|optimization|commitment_candidate", "record_id": "opaque", "revision_of": null, "resource_id": "resource:opaque", @@ -106,20 +137,28 @@ Minimum common envelope: "period_start": "YYYY-MM-DD", "period_end": "YYYY-MM-DD", "currency": "EUR", - "scenario": "low|base|high|observed", - "amount": 0, - "uncertainty": null, - "method": "versioned method identifier", - "assumptions": [], "source_evidence": [], "created_at": "RFC3339 timestamp" } ``` -Record-specific schemas add usage units, allocation drivers and shares, -service constraints, or optimization-case economics. Delivery is idempotent -on `record_id`; revisions are append-only. Fin-hub stores these as planning or -analytical evidence and never in its booked-cost ledger. +The common envelope deliberately has no generic `amount`. Executable, +record-specific schemas in `fin_hub.schemas.exchange` define: + +- `ForecastEvidence`: scenario/version, uncertainty, assumptions, and separate + infrastructure, internal-labor, external-service, setup, and other costs; +- `UsageObservation`: named decimal measures with explicit units and no booked + cost semantics; +- `AllocationEvidence`: referenced financial facts, method, allocated amount, + target shares, and explicit residual that must reconcile to one; +- `OptimizationEvidence`: baseline/alternative breakdown, one-time cost, and + expected period savings; and +- `CommitmentCandidate`: setup/recurring cost, cadence, term, and immutable + candidate approval status. + +Delivery is idempotent on `record_id`; revisions are append-only and must +reference the current record of the same type. Fin-hub stores these in +`planning_evidence`, never in its booked-cost ledger. ## Reconciliation invariants @@ -135,13 +174,10 @@ analytical evidence and never in its booked-cost ledger. 6. Missing joins, unknown tax semantics, and unattributed residuals are observable data-quality states, not zero values. -## Review questions +## Acceptance status -- Should client attribution keys and infrastructure allocation keys share one - namespaced grammar, or remain explicitly separate formats? -- Which repository assigns `service_id` and `workload_id` where no workload - repository already owns them? -- Does resource-control's `actual` record type mean technical observation only, - or should it be renamed to prevent confusion with fin-hub booked actuals? -- Which transport and schema registry become canonical for v0.1 after the - authority matrix is accepted? +Resource-control accepted the authority boundary in principle on 2026-08-11. +The review is preserved at +`history/2026-08-11-resource-control-review-fin-wp-0004.md`. Contract v0.1 is +not accepted until the implementation-integrity tasks and reconciled backup +round trip pass. diff --git a/history/2026-08-11-resource-control-review-fin-wp-0004.md b/history/2026-08-11-resource-control-review-fin-wp-0004.md new file mode 100644 index 0000000..e895176 --- /dev/null +++ b/history/2026-08-11-resource-control-review-fin-wp-0004.md @@ -0,0 +1,133 @@ +# Resource-control review of fin-hub results + +Date: 2026-08-11 +Reviewer: `resource-control` / Codex +Reviewed repository: `fin-hub` +Reviewed commits: `d2b9bc4^..2895077` +Primary workplans: `FIN-WP-0002`, `FIN-WP-0004`, `RESOURCE-WP-0003` +State: changes requested before accepting the v0.1 exchange contract + +## Conclusion + +The fin-hub result is directionally strong, but the v0.1 resource-cost +exchange contract is not yet ready for approval. Its authority matrix correctly +separates booked financial facts in fin-hub from resource identity, technical +usage, allocation evidence, forecasts, and optimization in resource-control. +The client-attribution and engagement-margin work is a useful foundation. + +Accounting-integrity and schema gaps remain. In particular, existing reports +can combine incompatible currencies and environments, changed CSV imports can +duplicate booked cost, and the SQLAlchemy and SQLite engagement-price paths do +not enforce the same invariants. The exchange envelopes also need exact money, +correction, and record-type semantics before implementation. + +Verification on 2026-08-10: + +- fin-hub worktree was clean at review time; +- `uv run pytest -q` passed: 33 tests, one dependency deprecation warning; +- a diagnostic reproduced EUR and USD service rows being combined into one + EUR-labelled total. + +## Findings and implementation pointers + +| Severity | Finding | Required task | +| --- | --- | --- | +| High | Service reporting groups by `service_id` and month without preserving currency and environment dimensions. | `FIN-WP-0004-T07` | +| High | CSV import identity is based on path and mtime; a changed or forced import appends the same booked rows again. | `FIN-WP-0004-T08` | +| Medium | SQLAlchemy and SQLite engagement-price paths enforce different attribution, currency, amount, period, and revision invariants. | `FIN-WP-0004-T09` | +| Medium | Booked-cost envelope does not define net, tax, gross, credit, refund, correction, decimal precision, or rounding relationships. | `FIN-WP-0004-T02` | +| Medium | The common planning envelope's single `amount` is insufficient for infrastructure, labor, setup, usage, allocation, and savings evidence. | `FIN-WP-0004-T03` | +| Low | fin-hub's canonical `INTENT.md` does not yet express booked-cost authority, client reporting, or the resource-control boundary. | `FIN-WP-0004-T01` | +| Low | Workplan lifecycle language and frontmatter status disagree. | `FIN-WP-0004-T01` | + +`FIN-WP-0004-T05` remains the acceptance gate: the first round trip must prove +that a cost is booked once, projected to resource-control, joined to technical +evidence, and returned as planning or variance evidence without becoming a +second financial fact. + +## Detailed findings + +### 1. Currency and environment mixing + +`src/fin_hub/coupling/ops_hub.py` keys service buckets only by `service_id` +and monthly totals only by month. A diagnostic with a EUR production row and a +USD development row for the same service produced one EUR-labelled total. + +Service aggregation must use at least: + +```text +service_id × environment × currency +month × currency +``` + +Unattributed and attributed reports must expose the same dimensional rules and +reconciliation totals. + +### 2. Duplicate booked cost on changed imports + +`src/fin_hub/services/ledger.py` treats `(source_path, source_mtime)` as import +identity and inserts every row from every newly observed file version. This is +file-delivery deduplication, not financial-fact idempotency. A corrected file or +forced import can inflate burn, attributed cost, and margin. + +Each booked row needs a stable `financial_fact_id`, source-document and +source-line identity, content fingerprint, and explicit append-only +correction/reversal relationship. Duplicate delivery must leave totals +unchanged. + +### 3. Divergent engagement-price invariants + +The SQLite command path validates attribution components, period, currency, +non-negative amount, source, and the current revision. The SQLAlchemy +`EngagementPrice` model does not enforce equivalent rules and can persist a key +that disagrees with its dimensions or an invalid price/revision state. + +Both persistence paths should use the same domain validator and database +constraints, with concurrency-safe uniqueness for the current price basis. + +### 4. Monetary contract semantics + +The booked-cost schema must define whether credits are signed, whether gross +equals net plus tax, where discounts apply, how refunds/reversals work, how +unknown tax is represented, and which precision and rounding rule applies. +Exchange money should use decimal strings or integer minor units, not binary +floating-point numbers. + +### 5. Record-specific planning schemas + +The common envelope may stay small, but `forecast`, `allocation`, `usage`, +`optimization`, and `commitment_candidate` each need a concrete schema. +Infrastructure cost, internal labor, external services, setup cost, recurring +cost, technical units, allocation shares and residual, and expected savings +must not be overloaded into one ambiguous `amount`. + +### 6. Canonical intent and lifecycle alignment + +`INTENT.md` should reciprocate the boundary already recorded in +resource-control: fin-hub owns booked financial facts, budgets, commitments, +burn, runway, and viability signals; it consumes resource-control projections +without becoming authoritative for resource identity or technical allocation. + +The workplan should no longer describe itself as proposed after its +frontmatter is active. Joint review remains incomplete until these review +results are incorporated and linked by both repositories. + +## Decisions recommended for v0.1 + +- Keep client and infrastructure attribution identifiers in distinct, + versioned namespaces under one common envelope. +- Native service and workload identity stays with its owning repository. + Resource-control may issue an explicitly scoped portfolio surrogate only + where no native authority exists. +- Rename resource-control `actual` records to `usage_observation` or + `cost_observation`; reserve “booked actual” for fin-hub facts. +- Each producer owns its outbound schema. The shared contract pins compatible + versions and reconciliation rules; transport is selected after the backup + round trip validates the model. + +## Approval condition + +Approve the authority boundary in principle. Keep `FIN-WP-0004-T01` in +progress until the contract incorporates this assessment. Do not mark the +v0.1 exchange accepted until `FIN-WP-0004-T02`, `T03`, `T07`, `T08`, and `T09` +are complete and `T05` demonstrates an idempotent reconciled round trip. diff --git a/src/fin_hub/cli.py b/src/fin_hub/cli.py index 32028b9..0d9ba4f 100644 --- a/src/fin_hub/cli.py +++ b/src/fin_hub/cli.py @@ -139,13 +139,13 @@ def _cmd_ledger_set_price(args: argparse.Namespace) -> int: revision_of=args.revision_of, ledger_path=_ledger_path(args), ) - print(json.dumps(price.__dict__, indent=2)) + print(json.dumps(price.__dict__, indent=2, default=str)) return 0 def _cmd_ledger_margins(args: argparse.Namespace) -> int: rows = client_margin_report(ledger_path=_ledger_path(args)) - print(json.dumps([row.as_dict() for row in rows], indent=2)) + print(json.dumps([row.as_dict() for row in rows], indent=2, default=str)) return 0 diff --git a/src/fin_hub/coupling/ops_hub.py b/src/fin_hub/coupling/ops_hub.py index 00f0602..035b685 100644 --- a/src/fin_hub/coupling/ops_hub.py +++ b/src/fin_hub/coupling/ops_hub.py @@ -35,12 +35,14 @@ class ServiceCostLine: def build_service_cost_report( lines: Iterable[ServiceCostLine], ) -> dict: - by_service: dict[str, dict] = {} - totals_by_month: dict[str, float] = defaultdict(float) - by_attribution: dict[tuple[str, str], dict] = {} + by_service: dict[tuple[str, str, str], dict] = {} + totals_by_period_currency: dict[tuple[str, str], float] = defaultdict(float) + by_attribution: dict[tuple[str, str, str], dict] = {} + unattributed: dict[tuple[str, str], dict] = {} for line in lines: + service_key = (line.service_id, line.environment, line.currency) bucket = by_service.setdefault( - line.service_id, + service_key, { "service_id": line.service_id, "environment": line.environment, @@ -52,8 +54,12 @@ def build_service_cost_report( month_total = bucket["months"].get(line.period_month, 0.0) + line.amount bucket["months"][line.period_month] = month_total bucket["total"] += line.amount - totals_by_month[line.period_month] += line.amount - key = (line.cost_attribution_key or "unattributed", line.currency) + totals_by_period_currency[(line.period_month, line.currency)] += line.amount + key = ( + line.cost_attribution_key or "unattributed", + line.environment, + line.currency, + ) attribution = by_attribution.setdefault( key, { @@ -61,6 +67,7 @@ def build_service_cost_report( "client_id": line.client_id, "application_id": line.application_id, "app_instance_id": line.app_instance_id, + "environment": line.environment, "currency": line.currency, "months": {}, "total": 0.0, @@ -70,6 +77,21 @@ def build_service_cost_report( attribution["months"].get(line.period_month, 0.0) + line.amount ) attribution["total"] += line.amount + if line.cost_attribution_key is None: + residual_key = (line.environment, line.currency) + residual = unattributed.setdefault( + residual_key, + { + "environment": line.environment, + "currency": line.currency, + "months": {}, + "total": 0.0, + }, + ) + residual["months"][line.period_month] = ( + residual["months"].get(line.period_month, 0.0) + line.amount + ) + residual["total"] += line.amount services = sorted(by_service.values(), key=lambda item: item["total"], reverse=True) return { "source_hub": "fin-hub", @@ -79,5 +101,11 @@ def build_service_cost_report( "attributions": sorted( by_attribution.values(), key=lambda item: item["total"], reverse=True ), - "totals_by_month": dict(sorted(totals_by_month.items())), + "unattributed": sorted( + unattributed.values(), key=lambda item: (item["currency"], item["environment"]) + ), + "totals_by_period_currency": [ + {"period_month": period, "currency": currency, "total": total} + for (period, currency), total in sorted(totals_by_period_currency.items()) + ], } diff --git a/src/fin_hub/models/engagement_price.py b/src/fin_hub/models/engagement_price.py index ce733a0..820f38a 100644 --- a/src/fin_hub/models/engagement_price.py +++ b/src/fin_hub/models/engagement_price.py @@ -5,29 +5,77 @@ from __future__ import annotations import uuid from datetime import date -from sqlalchemy import Date, Float, ForeignKey, String +from decimal import Decimal + +from sqlalchemy import Boolean, CheckConstraint, Date, ForeignKey, Index, Numeric, String, event, text from sqlalchemy.dialects.postgresql import UUID from sqlalchemy.orm import Mapped, mapped_column from hub_core.models.base import Base, TimestampMixin +from fin_hub.money import EngagementPriceTerms class EngagementPrice(Base, TimestampMixin): """A reporting entitlement price, not an invoice or payment record.""" __tablename__ = "fin_engagement_prices" + __table_args__ = ( + CheckConstraint("amount >= 0", name="ck_fin_engagement_prices_nonnegative"), + CheckConstraint("length(currency) = 3", name="ck_fin_engagement_prices_currency"), + CheckConstraint( + "effective_to IS NULL OR effective_to >= effective_from", + name="ck_fin_engagement_prices_effective_dates", + ), + Index( + "uq_fin_engagement_prices_current_basis", + "cost_attribution_key", + "period_month", + "currency", + unique=True, + postgresql_where=text("is_current"), + sqlite_where=text("is_current = 1"), + ), + ) id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4) client_id: Mapped[str] = mapped_column(String(128), nullable=False, index=True) application_id: Mapped[str] = mapped_column(String(128), nullable=False, index=True) app_instance_id: Mapped[str] = mapped_column(String(128), nullable=False, index=True) cost_attribution_key: Mapped[str] = mapped_column(String(423), nullable=False, index=True) + period_month: Mapped[str] = mapped_column(String(7), nullable=False, index=True) effective_from: Mapped[date] = mapped_column(Date, nullable=False, index=True) effective_to: Mapped[date | None] = mapped_column(Date, nullable=True) - amount: Mapped[float] = mapped_column(Float, nullable=False) + amount: Mapped[Decimal] = mapped_column(Numeric(18, 2), nullable=False) currency: Mapped[str] = mapped_column(String(3), nullable=False, default="EUR") source: Mapped[str] = mapped_column(String(128), nullable=False) revision_of: Mapped[uuid.UUID | None] = mapped_column( UUID(as_uuid=True), ForeignKey("fin_engagement_prices.id"), nullable=True ) + is_current: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True) notes: Mapped[str | None] = mapped_column(String(512), nullable=True) + + +@event.listens_for(EngagementPrice, "before_insert") +@event.listens_for(EngagementPrice, "before_update") +def _validate_engagement_price(_mapper, _connection, target: EngagementPrice) -> None: + if target.effective_to is not None and target.effective_to < target.effective_from: + raise ValueError("effective_to cannot precede effective_from") + terms = EngagementPriceTerms.validate( + client_id=target.client_id, + application_id=target.application_id, + app_instance_id=target.app_instance_id, + period_month=target.effective_from.strftime("%Y-%m"), + amount=target.amount, + currency=target.currency, + source=target.source, + ) + target.client_id = terms.attribution.client_id + target.application_id = terms.attribution.application_id + target.app_instance_id = terms.attribution.app_instance_id + if target.cost_attribution_key not in (None, terms.attribution.key): + raise ValueError("cost_attribution_key does not match its attribution dimensions") + target.cost_attribution_key = terms.attribution.key + target.period_month = terms.period_month + target.amount = terms.amount + target.currency = terms.currency + target.source = terms.source diff --git a/src/fin_hub/models/service_cost.py b/src/fin_hub/models/service_cost.py index c206bab..24a9ab0 100644 --- a/src/fin_hub/models/service_cost.py +++ b/src/fin_hub/models/service_cost.py @@ -4,13 +4,15 @@ from __future__ import annotations import uuid from datetime import date +from decimal import Decimal -from sqlalchemy import CheckConstraint, Date, Float, String, event +from sqlalchemy import CheckConstraint, Date, Numeric, String, event from sqlalchemy.dialects.postgresql import UUID from sqlalchemy.orm import Mapped, mapped_column from hub_core.models.base import Base, TimestampMixin from fin_hub.attribution import optional_attribution +from fin_hub.money import currency_code, money, reporting_month class ServiceCost(Base, TimestampMixin): @@ -29,7 +31,7 @@ class ServiceCost(Base, TimestampMixin): service_id: Mapped[str] = mapped_column(String(128), nullable=False, index=True) environment: Mapped[str] = mapped_column(String(64), nullable=False, default="production") period_month: Mapped[str] = mapped_column(String(7), nullable=False, index=True) - amount: Mapped[float] = mapped_column(Float, nullable=False, default=0.0) + amount: Mapped[Decimal] = mapped_column(Numeric(18, 2), nullable=False, default=Decimal("0")) currency: Mapped[str] = mapped_column(String(3), nullable=False, default="EUR") source: Mapped[str] = mapped_column(String(32), nullable=False) incurred_on: Mapped[date | None] = mapped_column(Date, nullable=True) @@ -50,3 +52,9 @@ def _normalize_service_cost_attribution(_mapper, _connection, target: ServiceCos if target.cost_attribution_key not in (None, expected_key): raise ValueError("cost_attribution_key does not match its attribution dimensions") target.cost_attribution_key = expected_key + target.period_month = reporting_month(target.period_month) + target.amount = money(target.amount) + target.currency = currency_code(target.currency) + target.source = target.source.strip() + if not target.source: + raise ValueError("source is required") diff --git a/src/fin_hub/money.py b/src/fin_hub/money.py new file mode 100644 index 0000000..7ec4bbf --- /dev/null +++ b/src/fin_hub/money.py @@ -0,0 +1,81 @@ +"""Shared financial-domain validation and decimal arithmetic.""" + +from __future__ import annotations + +from dataclasses import dataclass +from datetime import date +from decimal import Decimal, InvalidOperation, ROUND_HALF_EVEN + +from fin_hub.attribution import ClientAttribution + +MONEY_QUANTUM = Decimal("0.01") +MONEY_ROUNDING = ROUND_HALF_EVEN + + +def money(value: Decimal | str | int | float, *, non_negative: bool = False) -> Decimal: + try: + normalized = Decimal(str(value)).quantize(MONEY_QUANTUM, rounding=MONEY_ROUNDING) + except (InvalidOperation, ValueError) as exc: + raise ValueError("money must be a finite decimal value") from exc + if not normalized.is_finite(): + raise ValueError("money must be a finite decimal value") + if non_negative and normalized < 0: + raise ValueError("money cannot be negative") + return normalized + + +def money_minor(value: Decimal | str | int | float, *, non_negative: bool = False) -> int: + return int(money(value, non_negative=non_negative) / MONEY_QUANTUM) + + +def minor_money(value: int) -> Decimal: + return (Decimal(value) * MONEY_QUANTUM).quantize(MONEY_QUANTUM) + + +def currency_code(value: str) -> str: + normalized = value.strip().upper() + if len(normalized) != 3 or not normalized.isascii() or not normalized.isalpha(): + raise ValueError("currency must be a three-letter ASCII code") + return normalized + + +def reporting_month(value: str) -> str: + if len(value) != 7 or value[4] != "-": + raise ValueError("period_month must use YYYY-MM") + try: + date.fromisoformat(f"{value}-01") + except ValueError as exc: + raise ValueError("period_month must use YYYY-MM") from exc + return value + + +@dataclass(frozen=True) +class EngagementPriceTerms: + attribution: ClientAttribution + period_month: str + amount: Decimal + currency: str + source: str + + @classmethod + def validate( + cls, + *, + client_id: str, + application_id: str, + app_instance_id: str, + period_month: str, + amount: Decimal | str | int | float, + currency: str, + source: str, + ) -> "EngagementPriceTerms": + normalized_source = source.strip() + if not normalized_source: + raise ValueError("source is required") + return cls( + attribution=ClientAttribution(client_id, application_id, app_instance_id), + period_month=reporting_month(period_month), + amount=money(amount, non_negative=True), + currency=currency_code(currency), + source=normalized_source, + ) diff --git a/src/fin_hub/schemas/__init__.py b/src/fin_hub/schemas/__init__.py index 9104043..3a3fab0 100644 --- a/src/fin_hub/schemas/__init__.py +++ b/src/fin_hub/schemas/__init__.py @@ -1,5 +1,23 @@ """Pydantic schemas for fin-hub HTTP surfaces.""" from fin_hub.schemas.runway import MonthlyBurnRead, RunwayAlertRead, RunwaySummaryRead +from fin_hub.schemas.exchange import ( + AllocationEvidence, + BookedCostEvidence, + CommitmentCandidate, + ForecastEvidence, + OptimizationEvidence, + UsageObservation, +) -__all__ = ["MonthlyBurnRead", "RunwayAlertRead", "RunwaySummaryRead"] \ No newline at end of file +__all__ = [ + "AllocationEvidence", + "BookedCostEvidence", + "CommitmentCandidate", + "ForecastEvidence", + "MonthlyBurnRead", + "OptimizationEvidence", + "RunwayAlertRead", + "RunwaySummaryRead", + "UsageObservation", +] diff --git a/src/fin_hub/schemas/exchange.py b/src/fin_hub/schemas/exchange.py new file mode 100644 index 0000000..8436c2c --- /dev/null +++ b/src/fin_hub/schemas/exchange.py @@ -0,0 +1,237 @@ +"""Versioned fin-hub/resource-control exchange records.""" + +from __future__ import annotations + +from datetime import date, datetime +from decimal import Decimal +from typing import Annotated, Literal + +from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator + +from fin_hub.money import currency_code, money, reporting_month + + +class ExchangeRecord(BaseModel): + model_config = ConfigDict(extra="forbid") + + schema_version: Literal["0.1"] = "0.1" + record_id: str = Field(min_length=1, max_length=256) + revision_of: str | None = None + resource_id: str | None = None + service_id: str | None = None + workload_id: str | None = None + tenant_id: str | None = None + environment: str | None = None + cost_attribution_key: str | None = None + period_start: date + period_end: date + source_evidence: list[str] + created_at: datetime + + @model_validator(mode="after") + def validate_period(self): + if self.period_end < self.period_start: + raise ValueError("period_end cannot precede period_start") + return self + + +class MonetaryRecord(ExchangeRecord): + currency: str + + @field_validator("currency") + @classmethod + def validate_currency(cls, value: str) -> str: + return currency_code(value) + +class BookedCostEvidence(BaseModel): + """Authoritative fin-hub fact; amounts serialize as decimal strings.""" + + model_config = ConfigDict(extra="forbid") + + schema_version: Literal["0.1"] = "0.1" + record_type: Literal["booked_cost"] = "booked_cost" + financial_fact_id: str = Field(min_length=1, max_length=256) + correction_of: str | None = None + adjustment_kind: Literal["charge", "credit", "refund", "reversal", "correction"] + source_type: str = Field(min_length=1) + source_document_id: str = Field(min_length=1) + source_line_id: str = Field(min_length=1) + content_fingerprint: str = Field(min_length=1) + provider: str = Field(min_length=1) + provider_account_ref: str | None = None + accounting_period: str + service_period_start: date | None = None + service_period_end: date | None = None + currency: str + net_amount: Decimal = Decimal("0.00") + discount_amount: Decimal = Decimal("0.00") + tax_status: Literal["known", "unknown", "not_applicable"] + tax_amount: Decimal | None = None + gross_amount: Decimal + adjustment_amount: Decimal = Decimal("0.00") + effective_amount: Decimal + resource_id: str | None = None + service_id: str | None = None + workload_id: str | None = None + tenant_id: str | None = None + environment: str | None = None + cost_attribution_key: str | None = None + source_evidence_ref: str = Field(min_length=1) + recorded_at: datetime + + @field_validator( + "net_amount", "discount_amount", "gross_amount", mode="before" + ) + @classmethod + def validate_nonnegative_money(cls, value): + return money(value, non_negative=True) + + @field_validator("tax_amount", "adjustment_amount", "effective_amount", mode="before") + @classmethod + def validate_signed_money(cls, value): + return None if value is None else money(value) + + @field_validator("currency") + @classmethod + def validate_currency(cls, value: str) -> str: + return currency_code(value) + + @field_validator("accounting_period") + @classmethod + def validate_accounting_period(cls, value: str) -> str: + return reporting_month(value) + + @model_validator(mode="after") + def validate_accounting_relationships(self): + if self.tax_status == "unknown" and self.tax_amount is not None: + raise ValueError("tax_amount must be null when tax_status is unknown") + if self.tax_status != "unknown" and self.tax_amount is None: + raise ValueError("tax_amount is required when tax status is known") + tax = self.tax_amount or Decimal("0.00") + if self.tax_status != "unknown": + expected_gross = money(self.net_amount - self.discount_amount + tax) + if self.gross_amount != expected_gross: + raise ValueError("gross_amount must equal net - discount + tax") + if self.effective_amount != money(self.gross_amount + self.adjustment_amount): + raise ValueError("effective_amount must equal gross + adjustment") + if self.adjustment_kind != "charge" and self.correction_of is None: + raise ValueError("credit/refund/reversal/correction requires correction_of") + if self.adjustment_kind in {"credit", "refund", "reversal"} and self.adjustment_amount >= 0: + raise ValueError("credit/refund/reversal adjustment_amount must be negative") + if ( + self.service_period_start is not None + and self.service_period_end is not None + and self.service_period_end < self.service_period_start + ): + raise ValueError("service period end cannot precede start") + return self + + +class CostBreakdown(BaseModel): + model_config = ConfigDict(extra="forbid") + infrastructure: Decimal = Decimal("0.00") + internal_labor: Decimal = Decimal("0.00") + external_services: Decimal = Decimal("0.00") + setup: Decimal = Decimal("0.00") + other: Decimal = Decimal("0.00") + + @field_validator("*", mode="before") + @classmethod + def validate_money(cls, value): + return money(value, non_negative=True) + + @property + def total(self) -> Decimal: + return money( + sum( + (getattr(self, name) for name in self.__class__.model_fields), + Decimal("0"), + ) + ) + + +class ForecastEvidence(MonetaryRecord): + record_type: Literal["forecast"] = "forecast" + scenario: Literal["low", "base", "high"] + forecast_version: str = Field(min_length=1) + costs: CostBreakdown + uncertainty: str | None = None + assumptions: list[str] + + +class UsageMeasure(BaseModel): + model_config = ConfigDict(extra="forbid") + name: str = Field(min_length=1) + value: Decimal + unit: str = Field(min_length=1) + + +class UsageObservation(ExchangeRecord): + record_type: Literal["usage_observation"] = "usage_observation" + measures: list[UsageMeasure] + + +class AllocationShare(BaseModel): + model_config = ConfigDict(extra="forbid") + target_key: str = Field(min_length=1) + share: Decimal = Field(ge=0, le=1) + + +class AllocationEvidence(MonetaryRecord): + record_type: Literal["allocation"] = "allocation" + financial_fact_ids: list[str] = Field(min_length=1) + method: str = Field(min_length=1) + allocated_amount: Decimal + shares: list[AllocationShare] + residual_share: Decimal = Field(ge=0, le=1) + + @field_validator("allocated_amount", mode="before") + @classmethod + def validate_amount(cls, value): + return money(value, non_negative=True) + + @model_validator(mode="after") + def validate_shares(self): + total = sum((share.share for share in self.shares), Decimal("0")) + if total + self.residual_share != Decimal("1"): + raise ValueError("allocation shares plus residual_share must equal 1") + return self + + +class OptimizationEvidence(MonetaryRecord): + record_type: Literal["optimization"] = "optimization" + baseline: CostBreakdown + alternative: CostBreakdown + one_time_cost: Decimal + expected_period_savings: Decimal + assumptions: list[str] + + @field_validator("one_time_cost", "expected_period_savings", mode="before") + @classmethod + def validate_amount(cls, value): + return money(value, non_negative=True) + + +class CommitmentCandidate(MonetaryRecord): + record_type: Literal["commitment_candidate"] = "commitment_candidate" + setup_cost: Decimal + recurring_cost: Decimal + cadence: Literal["monthly", "quarterly", "annual", "one_time"] + term_start: date + term_end: date | None = None + approval_status: Literal["candidate"] = "candidate" + + @field_validator("setup_cost", "recurring_cost", mode="before") + @classmethod + def validate_amount(cls, value): + return money(value, non_negative=True) + + +PlanningEvidence = Annotated[ + ForecastEvidence + | UsageObservation + | AllocationEvidence + | OptimizationEvidence + | CommitmentCandidate, + Field(discriminator="record_type"), +] diff --git a/src/fin_hub/services/exchange.py b/src/fin_hub/services/exchange.py new file mode 100644 index 0000000..9304311 --- /dev/null +++ b/src/fin_hub/services/exchange.py @@ -0,0 +1,203 @@ +"""Persistence boundary for non-booked resource-control evidence.""" + +from __future__ import annotations + +import json +import sqlite3 +from calendar import monthrange +from datetime import date +from pathlib import Path + +from pydantic import TypeAdapter + +from fin_hub.money import minor_money, money +from fin_hub.schemas.exchange import BookedCostEvidence, PlanningEvidence +from fin_hub.services.ledger import _connect, default_ledger_path + +_PLANNING_ADAPTER = TypeAdapter(PlanningEvidence) + + +def _ensure_planning_schema(conn: sqlite3.Connection) -> None: + conn.execute( + """ + CREATE TABLE IF NOT EXISTS planning_evidence ( + record_id TEXT PRIMARY KEY, + record_type TEXT NOT NULL, + revision_of TEXT REFERENCES planning_evidence(record_id), + payload_json TEXT NOT NULL, + is_current INTEGER NOT NULL DEFAULT 1, + received_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP + ) + """ + ) + conn.execute( + "CREATE INDEX IF NOT EXISTS ix_planning_evidence_type_current " + "ON planning_evidence (record_type, is_current)" + ) + conn.commit() + + +def ingest_planning_evidence( + payload: dict, + *, + ledger_path: Path | None = None, +) -> PlanningEvidence: + """Validate and idempotently retain planning evidence outside booked spend.""" + + record = _PLANNING_ADAPTER.validate_python(payload) + canonical = record.model_dump_json() + ledger = ledger_path or default_ledger_path() + with _connect(ledger) as conn: + _ensure_planning_schema(conn) + conn.execute("BEGIN IMMEDIATE") + existing = conn.execute( + "SELECT payload_json FROM planning_evidence WHERE record_id = ?", + (record.record_id,), + ).fetchone() + if existing is not None: + if json.loads(existing["payload_json"]) != json.loads(canonical): + raise ValueError("record_id already exists with different content") + return record + if record.revision_of is not None: + predecessor = conn.execute( + "SELECT record_type, is_current FROM planning_evidence WHERE record_id = ?", + (record.revision_of,), + ).fetchone() + if predecessor is None or predecessor["record_type"] != record.record_type: + raise ValueError("revision_of must reference an existing record of the same type") + if predecessor["is_current"] != 1: + raise ValueError("revision_of must reference the current record") + conn.execute( + "UPDATE planning_evidence SET is_current = 0 WHERE record_id = ?", + (record.revision_of,), + ) + conn.execute( + "INSERT INTO planning_evidence " + "(record_id, record_type, revision_of, payload_json, is_current) " + "VALUES (?, ?, ?, ?, 1)", + (record.record_id, record.record_type, record.revision_of, canonical), + ) + conn.commit() + return record + + +def booked_cost_projection(*, ledger_path: Path | None = None) -> list[BookedCostEvidence]: + """Project current authoritative facts without exposing raw invoice content.""" + + ledger = ledger_path or default_ledger_path() + with _connect(ledger) as conn: + rows = conn.execute( + "SELECT * FROM ledger_entries WHERE is_current = 1 ORDER BY id" + ).fetchall() + projected: list[BookedCostEvidence] = [] + for row in rows: + amount = minor_money(int(row["amount_minor"])) + adjustment_kind = row["adjustment_kind"] + gross_amount = amount + adjustment_amount = minor_money(0) + if adjustment_kind == "reversal": + with _connect(ledger) as conn: + predecessor = conn.execute( + "SELECT amount_minor FROM ledger_entries WHERE financial_fact_id = ?", + (row["correction_of"],), + ).fetchone() + if predecessor is None: + raise ValueError("reversal predecessor is missing") + gross_amount = minor_money(int(predecessor["amount_minor"])) + adjustment_amount = -gross_amount + projected.append( + BookedCostEvidence( + financial_fact_id=row["financial_fact_id"], + correction_of=row["correction_of"], + adjustment_kind=adjustment_kind, + source_type=row["source_type"], + source_document_id=row["source_document_id"], + source_line_id=row["source_line_id"], + content_fingerprint=row["content_fingerprint"], + provider=row["source_type"], + accounting_period=row["period_month"], + service_period_start=row["incurred_on"], + service_period_end=row["incurred_on"], + currency=row["currency"], + net_amount="0.00", + discount_amount="0.00", + tax_status="unknown", + tax_amount=None, + gross_amount=gross_amount, + adjustment_amount=adjustment_amount, + effective_amount=amount, + service_id=row["category"], + environment=row["environment"], + cost_attribution_key=row["cost_attribution_key"], + source_evidence_ref=row["correction_source"] or row["source_document_id"], + recorded_at=row["imported_at"], + ) + ) + return projected + + +def ingest_resource_forecast( + payload: dict, + *, + resource_id: str, + source_evidence_ref: str, + ledger_path: Path | None = None, +) -> list[PlanningEvidence]: + """Adapt resource-control's v0.1 monthly forecast into typed evidence.""" + + if payload.get("record_type") != "forecast": + raise ValueError("resource forecast payload must have record_type=forecast") + created_at = payload["created_at"] + version = f"{payload['provider_id']}:{created_at}" + records: list[PlanningEvidence] = [] + for row in payload["rows"]: + expected_total = money(row["infrastructure_eur"]) + money(row["internal_labor_eur"]) + if money(row["total_eur"]) != expected_total: + raise ValueError("resource forecast total_eur does not match its cost breakdown") + year, month = (int(part) for part in row["period"].split("-")) + period_start = date(year, month, 1) + period_end = date(year, month, monthrange(year, month)[1]) + record = ingest_planning_evidence( + { + "schema_version": "0.1", + "record_type": "forecast", + "record_id": ( + f"forecast:{payload['provider_id']}:" + f"{payload['cost_attribution_key']}:{row['period']}:{created_at}" + ), + "revision_of": payload.get("forecast_ref"), + "resource_id": resource_id, + "service_id": payload["provider_id"], + "workload_id": payload["workload"], + "tenant_id": None, + "environment": "production", + "cost_attribution_key": payload["cost_attribution_key"], + "period_start": period_start.isoformat(), + "period_end": period_end.isoformat(), + "currency": "EUR", + "source_evidence": [source_evidence_ref, *row.get("evidence", [])], + "created_at": created_at, + "scenario": payload.get("scenario") or "base", + "forecast_version": version, + "costs": { + "infrastructure": row["infrastructure_eur"], + "internal_labor": row["internal_labor_eur"], + "external_services": 0, + "setup": 0, + "other": 0, + }, + "uncertainty": None, + "assumptions": [ + f"database_gb={row['database_gb']}", + f"stored_gb={row['stored_gb']}", + f"wal_gb={row['wal_gb']}", + f"restore_egress_gb={row['restore_egress_gb']}", + f"write_requests={row['write_requests']}", + f"read_requests={row['read_requests']}", + f"internal_labor_hours={row['internal_labor_hours']}", + ], + }, + ledger_path=ledger_path, + ) + records.append(record) + return records diff --git a/src/fin_hub/services/ledger.py b/src/fin_hub/services/ledger.py index e3f004b..a7c2504 100644 --- a/src/fin_hub/services/ledger.py +++ b/src/fin_hub/services/ledger.py @@ -3,15 +3,25 @@ from __future__ import annotations import json +import hashlib import sqlite3 import uuid from dataclasses import asdict, dataclass from datetime import date, datetime, timezone +from decimal import Decimal from pathlib import Path from fin_hub.ingest.anthropic import parse_anthropic_billing_csv from fin_hub.ingest.cloud import parse_cloud_cost_csv from fin_hub.ingest.hosteurope import parse_hosteurope_csv +from fin_hub.money import ( + EngagementPriceTerms, + currency_code, + minor_money, + money, + money_minor, + reporting_month, +) DEFAULT_LEDGER_PATH = Path(".fin-hub/ledger.db") @@ -21,16 +31,26 @@ class LedgerEntry: source_type: str category: str label: str - amount: float + amount: Decimal currency: str period_month: str incurred_on: date | None source_path: str + environment: str = "production" client_id: str | None = None application_id: str | None = None app_instance_id: str | None = None cost_attribution_key: str | None = None + def __post_init__(self) -> None: + object.__setattr__(self, "amount", money(self.amount)) + object.__setattr__(self, "currency", currency_code(self.currency)) + object.__setattr__(self, "period_month", reporting_month(self.period_month)) + normalized_environment = self.environment.strip() + if not normalized_environment: + raise ValueError("environment is required") + object.__setattr__(self, "environment", normalized_environment) + @dataclass(frozen=True) class MonthlyRollup: @@ -62,7 +82,7 @@ class EngagementPriceRecord: app_instance_id: str cost_attribution_key: str period_month: str - amount: float + amount: Decimal currency: str source: str revision_of: str | None @@ -77,9 +97,9 @@ class ClientMargin: app_instance_id: str period_month: str currency: str - revenue: float - attributed_cost: float - margin: float + revenue: Decimal + attributed_cost: Decimal + margin: Decimal price_id: str def as_dict(self) -> dict: @@ -136,14 +156,14 @@ def _ensure_schema(conn: sqlite3.Connection) -> None: cost_attribution_key TEXT NOT NULL, period_month TEXT NOT NULL, amount REAL NOT NULL, + amount_minor INTEGER, currency TEXT NOT NULL, source TEXT NOT NULL, revision_of TEXT REFERENCES engagement_prices(id), + is_current INTEGER NOT NULL DEFAULT 1, recorded_at TEXT NOT NULL ); - CREATE INDEX IF NOT EXISTS ix_engagement_prices_basis - ON engagement_prices (cost_attribution_key, period_month, currency); """ ) columns = {row["name"] for row in conn.execute("PRAGMA table_info(ledger_entries)")} @@ -152,15 +172,99 @@ def _ensure_schema(conn: sqlite3.Connection) -> None: "application_id", "app_instance_id", "cost_attribution_key", + "environment", ): if name not in columns: conn.execute(f"ALTER TABLE ledger_entries ADD COLUMN {name} TEXT") + entry_columns = {row["name"] for row in conn.execute("PRAGMA table_info(ledger_entries)")} + if "amount_minor" not in entry_columns: + conn.execute("ALTER TABLE ledger_entries ADD COLUMN amount_minor INTEGER") + for name, definition in ( + ("financial_fact_id", "TEXT"), + ("source_document_id", "TEXT"), + ("source_line_id", "TEXT"), + ("content_fingerprint", "TEXT"), + ("correction_of", "TEXT"), + ("adjustment_kind", "TEXT NOT NULL DEFAULT 'charge'"), + ("correction_source", "TEXT"), + ("is_current", "INTEGER NOT NULL DEFAULT 1"), + ): + if name not in entry_columns: + conn.execute(f"ALTER TABLE ledger_entries ADD COLUMN {name} {definition}") + conn.execute( + "UPDATE ledger_entries SET amount_minor = ROUND(amount * 100) " + "WHERE amount_minor IS NULL" + ) + conn.execute( + "UPDATE ledger_entries SET financial_fact_id = 'legacy:' || id, " + "source_document_id = 'legacy:' || source_path, " + "source_line_id = 'legacy:' || id, " + "content_fingerprint = 'legacy:' || id " + "WHERE financial_fact_id IS NULL" + ) + conn.execute( + "CREATE UNIQUE INDEX IF NOT EXISTS uq_ledger_financial_fact_id " + "ON ledger_entries (financial_fact_id)" + ) + conn.execute( + "CREATE UNIQUE INDEX IF NOT EXISTS uq_ledger_current_source_line " + "ON ledger_entries (source_line_id) WHERE is_current = 1" + ) + price_columns = { + row["name"] for row in conn.execute("PRAGMA table_info(engagement_prices)") + } + if "amount_minor" not in price_columns: + conn.execute("ALTER TABLE engagement_prices ADD COLUMN amount_minor INTEGER") + if "is_current" not in price_columns: + conn.execute( + "ALTER TABLE engagement_prices ADD COLUMN is_current INTEGER NOT NULL DEFAULT 1" + ) + conn.execute( + "UPDATE engagement_prices SET amount_minor = ROUND(amount * 100) " + "WHERE amount_minor IS NULL" + ) + conn.execute( + "UPDATE engagement_prices SET is_current = 0 WHERE id IN (" + "SELECT revision_of FROM engagement_prices WHERE revision_of IS NOT NULL)" + ) + conn.execute( + "CREATE INDEX IF NOT EXISTS ix_engagement_prices_basis " + "ON engagement_prices (cost_attribution_key, period_month, currency)" + ) + conn.execute( + "CREATE UNIQUE INDEX IF NOT EXISTS uq_engagement_prices_current_basis " + "ON engagement_prices (cost_attribution_key, period_month, currency) " + "WHERE is_current = 1" + ) + conn.commit() def _utc_now() -> str: return datetime.now(timezone.utc).isoformat() +def _digest(*parts: object) -> str: + payload = "\x1f".join("" if part is None else str(part) for part in parts) + return hashlib.sha256(payload.encode("utf-8")).hexdigest() + + +def _entry_business_key(entry: LedgerEntry) -> str: + return _digest( + entry.source_type, + entry.category, + entry.label, + entry.currency, + entry.period_month, + entry.incurred_on.isoformat() if entry.incurred_on else None, + entry.environment, + entry.cost_attribution_key, + ) + + +def _entry_fingerprint(entry: LedgerEntry) -> str: + return _digest(_entry_business_key(entry), money_minor(entry.amount)) + + def set_opening_balance(path: Path, balance: float, *, currency: str = "EUR") -> None: with _connect(path) as conn: conn.execute( @@ -234,6 +338,7 @@ def _entries_from_hosteurope(path: Path) -> list[LedgerEntry]: period_month=row.period_month, incurred_on=row.incurred_on, source_path=resolved, + environment=row.environment, client_id=row.client_id, application_id=row.application_id, app_instance_id=row.app_instance_id, @@ -279,20 +384,50 @@ def import_csv( skipped=True, ) - for entry in entries: + business_keys = [_entry_business_key(entry) for entry in entries] + source_document_id = f"{source_type}:{_digest(*sorted(business_keys))}" + occurrences: dict[str, int] = {} + rows_imported = 0 + for entry, business_key in zip(entries, business_keys, strict=True): + occurrence = occurrences.get(business_key, 0) + 1 + occurrences[business_key] = occurrence + source_line_id = f"{source_type}:{business_key}:{occurrence}" + fingerprint = _entry_fingerprint(entry) + current = conn.execute( + "SELECT financial_fact_id, content_fingerprint FROM ledger_entries " + "WHERE source_line_id = ? AND is_current = 1", + (source_line_id,), + ).fetchone() + if current is not None and current["content_fingerprint"] == fingerprint: + continue + if current is not None and not force: + raise ValueError( + "changed financial fact requires force=True to append an explicit correction: " + f"{current['financial_fact_id']}" + ) + correction_of = current["financial_fact_id"] if current is not None else None + financial_fact_id = f"fact:{_digest(source_line_id, fingerprint)}" + if current is not None: + conn.execute( + "UPDATE ledger_entries SET is_current = 0 WHERE financial_fact_id = ?", + (correction_of,), + ) conn.execute( """ INSERT INTO ledger_entries ( source_type, category, label, amount, currency, period_month, incurred_on, source_path, imported_at, - client_id, application_id, app_instance_id, cost_attribution_key - ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + client_id, application_id, app_instance_id, cost_attribution_key, + environment, amount_minor, financial_fact_id, source_document_id, + source_line_id, content_fingerprint, correction_of, + adjustment_kind, correction_source, is_current + ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 1) """, ( entry.source_type, entry.category, entry.label, - entry.amount, + float(entry.amount), entry.currency, entry.period_month, entry.incurred_on.isoformat() if entry.incurred_on else None, @@ -302,33 +437,108 @@ def import_csv( entry.application_id, entry.app_instance_id, entry.cost_attribution_key, + entry.environment, + money_minor(entry.amount), + financial_fact_id, + source_document_id, + source_line_id, + fingerprint, + correction_of, + "correction" if correction_of else "charge", + entry.source_path if correction_of else None, ), ) + rows_imported += 1 conn.execute( """ INSERT OR REPLACE INTO ledger_imports ( source_path, source_mtime, source_type, row_count, imported_at ) VALUES (?, ?, ?, ?, ?) """, - (resolved, mtime, source_type, len(entries), imported_at), + (resolved, mtime, source_type, rows_imported, imported_at), ) conn.commit() return ImportResult( source_type=source_type, source_path=resolved, - rows_imported=len(entries), - skipped=False, + rows_imported=rows_imported, + skipped=rows_imported == 0, ) +def reverse_financial_fact( + financial_fact_id: str, + *, + source: str, + ledger_path: Path | None = None, +) -> str: + """Append a zero-effective reversal and retain the reversed predecessor.""" + + normalized_source = source.strip() + if not normalized_source: + raise ValueError("source is required") + ledger = ledger_path or default_ledger_path() + with _connect(ledger) as conn: + conn.execute("BEGIN IMMEDIATE") + current = conn.execute( + "SELECT * FROM ledger_entries WHERE financial_fact_id = ? AND is_current = 1", + (financial_fact_id,), + ).fetchone() + if current is None: + raise ValueError("financial_fact_id must reference a current fact") + fingerprint = _digest(current["content_fingerprint"], "reversal", normalized_source) + reversal_id = f"fact:{_digest(current['source_line_id'], fingerprint)}" + conn.execute( + "UPDATE ledger_entries SET is_current = 0 WHERE financial_fact_id = ?", + (financial_fact_id,), + ) + conn.execute( + """ + INSERT INTO ledger_entries ( + source_type, category, label, amount, currency, period_month, + incurred_on, source_path, imported_at, client_id, application_id, + app_instance_id, cost_attribution_key, environment, amount_minor, + financial_fact_id, source_document_id, source_line_id, + content_fingerprint, correction_of, adjustment_kind, + correction_source, is_current + ) VALUES (?, ?, ?, 0, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 0, ?, ?, ?, ?, ?, 'reversal', ?, 1) + """, + ( + current["source_type"], + current["category"], + current["label"], + current["currency"], + current["period_month"], + current["incurred_on"], + current["source_path"], + _utc_now(), + current["client_id"], + current["application_id"], + current["app_instance_id"], + current["cost_attribution_key"], + current["environment"], + reversal_id, + current["source_document_id"], + current["source_line_id"], + fingerprint, + financial_fact_id, + normalized_source, + ), + ) + conn.commit() + return reversal_id + + def monthly_summary(*, ledger_path: Path | None = None) -> list[MonthlyRollup]: ledger = ledger_path or default_ledger_path() with _connect(ledger) as conn: rows = conn.execute( """ - SELECT period_month, currency, SUM(amount) AS total, COUNT(*) AS entry_count + SELECT period_month, currency, SUM(amount_minor) AS total_minor, + COUNT(*) AS entry_count FROM ledger_entries + WHERE is_current = 1 GROUP BY period_month, currency ORDER BY period_month, currency """ @@ -337,7 +547,7 @@ def monthly_summary(*, ledger_path: Path | None = None) -> list[MonthlyRollup]: MonthlyRollup( period_month=row["period_month"], currency=row["currency"], - total=float(row["total"]), + total=float(minor_money(int(row["total_minor"]))), entry_count=int(row["entry_count"]), ) for row in rows @@ -367,31 +577,25 @@ def record_engagement_price( price_id: str | None = None, revision_of: str | None = None, ) -> EngagementPriceRecord: - from fin_hub.attribution import ClientAttribution - - attribution = ClientAttribution(client_id, application_id, app_instance_id) - if len(period_month) != 7 or period_month[4] != "-": - raise ValueError("period_month must use YYYY-MM") - try: - date.fromisoformat(f"{period_month}-01") - except ValueError as exc: - raise ValueError("period_month must use YYYY-MM") from exc - normalized_currency = currency.strip().upper() - if len(normalized_currency) != 3 or not normalized_currency.isalpha(): - raise ValueError("currency must be a three-letter code") - if amount < 0: - raise ValueError("engagement price amount cannot be negative") - if not source.strip(): - raise ValueError("source is required") + terms = EngagementPriceTerms.validate( + client_id=client_id, + application_id=application_id, + app_instance_id=app_instance_id, + period_month=period_month, + amount=amount, + currency=currency, + source=source, + ) ledger = ledger_path or default_ledger_path() identifier = price_id or str(uuid.uuid4()) recorded_at = _utc_now() with _connect(ledger) as conn: - basis = (attribution.key, period_month, normalized_currency) + conn.execute("BEGIN IMMEDIATE") + basis = (terms.attribution.key, terms.period_month, terms.currency) current = conn.execute( "SELECT id FROM engagement_prices WHERE cost_attribution_key = ? " - "AND period_month = ? AND currency = ? " + "AND period_month = ? AND currency = ? AND is_current = 1 " "ORDER BY recorded_at DESC, rowid DESC LIMIT 1", basis, ).fetchone() @@ -399,21 +603,28 @@ def record_engagement_price( raise ValueError(f"revision_of must reference current price {current['id']}") if current is None and revision_of is not None: raise ValueError("revision_of cannot be used without an existing price") + if current is not None: + conn.execute( + "UPDATE engagement_prices SET is_current = 0 WHERE id = ?", + (current["id"],), + ) conn.execute( "INSERT INTO engagement_prices (" "id, client_id, application_id, app_instance_id, cost_attribution_key, " - "period_month, amount, currency, source, revision_of, recorded_at" - ") VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)", + "period_month, amount, amount_minor, currency, source, revision_of, " + "is_current, recorded_at" + ") VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 1, ?)", ( identifier, - attribution.client_id, - attribution.application_id, - attribution.app_instance_id, - attribution.key, - period_month, - amount, - normalized_currency, - source.strip(), + terms.attribution.client_id, + terms.attribution.application_id, + terms.attribution.app_instance_id, + terms.attribution.key, + terms.period_month, + float(terms.amount), + money_minor(terms.amount), + terms.currency, + terms.source, revision_of, recorded_at, ), @@ -421,14 +632,14 @@ def record_engagement_price( conn.commit() return EngagementPriceRecord( id=identifier, - client_id=attribution.client_id, - application_id=attribution.application_id, - app_instance_id=attribution.app_instance_id, - cost_attribution_key=attribution.key, - period_month=period_month, - amount=amount, - currency=normalized_currency, - source=source.strip(), + client_id=terms.attribution.client_id, + application_id=terms.attribution.application_id, + app_instance_id=terms.attribution.app_instance_id, + cost_attribution_key=terms.attribution.key, + period_month=terms.period_month, + amount=terms.amount, + currency=terms.currency, + source=terms.source, revision_of=revision_of, recorded_at=recorded_at, ) @@ -440,23 +651,18 @@ def client_margin_report(*, ledger_path: Path | None = None) -> list[ClientMargi rows = conn.execute( """ WITH latest_prices AS ( - SELECT p.* - FROM engagement_prices p - WHERE NOT EXISTS ( - SELECT 1 FROM engagement_prices revision - WHERE revision.revision_of = p.id - ) + SELECT p.* FROM engagement_prices p WHERE p.is_current = 1 ), attributed_costs AS ( SELECT cost_attribution_key, period_month, currency, - SUM(amount) AS attributed_cost + SUM(amount_minor) AS attributed_cost_minor FROM ledger_entries - WHERE cost_attribution_key IS NOT NULL + WHERE cost_attribution_key IS NOT NULL AND is_current = 1 GROUP BY cost_attribution_key, period_month, currency ) SELECT p.id AS price_id, p.client_id, p.application_id, p.app_instance_id, p.cost_attribution_key, p.period_month, - p.currency, p.amount AS revenue, - COALESCE(c.attributed_cost, 0) AS attributed_cost + p.currency, p.amount_minor AS revenue_minor, + COALESCE(c.attributed_cost_minor, 0) AS attributed_cost_minor FROM latest_prices p LEFT JOIN attributed_costs c ON c.cost_attribution_key = p.cost_attribution_key @@ -473,9 +679,11 @@ def client_margin_report(*, ledger_path: Path | None = None) -> list[ClientMargi app_instance_id=row["app_instance_id"], period_month=row["period_month"], currency=row["currency"], - revenue=float(row["revenue"]), - attributed_cost=float(row["attributed_cost"]), - margin=float(row["revenue"]) - float(row["attributed_cost"]), + revenue=minor_money(int(row["revenue_minor"])), + attributed_cost=minor_money(int(row["attributed_cost_minor"])), + margin=minor_money( + int(row["revenue_minor"]) - int(row["attributed_cost_minor"]) + ), price_id=row["price_id"], ) for row in rows diff --git a/tests/test_coupling.py b/tests/test_coupling.py index 625f1df..c9ce7b9 100644 --- a/tests/test_coupling.py +++ b/tests/test_coupling.py @@ -21,8 +21,11 @@ def test_build_service_cost_report_groups_by_service(): report = build_service_cost_report(lines) assert report["signal"] == "service_cost_attribution" assert len(report["services"]) == 2 - assert report["totals_by_month"]["2026-06"] == pytest.approx(99.8) + assert report["totals_by_period_currency"] == [ + {"period_month": "2026-06", "currency": "EUR", "total": pytest.approx(99.8)} + ] assert report["attributions"][0]["cost_attribution_key"] is None + assert report["unattributed"][0]["total"] == pytest.approx(99.8) def test_build_service_cost_report_groups_by_client_attribution(): @@ -49,6 +52,7 @@ def test_build_service_cost_report_groups_by_client_attribution(): "client_id": "acme", "application_id": "portal", "app_instance_id": "prod-01", + "environment": "production", "currency": "EUR", "months": {"2026-07": 42.0}, "total": 42.0, @@ -77,3 +81,31 @@ def test_client_attribution_totals_do_not_mix_currencies(): ("EUR", 42.0), ("USD", 50.0), } + + +def test_service_totals_do_not_mix_currency_or_environment(): + common = { + "service_id": "cluster", + "period_month": "2026-07", + "source": "fixture", + } + report = build_service_cost_report( + [ + ServiceCostLine(amount=10.0, currency="EUR", environment="production", **common), + ServiceCostLine(amount=20.0, currency="USD", environment="production", **common), + ServiceCostLine(amount=30.0, currency="EUR", environment="development", **common), + ] + ) + + assert { + (row["environment"], row["currency"], row["total"]) + for row in report["services"] + } == { + ("production", "EUR", 10.0), + ("production", "USD", 20.0), + ("development", "EUR", 30.0), + } + assert report["totals_by_period_currency"] == [ + {"period_month": "2026-07", "currency": "EUR", "total": 40.0}, + {"period_month": "2026-07", "currency": "USD", "total": 20.0}, + ] diff --git a/tests/test_exchange.py b/tests/test_exchange.py new file mode 100644 index 0000000..7487c46 --- /dev/null +++ b/tests/test_exchange.py @@ -0,0 +1,207 @@ +from datetime import date, datetime, timezone +from decimal import Decimal +from pathlib import Path + +import pytest +from pydantic import ValidationError + +from fin_hub.models.engagement_price import EngagementPrice, _validate_engagement_price +from fin_hub.schemas.exchange import ( + AllocationEvidence, + BookedCostEvidence, + ForecastEvidence, +) +from fin_hub.services.exchange import ( + booked_cost_projection, + ingest_planning_evidence, + ingest_resource_forecast, +) +from fin_hub.services.ledger import import_csv + + +def _common_planning() -> dict: + return { + "schema_version": "0.1", + "record_id": "forecast:platform-audit-storage:2026-08", + "revision_of": None, + "resource_id": "resource:platform_audit_storage", + "service_id": "object-storage", + "workload_id": "platform-pg", + "tenant_id": None, + "environment": "production", + "cost_attribution_key": "platform:audit-storage", + "period_start": "2026-09-01", + "period_end": "2027-08-31", + "currency": "eur", + "source_evidence": ["resource-control:data/forecasts/platform-audit-storage"], + "created_at": "2026-08-10T17:10:00Z", + } + + +def test_booked_cost_money_relationships_and_decimal_json(): + record = BookedCostEvidence( + financial_fact_id="fact:1", + correction_of=None, + adjustment_kind="charge", + source_type="provider_invoice", + source_document_id="invoice:1", + source_line_id="invoice:1:line:1", + content_fingerprint="sha256:abc", + provider="provider", + accounting_period="2026-07", + currency="eur", + net_amount="10.005", + discount_amount="1.00", + tax_status="known", + tax_amount="1.90", + gross_amount="10.90", + adjustment_amount="0", + effective_amount="10.90", + source_evidence_ref="invoice:1", + recorded_at=datetime.now(timezone.utc), + ) + + assert record.net_amount == Decimal("10.00") + assert record.currency == "EUR" + assert '"gross_amount":"10.90"' in record.model_dump_json() + + +def test_booked_cost_rejects_invalid_relationships(): + with pytest.raises(ValidationError, match="gross_amount"): + BookedCostEvidence( + financial_fact_id="fact:1", + adjustment_kind="charge", + source_type="invoice", + source_document_id="doc", + source_line_id="line", + content_fingerprint="hash", + provider="provider", + accounting_period="2026-07", + currency="EUR", + net_amount="10", + discount_amount="0", + tax_status="known", + tax_amount="1.90", + gross_amount="12", + effective_amount="12", + source_evidence_ref="doc", + recorded_at=datetime.now(timezone.utc), + ) + + +def test_planning_forecast_is_typed_and_idempotent(tmp_path: Path): + payload = { + **_common_planning(), + "record_type": "forecast", + "scenario": "base", + "forecast_version": "2026-08-base", + "costs": { + "infrastructure": "2.89", + "internal_labor": "60", + "external_services": "0", + "setup": "0", + "other": "0", + }, + "uncertainty": "provider price excludes unknown tax", + "assumptions": ["180 GB stored"], + } + ledger = tmp_path / "ledger.db" + + first = ingest_planning_evidence(payload, ledger_path=ledger) + second = ingest_planning_evidence(payload, ledger_path=ledger) + + assert isinstance(first, ForecastEvidence) + assert first.costs.total == Decimal("62.89") + assert second.record_id == first.record_id + + +def test_resource_control_backup_forecast_enters_planning_store(tmp_path: Path): + payload = { + "schema_version": "0.1", + "record_type": "forecast", + "workload": "platform-pg", + "cost_attribution_key": "platform:audit-storage", + "provider_id": "scaleway-standard-multi-az", + "created_at": "2026-08-10T17:10:00Z", + "scenario": "base", + "forecast_ref": None, + "rows": [ + { + "period": "2026-09", + "database_gb": 5, + "stored_gb": 180, + "wal_gb": 30, + "restore_egress_gb": 5, + "write_requests": 2500, + "read_requests": 1000, + "infrastructure_eur": 2.89, + "internal_labor_hours": 1, + "internal_labor_eur": 60, + "total_eur": 62.89, + } + ], + } + records = ingest_resource_forecast( + payload, + resource_id="resource:platform_audit_storage", + source_evidence_ref="resource-control:data/forecasts/platform-audit-storage", + ledger_path=tmp_path / "ledger.db", + ) + + assert len(records) == 1 + assert isinstance(records[0], ForecastEvidence) + assert records[0].costs.total == Decimal("62.89") + + +def test_allocation_requires_shares_and_residual_to_reconcile(): + payload = { + **_common_planning(), + "record_type": "allocation", + "record_id": "allocation:1", + "financial_fact_ids": ["fact:1"], + "method": "namespace-cpu-v1", + "allocated_amount": "10.00", + "shares": [{"target_key": "client:acme", "share": "0.75"}], + "residual_share": "0.20", + } + with pytest.raises(ValidationError, match="must equal 1"): + AllocationEvidence.model_validate(payload) + + +def test_booked_cost_projection_uses_current_corrected_fact(tmp_path: Path): + source = tmp_path / "cost.csv" + ledger = tmp_path / "ledger.db" + source.write_text( + "product,amount,currency,invoice_date,environment\n" + "Server,10.00,EUR,2026-07-01,production\n", + encoding="utf-8", + ) + import_csv(source, "hosteurope", ledger_path=ledger) + source.write_text( + "product,amount,currency,invoice_date,environment\n" + "Server,12.00,EUR,2026-07-01,production\n", + encoding="utf-8", + ) + import_csv(source, "hosteurope", ledger_path=ledger, force=True) + + records = booked_cost_projection(ledger_path=ledger) + + assert len(records) == 1 + assert records[0].effective_amount == Decimal("12.00") + assert records[0].correction_of is not None + assert records[0].environment == "production" + + +def test_sqlalchemy_price_uses_shared_invariants(): + price = EngagementPrice( + client_id="acme", + application_id="portal", + app_instance_id="prod-01", + cost_attribution_key="wrong", + effective_from=date(2026, 7, 1), + amount=Decimal("100.001"), + currency="eur", + source="agreement", + ) + with pytest.raises(ValueError, match="does not match"): + _validate_engagement_price(None, None, price) diff --git a/tests/test_ledger.py b/tests/test_ledger.py index c0b16e0..9e6184c 100644 --- a/tests/test_ledger.py +++ b/tests/test_ledger.py @@ -1,4 +1,6 @@ import sqlite3 +import os +import shutil from pathlib import Path import pytest @@ -11,6 +13,7 @@ from fin_hub.services.ledger import ( monthly_burn_series, monthly_summary, record_engagement_price, + reverse_financial_fact, set_opening_balance, ) @@ -44,6 +47,86 @@ def test_ledger_import_skips_unchanged_file(tmp_path: Path): assert second.rows_imported == 0 +def test_ledger_import_is_idempotent_when_file_is_renamed_or_touched(tmp_path: Path): + ledger = tmp_path / "ledger.db" + source = tmp_path / "costs.csv" + shutil.copy(FIXTURES / "cloud-costs.csv", source) + renamed = tmp_path / "renamed.csv" + + first = import_csv(source, "cloud", ledger_path=ledger) + os.utime(source, (source.stat().st_atime, source.stat().st_mtime + 1)) + touched = import_csv(source, "cloud", ledger_path=ledger) + shutil.copy(source, renamed) + delivered_again = import_csv(renamed, "cloud", ledger_path=ledger) + + assert first.rows_imported == 3 + assert touched.rows_imported == 0 + assert delivered_again.rows_imported == 0 + assert monthly_summary(ledger_path=ledger)[0].entry_count == 3 + + +def test_changed_fact_requires_and_applies_one_correction(tmp_path: Path): + ledger = tmp_path / "ledger.db" + source = tmp_path / "costs.csv" + source.write_text( + "service,amount,currency,period_month\ncompute,10.00,EUR,2026-07\n", + encoding="utf-8", + ) + import_csv(source, "cloud", ledger_path=ledger) + source.write_text( + "service,amount,currency,period_month\ncompute,12.00,EUR,2026-07\n", + encoding="utf-8", + ) + + with pytest.raises(ValueError, match="explicit correction"): + import_csv(source, "cloud", ledger_path=ledger) + corrected = import_csv(source, "cloud", ledger_path=ledger, force=True) + duplicate = import_csv(source, "cloud", ledger_path=ledger, force=True) + + assert corrected.rows_imported == 1 + assert duplicate.rows_imported == 0 + rollup = monthly_summary(ledger_path=ledger)[0] + assert rollup.total == 12.0 + assert rollup.entry_count == 1 + with sqlite3.connect(ledger) as conn: + facts = conn.execute( + "SELECT correction_of, is_current FROM ledger_entries ORDER BY id" + ).fetchall() + assert facts[0][1] == 0 + assert facts[1][0] is not None + assert facts[1][1] == 1 + + +def test_financial_fact_reversal_is_append_only_and_zeroes_effective_total(tmp_path: Path): + ledger = tmp_path / "ledger.db" + source = tmp_path / "costs.csv" + source.write_text( + "service,amount,currency,period_month\ncompute,10.00,EUR,2026-07\n", + encoding="utf-8", + ) + import_csv(source, "cloud", ledger_path=ledger) + with sqlite3.connect(ledger) as conn: + fact_id = conn.execute( + "SELECT financial_fact_id FROM ledger_entries" + ).fetchone()[0] + + reversal_id = reverse_financial_fact( + fact_id, source="provider-credit-note", ledger_path=ledger + ) + + rollup = monthly_summary(ledger_path=ledger)[0] + assert rollup.total == 0.0 + with sqlite3.connect(ledger) as conn: + rows = conn.execute( + "SELECT financial_fact_id, correction_of, adjustment_kind, is_current " + "FROM ledger_entries ORDER BY id" + ).fetchall() + assert rows == [ + (fact_id, None, "charge", 0), + (reversal_id, fact_id, "reversal", 1), + ] + + def test_ledger_import_preserves_client_attribution(tmp_path: Path): source = tmp_path / "attributed.csv" source.write_text( diff --git a/tests/test_models.py b/tests/test_models.py index c50509b..33cf36a 100644 --- a/tests/test_models.py +++ b/tests/test_models.py @@ -1,5 +1,10 @@ """Smoke tests for fin-hub model registration.""" +from datetime import date +from decimal import Decimal + +import pytest + from fin_hub.models import ( Budget, BurnRate, @@ -9,6 +14,8 @@ from fin_hub.models import ( ServiceCost, TokenSpend, ) +from fin_hub.models.engagement_price import _validate_engagement_price +from fin_hub.models.service_cost import _normalize_service_cost_attribution from hub_core.models.base import Base @@ -39,3 +46,41 @@ def test_service_cost_has_external_attribution_seam(): assert columns["application_id"].nullable is True assert columns["app_instance_id"].nullable is True assert columns["cost_attribution_key"].nullable is True + + +def test_service_cost_orm_uses_shared_money_and_period_invariants(): + cost = ServiceCost( + service_id="cluster", + environment="production", + period_month="2026-07", + amount=Decimal("10.005"), + currency="eur", + source=" invoice ", + ) + _normalize_service_cost_attribution(None, None, cost) + assert cost.amount == Decimal("10.00") + assert cost.currency == "EUR" + assert cost.source == "invoice" + + cost.period_month = "2026-13" + with pytest.raises(ValueError, match="YYYY-MM"): + _normalize_service_cost_attribution(None, None, cost) + + +def test_engagement_price_orm_normalizes_same_basis_as_sqlite(): + price = EngagementPrice( + client_id="acme", + application_id="portal", + app_instance_id="prod-01", + cost_attribution_key=None, + period_month="placeholder", + effective_from=date(2026, 7, 15), + amount=Decimal("100.005"), + currency="eur", + source=" agreement ", + ) + _validate_engagement_price(None, None, price) + assert price.period_month == "2026-07" + assert price.amount == Decimal("100.00") + assert price.currency == "EUR" + assert price.source == "agreement" diff --git a/uv.lock b/uv.lock index 521b31c..b07bf9f 100644 --- a/uv.lock +++ b/uv.lock @@ -568,7 +568,7 @@ wheels = [ [[package]] name = "hub-core" -version = "0.1.0" +version = "0.2.0" source = { editable = "../hub-core" } dependencies = [ { name = "fastapi" }, diff --git a/workplans/FIN-WP-0002-client-attribution-and-billing-basis.md b/workplans/FIN-WP-0002-client-attribution-and-billing-basis.md index 861cc8e..ba9e7fd 100644 --- a/workplans/FIN-WP-0002-client-attribution-and-billing-basis.md +++ b/workplans/FIN-WP-0002-client-attribution-and-billing-basis.md @@ -116,7 +116,7 @@ prices remain auditable, and duplicate prices require an explicit revision. ```task id: FIN-WP-0002-T03 -status: wait +status: todo priority: medium state_hub_task_id: "899a5c29-843c-481f-a63a-42d4d327af5d" ``` diff --git a/workplans/FIN-WP-0004-resource-cost-evidence-contract.md b/workplans/FIN-WP-0004-resource-cost-evidence-contract.md index 28da41e..caa9386 100644 --- a/workplans/FIN-WP-0004-resource-cost-evidence-contract.md +++ b/workplans/FIN-WP-0004-resource-cost-evidence-contract.md @@ -8,7 +8,7 @@ status: active owner: codex topic_slug: financials created: "2026-08-10" -updated: "2026-08-10" +updated: "2026-08-11" related: - FIN-WP-0001 - RESOURCE-WP-0002 @@ -31,14 +31,16 @@ signals. `resource-control` remains authoritative for resource identity, lifecycle, demand and capacity forecasts, technical usage, allocation evidence, unit economics, and infrastructure optimization cases. -This is a proposed companion to `RESOURCE-WP-0003` and should move to `ready` -only after both repository owners review the authority matrix. +This active companion to `RESOURCE-WP-0003` incorporates resource-control's +2026-08-11 review. The authority boundary is accepted in principle; the v0.1 +exchange remains unapproved until T02, T03, T05, and T07–T09 satisfy the +review conditions. ## T01 — Review and record the authority boundary ```task id: FIN-WP-0004-T01 -status: progress +status: done priority: high state_hub_task_id: "6616e6a0-b0f3-4b82-a081-083891e2fb6e" ``` @@ -66,11 +68,23 @@ defines the authority matrix, directional envelopes, identifiers, correction and idempotency rules, reconciliation invariants, and four review questions. Joint resource-control review remains required before completion. +Review 2026-08-11: resource-control accepted the authority boundary in +principle and requested changes before v0.1 approval. The timestamped result is +`history/2026-08-11-resource-control-review-fin-wp-0004.md`. T02 and T03 own +the envelope refinements; T07-T09 own the implementation-integrity findings. +Align `INTENT.md` and remove the proposed/active lifecycle contradiction as +part of incorporating the joint review. + +Completed 2026-08-11: incorporated all four review decisions into the +contract, aligned `INTENT.md`, and corrected lifecycle language. The authority +boundary is jointly reviewed and accepted in principle; implementation +acceptance remains gated by T05. + ## T02 — Specify fin-hub to resource-control booked-cost evidence ```task id: FIN-WP-0004-T02 -status: todo +status: done priority: high state_hub_task_id: "0a90dd6f-b0c9-42dc-b625-cfedf0471143" ``` @@ -90,11 +104,17 @@ Done when representative Host Europe or other provider rows can be exported idempotently and joined without copying invoice authority into `resource-control`. +Completed 2026-08-11: `BookedCostEvidence` defines decimal-string money, +round-half-even precision, net/discount/tax/gross/effective relationships, +unknown-tax handling, signed adjustment semantics, stable fact/document/line +identity, fingerprints, corrections, provenance, and join axes. Current +HostEurope facts project through this schema without raw invoice content. + ## T03 — Specify resource-control to fin-hub planning evidence ```task id: FIN-WP-0004-T03 -status: todo +status: done priority: high state_hub_task_id: "ad9dccb4-85be-4f19-b36e-2f7165a879f5" ``` @@ -111,6 +131,12 @@ forecasts remain queryable for later variance analysis. Done when the PostgreSQL backup forecast can enter fin-hub without being mistaken for an invoice or actual expenditure. +Completed 2026-08-11: added distinct forecast, usage-observation, allocation, +optimization, and commitment-candidate schemas plus an idempotent, +append-only `planning_evidence` store outside the booked ledger. The adapter +successfully ingested all 12 rows of resource-control's live PostgreSQL backup +forecast into a temporary fin-hub ledger. + ## T04 — Expose budgets, commitments, and viability constraints ```task @@ -170,6 +196,82 @@ unattributed booked cost, stale forecasts, and reconciliation failures. Done when the contract supports storage, compute or cluster capacity, and a shared-service allocation without changing the authority boundary. +## T07 — Correct cost-report dimensions and currency safety + +```task +id: FIN-WP-0004-T07 +status: done +priority: high +state_hub_task_id: "406ce484-db7f-4bb9-b3b3-23217c052e08" +``` + +Correct service and monthly aggregation so no report adds amounts across +currencies or silently combines environments. Define and test the canonical +dimensions for service, attribution, period, environment, and currency views. +Expose residual and unattributed totals under the same rules. + +Done when mixed-currency and mixed-environment fixtures remain separate, all +rollups reconcile per currency, and no result inherits a misleading currency +or environment label from its first row. + +Completed 2026-08-11: service buckets now key on service, environment, and +currency; attribution buckets include environment and currency; period totals +are currency-separated; and explicit unattributed residuals use the same +dimensions. Mixed fixtures are covered by regression tests. + +## T08 — Establish idempotent booked-fact imports and corrections + +```task +id: FIN-WP-0004-T08 +status: done +priority: high +state_hub_task_id: "aded815d-75a2-4496-9676-b44b5e4dc49a" +``` + +Replace file-path-and-mtime deduplication as the financial identity boundary. +Define stable `financial_fact_id`, source-document and source-line identity, +content fingerprint, duplicate-delivery behavior, and append-only correction, +reversal, and replacement semantics. Preserve raw provenance without storing +secret invoice material in exchange records. + +Done when unchanged, renamed, touched, corrected, and force-reimported inputs +cannot double-count a fact; corrections change effective totals exactly once +while predecessors remain auditable. + +Completed 2026-08-11: ledger rows now carry stable financial-fact, +source-document, and source-line identities, content fingerprints, +`correction_of`, and single-current state. Renamed, touched, duplicate, and +forced unchanged deliveries are idempotent; changed facts require an explicit +forced correction and effective totals include exactly one current version. +`reverse_financial_fact` appends a provenance-bearing reversal that zeroes the +effective fact without deleting its predecessor. + +## T09 — Unify money and engagement-price domain invariants + +```task +id: FIN-WP-0004-T09 +status: done +priority: high +state_hub_task_id: "c52d79b9-cbad-4d06-bd54-9b06125e12a0" +``` + +Use one domain validation path for SQLite commands and SQLAlchemy models. +Enforce attribution-key consistency, valid periods, normalized currency, +non-negative engagement prices, decimal money, effective-date rules, source +provenance, and a concurrency-safe append-only revision chain. Add equivalent +database constraints where practical. + +Done when both persistence paths reject the same invalid records, concurrent +revisions cannot create two current prices for one basis, and calculations use +documented precision and rounding. + +Completed 2026-08-11: shared money, currency, period, attribution, and price +validation now serves SQLite and SQLAlchemy. Calculations use integer minor +units or `Decimal(18,2)` with round-half-even; price revisions use immediate +transactions and partial unique current-basis indexes. ORM constraints cover +non-negative price, effective dates, currency, canonical attribution, and one +current reporting-month basis. + ## Acceptance - [ ] One reviewed authority matrix is linked from both repositories. @@ -182,3 +284,6 @@ shared-service allocation without changing the authority boundary. - [ ] Backup completes a reconciled round trip with no double counting. - [ ] At least one non-storage and one shared-service case validate the generalized contract. +- [ ] Cost reports never mix currency or environment dimensions. +- [ ] Booked-fact import and corrections are idempotent under changed delivery. +- [ ] SQLite and SQLAlchemy paths enforce the same money and price invariants.