feat: bound AI-plan work to resource-control and stop zeroing missing tokens

Record the FIN-WP-0007 split after checking resource-control: they keep
intelligence resource identity and provision-level class I metering;
session tokens stay with State Hub; work-effectiveness is a reporting
join. Missing token counts are now unknown, not zero.
This commit is contained in:
tegwick 2026-08-15 19:01:45 +02:00
parent 004c041dc0
commit 09ef7fa967
10 changed files with 326 additions and 38 deletions

View file

@ -11,6 +11,14 @@ forecasts, commitment candidates, and optimization scenarios from
forecast or allocation never becomes booked spend merely because fin-hub
stores or reports it.
Intelligence follows the same split. Resource-control owns portfolio
identity and provision-level class `I` metering when Railiance procures a
model API, reserved weights, or inference host. State Hub owns agent-session
token evidence. Fin-hub books the plan invoice, records entitlement, joins
those inputs, and reports effectiveness. It does not invent a `resource_id`,
parse sessions, or emit technical usage or allocation as if it were
resource-control.
Fin-hub does not generate legal invoices, perform bookkeeping, execute or
track payments, approve contractual commitments, own client/resource identity,
or provision infrastructure. It records, reconciles, reports, projects, and

View file

@ -1,7 +1,7 @@
# Fin Hub
Resource viability hub for the FOS federation — budgets, commitments, burn rate,
runway projection, and token spend tracking.
runway projection, and booked token / AI-plan spend.
Fin-hub extends `hub-core` with financial models and read surfaces. Generic hub
primitives (domains, repos, messages, progress events) come from hub-core;

View file

@ -2,11 +2,18 @@
## In Scope
- Budget, commitment, burn rate, runway projection, and token spend models
- Manual CSV import for cloud and API costs (v0.1)
- Budget, commitment, burn rate, runway projection, and booked token /
AI-plan spend
- Manual CSV import for cloud, API, and subscription costs (v0.1)
- Runway calculator with alert thresholds
- Client-level cost attribution, engagement revenue and margin reporting, and
billing-basis exports for external bookkeeping/invoicing systems
- AI-plan entitlements, joins to State Hub session-token evidence, reporting
allocation of booked plan cost onto work, and effectiveness overlays
(work per token and work per euro) that never become booked spend
- Consumption of resource-control planning evidence (forecasts, technical
usage, allocations, optimization, settlement statements) without taking
authority for those records
- FOS §9 cross-hub signals: fin→dev (budget pressure), fin→ops (cost attribution), fin→canon (viability alerts)
- hub-core generic primitives (domains, repos, messages, progress)
@ -15,4 +22,13 @@
- Legal invoice generation or numbering, bookkeeping, payment execution or
tracking, and banking integration
- Tax filing or legal entity management
- Authority for client identity or infrastructure resource identity
- Authority for client identity or infrastructure / intelligence resource
identity (`resource:…` remains resource-control)
- Provision-level class `I` token metering against a portfolio resource
- Technical usage, utilization, and allocation-driver evidence for
inventoried resources (resource-control, sourced from platform telemetry)
- Parsing coding-agent session files or storing transcripts (State Hub /
llm-connect)
- Inventing a `resource_id` for an AI plan that resource-control has not
published
- A canonical token↔hour or token↔euro exchange rate

View file

@ -0,0 +1,142 @@
# AI plan cost, token estimates, and effectiveness
Status: fin-hub design note for `FIN-WP-0007-T00`
Checked against: resource-control `SCOPE.md`, `INTENT.md`,
`docs/TermsOfResourceProcurement.md` §4.1 and §6,
`docs/fin-hub-resource-control-contract-v0.1.md`,
`docs/evidence-basis.md`,
`data/capability/platform-audit-storage.json` (2026-08-15),
and the ratified fin-hub contract
`docs/fin-resource-authority-contract-v0.1.md`.
This note records the authority split. It does not amend contract v0.1.
A proposed addendum is at the end for resource-control to accept or reject.
## Why a second check was required
`FIN-WP-0007` first treated resource-control as the wrong producer for
any AI-token number. That is too coarse.
Resource-control already names intelligence as a Railiance-procured
resource class — “commercial model APIs, reserved weights, inference
hosts” — and records class `I` consumption on a provision as *their*
gap (`owner: resource-control`), with unknown never stored as zero.
They do not yet inventory any AI plan, and they do not parse coding
sessions. Those two facts have to stay separate.
## Four quantities, four writers
| Quantity | Native unit | Authoritative writer | Fin-hub may |
| --- | --- | --- | --- |
| Booked plan / API invoice | currency | fin-hub | Book once as `booked_cost` |
| Recurring plan as commitment | currency / period | fin-hub | Record; not usage |
| Plan entitlement (included tokens or plan label) | token or declared capacity | fin-hub | Commercial fact, same discipline as engagement prices |
| Intelligence *resource* identity | `resource:…` | resource-control | Join when present; never invent |
| Provision-level class `I` usage | token | resource-control, when they meter a named provision | Consume via `FIN-WP-0004` `usage_observation` |
| Session token evidence | token | State Hub (`STATE-WP-0045`) | Join as session-token evidence |
| Technical allocation of a portfolio resource | share of a referenced fact | resource-control | Consume `AllocationEvidence`; do not emit it |
| Work done | task / outcome | State Hub / domain repos | Join key, not a cost driver |
| Reporting allocation of a plan invoice across work | share of a referenced fact | fin-hub | Local reporting derivation (`FIN-WP-0002` family) |
| Implied €/token and work-per-token / work-per-euro | derived | fin-hub | Overlay only; inherit the weakest input basis |
Session telemetry is not platform telemetry. Contract v0.1 says
technical usage is sourced from platform telemetry and written by
resource-control. Agent-session counts from State Hub therefore must
not be stored as resource-control `usage_observation`.
Reporting allocation of an invoice across workplans is not technical
allocation of a portfolio resource. Contract v0.1 keeps allocation
method and driver with resource-control. Fin-hub already has a
reporting-only consumer for that evidence (`FIN-WP-0002-T03`). Work
effectiveness uses a *local* reporting allocation and must not be
posted as `AllocationEvidence`.
## When a `resource_id` exists
If resource-control later inventories a Railiance-procured
intelligence resource (API account, reserved weights, inference
host, or license):
- fin-hub books the invoice and may attach the resource-control
`resource_id` they published;
- resource-control owns usage and technical allocation on that
resource, including class `I` tokens against the provision;
- fin-hub still owns entitlement, burn, runway, and the
work-effectiveness report.
Until that inventory row exists, `resource_id` stays `null`. A
null join is a visible gap, not a license to mint
`resource:anthropic:claude-max`.
## Financial entity
Resource-control requires every expense, estimate, and usage
observation to carry exactly one `financial_entity_id`, or an
explicit null with a named gap. Founder-private cards and uncleared
Auslagen stay outside their terms until booked to an entity *and* a
resource.
Fin-hub may still book those invoices for runway. It must not
silent-default the entity to Railiance or Binky, and it must not
pretend a private plan is a Railiance-procured resource.
## Measurement layers and estimates
Keep these records separate, joined by period, provider/plan, and
optional workplan/repo/`resource_id`:
1. booked plan cost
2. entitlement (or explicit unknown capacity)
3. consumption of class `I`, each row marked with how it was obtained
4. work done
Reuse resource-controls evidence-basis vocabulary rather than
inventing a dialect: `invoiced`, `measured`, `quoted`, `derived`,
`projected`, `estimated`, `assumed`, `unknown`. Unknown carries no
quantity. Zero means measured and none.
State Hub `measurement_kind` maps as:
| State Hub | Evidence basis used here |
| --- | --- |
| `measured` | `measured` |
| `allocated` | `derived` (from a measured parent; method required) |
| `estimated` | `estimated` |
| `superseded` | excluded from totals |
| no events | `unknown` residual, not a zero observation |
A derived overlay (implied €/token, work per token) is only as strong
as its weakest input. A month whose coverage is incomplete is
`insufficient` for trend comparison, not interpolated.
## What fin-hub will not do
- Invent resource identity or provider-resource identity.
- Meter tokens against a portfolio provision.
- Parse Codex, Claude, Cursor, or Grok session files.
- Emit `usage_observation` or `AllocationEvidence` as if it were
resource-control.
- Collapse tokens into euros at capture time.
- Store missing tokens as `0`.
- Treat a plan entitlement as an invoice, or an implied €/token as a
price.
## Proposed contract addendum (not ratified)
Offer to resource-control, without changing v0.1 until they accept:
1. Intelligence resources remain in their portfolio when Railiance
procures them. Class `I` on a provision stays their meter.
2. Agent-session token events stay with State Hub. Fin-hub may join
them for financial reporting; that join is not technical usage.
3. Work-effectiveness allocation of a booked AI-plan fact is a
fin-hub reporting derivation and is not technical allocation.
4. A booked AI-plan fact may carry `resource_id=null` until they
publish one. Fin-hub will not mint a surrogate.
`FIN-WP-0004-T05` (Scaleway `platform:audit-storage` booked-cost
round trip) is unchanged and is class `S`, not class `I`.
Offered to resource-control on 2026-08-15 as State Hub message
`4e75cd00-87f1-46e1-8d17-9169289640a2`. Not assumed until they
accept, reject, or name a conflict.

View file

@ -13,14 +13,28 @@ from fin_hub.ingest._csv import parse_amount, pick, read_csv_rows
class TokenSpendRow:
provider: str
model: str
tokens_in: int
tokens_out: int
tokens_in: int | None
tokens_out: int | None
cost: float
currency: str
session_id: str | None
recorded_at: datetime
def parse_optional_token_count(value: str) -> int | None:
"""Parse a token quantity. Empty is unknown; 0 is measured none."""
if not value:
return None
try:
count = int(value)
except ValueError as error:
raise ValueError(f"token count must be an integer, got {value!r}") from error
if count < 0:
raise ValueError(f"token count cannot be negative: {count}")
return count
def parse_anthropic_billing_csv(path: Path, *, default_currency: str = "USD") -> list[TokenSpendRow]:
rows: list[TokenSpendRow] = []
for row in read_csv_rows(path):
@ -28,8 +42,12 @@ def parse_anthropic_billing_csv(path: Path, *, default_currency: str = "USD") ->
cost_raw = pick(row, "cost", "amount", "total_cost", "usage_cost_usd")
if not model or not cost_raw:
continue
tokens_in = int(pick(row, "input_tokens", "tokens_in", "prompt_tokens") or "0")
tokens_out = int(pick(row, "output_tokens", "tokens_out", "completion_tokens") or "0")
tokens_in = parse_optional_token_count(
pick(row, "input_tokens", "tokens_in", "prompt_tokens")
)
tokens_out = parse_optional_token_count(
pick(row, "output_tokens", "tokens_out", "completion_tokens")
)
recorded_raw = pick(row, "date", "usage_date", "recorded_at", "timestamp")
recorded_at = (
datetime.fromisoformat(recorded_raw.replace("Z", "+00:00"))

View file

@ -69,8 +69,8 @@ class TokenSpend(Base, TimestampMixin):
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
provider: Mapped[str] = mapped_column(String(32), nullable=False, index=True)
model: Mapped[str] = mapped_column(String(64), nullable=False)
tokens_in: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
tokens_out: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
tokens_in: Mapped[int | None] = mapped_column(Integer, nullable=True)
tokens_out: Mapped[int | None] = mapped_column(Integer, nullable=True)
cost: Mapped[float] = mapped_column(Float, nullable=False, default=0.0)
session_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
recorded_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)

View file

@ -2,7 +2,7 @@ from pathlib import Path
import pytest
from fin_hub.ingest.anthropic import parse_anthropic_billing_csv
from fin_hub.ingest.anthropic import parse_anthropic_billing_csv, parse_optional_token_count
from fin_hub.ingest.cloud import parse_cloud_cost_csv
from fin_hub.ingest.hosteurope import parse_hosteurope_csv
@ -21,6 +21,43 @@ def test_parse_anthropic_billing_csv():
assert len(rows) == 2
assert rows[0].provider == "anthropic"
assert rows[0].tokens_in == 12000
assert rows[0].tokens_out == 3000
def test_parse_optional_token_count_treats_empty_as_unknown():
assert parse_optional_token_count("") is None
assert parse_optional_token_count("0") == 0
assert parse_optional_token_count("12") == 12
with pytest.raises(ValueError, match="integer"):
parse_optional_token_count("n/a")
with pytest.raises(ValueError, match="negative"):
parse_optional_token_count("-1")
def test_parse_anthropic_cost_only_row_does_not_invent_zero_tokens(tmp_path: Path):
source = tmp_path / "anthropic-cost-only.csv"
source.write_text(
"model,cost,currency,usage_date\n"
"claude-opus-4-20250514,20.00,USD,2026-08-01\n",
encoding="utf-8",
)
row = parse_anthropic_billing_csv(source)[0]
assert row.cost == 20.0
assert row.tokens_in is None
assert row.tokens_out is None
def test_parse_anthropic_explicit_zero_tokens_remain_zero(tmp_path: Path):
source = tmp_path / "anthropic-zero-tokens.csv"
source.write_text(
"model,input_tokens,output_tokens,cost,currency,usage_date\n"
"claude-haiku-3-20240307,0,0,0.00,USD,2026-08-02\n",
encoding="utf-8",
)
row = parse_anthropic_billing_csv(source)[0]
assert row.tokens_in == 0
assert row.tokens_out == 0
assert row.cost == 0.0
def test_parse_hosteurope_csv():

View file

@ -20,6 +20,20 @@ from fin_hub.services.ledger import (
FIXTURES = Path(__file__).parent / "fixtures"
def test_ledger_import_anthropic_cost_only_does_not_require_tokens(tmp_path: Path):
ledger = tmp_path / "ledger.db"
source = tmp_path / "anthropic-cost-only.csv"
source.write_text(
"model,cost,currency,usage_date\nclaude-opus-4-20250514,20.00,USD,2026-08-01\n",
encoding="utf-8",
)
result = import_csv(source, "anthropic", ledger_path=ledger)
assert result.rows_imported == 1
rollups = monthly_summary(ledger_path=ledger)
assert rollups[0].currency == "USD"
assert rollups[0].total == pytest.approx(20.0)
def test_ledger_import_and_monthly_summary(tmp_path: Path):
ledger = tmp_path / "ledger.db"
set_opening_balance(ledger, 12000.0)

View file

@ -40,6 +40,14 @@ def test_model_classes_importable():
assert EngagementPrice.__tablename__ == "fin_engagement_prices"
def test_token_spend_token_counts_are_nullable():
columns = TokenSpend.__table__.columns
assert columns["tokens_in"].nullable is True
assert columns["tokens_out"].nullable is True
assert columns["tokens_in"].default is None
assert columns["tokens_out"].default is None
def test_service_cost_has_external_attribution_seam():
columns = ServiceCost.__table__.columns
assert columns["client_id"].nullable is True

View file

@ -4,7 +4,7 @@ type: workplan
title: "AI plan cost, token estimates, and effectiveness"
domain: financials
repo: fin-hub
status: proposed
status: active
owner: grok
topic_slug: financials
created: "2026-08-15"
@ -12,7 +12,9 @@ updated: "2026-08-15"
related:
- FIN-WP-0002
- FIN-WP-0004
- FIN-WP-0006
- STATE-WP-0045
- RESOURCE-WP-0003
- ITC-CAP-0.2.0
origin: assessment
origin_ref: "info-tech-canon 0.3.0 review — AI token data-gap"
@ -44,22 +46,34 @@ wait for a new commerce or intelligence domain model.
## Boundaries
Checked 2026-08-15 against resource-control SCOPE, INTENT, procurement
terms §4.1/§6, their v0.1 ratification, evidence-basis vocabulary, and
the backup capability restatement. Design note:
`docs/ai-plan-token-effectiveness.md`. Contract v0.1 is not silently
amended; the addendum in that note is offered, not assumed.
| Concept | Authoritative writer | Fin-hub role |
| --- | --- | --- |
| Provider invoice / plan subscription charge | fin-hub | Book once as `booked_cost` |
| Recurring plan as a financial commitment | fin-hub | Record cadence and amount; not usage |
| Plan entitlement (included tokens or plan capacity) | fin-hub | Commercial fact, same discipline as engagement prices |
| Session token evidence (`measured` / `allocated` / `estimated` / `superseded`) | State Hub (`STATE-WP-0045`) | Consume aggregates; do not re-parse sessions |
| Technical infrastructure usage and resource identity | resource-control | Out of scope here (class `S`/`C`/`P`, `FIN-WP-0004`) |
| Intelligence / infrastructure `resource_id` | resource-control | Join when they publish one; never invent |
| Provision-level class `I` usage | resource-control (gap they already own) | Consume via `FIN-WP-0004` when emitted |
| Session token evidence (`measured` / `allocated` / `estimated` / `superseded`) | State Hub (`STATE-WP-0045`) | Join as session-token evidence; do not store as `usage_observation` |
| Technical allocation of a portfolio resource | resource-control | Consume `AllocationEvidence`; do not emit it |
| Work done (tasks, progress, outcomes) | State Hub / domain repos | Join key for effectiveness; not a cost driver |
| Reporting allocation of a plan invoice across work | fin-hub | Local `FIN-WP-0002`-family derivation |
| Implied €/token and work-per-token / work-per-euro | fin-hub | Reporting overlay; never a booked price |
| Vendor parsers, transcripts, model names, list prices | State Hub / llm-connect / operators | Not invented here |
Reuse `docs/fin-resource-authority-contract-v0.1.md` shapes
(`booked_cost`, `usage_observation`, `AllocationEvidence`). Resource-control
is the wrong *producer* for AI-session usage; the *envelope* is the right
one. A new AI-plan producer (State Hub aggregates, plus fin-hub's own
entitlement and booked facts) fills that role.
Reuse the *discipline* of contract v0.1 (unknown is not zero, residual
visible, estimates are not booked). Do not impersonate resource-control
record types for agent-session telemetry or work-effectiveness shares.
Founder-private AI plans may be booked for runway. They stay
`resource_id=null` and must not silent-default `financial_entity_id`
until the spend is booked to an entity and, if Railiance-procured, a
resource.
## Non-goals
@ -67,6 +81,9 @@ entitlement and booked facts) fills that role.
- Completing the Scaleway `platform:audit-storage` booked-cost round trip
(`FIN-WP-0004-T05`). That is class `S`, not class `I`.
- Owning session parsers or storing transcripts.
- Inventing a `resource_id` or metering tokens against a provision.
- Emitting `usage_observation` or `AllocationEvidence` as if this repo
were resource-control.
- A canonical token↔hour or token↔euro exchange rate (ITC-CAP forbids
asserting one; substitution is observed from the time series).
- A new definition of “work” beyond joining to existing task / progress
@ -99,7 +116,7 @@ estimator method starts a new series.
```task
id: FIN-WP-0007-T00
status: todo
status: done
priority: high
state_hub_task_id: "5e9bccfd-569a-43a2-b016-afed760add49"
```
@ -117,11 +134,21 @@ Done when the note is committed, `SCOPE.md` matches the boundary, and
no task below has an ambiguous owner for booked cost, entitlement,
session tokens, or work.
Completed 2026-08-15: boundary-checked against resource-control SCOPE,
INTENT, Terms §4.1/§6, their contract ratification, evidence-basis, and
the backup restatement. They own intelligence *resource* identity and
provision-level class `I` metering (not yet implemented). Session
telemetry and work-effectiveness stay outside that. Note:
`docs/ai-plan-token-effectiveness.md`. `SCOPE.md` and `INTENT.md`
updated. T04/T05 rewritten so fin-hub does not impersonate
`usage_observation` or `AllocationEvidence`. Proposed v0.1 addendum
offered to resource-control; not assumed.
## Stop treating missing tokens as zero
```task
id: FIN-WP-0007-T01
status: todo
status: done
priority: high
state_hub_task_id: "aefa287e-0e4d-40b1-adf6-a8e6df7caeb2"
```
@ -139,6 +166,11 @@ Done when the existing fixture still parses, a missing-token fixture
does not emit zeros, and CAP-R8 (`unknown` never stored as zero) holds
on the token path.
Completed 2026-08-15: `parse_optional_token_count` treats empty as
unknown and `0` as measured none. `TokenSpend.tokens_in/out` are
nullable with no zero default. Ledger import of a cost-only Anthropic
row still books the amount and does not invent token quantities.
## Book AI-plan invoices as facts and commitments
```task
@ -186,7 +218,7 @@ invent a token ceiling.
Done when a plan month can be reported as booked euros + entitled
capacity (or explicit unknown) without reading session logs.
## Consume State Hub token aggregates as usage observations
## Join State Hub session-token aggregates
```task
id: FIN-WP-0007-T04
@ -195,21 +227,28 @@ priority: high
state_hub_task_id: "29f1804f-a5ad-46b6-b9d7-8f074e31b04e"
```
After T00, ingest State Hub token summaries as `usage_observation`
records (or a compatible typed sibling). Preserve `measurement_kind`
(`measured`, `allocated`, `estimated`, `superseded`), period, model,
repo/workplan association, confidence, and coverage. Do not re-parse
Codex/Claude/Cursor/Grok session files in this repo.
After T00, ingest State Hub token summaries as **fin-hub session-token
evidence**, not as resource-control `usage_observation`. Preserve
`measurement_kind` (`measured`, `allocated`, `estimated`,
`superseded`), period, model, repo/workplan association, confidence,
and coverage. Map kinds onto the resource-control evidence-basis
vocabulary (`measured` / `derived` / `estimated` / excluded /
`unknown`). Do not re-parse Codex, Claude, Cursor, or Grok session
files. Do not invent a `resource_id`.
If resource-control later inventories a Railiance-procured
intelligence resource and emits `usage_observation` for it, consume
that through the existing `FIN-WP-0004` planning-evidence path.
Superseded rows must not enter totals. Estimated and measured must
remain separable. Months with no State Hub events produce an explicit
unknown residual, not a zero-consumption observation.
Done when a recorded fixture of mixed measured/estimated/superseded
aggregates stores as usage evidence, reconciles by kind, and cannot be
posted as booked spend.
aggregates stores as session-token evidence, reconciles by kind, and
cannot be posted as booked spend or as resource-control usage.
## Allocate booked plan cost across work
## Reporting-allocate booked plan cost across work
```task
id: FIN-WP-0007-T05
@ -219,17 +258,23 @@ state_hub_task_id: "db098143-90b7-475c-8846-e4c274e50b4a"
```
After T02 and T04, allocate each AI-plan `financial_fact_id` across
repos/workplans using `AllocationEvidence`: named method, shares, and
explicit residual. Allowed first methods: measured-token share,
measured+estimated-token share, session-count share, even split.
Method and estimator version are part of the record identity.
repos/workplans as a **fin-hub reporting allocation** (same family as
`FIN-WP-0002` billing basis). This is not resource-control
`AllocationEvidence` and must not be posted as technical allocation.
Shares plus residual must equal 1. Unattributed and unmeasured shares
stay visible. Allocation never writes a new booked fact.
Allowed first methods: measured-token share, measured+estimated-token
share, session-count share, even split. Method and estimator version
are part of the record identity. Shares plus residual must equal 1.
Unattributed and unmeasured shares stay visible. Allocation never
writes a new booked fact.
Technical allocation of a portfolio resource remains
resource-control's job and only applies when they have published a
`resource_id`.
Done when a one-plan, two-workplan fixture reconciles to the booked
effective amount, names its method, and keeps residual when coverage
is incomplete.
effective amount, names its method, keeps residual when coverage is
incomplete, and is not stored as `AllocationEvidence`.
## Publish an effectiveness report