tegwick
6f0eb7f7f8
Record the booked-vs-usage-vs-entitlement-vs-work split for AI plans so missing session tokens can be estimated without becoming booked spend.
278 lines
10 KiB
Markdown
278 lines
10 KiB
Markdown
---
|
||
id: FIN-WP-0007
|
||
type: workplan
|
||
title: "AI plan cost, token estimates, and effectiveness"
|
||
domain: financials
|
||
repo: fin-hub
|
||
status: proposed
|
||
owner: grok
|
||
topic_slug: financials
|
||
created: "2026-08-15"
|
||
updated: "2026-08-15"
|
||
related:
|
||
- FIN-WP-0002
|
||
- FIN-WP-0004
|
||
- STATE-WP-0045
|
||
- ITC-CAP-0.2.0
|
||
origin: assessment
|
||
origin_ref: "info-tech-canon 0.3.0 review — AI token data-gap"
|
||
---
|
||
|
||
# FIN-WP-0007 — AI plan cost, token estimates, and effectiveness
|
||
|
||
## Goal
|
||
|
||
Build a provenance-bearing dataset that lets fin-hub compare **what AI
|
||
plans cost**, **what token capacity they entitle**, **what was actually
|
||
or estimably consumed**, and **what work was delivered**, without treating
|
||
missing session usage as zero and without turning estimates into booked
|
||
spend.
|
||
|
||
The motivating failure is operational, not theoretical: most coding-agent
|
||
sessions do not emit trustworthy token counts or token costs. The facts
|
||
that *do* exist are monthly plan invoices, plan entitlements, occasional
|
||
provider or State Hub measurements, and independent work outcomes. Those
|
||
must stay distinct and joinable so effectiveness can be watched over
|
||
time.
|
||
|
||
This workplan applies the InfoTechCanon 0.3.0 / ITC-CAP 0.2.0 split
|
||
already used for infrastructure in `FIN-WP-0004`: booked euros are not
|
||
usage; usage is recorded in native units (`token` for class `I`);
|
||
`unknown` is never stored as `0`; currency is an overlay. It does **not**
|
||
wait for a new commerce or intelligence domain model.
|
||
|
||
## Boundaries
|
||
|
||
| Concept | Authoritative writer | Fin-hub role |
|
||
| --- | --- | --- |
|
||
| Provider invoice / plan subscription charge | fin-hub | Book once as `booked_cost` |
|
||
| Recurring plan as a financial commitment | fin-hub | Record cadence and amount; not usage |
|
||
| Plan entitlement (included tokens or plan capacity) | fin-hub | Commercial fact, same discipline as engagement prices |
|
||
| Session token evidence (`measured` / `allocated` / `estimated` / `superseded`) | State Hub (`STATE-WP-0045`) | Consume aggregates; do not re-parse sessions |
|
||
| Technical infrastructure usage and resource identity | resource-control | Out of scope here (class `S`/`C`/`P`, `FIN-WP-0004`) |
|
||
| Work done (tasks, progress, outcomes) | State Hub / domain repos | Join key for effectiveness; not a cost driver |
|
||
| Implied €/token and work-per-token / work-per-euro | fin-hub | Reporting overlay; never a booked price |
|
||
| Vendor parsers, transcripts, model names, list prices | State Hub / llm-connect / operators | Not invented here |
|
||
|
||
Reuse `docs/fin-resource-authority-contract-v0.1.md` shapes
|
||
(`booked_cost`, `usage_observation`, `AllocationEvidence`). Resource-control
|
||
is the wrong *producer* for AI-session usage; the *envelope* is the right
|
||
one. A new AI-plan producer (State Hub aggregates, plus fin-hub's own
|
||
entitlement and booked facts) fills that role.
|
||
|
||
## Non-goals
|
||
|
||
- Legal invoices, DATEV postings, or payment execution (`FIN-WP-0005`).
|
||
- Completing the Scaleway `platform:audit-storage` booked-cost round trip
|
||
(`FIN-WP-0004-T05`). That is class `S`, not class `I`.
|
||
- Owning session parsers or storing transcripts.
|
||
- A canonical token↔hour or token↔euro exchange rate (ITC-CAP forbids
|
||
asserting one; substitution is observed from the time series).
|
||
- A new definition of “work” beyond joining to existing task / progress
|
||
outcomes.
|
||
- Editing InfoTechCanon inline. Canon pressure is a demand message, not a
|
||
kernel patch from this repo.
|
||
|
||
## Measurement layers
|
||
|
||
Keep these four quantities in separate records, joined by period,
|
||
provider/plan, and optional workplan/repo:
|
||
|
||
1. **Booked plan cost** — euros for the subscription month.
|
||
2. **Entitlement** — included tokens or declared plan capacity.
|
||
3. **Consumption of class `I`** — tokens, each row marked
|
||
`measured` | `inferred` | `unknown`. Missing is `unknown`, never `0`.
|
||
4. **Work done** — tasks or accepted outcomes in the same period.
|
||
|
||
Derived overlays, never booked facts:
|
||
|
||
- implied €/token = booked ÷ (measured + estimated + residual)
|
||
- effectiveness = work / measured tokens, work / (measured+estimated)
|
||
tokens, and work / euro
|
||
- coverage = measured / (measured + estimated + residual)
|
||
|
||
A month with worse coverage must not look more efficient. Changing an
|
||
estimator method starts a new series.
|
||
|
||
## Record the authority and measurement split
|
||
|
||
```task
|
||
id: FIN-WP-0007-T00
|
||
status: todo
|
||
priority: high
|
||
```
|
||
|
||
Write a short design note in `docs/` that records the four-layer split,
|
||
the authority table above, reuse of the v0.1 exchange envelopes, and
|
||
the State Hub join (token aggregates by
|
||
`measurement_kind`, period, repo/workplan). Confirm `SCOPE.md` already
|
||
covers token-spend models; if the current wording implies session-metered
|
||
CSV rows are the only token surface, update it so plan invoices,
|
||
entitlements, estimated usage, and effectiveness reports are in scope
|
||
while session parsing stays out.
|
||
|
||
Done when the note is committed, `SCOPE.md` matches the boundary, and
|
||
no task below has an ambiguous owner for booked cost, entitlement,
|
||
session tokens, or work.
|
||
|
||
## Stop treating missing tokens as zero
|
||
|
||
```task
|
||
id: FIN-WP-0007-T01
|
||
status: todo
|
||
priority: high
|
||
```
|
||
|
||
After T00, fix `TokenSpend` and `parse_anthropic_billing_csv` so absent
|
||
token fields are `null` / omitted, not `0`. `0` means measured and none.
|
||
Align ledger import: token quantities that are unknown must not become
|
||
zero ledger attributes, and a cost-only row must remain a cost-only row.
|
||
|
||
Add regression tests for missing `input_tokens` / `output_tokens`,
|
||
cost-only provider exports, and the invariant that incomplete months
|
||
cannot under-report consumption as zero.
|
||
|
||
Done when the existing fixture still parses, a missing-token fixture
|
||
does not emit zeros, and CAP-R8 (`unknown` never stored as zero) holds
|
||
on the token path.
|
||
|
||
## Book AI-plan invoices as facts and commitments
|
||
|
||
```task
|
||
id: FIN-WP-0007-T02
|
||
status: todo
|
||
priority: high
|
||
```
|
||
|
||
After T00, ingest monthly AI-plan / subscription charges as
|
||
`booked_cost` facts (one current fact per provider-account × plan ×
|
||
accounting period) and as recurring `Commitment` rows where the plan is
|
||
ongoing. Use the existing financial-fact identity, fingerprint,
|
||
correction, and unknown-tax rules from `FIN-WP-0004`.
|
||
|
||
Do not explode a flat monthly invoice into synthetic per-session costs.
|
||
Session-level euros, if shown at all, come from a later allocation
|
||
(T05), not from the booked ledger.
|
||
|
||
Done when a fixture for at least two providers (for example a flat
|
||
subscription and a usage-top-up) books exactly once per period,
|
||
corrects append-only, and appears on burn/runway like any other
|
||
commitment.
|
||
|
||
## Record plan entitlements
|
||
|
||
```task
|
||
id: FIN-WP-0007-T03
|
||
status: todo
|
||
priority: high
|
||
```
|
||
|
||
After T02, add an append-only plan-entitlement record: provider, plan
|
||
identity, period, entitled quantity, unit (`token` or a declared plan
|
||
capacity such as a multiplier), source provenance, and revision link.
|
||
Follow the engagement-price discipline from `FIN-WP-0002-T02`: a
|
||
revision supersedes, a price or capacity is not an invoice, currency
|
||
mismatches cannot silently join.
|
||
|
||
When entitled tokens are themselves unknown (common for “Max 20×”
|
||
style plans), store `unknown` plus the declared plan label. Do not
|
||
invent a token ceiling.
|
||
|
||
Done when a plan month can be reported as booked euros + entitled
|
||
capacity (or explicit unknown) without reading session logs.
|
||
|
||
## Consume State Hub token aggregates as usage observations
|
||
|
||
```task
|
||
id: FIN-WP-0007-T04
|
||
status: todo
|
||
priority: high
|
||
```
|
||
|
||
After T00, ingest State Hub token summaries as `usage_observation`
|
||
records (or a compatible typed sibling). Preserve `measurement_kind`
|
||
(`measured`, `allocated`, `estimated`, `superseded`), period, model,
|
||
repo/workplan association, confidence, and coverage. Do not re-parse
|
||
Codex/Claude/Cursor/Grok session files in this repo.
|
||
|
||
Superseded rows must not enter totals. Estimated and measured must
|
||
remain separable. Months with no State Hub events produce an explicit
|
||
unknown residual, not a zero-consumption observation.
|
||
|
||
Done when a recorded fixture of mixed measured/estimated/superseded
|
||
aggregates stores as usage evidence, reconciles by kind, and cannot be
|
||
posted as booked spend.
|
||
|
||
## Allocate booked plan cost across work
|
||
|
||
```task
|
||
id: FIN-WP-0007-T05
|
||
status: todo
|
||
priority: medium
|
||
```
|
||
|
||
After T02 and T04, allocate each AI-plan `financial_fact_id` across
|
||
repos/workplans using `AllocationEvidence`: named method, shares, and
|
||
explicit residual. Allowed first methods: measured-token share,
|
||
measured+estimated-token share, session-count share, even split.
|
||
Method and estimator version are part of the record identity.
|
||
|
||
Shares plus residual must equal 1. Unattributed and unmeasured shares
|
||
stay visible. Allocation never writes a new booked fact.
|
||
|
||
Done when a one-plan, two-workplan fixture reconciles to the booked
|
||
effective amount, names its method, and keeps residual when coverage
|
||
is incomplete.
|
||
|
||
## Publish an effectiveness report
|
||
|
||
```task
|
||
id: FIN-WP-0007-T06
|
||
status: todo
|
||
priority: medium
|
||
```
|
||
|
||
After T03–T05, add a period report (CLI and, if cheap, the existing
|
||
read API) that emits, per plan and currency:
|
||
|
||
- booked effective amount
|
||
- entitled capacity (or unknown)
|
||
- tokens measured / estimated / residual-unknown
|
||
- coverage ratio
|
||
- implied €/token, labelled inferred
|
||
- work joined (task counts or accepted outcomes) for the same period
|
||
- work per measured token, work per (measured+estimated) token, work
|
||
per euro — three series, not one blend
|
||
|
||
Refuse to present a single blended “efficiency” number. A month whose
|
||
coverage is below a documented threshold is marked unfit for trend
|
||
comparison, not interpolated.
|
||
|
||
Done when two fixture months with different coverage cannot produce a
|
||
chart in which the worse-measured month looks cheaper per unit of
|
||
work unless that is also true on the measured-only series.
|
||
|
||
## Send canon feedback
|
||
|
||
```task
|
||
id: FIN-WP-0007-T07
|
||
status: todo
|
||
priority: low
|
||
```
|
||
|
||
After T00, send a demand signal to `info-tech-canon` (State Hub
|
||
message and/or a `demand/` note they can assimilate). Ask for:
|
||
|
||
- a `measured | inferred | declared | unknown` qualifier on
|
||
`CapabilityConsumption` (reuse ITC-DATA lineage states; do not
|
||
invent a local dialect);
|
||
- enough `commerce.entitlement` / `commerce.metering` structure for
|
||
plan capacity and incomplete meters (OQ-5 still open for invoices
|
||
and entitlements);
|
||
- dropping or re-homing the `cost` quality dimension on
|
||
`intelligence.generation` so it does not fight ITC-CAP §4.12.
|
||
|
||
Do not patch the canon from this repo.
|
||
|
||
Done when the demand is sent, referenced from this workplan, and
|
||
explicitly does not block T01–T06.
|