diff --git a/workplans/FIN-WP-0007-ai-plan-token-effectiveness.md b/workplans/FIN-WP-0007-ai-plan-token-effectiveness.md new file mode 100644 index 0000000..0e6c844 --- /dev/null +++ b/workplans/FIN-WP-0007-ai-plan-token-effectiveness.md @@ -0,0 +1,278 @@ +--- +id: FIN-WP-0007 +type: workplan +title: "AI plan cost, token estimates, and effectiveness" +domain: financials +repo: fin-hub +status: proposed +owner: grok +topic_slug: financials +created: "2026-08-15" +updated: "2026-08-15" +related: + - FIN-WP-0002 + - FIN-WP-0004 + - STATE-WP-0045 + - ITC-CAP-0.2.0 +origin: assessment +origin_ref: "info-tech-canon 0.3.0 review — AI token data-gap" +--- + +# FIN-WP-0007 — AI plan cost, token estimates, and effectiveness + +## Goal + +Build a provenance-bearing dataset that lets fin-hub compare **what AI +plans cost**, **what token capacity they entitle**, **what was actually +or estimably consumed**, and **what work was delivered**, without treating +missing session usage as zero and without turning estimates into booked +spend. + +The motivating failure is operational, not theoretical: most coding-agent +sessions do not emit trustworthy token counts or token costs. The facts +that *do* exist are monthly plan invoices, plan entitlements, occasional +provider or State Hub measurements, and independent work outcomes. Those +must stay distinct and joinable so effectiveness can be watched over +time. + +This workplan applies the InfoTechCanon 0.3.0 / ITC-CAP 0.2.0 split +already used for infrastructure in `FIN-WP-0004`: booked euros are not +usage; usage is recorded in native units (`token` for class `I`); +`unknown` is never stored as `0`; currency is an overlay. It does **not** +wait for a new commerce or intelligence domain model. + +## Boundaries + +| Concept | Authoritative writer | Fin-hub role | +| --- | --- | --- | +| Provider invoice / plan subscription charge | fin-hub | Book once as `booked_cost` | +| Recurring plan as a financial commitment | fin-hub | Record cadence and amount; not usage | +| Plan entitlement (included tokens or plan capacity) | fin-hub | Commercial fact, same discipline as engagement prices | +| Session token evidence (`measured` / `allocated` / `estimated` / `superseded`) | State Hub (`STATE-WP-0045`) | Consume aggregates; do not re-parse sessions | +| Technical infrastructure usage and resource identity | resource-control | Out of scope here (class `S`/`C`/`P`, `FIN-WP-0004`) | +| Work done (tasks, progress, outcomes) | State Hub / domain repos | Join key for effectiveness; not a cost driver | +| Implied €/token and work-per-token / work-per-euro | fin-hub | Reporting overlay; never a booked price | +| Vendor parsers, transcripts, model names, list prices | State Hub / llm-connect / operators | Not invented here | + +Reuse `docs/fin-resource-authority-contract-v0.1.md` shapes +(`booked_cost`, `usage_observation`, `AllocationEvidence`). Resource-control +is the wrong *producer* for AI-session usage; the *envelope* is the right +one. A new AI-plan producer (State Hub aggregates, plus fin-hub's own +entitlement and booked facts) fills that role. + +## Non-goals + +- Legal invoices, DATEV postings, or payment execution (`FIN-WP-0005`). +- Completing the Scaleway `platform:audit-storage` booked-cost round trip + (`FIN-WP-0004-T05`). That is class `S`, not class `I`. +- Owning session parsers or storing transcripts. +- A canonical token↔hour or token↔euro exchange rate (ITC-CAP forbids + asserting one; substitution is observed from the time series). +- A new definition of “work” beyond joining to existing task / progress + outcomes. +- Editing InfoTechCanon inline. Canon pressure is a demand message, not a + kernel patch from this repo. + +## Measurement layers + +Keep these four quantities in separate records, joined by period, +provider/plan, and optional workplan/repo: + +1. **Booked plan cost** — euros for the subscription month. +2. **Entitlement** — included tokens or declared plan capacity. +3. **Consumption of class `I`** — tokens, each row marked + `measured` | `inferred` | `unknown`. Missing is `unknown`, never `0`. +4. **Work done** — tasks or accepted outcomes in the same period. + +Derived overlays, never booked facts: + +- implied €/token = booked ÷ (measured + estimated + residual) +- effectiveness = work / measured tokens, work / (measured+estimated) + tokens, and work / euro +- coverage = measured / (measured + estimated + residual) + +A month with worse coverage must not look more efficient. Changing an +estimator method starts a new series. + +## Record the authority and measurement split + +```task +id: FIN-WP-0007-T00 +status: todo +priority: high +``` + +Write a short design note in `docs/` that records the four-layer split, +the authority table above, reuse of the v0.1 exchange envelopes, and +the State Hub join (token aggregates by +`measurement_kind`, period, repo/workplan). Confirm `SCOPE.md` already +covers token-spend models; if the current wording implies session-metered +CSV rows are the only token surface, update it so plan invoices, +entitlements, estimated usage, and effectiveness reports are in scope +while session parsing stays out. + +Done when the note is committed, `SCOPE.md` matches the boundary, and +no task below has an ambiguous owner for booked cost, entitlement, +session tokens, or work. + +## Stop treating missing tokens as zero + +```task +id: FIN-WP-0007-T01 +status: todo +priority: high +``` + +After T00, fix `TokenSpend` and `parse_anthropic_billing_csv` so absent +token fields are `null` / omitted, not `0`. `0` means measured and none. +Align ledger import: token quantities that are unknown must not become +zero ledger attributes, and a cost-only row must remain a cost-only row. + +Add regression tests for missing `input_tokens` / `output_tokens`, +cost-only provider exports, and the invariant that incomplete months +cannot under-report consumption as zero. + +Done when the existing fixture still parses, a missing-token fixture +does not emit zeros, and CAP-R8 (`unknown` never stored as zero) holds +on the token path. + +## Book AI-plan invoices as facts and commitments + +```task +id: FIN-WP-0007-T02 +status: todo +priority: high +``` + +After T00, ingest monthly AI-plan / subscription charges as +`booked_cost` facts (one current fact per provider-account × plan × +accounting period) and as recurring `Commitment` rows where the plan is +ongoing. Use the existing financial-fact identity, fingerprint, +correction, and unknown-tax rules from `FIN-WP-0004`. + +Do not explode a flat monthly invoice into synthetic per-session costs. +Session-level euros, if shown at all, come from a later allocation +(T05), not from the booked ledger. + +Done when a fixture for at least two providers (for example a flat +subscription and a usage-top-up) books exactly once per period, +corrects append-only, and appears on burn/runway like any other +commitment. + +## Record plan entitlements + +```task +id: FIN-WP-0007-T03 +status: todo +priority: high +``` + +After T02, add an append-only plan-entitlement record: provider, plan +identity, period, entitled quantity, unit (`token` or a declared plan +capacity such as a multiplier), source provenance, and revision link. +Follow the engagement-price discipline from `FIN-WP-0002-T02`: a +revision supersedes, a price or capacity is not an invoice, currency +mismatches cannot silently join. + +When entitled tokens are themselves unknown (common for “Max 20×” +style plans), store `unknown` plus the declared plan label. Do not +invent a token ceiling. + +Done when a plan month can be reported as booked euros + entitled +capacity (or explicit unknown) without reading session logs. + +## Consume State Hub token aggregates as usage observations + +```task +id: FIN-WP-0007-T04 +status: todo +priority: high +``` + +After T00, ingest State Hub token summaries as `usage_observation` +records (or a compatible typed sibling). Preserve `measurement_kind` +(`measured`, `allocated`, `estimated`, `superseded`), period, model, +repo/workplan association, confidence, and coverage. Do not re-parse +Codex/Claude/Cursor/Grok session files in this repo. + +Superseded rows must not enter totals. Estimated and measured must +remain separable. Months with no State Hub events produce an explicit +unknown residual, not a zero-consumption observation. + +Done when a recorded fixture of mixed measured/estimated/superseded +aggregates stores as usage evidence, reconciles by kind, and cannot be +posted as booked spend. + +## Allocate booked plan cost across work + +```task +id: FIN-WP-0007-T05 +status: todo +priority: medium +``` + +After T02 and T04, allocate each AI-plan `financial_fact_id` across +repos/workplans using `AllocationEvidence`: named method, shares, and +explicit residual. Allowed first methods: measured-token share, +measured+estimated-token share, session-count share, even split. +Method and estimator version are part of the record identity. + +Shares plus residual must equal 1. Unattributed and unmeasured shares +stay visible. Allocation never writes a new booked fact. + +Done when a one-plan, two-workplan fixture reconciles to the booked +effective amount, names its method, and keeps residual when coverage +is incomplete. + +## Publish an effectiveness report + +```task +id: FIN-WP-0007-T06 +status: todo +priority: medium +``` + +After T03–T05, add a period report (CLI and, if cheap, the existing +read API) that emits, per plan and currency: + +- booked effective amount +- entitled capacity (or unknown) +- tokens measured / estimated / residual-unknown +- coverage ratio +- implied €/token, labelled inferred +- work joined (task counts or accepted outcomes) for the same period +- work per measured token, work per (measured+estimated) token, work + per euro — three series, not one blend + +Refuse to present a single blended “efficiency” number. A month whose +coverage is below a documented threshold is marked unfit for trend +comparison, not interpolated. + +Done when two fixture months with different coverage cannot produce a +chart in which the worse-measured month looks cheaper per unit of +work unless that is also true on the measured-only series. + +## Send canon feedback + +```task +id: FIN-WP-0007-T07 +status: todo +priority: low +``` + +After T00, send a demand signal to `info-tech-canon` (State Hub +message and/or a `demand/` note they can assimilate). Ask for: + +- a `measured | inferred | declared | unknown` qualifier on + `CapabilityConsumption` (reuse ITC-DATA lineage states; do not + invent a local dialect); +- enough `commerce.entitlement` / `commerce.metering` structure for + plan capacity and incomplete meters (OQ-5 still open for invoices + and entitlements); +- dropping or re-homing the `cost` quality dimension on + `intelligence.generation` so it does not fight ITC-CAP §4.12. + +Do not patch the canon from this repo. + +Done when the demand is sent, referenced from this workplan, and +explicitly does not block T01–T06.