target-revenue/workplans/TREV-WP-0006-trust-service-implementation.md
tegwick b7b985d96d Implement hosted Breach/Compliance Record publication (WP-0006-T09)
migrations/0004_breach_records.sql models a case's lifecycle as
append-only events (alleged/cured/determined/terminated) grouped by
case_id rather than one mutable row - resolution is always a new,
later event, never an edit. A CHECK constraint makes the
anonymized-default rule (License V1C1 §7.4) a database fact:
named_entitlement_holder can be set if and only if anonymized = false.

src/target_revenue/breach_record.py's publish_breach_event() enforces
per-Licensor phase ownership and rejects named-disclosure requests
that don't also set named_disclosure_authorized_under_cua: true - the
Trust Service records the Licensor's assertion that the CUA's naming
clause authorizes it, it never verifies the underlying CUA text
itself. Signs every event with the same instance Ed25519 key already
used for Ledger entries and Attestations.

Adds POST/GET /phases/{id}/breach-records. Guarded the .registry
import behind a lazy in-function import (matching attestation.py's
TYPE_CHECKING pattern) so tests/test_breach_record.py (7 tests) runs
under plain system Python with no psycopg dependency. 5 new
Docker-gated tests cover the default-anonymized lifecycle, the
named-disclosure authorization gate, cross-Licensor rejection,
signature verification, and DB-level UPDATE/DELETE rejection.

This closes WP-0006 again - all 9 tasks done.
2026-07-29 22:23:41 +02:00

22 KiB
Raw Blame History

id type title domain repo status owner topic_slug created updated state_hub_workstream_id
TREV-WP-0006 workplan Trust Service reference implementation (PRD Phase 4b) infotech target-revenue finished claude infotech 2026-07-29 2026-07-29 03f63e7e-c17a-4dcc-aa34-a79824dafcf3

Trust Service reference implementation (PRD Phase 4b)

Builds the hosted Trust Service that WP-0002 deliberately deferred: Phase Registry, Extension Registry, hosted Target Ledger, Metrics, and Conversion Attestation publication, serving multiple repos rather than the single offline golden-Phase package. This is the piece explicitly named in SCOPE.md §3 as out of scope until now.

Unblocked as of 2026-07-29: SCOPE.md §4 rule 2 required (a) WP-0003's core extract deliverables to exist, and (b) working defaults for Longstop, currency, recognition, and minimum evidence to be published, before full Trust Service implementation could start. Both conditions are now met (WP-0003 finished 2026-07-29; specs/OpenQuestions-WorkingDefaults.md published since Stage 0). This workplan is the first to actually start that implementation.

Why now: the midterm goal is to govern and monetize the evolution of repos across the coulomb Forgejo org's product lines (coulomb-loop, net-kingdom, helix-forge, the railiance-* family) — that requires a real, multi-repo-capable Trust Service, not the single-Phase offline library WP-0002 produced.

Does not include: accepting real payments or tracking a real Phase for any production repo — see workplans/TREV-WP-0008-governance-and-pilot-rollout.md for the human-gated go-live step. This workplan builds the infrastructure; it does not turn it on for anyone's real money.

Trust Service Product Requirements Document

id: TREV-WP-0006-T01
status: done
priority: high
state_hub_task_id: "c98a1871-ae5c-4c91-b59c-88a6e4ccc193"

Produce specs/TrustServiceProductRequirementsDocument.md: scope, stakeholders, and requirements for the hosted service, building directly on specs/TechnicalSpecificationDocument.md §4 (component architecture) and §3's schemas. Must state explicitly which of WP-0002's Stage 0 guarantees (determinism, offline verifiability, no Trust Service discretion over conversion) the hosted service inherits unchanged versus which Stage-0-only constraints (single golden Phase, no auth, no multi-tenancy) it must now lift.

Result: specs/TrustServiceProductRequirementsDocument.md produced. Nine stakeholders identified (including Enforcement Partner, a category that did not exist at framework-PRD-drafting time). §3 states five inherited-unchanged Stage 0 guarantees (determinism, offline verifiability, no discretionary conversion authority, append-only ledger semantics, closed core vocabulary); §4 tables ten Stage-0-only constraints that must lift, mapped to their owning T03T08 task, plus one flagged gap: no WP-0006 task currently owns the Breach/Compliance Record hosting component (TSD §4.1's 2026-07-29 addition, post-dates WP-0002) — recommend either a new T09 or folding it into T06 before implementation starts. Nine functional requirements (TS-FR-19) elaborate framework PRD FR-8/9/10 for a hosted, multi-tenant, authenticated context without redefining any schema field.

Stack and storage ADR (hosted service — distinct scope from ADR-0001)

id: TREV-WP-0006-T02
status: done
priority: high
human_accept_required: true
human_accepted_by: Bernd
human_accepted_at: "2026-07-29"
state_hub_task_id: "06eabd46-5de5-4d80-9bb7-c70ef6d80df9"

docs/adr/ADR-0001-stage0-library-stack.md explicitly scoped out hosting, storage, and multi-tenancy decisions (its Revisit trigger names this exact moment). Record a new ADR for: storage engine for the append-only Target Ledger at multi-repo scale, authentication/authorization model for Phase Registry writes, and hosting/deployment target — railiance-infra and railiance-cluster are the existing platform-infrastructure repos in this Forgejo org and are the natural first candidates to investigate, not assumed by default.

Human accept gate: per the same policy as ADR-0001 — agents may draft, must not treat as accepted without explicit maintainer sign-off.

Result: Accepted 2026-07-29 by the maintainer (Bernd), no changes requested. docs/adr/ADR-0002-hosted-trust-service-stack.md: PostgreSQL (append-only enforced at the database-grant/trigger level, not just convention) for storage; per-Licensor API token for write/read-grant access control, decoupled from the existing Ed25519 per-entry signature; railiance-cluster's existing k3s/Helm/GitOps baseline (on the railiance-infra-provisioned HostEurope server) for hosting, rather than new infrastructure; Python reusing src/target_revenue/ directly behind a thin HTTP layer. T03T08 may now build against this decision.

Phase Registry and Extension Registry hosting

id: TREV-WP-0006-T03
status: done
priority: high
state_hub_task_id: "fb9f00e5-5755-44e4-ad07-ba0dc73596b8"

Implement hosted versions of the Phase Registry and Extension Registry components (specs/TechnicalSpecificationDocument.md §4.1), serving multiple Phase Manifests and extension registrations across repos rather than one golden fixture. Reject non-conformant manifests/extensions at registration, exactly as the Stage 0 validators already do — this task adds multi-tenancy and persistence, not new validation logic.

Result: Implemented per ADR-0002. migrations/0001_registries.sql creates licensors, phase_manifests, extensions tables, a trf_app role with no UPDATE/DELETE grant on phase_manifests or extensions (database-enforced append-only, not just application convention), and a set_extension_status() SECURITY DEFINER function as the only sanctioned path to canonical/deprecated — canonicalization is therefore a recorded, attributable governance action the application role cannot perform via ordinary UPDATE. src/target_revenue/registry.py wraps the existing validation.py checks (no new validation logic) with persistence and per-Licensor token authentication. src/target_revenue/service/app.py exposes a thin FastAPI surface (POST/GET /phases, POST/GET /extensions). Added a service/service-dev optional-dependency group (pyproject.toml) so the core offline library keeps zero new hard dependencies. tests/test_registry_hosting.py (7 tests, requires Docker, auto-skips otherwise) spins an ephemeral, disposable Postgres container — never the shared state-hub infra-postgres-1/custodian instance — covering registration, rejection with diff, duplicate-phase_id rejection, unknown-token rejection, extension canonicalization via the governance function, and two explicit database-privilege tests proving trf_app cannot UPDATE/DELETE phase_manifests or UPDATE extensions directly. Original 36-test offline suite verified unchanged and passing with plain system Python (no service deps required).

Hosted Target Ledger append API

id: TREV-WP-0006-T04
status: done
priority: high
state_hub_task_id: "591e87d6-3b64-4f1f-b870-edf834edc522"

Implement an authenticated append API for Target Ledger entries per Phase, preserving WP-0002's append-only, hash-chained, deterministically-foldable guarantees exactly. Any party who can compute the fold offline from an export must get the same Outstanding Target as the hosted service — this is the property to test most aggressively, since it's the one a hosted service could most easily and least visibly break.

Result: migrations/0002_ledger.sql adds ledger_entries (GENERATED ALWAYS AS IDENTITY sequence for exact append order; trf_app again has no UPDATE/DELETE grant). src/target_revenue/ledger.py's append_entry(): rejects any caller-supplied previous_entry_hash/ signature outright (server-computed only), enforces phase ownership (a Licensor may only append to its own registered Phase — TS-FR-8 isolation), serializes concurrent appends per Phase via pg_advisory_xact_lock, computes the chain tip from the last stored entry and signs with the Trust Service instance's own Ed25519 key (service/keys.py, env-configured or ephemeral-with-warning for dev), reusing validation.py's existing schema/currency checks unchanged. service/app.py adds POST/GET /phases/{id}/ledger and an unauthenticated GET /public-key (so an external party can verify signatures without trusting this API's own access control at all). Fixed an unrelated route ordering bug found while wiring this in: {phase_id:path} on the plain GET /phases/{id} route was greedily matching /ledger-suffixed paths too, since phase IDs contain colons but no slashes and never needed the :path converter — switched all phase routes to plain {phase_id}. tests/test_ledger_hosting.py (8 tests, Docker-gated): hash-chain linkage across appends, forged-hash/signature rejection, cross-Licensor isolation, currency-mismatch rejection, duplicate-entry-id rejection, DB-level UPDATE/DELETE privilege checks, signature verification via the public-key endpoint, and — the task's own highest-priority property — appending a mixed development-credit/remission-credit/credit-reversal sequence through the API, exporting it, and confirming fold.fold_outstanding_target over the export reproduces the exact expected Development Credit, Remission Credit, and Outstanding Target. Offline 36-test suite re-verified unchanged with plain system Python; no stray Docker containers left running.

Metrics service

id: TREV-WP-0006-T05
status: done
priority: medium
state_hub_task_id: "6bab5e64-0091-4a25-906d-758310cbe723"

Implement the public metrics set from specs/OpenQuestions-WorkingDefaults.md Q9 (Initial Target, cumulative credits, Outstanding Target, conversion status, last checkpoint, Longstop timestamp as mandatory; velocity/forecast as recommended, clearly labeled as forecasts per concept §14.6).

Result: src/target_revenue/metrics.py's compute_metrics(manifest, entries, as_of) — pure and deterministic like fold.py (the only input that varies with wall-clock time, as_of, is an explicit parameter, never read internally), reusing fold.py/conversion.py unchanged. Returns three explicitly separated blocks per TrustServicePRD TS-FR-5: facts (Q9's mandatory set — Initial Target, cumulative Development/Remission Credit, Outstanding Target, conversion status, last ledger entry id, Longstop timestamp), calculations (target satisfaction percentage, Development/Remission Credit velocity per day, days since last material progress), and forecasts (projected conversion date, null whenever already converted or velocity is zero/negative — never populated as a disguised fact). Added GET /phases/{id}/metrics to service/app.py, unauthenticated per FR-9/FR-10. tests/test_metrics.py (6 tests, no Docker/Postgres required, runs under plain system Python — no new hard dependency) covers the mandatory-facts set, fact/calculation/forecast separation, percentage correctness, the zero-material-entries case, and as_of timezone-awareness. Added one Docker-gated test to tests/test_ledger_hosting.py proving the hosted /metrics response (fetched with no Authorization header) exactly equals compute_metrics() run offline against the same exported Manifest + Ledger. Full suite (offline 36 + metrics 6 + Docker-gated 22) verified passing; no stray containers left running.

Conversion Attestation publication

id: TREV-WP-0006-T06
status: done
priority: high
state_hub_task_id: "f08d3d73-4c75-4049-97d0-c27ae8513336"

Implement Conversion Attestation generation and publication as a hosted service, preserving the non-discretionary rule from src/target_revenue/conversion.py: the service publishes evidence of a conversion already true from the Manifest + Ledger, never a precondition for it. Any repo's own tooling must be able to recompute conversion status without querying this service at all.

Result: migrations/0003_attestations.sql adds attestations (one row per Phase, trf_app again with no UPDATE/DELETE grant — a published attestation is a permanent record, never revised). src/target_revenue/attestation.py's publish_attestation() is idempotent (an existing row is returned unchanged, never regenerated) and derives conversion_timestamp from the ledger itself — _find_conversion_prefix() walks increasing ledger prefixes to find the earliest point the fold reaches Outstanding Target = 0, rather than using the publish call's wall-clock time or blindly trusting entries[-1], so a later, unrelated ledger entry can never retroactively change an already-converged Phase's recorded conversion moment. Raises NotConvertedError (never fabricates a conversion) if the ledger never reaches zero. Reuses conversion.generate_attestation() unchanged. service/app.py adds GET /phases/{id}/attestation, unauthenticated, publishing on first observation and simply returning the stored record thereafter. tests/test_attestation.py (3 tests, no Docker/Postgres, plain system Python — required a TYPE_CHECKING-guarded psycopg import to keep it dependency-free) proves the earliest-crossing behavior explicitly, including the case where a later entry exists past the conversion point. 6 new Docker-gated tests in tests/test_ledger_hosting.py cover 404-before-conversion, the core legal-technical property (/metrics facts/calculations/forecasts are bit-for-bit identical whether or not /attestation has ever been called), one-time publication with identical output on reread, public-key signature verification, and DB-level UPDATE/DELETE rejection on attestations. Full suite verified: 45 passing offline (36 + 6 metrics + 3 attestation) under plain system Python, 30 passing under the Docker-gated suite; no stray containers left running.

Multi-repo onboarding flow

id: TREV-WP-0006-T07
status: done
priority: medium
state_hub_task_id: "44c5846e-1f37-41fe-8ac0-fc7d88628e9f"

Define and implement how a repo (e.g. coulomb-loop, net-kingdom, helix-forge, a railiance-* repo) registers a Phase against the hosted Trust Service: what it commits to its own repo (a Phase Manifest file, per specs/PhaseManifestSpecification.md) versus what it submits to the Trust Service (the registration call). Coordinate with workplans/TREV-WP-0008-governance-and-pilot-rollout.md, which selects the actual first repos — this task defines the mechanism, not the pilot selection.

Result: specs/TrustServiceOnboarding.md defines the mechanism: a Phase Manifest file is committed to the declaring repo (durable, git-history-backed, independently foldable forever) and registered with the hosted service (the multi-tenant, publicly-servable copy); once registered, the Ledger's live authoritative copy is the hosted service only, not a second competing file — a repo may keep periodic exports for its own offline re-verification, but those are copies, not a source of truth. Licensor token bootstrapping is explicitly out of this mechanism's scope (a WP-0008-T01 governance action). scripts/trf_onboard.py implements it as a dependency-light CLI (stdlib urllib + the existing target_revenue.validation, no FastAPI/psycopg required to onboard a Phase) with four subcommands: validate (offline only, no network), register-phase (validates offline before ever attempting the network call), append-entry, and status (public, no token). The Licensor API token is read only from an environment variable named on the command line, never accepted as a literal argument or committed. tests/test_trf_onboard.py (4 tests, no network, no Docker) proves both invalid-manifest and missing-token-env cases fail before any HTTP attempt is made — asserted by monkeypatching the request function to raise if called at all. tests/test_onboarding_hosted.py (1 Docker-gated test) is the one test in the suite that runs an actual uvicorn server on a real socket (rather than FastAPI's in-process TestClient, since the CLI genuinely speaks HTTP) and drives the full register → append → status round trip through the CLI exactly as an external repo would invoke it. Full suite: 49 passing offline, 71 passing with Docker; no stray containers left running.

Conformance test suite at hosted scale

id: TREV-WP-0006-T08
status: done
priority: medium
state_hub_task_id: "359c9c5e-a49a-40c1-82ea-2bf80e93e7d6"

Extend WP-0002's conformance suite to run against the hosted service: multiple concurrent Phases across different repos, the golden Phase example replayed through the hosted API rather than only the offline library, and a specific regression test that hosted and offline folds of the same exported ledger always agree.

Result: tests/test_hosted_conformance.py (6 Docker-gated tests) addresses all three named requirements directly: (1) the entire golden Phase (examples/phase-001/manifest.json + full 6-entry ledger + all 4 extensions) replayed through the hosted API reproduces exactly the 36-test-suite-established offline outcome — Development Credit 67000, Remission Credit 33000, Outstanding Target 0, future_license MIT — and the hosted Attestation matches it field-for-field; (2) two Phases under two different Licensors (standing in for coulomb-loop/net-kingdom) operate with interleaved, concurrent-style appends and are proven isolated both positively (each Phase's metrics reflect only its own entries) and negatively (a cross-Licensor write attempt is rejected); (3) a parametrized regression test across four distinct ledger shapes (credits-only, with remission, with a reversal, with both administrative correction types) asserts hosted-append-then-offline-fold always reproduces the exact expected totals, generalizing the single-shape check already added ad hoc in T04. Full suite: 49 passing offline (plain system Python, no new dependency), 77 passing with Docker; no stray containers left running.

Breach/Compliance Record hosting

id: TREV-WP-0006-T09
status: done
priority: medium
state_hub_task_id: "1a25114c-ccbb-417a-aa73-68f33120c5e4"

T01's Trust Service PRD flagged this gap explicitly: the Breach/Compliance Record component (specs/TechnicalSpecificationDocument.md §4.1 table, added 2026-07-29 alongside License V1C1 §7.4) postdates WP-0002 and was not assigned to any of T03T08. Implement it now as this workplan's ninth task, per TrustServicePRD TS-FR-7:

  • publish the Licensor's breach and termination determinations for a Phase, distinguishing alleged (notice given) from determined (cure period expired or breach otherwise established per the License's own terms) — the Trust Service never adjudicates whether a breach occurred, it only publishes the Licensor's own determination (TSD §4.1 Forbidden column, restated in TrustServicePRD TS-FR-7);
  • default to an anonymized Phase-and-category record (License V1C1 §7.4); named disclosure of the Commercial Entitlement holder occurs only where the applicable Commercial Use Agreement's naming/disclosure clause (CUA V1C1 §9) opts in, following that clause's notice period;
  • append-only per the same pattern as phase_manifests/ledger_entries/ attestations — a determination, once published, is not silently revised; a correction is a new, dated record, not an edit;
  • add the read/write API surface to service/app.py and reuse registry.authenticate's per-Licensor token model for the write side (only the Phase's own Licensor may publish a determination for it) — reads are public per the anonymized-by-default rule above.

Deliverable: a new migration (migrations/0004_breach_records.sql or similar), src/target_revenue/breach_record.py, corresponding service/app.py endpoints, and Docker-gated tests following the existing pattern in tests/test_hosted_conformance.py/test_ledger_hosting.py.

Result: migrations/0004_breach_records.sql models a case's lifecycle as append-only events (alleged/cured/determined/terminated) grouped by case_id, rather than one mutable row per case — resolution is always a new, later event, never an edit of the alleged one (trf_app again has no UPDATE/DELETE grant). A CHECK constraint makes the anonymized-default rule a database fact, not just an API convention: named_entitlement_holder can be set if and only if anonymized = false. src/target_revenue/breach_record.py's publish_breach_event() enforces per-Licensor phase ownership (same pattern as ledger.py), and — the naming-authorization requirement License V1C1 §7.4/CUA §9 impose — rejects any named-disclosure request that doesn't also set named_disclosure_authorized_under_cua: true; the Trust Service never verifies the underlying CUA text, it only records that the Licensor asserted the clause authorizes it. Signs every published event with the same instance Ed25519 key already used for Ledger entries and Attestations. Added POST/GET /phases/{id}/breach-records to service/app.py (write requires the Phase's own Licensor token; read is public). Guarded the .registry import behind a lazy, in-function import (matching attestation.py's TYPE_CHECKING pattern) so tests/test_breach_record.py (7 tests) runs its pure validation-logic checks under plain system Python with no psycopg dependency. 5 new Docker-gated tests in tests/test_ledger_hosting.py cover the default- anonymized lifecycle (alleged → determined, both events preserved, neither edited), the named-disclosure authorization gate (rejected without it, accepted with it), cross-Licensor rejection, signature verification, and DB-level UPDATE/DELETE rejection. Full suite: 56 passing offline (plain system Python), 89 passing with Docker; no stray containers left running. WP-0006 is finished again — all 9 tasks done.