diff --git a/docs/evidence-emission.md b/docs/evidence-emission.md index 508f901..ec2d191 100644 --- a/docs/evidence-emission.md +++ b/docs/evidence-emission.md @@ -34,9 +34,62 @@ revisited. ## Envelope -`audit-core.event.v1alpha1`. `source` is `tenant-engine`. See -`tenant_engine.audit_core.envelope_for`. Duplicate event ids are 200 -and not retried; 400/409 dead-letter; 503/transport retry. +The wire contract is audit-core's `docs/event-envelope.md`, not a schema +name. There is **no version negotiation** at that receiver: an envelope +either carries all eight required fields, truthy, or it is rejected +whole. `tenant_engine.audit_core.envelope_for` builds it. + +| Field | This engine sends | +| --- | --- | +| `id` | the event id, repeated in the `Idempotency-Key` header (a mismatch is rejected) | +| `type` | the domain event type (`tenant_created`, `role_granted`, …) | +| `source` | `tenant-engine` | +| `subject` | `tenant:` | +| `tenant` | the affected tenant id | +| `correlation_id` | the caller's, or one this engine minted for the operation | +| `occurred_at` | RFC 3339 **with an explicit offset** — a naive timestamp is rejected | +| `data` | the event payload, including the acting principal | + +`observed_at`, `action`, `resource`, `scope`, `outcome`, and `actor` are +**derived by audit-core** and deliberately not sent: sending them would +suggest this engine controls values it does not. The acting principal +travels inside `data`, where it reads as this service's claim rather than +the archive's finding. + +An earlier emitter sent its own names (`event_id`, `action`, `resource`, +`observed_at`, `details`) and no `correlation_id` at all. Every event +would have been rejected. `wire_envelope` upgrades any such row still +sitting in an outbox at send time, and refuses to send one whose +correlation cannot be recovered from its payload rather than inventing +one. + +### Correlation + +audit-core requires a `correlation_id` and will not synthesize one — an +id the *archive* invented would tie an event to an operation it never +observed. This engine is differently placed: it performed the operation. +Every mutation route accepts a `correlation_id`; the three older routes +(create, revoke, plan) treat it as optional for compatibility and mint +`req-` when a caller supplies none, returning it in the response. +The minted id is written into the local event payload too, so the local +and external records agree. + +### Delivery outcomes + +| Response | Outbox row | +| --- | --- | +| `202` | delivered | +| `200` | duplicate id, already stored — not retried | +| `400` | **stays pending**, reason recorded on the row | +| `409` | dead-lettered | +| `401` / `403` / `503` / transport | retried | + +A `400` is deliberately **not** terminal. It means the event is not in +the archive — audit-core holds only an unchained dead letter, which is a +diagnostic queue and not custody. Marking our row handled would lose the +event on both sides, silently, while the integration looked healthy. A +400 is an integration defect to fix; the row stays pending so it is +delivered once the defect is. ## Credentials diff --git a/src/tenant_engine/app.py b/src/tenant_engine/app.py index 22c4ec8..1775621 100644 --- a/src/tenant_engine/app.py +++ b/src/tenant_engine/app.py @@ -3,6 +3,7 @@ from __future__ import annotations import hashlib import json from datetime import UTC, datetime +from uuid import uuid4 from fastapi import FastAPI, Header, HTTPException, Query, Request, Response from fastapi.responses import JSONResponse @@ -64,6 +65,11 @@ class CreateTenantRequest(BaseModel): actor: str display_name: str | None = None contact_email: str | None = None + # Optional, unlike the lifecycle bodies, because these three routes + # shipped without it and existing clients must keep working. When a + # caller has no correlation of its own this service mints one for the + # request and returns it -- see `_correlation`. + correlation_id: str | None = None class GrantRoleRequest(BaseModel): @@ -79,11 +85,13 @@ class GrantRoleRequest(BaseModel): class RevokeRoleRequest(BaseModel): grant_id: str actor: str + correlation_id: str | None = None class AssignPlanRequest(BaseModel): plan_id: str actor: str + correlation_id: str | None = None class TenantMetadata(BaseModel): @@ -277,6 +285,7 @@ def create_app( @app.post("/tenants", status_code=201) async def create_tenant(payload: CreateTenantRequest) -> dict: + correlation_id = _correlation(payload.correlation_id) outcome = _authorize( authorizer, store, @@ -292,7 +301,9 @@ def create_app( contact_email=payload.contact_email, created_at=datetime.now(UTC), ) - store.create_tenant(tenant, authz=outcome.as_payload()) + store.create_tenant( + tenant, authz=outcome.as_payload(), correlation_id=correlation_id + ) _drain_outbox(store, app.state.audit_core) except InvalidTenantIdentifierError as exc: raise HTTPException(status_code=400, detail=str(exc)) from exc @@ -300,7 +311,7 @@ def create_app( raise HTTPException(status_code=409, detail=str(exc)) from exc # Superset of the pre-lifecycle response (tenant_id/identifier/grouping # are unchanged), so existing create clients keep working. - return _tenant_response(tenant) + return {**_tenant_response(tenant), "correlation_id": correlation_id} @app.post("/tenants/{tenant_id}/roles/grant", status_code=201) async def grant_role(tenant_id: str, payload: GrantRoleRequest) -> dict: @@ -327,10 +338,16 @@ def create_app( raise HTTPException(status_code=409, detail="tenant_retired") from exc except InvalidGrantError as exc: raise HTTPException(status_code=400, detail=str(exc)) from exc - return {"grant_id": grant.grant_id, "tenant_id": tenant_id, "role": grant.role.value} + return { + "grant_id": grant.grant_id, + "tenant_id": tenant_id, + "role": grant.role.value, + "correlation_id": payload.correlation_id, + } @app.post("/tenants/{tenant_id}/roles/revoke") async def revoke_role(tenant_id: str, payload: RevokeRoleRequest) -> dict: + correlation_id = _correlation(payload.correlation_id) outcome = _authorize( authorizer, store, action="tenant.role.revoke", tenant_id=tenant_id, actor=payload.actor ) @@ -340,6 +357,7 @@ def create_app( grant_id=payload.grant_id, at=datetime.now(UTC), authz=outcome.as_payload(), + correlation_id=correlation_id, ) _drain_outbox(store, app.state.audit_core) except TenantNotFoundError as exc: @@ -348,10 +366,16 @@ def create_app( raise HTTPException(status_code=404, detail="grant_not_found") from exc except InvalidGrantError as exc: raise HTTPException(status_code=409, detail=str(exc)) from exc - return {"grant_id": revoked.grant_id, "tenant_id": tenant_id, "revoked": True} + return { + "grant_id": revoked.grant_id, + "tenant_id": tenant_id, + "revoked": True, + "correlation_id": correlation_id, + } @app.post("/tenants/{tenant_id}/plan") async def assign_plan(tenant_id: str, payload: AssignPlanRequest) -> dict: + correlation_id = _correlation(payload.correlation_id) outcome = _authorize( authorizer, store, action="tenant.plan.assign", tenant_id=tenant_id, actor=payload.actor ) @@ -361,13 +385,18 @@ def create_app( tenant_id=tenant_id, plan_id=payload.plan_id, assigned_at=datetime.now(UTC) ), authz=outcome.as_payload(), + correlation_id=correlation_id, ) _drain_outbox(store, app.state.audit_core) except TenantNotFoundError as exc: raise HTTPException(status_code=404, detail="tenant_not_found") from exc except TenantRetiredError as exc: raise HTTPException(status_code=409, detail="tenant_retired") from exc - return {"tenant_id": tenant_id, "plan_id": payload.plan_id} + return { + "tenant_id": tenant_id, + "plan_id": payload.plan_id, + "correlation_id": correlation_id, + } # -- Lifecycle write API (TEN-WP-0005) -------------------------------- # Update, retire, and reactivate are separately authorized actions, not @@ -881,6 +910,18 @@ def _persist_authz(store: TenantStore, outcome: AuthorizationOutcome) -> None: ) +def _correlation(supplied: str | None) -> str: + """The correlation id for one mutation. + + audit-core requires a truthy `correlation_id` on every event and will not + synthesize one, correctly: an id the archive invented would tie an event + to an operation nobody observed. This service *did* observe the operation, + so when a caller supplies none it mints an id for this request and returns + it in the response, where the caller can use it to correlate. + """ + return supplied or f"req-{uuid4()}" + + def _drain_outbox(store: TenantStore, client: AuditCoreClient | None) -> None: """Best-effort drain. Never fails the mutation (attributive, non-blocking).""" if client is None: diff --git a/src/tenant_engine/audit_core.py b/src/tenant_engine/audit_core.py index 96122a4..bac1689 100644 --- a/src/tenant_engine/audit_core.py +++ b/src/tenant_engine/audit_core.py @@ -19,42 +19,129 @@ from uuid import uuid4 import httpx -SCHEMA_VERSION = "audit-core.event.v1alpha1" SOURCE = "tenant-engine" +#: The eight fields `POST /v1/events` requires, all of which must be truthy. +#: Contract: audit-core `docs/event-envelope.md`. There is no version +#: negotiation at that receiver -- an envelope either matches or is rejected +#: whole -- so this tuple is the contract, not a schema string. +REQUIRED_FIELDS = ( + "id", + "type", + "source", + "subject", + "tenant", + "correlation_id", + "occurred_at", + "data", +) + + +class EnvelopeError(ValueError): + """An envelope this engine would not be able to deliver. + + Raised at emission, inside the mutation transaction, rather than at + drain time. A malformed envelope is a defect in this service, and the + place to surface it is the mutation that produced it -- not the audit + trail, where audit-core would reject it into a dead-letter queue that + is a diagnostic surface and not custody. + """ + def new_event_id() -> str: return str(uuid4()) +def correlation_for(payload: dict[str, Any]) -> str: + """The correlation id for an event, minting one only if none was supplied. + + audit-core requires a truthy `correlation_id` and will not synthesize one, + correctly: an id the *archive* invented would tie an event to an operation + it never observed. This engine is in a different position -- it performed + the operation -- so when a caller supplies no correlation it names its own + operation. The minted id is written back into the local event payload too, + so the local record and the external one carry the same id rather than + disagreeing about what the event belongs to. + """ + return str(payload.get("correlation_id") or "") or f"op-{uuid4()}" + + def envelope_for( *, event_id: str, event_type: str, tenant_id: str, - observed_at: str, + occurred_at: str, + correlation_id: str, payload: dict[str, Any], ) -> dict[str, Any]: - return { - "schema_version": SCHEMA_VERSION, - "event_id": event_id, - "observed_at": observed_at, - "tenant": tenant_id, - "scope": "tenant-engine", + """Build the audit-core ingest envelope for one mutation event. + + Only the fields the receiver reads are sent. `observed_at`, `action`, + `resource`, `scope`, `outcome`, and `actor` are derived by audit-core + from these; sending them would suggest this engine controls values it + does not. The acting principal travels inside `data`, where it reads as + this service's claim rather than the archive's finding. + """ + envelope = { + "id": event_id, + "type": event_type, "source": SOURCE, - "actor": payload.get("actor") or payload.get("granted_by") or payload.get("changed_by"), - "action": event_type, - "resource": f"tenant:{tenant_id}", - "outcome": "recorded", - "reason": payload.get("reason") or payload.get("authorization_reason"), - "details": payload, + "subject": f"tenant:{tenant_id}", + "tenant": tenant_id, + "correlation_id": correlation_id, + "occurred_at": occurred_at, + "data": payload, } + missing = [field for field in REQUIRED_FIELDS if not envelope.get(field)] + if missing: + raise EnvelopeError(f"envelope missing required field(s): {', '.join(missing)}") + return envelope + + +#: Envelopes written before the eight-field contract was published are still +#: sitting in outboxes. Map the old key onto the new one at send time so a +#: pending row from before the fix is deliverable rather than permanently +#: rejected. `schema_version`, `scope`, `outcome`, `actor`, and `reason` are +#: dropped: the receiver derives or ignores every one of them. +_LEGACY_KEYS = { + "event_id": "id", + "action": "type", + "resource": "subject", + "observed_at": "occurred_at", + "details": "data", +} + + +def wire_envelope(envelope: dict[str, Any]) -> dict[str, Any]: + """The body to POST, upgrading a legacy stored envelope if needed.""" + if all(envelope.get(field) for field in REQUIRED_FIELDS): + return {field: envelope[field] for field in REQUIRED_FIELDS} + upgraded = {key: envelope[key] for key in REQUIRED_FIELDS if key in envelope} + for legacy, current in _LEGACY_KEYS.items(): + if not upgraded.get(current) and envelope.get(legacy): + upgraded[current] = envelope[legacy] + upgraded.setdefault("source", SOURCE) + data = upgraded.get("data") + # A legacy envelope predates the correlation_id requirement, but it + # carried the payload -- and the payload is where mutations recorded + # their correlation. Take it from there or not at all; audit-core is + # explicit that a synthesized correlation ties an event to an operation + # nobody observed. + if not upgraded.get("correlation_id") and isinstance(data, dict): + correlation = data.get("correlation_id") + if correlation: + upgraded["correlation_id"] = correlation + missing = [field for field in REQUIRED_FIELDS if not upgraded.get(field)] + if missing: + raise EnvelopeError(f"envelope missing required field(s): {', '.join(missing)}") + return upgraded @dataclass(frozen=True, slots=True) class DeliveryResult: event_id: str - status: str # delivered | duplicate | retry | dead | skipped + status: str # delivered | duplicate | retry | invalid | dead | skipped http_status: int | None = None detail: str = "" @@ -79,8 +166,18 @@ class AuditCoreClient: ) def post_event(self, envelope: dict[str, Any]) -> DeliveryResult: - event_id = str(envelope.get("event_id") or "") - headers: dict[str, str] = {"Content-Type": "application/json"} + event_id = str(envelope.get("id") or envelope.get("event_id") or "") + try: + body = wire_envelope(envelope) + except EnvelopeError as exc: + # Not sent. An unsendable envelope stays pending and visible + # rather than being recorded as a delivery outcome. + return DeliveryResult(event_id, "invalid", None, f"envelope:{exc}") + headers: dict[str, str] = { + "Content-Type": "application/json", + # The receiver rejects a body whose id disagrees with this header. + "Idempotency-Key": str(body["id"]), + } if self.token_file: try: token = open(self.token_file, encoding="utf-8").read().strip() @@ -90,17 +187,40 @@ class AuditCoreClient: return DeliveryResult(event_id, "retry", None, "token_empty") headers["Authorization"] = f"Bearer {token}" try: - response = self._client.post("/v1/events", json=envelope, headers=headers) + response = self._client.post("/v1/events", json=body, headers=headers) except (httpx.HTTPError, OSError) as exc: return DeliveryResult(event_id, "retry", None, f"unreachable:{exc.__class__.__name__}") if response.status_code in (200, 202): status = "duplicate" if response.status_code == 200 else "delivered" return DeliveryResult(event_id, status, response.status_code) - if response.status_code in (400, 409): - return DeliveryResult(event_id, "dead", response.status_code, "rejected") + if response.status_code == 400: + # A 400 means the event is NOT in the archive: audit-core holds + # only an unchained dead letter. Treating that as terminal would + # mark our row handled and lose the event on both sides, so the + # row stays pending and the reason is recorded on it. A 400 is an + # integration defect to fix, not a delivery outcome. + return DeliveryResult( + event_id, "invalid", 400, f"rejected:{_reason(response)}" + ) + if response.status_code == 409: + return DeliveryResult(event_id, "dead", 409, "rejected") if response.status_code in (401, 403): return DeliveryResult(event_id, "retry", response.status_code, "unauthorized") return DeliveryResult(event_id, "retry", response.status_code, "unavailable") def close(self) -> None: self._client.close() + + +def _reason(response: httpx.Response) -> str: + """audit-core's rejection reason, so the outbox row says what to fix.""" + try: + body = response.json() + except ValueError: + return "unparseable" + if isinstance(body, dict): + for key in ("reason", "error", "detail"): + value = body.get(key) + if isinstance(value, str) and value: + return value + return "unknown" diff --git a/src/tenant_engine/postgres_store.py b/src/tenant_engine/postgres_store.py index 7e34577..4112176 100644 --- a/src/tenant_engine/postgres_store.py +++ b/src/tenant_engine/postgres_store.py @@ -19,7 +19,7 @@ except ImportError: # pragma: no cover - exercised by the deployment guard ConnectionPool = None PoolTimeout = None -from tenant_engine.audit_core import envelope_for, new_event_id +from tenant_engine.audit_core import correlation_for, envelope_for, new_event_id from tenant_engine.domain import ( CapabilityRole, PlanAssignment, @@ -121,7 +121,13 @@ class PostgresTenantStore: with self._pool.connection() as conn: conn.execute("SELECT 1").fetchone() - def create_tenant(self, tenant: Tenant, *, authz: dict[str, Any] | None = None) -> None: + def create_tenant( + self, + tenant: Tenant, + *, + authz: dict[str, Any] | None = None, + correlation_id: str | None = None, + ) -> None: try: with self._pool.connection() as conn, conn.transaction(): conn.execute( @@ -147,7 +153,12 @@ class PostgresTenantStore: conn, "tenant_created", tenant.tenant_id, - {"identifier": tenant.identifier, "grouping": tenant.grouping, **(authz or {})}, + { + "identifier": tenant.identifier, + "grouping": tenant.grouping, + "correlation_id": correlation_id, + **(authz or {}), + }, ) except StoreUnavailableError as exc: if isinstance(exc.__cause__, UniqueViolation): @@ -254,7 +265,13 @@ class PostgresTenantStore: ) def revoke_role( - self, *, tenant_id: str, grant_id: str, at: datetime, authz: dict[str, Any] | None = None + self, + *, + tenant_id: str, + grant_id: str, + at: datetime, + authz: dict[str, Any] | None = None, + correlation_id: str | None = None, ) -> RoleGrant: tenant = self.get_tenant(tenant_id) with self._pool.connection() as conn, conn.transaction(): @@ -270,7 +287,12 @@ class PostgresTenantStore: conn, "role_revoked", tenant.tenant_id, - {"grant_id": grant_id, "role": grant.role.value, **(authz or {})}, + { + "grant_id": grant_id, + "role": grant.role.value, + "correlation_id": correlation_id or grant.correlation_id, + **(authz or {}), + }, ) return grant @@ -284,7 +306,11 @@ class PostgresTenantStore: return frozenset(CapabilityRole(row["role"]) for row in rows) def assign_plan( - self, assignment: PlanAssignment, *, authz: dict[str, Any] | None = None + self, + assignment: PlanAssignment, + *, + authz: dict[str, Any] | None = None, + correlation_id: str | None = None, ) -> None: tenant = self.get_tenant(assignment.tenant_id) self._require_active(tenant, "assign a plan") @@ -299,7 +325,11 @@ class PostgresTenantStore: conn, "plan_assigned", tenant.tenant_id, - {"plan_id": assignment.plan_id, **(authz or {})}, + { + "plan_id": assignment.plan_id, + "correlation_id": correlation_id, + **(authz or {}), + }, ) def events_for(self, tenant_id: str) -> list[DomainEvent]: @@ -548,7 +578,11 @@ class PostgresTenantStore: @staticmethod def _emit(conn: Any, event_type: str, tenant_id: str, payload: dict[str, Any]) -> None: event_id = str(payload.get("event_id") or new_event_id()) - payload = {**payload, "event_id": event_id} + # audit-core requires a truthy correlation_id on every event + # (its docs/event-envelope.md). Normalize before the local + # insert so the local record and the emitted envelope carry + # the same id. + payload = {**payload, "event_id": event_id, "correlation_id": correlation_for(payload)} at = datetime.now(UTC) conn.execute( "INSERT INTO events (event_type, tenant_id, at, payload) VALUES (%s, %s, %s, %s)", @@ -558,7 +592,8 @@ class PostgresTenantStore: event_id=event_id, event_type=event_type, tenant_id=tenant_id, - observed_at=at.isoformat(), + occurred_at=at.isoformat(), + correlation_id=payload["correlation_id"], payload=payload, ) conn.execute( diff --git a/src/tenant_engine/sqlite_store.py b/src/tenant_engine/sqlite_store.py index 791a3f2..fc82c1f 100644 --- a/src/tenant_engine/sqlite_store.py +++ b/src/tenant_engine/sqlite_store.py @@ -8,7 +8,7 @@ from datetime import datetime from threading import RLock from typing import Any -from tenant_engine.audit_core import envelope_for, new_event_id +from tenant_engine.audit_core import correlation_for, envelope_for, new_event_id from tenant_engine.domain import ( CapabilityRole, PlanAssignment, @@ -126,7 +126,13 @@ class SQLiteTenantStore: with self._lock: self._db.execute("SELECT 1").fetchone() - def create_tenant(self, tenant: Tenant, *, authz: dict[str, Any] | None = None) -> None: + def create_tenant( + self, + tenant: Tenant, + *, + authz: dict[str, Any] | None = None, + correlation_id: str | None = None, + ) -> None: with self._lock, self._db: try: self._db.execute( @@ -156,6 +162,7 @@ class SQLiteTenantStore: { "identifier": tenant.identifier, "grouping": tenant.grouping, + "correlation_id": correlation_id, **(authz or {}), }, ) @@ -453,7 +460,13 @@ class SQLiteTenantStore: ) def revoke_role( - self, *, tenant_id: str, grant_id: str, at: datetime, authz: dict[str, Any] | None = None + self, + *, + tenant_id: str, + grant_id: str, + at: datetime, + authz: dict[str, Any] | None = None, + correlation_id: str | None = None, ) -> RoleGrant: tenant = self.get_tenant(tenant_id) with self._lock: @@ -471,7 +484,12 @@ class SQLiteTenantStore: self._emit( "role_revoked", tenant.tenant_id, - {"grant_id": grant_id, "role": grant.role.value, **(authz or {})}, + { + "grant_id": grant_id, + "role": grant.role.value, + "correlation_id": correlation_id or grant.correlation_id, + **(authz or {}), + }, ) return grant @@ -485,7 +503,11 @@ class SQLiteTenantStore: return frozenset(CapabilityRole(row["role"]) for row in rows) def assign_plan( - self, assignment: PlanAssignment, *, authz: dict[str, Any] | None = None + self, + assignment: PlanAssignment, + *, + authz: dict[str, Any] | None = None, + correlation_id: str | None = None, ) -> None: tenant = self.get_tenant(assignment.tenant_id) self._require_active(tenant, "assign a plan") @@ -497,7 +519,11 @@ class SQLiteTenantStore: self._emit( "plan_assigned", tenant.tenant_id, - {"plan_id": assignment.plan_id, **(authz or {})}, + { + "plan_id": assignment.plan_id, + "correlation_id": correlation_id, + **(authz or {}), + }, ) def events_for(self, tenant_id: str) -> list[DomainEvent]: @@ -590,7 +616,11 @@ class SQLiteTenantStore: def _emit(self, event_type: str, tenant_id: str, payload: dict) -> None: event_id = str(payload.get("event_id") or new_event_id()) - payload = {**payload, "event_id": event_id} + # audit-core requires a truthy correlation_id on every event + # (its docs/event-envelope.md). Normalize before the local + # insert so the local record and the emitted envelope carry + # the same id. + payload = {**payload, "event_id": event_id, "correlation_id": correlation_for(payload)} now = datetime.now().astimezone() self._db.execute( "INSERT INTO events(event_type,tenant_id,at,payload) VALUES(?,?,?,?)", @@ -600,7 +630,8 @@ class SQLiteTenantStore: event_id=event_id, event_type=event_type, tenant_id=tenant_id, - observed_at=now.isoformat(), + occurred_at=now.isoformat(), + correlation_id=payload["correlation_id"], payload=payload, ) self._db.execute( diff --git a/src/tenant_engine/store.py b/src/tenant_engine/store.py index f277e4f..4bb1629 100644 --- a/src/tenant_engine/store.py +++ b/src/tenant_engine/store.py @@ -5,7 +5,7 @@ from dataclasses import dataclass, replace from datetime import UTC, datetime from typing import Any, Protocol -from tenant_engine.audit_core import envelope_for, new_event_id +from tenant_engine.audit_core import correlation_for, envelope_for, new_event_id from tenant_engine.domain import ( CapabilityRole, PlanAssignment, @@ -109,20 +109,36 @@ class TenantStore(Protocol): tenant_roles claim, KEY-WP-0005-T02). """ - def create_tenant(self, tenant: Tenant, *, authz: dict[str, Any] | None = None) -> None: ... + def create_tenant( + self, + tenant: Tenant, + *, + authz: dict[str, Any] | None = None, + correlation_id: str | None = None, + ) -> None: ... def get_tenant(self, tenant_id: str) -> Tenant: ... def grant_role(self, grant: RoleGrant, *, authz: dict[str, Any] | None = None) -> None: ... def revoke_role( - self, *, tenant_id: str, grant_id: str, at: datetime, authz: dict[str, Any] | None = None + self, + *, + tenant_id: str, + grant_id: str, + at: datetime, + authz: dict[str, Any] | None = None, + correlation_id: str | None = None, ) -> RoleGrant: ... def active_roles(self, tenant_id: str) -> frozenset[CapabilityRole]: ... def assign_plan( - self, assignment: PlanAssignment, *, authz: dict[str, Any] | None = None + self, + assignment: PlanAssignment, + *, + authz: dict[str, Any] | None = None, + correlation_id: str | None = None, ) -> None: ... def events_for(self, tenant_id: str) -> list[DomainEvent]: ... @@ -253,7 +269,13 @@ class InMemoryTenantStore: self._overrides: dict[str, dict[str, LimitValue]] = {} self._guardrail_changes: list[GuardrailChange] = [] - def create_tenant(self, tenant: Tenant, *, authz: dict[str, Any] | None = None) -> None: + def create_tenant( + self, + tenant: Tenant, + *, + authz: dict[str, Any] | None = None, + correlation_id: str | None = None, + ) -> None: if tenant.tenant_id in self._tenants: raise TenantAlreadyExistsError(tenant.tenant_id) if tenant.identifier in self._by_identifier: @@ -264,7 +286,12 @@ class InMemoryTenantStore: self._emit( "tenant_created", tenant.tenant_id, - {"identifier": tenant.identifier, "grouping": tenant.grouping, **(authz or {})}, + { + "identifier": tenant.identifier, + "grouping": tenant.grouping, + "correlation_id": correlation_id, + **(authz or {}), + }, ) def get_tenant(self, tenant_id: str) -> Tenant: @@ -293,6 +320,7 @@ class InMemoryTenantStore: grant_id: str, at: datetime, authz: dict[str, Any] | None = None, + correlation_id: str | None = None, ) -> RoleGrant: resolved = self._resolve(tenant_id) try: @@ -304,7 +332,12 @@ class InMemoryTenantStore: self._emit( "role_revoked", resolved, - {"grant_id": grant_id, "role": revoked.role.value, **(authz or {})}, + { + "grant_id": grant_id, + "role": revoked.role.value, + "correlation_id": correlation_id or grant.correlation_id, + **(authz or {}), + }, ) return revoked @@ -315,7 +348,11 @@ class InMemoryTenantStore: ) def assign_plan( - self, assignment: PlanAssignment, *, authz: dict[str, Any] | None = None + self, + assignment: PlanAssignment, + *, + authz: dict[str, Any] | None = None, + correlation_id: str | None = None, ) -> None: resolved = self._resolve(assignment.tenant_id) self._require_active(resolved, "assign a plan") @@ -323,7 +360,11 @@ class InMemoryTenantStore: self._emit( "plan_assigned", resolved, - {"plan_id": assignment.plan_id, **(authz or {})}, + { + "plan_id": assignment.plan_id, + "correlation_id": correlation_id, + **(authz or {}), + }, ) def events_for(self, tenant_id: str) -> list[DomainEvent]: @@ -494,7 +535,11 @@ class InMemoryTenantStore: def _emit(self, event_type: str, tenant_id: str, payload: dict[str, Any]) -> None: event_id = str(payload.get("event_id") or new_event_id()) - payload = {**payload, "event_id": event_id} + # audit-core requires a truthy correlation_id on every event + # (its docs/event-envelope.md). Normalize before the local + # insert so the local record and the emitted envelope carry + # the same id. + payload = {**payload, "event_id": event_id, "correlation_id": correlation_for(payload)} at = datetime.now(UTC) self._events.append( DomainEvent(event_type=event_type, tenant_id=tenant_id, at=at, payload=payload) @@ -503,7 +548,8 @@ class InMemoryTenantStore: event_id=event_id, event_type=event_type, tenant_id=tenant_id, - observed_at=at.isoformat(), + occurred_at=at.isoformat(), + correlation_id=payload["correlation_id"], payload=payload, ) self._outbox[event_id] = OutboxRow( diff --git a/tests/test_audit_core.py b/tests/test_audit_core.py index 4eef957..2ee9557 100644 --- a/tests/test_audit_core.py +++ b/tests/test_audit_core.py @@ -1,11 +1,21 @@ """TEN-WP-0011-T04: local outbox and attributive drain to audit-core.""" +import json +from datetime import datetime + import httpx +import pytest from fastapi.testclient import TestClient from helpers import AllowAllAuthorizer from tenant_engine.app import create_app -from tenant_engine.audit_core import SOURCE, AuditCoreClient +from tenant_engine.audit_core import ( + REQUIRED_FIELDS, + SOURCE, + AuditCoreClient, + EnvelopeError, + wire_envelope, +) from tenant_engine.config import Settings from tenant_engine.domain import Tenant from tenant_engine.store import InMemoryTenantStore @@ -18,9 +28,139 @@ def test_mutation_enqueues_an_outbox_envelope(): assert pending envelope = pending[0].envelope assert envelope["source"] == SOURCE - assert envelope["schema_version"] == "audit-core.event.v1alpha1" assert envelope["tenant"] == "t-1" - assert "event_id" in envelope + assert envelope["id"] + + +def test_the_envelope_carries_exactly_the_eight_fields_the_receiver_requires(): + """audit-core `docs/event-envelope.md`: eight fields, all truthy. + + Pinned because the first shipped emitter used its own names -- event_id, + action, resource, observed_at, details -- and omitted correlation_id + entirely, which the receiver rejects whole. That mismatch would have + dead-lettered every event while looking like a working integration. + """ + store = InMemoryTenantStore() + store.create_tenant( + Tenant.create(tenant_id="t-1", identifier="tenant:friendly:binky"), + correlation_id="corr-1", + ) + envelope = store.pending_outbox()[0].envelope + assert set(envelope) == set(REQUIRED_FIELDS) + assert all(envelope[field] for field in REQUIRED_FIELDS) + assert envelope["type"] == "tenant_created" + assert envelope["subject"] == "tenant:t-1" + assert envelope["correlation_id"] == "corr-1" + assert envelope["data"]["identifier"] == "tenant:friendly:binky" + # The receiver derives these; sending them would imply we control them. + for derived in ("observed_at", "action", "resource", "scope", "outcome", "actor"): + assert derived not in envelope + # A naive timestamp is ambiguous by up to a day and is rejected. + assert datetime.fromisoformat(envelope["occurred_at"]).tzinfo is not None + + +def test_a_caller_that_supplies_no_correlation_still_produces_a_deliverable_event(): + """The engine names its own operation rather than emitting an undeliverable + envelope. It observed the mutation; audit-core, which did not, refuses to + synthesize a correlation and is right to.""" + store = InMemoryTenantStore() + store.create_tenant(Tenant.create(tenant_id="t-1", identifier="tenant:friendly:binky")) + envelope = store.pending_outbox()[0].envelope + assert envelope["correlation_id"] + # The local record and the emitted one agree about the operation. + assert store.events_for("t-1")[0].payload["correlation_id"] == envelope["correlation_id"] + + +def test_the_idempotency_key_header_equals_the_body_id(): + """A mismatch is rejected `idempotency_key_mismatch` by the receiver.""" + seen: list[httpx.Request] = [] + + def handler(request: httpx.Request) -> httpx.Response: + seen.append(request) + return httpx.Response(202, json={"status": "accepted"}) + + client = AuditCoreClient( + base_url="https://audit-core.example.test", transport=httpx.MockTransport(handler) + ) + store = InMemoryTenantStore() + store.create_tenant(Tenant.create(tenant_id="t-1", identifier="tenant:friendly:binky")) + envelope = store.pending_outbox()[0].envelope + assert client.post_event(envelope).status == "delivered" + assert seen[0].headers["Idempotency-Key"] == envelope["id"] + assert json.loads(seen[0].content)["id"] == envelope["id"] + + +def test_a_rejected_event_is_retained_rather_than_recorded_as_handled(): + """A 400 means the event is NOT in the archive -- audit-core holds only an + unchained dead letter. Marking our row handled would lose it on both + sides, silently. It stays pending, with the reason on the row.""" + + def handler(request: httpx.Request) -> httpx.Response: + return httpx.Response(400, json={"reason": "invalid_event"}) + + client = AuditCoreClient( + base_url="https://audit-core.example.test", transport=httpx.MockTransport(handler) + ) + store = InMemoryTenantStore() + store.create_tenant(Tenant.create(tenant_id="t-1", identifier="tenant:friendly:binky")) + row = store.pending_outbox()[0] + result = client.post_event(row.envelope) + assert (result.status, result.http_status) == ("invalid", 400) + assert "invalid_event" in result.detail + + store.mark_outbox(row.event_id, status=result.status, detail=result.detail) + still_pending = store.pending_outbox() + assert [r.event_id for r in still_pending] == [row.event_id] + assert still_pending[0].dead_at is None + + +def test_a_legacy_stored_envelope_is_upgraded_at_send_time(): + """Rows written before the contract was published are still in outboxes.""" + legacy = { + "schema_version": "audit-core.event.v1alpha1", + "event_id": "e-1", + "observed_at": "2026-09-10T11:04:12+00:00", + "tenant": "t-1", + "scope": "tenant-engine", + "source": SOURCE, + "actor": "ops", + "action": "role_granted", + "resource": "tenant:t-1", + "outcome": "recorded", + "reason": "onboarding", + "details": {"role": "PLTF", "correlation_id": "corr-1"}, + } + body = wire_envelope(legacy) + assert set(body) == set(REQUIRED_FIELDS) + assert body["id"] == "e-1" + assert body["type"] == "role_granted" + assert body["subject"] == "tenant:t-1" + assert body["occurred_at"] == "2026-09-10T11:04:12+00:00" + assert body["correlation_id"] == "corr-1" + assert body["data"] == legacy["details"] + + +def test_a_legacy_envelope_with_no_recoverable_correlation_is_not_sent(): + """Never invent one: it would tie the event to an operation nobody saw.""" + legacy = { + "event_id": "e-1", + "observed_at": "2026-09-10T11:04:12+00:00", + "tenant": "t-1", + "source": SOURCE, + "action": "role_granted", + "resource": "tenant:t-1", + "details": {"role": "PLTF"}, + } + with pytest.raises(EnvelopeError): + wire_envelope(legacy) + + client = AuditCoreClient( + base_url="https://audit-core.example.test", + transport=httpx.MockTransport(lambda request: pytest.fail("must not be sent")), + ) + result = client.post_event(legacy) + assert result.status == "invalid" + assert result.http_status is None def test_drain_marks_delivered_and_does_not_hold_audit_core_sql(): diff --git a/workplans/TEN-WP-0012-external-conformance-waits.md b/workplans/TEN-WP-0012-external-conformance-waits.md index 89750aa..406ea19 100644 --- a/workplans/TEN-WP-0012-external-conformance-waits.md +++ b/workplans/TEN-WP-0012-external-conformance-waits.md @@ -8,7 +8,7 @@ status: blocked owner: claude topic_slug: netkingdom created: "2026-09-07" -updated: "2026-09-07" +updated: "2026-09-10" depends_on: - TEN-WP-0011 unblocks: [] @@ -49,8 +49,11 @@ convention's status for work that is blocked on another party. explicit that `POST /messages/` asking for a token is an anti-pattern; sender registration is audit-core's to issue. - Not re-opening `TEN-WP-0011`. Our side of both items shipped. -- No code change is expected from this workplan. If either disposition - arrives and *does* require code, that becomes its own workplan. +- No code change was expected from this workplan. That held until + 2026-09-10, when audit-core's disposition on T01 arrived carrying a + blocking defect in our emitter. The fix was small, wholly inside this + repo, and belonged to the wait it unblocks, so it was made here rather + than spun into a workplan of its own. A larger one still would. ## T01 — audit-core sender registration (AUDIT-IN-0002) @@ -70,6 +73,49 @@ holds no audit-core credential beyond that sender token. Waiting on: `AUDIT-IN-0002`, filed on audit-core. +**2026-09-10 — registered, and a blocking defect found on our side.** +audit-core registered the sender (attributive, `source` pinned to +`tenant-engine`, `may_write`, `secret_policy: redact`, `tenants: ["*"]` +justified per sender) and recorded our declared completeness trade on the +sender identity itself. The token and the operator apply are still theirs +to issue, so this task stays `wait`. + +Their review then found that **no event we emitted could ever have been +accepted**. `envelope_for` sent five of the eight required fields under +its own names (`event_id`, `action`, `resource`, `observed_at`, +`details`) and omitted `correlation_id` entirely; `normalize()` rejects +that whole, `400`. Our drain treated `400` as terminal, so every event +would have been marked handled here while audit-core held only an +unchained dead letter — lost on both sides, silently, with the +integration looking healthy. The root cause was on their side: the wire +contract was published nowhere we could read, and +`docs/audit-backend-contract.md` describes the *stored* record, from +which the names we chose were a reasonable inference. It is now published +as audit-core `docs/event-envelope.md`. + +Fixed here the same day, ahead of any token: + +- `envelope_for` emits exactly the eight required fields and sends none + of the six audit-core derives. +- `correlation_id` is threaded through create / revoke / plan, which had + no such field. It is optional on those three request bodies for + compatibility; when a caller supplies none this engine mints + `req-` for the operation it actually performed and returns it. + Never synthesized at the archive, which is what audit-core refuses and + is right to refuse. +- `Idempotency-Key` is sent and equals the body `id`. +- `400` no longer dead-letters. The row stays pending with the rejection + reason recorded on it, per their guidance that a 400 is an integration + defect to fix, not a delivery outcome to record. +- `wire_envelope` upgrades outbox rows written in the old shape at send + time, and refuses to send one whose correlation cannot be recovered + from its payload. + +Verified by running all nine event types, through the API, through +audit-core's actual `normalize()` with a matching `SenderIdentity` — all +accepted. That is the check their message asked for, short of a live +non-production `202`, which still needs the token. + Done when: audit-core registers the sender and production mutations land externally, **or** audit-core declines and `SCOPE.md` row "Independent audit-core emission" is restated to say so.