fix: report truthful compatibility outcomes and verify audit coverage
Some checks failed
CI Smoke / host-smoke (push) Successful in 1s
CI Smoke / pytest-smoke (push) Failing after 3s

Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a0e747-8f27-7242-8df8-8bc44f88c929
This commit is contained in:
tegwick 2026-09-28 14:59:33 +02:00
parent c0383c9c2b
commit 72f3513954
7 changed files with 246 additions and 16 deletions

View file

@ -56,9 +56,9 @@ A host composes `create_app(access_controller=AccessController(...))` with:
only after an operational-custody probe and explicit durable acceptance. Every allow must reach this sink before handler execution;
a failed sink blocks reads as well as writes. Authorization receipts say
`authorized`, not “operation completed.” Domain commit/outcome audit remains a
separate concern. Native durable mutations now have a
[transaction-linked outcome outbox](operation-outcome-audit.md); compatibility
and external mutations remain outside that slice.
separate concern. Native durable and implemented SQL compatibility mutations have a
[transaction-linked outcome outbox](operation-outcome-audit.md); arbitrary
embedded-host and external mutations remain outside that slice.
- The root's existing immutable issuer and subject, supplied after owner resolution.
No username, email, first-login promotion or generic role establishes root.

View file

@ -1,4 +1,4 @@
# Transaction-linked native operation outcomes
# Transaction-linked operation outcomes
HUB-WP-0012 source candidate, 2026-09-28. Live sender admission, database rollout
and owner acceptance remain open.
@ -26,8 +26,20 @@ retained separately from verified authorization correlation IDs. That verified
ID and decision ID join the outcome to the pre-execution signed decision already
held by Audit Core. The outcome does not duplicate the signed envelope.
This slice covers the native durable mutation ledger. It does not claim outcome
coverage for compatibility/embedded-host mutations, background projection refresh,
Implemented SQL compatibility writes share this transaction boundary: hub,
manifest create/update/activate, consumer, API key, widget and interaction-event
writes, through both `/api/v2` and root aliases. Key issuance also updates the
consumer in the same transaction; an outbox failure rolls both changes back.
Generated keys never enter the ledger/outcome envelope and cannot authenticate
enforced routes, which still require verified OIDC identity.
Unimplemented token issuance and deferred POST operations return `501`, including
in OpenAPI, rather than acknowledging a mutation that did not occur. Missing
manifest update/activation returns `404`. Neither creates a committed outcome.
Interaction responses return the persisted ID even when the caller supplies an ID.
This does not claim outcome coverage for arbitrary embedded-host mutations,
background projection refresh,
external services, or in-memory development stores. Non-enforced maintenance
writes retain their existing local ledger behavior without fabricating an actor
or emitting an attributed remote outcome.
@ -75,7 +87,7 @@ This document neither issues that grant nor claims production delivery.
## Local evidence
Tests use SQLite-backed durable stores to prove atomic rollback, handler rejection,
all native mutation families, actor/decision attribution, concurrent request
all native and implemented SQL compatibility mutation families, actor/decision attribution, concurrent request
isolation, persisted retry delay, reopen/replay, cancellation, stale-backlog
readiness, dispatcher drain/shutdown and migration shape/downgrade refusal.
The real Audit Core receiver/storage source accepts the eight-field envelope and
@ -83,7 +95,8 @@ returns a duplicate receipt on replay. Its operational-readiness classification
is explicitly a test fixture; this is not live custody evidence.
Disposable PostgreSQL tests now verify multiworker lock scheduling, the full
migration chain, rollback, persisted retries and process-exit replay. See the
migration chain, native and compatibility-key rollback, persisted retries and
process-exit replay. See the
[PostgreSQL gate](conformance.md#disposable-postgresql-gate). Production grants,
retention and deployed failure-detection/receiver acceptance still require receipts.

View file

@ -46,8 +46,8 @@ at three seconds, uses TLS, and never follows redirects.
Allow is blocked until the archive accepts the authorization record. A lost
receipt blocks the business operation even if the attempt reached storage. This
is a pre-execution authorization journal, not proof that an operation committed.
Authorization cannot use a local success buffer or silent redaction. Native
transaction outcomes now use a separate [durable outbox](operation-outcome-audit.md);
Authorization cannot use a local success buffer or silent redaction. Native and
implemented SQL compatibility transaction outcomes use a separate [durable outbox](operation-outcome-audit.md);
its production delivery and broader mutation coverage remain T03/T04 gates.
The exact verified signed decision is retained under `data.signed_decision` as

View file

@ -136,7 +136,7 @@ class SQLCompatibilityStore:
)
).mappings().one_or_none()
if current is None:
return {"id": manifest_id, "status": "missing"}
raise HTTPException(status_code=404, detail="manifest not found")
payload = dict(current["body"] or {})
payload.update(body)
values = {
@ -296,7 +296,7 @@ class SQLCompatibilityStore:
payload={"legacy": body},
)
accepted = await self.ports.append_interaction(command)
return {"id": accepted.id, **body}
return {**body, "id": accepted.id}
def create_compatibility_router() -> APIRouter:
@ -407,10 +407,10 @@ def create_compatibility_router() -> APIRouter:
await _protected(request, "credentials", authorization, write=True)
return await _store(request).create_key(consumer_id, body.get("scopes"))
@router.post("/api/v2/token")
@router.post("/api/v2/token", status_code=501)
async def token(request: Request, authorization: AuthorizationHeader = None) -> dict:
await _protected(request, "credentials", authorization)
return {"access_token": "already-authenticated", "token_type": "bearer"}
raise HTTPException(status_code=501, detail="compatibility token issuance is not implemented")
@router.get("/api/v2/widgets")
async def list_widgets(
@ -457,7 +457,7 @@ def create_compatibility_router() -> APIRouter:
request: Request, authorization: AuthorizationHeader = None
) -> dict:
await _protected(request, "deferred", authorization, write=True)
return {"data": []}
raise HTTPException(status_code=501, detail="compatibility operation is not implemented")
stem = path.replace("-", "_")
router.add_api_route(
@ -470,7 +470,7 @@ def create_compatibility_router() -> APIRouter:
f"/api/v2/{path}",
accept_deferred,
methods=["POST"],
status_code=201,
status_code=501,
operation_id=f"compat_create_{stem}_{index}",
)

View file

@ -0,0 +1,140 @@
"""Enforced compatibility writes: real SQL transactions, synthetic authority."""
import json
from dataclasses import replace
from uuid import uuid4
import pytest
import sqlalchemy as sa
from fastapi.testclient import TestClient
from hub_core.runtime.app import create_app
from hub_core.runtime.config import RuntimeSettings
from hub_core.runtime.tables import (compat_api_keys, compat_api_consumers,
runtime_audit_ledger, runtime_outcome_outbox, runtime_interaction_events)
from test_access_boundary import HEADERS
from test_outcome_audit import target, rows
GROUPS = frozenset({'registry','credentials','interaction','deferred','system'})
@pytest.fixture
def compatibility(target):
_,store,owners,url = target
settings = RuntimeSettings(environment='test',access_mode='enforce',backend='postgresql',
database_url=url,v2_groups=GROUPS,v2_write_groups=GROUPS)
app = create_app(settings=settings,port_store=store,access_controller=owners.controller())
with TestClient(app,raise_server_exceptions=False) as client:
yield client,store,owners
@pytest.mark.parametrize('prefix',['/api/v2',''])
def test_all_implemented_compatibility_mutations_have_attributed_outcomes(compatibility,prefix):
client,store,owners = compatibility
ids = []
def write(path,body,method='POST',status=201):
response = client.request(method,prefix+path,headers=HEADERS,json=body)
assert response.status_code == status, response.text
ids.append(response.headers['x-correlation-id'])
return response.json()
hub = write('/hubs',{'name':'Fixture Hub','slug':'fixture-hub'})
manifest = write('/hub-capability-manifests',{'hubId':hub['id'],'manifestVersion':'1.0'})
write('/hub-capability-manifests/'+manifest['id'],{'description':'private manifest value'},'PATCH',200)
write('/hub-capability-manifests/'+manifest['id']+'/activate',{},status=200)
consumer = write('/api-consumers',{'name':'Fixture Consumer'})
issued = write('/api-consumers/'+consumer['id']+'/api-keys',{'scopes':'fixture-only'})
widget = write('/widgets',{'hubId':hub['id'],'name':'Fixture Widget'})
interaction = write('/interaction-events',{'widgetId':widget['id'],'eventType':'fixture.interaction',
'id':'caller-forged-id','metadata':{'text':'private event body'}})
assert interaction['id'] != 'caller-forged-id'
event, = rows(store,runtime_interaction_events)
assert interaction['id'] == event['id']
pending = rows(store,runtime_outcome_outbox)
assert len(pending) == len(ids) == 8
assert {row['envelope']['correlation_id'] for row in pending} == set(ids)
decisions = {record['correlation_id']: record for record in owners.records}
operations = set()
for row in pending:
event = row['envelope']
auth = event['data']['authorization']
assert auth['subject'] == 'immutable-root'
assert auth['decision_id'] == decisions[event['correlation_id']]['decision_id']
operations.add(event['data']['operation'])
assert operations == {'compat.hub.created','compat.manifest.created','compat.manifest.updated',
'compat.manifest.activated','compat.consumer.created','compat.api_key.created',
'compat.widget.created','event.interaction.accepted'}
serialized = json.dumps(pending)+json.dumps(rows(store,runtime_audit_ledger),default=str)
assert issued['fullKey'] not in serialized
assert 'private manifest value' not in serialized and 'private event body' not in serialized
# Legacy compatibility keys cannot authenticate an enforced route.
assert client.get(prefix+'/hubs',headers={'Authorization':'Bearer '+issued['fullKey']}).status_code == 401
@pytest.mark.parametrize('prefix',['/api/v2',''])
@pytest.mark.parametrize('path',['annotations','requirement-candidates','decision-records',
'deployment-records','outcome-signals','token'])
def test_unimplemented_compatibility_writes_never_report_success(compatibility,prefix,path):
client,store,_ = compatibility
response = client.post(prefix+'/'+path,headers=HEADERS,json={})
assert response.status_code == 501
responses = client.app.openapi()['paths']['/api/v2/'+path]['post']['responses']
assert '501' in responses and '200' not in responses and '201' not in responses
assert not rows(store,runtime_outcome_outbox)
assert not rows(store,runtime_audit_ledger)
@pytest.mark.parametrize('prefix',['/api/v2',''])
@pytest.mark.parametrize('activate',[False,True])
def test_missing_manifest_is_not_a_successful_mutation(compatibility,prefix,activate):
client,store,_ = compatibility
path = prefix+'/hub-capability-manifests/'+str(uuid4())
response = client.request('POST' if activate else 'PATCH',path+('/activate' if activate else ''),
headers=HEADERS,json={})
assert response.status_code == 404
assert not rows(store,runtime_outcome_outbox)
def test_key_issuance_and_consumer_update_roll_back_on_outbox_failure(compatibility):
client,store,_ = compatibility
consumer = client.post('/api/v2/api-consumers',headers=HEADERS,json={'name':'Fixture Consumer'}).json()
before = rows(store,compat_api_consumers)
def fail(connection,cursor,statement,parameters,context,many):
if statement.startswith('INSERT INTO runtime_outcome_outbox'):
raise RuntimeError('fixture outbox failure')
sa.event.listen(store.engine.sync_engine,'before_cursor_execute',fail)
try:
response = client.post('/api/v2/api-consumers/'+consumer['id']+'/api-keys',headers=HEADERS,json={})
assert response.status_code == 500
assert 'fullKey' not in response.text
finally:
sa.event.remove(store.engine.sync_engine,'before_cursor_execute',fail)
assert not rows(store,compat_api_keys)
assert rows(store,compat_api_consumers) == before
assert len(rows(store,runtime_outcome_outbox)) == 1
assert len(rows(store,runtime_audit_ledger)) == 1
@pytest.mark.parametrize('failure,status',[('anonymous',401),('invalid',401),('ordinary',403),
('revoked',403),('wrong_tenant',403),('policy',503),('audit',503)])
def test_compatibility_denials_do_not_mutate_or_queue_outcomes(compatibility,failure,status):
client,store,owners = compatibility
headers = HEADERS
if failure == 'anonymous': headers = {}
elif failure == 'invalid': headers = {'Authorization':'Bearer wrong'}
elif failure == 'ordinary': owners.actor = replace(owners.actor,subject='ordinary')
elif failure == 'revoked': owners.facts = replace(owners.facts,root_entitled=False)
elif failure == 'wrong_tenant': owners.actor = replace(owners.actor,tenant='tenant:other')
elif failure == 'policy': owners.policy_down = True
else: owners.audit_down = True
for prefix in ['/api/v2','']:
assert client.post(prefix+'/api-consumers',headers=headers,json={'name':'Never created'}).status_code == status
assert not rows(store,compat_api_consumers)
assert not rows(store,runtime_outcome_outbox)
def test_read_only_and_disabled_group_still_prevent_writes(compatibility):
client,store,_ = compatibility
client.app.state.settings = replace(client.app.state.settings,v2_write_groups=frozenset())
assert client.post('/api-consumers',headers=HEADERS,json={'name':'Blocked'}).status_code == 503
client.app.state.settings = replace(client.app.state.settings,v2_groups=frozenset())
assert client.post('/api-consumers',headers=HEADERS,json={'name':'Blocked'}).status_code == 404
assert not rows(store,runtime_outcome_outbox)

View file

@ -256,3 +256,57 @@ def test_receiver_failure_persists_backoff_across_connection_reopen(database):
finally:
await reopened.aclose()
asyncio.run(run())
def test_compatibility_key_transaction_and_outcome_rollback(database):
import httpx
from hub_core.runtime.app import create_app
from hub_core.runtime.config import RuntimeSettings
from hub_core.runtime.tables import compat_api_consumers, compat_api_keys
from test_access_boundary import HEADERS
url,_ = database
async def run():
store = store_for(url)
owners = Owners()
groups = frozenset({'credentials'})
app = create_app(settings=RuntimeSettings(environment='test',access_mode='enforce',
backend='postgresql',v2_groups=groups,v2_write_groups=groups),
port_store=store,access_controller=owners.controller())
async def snapshot(table):
async with store.sessions() as session:
return [dict(row) for row in (await session.execute(sa.select(table))).mappings()]
def fail(connection,cursor,statement,parameters,context,many):
if statement.startswith('INSERT INTO runtime_outcome_outbox'):
raise RuntimeError('fixture outbox failure')
try:
async with httpx.AsyncClient(transport=httpx.ASGITransport(app=app,raise_app_exceptions=False),
base_url='http://test') as client:
created = await client.post('/api/v2/api-consumers',headers=HEADERS,json={'name':'Fixture'})
assert created.status_code == 201
path = '/api-consumers/'+created.json()['id']+'/api-keys'
before = await snapshot(compat_api_consumers)
sa.event.listen(store.engine.sync_engine,'before_cursor_execute',fail)
try:
failed = await client.post(path,headers=HEADERS,json={})
assert failed.status_code == 500
finally:
sa.event.remove(store.engine.sync_engine,'before_cursor_execute',fail)
assert await snapshot(compat_api_consumers) == before
assert not await snapshot(compat_api_keys)
assert len(await pending(store)) == 1
issued = await client.post(path,headers=HEADERS,json={})
assert issued.status_code == 201
assert len(await snapshot(compat_api_keys)) == 1
assert await snapshot(compat_api_consumers) != before
outcomes = await pending(store)
assert len(outcomes) == 2
assert {row['envelope']['data']['operation'] for row in outcomes} == {
'compat.consumer.created','compat.api_key.created'}
assert all(row['envelope']['data']['authorization']['subject'] == 'immutable-root'
for row in outcomes)
assert issued.json()['fullKey'] not in json.dumps(outcomes)
assert len(await snapshot(runtime_audit_ledger)) == 2
finally:
await store.aclose()
asyncio.run(run())

View file

@ -426,6 +426,29 @@ open. Validation: **372 ordinary tests**, **five real PostgreSQL tests** and
**six owner-source Audit Core tests** pass; inventory, package build and isolated
wheel validation pass. No deployment or external owner contract changed.
## Compatibility outcome continuation — 2026-09-28
Verified that all eight implemented SQL compatibility mutation operations share
transaction-linked ledger/outcome custody, through canonical routes and root
aliases. Added enforced authorization, denial, attribution, secret-exclusion and
rollback coverage. Disposable PostgreSQL verifies consumer/key/outcome atomicity,
including rollback of the consumer prefix when outbox insertion fails.
Unimplemented token issuance and deferred POST operations now return `501`, with
matching OpenAPI responses, instead of false success. Missing manifest updates
and activations return `404`. Interaction responses preserve the persisted ID
when callers supply their own ID. These errors do not create committed outcomes.
The [outcome coverage contract](../docs/operation-outcome-audit.md) now includes
implemented SQL compatibility writes. Arbitrary embedded-host/external writes,
owner admission and live acceptance remain open; T03/T04 remain `progress`.
All credentials used in the new tests are ephemeral local fixtures. No live key,
entitlement, database migration or deployment was created.
Validation: `make ci-check` passed **399 ordinary tests** (two opt-in modules
skipped), inventory drift, distribution/isolated-wheel checks and **six disposable
PostgreSQL tests**.
## Acceptance checkpoints
- [x] Architecture/source/runtime review captured; new implementation owner is hub-core