fix: report truthful compatibility outcomes and verify audit coverage
Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a0e747-8f27-7242-8df8-8bc44f88c929
This commit is contained in:
parent
c0383c9c2b
commit
72f3513954
7 changed files with 246 additions and 16 deletions
|
|
@ -56,9 +56,9 @@ A host composes `create_app(access_controller=AccessController(...))` with:
|
|||
only after an operational-custody probe and explicit durable acceptance. Every allow must reach this sink before handler execution;
|
||||
a failed sink blocks reads as well as writes. Authorization receipts say
|
||||
`authorized`, not “operation completed.” Domain commit/outcome audit remains a
|
||||
separate concern. Native durable mutations now have a
|
||||
[transaction-linked outcome outbox](operation-outcome-audit.md); compatibility
|
||||
and external mutations remain outside that slice.
|
||||
separate concern. Native durable and implemented SQL compatibility mutations have a
|
||||
[transaction-linked outcome outbox](operation-outcome-audit.md); arbitrary
|
||||
embedded-host and external mutations remain outside that slice.
|
||||
- The root's existing immutable issuer and subject, supplied after owner resolution.
|
||||
No username, email, first-login promotion or generic role establishes root.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
# Transaction-linked native operation outcomes
|
||||
# Transaction-linked operation outcomes
|
||||
|
||||
HUB-WP-0012 source candidate, 2026-09-28. Live sender admission, database rollout
|
||||
and owner acceptance remain open.
|
||||
|
|
@ -26,8 +26,20 @@ retained separately from verified authorization correlation IDs. That verified
|
|||
ID and decision ID join the outcome to the pre-execution signed decision already
|
||||
held by Audit Core. The outcome does not duplicate the signed envelope.
|
||||
|
||||
This slice covers the native durable mutation ledger. It does not claim outcome
|
||||
coverage for compatibility/embedded-host mutations, background projection refresh,
|
||||
Implemented SQL compatibility writes share this transaction boundary: hub,
|
||||
manifest create/update/activate, consumer, API key, widget and interaction-event
|
||||
writes, through both `/api/v2` and root aliases. Key issuance also updates the
|
||||
consumer in the same transaction; an outbox failure rolls both changes back.
|
||||
Generated keys never enter the ledger/outcome envelope and cannot authenticate
|
||||
enforced routes, which still require verified OIDC identity.
|
||||
|
||||
Unimplemented token issuance and deferred POST operations return `501`, including
|
||||
in OpenAPI, rather than acknowledging a mutation that did not occur. Missing
|
||||
manifest update/activation returns `404`. Neither creates a committed outcome.
|
||||
Interaction responses return the persisted ID even when the caller supplies an ID.
|
||||
|
||||
This does not claim outcome coverage for arbitrary embedded-host mutations,
|
||||
background projection refresh,
|
||||
external services, or in-memory development stores. Non-enforced maintenance
|
||||
writes retain their existing local ledger behavior without fabricating an actor
|
||||
or emitting an attributed remote outcome.
|
||||
|
|
@ -75,7 +87,7 @@ This document neither issues that grant nor claims production delivery.
|
|||
## Local evidence
|
||||
|
||||
Tests use SQLite-backed durable stores to prove atomic rollback, handler rejection,
|
||||
all native mutation families, actor/decision attribution, concurrent request
|
||||
all native and implemented SQL compatibility mutation families, actor/decision attribution, concurrent request
|
||||
isolation, persisted retry delay, reopen/replay, cancellation, stale-backlog
|
||||
readiness, dispatcher drain/shutdown and migration shape/downgrade refusal.
|
||||
The real Audit Core receiver/storage source accepts the eight-field envelope and
|
||||
|
|
@ -83,7 +95,8 @@ returns a duplicate receipt on replay. Its operational-readiness classification
|
|||
is explicitly a test fixture; this is not live custody evidence.
|
||||
|
||||
Disposable PostgreSQL tests now verify multiworker lock scheduling, the full
|
||||
migration chain, rollback, persisted retries and process-exit replay. See the
|
||||
migration chain, native and compatibility-key rollback, persisted retries and
|
||||
process-exit replay. See the
|
||||
[PostgreSQL gate](conformance.md#disposable-postgresql-gate). Production grants,
|
||||
retention and deployed failure-detection/receiver acceptance still require receipts.
|
||||
|
||||
|
|
|
|||
|
|
@ -46,8 +46,8 @@ at three seconds, uses TLS, and never follows redirects.
|
|||
Allow is blocked until the archive accepts the authorization record. A lost
|
||||
receipt blocks the business operation even if the attempt reached storage. This
|
||||
is a pre-execution authorization journal, not proof that an operation committed.
|
||||
Authorization cannot use a local success buffer or silent redaction. Native
|
||||
transaction outcomes now use a separate [durable outbox](operation-outcome-audit.md);
|
||||
Authorization cannot use a local success buffer or silent redaction. Native and
|
||||
implemented SQL compatibility transaction outcomes use a separate [durable outbox](operation-outcome-audit.md);
|
||||
its production delivery and broader mutation coverage remain T03/T04 gates.
|
||||
|
||||
The exact verified signed decision is retained under `data.signed_decision` as
|
||||
|
|
|
|||
|
|
@ -136,7 +136,7 @@ class SQLCompatibilityStore:
|
|||
)
|
||||
).mappings().one_or_none()
|
||||
if current is None:
|
||||
return {"id": manifest_id, "status": "missing"}
|
||||
raise HTTPException(status_code=404, detail="manifest not found")
|
||||
payload = dict(current["body"] or {})
|
||||
payload.update(body)
|
||||
values = {
|
||||
|
|
@ -296,7 +296,7 @@ class SQLCompatibilityStore:
|
|||
payload={"legacy": body},
|
||||
)
|
||||
accepted = await self.ports.append_interaction(command)
|
||||
return {"id": accepted.id, **body}
|
||||
return {**body, "id": accepted.id}
|
||||
|
||||
|
||||
def create_compatibility_router() -> APIRouter:
|
||||
|
|
@ -407,10 +407,10 @@ def create_compatibility_router() -> APIRouter:
|
|||
await _protected(request, "credentials", authorization, write=True)
|
||||
return await _store(request).create_key(consumer_id, body.get("scopes"))
|
||||
|
||||
@router.post("/api/v2/token")
|
||||
@router.post("/api/v2/token", status_code=501)
|
||||
async def token(request: Request, authorization: AuthorizationHeader = None) -> dict:
|
||||
await _protected(request, "credentials", authorization)
|
||||
return {"access_token": "already-authenticated", "token_type": "bearer"}
|
||||
raise HTTPException(status_code=501, detail="compatibility token issuance is not implemented")
|
||||
|
||||
@router.get("/api/v2/widgets")
|
||||
async def list_widgets(
|
||||
|
|
@ -457,7 +457,7 @@ def create_compatibility_router() -> APIRouter:
|
|||
request: Request, authorization: AuthorizationHeader = None
|
||||
) -> dict:
|
||||
await _protected(request, "deferred", authorization, write=True)
|
||||
return {"data": []}
|
||||
raise HTTPException(status_code=501, detail="compatibility operation is not implemented")
|
||||
|
||||
stem = path.replace("-", "_")
|
||||
router.add_api_route(
|
||||
|
|
@ -470,7 +470,7 @@ def create_compatibility_router() -> APIRouter:
|
|||
f"/api/v2/{path}",
|
||||
accept_deferred,
|
||||
methods=["POST"],
|
||||
status_code=201,
|
||||
status_code=501,
|
||||
operation_id=f"compat_create_{stem}_{index}",
|
||||
)
|
||||
|
||||
|
|
|
|||
140
tests/test_compatibility_access.py
Normal file
140
tests/test_compatibility_access.py
Normal file
|
|
@ -0,0 +1,140 @@
|
|||
"""Enforced compatibility writes: real SQL transactions, synthetic authority."""
|
||||
import json
|
||||
from dataclasses import replace
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
import sqlalchemy as sa
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from hub_core.runtime.app import create_app
|
||||
from hub_core.runtime.config import RuntimeSettings
|
||||
from hub_core.runtime.tables import (compat_api_keys, compat_api_consumers,
|
||||
runtime_audit_ledger, runtime_outcome_outbox, runtime_interaction_events)
|
||||
from test_access_boundary import HEADERS
|
||||
from test_outcome_audit import target, rows
|
||||
|
||||
GROUPS = frozenset({'registry','credentials','interaction','deferred','system'})
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def compatibility(target):
|
||||
_,store,owners,url = target
|
||||
settings = RuntimeSettings(environment='test',access_mode='enforce',backend='postgresql',
|
||||
database_url=url,v2_groups=GROUPS,v2_write_groups=GROUPS)
|
||||
app = create_app(settings=settings,port_store=store,access_controller=owners.controller())
|
||||
with TestClient(app,raise_server_exceptions=False) as client:
|
||||
yield client,store,owners
|
||||
|
||||
|
||||
@pytest.mark.parametrize('prefix',['/api/v2',''])
|
||||
def test_all_implemented_compatibility_mutations_have_attributed_outcomes(compatibility,prefix):
|
||||
client,store,owners = compatibility
|
||||
ids = []
|
||||
def write(path,body,method='POST',status=201):
|
||||
response = client.request(method,prefix+path,headers=HEADERS,json=body)
|
||||
assert response.status_code == status, response.text
|
||||
ids.append(response.headers['x-correlation-id'])
|
||||
return response.json()
|
||||
hub = write('/hubs',{'name':'Fixture Hub','slug':'fixture-hub'})
|
||||
manifest = write('/hub-capability-manifests',{'hubId':hub['id'],'manifestVersion':'1.0'})
|
||||
write('/hub-capability-manifests/'+manifest['id'],{'description':'private manifest value'},'PATCH',200)
|
||||
write('/hub-capability-manifests/'+manifest['id']+'/activate',{},status=200)
|
||||
consumer = write('/api-consumers',{'name':'Fixture Consumer'})
|
||||
issued = write('/api-consumers/'+consumer['id']+'/api-keys',{'scopes':'fixture-only'})
|
||||
widget = write('/widgets',{'hubId':hub['id'],'name':'Fixture Widget'})
|
||||
interaction = write('/interaction-events',{'widgetId':widget['id'],'eventType':'fixture.interaction',
|
||||
'id':'caller-forged-id','metadata':{'text':'private event body'}})
|
||||
assert interaction['id'] != 'caller-forged-id'
|
||||
event, = rows(store,runtime_interaction_events)
|
||||
assert interaction['id'] == event['id']
|
||||
pending = rows(store,runtime_outcome_outbox)
|
||||
assert len(pending) == len(ids) == 8
|
||||
assert {row['envelope']['correlation_id'] for row in pending} == set(ids)
|
||||
decisions = {record['correlation_id']: record for record in owners.records}
|
||||
operations = set()
|
||||
for row in pending:
|
||||
event = row['envelope']
|
||||
auth = event['data']['authorization']
|
||||
assert auth['subject'] == 'immutable-root'
|
||||
assert auth['decision_id'] == decisions[event['correlation_id']]['decision_id']
|
||||
operations.add(event['data']['operation'])
|
||||
assert operations == {'compat.hub.created','compat.manifest.created','compat.manifest.updated',
|
||||
'compat.manifest.activated','compat.consumer.created','compat.api_key.created',
|
||||
'compat.widget.created','event.interaction.accepted'}
|
||||
serialized = json.dumps(pending)+json.dumps(rows(store,runtime_audit_ledger),default=str)
|
||||
assert issued['fullKey'] not in serialized
|
||||
assert 'private manifest value' not in serialized and 'private event body' not in serialized
|
||||
# Legacy compatibility keys cannot authenticate an enforced route.
|
||||
assert client.get(prefix+'/hubs',headers={'Authorization':'Bearer '+issued['fullKey']}).status_code == 401
|
||||
|
||||
|
||||
@pytest.mark.parametrize('prefix',['/api/v2',''])
|
||||
@pytest.mark.parametrize('path',['annotations','requirement-candidates','decision-records',
|
||||
'deployment-records','outcome-signals','token'])
|
||||
def test_unimplemented_compatibility_writes_never_report_success(compatibility,prefix,path):
|
||||
client,store,_ = compatibility
|
||||
response = client.post(prefix+'/'+path,headers=HEADERS,json={})
|
||||
assert response.status_code == 501
|
||||
responses = client.app.openapi()['paths']['/api/v2/'+path]['post']['responses']
|
||||
assert '501' in responses and '200' not in responses and '201' not in responses
|
||||
assert not rows(store,runtime_outcome_outbox)
|
||||
assert not rows(store,runtime_audit_ledger)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('prefix',['/api/v2',''])
|
||||
@pytest.mark.parametrize('activate',[False,True])
|
||||
def test_missing_manifest_is_not_a_successful_mutation(compatibility,prefix,activate):
|
||||
client,store,_ = compatibility
|
||||
path = prefix+'/hub-capability-manifests/'+str(uuid4())
|
||||
response = client.request('POST' if activate else 'PATCH',path+('/activate' if activate else ''),
|
||||
headers=HEADERS,json={})
|
||||
assert response.status_code == 404
|
||||
assert not rows(store,runtime_outcome_outbox)
|
||||
|
||||
|
||||
def test_key_issuance_and_consumer_update_roll_back_on_outbox_failure(compatibility):
|
||||
client,store,_ = compatibility
|
||||
consumer = client.post('/api/v2/api-consumers',headers=HEADERS,json={'name':'Fixture Consumer'}).json()
|
||||
before = rows(store,compat_api_consumers)
|
||||
def fail(connection,cursor,statement,parameters,context,many):
|
||||
if statement.startswith('INSERT INTO runtime_outcome_outbox'):
|
||||
raise RuntimeError('fixture outbox failure')
|
||||
sa.event.listen(store.engine.sync_engine,'before_cursor_execute',fail)
|
||||
try:
|
||||
response = client.post('/api/v2/api-consumers/'+consumer['id']+'/api-keys',headers=HEADERS,json={})
|
||||
assert response.status_code == 500
|
||||
assert 'fullKey' not in response.text
|
||||
finally:
|
||||
sa.event.remove(store.engine.sync_engine,'before_cursor_execute',fail)
|
||||
assert not rows(store,compat_api_keys)
|
||||
assert rows(store,compat_api_consumers) == before
|
||||
assert len(rows(store,runtime_outcome_outbox)) == 1
|
||||
assert len(rows(store,runtime_audit_ledger)) == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize('failure,status',[('anonymous',401),('invalid',401),('ordinary',403),
|
||||
('revoked',403),('wrong_tenant',403),('policy',503),('audit',503)])
|
||||
def test_compatibility_denials_do_not_mutate_or_queue_outcomes(compatibility,failure,status):
|
||||
client,store,owners = compatibility
|
||||
headers = HEADERS
|
||||
if failure == 'anonymous': headers = {}
|
||||
elif failure == 'invalid': headers = {'Authorization':'Bearer wrong'}
|
||||
elif failure == 'ordinary': owners.actor = replace(owners.actor,subject='ordinary')
|
||||
elif failure == 'revoked': owners.facts = replace(owners.facts,root_entitled=False)
|
||||
elif failure == 'wrong_tenant': owners.actor = replace(owners.actor,tenant='tenant:other')
|
||||
elif failure == 'policy': owners.policy_down = True
|
||||
else: owners.audit_down = True
|
||||
for prefix in ['/api/v2','']:
|
||||
assert client.post(prefix+'/api-consumers',headers=headers,json={'name':'Never created'}).status_code == status
|
||||
assert not rows(store,compat_api_consumers)
|
||||
assert not rows(store,runtime_outcome_outbox)
|
||||
|
||||
|
||||
def test_read_only_and_disabled_group_still_prevent_writes(compatibility):
|
||||
client,store,_ = compatibility
|
||||
client.app.state.settings = replace(client.app.state.settings,v2_write_groups=frozenset())
|
||||
assert client.post('/api-consumers',headers=HEADERS,json={'name':'Blocked'}).status_code == 503
|
||||
client.app.state.settings = replace(client.app.state.settings,v2_groups=frozenset())
|
||||
assert client.post('/api-consumers',headers=HEADERS,json={'name':'Blocked'}).status_code == 404
|
||||
assert not rows(store,runtime_outcome_outbox)
|
||||
|
|
@ -256,3 +256,57 @@ def test_receiver_failure_persists_backoff_across_connection_reopen(database):
|
|||
finally:
|
||||
await reopened.aclose()
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_compatibility_key_transaction_and_outcome_rollback(database):
|
||||
import httpx
|
||||
from hub_core.runtime.app import create_app
|
||||
from hub_core.runtime.config import RuntimeSettings
|
||||
from hub_core.runtime.tables import compat_api_consumers, compat_api_keys
|
||||
from test_access_boundary import HEADERS
|
||||
|
||||
url,_ = database
|
||||
async def run():
|
||||
store = store_for(url)
|
||||
owners = Owners()
|
||||
groups = frozenset({'credentials'})
|
||||
app = create_app(settings=RuntimeSettings(environment='test',access_mode='enforce',
|
||||
backend='postgresql',v2_groups=groups,v2_write_groups=groups),
|
||||
port_store=store,access_controller=owners.controller())
|
||||
async def snapshot(table):
|
||||
async with store.sessions() as session:
|
||||
return [dict(row) for row in (await session.execute(sa.select(table))).mappings()]
|
||||
def fail(connection,cursor,statement,parameters,context,many):
|
||||
if statement.startswith('INSERT INTO runtime_outcome_outbox'):
|
||||
raise RuntimeError('fixture outbox failure')
|
||||
try:
|
||||
async with httpx.AsyncClient(transport=httpx.ASGITransport(app=app,raise_app_exceptions=False),
|
||||
base_url='http://test') as client:
|
||||
created = await client.post('/api/v2/api-consumers',headers=HEADERS,json={'name':'Fixture'})
|
||||
assert created.status_code == 201
|
||||
path = '/api-consumers/'+created.json()['id']+'/api-keys'
|
||||
before = await snapshot(compat_api_consumers)
|
||||
sa.event.listen(store.engine.sync_engine,'before_cursor_execute',fail)
|
||||
try:
|
||||
failed = await client.post(path,headers=HEADERS,json={})
|
||||
assert failed.status_code == 500
|
||||
finally:
|
||||
sa.event.remove(store.engine.sync_engine,'before_cursor_execute',fail)
|
||||
assert await snapshot(compat_api_consumers) == before
|
||||
assert not await snapshot(compat_api_keys)
|
||||
assert len(await pending(store)) == 1
|
||||
issued = await client.post(path,headers=HEADERS,json={})
|
||||
assert issued.status_code == 201
|
||||
assert len(await snapshot(compat_api_keys)) == 1
|
||||
assert await snapshot(compat_api_consumers) != before
|
||||
outcomes = await pending(store)
|
||||
assert len(outcomes) == 2
|
||||
assert {row['envelope']['data']['operation'] for row in outcomes} == {
|
||||
'compat.consumer.created','compat.api_key.created'}
|
||||
assert all(row['envelope']['data']['authorization']['subject'] == 'immutable-root'
|
||||
for row in outcomes)
|
||||
assert issued.json()['fullKey'] not in json.dumps(outcomes)
|
||||
assert len(await snapshot(runtime_audit_ledger)) == 2
|
||||
finally:
|
||||
await store.aclose()
|
||||
asyncio.run(run())
|
||||
|
|
|
|||
|
|
@ -426,6 +426,29 @@ open. Validation: **372 ordinary tests**, **five real PostgreSQL tests** and
|
|||
**six owner-source Audit Core tests** pass; inventory, package build and isolated
|
||||
wheel validation pass. No deployment or external owner contract changed.
|
||||
|
||||
## Compatibility outcome continuation — 2026-09-28
|
||||
|
||||
Verified that all eight implemented SQL compatibility mutation operations share
|
||||
transaction-linked ledger/outcome custody, through canonical routes and root
|
||||
aliases. Added enforced authorization, denial, attribution, secret-exclusion and
|
||||
rollback coverage. Disposable PostgreSQL verifies consumer/key/outcome atomicity,
|
||||
including rollback of the consumer prefix when outbox insertion fails.
|
||||
|
||||
Unimplemented token issuance and deferred POST operations now return `501`, with
|
||||
matching OpenAPI responses, instead of false success. Missing manifest updates
|
||||
and activations return `404`. Interaction responses preserve the persisted ID
|
||||
when callers supply their own ID. These errors do not create committed outcomes.
|
||||
|
||||
The [outcome coverage contract](../docs/operation-outcome-audit.md) now includes
|
||||
implemented SQL compatibility writes. Arbitrary embedded-host/external writes,
|
||||
owner admission and live acceptance remain open; T03/T04 remain `progress`.
|
||||
All credentials used in the new tests are ephemeral local fixtures. No live key,
|
||||
entitlement, database migration or deployment was created.
|
||||
|
||||
Validation: `make ci-check` passed **399 ordinary tests** (two opt-in modules
|
||||
skipped), inventory drift, distribution/isolated-wheel checks and **six disposable
|
||||
PostgreSQL tests**.
|
||||
|
||||
## Acceptance checkpoints
|
||||
|
||||
- [x] Architecture/source/runtime review captured; new implementation owner is hub-core
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue