diff --git a/docs/access-profile-v1.md b/docs/access-profile-v1.md index 7f7c21a..10b7311 100644 --- a/docs/access-profile-v1.md +++ b/docs/access-profile-v1.md @@ -56,9 +56,9 @@ A host composes `create_app(access_controller=AccessController(...))` with: only after an operational-custody probe and explicit durable acceptance. Every allow must reach this sink before handler execution; a failed sink blocks reads as well as writes. Authorization receipts say `authorized`, not “operation completed.” Domain commit/outcome audit remains a - separate concern. Native durable mutations now have a - [transaction-linked outcome outbox](operation-outcome-audit.md); compatibility - and external mutations remain outside that slice. + separate concern. Native durable and implemented SQL compatibility mutations have a + [transaction-linked outcome outbox](operation-outcome-audit.md); arbitrary + embedded-host and external mutations remain outside that slice. - The root's existing immutable issuer and subject, supplied after owner resolution. No username, email, first-login promotion or generic role establishes root. diff --git a/docs/operation-outcome-audit.md b/docs/operation-outcome-audit.md index fcfe0aa..7c91730 100644 --- a/docs/operation-outcome-audit.md +++ b/docs/operation-outcome-audit.md @@ -1,4 +1,4 @@ -# Transaction-linked native operation outcomes +# Transaction-linked operation outcomes HUB-WP-0012 source candidate, 2026-09-28. Live sender admission, database rollout and owner acceptance remain open. @@ -26,8 +26,20 @@ retained separately from verified authorization correlation IDs. That verified ID and decision ID join the outcome to the pre-execution signed decision already held by Audit Core. The outcome does not duplicate the signed envelope. -This slice covers the native durable mutation ledger. It does not claim outcome -coverage for compatibility/embedded-host mutations, background projection refresh, +Implemented SQL compatibility writes share this transaction boundary: hub, +manifest create/update/activate, consumer, API key, widget and interaction-event +writes, through both `/api/v2` and root aliases. Key issuance also updates the +consumer in the same transaction; an outbox failure rolls both changes back. +Generated keys never enter the ledger/outcome envelope and cannot authenticate +enforced routes, which still require verified OIDC identity. + +Unimplemented token issuance and deferred POST operations return `501`, including +in OpenAPI, rather than acknowledging a mutation that did not occur. Missing +manifest update/activation returns `404`. Neither creates a committed outcome. +Interaction responses return the persisted ID even when the caller supplies an ID. + +This does not claim outcome coverage for arbitrary embedded-host mutations, +background projection refresh, external services, or in-memory development stores. Non-enforced maintenance writes retain their existing local ledger behavior without fabricating an actor or emitting an attributed remote outcome. @@ -75,7 +87,7 @@ This document neither issues that grant nor claims production delivery. ## Local evidence Tests use SQLite-backed durable stores to prove atomic rollback, handler rejection, -all native mutation families, actor/decision attribution, concurrent request +all native and implemented SQL compatibility mutation families, actor/decision attribution, concurrent request isolation, persisted retry delay, reopen/replay, cancellation, stale-backlog readiness, dispatcher drain/shutdown and migration shape/downgrade refusal. The real Audit Core receiver/storage source accepts the eight-field envelope and @@ -83,7 +95,8 @@ returns a duplicate receipt on replay. Its operational-readiness classification is explicitly a test fixture; this is not live custody evidence. Disposable PostgreSQL tests now verify multiworker lock scheduling, the full -migration chain, rollback, persisted retries and process-exit replay. See the +migration chain, native and compatibility-key rollback, persisted retries and +process-exit replay. See the [PostgreSQL gate](conformance.md#disposable-postgresql-gate). Production grants, retention and deployed failure-detection/receiver acceptance still require receipts. diff --git a/docs/owner-access-integration.md b/docs/owner-access-integration.md index 2929986..130df31 100644 --- a/docs/owner-access-integration.md +++ b/docs/owner-access-integration.md @@ -46,8 +46,8 @@ at three seconds, uses TLS, and never follows redirects. Allow is blocked until the archive accepts the authorization record. A lost receipt blocks the business operation even if the attempt reached storage. This is a pre-execution authorization journal, not proof that an operation committed. -Authorization cannot use a local success buffer or silent redaction. Native -transaction outcomes now use a separate [durable outbox](operation-outcome-audit.md); +Authorization cannot use a local success buffer or silent redaction. Native and +implemented SQL compatibility transaction outcomes use a separate [durable outbox](operation-outcome-audit.md); its production delivery and broader mutation coverage remain T03/T04 gates. The exact verified signed decision is retained under `data.signed_decision` as diff --git a/hub_core/runtime/compat.py b/hub_core/runtime/compat.py index 678b09f..36ae5f1 100644 --- a/hub_core/runtime/compat.py +++ b/hub_core/runtime/compat.py @@ -136,7 +136,7 @@ class SQLCompatibilityStore: ) ).mappings().one_or_none() if current is None: - return {"id": manifest_id, "status": "missing"} + raise HTTPException(status_code=404, detail="manifest not found") payload = dict(current["body"] or {}) payload.update(body) values = { @@ -296,7 +296,7 @@ class SQLCompatibilityStore: payload={"legacy": body}, ) accepted = await self.ports.append_interaction(command) - return {"id": accepted.id, **body} + return {**body, "id": accepted.id} def create_compatibility_router() -> APIRouter: @@ -407,10 +407,10 @@ def create_compatibility_router() -> APIRouter: await _protected(request, "credentials", authorization, write=True) return await _store(request).create_key(consumer_id, body.get("scopes")) - @router.post("/api/v2/token") + @router.post("/api/v2/token", status_code=501) async def token(request: Request, authorization: AuthorizationHeader = None) -> dict: await _protected(request, "credentials", authorization) - return {"access_token": "already-authenticated", "token_type": "bearer"} + raise HTTPException(status_code=501, detail="compatibility token issuance is not implemented") @router.get("/api/v2/widgets") async def list_widgets( @@ -457,7 +457,7 @@ def create_compatibility_router() -> APIRouter: request: Request, authorization: AuthorizationHeader = None ) -> dict: await _protected(request, "deferred", authorization, write=True) - return {"data": []} + raise HTTPException(status_code=501, detail="compatibility operation is not implemented") stem = path.replace("-", "_") router.add_api_route( @@ -470,7 +470,7 @@ def create_compatibility_router() -> APIRouter: f"/api/v2/{path}", accept_deferred, methods=["POST"], - status_code=201, + status_code=501, operation_id=f"compat_create_{stem}_{index}", ) diff --git a/tests/test_compatibility_access.py b/tests/test_compatibility_access.py new file mode 100644 index 0000000..843a635 --- /dev/null +++ b/tests/test_compatibility_access.py @@ -0,0 +1,140 @@ +"""Enforced compatibility writes: real SQL transactions, synthetic authority.""" +import json +from dataclasses import replace +from uuid import uuid4 + +import pytest +import sqlalchemy as sa +from fastapi.testclient import TestClient + +from hub_core.runtime.app import create_app +from hub_core.runtime.config import RuntimeSettings +from hub_core.runtime.tables import (compat_api_keys, compat_api_consumers, + runtime_audit_ledger, runtime_outcome_outbox, runtime_interaction_events) +from test_access_boundary import HEADERS +from test_outcome_audit import target, rows + +GROUPS = frozenset({'registry','credentials','interaction','deferred','system'}) + + +@pytest.fixture +def compatibility(target): + _,store,owners,url = target + settings = RuntimeSettings(environment='test',access_mode='enforce',backend='postgresql', + database_url=url,v2_groups=GROUPS,v2_write_groups=GROUPS) + app = create_app(settings=settings,port_store=store,access_controller=owners.controller()) + with TestClient(app,raise_server_exceptions=False) as client: + yield client,store,owners + + +@pytest.mark.parametrize('prefix',['/api/v2','']) +def test_all_implemented_compatibility_mutations_have_attributed_outcomes(compatibility,prefix): + client,store,owners = compatibility + ids = [] + def write(path,body,method='POST',status=201): + response = client.request(method,prefix+path,headers=HEADERS,json=body) + assert response.status_code == status, response.text + ids.append(response.headers['x-correlation-id']) + return response.json() + hub = write('/hubs',{'name':'Fixture Hub','slug':'fixture-hub'}) + manifest = write('/hub-capability-manifests',{'hubId':hub['id'],'manifestVersion':'1.0'}) + write('/hub-capability-manifests/'+manifest['id'],{'description':'private manifest value'},'PATCH',200) + write('/hub-capability-manifests/'+manifest['id']+'/activate',{},status=200) + consumer = write('/api-consumers',{'name':'Fixture Consumer'}) + issued = write('/api-consumers/'+consumer['id']+'/api-keys',{'scopes':'fixture-only'}) + widget = write('/widgets',{'hubId':hub['id'],'name':'Fixture Widget'}) + interaction = write('/interaction-events',{'widgetId':widget['id'],'eventType':'fixture.interaction', + 'id':'caller-forged-id','metadata':{'text':'private event body'}}) + assert interaction['id'] != 'caller-forged-id' + event, = rows(store,runtime_interaction_events) + assert interaction['id'] == event['id'] + pending = rows(store,runtime_outcome_outbox) + assert len(pending) == len(ids) == 8 + assert {row['envelope']['correlation_id'] for row in pending} == set(ids) + decisions = {record['correlation_id']: record for record in owners.records} + operations = set() + for row in pending: + event = row['envelope'] + auth = event['data']['authorization'] + assert auth['subject'] == 'immutable-root' + assert auth['decision_id'] == decisions[event['correlation_id']]['decision_id'] + operations.add(event['data']['operation']) + assert operations == {'compat.hub.created','compat.manifest.created','compat.manifest.updated', + 'compat.manifest.activated','compat.consumer.created','compat.api_key.created', + 'compat.widget.created','event.interaction.accepted'} + serialized = json.dumps(pending)+json.dumps(rows(store,runtime_audit_ledger),default=str) + assert issued['fullKey'] not in serialized + assert 'private manifest value' not in serialized and 'private event body' not in serialized + # Legacy compatibility keys cannot authenticate an enforced route. + assert client.get(prefix+'/hubs',headers={'Authorization':'Bearer '+issued['fullKey']}).status_code == 401 + + +@pytest.mark.parametrize('prefix',['/api/v2','']) +@pytest.mark.parametrize('path',['annotations','requirement-candidates','decision-records', + 'deployment-records','outcome-signals','token']) +def test_unimplemented_compatibility_writes_never_report_success(compatibility,prefix,path): + client,store,_ = compatibility + response = client.post(prefix+'/'+path,headers=HEADERS,json={}) + assert response.status_code == 501 + responses = client.app.openapi()['paths']['/api/v2/'+path]['post']['responses'] + assert '501' in responses and '200' not in responses and '201' not in responses + assert not rows(store,runtime_outcome_outbox) + assert not rows(store,runtime_audit_ledger) + + +@pytest.mark.parametrize('prefix',['/api/v2','']) +@pytest.mark.parametrize('activate',[False,True]) +def test_missing_manifest_is_not_a_successful_mutation(compatibility,prefix,activate): + client,store,_ = compatibility + path = prefix+'/hub-capability-manifests/'+str(uuid4()) + response = client.request('POST' if activate else 'PATCH',path+('/activate' if activate else ''), + headers=HEADERS,json={}) + assert response.status_code == 404 + assert not rows(store,runtime_outcome_outbox) + + +def test_key_issuance_and_consumer_update_roll_back_on_outbox_failure(compatibility): + client,store,_ = compatibility + consumer = client.post('/api/v2/api-consumers',headers=HEADERS,json={'name':'Fixture Consumer'}).json() + before = rows(store,compat_api_consumers) + def fail(connection,cursor,statement,parameters,context,many): + if statement.startswith('INSERT INTO runtime_outcome_outbox'): + raise RuntimeError('fixture outbox failure') + sa.event.listen(store.engine.sync_engine,'before_cursor_execute',fail) + try: + response = client.post('/api/v2/api-consumers/'+consumer['id']+'/api-keys',headers=HEADERS,json={}) + assert response.status_code == 500 + assert 'fullKey' not in response.text + finally: + sa.event.remove(store.engine.sync_engine,'before_cursor_execute',fail) + assert not rows(store,compat_api_keys) + assert rows(store,compat_api_consumers) == before + assert len(rows(store,runtime_outcome_outbox)) == 1 + assert len(rows(store,runtime_audit_ledger)) == 1 + + +@pytest.mark.parametrize('failure,status',[('anonymous',401),('invalid',401),('ordinary',403), + ('revoked',403),('wrong_tenant',403),('policy',503),('audit',503)]) +def test_compatibility_denials_do_not_mutate_or_queue_outcomes(compatibility,failure,status): + client,store,owners = compatibility + headers = HEADERS + if failure == 'anonymous': headers = {} + elif failure == 'invalid': headers = {'Authorization':'Bearer wrong'} + elif failure == 'ordinary': owners.actor = replace(owners.actor,subject='ordinary') + elif failure == 'revoked': owners.facts = replace(owners.facts,root_entitled=False) + elif failure == 'wrong_tenant': owners.actor = replace(owners.actor,tenant='tenant:other') + elif failure == 'policy': owners.policy_down = True + else: owners.audit_down = True + for prefix in ['/api/v2','']: + assert client.post(prefix+'/api-consumers',headers=headers,json={'name':'Never created'}).status_code == status + assert not rows(store,compat_api_consumers) + assert not rows(store,runtime_outcome_outbox) + + +def test_read_only_and_disabled_group_still_prevent_writes(compatibility): + client,store,_ = compatibility + client.app.state.settings = replace(client.app.state.settings,v2_write_groups=frozenset()) + assert client.post('/api-consumers',headers=HEADERS,json={'name':'Blocked'}).status_code == 503 + client.app.state.settings = replace(client.app.state.settings,v2_groups=frozenset()) + assert client.post('/api-consumers',headers=HEADERS,json={'name':'Blocked'}).status_code == 404 + assert not rows(store,runtime_outcome_outbox) diff --git a/tests/test_postgres_integration.py b/tests/test_postgres_integration.py index 9805338..95a41fb 100644 --- a/tests/test_postgres_integration.py +++ b/tests/test_postgres_integration.py @@ -256,3 +256,57 @@ def test_receiver_failure_persists_backoff_across_connection_reopen(database): finally: await reopened.aclose() asyncio.run(run()) + + +def test_compatibility_key_transaction_and_outcome_rollback(database): + import httpx + from hub_core.runtime.app import create_app + from hub_core.runtime.config import RuntimeSettings + from hub_core.runtime.tables import compat_api_consumers, compat_api_keys + from test_access_boundary import HEADERS + + url,_ = database + async def run(): + store = store_for(url) + owners = Owners() + groups = frozenset({'credentials'}) + app = create_app(settings=RuntimeSettings(environment='test',access_mode='enforce', + backend='postgresql',v2_groups=groups,v2_write_groups=groups), + port_store=store,access_controller=owners.controller()) + async def snapshot(table): + async with store.sessions() as session: + return [dict(row) for row in (await session.execute(sa.select(table))).mappings()] + def fail(connection,cursor,statement,parameters,context,many): + if statement.startswith('INSERT INTO runtime_outcome_outbox'): + raise RuntimeError('fixture outbox failure') + try: + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=app,raise_app_exceptions=False), + base_url='http://test') as client: + created = await client.post('/api/v2/api-consumers',headers=HEADERS,json={'name':'Fixture'}) + assert created.status_code == 201 + path = '/api-consumers/'+created.json()['id']+'/api-keys' + before = await snapshot(compat_api_consumers) + sa.event.listen(store.engine.sync_engine,'before_cursor_execute',fail) + try: + failed = await client.post(path,headers=HEADERS,json={}) + assert failed.status_code == 500 + finally: + sa.event.remove(store.engine.sync_engine,'before_cursor_execute',fail) + assert await snapshot(compat_api_consumers) == before + assert not await snapshot(compat_api_keys) + assert len(await pending(store)) == 1 + issued = await client.post(path,headers=HEADERS,json={}) + assert issued.status_code == 201 + assert len(await snapshot(compat_api_keys)) == 1 + assert await snapshot(compat_api_consumers) != before + outcomes = await pending(store) + assert len(outcomes) == 2 + assert {row['envelope']['data']['operation'] for row in outcomes} == { + 'compat.consumer.created','compat.api_key.created'} + assert all(row['envelope']['data']['authorization']['subject'] == 'immutable-root' + for row in outcomes) + assert issued.json()['fullKey'] not in json.dumps(outcomes) + assert len(await snapshot(runtime_audit_ledger)) == 2 + finally: + await store.aclose() + asyncio.run(run()) diff --git a/workplans/HUB-WP-0012-netkingdom-platform-root-access.md b/workplans/HUB-WP-0012-netkingdom-platform-root-access.md index 55f09d0..d2b0856 100644 --- a/workplans/HUB-WP-0012-netkingdom-platform-root-access.md +++ b/workplans/HUB-WP-0012-netkingdom-platform-root-access.md @@ -426,6 +426,29 @@ open. Validation: **372 ordinary tests**, **five real PostgreSQL tests** and **six owner-source Audit Core tests** pass; inventory, package build and isolated wheel validation pass. No deployment or external owner contract changed. +## Compatibility outcome continuation — 2026-09-28 + +Verified that all eight implemented SQL compatibility mutation operations share +transaction-linked ledger/outcome custody, through canonical routes and root +aliases. Added enforced authorization, denial, attribution, secret-exclusion and +rollback coverage. Disposable PostgreSQL verifies consumer/key/outcome atomicity, +including rollback of the consumer prefix when outbox insertion fails. + +Unimplemented token issuance and deferred POST operations now return `501`, with +matching OpenAPI responses, instead of false success. Missing manifest updates +and activations return `404`. Interaction responses preserve the persisted ID +when callers supply their own ID. These errors do not create committed outcomes. + +The [outcome coverage contract](../docs/operation-outcome-audit.md) now includes +implemented SQL compatibility writes. Arbitrary embedded-host/external writes, +owner admission and live acceptance remain open; T03/T04 remain `progress`. +All credentials used in the new tests are ephemeral local fixtures. No live key, +entitlement, database migration or deployment was created. + +Validation: `make ci-check` passed **399 ordinary tests** (two opt-in modules +skipped), inventory drift, distribution/isolated-wheel checks and **six disposable +PostgreSQL tests**. + ## Acceptance checkpoints - [x] Architecture/source/runtime review captured; new implementation owner is hub-core