Bind sender identities, add operator read surface, real serving layer
AUDIT-WP-0004 T03, T05, T06.
T03 - the receiver accepted whatever tenant and source a caller sent as long
as it held the one shared token, despite WP-0003 recording tenant isolation as
delivered. audit_core.senders binds each credential to the sources and tenants
it may assert, driven by AUDIT_CORE_SENDERS rather than literals. Identities
hold a list of tokens so rotation publishes the replacement alongside the
incumbent and needs no delivery gap. Read is a separate privilege from write,
so a sender credential cannot read the audit trail back.
T05 - lookup by event id, lookup by correlation id, and a dead-letter view.
Rejections are recorded rather than silently dropped. An event rejected for
carrying secret-shaped material has its payload withheld: storing it would
write that material into the audit store, which is what the rejection exists
to prevent. Reason and payload hash are kept so it stays traceable.
Replay is deliberately not built here. Idempotent replay is a property of the
durable store and building it against SQLite would produce a second
implementation to throw away; it lands with the Postgres backend in
AUDIT-WP-0005-T01.
T06 - serving moves to waitress with configurable threads and channel timeout,
installed in the image via the serve extra. Without it the entrypoint falls
back to a threaded wsgiref server with a socket timeout and graceful shutdown
on SIGTERM, and logs a warning so a deployment cannot quietly land on the
fallback. Metric counters deferred to WP-0005-T03 to be designed against the
real scrape path.
Tests 36 -> 46, covering cross-tenant and cross-source refusal, token
rotation, read/write privilege separation, correlation lookup, and payload
withholding on secret rejection.
Remaining in WP-0004: T04 redaction policy, which needs a decision on whether
a secret-shaped field is a rejection or a redaction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-10 14:50:02 +02:00
|
|
|
"""Sender identities and what they are permitted to claim.
|
|
|
|
|
|
|
|
|
|
AUDIT-WP-0004-T03. WP-0003 recorded tenant isolation as delivered, but the
|
|
|
|
|
receiver accepted whatever ``tenant`` and ``source`` a caller sent as long as
|
|
|
|
|
it held the one shared token. This module binds a credential to the sources and
|
|
|
|
|
tenants it may write for, and drives that binding from configuration rather
|
|
|
|
|
than literals.
|
|
|
|
|
|
|
|
|
|
Each identity carries a *list* of tokens so a credential can be rotated without
|
|
|
|
|
a delivery gap: publish the replacement alongside the incumbent, move the
|
|
|
|
|
sender, then drop the old one.
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
from __future__ import annotations
|
|
|
|
|
|
|
|
|
|
import hmac
|
|
|
|
|
import json
|
|
|
|
|
import os
|
|
|
|
|
from dataclasses import dataclass, field
|
2026-08-16 00:24:33 +02:00
|
|
|
from pathlib import Path
|
Bind sender identities, add operator read surface, real serving layer
AUDIT-WP-0004 T03, T05, T06.
T03 - the receiver accepted whatever tenant and source a caller sent as long
as it held the one shared token, despite WP-0003 recording tenant isolation as
delivered. audit_core.senders binds each credential to the sources and tenants
it may assert, driven by AUDIT_CORE_SENDERS rather than literals. Identities
hold a list of tokens so rotation publishes the replacement alongside the
incumbent and needs no delivery gap. Read is a separate privilege from write,
so a sender credential cannot read the audit trail back.
T05 - lookup by event id, lookup by correlation id, and a dead-letter view.
Rejections are recorded rather than silently dropped. An event rejected for
carrying secret-shaped material has its payload withheld: storing it would
write that material into the audit store, which is what the rejection exists
to prevent. Reason and payload hash are kept so it stays traceable.
Replay is deliberately not built here. Idempotent replay is a property of the
durable store and building it against SQLite would produce a second
implementation to throw away; it lands with the Postgres backend in
AUDIT-WP-0005-T01.
T06 - serving moves to waitress with configurable threads and channel timeout,
installed in the image via the serve extra. Without it the entrypoint falls
back to a threaded wsgiref server with a socket timeout and graceful shutdown
on SIGTERM, and logs a warning so a deployment cannot quietly land on the
fallback. Metric counters deferred to WP-0005-T03 to be designed against the
real scrape path.
Tests 36 -> 46, covering cross-tenant and cross-source refusal, token
rotation, read/write privilege separation, correlation lookup, and payload
withholding on secret rejection.
Remaining in WP-0004: T04 redaction policy, which needs a decision on whether
a secret-shaped field is a rejection or a redaction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-10 14:50:02 +02:00
|
|
|
from typing import Any, Iterable
|
|
|
|
|
|
Redact secret-shaped fields by default, countable per field path
AUDIT-WP-0004-T04, closing the workplan.
Decision (Bernd): default to redaction, allow rejection per sender. Losing an
audit record over one field is worse than storing it masked, but a
higher-assurance channel must be able to refuse rather than mask. secret_policy
is set per sender identity in AUDIT_CORE_SENDERS and defaults to redact.
Detection now covers the whole payload at any depth, including lists, rather
than only the top level of data. Under redaction the value is masked and the
key is preserved: dropping the key would hide that the sender transmitted the
field at all, which is exactly what an operator needs in order to stop it. The
stored record carries details.redaction with policy and affected paths, so a
reader never has to infer whether what they see is what was sent.
Idempotency is unaffected - the payload hash is taken over the original request
body, so redaction is deterministic and a resubmission still reconciles as a
duplicate.
Both outcomes are counted durably by sender, source, action and field path,
exposed at GET /v1/secret-findings. Per-path aggregation is the point: the
actionable unit is "stop emitting data.auth.token on membership.added", not
"there were 47 redactions". Counters survive restart because the fix they drive
lives in another service.
Contract doc updated to match. Tests 46 -> 50. WP-0004 is finished.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-10 16:02:22 +02:00
|
|
|
from audit_core.redaction import POLICIES, POLICY_REDACT
|
|
|
|
|
|
Bind sender identities, add operator read surface, real serving layer
AUDIT-WP-0004 T03, T05, T06.
T03 - the receiver accepted whatever tenant and source a caller sent as long
as it held the one shared token, despite WP-0003 recording tenant isolation as
delivered. audit_core.senders binds each credential to the sources and tenants
it may assert, driven by AUDIT_CORE_SENDERS rather than literals. Identities
hold a list of tokens so rotation publishes the replacement alongside the
incumbent and needs no delivery gap. Read is a separate privilege from write,
so a sender credential cannot read the audit trail back.
T05 - lookup by event id, lookup by correlation id, and a dead-letter view.
Rejections are recorded rather than silently dropped. An event rejected for
carrying secret-shaped material has its payload withheld: storing it would
write that material into the audit store, which is what the rejection exists
to prevent. Reason and payload hash are kept so it stays traceable.
Replay is deliberately not built here. Idempotent replay is a property of the
durable store and building it against SQLite would produce a second
implementation to throw away; it lands with the Postgres backend in
AUDIT-WP-0005-T01.
T06 - serving moves to waitress with configurable threads and channel timeout,
installed in the image via the serve extra. Without it the entrypoint falls
back to a threaded wsgiref server with a socket timeout and graceful shutdown
on SIGTERM, and logs a warning so a deployment cannot quietly land on the
fallback. Metric counters deferred to WP-0005-T03 to be designed against the
real scrape path.
Tests 36 -> 46, covering cross-tenant and cross-source refusal, token
rotation, read/write privilege separation, correlation lookup, and payload
withholding on secret rejection.
Remaining in WP-0004: T04 redaction policy, which needs a decision on whether
a secret-shaped field is a rejection or a redaction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-10 14:50:02 +02:00
|
|
|
WILDCARD = "*"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@dataclass(frozen=True)
|
|
|
|
|
class SenderIdentity:
|
|
|
|
|
"""A credential holder and the claims it is allowed to make."""
|
|
|
|
|
|
|
|
|
|
name: str
|
|
|
|
|
tokens: tuple[str, ...]
|
|
|
|
|
sources: frozenset[str]
|
|
|
|
|
tenants: frozenset[str] = field(default_factory=lambda: frozenset({WILDCARD}))
|
|
|
|
|
may_write: bool = True
|
|
|
|
|
may_read: bool = False
|
Redact secret-shaped fields by default, countable per field path
AUDIT-WP-0004-T04, closing the workplan.
Decision (Bernd): default to redaction, allow rejection per sender. Losing an
audit record over one field is worse than storing it masked, but a
higher-assurance channel must be able to refuse rather than mask. secret_policy
is set per sender identity in AUDIT_CORE_SENDERS and defaults to redact.
Detection now covers the whole payload at any depth, including lists, rather
than only the top level of data. Under redaction the value is masked and the
key is preserved: dropping the key would hide that the sender transmitted the
field at all, which is exactly what an operator needs in order to stop it. The
stored record carries details.redaction with policy and affected paths, so a
reader never has to infer whether what they see is what was sent.
Idempotency is unaffected - the payload hash is taken over the original request
body, so redaction is deterministic and a resubmission still reconciles as a
duplicate.
Both outcomes are counted durably by sender, source, action and field path,
exposed at GET /v1/secret-findings. Per-path aggregation is the point: the
actionable unit is "stop emitting data.auth.token on membership.added", not
"there were 47 redactions". Counters survive restart because the fix they drive
lives in another service.
Contract doc updated to match. Tests 46 -> 50. WP-0004 is finished.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-10 16:02:22 +02:00
|
|
|
# Secret-shaped field handling. Defaults to redact-and-accept, so a
|
|
|
|
|
# legitimate event is not lost over one field; a higher-assurance channel
|
|
|
|
|
# can be set to reject instead (AUDIT-WP-0004-T04).
|
|
|
|
|
secret_policy: str = POLICY_REDACT
|
Bind sender identities, add operator read surface, real serving layer
AUDIT-WP-0004 T03, T05, T06.
T03 - the receiver accepted whatever tenant and source a caller sent as long
as it held the one shared token, despite WP-0003 recording tenant isolation as
delivered. audit_core.senders binds each credential to the sources and tenants
it may assert, driven by AUDIT_CORE_SENDERS rather than literals. Identities
hold a list of tokens so rotation publishes the replacement alongside the
incumbent and needs no delivery gap. Read is a separate privilege from write,
so a sender credential cannot read the audit trail back.
T05 - lookup by event id, lookup by correlation id, and a dead-letter view.
Rejections are recorded rather than silently dropped. An event rejected for
carrying secret-shaped material has its payload withheld: storing it would
write that material into the audit store, which is what the rejection exists
to prevent. Reason and payload hash are kept so it stays traceable.
Replay is deliberately not built here. Idempotent replay is a property of the
durable store and building it against SQLite would produce a second
implementation to throw away; it lands with the Postgres backend in
AUDIT-WP-0005-T01.
T06 - serving moves to waitress with configurable threads and channel timeout,
installed in the image via the serve extra. Without it the entrypoint falls
back to a threaded wsgiref server with a socket timeout and graceful shutdown
on SIGTERM, and logs a warning so a deployment cannot quietly land on the
fallback. Metric counters deferred to WP-0005-T03 to be designed against the
real scrape path.
Tests 36 -> 46, covering cross-tenant and cross-source refusal, token
rotation, read/write privilege separation, correlation lookup, and payload
withholding on secret rejection.
Remaining in WP-0004: T04 redaction policy, which needs a decision on whether
a secret-shaped field is a rejection or a redaction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-10 14:50:02 +02:00
|
|
|
|
|
|
|
|
def __post_init__(self) -> None:
|
|
|
|
|
if not self.name:
|
|
|
|
|
raise ValueError("sender identity needs a name")
|
Redact secret-shaped fields by default, countable per field path
AUDIT-WP-0004-T04, closing the workplan.
Decision (Bernd): default to redaction, allow rejection per sender. Losing an
audit record over one field is worse than storing it masked, but a
higher-assurance channel must be able to refuse rather than mask. secret_policy
is set per sender identity in AUDIT_CORE_SENDERS and defaults to redact.
Detection now covers the whole payload at any depth, including lists, rather
than only the top level of data. Under redaction the value is masked and the
key is preserved: dropping the key would hide that the sender transmitted the
field at all, which is exactly what an operator needs in order to stop it. The
stored record carries details.redaction with policy and affected paths, so a
reader never has to infer whether what they see is what was sent.
Idempotency is unaffected - the payload hash is taken over the original request
body, so redaction is deterministic and a resubmission still reconciles as a
duplicate.
Both outcomes are counted durably by sender, source, action and field path,
exposed at GET /v1/secret-findings. Per-path aggregation is the point: the
actionable unit is "stop emitting data.auth.token on membership.added", not
"there were 47 redactions". Counters survive restart because the fix they drive
lives in another service.
Contract doc updated to match. Tests 46 -> 50. WP-0004 is finished.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-10 16:02:22 +02:00
|
|
|
if self.secret_policy not in POLICIES:
|
|
|
|
|
raise ValueError(
|
|
|
|
|
f"sender {self.name!r}: secret_policy must be one of {POLICIES}, "
|
|
|
|
|
f"got {self.secret_policy!r}"
|
|
|
|
|
)
|
Bind sender identities, add operator read surface, real serving layer
AUDIT-WP-0004 T03, T05, T06.
T03 - the receiver accepted whatever tenant and source a caller sent as long
as it held the one shared token, despite WP-0003 recording tenant isolation as
delivered. audit_core.senders binds each credential to the sources and tenants
it may assert, driven by AUDIT_CORE_SENDERS rather than literals. Identities
hold a list of tokens so rotation publishes the replacement alongside the
incumbent and needs no delivery gap. Read is a separate privilege from write,
so a sender credential cannot read the audit trail back.
T05 - lookup by event id, lookup by correlation id, and a dead-letter view.
Rejections are recorded rather than silently dropped. An event rejected for
carrying secret-shaped material has its payload withheld: storing it would
write that material into the audit store, which is what the rejection exists
to prevent. Reason and payload hash are kept so it stays traceable.
Replay is deliberately not built here. Idempotent replay is a property of the
durable store and building it against SQLite would produce a second
implementation to throw away; it lands with the Postgres backend in
AUDIT-WP-0005-T01.
T06 - serving moves to waitress with configurable threads and channel timeout,
installed in the image via the serve extra. Without it the entrypoint falls
back to a threaded wsgiref server with a socket timeout and graceful shutdown
on SIGTERM, and logs a warning so a deployment cannot quietly land on the
fallback. Metric counters deferred to WP-0005-T03 to be designed against the
real scrape path.
Tests 36 -> 46, covering cross-tenant and cross-source refusal, token
rotation, read/write privilege separation, correlation lookup, and payload
withholding on secret rejection.
Remaining in WP-0004: T04 redaction policy, which needs a decision on whether
a secret-shaped field is a rejection or a redaction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-10 14:50:02 +02:00
|
|
|
if not self.tokens or any(not t for t in self.tokens):
|
|
|
|
|
raise ValueError(f"sender {self.name!r} needs at least one non-empty token")
|
|
|
|
|
if not self.sources:
|
|
|
|
|
raise ValueError(f"sender {self.name!r} needs at least one permitted source")
|
|
|
|
|
|
|
|
|
|
def permits_source(self, source: str) -> bool:
|
|
|
|
|
return WILDCARD in self.sources or source in self.sources
|
|
|
|
|
|
Scope the read path by tenant (AUDIT-WP-0008-T04).
_read gated on may_read alone and never called permits_tenant, so any reader
credential could read every tenant through /v1/events, /v1/events/<id>,
/v1/dead-letters and /v1/secret-findings. Deployment bounded the exposure --
the only production sender holds may_read: false -- but the boundary was not in
the code, which is the difference between E2 and E1 on the tenancy posture
enforcement ladder.
Two rules, because the surfaces divide cleanly. Event reads are filtered to the
tenants the credential may act for. Surfaces with no tenant key to filter on --
stats, integrity, dead letters, secret findings -- require full tenant scope and
are refused rather than served instance-wide facts to a scoped reader.
A cross-tenant fetch returns 404 rather than 403. A distinguishable forbidden
would confirm that an event id exists and which tenant holds it, turning the
read surface into an existence oracle. Correlation lookup is filtered rather
than refused, since a correlation id legitimately spans tenants.
_readable_by fails closed: a record with no tenant is readable only at full
scope. Three existing tests read instance-wide surfaces with a scoped
credential, which this makes a 403; bound_app now carries an unrestricted
operator identity and those reads use it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-17 22:05:52 +02:00
|
|
|
def has_full_tenant_scope(self) -> bool:
|
|
|
|
|
"""Whether this identity is unrestricted across tenants.
|
|
|
|
|
|
|
|
|
|
The read surfaces that are not tenant-keyed — chain verification,
|
|
|
|
|
counters, dead letters, secret findings — cannot be filtered, so they
|
|
|
|
|
are reserved for an identity that could read every tenant anyway.
|
|
|
|
|
Anything else would hand a scoped reader instance-wide facts.
|
|
|
|
|
"""
|
|
|
|
|
return WILDCARD in self.tenants
|
|
|
|
|
|
Bind sender identities, add operator read surface, real serving layer
AUDIT-WP-0004 T03, T05, T06.
T03 - the receiver accepted whatever tenant and source a caller sent as long
as it held the one shared token, despite WP-0003 recording tenant isolation as
delivered. audit_core.senders binds each credential to the sources and tenants
it may assert, driven by AUDIT_CORE_SENDERS rather than literals. Identities
hold a list of tokens so rotation publishes the replacement alongside the
incumbent and needs no delivery gap. Read is a separate privilege from write,
so a sender credential cannot read the audit trail back.
T05 - lookup by event id, lookup by correlation id, and a dead-letter view.
Rejections are recorded rather than silently dropped. An event rejected for
carrying secret-shaped material has its payload withheld: storing it would
write that material into the audit store, which is what the rejection exists
to prevent. Reason and payload hash are kept so it stays traceable.
Replay is deliberately not built here. Idempotent replay is a property of the
durable store and building it against SQLite would produce a second
implementation to throw away; it lands with the Postgres backend in
AUDIT-WP-0005-T01.
T06 - serving moves to waitress with configurable threads and channel timeout,
installed in the image via the serve extra. Without it the entrypoint falls
back to a threaded wsgiref server with a socket timeout and graceful shutdown
on SIGTERM, and logs a warning so a deployment cannot quietly land on the
fallback. Metric counters deferred to WP-0005-T03 to be designed against the
real scrape path.
Tests 36 -> 46, covering cross-tenant and cross-source refusal, token
rotation, read/write privilege separation, correlation lookup, and payload
withholding on secret rejection.
Remaining in WP-0004: T04 redaction policy, which needs a decision on whether
a secret-shaped field is a rejection or a redaction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-10 14:50:02 +02:00
|
|
|
def permits_tenant(self, tenant: str) -> bool:
|
|
|
|
|
"""Whether this identity may write for ``tenant``.
|
|
|
|
|
|
|
|
|
|
Tenant identifiers are opaque here. Their shape
|
|
|
|
|
(``tenant:<grouping>:<name>``) is owned by the IAM Profile and
|
|
|
|
|
tenant-engine; matching is exact, never parsed or pattern-derived.
|
|
|
|
|
"""
|
|
|
|
|
return WILDCARD in self.tenants or tenant in self.tenants
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class SenderRegistry:
|
|
|
|
|
"""Resolves a credential to the identity that holds it."""
|
|
|
|
|
|
|
|
|
|
def __init__(self, identities: Iterable[SenderIdentity]) -> None:
|
|
|
|
|
self.identities = tuple(identities)
|
|
|
|
|
if not self.identities:
|
|
|
|
|
raise ValueError("at least one sender identity is required")
|
|
|
|
|
|
|
|
|
|
def authenticate(self, authorization_header: str | None) -> SenderIdentity | None:
|
|
|
|
|
"""Return the matching identity, or None.
|
|
|
|
|
|
|
|
|
|
Every candidate token is compared even after a match is found, so the
|
|
|
|
|
work done does not depend on which credential was supplied or how many
|
|
|
|
|
identities precede it.
|
|
|
|
|
"""
|
|
|
|
|
supplied = str(authorization_header or "")
|
|
|
|
|
matched: SenderIdentity | None = None
|
|
|
|
|
for identity in self.identities:
|
|
|
|
|
for token in identity.tokens:
|
|
|
|
|
try:
|
|
|
|
|
hit = hmac.compare_digest(supplied, f"Bearer {token}")
|
|
|
|
|
except TypeError:
|
|
|
|
|
# compare_digest rejects non-ASCII str; such a header is
|
|
|
|
|
# simply not a valid credential.
|
|
|
|
|
hit = False
|
|
|
|
|
if hit and matched is None:
|
|
|
|
|
matched = identity
|
|
|
|
|
return matched
|
|
|
|
|
|
|
|
|
|
@classmethod
|
|
|
|
|
def from_env(cls, env: dict[str, str] | None = None) -> "SenderRegistry":
|
|
|
|
|
"""Build a registry from ``AUDIT_CORE_SENDERS`` (JSON).
|
|
|
|
|
|
|
|
|
|
Shape::
|
|
|
|
|
|
|
|
|
|
[{"name": "user-engine",
|
|
|
|
|
"tokens": ["current", "next"],
|
|
|
|
|
"sources": ["user-engine"],
|
|
|
|
|
"tenants": ["tenant:friendly:binky"],
|
|
|
|
|
"may_read": false}]
|
|
|
|
|
|
|
|
|
|
``tenants`` defaults to ``["*"]``. Omitting it is a deliberate choice to
|
|
|
|
|
accept any tenant from that sender and should be justified per sender,
|
|
|
|
|
not adopted by default.
|
|
|
|
|
"""
|
|
|
|
|
env = env if env is not None else dict(os.environ)
|
|
|
|
|
raw = env.get("AUDIT_CORE_SENDERS")
|
|
|
|
|
if raw:
|
2026-08-16 00:24:33 +02:00
|
|
|
return cls(_apply_scope_overlay(_parse_identities(raw), env))
|
Bind sender identities, add operator read surface, real serving layer
AUDIT-WP-0004 T03, T05, T06.
T03 - the receiver accepted whatever tenant and source a caller sent as long
as it held the one shared token, despite WP-0003 recording tenant isolation as
delivered. audit_core.senders binds each credential to the sources and tenants
it may assert, driven by AUDIT_CORE_SENDERS rather than literals. Identities
hold a list of tokens so rotation publishes the replacement alongside the
incumbent and needs no delivery gap. Read is a separate privilege from write,
so a sender credential cannot read the audit trail back.
T05 - lookup by event id, lookup by correlation id, and a dead-letter view.
Rejections are recorded rather than silently dropped. An event rejected for
carrying secret-shaped material has its payload withheld: storing it would
write that material into the audit store, which is what the rejection exists
to prevent. Reason and payload hash are kept so it stays traceable.
Replay is deliberately not built here. Idempotent replay is a property of the
durable store and building it against SQLite would produce a second
implementation to throw away; it lands with the Postgres backend in
AUDIT-WP-0005-T01.
T06 - serving moves to waitress with configurable threads and channel timeout,
installed in the image via the serve extra. Without it the entrypoint falls
back to a threaded wsgiref server with a socket timeout and graceful shutdown
on SIGTERM, and logs a warning so a deployment cannot quietly land on the
fallback. Metric counters deferred to WP-0005-T03 to be designed against the
real scrape path.
Tests 36 -> 46, covering cross-tenant and cross-source refusal, token
rotation, read/write privilege separation, correlation lookup, and payload
withholding on secret rejection.
Remaining in WP-0004: T04 redaction policy, which needs a decision on whether
a secret-shaped field is a rejection or a redaction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-10 14:50:02 +02:00
|
|
|
|
|
|
|
|
legacy = (env.get("AUDIT_CORE_INGEST_TOKEN") or "").strip()
|
|
|
|
|
if not legacy:
|
|
|
|
|
raise ValueError(
|
|
|
|
|
"no sender configuration: set AUDIT_CORE_SENDERS (preferred) "
|
|
|
|
|
"or AUDIT_CORE_INGEST_TOKEN"
|
|
|
|
|
)
|
|
|
|
|
# Legacy single-token form. Kept so an existing deployment keeps
|
|
|
|
|
# working, but it grants every tenant — which is what T03 exists to
|
|
|
|
|
# stop, so it is deliberately noisy about what it is.
|
|
|
|
|
return cls([
|
|
|
|
|
SenderIdentity(
|
|
|
|
|
name="legacy-ingest-token",
|
|
|
|
|
tokens=(legacy,),
|
|
|
|
|
sources=frozenset({"user-engine"}),
|
|
|
|
|
tenants=frozenset({WILDCARD}),
|
|
|
|
|
)
|
|
|
|
|
])
|
|
|
|
|
|
|
|
|
|
|
2026-08-16 00:24:33 +02:00
|
|
|
def _load_scope_overlay(env: dict[str, str]) -> list[dict[str, Any]]:
|
|
|
|
|
"""Non-secret sender policy. Tokens never come from here.
|
|
|
|
|
|
|
|
|
|
``AUDIT_CORE_SENDERS_SCOPE`` is inline JSON (tests).
|
|
|
|
|
``AUDIT_CORE_SENDERS_SCOPE_PATH`` is a file (production ConfigMap).
|
|
|
|
|
"""
|
|
|
|
|
inline = env.get("AUDIT_CORE_SENDERS_SCOPE")
|
|
|
|
|
if inline:
|
|
|
|
|
payload = json.loads(inline)
|
|
|
|
|
else:
|
|
|
|
|
path = env.get("AUDIT_CORE_SENDERS_SCOPE_PATH")
|
|
|
|
|
if not path:
|
|
|
|
|
return []
|
|
|
|
|
payload = json.loads(Path(path).read_text())
|
|
|
|
|
if not isinstance(payload, list):
|
|
|
|
|
raise ValueError("sender scope overlay must be a JSON list")
|
|
|
|
|
return [entry for entry in payload if isinstance(entry, dict) and entry.get("name")]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _apply_scope_overlay(
|
|
|
|
|
identities: list[SenderIdentity], env: dict[str, str]
|
|
|
|
|
) -> list[SenderIdentity]:
|
|
|
|
|
"""Overlay non-secret fields from Git/ConfigMap onto Secret-backed tokens.
|
|
|
|
|
|
|
|
|
|
ExternalSecret refresh cannot shrink ``user-engine`` tenants below the
|
|
|
|
|
declared scope (AUDIT-WP-0006-T05). Tokens are never taken from the overlay.
|
|
|
|
|
"""
|
|
|
|
|
overlay = {entry["name"]: entry for entry in _load_scope_overlay(env)}
|
|
|
|
|
if not overlay:
|
|
|
|
|
return identities
|
|
|
|
|
merged: list[SenderIdentity] = []
|
|
|
|
|
for identity in identities:
|
|
|
|
|
extra = overlay.get(identity.name)
|
|
|
|
|
if extra is None:
|
|
|
|
|
merged.append(identity)
|
|
|
|
|
continue
|
|
|
|
|
tenants = extra.get("tenants")
|
|
|
|
|
sources = extra.get("sources")
|
|
|
|
|
merged.append(
|
|
|
|
|
SenderIdentity(
|
|
|
|
|
name=identity.name,
|
|
|
|
|
tokens=identity.tokens,
|
|
|
|
|
sources=(
|
|
|
|
|
frozenset(str(s) for s in sources) if sources else identity.sources
|
|
|
|
|
),
|
|
|
|
|
tenants=(
|
|
|
|
|
frozenset(str(t) for t in tenants) if tenants else identity.tenants
|
|
|
|
|
),
|
|
|
|
|
may_write=(
|
|
|
|
|
bool(extra["may_write"]) if "may_write" in extra else identity.may_write
|
|
|
|
|
),
|
|
|
|
|
may_read=(
|
|
|
|
|
bool(extra["may_read"]) if "may_read" in extra else identity.may_read
|
|
|
|
|
),
|
|
|
|
|
secret_policy=identity.secret_policy,
|
|
|
|
|
)
|
|
|
|
|
)
|
|
|
|
|
return merged
|
|
|
|
|
|
|
|
|
|
|
Bind sender identities, add operator read surface, real serving layer
AUDIT-WP-0004 T03, T05, T06.
T03 - the receiver accepted whatever tenant and source a caller sent as long
as it held the one shared token, despite WP-0003 recording tenant isolation as
delivered. audit_core.senders binds each credential to the sources and tenants
it may assert, driven by AUDIT_CORE_SENDERS rather than literals. Identities
hold a list of tokens so rotation publishes the replacement alongside the
incumbent and needs no delivery gap. Read is a separate privilege from write,
so a sender credential cannot read the audit trail back.
T05 - lookup by event id, lookup by correlation id, and a dead-letter view.
Rejections are recorded rather than silently dropped. An event rejected for
carrying secret-shaped material has its payload withheld: storing it would
write that material into the audit store, which is what the rejection exists
to prevent. Reason and payload hash are kept so it stays traceable.
Replay is deliberately not built here. Idempotent replay is a property of the
durable store and building it against SQLite would produce a second
implementation to throw away; it lands with the Postgres backend in
AUDIT-WP-0005-T01.
T06 - serving moves to waitress with configurable threads and channel timeout,
installed in the image via the serve extra. Without it the entrypoint falls
back to a threaded wsgiref server with a socket timeout and graceful shutdown
on SIGTERM, and logs a warning so a deployment cannot quietly land on the
fallback. Metric counters deferred to WP-0005-T03 to be designed against the
real scrape path.
Tests 36 -> 46, covering cross-tenant and cross-source refusal, token
rotation, read/write privilege separation, correlation lookup, and payload
withholding on secret rejection.
Remaining in WP-0004: T04 redaction policy, which needs a decision on whether
a secret-shaped field is a rejection or a redaction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-10 14:50:02 +02:00
|
|
|
def _parse_identities(raw: str) -> list[SenderIdentity]:
|
|
|
|
|
try:
|
|
|
|
|
entries = json.loads(raw)
|
|
|
|
|
except json.JSONDecodeError as exc:
|
|
|
|
|
raise ValueError(f"AUDIT_CORE_SENDERS is not valid JSON: {exc}") from exc
|
|
|
|
|
if not isinstance(entries, list) or not entries:
|
|
|
|
|
raise ValueError("AUDIT_CORE_SENDERS must be a non-empty JSON list")
|
|
|
|
|
return [_identity_from(entry) for entry in entries]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _identity_from(entry: Any) -> SenderIdentity:
|
|
|
|
|
if not isinstance(entry, dict):
|
|
|
|
|
raise ValueError("each sender entry must be an object")
|
|
|
|
|
tokens = entry.get("tokens") or ([entry["token"]] if entry.get("token") else [])
|
|
|
|
|
return SenderIdentity(
|
|
|
|
|
name=str(entry.get("name") or ""),
|
|
|
|
|
tokens=tuple(str(t) for t in tokens),
|
|
|
|
|
sources=frozenset(str(s) for s in (entry.get("sources") or [])),
|
|
|
|
|
tenants=frozenset(str(t) for t in (entry.get("tenants") or [WILDCARD])),
|
|
|
|
|
may_write=bool(entry.get("may_write", True)),
|
|
|
|
|
may_read=bool(entry.get("may_read", False)),
|
Redact secret-shaped fields by default, countable per field path
AUDIT-WP-0004-T04, closing the workplan.
Decision (Bernd): default to redaction, allow rejection per sender. Losing an
audit record over one field is worse than storing it masked, but a
higher-assurance channel must be able to refuse rather than mask. secret_policy
is set per sender identity in AUDIT_CORE_SENDERS and defaults to redact.
Detection now covers the whole payload at any depth, including lists, rather
than only the top level of data. Under redaction the value is masked and the
key is preserved: dropping the key would hide that the sender transmitted the
field at all, which is exactly what an operator needs in order to stop it. The
stored record carries details.redaction with policy and affected paths, so a
reader never has to infer whether what they see is what was sent.
Idempotency is unaffected - the payload hash is taken over the original request
body, so redaction is deterministic and a resubmission still reconciles as a
duplicate.
Both outcomes are counted durably by sender, source, action and field path,
exposed at GET /v1/secret-findings. Per-path aggregation is the point: the
actionable unit is "stop emitting data.auth.token on membership.added", not
"there were 47 redactions". Counters survive restart because the fix they drive
lives in another service.
Contract doc updated to match. Tests 46 -> 50. WP-0004 is finished.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-10 16:02:22 +02:00
|
|
|
secret_policy=str(entry.get("secret_policy", POLICY_REDACT)),
|
Bind sender identities, add operator read surface, real serving layer
AUDIT-WP-0004 T03, T05, T06.
T03 - the receiver accepted whatever tenant and source a caller sent as long
as it held the one shared token, despite WP-0003 recording tenant isolation as
delivered. audit_core.senders binds each credential to the sources and tenants
it may assert, driven by AUDIT_CORE_SENDERS rather than literals. Identities
hold a list of tokens so rotation publishes the replacement alongside the
incumbent and needs no delivery gap. Read is a separate privilege from write,
so a sender credential cannot read the audit trail back.
T05 - lookup by event id, lookup by correlation id, and a dead-letter view.
Rejections are recorded rather than silently dropped. An event rejected for
carrying secret-shaped material has its payload withheld: storing it would
write that material into the audit store, which is what the rejection exists
to prevent. Reason and payload hash are kept so it stays traceable.
Replay is deliberately not built here. Idempotent replay is a property of the
durable store and building it against SQLite would produce a second
implementation to throw away; it lands with the Postgres backend in
AUDIT-WP-0005-T01.
T06 - serving moves to waitress with configurable threads and channel timeout,
installed in the image via the serve extra. Without it the entrypoint falls
back to a threaded wsgiref server with a socket timeout and graceful shutdown
on SIGTERM, and logs a warning so a deployment cannot quietly land on the
fallback. Metric counters deferred to WP-0005-T03 to be designed against the
real scrape path.
Tests 36 -> 46, covering cross-tenant and cross-source refusal, token
rotation, read/write privilege separation, correlation lookup, and payload
withholding on secret rejection.
Remaining in WP-0004: T04 redaction policy, which needs a decision on whether
a secret-shaped field is a rejection or a redaction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-10 14:50:02 +02:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def development_registry(token: str) -> SenderRegistry:
|
|
|
|
|
"""A single permissive sender, for tests and local development.
|
|
|
|
|
|
|
|
|
|
Accepts any tenant. Not a production configuration — production binds each
|
|
|
|
|
sender to the tenants it may write for via ``AUDIT_CORE_SENDERS``.
|
|
|
|
|
"""
|
|
|
|
|
return SenderRegistry([
|
|
|
|
|
SenderIdentity(
|
|
|
|
|
name="development",
|
|
|
|
|
tokens=(token,),
|
|
|
|
|
sources=frozenset({"user-engine"}),
|
|
|
|
|
tenants=frozenset({WILDCARD}),
|
|
|
|
|
may_read=True,
|
|
|
|
|
)
|
|
|
|
|
])
|