Implement AUDIT-WP-0006 honest operational custody.
Postgres now reports custody_class=operational with a cited 30-day recoverable window. Join ITC-CAP operations.audit at D4, publish the interface card, and overlay user-engine tenants [*] from Git so an ExternalSecret refresh cannot shrink it.
This commit is contained in:
parent
0a3d05ff1c
commit
ded432a63f
25 changed files with 832 additions and 94 deletions
1
Makefile
1
Makefile
|
|
@ -45,6 +45,7 @@ image-publish: image-build ## Push the commit-tagged image to Forgejo
|
|||
deploy-dry-run: ## Server-side validate the railiance01 manifests
|
||||
KUBECONFIG=$(KUBECONFIG_RAILIANCE) kubectl apply --dry-run=server --validate=strict \
|
||||
-f deploy/audit-core.yaml \
|
||||
-f deploy/senders-scope.yaml \
|
||||
-f deploy/networkpolicies.yaml \
|
||||
-f deploy/clustersecretstore.yaml \
|
||||
-f deploy/externalsecrets.yaml \
|
||||
|
|
|
|||
17
SCOPE.md
17
SCOPE.md
|
|
@ -15,21 +15,28 @@ audit-core exists to provide the capability described in INTENT.md.
|
|||
|
||||
- Maintain the repository's primary implementation.
|
||||
- Keep docs, tests, and operational metadata current.
|
||||
- Operational audit custody (`operations.audit`) and its declared recovery bound.
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- Own unrelated adjacent systems.
|
||||
- Make irreversible operational decisions without human approval.
|
||||
- Procuring or operating S3 / Barman / WAL.
|
||||
- Booked cost or a second usage stream for `platform:audit-storage`.
|
||||
- A `rapp.yaml` in this repo (schema requires `rapp-*`).
|
||||
- Public ingest.
|
||||
|
||||
## Current State
|
||||
|
||||
- Status: active
|
||||
- Status: production
|
||||
- Production receiver on railiance01 (`namespace audit-core`), Postgres
|
||||
append-only store on `platform-pg`, sender `user-engine`.
|
||||
operational custody on `platform-pg`, sender `user-engine`.
|
||||
- Recovery is the platform `data.backup` window (30 days, RESOURCE-WP-0002
|
||||
live), not an audit-core deletion window. Production still reports
|
||||
`custody_class=archive`; AUDIT-WP-0006 is the honesty/canon-join pass.
|
||||
- Open workplan: `workplans/AUDIT-WP-0006-honest-custody-and-canon-join.md`.
|
||||
live). Manifest `/readyz` reports `custody_class=operational` and
|
||||
`recoverable_days=30` once the AUDIT-WP-0006 image is pinned.
|
||||
- ITC-CAP case: `data/capability/audit-core-operational.json`.
|
||||
`data.archive` is an unmet requirement.
|
||||
- AUDIT-WP-0001…0006 closed. No open workplan.
|
||||
|
||||
## Getting Oriented
|
||||
|
||||
|
|
|
|||
|
|
@ -13,6 +13,7 @@ from audit_core.interface import (
|
|||
IdempotentAuditBackend,
|
||||
RetentionPolicy,
|
||||
SCHEMA_VERSION_V1ALPHA1,
|
||||
custody_class_satisfies,
|
||||
validate_event,
|
||||
)
|
||||
from audit_core.mock_file_backend import MockFileAuditBackend
|
||||
|
|
@ -30,5 +31,6 @@ __all__ = [
|
|||
"RetentionPolicy",
|
||||
"SCHEMA_VERSION_V1ALPHA1",
|
||||
"SQLiteAuditBackend",
|
||||
"custody_class_satisfies",
|
||||
"validate_event",
|
||||
]
|
||||
|
|
|
|||
|
|
@ -38,6 +38,7 @@ from audit_core.interface import (
|
|||
EventConflictError,
|
||||
EventValidationError,
|
||||
IdempotentAuditBackend,
|
||||
custody_class_satisfies,
|
||||
)
|
||||
from audit_core.redaction import (
|
||||
POLICY_REDACT,
|
||||
|
|
@ -102,10 +103,13 @@ class IngestionApplication:
|
|||
require_custody_class: str | None = None,
|
||||
) -> None:
|
||||
policy = backend.retention_policy
|
||||
if require_custody_class and policy.custody_class != require_custody_class:
|
||||
if require_custody_class and not custody_class_satisfies(
|
||||
policy.custody_class, require_custody_class
|
||||
):
|
||||
# Production sets this. Without it, losing AUDIT_CORE_DATABASE_URL
|
||||
# silently downgrades custody to the development store instead of
|
||||
# failing to start.
|
||||
# failing to start. ``operational`` and ``archive`` alias each
|
||||
# other for one mixed-rollout deploy (AUDIT-WP-0006-T01).
|
||||
raise ValueError(
|
||||
f"backend custody_class={policy.custody_class!r} does not meet the "
|
||||
f"required {require_custody_class!r}; refusing to start"
|
||||
|
|
@ -330,11 +334,10 @@ class IngestionApplication:
|
|||
return self._json(
|
||||
start_response, HTTPStatus.SERVICE_UNAVAILABLE, {"status": "unavailable"}
|
||||
)
|
||||
policy = self.backend.retention_policy
|
||||
return self._json(
|
||||
start_response,
|
||||
HTTPStatus.OK,
|
||||
{"status": "ok", "custody_class": policy.custody_class, "durable": policy.durable},
|
||||
self.backend.retention_policy.as_readiness(),
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
|
|
@ -490,11 +493,15 @@ def build_backend() -> IdempotentAuditBackend:
|
|||
else "AUDIT_CORE_DATABASE_URL" if url
|
||||
else "brokered libpq environment")
|
||||
log.info("custody backend: postgresql (%s)", source)
|
||||
recoverable = os.environ.get("AUDIT_CORE_RECOVERABLE_DAYS")
|
||||
return PostgresAuditBackend(
|
||||
url or "",
|
||||
credential_dir=credential_dir,
|
||||
schema=os.environ.get("AUDIT_CORE_DATABASE_SCHEMA", "audit_core"),
|
||||
retention_days=int(retention) if retention else None,
|
||||
recoverable_days=(
|
||||
int(recoverable) if recoverable else 30
|
||||
),
|
||||
max_size=int(os.environ.get("AUDIT_CORE_DB_POOL_MAX", "8")),
|
||||
statement_timeout_ms=int(
|
||||
os.environ.get("AUDIT_CORE_DB_STATEMENT_TIMEOUT_MS", "30000")
|
||||
|
|
|
|||
|
|
@ -13,7 +13,23 @@ from uuid import uuid4
|
|||
|
||||
SCHEMA_VERSION_V1ALPHA1 = "audit-core.event.v1alpha1"
|
||||
|
||||
CustodyClass = Literal["development", "archive", "hot_search"]
|
||||
CustodyClass = Literal["development", "operational", "archive", "hot_search"]
|
||||
|
||||
# Production Postgres reports ``operational``. Manifests written before
|
||||
# AUDIT-WP-0006 required ``archive``. The two are aliases for one deploy so
|
||||
# a mixed rollout cannot refuse to start. ``development`` is never an alias.
|
||||
_PRODUCTION_CUSTODY_CLASSES = frozenset({"operational", "archive"})
|
||||
|
||||
|
||||
def custody_class_satisfies(actual: str, required: str) -> bool:
|
||||
"""Whether a backend's class meets a startup requirement.
|
||||
|
||||
``operational`` and ``archive`` satisfy each other. A development
|
||||
backend satisfies only ``development``.
|
||||
"""
|
||||
if actual == required:
|
||||
return True
|
||||
return {actual, required} <= _PRODUCTION_CUSTODY_CLASSES
|
||||
|
||||
_REQUIRED_STRING_FIELDS = (
|
||||
"schema_version",
|
||||
|
|
@ -41,6 +57,24 @@ class RetentionPolicy:
|
|||
immutable: bool
|
||||
tamper_evidence: bool
|
||||
durable: bool
|
||||
# Recoverable history is the platform backup window, not a deletion
|
||||
# policy. ``None`` means not declared (development backends).
|
||||
recoverable_days: int | None = None
|
||||
recoverable_source: str | None = None
|
||||
recoverable_basis: str | None = None
|
||||
|
||||
def as_readiness(self) -> dict[str, Any]:
|
||||
"""Sender-visible /readyz body. Keeps ``custody_class`` and adds recovery."""
|
||||
payload: dict[str, Any] = {
|
||||
"status": "ok",
|
||||
"custody_class": self.custody_class,
|
||||
"durable": self.durable,
|
||||
}
|
||||
if self.recoverable_days is not None or self.recoverable_source:
|
||||
payload["recoverable_days"] = self.recoverable_days
|
||||
payload["recoverable_source"] = self.recoverable_source
|
||||
payload["recoverable_basis"] = self.recoverable_basis
|
||||
return payload
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
|
|
@ -142,7 +176,7 @@ def validate_event(event: AuditEvent) -> None:
|
|||
class AuditBackend(Protocol):
|
||||
"""Protocol implemented by replaceable audit sinks.
|
||||
|
||||
Production backends provide durable archive or hot-search custody.
|
||||
Production backends provide durable operational or archive custody.
|
||||
Development backends (such as :class:`~audit_core.mock_file_backend.MockFileAuditBackend`)
|
||||
are for wiring only and must not be treated as audit custody.
|
||||
"""
|
||||
|
|
|
|||
|
|
@ -151,6 +151,12 @@ class PostgresAuditBackend:
|
|||
*,
|
||||
schema: str = DEFAULT_SCHEMA,
|
||||
retention_days: int | None = None,
|
||||
recoverable_days: int | None = 30,
|
||||
recoverable_source: str | None = (
|
||||
"resource-control/data/capability/platform-audit-storage.json"
|
||||
"#provisions[capability=data.backup]"
|
||||
),
|
||||
recoverable_basis: str | None = "measured",
|
||||
min_size: int = 1,
|
||||
max_size: int = 8,
|
||||
statement_timeout_ms: int = 30_000,
|
||||
|
|
@ -178,6 +184,9 @@ class PostgresAuditBackend:
|
|||
raise ValueError(f"unsafe schema name: {schema!r}")
|
||||
self.schema = schema
|
||||
self.retention_days = retention_days
|
||||
self.recoverable_days = recoverable_days
|
||||
self.recoverable_source = recoverable_source
|
||||
self.recoverable_basis = recoverable_basis
|
||||
base_kwargs = {
|
||||
"autocommit": True,
|
||||
# A stalled write must surface as unavailable rather than hold a
|
||||
|
|
@ -266,13 +275,22 @@ class PostgresAuditBackend:
|
|||
who can drop the trigger; ``tamper_evidence`` is correspondingly False,
|
||||
because nothing here would *prove* they had. Hash-chaining or external
|
||||
anchoring would be needed for that, and is not implemented.
|
||||
|
||||
``custody_class`` is ``operational``, not ``archive``. This store is
|
||||
durable append-only Postgres recovered through the platform
|
||||
``data.backup`` provision. It is not ITC-CAP ``data.archive`` (WORM
|
||||
object storage, manifests, retrieval tests). Recoverable history is
|
||||
the cited platform window, not ``retention_days``.
|
||||
"""
|
||||
return RetentionPolicy(
|
||||
custody_class="archive",
|
||||
custody_class="operational",
|
||||
retention_days=self.retention_days,
|
||||
immutable=True,
|
||||
tamper_evidence=False,
|
||||
durable=True,
|
||||
recoverable_days=self.recoverable_days,
|
||||
recoverable_source=self.recoverable_source,
|
||||
recoverable_basis=self.recoverable_basis,
|
||||
)
|
||||
|
||||
def emit(self, event: AuditEvent) -> str:
|
||||
|
|
|
|||
|
|
@ -17,6 +17,7 @@ import hmac
|
|||
import json
|
||||
import os
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any, Iterable
|
||||
|
||||
from audit_core.redaction import POLICIES, POLICY_REDACT
|
||||
|
|
@ -113,7 +114,7 @@ class SenderRegistry:
|
|||
env = env if env is not None else dict(os.environ)
|
||||
raw = env.get("AUDIT_CORE_SENDERS")
|
||||
if raw:
|
||||
return cls(_parse_identities(raw))
|
||||
return cls(_apply_scope_overlay(_parse_identities(raw), env))
|
||||
|
||||
legacy = (env.get("AUDIT_CORE_INGEST_TOKEN") or "").strip()
|
||||
if not legacy:
|
||||
|
|
@ -134,6 +135,66 @@ class SenderRegistry:
|
|||
])
|
||||
|
||||
|
||||
def _load_scope_overlay(env: dict[str, str]) -> list[dict[str, Any]]:
|
||||
"""Non-secret sender policy. Tokens never come from here.
|
||||
|
||||
``AUDIT_CORE_SENDERS_SCOPE`` is inline JSON (tests).
|
||||
``AUDIT_CORE_SENDERS_SCOPE_PATH`` is a file (production ConfigMap).
|
||||
"""
|
||||
inline = env.get("AUDIT_CORE_SENDERS_SCOPE")
|
||||
if inline:
|
||||
payload = json.loads(inline)
|
||||
else:
|
||||
path = env.get("AUDIT_CORE_SENDERS_SCOPE_PATH")
|
||||
if not path:
|
||||
return []
|
||||
payload = json.loads(Path(path).read_text())
|
||||
if not isinstance(payload, list):
|
||||
raise ValueError("sender scope overlay must be a JSON list")
|
||||
return [entry for entry in payload if isinstance(entry, dict) and entry.get("name")]
|
||||
|
||||
|
||||
def _apply_scope_overlay(
|
||||
identities: list[SenderIdentity], env: dict[str, str]
|
||||
) -> list[SenderIdentity]:
|
||||
"""Overlay non-secret fields from Git/ConfigMap onto Secret-backed tokens.
|
||||
|
||||
ExternalSecret refresh cannot shrink ``user-engine`` tenants below the
|
||||
declared scope (AUDIT-WP-0006-T05). Tokens are never taken from the overlay.
|
||||
"""
|
||||
overlay = {entry["name"]: entry for entry in _load_scope_overlay(env)}
|
||||
if not overlay:
|
||||
return identities
|
||||
merged: list[SenderIdentity] = []
|
||||
for identity in identities:
|
||||
extra = overlay.get(identity.name)
|
||||
if extra is None:
|
||||
merged.append(identity)
|
||||
continue
|
||||
tenants = extra.get("tenants")
|
||||
sources = extra.get("sources")
|
||||
merged.append(
|
||||
SenderIdentity(
|
||||
name=identity.name,
|
||||
tokens=identity.tokens,
|
||||
sources=(
|
||||
frozenset(str(s) for s in sources) if sources else identity.sources
|
||||
),
|
||||
tenants=(
|
||||
frozenset(str(t) for t in tenants) if tenants else identity.tenants
|
||||
),
|
||||
may_write=(
|
||||
bool(extra["may_write"]) if "may_write" in extra else identity.may_write
|
||||
),
|
||||
may_read=(
|
||||
bool(extra["may_read"]) if "may_read" in extra else identity.may_read
|
||||
),
|
||||
secret_policy=identity.secret_policy,
|
||||
)
|
||||
)
|
||||
return merged
|
||||
|
||||
|
||||
def _parse_identities(raw: str) -> list[SenderIdentity]:
|
||||
try:
|
||||
entries = json.loads(raw)
|
||||
|
|
|
|||
124
data/capability/audit-core-operational.json
Normal file
124
data/capability/audit-core-operational.json
Normal file
|
|
@ -0,0 +1,124 @@
|
|||
{
|
||||
"schema_version": "0.1",
|
||||
"record_scope": "operational",
|
||||
"canon": {
|
||||
"model": "ITC-CAP",
|
||||
"model_version": "0.4.0",
|
||||
"canon_version": "0.6.0",
|
||||
"status": "draft",
|
||||
"catalog": "info-tech-canon/infospace/models/capability/capabilities.yaml",
|
||||
"evidence_basis_catalog": "info-tech-canon/infospace/models/governance/evidence-basis.yaml"
|
||||
},
|
||||
"record_id": "capability-case:audit-core-operational:2026-08",
|
||||
"created_at": "2026-08-16T00:00:00Z",
|
||||
"subject": "Live audit-core receiver on railiance01, restated in canon terms",
|
||||
"note": "Joins reuse-surface id capability.audit.event-retain to ITC-CAP operations.audit. Maturity attaches to this provision, not the abstract capability. data.backup is cited, not restated.",
|
||||
"reuse_surface": {
|
||||
"id": "capability.audit.event-retain",
|
||||
"path": "registry/capabilities/capability.audit.event-retain.md"
|
||||
},
|
||||
"requires": [
|
||||
{
|
||||
"consumer": "audit-core.railiance01",
|
||||
"capability": "operations.audit",
|
||||
"profile": "administrative",
|
||||
"minimum_maturity": "D4",
|
||||
"requirement_note": "Production dependency for user-engine outbox delivery. D4 is the current ask; D5 would need measured integrity verification and actively controlled reliability."
|
||||
},
|
||||
{
|
||||
"consumer": "audit-core.railiance01",
|
||||
"capability": "data.archive",
|
||||
"profile": "operational",
|
||||
"requirement_note": "INTENT still wants unbounded WORM archive beyond the 30-day platform backup window. Unmet. Owner: audit-core to write a demand; resource-control to procure a different bucket/lifecycle than Barman. No provision is invented here."
|
||||
}
|
||||
],
|
||||
"provisions": [
|
||||
{
|
||||
"provider": "audit-core.railiance01",
|
||||
"capability": "operations.audit",
|
||||
"profile": "administrative",
|
||||
"environment": "production",
|
||||
"maturity": "D4",
|
||||
"implements": "HTTP POST /v1/events into append-only PostgreSQL on platform-pg, namespace audit-core, ClusterIP + default-deny",
|
||||
"maturity_rationale": "Approved for production dependency since AUDIT-WP-0005. Not D5: tamper_evidence is false (trigger is not a proof), one replica, reliability is not actively controlled.",
|
||||
"uses_provisions": [
|
||||
{
|
||||
"capability": "data.transactional",
|
||||
"provider": "rapp-postgres/platform-pg database audit_core",
|
||||
"relation": "may_use",
|
||||
"note": "operations.audit does not declare depends_on data.transactional in the catalog. The live store is Postgres; this names which provision satisfies it."
|
||||
},
|
||||
{
|
||||
"capability": "data.backup",
|
||||
"provider": "rapp-postgres/platform-pg CNPG barmanObjectStore",
|
||||
"relation": "may_use",
|
||||
"note": "Cite resource-control/data/capability/platform-audit-storage.json. Do not restate that case. Recoverable window 30 days, provision D4 against a D5 requirement.",
|
||||
"evidence_basis": "measured",
|
||||
"observed_at": "2026-08-14"
|
||||
},
|
||||
{
|
||||
"capability": "security.secrets",
|
||||
"provider": "OpenBao / external-secrets on reef-railiance",
|
||||
"relation": "may_use",
|
||||
"note": "ClusterSecretStore openbao-audit-core and openbao-audit-core-database. Never class P."
|
||||
}
|
||||
],
|
||||
"consumes": [
|
||||
{
|
||||
"class": "S",
|
||||
"name": "retained events",
|
||||
"quantity": {
|
||||
"value": null,
|
||||
"unit": "GB"
|
||||
},
|
||||
"period": "month",
|
||||
"basis": "unknown",
|
||||
"gap": "pg_total_relation_size of audit_core has not been recorded against this provision (owner: audit-core)"
|
||||
},
|
||||
{
|
||||
"class": "H",
|
||||
"name": "receiver operation",
|
||||
"quantity": {
|
||||
"value": null,
|
||||
"unit": "hour"
|
||||
},
|
||||
"period": "month",
|
||||
"supply": "internal",
|
||||
"basis": "unknown",
|
||||
"gap": "operator hours are not recorded (owner: audit-core; start a time record later)"
|
||||
},
|
||||
{
|
||||
"class": "I",
|
||||
"name": "intelligence",
|
||||
"quantity": {
|
||||
"value": null,
|
||||
"unit": "token"
|
||||
},
|
||||
"period": "month",
|
||||
"basis": "unknown",
|
||||
"gap": "audit-core does not meter token consumption against this provision (owner: audit-core)"
|
||||
}
|
||||
],
|
||||
"evidence": [
|
||||
{
|
||||
"hook": "audit_records",
|
||||
"basis": "measured",
|
||||
"value": "in-pod remote failure matrix 12 pass / 0 fail / 3 skip; live accept 202 before and after lease rotation",
|
||||
"observed_at": "2026-08-13",
|
||||
"ref": "evidence/failure-matrix-20260813T103908Z.json"
|
||||
},
|
||||
{
|
||||
"hook": "integrity_verification",
|
||||
"basis": "unknown",
|
||||
"gap": "events_append_only rejects UPDATE/DELETE for the runtime role; that is not a proof a database owner did not drop the trigger. Hash-chain or external anchor is not implemented (owner: audit-core). Do not borrow the restore drill for this hook."
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"open_items": [
|
||||
"data.archive is required by INTENT and unprovided. A founder decision is needed before resource-control procures a WORM/object-lock destination distinct from the 30-day Barman bucket.",
|
||||
"integrity_verification is unknown. Trigger enforcement is not tamper evidence.",
|
||||
"Class S/H/I consumption is unknown on this provision.",
|
||||
"Do not emit booked cost or a second usage stream for platform:audit-storage."
|
||||
]
|
||||
}
|
||||
|
|
@ -10,7 +10,8 @@ Deployment until:
|
|||
|
||||
1. The image digest is pinned (currently `sha256:41493cd5…` from commit `3a7d63e`).
|
||||
2. Secrets `audit-core-database`, `audit-core-database-migrate`, and
|
||||
`audit-core-senders` exist.
|
||||
`audit-core-senders` exist. ConfigMap `audit-core-senders-scope` is
|
||||
applied (`deploy/senders-scope.yaml`) before the Deployment mounts it.
|
||||
3. Job `audit-core-migrate` has completed.
|
||||
|
||||
```bash
|
||||
|
|
|
|||
|
|
@ -90,11 +90,11 @@ spec:
|
|||
value: "0.0.0.0"
|
||||
- name: AUDIT_CORE_HTTP_PORT
|
||||
value: "8080"
|
||||
# Refuses to start on anything but archive-class custody, so a
|
||||
# missing database URL fails loudly instead of silently downgrading
|
||||
# to the development store.
|
||||
# Refuses to start on anything but operational (durable Postgres)
|
||||
# custody. ``archive`` remains an accepted alias for one mixed
|
||||
# rollout so an old manifest cannot refuse a new image.
|
||||
- name: AUDIT_CORE_REQUIRE_CUSTODY_CLASS
|
||||
value: archive
|
||||
value: operational
|
||||
# Runtime role cannot CREATE TABLE. Schema changes are a Job
|
||||
# with the migration lease (deploy/migrate-job.yaml).
|
||||
- name: AUDIT_CORE_AUTO_MIGRATE
|
||||
|
|
@ -122,6 +122,11 @@ spec:
|
|||
secretKeyRef:
|
||||
name: audit-core-senders
|
||||
key: senders.json
|
||||
# Non-secret tenant/source scope. Tokens stay in the Secret;
|
||||
# ExternalSecret refresh cannot shrink user-engine tenants
|
||||
# below deploy/senders-scope.json.
|
||||
- name: AUDIT_CORE_SENDERS_SCOPE_PATH
|
||||
value: /etc/audit-core/senders-scope.json
|
||||
resources:
|
||||
requests:
|
||||
cpu: 50m
|
||||
|
|
@ -144,6 +149,10 @@ spec:
|
|||
- name: database-credential
|
||||
mountPath: /etc/audit-core/db
|
||||
readOnly: true
|
||||
- name: senders-scope
|
||||
mountPath: /etc/audit-core/senders-scope.json
|
||||
subPath: senders-scope.json
|
||||
readOnly: true
|
||||
startupProbe:
|
||||
httpGet: {path: /healthz, port: http}
|
||||
periodSeconds: 3
|
||||
|
|
@ -175,3 +184,7 @@ spec:
|
|||
secretName: audit-core-database
|
||||
# 0440 + fsGroup 10001: 0400 is root-only and the process cannot read it.
|
||||
defaultMode: 0440
|
||||
- name: senders-scope
|
||||
configMap:
|
||||
name: audit-core-senders-scope
|
||||
defaultMode: 0444
|
||||
|
|
|
|||
9
deploy/senders-scope.json
Normal file
9
deploy/senders-scope.json
Normal file
|
|
@ -0,0 +1,9 @@
|
|||
[
|
||||
{
|
||||
"name": "user-engine",
|
||||
"sources": ["user-engine"],
|
||||
"tenants": ["*"],
|
||||
"may_write": true,
|
||||
"may_read": false
|
||||
}
|
||||
]
|
||||
23
deploy/senders-scope.yaml
Normal file
23
deploy/senders-scope.yaml
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
# Non-secret sender scope (AUDIT-WP-0006-T05). Tokens stay in Secret
|
||||
# audit-core-senders. This ConfigMap is the authority for tenants/sources
|
||||
# so an ExternalSecret refresh cannot revert user-engine to a single tenant.
|
||||
# Keep in lockstep with deploy/senders-scope.json.
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: audit-core-senders-scope
|
||||
namespace: audit-core
|
||||
labels:
|
||||
app.kubernetes.io/name: audit-core
|
||||
data:
|
||||
senders-scope.json: |
|
||||
[
|
||||
{
|
||||
"name": "user-engine",
|
||||
"sources": ["user-engine"],
|
||||
"tenants": ["*"],
|
||||
"may_write": true,
|
||||
"may_read": false
|
||||
}
|
||||
]
|
||||
|
|
@ -117,20 +117,26 @@ Compatibility rules (not yet implemented):
|
|||
|
||||
| Field | Meaning |
|
||||
| --- | --- |
|
||||
| `custody_class` | `development`, `archive`, or `hot_search` |
|
||||
| `retention_days` | Maximum age before eligible deletion; `None` means indefinite |
|
||||
| `custody_class` | `development`, `operational`, `archive`, or `hot_search` |
|
||||
| `retention_days` | Maximum age before eligible deletion; `None` is a lifecycle statement (no expiry), not a recovery guarantee |
|
||||
| `immutable` | Whether stored records are protected from in-place alteration |
|
||||
| `tamper_evidence` | Whether manifests, hash chains, or signatures exist |
|
||||
| `durable` | Whether survival is expected across process restarts and host reboots |
|
||||
| `recoverable_days` | Cited platform backup window; `None` if not declared |
|
||||
| `recoverable_source` | Where the recoverable window is cited from |
|
||||
| `recoverable_basis` | ITC-GOV EvidenceBasis of that citation (`measured`, `quoted`, …) |
|
||||
|
||||
### Custody classes
|
||||
|
||||
| Class | Purpose | Guarantees |
|
||||
| --- | --- | --- |
|
||||
| `development` | Local integration and bootstrap wiring | Ephemeral local files; best-effort cleanup; **not audit custody** |
|
||||
| `archive` | Long-term evidence (planned) | Durable object storage, batch manifests, explicit retention |
|
||||
| `operational` | Durable production custody | Append-only Postgres; recoverable through the platform `data.backup` provision; not ITC-CAP `data.archive` |
|
||||
| `archive` | Long-term evidence (future sink) | Reserved for a backend that can satisfy `data.archive` hooks (retention policy, integrity verification, retrieval test) |
|
||||
| `hot_search` | Operational investigation (planned) | Shorter retention; searchable; not the evidence record |
|
||||
|
||||
`AUDIT_CORE_REQUIRE_CUSTODY_CLASS=operational` is the production fail-closed gate. `archive` is accepted as an alias of `operational` for one mixed rollout so an old manifest cannot refuse a new image. A `development` backend satisfies neither.
|
||||
|
||||
### Mock file backend policy
|
||||
|
||||
`MockFileAuditBackend.retention_policy`:
|
||||
|
|
@ -152,15 +158,20 @@ Enforcement:
|
|||
**Not guaranteed:** crash-safe writes, replication, encryption, tenant isolation,
|
||||
integrity proofs, or survival of `/tmp` across reboots.
|
||||
|
||||
### Production archive policy (planned)
|
||||
### Production operational policy (Postgres, live)
|
||||
|
||||
Target guarantees for the first durable backend:
|
||||
`PostgresAuditBackend.retention_policy`:
|
||||
|
||||
- `custody_class`: `archive`
|
||||
- `retention_days`: scope policy (often years, sometimes indefinite)
|
||||
- `immutable`: true (WORM / object lock where available)
|
||||
- `tamper_evidence`: true (batch manifests with content hashes)
|
||||
- `custody_class`: `operational`
|
||||
- `retention_days`: unset in production (the service does not expire rows)
|
||||
- `immutable`: true (trigger `events_append_only`; not a claim against the database owner)
|
||||
- `tamper_evidence`: false (a superuser can drop the trigger; no hash-chain)
|
||||
- `durable`: true
|
||||
- `recoverable_days`: 30, cited from the platform `data.backup` provision
|
||||
- `recoverable_source`: `resource-control/data/capability/platform-audit-storage.json#provisions[capability=data.backup]`
|
||||
- `recoverable_basis`: `measured`
|
||||
|
||||
`None` retention is a lifecycle statement, not unbounded archive. Rows older than the recoverable window are not promised after a restore. A future `archive` backend that satisfies ITC-CAP `data.archive` is not implemented.
|
||||
|
||||
## Migration path: mock file → durable backend
|
||||
|
||||
|
|
@ -189,15 +200,15 @@ backend.emit(AuditEvent(source="...", action="...", resource="...", outcome="suc
|
|||
|
||||
1. Register a durable archive backend implementing `AuditBackend`.
|
||||
2. Configure routing: development scopes may keep mock; production scopes require
|
||||
`custody_class=archive`.
|
||||
`custody_class=operational` (the live Postgres backend).
|
||||
3. Readiness checks compare `backend.retention_policy` against scope policy and
|
||||
fail closed when custody is insufficient.
|
||||
|
||||
### Phase 2 — archive primary (planned)
|
||||
### Phase 2 — operational primary (live)
|
||||
|
||||
1. Point `emit` calls (or HTTP ingestion) at the archive backend.
|
||||
1. HTTP ingestion writes through `PostgresAuditBackend` (`custody_class=operational`).
|
||||
2. Retain mock only for local `make mock-audit-smoke` and unit tests.
|
||||
3. Export historical mock JSONL into archive batches with manifest generation.
|
||||
3. A future `data.archive` sink is a separate backend, not a rename of Postgres.
|
||||
|
||||
### Phase 3 — hot search adjunct (planned)
|
||||
|
||||
|
|
@ -220,7 +231,9 @@ Archive remains the evidence record; hot search may use shorter `retention_days`
|
|||
| Backend | Module | Custody class |
|
||||
| --- | --- | --- |
|
||||
| Mock file JSONL | `audit_core.mock_file_backend.MockFileAuditBackend` | `development` |
|
||||
| Archive (planned) | TBD | `archive` |
|
||||
| SQLite (local / test) | `audit_core.sqlite_backend.SQLiteAuditBackend` | `development` |
|
||||
| PostgreSQL (production) | `audit_core.postgres_backend.PostgresAuditBackend` | `operational` |
|
||||
| Archive (planned `data.archive` sink) | TBD | `archive` |
|
||||
| Hot search (planned) | TBD | `hot_search` |
|
||||
|
||||
## Related documents
|
||||
|
|
|
|||
|
|
@ -27,10 +27,11 @@
|
|||
"platform_logical_seconds": 3.468,
|
||||
"retention": {
|
||||
"audit_core_retention_days": null,
|
||||
"audit_core_meaning": "does not expire or delete events",
|
||||
"audit_core_meaning": "does not expire or delete events; lifecycle statement, not a recovery guarantee",
|
||||
"platform_planned_window_days": 30,
|
||||
"production_barman": "fail-closed; no off-host copy yet",
|
||||
"recovery_bound": "platform backup retention, not audit-core deletion"
|
||||
"production_barman": "superseded 2026-08-14 by RESOURCE-WP-0002-T05: live Scaleway Barman, 30-day window, audit_core.events 30=30, full 65s / PITR 65s",
|
||||
"recovery_bound": "platform data.backup window (resource:platform:audit-storage), not audit-core deletion",
|
||||
"provision_honesty": "requirement D5, provision D4; cited, not re-scored here"
|
||||
},
|
||||
"verified": true
|
||||
}
|
||||
|
|
|
|||
99
docs/interface-card.yaml
Normal file
99
docs/interface-card.yaml
Normal file
|
|
@ -0,0 +1,99 @@
|
|||
schema: info-tech-canon.interface-card.v1
|
||||
id: audit-core/interface-card
|
||||
title: audit-core Canon Interface Card
|
||||
consumer: audit-core
|
||||
consumer_profile:
|
||||
repo: audit-core
|
||||
domain: infotech
|
||||
owner: audit-core
|
||||
intent: >
|
||||
Provide durable, tenant-aware operational custody for audit events so
|
||||
senders can treat a 202 as evidence-in-store, not a log-forwarding hint.
|
||||
scope:
|
||||
- audit event ingestion
|
||||
- append-only operational custody
|
||||
- sender binding
|
||||
- recovery bound to platform backup
|
||||
purposes:
|
||||
- id: audit-core/operational-custody
|
||||
use_case: Accept normalized events from registered senders and retain them in an append-only store.
|
||||
consumer_need: A joinable operations.audit provision with an honest recovery claim.
|
||||
demand_signals:
|
||||
- user-engine delivers platform and tenant events over POST /v1/events
|
||||
- neighbours already require data.backup in ITC-CAP terms
|
||||
canon_surfaces:
|
||||
- model/capability
|
||||
- model/governance
|
||||
- model/data
|
||||
- model/security
|
||||
surfaces:
|
||||
implemented_profiles: []
|
||||
consumed_artifacts:
|
||||
- model/capability
|
||||
- model/governance
|
||||
- model/data
|
||||
- model/security
|
||||
owned_concepts: []
|
||||
produced_concepts:
|
||||
- Evidence
|
||||
- AuditRecord
|
||||
consumed_concepts:
|
||||
- Evidence
|
||||
- EvidenceBasis
|
||||
- CapabilityProvision
|
||||
- RetentionRuleReference
|
||||
mappings:
|
||||
- from: stored event
|
||||
to: Evidence / AuditRecord
|
||||
note: This service stores evidence. It is not an independent Audit-as-assessment.
|
||||
- from: capability.audit.event-retain
|
||||
to: operations.audit
|
||||
note: data/capability/audit-core-operational.json
|
||||
validation_expectations:
|
||||
commands:
|
||||
- PYTHONPATH=src python3 -m info_tech_canon capability-review /home/worsch/audit-core/data/capability/audit-core-operational.json
|
||||
evidence_required:
|
||||
- data/capability/audit-core-operational.json
|
||||
- docs/operator-runbook.md Restore section
|
||||
- docs/evidence/restore-walk-20260813T121200Z.json
|
||||
known_gaps:
|
||||
- id: data.archive-unprovided
|
||||
owner: audit-core
|
||||
disposition: unmet requirement recorded on the ITC-CAP case; do not build the sink in AUDIT-WP-0006
|
||||
- id: tamper-evidence-false
|
||||
owner: audit-core
|
||||
disposition: integrity_verification hook is unknown; trigger is not a proof
|
||||
- id: no-hash-chain
|
||||
owner: audit-core
|
||||
disposition: INTENT residual, not this workplan
|
||||
- id: single-sender
|
||||
owner: audit-core
|
||||
disposition: NetworkPolicy admits user-engine only; other sources remain adapters
|
||||
- id: no-rapp-yaml
|
||||
owner: railiance-master
|
||||
disposition: schema requires rapp-*; extraction is a first-wave family decision
|
||||
- id: historic-archive-overclaim
|
||||
owner: audit-core
|
||||
disposition: closed by AUDIT-WP-0006-T01; /readyz reports operational
|
||||
purpose_fit:
|
||||
state: partial
|
||||
matched_capabilities:
|
||||
- operations.audit
|
||||
scope_pressure: >
|
||||
INTENT describes a control plane, object archive, hot search, and export.
|
||||
The live service is a single-sender operational custody receiver.
|
||||
recommended_disposition: keep the operational provision honest; do not inflate to data.archive
|
||||
consumer_needs:
|
||||
current:
|
||||
- Honest custody_class and recoverable window on /readyz
|
||||
- Joinable operations.audit provision
|
||||
requested_extensions:
|
||||
- data.archive sink if events must survive past the 30-day backup window
|
||||
feedback: []
|
||||
known_deviations:
|
||||
- no data.archive sink
|
||||
- tamper_evidence=False
|
||||
- no hash-chain
|
||||
- single sender user-engine
|
||||
- no rapp.yaml (not a rapp-* repo)
|
||||
- /readyz historically overclaimed archive (closed by T01)
|
||||
|
|
@ -29,12 +29,13 @@ warden route show database-dynamic-credentials --json
|
|||
| Check | Meaning |
|
||||
| --- | --- |
|
||||
| `GET /healthz` | Process is up. Liveness uses this. A database outage must **not** restart the pod. |
|
||||
| `GET /readyz` | Custody is reachable and `custody_class=archive`. Readiness uses this; the pod leaves the Service rather than accept events it cannot store. |
|
||||
| `GET /readyz` | Custody is reachable and `custody_class=operational`. Also reports `recoverable_days` (cited platform backup window). Readiness uses this; the pod leaves the Service rather than accept events it cannot store. |
|
||||
| `GET /v1/stats` | In-process counters since start (`accepted`, `duplicate`, `conflict`, `rejected`, `unauthorized`, `forbidden`, `unavailable`, `error`). Resets on restart. Requires `may_read`. |
|
||||
|
||||
A missing `AUDIT_CORE_DATABASE_URL` / credential directory is a startup
|
||||
failure (`AUDIT_CORE_REQUIRE_CUSTODY_CLASS=archive`), not a silent downgrade
|
||||
to SQLite.
|
||||
failure (`AUDIT_CORE_REQUIRE_CUSTODY_CLASS=operational`), not a silent
|
||||
downgrade to SQLite. An older manifest that still requires `archive` is
|
||||
accepted as an alias for one mixed rollout.
|
||||
|
||||
## Lookup
|
||||
|
||||
|
|
@ -96,9 +97,14 @@ replace that path with a founder paste. Rotation is overlap-first:
|
|||
5. Either wait for the 1h refresh or annotate the ExternalSecret to force a
|
||||
sync, then restart the pod so it re-reads `AUDIT_CORE_SENDERS`.
|
||||
|
||||
The write token is bound to `source=user-engine` and the tenants that
|
||||
identity may claim. The operator token is a separate identity with
|
||||
`may_read: true`. Do not reuse one token for both.
|
||||
The write token is bound to `source=user-engine`. Tenant scope for that
|
||||
identity is **not a secret**: `deploy/senders-scope.json` (ConfigMap
|
||||
`audit-core-senders-scope`) overlays `tenants: ["*"]` onto the Secret
|
||||
at start. An ExternalSecret refresh cannot revert it to a single tenant.
|
||||
Tokens stay in Secret `audit-core-senders` / OpenBao KV.
|
||||
|
||||
The operator token is a separate identity with `may_read: true`. Do not
|
||||
reuse one token for both.
|
||||
|
||||
A shape (values are placeholders) is in `docs/senders.example.json`.
|
||||
|
||||
|
|
@ -142,17 +148,34 @@ behind the read privilege). Watch:
|
|||
|
||||
## Restore
|
||||
|
||||
audit-core does **not** expire events (`retention_days` unset). Recovery is
|
||||
bounded by what rapp-postgres can restore, not by an audit-core deletion
|
||||
window. The platform's planned Barman window is 30 days. Production Barman
|
||||
is still fail-closed (no governed off-host target). Do not promise an RPO
|
||||
until that target exists. Local WAL on the node is not an off-host copy.
|
||||
audit-core does **not** expire events (`retention_days` unset). That is a
|
||||
lifecycle policy statement, not a recovery guarantee. Recoverable history
|
||||
is the platform `data.backup` window: **30 days**, prefix `platform-pg/`,
|
||||
bucket owned by `resource:platform:audit-storage`. Cite, do not copy:
|
||||
|
||||
- `resource-control/data/capability/platform-audit-storage.json`
|
||||
- `rapp-postgres/docs/restore.md`
|
||||
|
||||
The platform requirement is `data.backup` profile `database` at D5
|
||||
(RPO 5 min, RTO 60 min, 30-day retention, not in the railiance01 /
|
||||
host-europe failure domain). The live provision is **D4**, not D5:
|
||||
resource-control scored one backup, one full restore, one PITR, and
|
||||
does not yet emit `wal_archive_gap_minutes`. RPO/RTO numbers are
|
||||
theirs (`measured`, single observation). audit-core does not claim a
|
||||
better grade.
|
||||
|
||||
Rows older than the 30-day window are not promised after a restore.
|
||||
Local WAL on the node is not a second copy.
|
||||
|
||||
Physical restore is instance-wide. A consumer-only restore is a logical
|
||||
export of `audit_core` from a scratch physical restore, then a controlled
|
||||
import. Never recover in place. Procedure: `rapp-postgres/docs/restore.md`.
|
||||
|
||||
Walked 2026-08-13:
|
||||
The 2026-08-13 fail-closed sentence is superseded by RESOURCE-WP-0002-T05
|
||||
(2026-08-14): production Barman to Scaleway, `audit_core.events` 30=30,
|
||||
full restore 65 s, PITR 65 s.
|
||||
|
||||
Walked 2026-08-13 (pre-commissioning historical evidence):
|
||||
|
||||
| Path | Evidence | Elapsed | Result |
|
||||
| --- | --- | --- | --- |
|
||||
|
|
@ -185,5 +208,6 @@ narrows a column must replace that note before release.
|
|||
3. Sender registry is Secret `audit-core-senders` (in-cluster mint).
|
||||
Database leases come from `openbao-audit-core-database`.
|
||||
4. Job `audit-core-migrate` with `AUDIT_CORE_MIGRATE_ROLE=audit_core_migrate`.
|
||||
5. Deployment. `/readyz` must report `custody_class=archive`.
|
||||
5. Deployment. `/readyz` must report `custody_class=operational` and
|
||||
`recoverable_days=30`.
|
||||
6. In-pod `MODE=remote DISRUPT=0` failure matrix. Evidence goes to NK-WP-0024.
|
||||
|
|
|
|||
|
|
@ -3,40 +3,37 @@ id: capability.audit.event-retain
|
|||
name: Audit Event Retention
|
||||
summary: Collect, normalize, retain, and search audit events with integrity evidence across tenants.
|
||||
owner: audit-core
|
||||
status: draft
|
||||
domain: helix_forge
|
||||
status: production
|
||||
domain: infotech
|
||||
tags: [audit, retention, compliance]
|
||||
|
||||
maturity:
|
||||
discovery:
|
||||
current: D4
|
||||
target: D6
|
||||
confidence: medium
|
||||
rationale: audit-core INTENT defines full audit fabric scope and integration boundaries.
|
||||
availability:
|
||||
current: A2
|
||||
target: A5
|
||||
confidence: low
|
||||
rationale: Core modules exist; deployable service packaging in progress.
|
||||
joins:
|
||||
itc_cap: operations.audit
|
||||
provision: data/capability/audit-core-operational.json
|
||||
provision_maturity: D4
|
||||
|
||||
external_evidence:
|
||||
completeness:
|
||||
level: C2
|
||||
name: Partial
|
||||
confidence: low
|
||||
basis: scope_vs_intent_and_consumer_expectations
|
||||
level: C3
|
||||
name: Substantial
|
||||
confidence: medium
|
||||
basis: live_receiver_and_restore_walk
|
||||
satisfied_expectations:
|
||||
- retention and integrity goals documented
|
||||
- HTTP ingest through the backend contract
|
||||
- append-only Postgres custody on platform-pg
|
||||
- recovery cited to the live platform data.backup provision
|
||||
broken_expectations:
|
||||
- federation with all platform runtimes not proven in registry
|
||||
- data.archive sink not provided
|
||||
- tamper evidence not implemented
|
||||
out_of_scope_expectations:
|
||||
- application business audit semantics ownership
|
||||
- booked-cost origination
|
||||
reliability:
|
||||
level: R1
|
||||
confidence: low
|
||||
basis: consumer_quality_signals
|
||||
level: R2
|
||||
confidence: medium
|
||||
basis: failure_matrix_and_restore_walk
|
||||
known_reliability_risks:
|
||||
- multi-tenant isolation not evidenced here
|
||||
- single replica
|
||||
- integrity_verification hook unmet
|
||||
|
||||
discovery:
|
||||
intent: >
|
||||
|
|
@ -49,14 +46,19 @@ discovery:
|
|||
- tamper evidence
|
||||
excludes:
|
||||
- generating domain business events
|
||||
- procuring or operating platform backup
|
||||
- booked financial facts
|
||||
use_cases: []
|
||||
|
||||
availability:
|
||||
current_level: A2
|
||||
current_level: A4
|
||||
target_level: A5
|
||||
current_artifacts:
|
||||
- audit-core/
|
||||
- audit-core/deploy/audit-core.yaml
|
||||
- audit-core/audit_core/postgres_backend.py
|
||||
- rapp-postgres/consumers/audit-core.yaml
|
||||
consumption_modes:
|
||||
- http ingest
|
||||
- source module
|
||||
|
||||
relations:
|
||||
|
|
@ -64,17 +66,25 @@ relations:
|
|||
related_to:
|
||||
- capability.activity.event-coordinate
|
||||
- capability.statehub.progress-log
|
||||
uses_provisions:
|
||||
- data.transactional (rapp-postgres/platform-pg)
|
||||
- data.backup (resource:platform:audit-storage, cited)
|
||||
- security.secrets (OpenBao / ESO)
|
||||
|
||||
consumer_guidance:
|
||||
recommended_for:
|
||||
- planning audit retention independent of a single product
|
||||
- platform and application audit event delivery over POST /v1/events
|
||||
not_recommended_for:
|
||||
- treating this store as WORM archive
|
||||
- replacing application-level logging only
|
||||
known_limitations:
|
||||
- consumer evidence not yet collected in registry
|
||||
- recoverable history is the 30-day platform backup window
|
||||
- no hash-chain or export API yet
|
||||
---
|
||||
|
||||
# Audit Event Retention
|
||||
|
||||
Audit Core provides the retention and integrity layer for audit events across
|
||||
the platform.
|
||||
Audit Core provides the operational custody layer for audit events.
|
||||
ITC-CAP join: `operations.audit` at provision maturity D4
|
||||
(`data/capability/audit-core-operational.json`). Maturity is not a
|
||||
property of this abstract capability.
|
||||
|
|
|
|||
|
|
@ -1,14 +1,16 @@
|
|||
version: 1
|
||||
updated: '2026-06-16'
|
||||
domain: helix_forge
|
||||
updated: '2026-08-16'
|
||||
domain: infotech
|
||||
capabilities:
|
||||
- id: capability.audit.event-retain
|
||||
name: Audit Event Retention
|
||||
summary: Collect, normalize, retain, and search audit events with integrity evidence
|
||||
across tenants.
|
||||
vector: D4 / A2 / C2 / R1
|
||||
domain: helix_forge
|
||||
status: draft
|
||||
joins: operations.audit
|
||||
provision: data/capability/audit-core-operational.json
|
||||
vector: D4 provision / A4 / C3 / R2
|
||||
domain: infotech
|
||||
status: production
|
||||
owner: audit-core
|
||||
path: registry/capabilities/capability.audit.event-retain.md
|
||||
tags:
|
||||
|
|
@ -16,4 +18,5 @@ capabilities:
|
|||
- retention
|
||||
- compliance
|
||||
consumption_modes:
|
||||
- http ingest
|
||||
- source module
|
||||
|
|
|
|||
|
|
@ -184,7 +184,7 @@ def start_stack() -> subprocess.Popen:
|
|||
"AUDIT_CORE_SENDERS": senders,
|
||||
"AUDIT_CORE_HOST": "127.0.0.1",
|
||||
"AUDIT_CORE_HTTP_PORT": str(APP_PORT),
|
||||
"AUDIT_CORE_REQUIRE_CUSTODY_CLASS": "archive",
|
||||
"AUDIT_CORE_REQUIRE_CUSTODY_CLASS": "operational",
|
||||
"AUDIT_CORE_DB_STATEMENT_TIMEOUT_MS": "5000",
|
||||
"AUDIT_CORE_LOG_LEVEL": "WARNING",
|
||||
}
|
||||
|
|
@ -391,7 +391,7 @@ def start_app_process() -> subprocess.Popen:
|
|||
"AUDIT_CORE_SENDERS": senders,
|
||||
"AUDIT_CORE_HOST": "127.0.0.1",
|
||||
"AUDIT_CORE_HTTP_PORT": str(APP_PORT),
|
||||
"AUDIT_CORE_REQUIRE_CUSTODY_CLASS": "archive",
|
||||
"AUDIT_CORE_REQUIRE_CUSTODY_CLASS": "operational",
|
||||
"AUDIT_CORE_DB_STATEMENT_TIMEOUT_MS": "5000",
|
||||
"AUDIT_CORE_LOG_LEVEL": "WARNING",
|
||||
}
|
||||
|
|
|
|||
|
|
@ -86,7 +86,11 @@ def digest(event: AuditEvent) -> str:
|
|||
def test_declares_a_retention_policy(backend):
|
||||
policy = backend.retention_policy
|
||||
assert policy.durable is True
|
||||
assert policy.custody_class in ("development", "archive", "hot_search")
|
||||
assert policy.custody_class in ("development", "operational", "archive", "hot_search")
|
||||
if policy.custody_class == "operational":
|
||||
assert policy.recoverable_days == 30
|
||||
assert policy.recoverable_basis == "measured"
|
||||
assert policy.recoverable_source
|
||||
# A backend claiming tamper evidence must also claim immutability;
|
||||
# the reverse is allowed.
|
||||
if policy.tamper_evidence:
|
||||
|
|
|
|||
65
tests/test_capability_case.py
Normal file
65
tests/test_capability_case.py
Normal file
|
|
@ -0,0 +1,65 @@
|
|||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
RECORD = ROOT / "data" / "capability" / "audit-core-operational.json"
|
||||
CARD = ROOT / "docs" / "interface-card.yaml"
|
||||
SCHEMA = Path.home() / "info-tech-canon" / "infospace" / "schemas" / "interface-card.schema.yaml"
|
||||
|
||||
|
||||
def test_capability_record_exists_and_joins_operations_audit():
|
||||
record = json.loads(RECORD.read_text())
|
||||
assert any(item["capability"] == "operations.audit" for item in record["requires"])
|
||||
assert any(item["capability"] == "data.archive" for item in record["requires"])
|
||||
provisions = {item["capability"]: item for item in record["provisions"]}
|
||||
assert "data.archive" not in provisions
|
||||
audit = provisions["operations.audit"]
|
||||
assert audit["maturity"] == "D4"
|
||||
assert audit["provider"] == "audit-core.railiance01"
|
||||
used = {item["capability"]: item["relation"] for item in audit["uses_provisions"]}
|
||||
assert used["data.backup"] == "may_use"
|
||||
assert used["security.secrets"] == "may_use"
|
||||
unknown = [row for row in audit["consumes"] if row["basis"] == "unknown"]
|
||||
assert unknown
|
||||
assert all(row["quantity"]["value"] is None and row.get("gap") for row in unknown)
|
||||
|
||||
|
||||
def test_capability_review_against_live_catalog():
|
||||
import sys
|
||||
|
||||
canon_src = Path.home() / "info-tech-canon" / "src"
|
||||
if canon_src.is_dir() and str(canon_src) not in sys.path:
|
||||
sys.path.insert(0, str(canon_src))
|
||||
try:
|
||||
from info_tech_canon.capability import review_path
|
||||
except ImportError:
|
||||
pytest.skip("info-tech-canon is not importable")
|
||||
result = review_path(RECORD)
|
||||
assert result["ok"] is True
|
||||
by_cap = {item["capability"]: item for item in result["requirements"]}
|
||||
assert by_cap["operations.audit"]["status"] == "met"
|
||||
assert by_cap["data.archive"]["status"] == "unprovided"
|
||||
|
||||
|
||||
def test_interface_card_has_required_fields():
|
||||
text = CARD.read_text()
|
||||
assert "id: audit-core/interface-card" in text
|
||||
assert "title:" in text
|
||||
assert "consumer:" in text
|
||||
assert "canon_surfaces:" in text
|
||||
assert "Evidence" in text
|
||||
assert "AuditRecord" in text
|
||||
assert "data.archive-unprovided" in text
|
||||
assert "no-rapp-yaml" in text
|
||||
|
||||
|
||||
def test_interface_card_validates_against_canon_schema():
|
||||
if not SCHEMA.is_file():
|
||||
pytest.skip("info-tech-canon interface-card schema not on disk")
|
||||
yaml = pytest.importorskip("yaml")
|
||||
jsonschema = pytest.importorskip("jsonschema")
|
||||
schema = yaml.safe_load(SCHEMA.read_text())
|
||||
card = yaml.safe_load(CARD.read_text())
|
||||
jsonschema.Draft202012Validator(schema).validate(card)
|
||||
|
|
@ -4,7 +4,11 @@ import json
|
|||
import pytest
|
||||
|
||||
from audit_core.ingestion import IngestionApplication
|
||||
from audit_core.interface import BackendUnavailableError, RetentionPolicy
|
||||
from audit_core.interface import (
|
||||
BackendUnavailableError,
|
||||
RetentionPolicy,
|
||||
custody_class_satisfies,
|
||||
)
|
||||
from audit_core.mock_file_backend import MockFileAuditBackend
|
||||
from audit_core.sqlite_backend import SQLiteAuditBackend
|
||||
|
||||
|
|
@ -408,10 +412,71 @@ def test_required_custody_class_refuses_a_development_backend(tmp_path):
|
|||
"""Losing AUDIT_CORE_DATABASE_URL must fail to start, not silently
|
||||
downgrade custody to the development store."""
|
||||
backend = SQLiteAuditBackend(str(tmp_path / "dev.db"))
|
||||
with pytest.raises(ValueError, match="does not meet the required"):
|
||||
IngestionApplication(backend, "opaque", require_custody_class="operational")
|
||||
with pytest.raises(ValueError, match="does not meet the required"):
|
||||
IngestionApplication(backend, "opaque", require_custody_class="archive")
|
||||
|
||||
|
||||
def test_operational_and_archive_alias_for_one_deploy():
|
||||
"""A mixed rollout must start: new backend + old require, and the reverse."""
|
||||
|
||||
class _Operational(_BrokenBackend):
|
||||
@property
|
||||
def retention_policy(self):
|
||||
return RetentionPolicy(
|
||||
custody_class="operational",
|
||||
retention_days=None,
|
||||
immutable=True,
|
||||
tamper_evidence=False,
|
||||
durable=True,
|
||||
recoverable_days=30,
|
||||
recoverable_source="cited",
|
||||
recoverable_basis="measured",
|
||||
)
|
||||
|
||||
IngestionApplication(_Operational(), "opaque", require_custody_class="archive")
|
||||
IngestionApplication(_Operational(), "opaque", require_custody_class="operational")
|
||||
|
||||
|
||||
def test_custody_class_alias_is_not_development():
|
||||
assert custody_class_satisfies("operational", "archive")
|
||||
assert custody_class_satisfies("archive", "operational")
|
||||
assert not custody_class_satisfies("development", "operational")
|
||||
assert not custody_class_satisfies("development", "archive")
|
||||
assert custody_class_satisfies("development", "development")
|
||||
|
||||
|
||||
def test_readiness_reports_recovery_fields_for_operational_backend():
|
||||
class _Operational(_BrokenBackend):
|
||||
@property
|
||||
def retention_policy(self):
|
||||
return RetentionPolicy(
|
||||
custody_class="operational",
|
||||
retention_days=None,
|
||||
immutable=True,
|
||||
tamper_evidence=False,
|
||||
durable=True,
|
||||
recoverable_days=30,
|
||||
recoverable_source="resource-control/data/capability/platform-audit-storage.json",
|
||||
recoverable_basis="measured",
|
||||
)
|
||||
|
||||
def health(self):
|
||||
return None
|
||||
|
||||
status, body = invoke(
|
||||
IngestionApplication(_Operational(), "opaque"),
|
||||
None, path="/readyz", method="GET", body=b"",
|
||||
)
|
||||
assert status.startswith("200")
|
||||
assert body["custody_class"] == "operational"
|
||||
assert body["durable"] is True
|
||||
assert body["recoverable_days"] == 30
|
||||
assert body["recoverable_basis"] == "measured"
|
||||
assert "platform-audit-storage" in body["recoverable_source"]
|
||||
|
||||
|
||||
def test_counters_track_each_outcome(tmp_path):
|
||||
app, _ = bound_app(tmp_path, may_read=True)
|
||||
invoke(app, event()) # accepted
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ from audit_core.interface import (
|
|||
EventValidationError,
|
||||
RetentionPolicy,
|
||||
SCHEMA_VERSION_V1ALPHA1,
|
||||
custody_class_satisfies,
|
||||
validate_event,
|
||||
)
|
||||
from audit_core.mock_file_backend import MockFileAuditBackend
|
||||
|
|
@ -79,6 +80,29 @@ def test_mock_backend_retention_policy_none_when_cleanup_disabled():
|
|||
assert backend.retention_policy.retention_days is None
|
||||
|
||||
|
||||
def test_custody_class_satisfies_aliases_operational_and_archive():
|
||||
assert custody_class_satisfies("operational", "archive") is True
|
||||
assert custody_class_satisfies("archive", "operational") is True
|
||||
assert custody_class_satisfies("operational", "development") is False
|
||||
|
||||
|
||||
def test_readiness_payload_includes_recovery_when_declared():
|
||||
policy = RetentionPolicy(
|
||||
custody_class="operational",
|
||||
retention_days=None,
|
||||
immutable=True,
|
||||
tamper_evidence=False,
|
||||
durable=True,
|
||||
recoverable_days=30,
|
||||
recoverable_source="cited",
|
||||
recoverable_basis="measured",
|
||||
)
|
||||
body = policy.as_readiness()
|
||||
assert body["status"] == "ok"
|
||||
assert body["custody_class"] == "operational"
|
||||
assert body["recoverable_days"] == 30
|
||||
|
||||
|
||||
def test_audit_event_record_uses_v1alpha1_schema():
|
||||
event = AuditEvent(
|
||||
source="audit-core",
|
||||
|
|
|
|||
97
tests/test_senders.py
Normal file
97
tests/test_senders.py
Normal file
|
|
@ -0,0 +1,97 @@
|
|||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from audit_core.senders import SenderRegistry, WILDCARD
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
SCOPE_FILE = ROOT / "deploy" / "senders-scope.json"
|
||||
|
||||
|
||||
def test_declared_scope_keeps_user_engine_tenants_wildcard():
|
||||
scope = json.loads(SCOPE_FILE.read_text())
|
||||
user_engine = next(entry for entry in scope if entry["name"] == "user-engine")
|
||||
assert user_engine["tenants"] == ["*"]
|
||||
assert user_engine["sources"] == ["user-engine"]
|
||||
assert user_engine["may_write"] is True
|
||||
assert user_engine["may_read"] is False
|
||||
assert "tokens" not in user_engine
|
||||
assert "token" not in user_engine
|
||||
|
||||
|
||||
def test_scope_overlay_widens_narrow_secret_tenants(tmp_path):
|
||||
secret = json.dumps(
|
||||
[
|
||||
{
|
||||
"name": "user-engine",
|
||||
"tokens": ["live-token"],
|
||||
"sources": ["user-engine"],
|
||||
"tenants": ["tenant:friendly:binky"],
|
||||
"may_write": True,
|
||||
"may_read": False,
|
||||
}
|
||||
]
|
||||
)
|
||||
env = {
|
||||
"AUDIT_CORE_SENDERS": secret,
|
||||
"AUDIT_CORE_SENDERS_SCOPE_PATH": str(SCOPE_FILE),
|
||||
}
|
||||
registry = SenderRegistry.from_env(env)
|
||||
identity = registry.authenticate("Bearer live-token")
|
||||
assert identity is not None
|
||||
assert WILDCARD in identity.tenants
|
||||
assert identity.permits_tenant("tenant:other:x")
|
||||
assert identity.tokens == ("live-token",)
|
||||
|
||||
|
||||
def test_scope_overlay_does_not_take_tokens_from_git():
|
||||
overlay = json.dumps(
|
||||
[
|
||||
{
|
||||
"name": "user-engine",
|
||||
"tokens": ["must-not-be-used"],
|
||||
"tenants": ["*"],
|
||||
}
|
||||
]
|
||||
)
|
||||
secret = json.dumps(
|
||||
[
|
||||
{
|
||||
"name": "user-engine",
|
||||
"tokens": ["live-token"],
|
||||
"sources": ["user-engine"],
|
||||
"tenants": ["tenant:friendly:binky"],
|
||||
}
|
||||
]
|
||||
)
|
||||
registry = SenderRegistry.from_env(
|
||||
{"AUDIT_CORE_SENDERS": secret, "AUDIT_CORE_SENDERS_SCOPE": overlay}
|
||||
)
|
||||
assert registry.authenticate("Bearer must-not-be-used") is None
|
||||
assert registry.authenticate("Bearer live-token") is not None
|
||||
|
||||
|
||||
def test_missing_overlay_leaves_secret_as_is():
|
||||
secret = json.dumps(
|
||||
[
|
||||
{
|
||||
"name": "user-engine",
|
||||
"tokens": ["live-token"],
|
||||
"sources": ["user-engine"],
|
||||
"tenants": ["tenant:friendly:binky"],
|
||||
}
|
||||
]
|
||||
)
|
||||
identity = SenderRegistry.from_env({"AUDIT_CORE_SENDERS": secret}).identities[0]
|
||||
assert identity.tenants == frozenset({"tenant:friendly:binky"})
|
||||
|
||||
|
||||
def test_invalid_scope_overlay_is_a_startup_error():
|
||||
secret = json.dumps(
|
||||
[{"name": "user-engine", "tokens": ["t"], "sources": ["user-engine"]}]
|
||||
)
|
||||
with pytest.raises(ValueError, match="JSON list"):
|
||||
SenderRegistry.from_env(
|
||||
{"AUDIT_CORE_SENDERS": secret, "AUDIT_CORE_SENDERS_SCOPE": "{}"}
|
||||
)
|
||||
|
|
@ -4,16 +4,17 @@ type: workplan
|
|||
title: "Honest operational custody against ITC-CAP and the live platform backup"
|
||||
domain: infotech
|
||||
repo: audit-core
|
||||
status: ready
|
||||
status: finished
|
||||
owner: grok
|
||||
topic_slug: railiance
|
||||
created: "2026-08-15"
|
||||
updated: "2026-08-15"
|
||||
updated: "2026-08-16"
|
||||
depends_on:
|
||||
- AUDIT-WP-0005
|
||||
- RESOURCE-WP-0002
|
||||
- ITC-WP-0014
|
||||
- ITC-WP-0015
|
||||
state_hub_workstream_id: "8d775ffb-3c83-4c33-9ffa-05ce52c5ff91"
|
||||
---
|
||||
|
||||
# AUDIT-WP-0006 — Honest operational custody against ITC-CAP and the live platform backup
|
||||
|
|
@ -122,8 +123,9 @@ not build the sink here.
|
|||
|
||||
```task
|
||||
id: AUDIT-WP-0006-T01
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "bbf476eb-7bc3-4cb6-bd1c-9b2c75903906"
|
||||
```
|
||||
|
||||
`CustodyClass` is currently `development | archive | hot_search`. The
|
||||
|
|
@ -164,12 +166,21 @@ Done when: unit tests cover the new class and the alias; production
|
|||
manifest requires the honest class; contract and `/readyz` no longer
|
||||
call Postgres `data.archive`.
|
||||
|
||||
Done 2026-08-16: `CustodyClass` includes `operational`; Postgres reports
|
||||
it with a 30-day cited recoverable window (`measured`).
|
||||
`AUDIT_CORE_REQUIRE_CUSTODY_CLASS=operational` in the manifest; `archive`
|
||||
is a one-deploy alias. `/readyz` publishes recovery fields. Contract
|
||||
replaced "Production archive policy (planned)" with the live operational
|
||||
policy. Suite 84 passed. **Do not apply the Deployment until a new image
|
||||
is pinned** — the live image still reports `archive` and has no alias.
|
||||
|
||||
## T02 — Bind the recovery promise to the live platform backup
|
||||
|
||||
```task
|
||||
id: AUDIT-WP-0006-T02
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "06907b2c-a822-48f0-8edc-50dd8914342b"
|
||||
```
|
||||
|
||||
`docs/operator-runbook.md` still says production Barman is fail-closed
|
||||
|
|
@ -198,12 +209,17 @@ RESOURCE-WP-0002-T05 (30 events, 65 s full / 65 s PITR).
|
|||
Done when the runbook and the T06 evidence note no longer contradict
|
||||
the live backup provision.
|
||||
|
||||
Done 2026-08-16: Restore section cites `resource:platform:audit-storage`,
|
||||
D5 requirement / D4 provision, and supersedes the 2026-08-13 fail-closed
|
||||
sentence with RESOURCE-WP-0002-T05. Evidence JSON updated in place.
|
||||
|
||||
## T03 — Publish an ITC-CAP case for the live provision
|
||||
|
||||
```task
|
||||
id: AUDIT-WP-0006-T03
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "eba29df0-36b9-4f43-add6-760285c56bde"
|
||||
```
|
||||
|
||||
Neighbours already restate real provisions against the live catalog
|
||||
|
|
@ -265,12 +281,18 @@ catalog. Gaps are allowed when they name owner and disposition.
|
|||
Done when the record validates and the reuse-surface card no longer
|
||||
contradicts the live receiver.
|
||||
|
||||
Done 2026-08-16: `data/capability/audit-core-operational.json` reviews
|
||||
`ok` against ITC-CAP 0.4.0 (`operations.audit` met at D4; `data.archive`
|
||||
unprovided). Reuse-surface card joins that provision, domain `infotech`,
|
||||
status `production`.
|
||||
|
||||
## T04 — Publish a Canon Interface Card
|
||||
|
||||
```task
|
||||
id: AUDIT-WP-0006-T04
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "c5f5a883-834f-42eb-8f05-de2857d3904f"
|
||||
```
|
||||
|
||||
ITC-GOV / ITC-SEC / ITC-DATA say subsystems that produce Evidence
|
||||
|
|
@ -297,12 +319,17 @@ Declare at least:
|
|||
Done when the card exists in-repo and validates against
|
||||
`interface-card.schema.yaml`.
|
||||
|
||||
Done 2026-08-16: `docs/interface-card.yaml` validates. Schema wants
|
||||
`consumer` as a string and `canon_surfaces` as a string array; richer
|
||||
template fields live under `consumer_profile` / `surfaces`.
|
||||
|
||||
## T05 — Persist sender tenant scope and refresh SCOPE
|
||||
|
||||
```task
|
||||
id: AUDIT-WP-0006-T05
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "c629d894-2fe6-4b46-9909-7267d9170c84"
|
||||
```
|
||||
|
||||
Inbox 2026-08-13 from net-kingdom (NK-WP-0024): live user-engine sender
|
||||
|
|
@ -327,6 +354,12 @@ Done when the live sender document cannot revert to a single-tenant
|
|||
list on refresh, SCOPE matches the repo, and the net-kingdom message
|
||||
is answered.
|
||||
|
||||
Done 2026-08-16: Live Secret already has user-engine `tenants: ["*"]`.
|
||||
`deploy/senders-scope.json` + ConfigMap `audit-core-senders-scope`
|
||||
applied on railiance01. The process overlays that file over the Secret
|
||||
so a later KV refresh cannot shrink tenants. Tokens stay out of Git.
|
||||
SCOPE current state updated.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- Production fail-closed gate no longer keys off the word `archive`.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue