activity-core/docs/release-broker.md

# Image-only release broker core

ACTIVITY-WP-0041-T03 owns this implementation. **Not activated in production.**
The modules are not wired to a production API, worker, schedule or credential source.
Construction requires `admitted=True` from trusted deployment configuration; that
switch is a local guard, not proof that an identity has actually been admitted.
The current deployed revision is unchanged, so this code does not restart soak.

## Implemented boundary

`activity_core.release_broker.Receipts` checks Ed25519 signatures against a locally
configured key-to-principal/role registry. Requests cannot introduce trusted keys.
Build, independent review, health and retention attestations must all name the
same fixed repository/application, full candidate and rollback commits, and exact
before/after manifest hashes. Evidence expires within five minutes. Build and
review require different principals **and keys**. Build evidence must cover exact
candidate image digests and all three mandatory CI contexts. Retention must cover
both live and rollback images. Health must attest continuous healthy observation
of the prior revision for at least 24 hours. The trusted issuers must verify these
facts against their authorities; signatures alone cannot make assertions true.

`gitops_policy` is the shared image-only validator used by both broker and the
existing CLI. It forbids non-image deployment changes and resource-set changes.
`release_operations` constructs a one-field update of the validated platform child
Application and fixed selective root/child sync payloads. No arbitrary command,
repository path, application, prune flag or override is exposed by these builders.

## Durable recovery

A local SQLite ledger stores signed receipts, their hashes, the release binding,
phase and transition history. One active row and an immediate transaction serialize
all releases and adapter calls on that ledger. The database must reside on durable,
operator-owned local storage, shared by all instances handling this application;
this is not a distributed lock across independent databases or network filesystems.

Phases are planned → publish_pending → published → synced → complete. Publication
intent is committed before calling the adapter. Lost responses therefore cannot
turn a possibly published change into an assumed cancellation. A release that
expires before any attempt is cancelled; after intent, timeout or failed health
enters rollback_planned → rollback_published → rollback_synced → rolled_back.
Failed rollback retains the active slot. Retried publication/sync must be
idempotent, using compare-and-swap and accepting already-at-target as success.
The adapter must refuse any unexpected third revision rather than overwriting it.

Tests use generated fixture keys, disposable real Git repositories and a simulated
Kubernetes transport. They prove signed admission, real Git publication/rollback,
concurrent-writer rejection and lost-push-response recovery. Kubernetes authorization,
production credentials and live Git/ArgoCD rollback are **not** proven by these fixtures.

## Still required before activation

1. Admit and exercise the implemented transport adapter against an isolated
   authenticated Git/Kubernetes environment. Supply the actual bounded report/schedule
   invariant probe, pinned cluster UID and immutable credential/context configuration.
   Verify the real authorization denials and restart behavior; simulated Kubernetes
   responses do not prove those. A generic repository-write token is not path enforcement.
2. Admit the dedicated principal and credential custody through the owner lane.
   ArgoCD Core has no API-server token lane. Kubernetes Application patch RBAC
   alone cannot restrict fields: keep it behind the reviewed broker boundary.
   Publish negative access tests and revocation behavior; no broad operator key.
3. Supply independent trusted build/review/health/retention issuers and their key
   custody/rotation. The observer must measure continuous health; it must not
   manufacture a 24-hour interval from two snapshots. Preserve signing public
   keys for audit and protect the ledger from producer writes.
4. Deploy/register the implemented Temporal dispatcher and supply its bounded,
   idempotent sanitized evidence sink. Prove authenticated transport failure, concurrency with other
   publishers, restart, denial and rollback in an isolated deployment environment.
   Then finish the production observation gate and enable the bounded scope.

Platform enforcement contract: `railiance-platform/docs/activity-core-release-admission.md`.
These requirements remain live work in ACTIVITY-WP-0041-T03 and RPF-WP-0048-T02.


## Transport and observer implementation

`release_transport.GitArgoBackend` retrieves both full source commits from the fixed
activity-core repository and compares parsed manifest hashes with the admitted
binding. It accepts only the audited static `runtime.yaml` Kustomization; proposed
source cannot execute a renderer/plugin. Git publication creates a commit changing
only the platform child revision field. Non-force pushes reject concurrent branch
advances. After a lost response, remote Git is the durable source of truth: a retry
accepts already-at-target and selective sync uses the current verified platform tip.
Unexpected third revisions and unrelated changed paths stop the operation.

The adapter verifies the operator-pinned kube-system namespace UID before Git
publication or Kubernetes access. Context/credentials must remain immutable during
its lifetime. Every subprocess is non-shell, with timeouts and sanitized failures.
Kubernetes JSON patches compare resourceVersion and spec before setting the fixed
sync operation. Existing foreign operations are never overwritten. Root sync selects
only activity-core; child sync never prunes. Health checks require exact revision,
Synced/Healthy and all three deployments ready at their observed generations, plus
a mandatory report/schedule invariant probe. That trusted callback must itself use
bounded I/O; no production probe is supplied or implied by the fixture implementation.
Polling windows are bounded, with individual command timeout overhead possible.

`release_observer.HealthObserver` records samples durably. It requires consecutive
healthy samples of one revision with gaps no greater than 90 seconds. Failed samples,
revision changes and missed samples reset the interval; clock regression durably
invalidates it. Attestation requires a complete 24-hour sampled interval and a fresh
last sample, including after restart. This is evidence at a bounded sampling cadence,
not a claim that every instant between samples was observed. Two distant healthy
snapshots cannot establish the interval. New observers start from their first actual
sample; they do not backfill the earlier deployment timestamp.

Neither module is connected to a production schedule, endpoint or credential source.
Trusted signer custody, real invariant probes, production Temporal registration and isolated
Kubernetes authorization/rollback tests remain required before activation. Production
revision and the deployment observation clock are unchanged by this source-only work.

## Temporal dispatch and durable audit delivery

`release_dispatch.ReleaseDispatchWorkflow` resumes an already-admitted release ID.
`ReleaseActivities` binds its broker, backend factory and audit sink at trusted worker
construction; Temporal inputs cannot supply manifests, keys, transport configuration
or an admission flag. `release_worker` constructs a separate
`activity-core-release-tq` worker, which the admitted deployment must explicitly run.
It does not modify the ordinary orchestrator worker or start a workflow/schedule.
Use a stable workflow ID such as `activity-core-release:<ledger release ID>` when
starting it. Broker serialization remains authoritative even with duplicate dispatch.

Adapter calls run outside the workflow and resume from the durable ledger after
activity retry or restart. Failed rollback keeps the release slot and is retried
with durable timers; continue-as-new bounds workflow history. Temporal receives
sanitized errors rather than raw transport exceptions. Disabling the broker's
trusted admission setting prevents subsequent activity calls; operational revocation
still requires the admitted worker/credential lifecycle, not a workflow input.

The transition ledger is also the audit outbox. Acknowledgements are persisted only
after the configured sink returns. Delivery is at least once with stable event IDs;
the sink must durably upsert those IDs and use bounded I/O. It receives only release
ID, phase, time, candidate/rollback commits and the fixed application name. Sink
credentials, signed envelopes and transport output never enter these events.
Nonterminal sink failure retains pending events and permits recovery to proceed;
terminal workflow completion waits for acknowledged audit delivery. One ledger is
bound to one logical audit sink; changing sinks requires an explicit replay/migration
of delivery acknowledgements. The sink can fan out to approved evidence destinations.

Tests exercise the real ledger through the activities and validate Temporal sandbox
construction, with in-memory transports/sinks and orchestration stubs. This does not
prove a deployed Temporal server, real Kubernetes authorization or production sinks.
Continuous observer scheduling, actual sink adapters, admission credentials and
authenticated deployment proof remain activation prerequisites in T03.