The founder ran the release:
- access-engine compact-sitting v14 (Helm revision 14) first, then Argo
005709a: image ca75f0be, runtime ConfigMap b488b96a0953
- backup ed4469e7, held off-cluster
- health and readiness 200; private routes 404 on the public origin
- store counts unchanged, and an existing memo loaded under v14
- executor-identity probe 503 approval_unreadable; sso/default probe 403
submitter_unknown
T02 stays open until the executor's present-existing memo is accepted at aal2 and
consumed (NK-WP-0056 pilot).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 5502@bnt-lap001
Assistant-Session: 68b3b02f-5d0f-4e71-8421-796f62e50196
17 KiB
Release runbook: present_existing for nk-bind-executor + compact-sitting v14 (INFD-WP-0010-T02)
Prepared 2026-11-01 by the informed-decision owner agent. Applied by the founder on 2026-11-01 (RELEASE_SHA
005709a, Helm revision 14). Results are in docs/evidence/2026-11-01-present-existing-release/release.json and in
the deploy/lifecycle/README.md section "Present-existing release — 2026-11-01".
Founder approval (2026-11-01, relayed by the net-kingdom coordinator): release present_existing together with
access-engine compact-sitting v14. Every command below is founder-run, with
export KUBECONFIG=$HOME/.kube/config-railiance01.
Status: image pinned (gate G0 closed 2026-11-01)
Owner CI (service-image.yaml, run 6, source c228583) published main-c228583, digest
forgejo.coulomb.social/coulomb/informed-decision@sha256:ca75f0be03af0bd96a66b08d2949bdc2d16365a93ffabcd6593074b565e8f01b. The owner verified it read-only:
- the label revision is
c228583, and the user is 10001 apk info --exists 'zlib>=1.3.2-r1'passes; zlib is 1.3.2-r1 (runtime 1.3.2), libuuid is 2.42.3-r1, and pip is absent- the runtime smoke passes, and the installed modules equal the source at
c228583 tools/smoke_container.py --workloadpasses 13/13- Trivy (the same pinned scanner as earlier releases, DB of 2026-11-01) finds 0 HIGH/CRITICAL. It reports
2 UNKNOWN: CVE-2026-107891 in musl/musl-utils 1.2.6-r2, fixed in 1.2.6-r3. The live
f5588f49image carries the same musl, so this is not a regression.
Evidence: docs/evidence/2026-11-01-present-existing-release/. RELEASE_SHA is the commit that pinned this digest:
git -C ~/informed-decision log -1 --format=%H -- deploy/lifecycle/admission.candidate.json on main.
History: why the gate existed
No CI image exists for current main. This repository has no Forgejo workflow that builds the full service
image. Its two workflows build single-purpose overlays on fixed base digests:
.forgejo/workflows/purpose-overlay-image.yamlswaps one module,policy.py, on base7f5c6cb2…, and checks it against the pinned hash9d7c1be4…. Commit80ef840changedpolicy.py(now912ad3a3…), so run 5 (heade617bb5, 2026-11-01T03:07Z) failed at that check. That is the intended behaviour. Even if the hash were updated, the overlay could not carry this release: it touches nine modules (intake.py,intake_service.py,intake_approval.py,outcome.py(new),policy.py,requester.py,review.py,store.py,workload.py)..forgejo/workflows/zlib-overlay-image.yamlupgrades zlib only, on base763c3640….
The registry has no tag for 3b7031a, main or latest. docker buildx imagetools inspect returns
not found for each.
Before this release can go out, an owner-CI full-image build from the release source must exist. It needs
its own workflow, and it must keep zlib at 1.3.2-r1 or later. A plain rebuild of Containerfile uses the
pinned python:3.12-alpine@sha256:b64631e0… base and only adds libuuid. That would reintroduce the zlib
finding that the live image f5588f49… remediated. Then pin the image as in step 0.
Update, same day: the full-image workflow now exists in source: .forgejo/workflows/service-image.yaml. It
builds Containerfile from the pushed commit's tarball, with the wheel built in-image. It installs the overlay's
hash-pinned zlib-1.3.2-r1 package offline and enforces zlib>=1.3.2-r1. It checks the runtime smoke and that the
installed source matches the commit before pushing forgejo.coulomb.social/coulomb/informed-decision:main-<short sha>,
then prints the digest in "Report immutable digest". A local build of the same Containerfile passed 13/13 container
checks (tools/smoke_container.py --workload), with zlib 1.3.2-r1 and the installed modules identical to source.
The gate is now: the push runs this workflow, then step 0 pins the digest it reports.
Do not sync this candidate with the current image. The live image f5588f49… accepts only the four basic
registry keys: set(row) != {submitter_id, memo_classes, review_groups, daily_limit}. It would reject the
executor row, so every intake request would fail with 503 intake_configuration_unavailable, State Hub's
included. It also does not send approval_action.
Pins
| Current, live (rollback target) | Candidate | |
|---|---|---|
Argo informed-decision targetRevision |
312ab97e876e17341bf16f9e95397e14c2ac2720 |
RELEASE_SHA (the image-pin commit, see Status) |
| Service image | …/informed-decision@sha256:f5588f496b1f267d07cdc9f5208e6f2f241f74b739c23924b03702dec13a5c23 |
…/informed-decision@sha256:ca75f0be03af0bd96a66b08d2949bdc2d16365a93ffabcd6593074b565e8f01b (main-c228583) |
| Runtime ConfigMap | informed-decision-runtime-6c221878d051 |
informed-decision-runtime-b488b96a0953 |
| Review policy pin | compact-sitting v13 sha256:31be5985c0c1e6b0c3a039ed7310d1e94daf523a94f6d926e6e8d759d3711873 |
v14 sha256:8911458f36f1a60598aa19592a2d752a74dffe2617fd86238407ac87bc300fde |
Helm flex-auth-informed-decision-sitting (access-engine) |
revision 13, image 6c6dc263…, v13 policy path |
access-engine's v14 release (2de0f86), prepared in parallel |
| Lifecycle policy pin | informed-decision.lifecycle v2 da01e045… |
unchanged |
| Store schema | 5 | 5 (no migration) |
Candidate source changes in this commit:
admission.candidate.json: the policy pin moves to v14, and the workload peer{namespace: sso, app: bounded-executor}is added.service.candidate.jsonis re-rendered withtools/render_deployment.py.informed-decision-reviewNetworkPolicy: the 8081 ingress rulefromnow lists exactly two peers:state-hub/app.kubernetes.io/name=state-hub(unchanged) andkubernetes.io/metadata.name=ssoANDapp.kubernetes.io/name=bounded-executor. No other sso pod is admitted, and nothing new is admitted on 8080.intake-submitters.json(projected intoregistries.candidate.json): new rowsystem:serviceaccount:sso:nk-bind-executor→nk-bind-executor, memo classbounded-execution, review groupnet-kingdom-admins,daily_limit: 10,present_existing {approval_actor: service:nk-bind-executor, actions: [bounded-execution]}. The state-hub row is byte-identical, andoutcome_readstays off (T05 waits for STATE-WP-0101).
Step 0: pin the image (DONE 2026-11-01, owner agent; kept for reference)
cd ~/informed-decision
docker buildx imagetools inspect forgejo.coulomb.social/coulomb/informed-decision:main-<short sha> # or the CI "Report immutable digest" line
# set "image" in deploy/lifecycle/admission.candidate.json to forgejo.coulomb.social/coulomb/informed-decision@<IMAGE_DIGEST>
python3 tools/render_deployment.py --input deploy/lifecycle/admission.candidate.json \
--output deploy/lifecycle/service.candidate.json
.venv/bin/python -m pytest -q # expect 0 failures
python3 tools/smoke_container.py --image forgejo.coulomb.social/coulomb/informed-decision@<IMAGE_DIGEST> \
--receipt docs/evidence/<date>-present-existing-release/container.json --workload
git commit -am "INFD-WP-0010-T02: pin service image <IMAGE_DIGEST> for the v14 release" && git push
# Founder, before step 1:
RELEASE_SHA=$(git -C ~/informed-decision log -1 --format=%H -- deploy/lifecycle/admission.candidate.json)
Only the image line and the re-rendered service.candidate.json change. The runtime ConfigMap name stays
…b488b96a0953 because the image is not part of runtime.json.
Step 1: preflight (read-only)
export KUBECONFIG=$HOME/.kube/config-railiance01
kubectl -n argocd get application informed-decision \
-o jsonpath='{.spec.source.targetRevision} {.status.sync.status} {.status.health.status}{"\n"}'
# expect 312ab97e876e17341bf16f9e95397e14c2ac2720, Healthy (OutOfSync is the known orphan ConfigMap
# informed-decision-runtime-c81577741a09; prune stays off)
kubectl -n informed-decision get deploy informed-decision -o jsonpath='{.spec.template.spec.containers[0].image}{"\n"}'
helm -n flex-auth history flex-auth-informed-decision-sitting --max 3 # expect revision 13 deployed
kubectl -n sso get sa nk-bind-executor
kubectl -n informed-decision exec deploy/informed-decision -- \
informed-decision-admin inspect --db /data/private/review.sqlite # record counts (schema 5)
# Server dry-run of the exact release source (no persistence):
git -C ~/informed-decision checkout --detach "$RELEASE_SHA"
kubectl apply --dry-run=server -k ~/informed-decision/deploy/lifecycle
kubectl diff -k ~/informed-decision/deploy/lifecycle # expect: image, runtime ConfigMap name/new ConfigMap,
# registry ConfigMap, review NetworkPolicy (+ Argo annotations)
git -C ~/informed-decision checkout main
Step 2: backup (schema is unchanged, but the release convention still requires it)
STAMP=$(date -u +%Y%m%dT%H%M%SZ)
kubectl -n informed-decision exec deploy/informed-decision -- \
informed-decision-admin backup --db /data/private/review.sqlite --output /data/private/review-$STAMP-pre-v14.sqlite
mkdir -p -m 0700 ~/informed-decision/.private-backups/$STAMP-pre-v14
kubectl -n informed-decision exec deploy/informed-decision -- cat /data/private/review-$STAMP-pre-v14.sqlite \
> ~/informed-decision/.private-backups/$STAMP-pre-v14/review.sqlite
chmod 0600 ~/informed-decision/.private-backups/$STAMP-pre-v14/review.sqlite
sha256sum ~/informed-decision/.private-backups/$STAMP-pre-v14/review.sqlite
Step 3: one window, access-engine first, then informed-decision
The service checks the PDP's provenance on every review call: policy_package, policy_version,
policy_package_digest and matched_policy_version must all equal its pin. Between the two steps, the
version mismatch makes human review reads and acts fail closed. No review is lost, but pages refuse.
Workload intake (POST /v1/memos) does not consult the PDP, so State Hub posing continues throughout. Keep
the gap to minutes.
3a. access-engine v14 follows access-engine's own release runbook (FLEX-WP-0040-T07,
access-engine/docs/compact-sitting-v14-present-existing.md, "Live steps"). It runs a Helm upgrade of
flex-auth-informed-decision-sitting with --atomic: a new image built from 2de0f86 or later, and
--policy /opt/flex-auth/examples/informed-decision-present-existing-review/policy.md. Check:
helm -n flex-auth history flex-auth-informed-decision-sitting --max 2 # expect revision 14 deployed
kubectl -n flex-auth rollout status deploy/flex-auth-informed-decision-sitting --timeout=180s
If the --atomic upgrade rolls itself back, stop here. Do not run 3b. Nothing on the informed-decision side has changed.
3b. informed-decision, through the manually pinned Argo application:
kubectl -n argocd patch application informed-decision --type merge \
-p "{\"spec\":{\"source\":{\"targetRevision\":\"$RELEASE_SHA\"}}}"
kubectl -n argocd patch application informed-decision --type merge \
-p "{\"operation\":{\"initiatedBy\":{\"username\":\"founder\"},\"sync\":{\"revision\":\"$RELEASE_SHA\",\"prune\":false,\"syncOptions\":[\"ApplyOutOfSyncOnly=true\"]}}}"
kubectl -n argocd get application informed-decision \
-o jsonpath='{.status.operationState.phase} {.status.sync.revision} {.status.health.status}{"\n"}'
kubectl -n informed-decision rollout status deploy/informed-decision --timeout=300s # Recreate: short restart
Step 4: verification
Possible without the executor:
- Pins live. Check the Deployment image and that its
configurationvolume namesinformed-decision-runtime-b488b96a0953:kubectl -n informed-decision get deploy informed-decision \ -o jsonpath='{.spec.template.spec.containers[0].image} {.spec.template.spec.volumes[?(@.name=="configuration")].configMap.name}{"\n"}' kubectl -n informed-decision get networkpolicy informed-decision-review -o jsonpath='{.spec.ingress[1].from}{"\n"}' # expect exactly the state-hub peer and the sso + app.kubernetes.io/name=bounded-executor peer - Public surface.
curl -s -o /dev/null -w '%{http_code}\n' https://decisions.coulomb.social/healthzand/readyzreturn 200./v1/memosand/v1/decisionson the public origin return 404. - v14 pin match (human, aal2). Open
https://decisions.coulomb.social/and one existing memo you were presented. The page loads only if the PDP's v14 provenance equals the new pin. A refusal mentioning policy or provenance means 3a and 3b disagree: roll back (step 5). - Store unchanged. Run the
informed-decision-admin inspectcommand from step 1 again. Schema 5, and the counts are no lower than the preflight. - Executor path, synthetic and side-effect free. Run this from a temporary pod that carries the
executor's identity and label. It is a founder-run kubectl write: a short-lived pod in sso, removed by
--rm. The probe pins a deliberately wrong binding digest, so even a successful approval read ends in a 409. No memo is stored, no budget is spent, and nothing is created, entered or consumed in approval-engine.
Expected result for a random UUID:cd ~/informed-decision IMAGE=forgejo.coulomb.social/coulomb/informed-decision@sha256:ca75f0be03af0bd96a66b08d2949bdc2d16365a93ffabcd6593074b565e8f01b NAME=infd-probe-$(date -u +%H%M%S) # --overrides (merge patch) replaces the generated container: it carries name and image OVR='{"spec":{"serviceAccountName":"nk-bind-executor","automountServiceAccountToken":false, "securityContext":{"runAsNonRoot":true,"runAsUser":10001,"seccompProfile":{"type":"RuntimeDefault"}}, "containers":[{"name":"'$NAME'","image":"'$IMAGE'","command":["python","-"],"stdin":true,"stdinOnce":true, "securityContext":{"allowPrivilegeEscalation":false,"readOnlyRootFilesystem":true,"capabilities":{"drop":["ALL"]}}, "volumeMounts":[{"name":"intake","mountPath":"/var/run/secrets/informed-decision-intake","readOnly":true}]}], "volumes":[{"name":"intake","projected":{"sources":[{"serviceAccountToken":{"path":"token", "audience":"informed-decision-intake","expirationSeconds":600}}]}}]}}' .venv/bin/python tools/present_existing_probe.py --approval-id "$(python3 -c 'import uuid;print(uuid.uuid4())')" --program \ | kubectl -n sso run "$NAME" --rm -i --restart=Never --image="$IMAGE" \ --labels=app.kubernetes.io/name=bounded-executor,app.kubernetes.io/component=release-probe --overrides="$OVR"503 {"error": "approval_unreadable"}. Getting there proves the NetworkPolicy admits the sso executor label on 8081, TokenReview acceptsnk-bind-executorwith audienceinformed-decision-intake, the reloaded registry parses (so the state-hub row is also loaded), thepresent_existingprofile and envelope validation pass, and the reservation is released.- If you know an approval id in approval-engine that no informed-decision memo has presented yet (for example
one created during NK-WP-0056 executor testing), use it in place of the random UUID. Expect a 409 with an
approval_*reason (approval_digest_mismatchor an earlier one). That proves the read through KeyCapeapproval:readas well. - Negative check: rerun with
"serviceAccountName":"default"and expect403 submitter_unknown. - Other outcomes:
401 workload_authentication_failedpoints to the token or audience. A timeout points to the NetworkPolicy (check the executor egress policy too).503 intake_configuration_unavailablemeans the registry did not parse: roll back.
- If you know an approval id in approval-engine that no informed-decision memo has presented yet (for example
one created during NK-WP-0056 executor testing), use it in place of the random UUID. Expect a 409 with an
Needs the executor (this is T02's own done-when): the executor creates an approval as service:nk-bind-executor and
presents it, the founder accepts at aal2 on /review, and the executor consumes it. That is the
NK-WP-0056 pilot run. It is not part of this release window.
Step 5: rollback to the current release
The database is retained and stays on schema 5, so this is not a downgrade. Do both halves in the same window:
kubectl -n argocd patch application informed-decision --type merge \
-p '{"spec":{"source":{"targetRevision":"312ab97e876e17341bf16f9e95397e14c2ac2720"}}}'
kubectl -n argocd patch application informed-decision --type merge \
-p '{"operation":{"initiatedBy":{"username":"founder"},"sync":{"revision":"312ab97e876e17341bf16f9e95397e14c2ac2720","prune":false,"syncOptions":["ApplyOutOfSyncOnly=true"]}}}'
kubectl -n informed-decision rollout status deploy/informed-decision --timeout=300s
helm -n flex-auth rollback flex-auth-informed-decision-sitting 13 --wait # access-engine's v13 release
This restores image f5588f49…, runtime ConfigMap …6c221878d051 (v13 pin), the registry without the executor
row, and the NetworkPolicy without the sso peer. The new immutable ConfigMap …b488b96a0953 is left in place
because prune is off. It is harmless. Any present-existing memo posed during the window stays in the store but
cannot be reviewed under v13 (unknown_submitter): it fails closed. Record it, and do not delete it.
Step 6: record
Write docs/evidence/<date>-present-existing-release/ (pins, Argo revision, Helm revision, backup sha256,
probe outputs without tokens) and append a "Present-existing release" section to deploy/lifecycle/README.md.
Then update INFD-WP-0010-T02.