informed-decision/deploy/lifecycle/RELEASE-INFD-WP-0010-T02.md
tegwick 83fd105465 INFD-WP-0010-T02: record the founder-run present-existing release (2026-11-01)
The founder ran the release:
- access-engine compact-sitting v14 (Helm revision 14) first, then Argo
  005709a: image ca75f0be, runtime ConfigMap b488b96a0953
- backup ed4469e7, held off-cluster
- health and readiness 200; private routes 404 on the public origin
- store counts unchanged, and an existing memo loaded under v14
- executor-identity probe 503 approval_unreadable; sso/default probe 403
  submitter_unknown

T02 stays open until the executor's present-existing memo is accepted at aal2 and
consumed (NK-WP-0056 pilot).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 5502@bnt-lap001
Assistant-Session: 68b3b02f-5d0f-4e71-8421-796f62e50196
2026-11-01 14:40:35 +01:00

17 KiB

Release runbook: present_existing for nk-bind-executor + compact-sitting v14 (INFD-WP-0010-T02)

Prepared 2026-11-01 by the informed-decision owner agent. Applied by the founder on 2026-11-01 (RELEASE_SHA 005709a, Helm revision 14). Results are in docs/evidence/2026-11-01-present-existing-release/release.json and in the deploy/lifecycle/README.md section "Present-existing release — 2026-11-01". Founder approval (2026-11-01, relayed by the net-kingdom coordinator): release present_existing together with access-engine compact-sitting v14. Every command below is founder-run, with export KUBECONFIG=$HOME/.kube/config-railiance01.

Status: image pinned (gate G0 closed 2026-11-01)

Owner CI (service-image.yaml, run 6, source c228583) published main-c228583, digest forgejo.coulomb.social/coulomb/informed-decision@sha256:ca75f0be03af0bd96a66b08d2949bdc2d16365a93ffabcd6593074b565e8f01b. The owner verified it read-only:

  • the label revision is c228583, and the user is 10001
  • apk info --exists 'zlib>=1.3.2-r1' passes; zlib is 1.3.2-r1 (runtime 1.3.2), libuuid is 2.42.3-r1, and pip is absent
  • the runtime smoke passes, and the installed modules equal the source at c228583
  • tools/smoke_container.py --workload passes 13/13
  • Trivy (the same pinned scanner as earlier releases, DB of 2026-11-01) finds 0 HIGH/CRITICAL. It reports 2 UNKNOWN: CVE-2026-107891 in musl/musl-utils 1.2.6-r2, fixed in 1.2.6-r3. The live f5588f49 image carries the same musl, so this is not a regression.

Evidence: docs/evidence/2026-11-01-present-existing-release/. RELEASE_SHA is the commit that pinned this digest: git -C ~/informed-decision log -1 --format=%H -- deploy/lifecycle/admission.candidate.json on main.

History: why the gate existed

No CI image exists for current main. This repository has no Forgejo workflow that builds the full service image. Its two workflows build single-purpose overlays on fixed base digests:

  • .forgejo/workflows/purpose-overlay-image.yaml swaps one module, policy.py, on base 7f5c6cb2…, and checks it against the pinned hash 9d7c1be4…. Commit 80ef840 changed policy.py (now 912ad3a3…), so run 5 (head e617bb5, 2026-11-01T03:07Z) failed at that check. That is the intended behaviour. Even if the hash were updated, the overlay could not carry this release: it touches nine modules (intake.py, intake_service.py, intake_approval.py, outcome.py (new), policy.py, requester.py, review.py, store.py, workload.py).
  • .forgejo/workflows/zlib-overlay-image.yaml upgrades zlib only, on base 763c3640….

The registry has no tag for 3b7031a, main or latest. docker buildx imagetools inspect returns not found for each.

Before this release can go out, an owner-CI full-image build from the release source must exist. It needs its own workflow, and it must keep zlib at 1.3.2-r1 or later. A plain rebuild of Containerfile uses the pinned python:3.12-alpine@sha256:b64631e0… base and only adds libuuid. That would reintroduce the zlib finding that the live image f5588f49… remediated. Then pin the image as in step 0.

Update, same day: the full-image workflow now exists in source: .forgejo/workflows/service-image.yaml. It builds Containerfile from the pushed commit's tarball, with the wheel built in-image. It installs the overlay's hash-pinned zlib-1.3.2-r1 package offline and enforces zlib>=1.3.2-r1. It checks the runtime smoke and that the installed source matches the commit before pushing forgejo.coulomb.social/coulomb/informed-decision:main-<short sha>, then prints the digest in "Report immutable digest". A local build of the same Containerfile passed 13/13 container checks (tools/smoke_container.py --workload), with zlib 1.3.2-r1 and the installed modules identical to source. The gate is now: the push runs this workflow, then step 0 pins the digest it reports.

Do not sync this candidate with the current image. The live image f5588f49… accepts only the four basic registry keys: set(row) != {submitter_id, memo_classes, review_groups, daily_limit}. It would reject the executor row, so every intake request would fail with 503 intake_configuration_unavailable, State Hub's included. It also does not send approval_action.

Pins

Current, live (rollback target) Candidate
Argo informed-decision targetRevision 312ab97e876e17341bf16f9e95397e14c2ac2720 RELEASE_SHA (the image-pin commit, see Status)
Service image …/informed-decision@sha256:f5588f496b1f267d07cdc9f5208e6f2f241f74b739c23924b03702dec13a5c23 …/informed-decision@sha256:ca75f0be03af0bd96a66b08d2949bdc2d16365a93ffabcd6593074b565e8f01b (main-c228583)
Runtime ConfigMap informed-decision-runtime-6c221878d051 informed-decision-runtime-b488b96a0953
Review policy pin compact-sitting v13 sha256:31be5985c0c1e6b0c3a039ed7310d1e94daf523a94f6d926e6e8d759d3711873 v14 sha256:8911458f36f1a60598aa19592a2d752a74dffe2617fd86238407ac87bc300fde
Helm flex-auth-informed-decision-sitting (access-engine) revision 13, image 6c6dc263…, v13 policy path access-engine's v14 release (2de0f86), prepared in parallel
Lifecycle policy pin informed-decision.lifecycle v2 da01e045… unchanged
Store schema 5 5 (no migration)

Candidate source changes in this commit:

  • admission.candidate.json: the policy pin moves to v14, and the workload peer {namespace: sso, app: bounded-executor} is added. service.candidate.json is re-rendered with tools/render_deployment.py.
  • informed-decision-review NetworkPolicy: the 8081 ingress rule from now lists exactly two peers: state-hub/app.kubernetes.io/name=state-hub (unchanged) and kubernetes.io/metadata.name=sso AND app.kubernetes.io/name=bounded-executor. No other sso pod is admitted, and nothing new is admitted on 8080.
  • intake-submitters.json (projected into registries.candidate.json): new row system:serviceaccount:sso:nk-bind-executor → nk-bind-executor, memo class bounded-execution, review group net-kingdom-admins, daily_limit: 10, present_existing {approval_actor: service:nk-bind-executor, actions: [bounded-execution]}. The state-hub row is byte-identical, and outcome_read stays off (T05 waits for STATE-WP-0101).

Step 0: pin the image (DONE 2026-11-01, owner agent; kept for reference)

cd ~/informed-decision
docker buildx imagetools inspect forgejo.coulomb.social/coulomb/informed-decision:main-<short sha>   # or the CI "Report immutable digest" line
# set "image" in deploy/lifecycle/admission.candidate.json to forgejo.coulomb.social/coulomb/informed-decision@<IMAGE_DIGEST>
python3 tools/render_deployment.py --input deploy/lifecycle/admission.candidate.json \
  --output deploy/lifecycle/service.candidate.json
.venv/bin/python -m pytest -q                       # expect 0 failures
python3 tools/smoke_container.py --image forgejo.coulomb.social/coulomb/informed-decision@<IMAGE_DIGEST> \
  --receipt docs/evidence/<date>-present-existing-release/container.json --workload
git commit -am "INFD-WP-0010-T02: pin service image <IMAGE_DIGEST> for the v14 release" && git push
# Founder, before step 1:
RELEASE_SHA=$(git -C ~/informed-decision log -1 --format=%H -- deploy/lifecycle/admission.candidate.json)

Only the image line and the re-rendered service.candidate.json change. The runtime ConfigMap name stays …b488b96a0953 because the image is not part of runtime.json.

Step 1: preflight (read-only)

export KUBECONFIG=$HOME/.kube/config-railiance01
kubectl -n argocd get application informed-decision \
  -o jsonpath='{.spec.source.targetRevision} {.status.sync.status} {.status.health.status}{"\n"}'
#   expect 312ab97e876e17341bf16f9e95397e14c2ac2720, Healthy (OutOfSync is the known orphan ConfigMap
#   informed-decision-runtime-c81577741a09; prune stays off)
kubectl -n informed-decision get deploy informed-decision -o jsonpath='{.spec.template.spec.containers[0].image}{"\n"}'
helm -n flex-auth history flex-auth-informed-decision-sitting --max 3     # expect revision 13 deployed
kubectl -n sso get sa nk-bind-executor
kubectl -n informed-decision exec deploy/informed-decision -- \
  informed-decision-admin inspect --db /data/private/review.sqlite        # record counts (schema 5)
# Server dry-run of the exact release source (no persistence):
git -C ~/informed-decision checkout --detach "$RELEASE_SHA"
kubectl apply --dry-run=server -k ~/informed-decision/deploy/lifecycle
kubectl diff -k ~/informed-decision/deploy/lifecycle    # expect: image, runtime ConfigMap name/new ConfigMap,
                                                       # registry ConfigMap, review NetworkPolicy (+ Argo annotations)
git -C ~/informed-decision checkout main

Step 2: backup (schema is unchanged, but the release convention still requires it)

STAMP=$(date -u +%Y%m%dT%H%M%SZ)
kubectl -n informed-decision exec deploy/informed-decision -- \
  informed-decision-admin backup --db /data/private/review.sqlite --output /data/private/review-$STAMP-pre-v14.sqlite
mkdir -p -m 0700 ~/informed-decision/.private-backups/$STAMP-pre-v14
kubectl -n informed-decision exec deploy/informed-decision -- cat /data/private/review-$STAMP-pre-v14.sqlite \
  > ~/informed-decision/.private-backups/$STAMP-pre-v14/review.sqlite
chmod 0600 ~/informed-decision/.private-backups/$STAMP-pre-v14/review.sqlite
sha256sum ~/informed-decision/.private-backups/$STAMP-pre-v14/review.sqlite

Step 3: one window, access-engine first, then informed-decision

The service checks the PDP's provenance on every review call: policy_package, policy_version, policy_package_digest and matched_policy_version must all equal its pin. Between the two steps, the version mismatch makes human review reads and acts fail closed. No review is lost, but pages refuse. Workload intake (POST /v1/memos) does not consult the PDP, so State Hub posing continues throughout. Keep the gap to minutes.

3a. access-engine v14 follows access-engine's own release runbook (FLEX-WP-0040-T07, access-engine/docs/compact-sitting-v14-present-existing.md, "Live steps"). It runs a Helm upgrade of flex-auth-informed-decision-sitting with --atomic: a new image built from 2de0f86 or later, and --policy /opt/flex-auth/examples/informed-decision-present-existing-review/policy.md. Check:

helm -n flex-auth history flex-auth-informed-decision-sitting --max 2      # expect revision 14 deployed
kubectl -n flex-auth rollout status deploy/flex-auth-informed-decision-sitting --timeout=180s

If the --atomic upgrade rolls itself back, stop here. Do not run 3b. Nothing on the informed-decision side has changed.

3b. informed-decision, through the manually pinned Argo application:

kubectl -n argocd patch application informed-decision --type merge \
  -p "{\"spec\":{\"source\":{\"targetRevision\":\"$RELEASE_SHA\"}}}"
kubectl -n argocd patch application informed-decision --type merge \
  -p "{\"operation\":{\"initiatedBy\":{\"username\":\"founder\"},\"sync\":{\"revision\":\"$RELEASE_SHA\",\"prune\":false,\"syncOptions\":[\"ApplyOutOfSyncOnly=true\"]}}}"
kubectl -n argocd get application informed-decision \
  -o jsonpath='{.status.operationState.phase} {.status.sync.revision} {.status.health.status}{"\n"}'
kubectl -n informed-decision rollout status deploy/informed-decision --timeout=300s   # Recreate: short restart

Step 4: verification

Possible without the executor:

  1. Pins live. Check the Deployment image and that its configuration volume names informed-decision-runtime-b488b96a0953:
    kubectl -n informed-decision get deploy informed-decision \
      -o jsonpath='{.spec.template.spec.containers[0].image} {.spec.template.spec.volumes[?(@.name=="configuration")].configMap.name}{"\n"}'
    kubectl -n informed-decision get networkpolicy informed-decision-review -o jsonpath='{.spec.ingress[1].from}{"\n"}'
    #   expect exactly the state-hub peer and the sso + app.kubernetes.io/name=bounded-executor peer
    
  2. Public surface. curl -s -o /dev/null -w '%{http_code}\n' https://decisions.coulomb.social/healthz and /readyz return 200. /v1/memos and /v1/decisions on the public origin return 404.
  3. v14 pin match (human, aal2). Open https://decisions.coulomb.social/ and one existing memo you were presented. The page loads only if the PDP's v14 provenance equals the new pin. A refusal mentioning policy or provenance means 3a and 3b disagree: roll back (step 5).
  4. Store unchanged. Run the informed-decision-admin inspect command from step 1 again. Schema 5, and the counts are no lower than the preflight.
  5. Executor path, synthetic and side-effect free. Run this from a temporary pod that carries the executor's identity and label. It is a founder-run kubectl write: a short-lived pod in sso, removed by --rm. The probe pins a deliberately wrong binding digest, so even a successful approval read ends in a 409. No memo is stored, no budget is spent, and nothing is created, entered or consumed in approval-engine.
    cd ~/informed-decision
    IMAGE=forgejo.coulomb.social/coulomb/informed-decision@sha256:ca75f0be03af0bd96a66b08d2949bdc2d16365a93ffabcd6593074b565e8f01b
    NAME=infd-probe-$(date -u +%H%M%S)   # --overrides (merge patch) replaces the generated container: it carries name and image
    OVR='{"spec":{"serviceAccountName":"nk-bind-executor","automountServiceAccountToken":false,
      "securityContext":{"runAsNonRoot":true,"runAsUser":10001,"seccompProfile":{"type":"RuntimeDefault"}},
      "containers":[{"name":"'$NAME'","image":"'$IMAGE'","command":["python","-"],"stdin":true,"stdinOnce":true,
        "securityContext":{"allowPrivilegeEscalation":false,"readOnlyRootFilesystem":true,"capabilities":{"drop":["ALL"]}},
        "volumeMounts":[{"name":"intake","mountPath":"/var/run/secrets/informed-decision-intake","readOnly":true}]}],
      "volumes":[{"name":"intake","projected":{"sources":[{"serviceAccountToken":{"path":"token",
        "audience":"informed-decision-intake","expirationSeconds":600}}]}}]}}'
    .venv/bin/python tools/present_existing_probe.py --approval-id "$(python3 -c 'import uuid;print(uuid.uuid4())')" --program \
    | kubectl -n sso run "$NAME" --rm -i --restart=Never --image="$IMAGE" \
        --labels=app.kubernetes.io/name=bounded-executor,app.kubernetes.io/component=release-probe --overrides="$OVR"
    
    Expected result for a random UUID: 503 {"error": "approval_unreadable"}. Getting there proves the NetworkPolicy admits the sso executor label on 8081, TokenReview accepts nk-bind-executor with audience informed-decision-intake, the reloaded registry parses (so the state-hub row is also loaded), the present_existing profile and envelope validation pass, and the reservation is released.
    • If you know an approval id in approval-engine that no informed-decision memo has presented yet (for example one created during NK-WP-0056 executor testing), use it in place of the random UUID. Expect a 409 with an approval_* reason (approval_digest_mismatch or an earlier one). That proves the read through KeyCape approval:read as well.
    • Negative check: rerun with "serviceAccountName":"default" and expect 403 submitter_unknown.
    • Other outcomes: 401 workload_authentication_failed points to the token or audience. A timeout points to the NetworkPolicy (check the executor egress policy too). 503 intake_configuration_unavailable means the registry did not parse: roll back.

Needs the executor (this is T02's own done-when): the executor creates an approval as service:nk-bind-executor and presents it, the founder accepts at aal2 on /review, and the executor consumes it. That is the NK-WP-0056 pilot run. It is not part of this release window.

Step 5: rollback to the current release

The database is retained and stays on schema 5, so this is not a downgrade. Do both halves in the same window:

kubectl -n argocd patch application informed-decision --type merge \
  -p '{"spec":{"source":{"targetRevision":"312ab97e876e17341bf16f9e95397e14c2ac2720"}}}'
kubectl -n argocd patch application informed-decision --type merge \
  -p '{"operation":{"initiatedBy":{"username":"founder"},"sync":{"revision":"312ab97e876e17341bf16f9e95397e14c2ac2720","prune":false,"syncOptions":["ApplyOutOfSyncOnly=true"]}}}'
kubectl -n informed-decision rollout status deploy/informed-decision --timeout=300s
helm -n flex-auth rollback flex-auth-informed-decision-sitting 13 --wait     # access-engine's v13 release

This restores image f5588f49…, runtime ConfigMap …6c221878d051 (v13 pin), the registry without the executor row, and the NetworkPolicy without the sso peer. The new immutable ConfigMap …b488b96a0953 is left in place because prune is off. It is harmless. Any present-existing memo posed during the window stays in the store but cannot be reviewed under v13 (unknown_submitter): it fails closed. Record it, and do not delete it.

Step 6: record

Write docs/evidence/<date>-present-existing-release/ (pins, Argo revision, Helm revision, backup sha256, probe outputs without tokens) and append a "Present-existing release" section to deploy/lifecycle/README.md. Then update INFD-WP-0010-T02.