Advance supervised agent records and close verified Secret annotation guard
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 3s
Python Tests / pytest (push) Successful in 25s

This commit is contained in:
codex 2026-09-28 18:15:27 +02:00
parent b2f6713721
commit db91818e84
44 changed files with 6868 additions and 54 deletions

View file

@ -0,0 +1,157 @@
# Agent-specific supervised and autopilot modes
Founder decision, 2026-09-28, `GOVERN @ estate`.
Recorded under CUST-WP-0073-T01. This supersedes the proposed permanent choice
between observation-only agents and unrestricted deployment access.
## Accepted direction
Autonomy is a characteristic of an identified agent, scoped to the work it is
trusted to perform. Start agents in **supervised-mode**. Record how often the
supervisor accepts privileged-action proposals unchanged and how well those
exact proposals work when executed. Only a demonstrated record of successful
operation without adaptation or refinement supports promotion to
**autopilot-mode**. Autopilot has both a cost budget and a risk limit in EUR,
which the supervisor can tune per agent and scope.
This decision accepts the model. It does not promote an agent, choose numerical
limits, authorize a live credential cutover or claim runtime enforcement exists.
## Minimal operating contract
An agent instance/assignment carries its stable identity, mode, supervisor,
allowed action/target scope, execution-profile revision and reference to its
promotion/limit decision. Mode persists across sessions. Trust in one scope does
not automatically grant trust in another. New scopes start supervised; material
model, tool-profile or execution-environment changes require a supervisor review
of whether prior evidence still applies.
In supervised-mode, ordinary already-authorized preparation, reads, local edits
and tests continue. For a privileged action the agent prepares the exact action,
target, expected result, verification/rollback, cost estimate and risk estimate.
The supervisor approves that proposal or performs it through the privileged
execution path. The receipt records whether approval or human execution was
needed. Approval is bound to the reviewed proposal revision; changing that
revision requires review again. The agent cannot approve itself.
In autopilot-mode, a privileged action may run without per-action approval only
inside the agent's granted scope and both monetary limits. Missing estimates,
expired grants, insufficient remaining budget, unbounded risk, or actions outside
scope return that action to supervision. Existing human-only lanes remain
human-only unless explicitly changed by the governing authority. Mode and work
record `lane` are separate dimensions; effective authority is their intersection.
## Evidence for promotion
Use existing approval and execution receipts, keyed by agent, scope, profile
revision and a stable proposal ID/digest. Record:
- the original proposal, supervisor disposition (unchanged / revised /
rejected), revisions and reason; pending/withdrawn proposals stay visible;
- approval and execution identities, timestamps and the exact executed revision;
- outcome verification, interventions, rollback/recovery, observed cost and
realized loss/incident evidence. Store references, not credentials.
For a declared review window, report counts as well as rates:
- **Unchanged acceptance rate:** proposals accepted without changes divided by
all adjudicated original proposals. Revised and rejected proposals remain in
the denominator; retries do not become fresh successes.
- **Unchanged execution success rate:** original proposals executed as accepted,
passing the agreed verification with no corrective refinement or rescue,
divided by all executed original proposals with completed verification.
Execution of a supervisor-revised proposal does not earn an unchanged success.
- Also report pending/unverified outcomes and supervisor interventions/time.
Approval without execution is not a successful outcome. No observations means
unknown, never 100%.
The supervisor promotes explicitly for a named scope using an agreed minimum
sample, review window, acceptance/success thresholds and incident tolerance.
These values are not set by this decision; no default percentage grants access.
Successful harmless work alone does not establish competence for higher-risk
privileged actions. Promotion, revocation and limit changes retain history.
The supervisor can reduce autonomy immediately; failed verification or missing
enforcement stops further autonomous privileged actions pending review.
## Two distinct EUR controls
**Cost budget** bounds attributable spend and commitments over an explicit period,
including execution costs and resources/services the action commits to. Reserve
the estimated maximum cost before execution, reconcile actuals afterwards, and
count concurrent reservations against the same remaining budget. Unknown cost is
not free. Record the budget source, currency and reset period. Do not confuse a
token limit or provider subscription with a complete euro-denominated budget.
**Risk limit** bounds potential loss, separately from normal spending. Proposed
operational interpretation for each grant: a conservatively assessed credible
loss bound per action plus aggregate outstanding exposure from concurrent or
dependent actions. Include recovery expense, service interruption and data loss
where applicable, with assumptions and uncertainty. An expected-loss average
alone must not hide a much larger credible downside. An unpriced or unbounded
consequence requires supervision. EUR limits do not price away human-only or
other non-monetary prohibitions. The supervisor accepts the valuation method
and the action/exposure limits when granting autopilot; the agent cannot raise
its own limit or declare its own estimate authoritative.
Before enabling autopilot, verify that the execution path enforces reservations,
limits and revocation across concurrent actions. Recording fields in a manifest
does not enforce a budget. Until that proof exists, the mode remains supervised.
## Credential boundary and existing owners
Keep admin credentials out of the agent's direct reach in both modes. A scoped
privileged execution path performs approved actions in supervised-mode and
policy-authorized actions in autopilot-mode, returning sanitized outcome evidence.
Unrestricted sudo, admin kubeconfig access or unreviewed GitOps deployment would
bypass that path. Separate identities/isolation remain necessary, but the
observation-only profile is the supervised starting profile, not a permanent
limit on what an agent may accomplish.
Reuse existing boundaries rather than build another supervisor service:
| Concern | Existing surface / limit |
|---|---|
| Agent identity, mode and performance | Consumer-owned agent instance/assignment records; `agentic-resources` performance loop. Its broader workforce inventory/assignment contracts are still proposed. |
| Exact human approval and execution outcome | `approval-engine` approval object, `informed-decision` supervisor presentation, execution receipts. State Hub decisions record governance; they are not runtime approval tokens. |
| Runtime enforcement and credential custody | `glas-harness`, selected rein and sandbox; existing authorization and credential-owner paths. A prompt or mode label grants nothing. |
| Monetary authority | `fin-hub` budget source, with execution accounting supplied by the relevant runtime/service. |
| Risk judgement | Supervisor-approved estimates; `risk-nexus` can hold evidence but is not a live EUR risk gate. |
| Work and review evidence | Existing CUST-WP-0073 tasks and State Hub progress; no new task/workplan or service. |
## Bounded application to CUST-WP-0073
T01's policy choice is resolved by this decision. T02 implements and proves the
supervised starting boundary and records the existing execution path selected;
T04 adds per-agent mode and proposal/outcome evidence to guidance and the chosen
existing receipts. First implementation may use reviewed file records and
existing receipts. No new dashboard, automatic promotion engine, monetary risk
estimator or general workforce system is required to finish credential separation.
Autopilot is a promotion option requiring its own concrete grant and demonstrated
enforcement, not an activation promised by this workplan. No such grant exists
from this decision. T03's annotation policy and T05's deferred rotation remain
unchanged. CUST-WP-0071 retains its measurement and weekly-review scope.
## Supervised record in use
The initial consumer-owned record is
`.kaizen/agents/custodian-codex/supervision.json`. Generate its descriptive report:
```bash
python3 scripts/summarize_agent_supervision.py .kaizen/agents/custodian-codex/supervision.json
```
For each future scored proposal, retain the original digest before submission,
the supervisor's approval reference and approved digest, then the executed digest
and verification receipt. Use one stable proposal ID across revisions. Record
`refinement_or_rescue` explicitly. Accepted revised proposals, rejections and
rescued executions cannot earn unchanged success. Pending/unverified outcomes
remain visible. These records contain references and outcomes, never credential
values or executable approval tokens.
Earlier conversation authorization is retained as `unscored` where an exact
submitted revision was not recorded. It is not backfilled into promotion evidence.
The initial rates are unknown. The utility is descriptive and grants no authority;
autopilot remains disabled and the interactive runtime has not been migrated.
The existing sand-boxer isolation mechanism has a separate non-model proof in
`docs/evidence/2026-09-28-supervised-sandbox-proof.json`.

View file

@ -1,7 +1,7 @@
# Agent environment orientation
**Audience:** every coding agent working in this estate (Claude Code, Codex, Grok, custodian workers). It is tool-neutral.
**Owner:** the-custodian. **Last verified:** 2026-09-24.
**Owner:** the-custodian. **Last verified:** 2026-09-28 (§6); other sections retain their dated evidence.
These are facts about the *environment*: where things run, how to reach them, and the traps that cost real time. Each section names its owner. When a fact changes, fix it here and in the owner's record.
@ -76,9 +76,24 @@ Owner: railiance-platform (OpenBao) and ops-warden (the `warden access` lane).
## 6. Secrets and credentials
- Run the credential-routing check (`warden route find "<need>"`) before requesting anything. See the "Credential and access routing" section in every repo's `AGENTS.md`.
- **Never read a Secret with `-o yaml`, `-o json`, `describe`-style tools, or even `-o jsonpath='{.metadata}'`.** A Secret created with `kubectl apply` carries its full data in the `kubectl.kubernetes.io/last-applied-configuration` annotation, so a metadata read prints the secret. To test for the annotation without printing a value:
`kubectl get secret <n> -o go-template='{{ if index .metadata.annotations "kubectl.kubernetes.io/last-applied-configuration" }}HAS-ANNOTATION{{ else }}clean{{ end }}'`
- **Do not run any other go-template or jsonpath against a Secret.** On 2026-09-23 a template that only asked for `len .metadata.ownerReferences` failed because the field was absent, and kubectl printed the raw object, including `.data` and the last-applied annotation. A metadata-only template is not safe on that basis. The presence check above is the only template to use.
- **Never print Secret objects, annotations, or raw kubectl error output.** A
client-side apply annotation can contain a second copy of every value, and a
failing template can dump the object. Metadata-only intent does not make a
command safe.
- **Use the output-suppressing maintenance helper for annotation checks:**
`railiance-platform/scripts/secret_annotation_maintenance.py` (no arguments
inspects; `--clean` is the supervised mutation). It uses only validated
namespace/name fields and a presence template tested for absent, empty and
populated annotation maps and empty annotation values. It captures and discards
raw kubectl output/errors; receipts contain only names, counts and booleans.
- **The September 24 inline presence template is withdrawn.** On September 28,
`index .metadata.annotations` failed on an absent map. The wrapper suppressed
the resulting object dump; no values reached the agent transcript. Do not run
standalone go-template/jsonpath against real Secrets, including the old check.
- Agent autonomy is per identified agent and scope: start supervised, record
exact proposals and verified outcomes, promote explicitly only under bounded
EUR cost and risk grants. See `docs/agent-autonomy-decision.md`. A mode label or
approval rate does not isolate credentials or grant runtime permissions.
- ExternalSecrets use ClusterSecretStores. The target pattern is **OpenBao Kubernetes auth**: one ServiceAccount per consumer, 15-minute tokens, and a policy scoped to exact paths. Five stores still use static `*-eso-token` Secrets, which expire. Do not re-run the old `*-eso-token-apply` scripts.
## 7. GitOps (ArgoCD) on railiance01

View file

@ -0,0 +1,125 @@
# Credential separation — reviewed change package
2026-09-28. Prepared under the existing CUST-WP-0073 tasks. The initial admission rollout was rolled back, then corrected and re-enabled; see the
[rollout receipt](../../evidence/2026-09-28-secret-annotation-rollout.md). The founder selected agent-specific supervised/autopilot modes in
[the decision record](../../agent-autonomy-decision.md). The concrete supervised
execution path and account cutover remain unimplemented.
## Verified current state
`ssh railiance01` runs as `tegwick` (uid 1000), with `(ALL) NOPASSWD: ALL`.
`/etc/rancher/k3s/k3s.yaml` is `644 root root`.
`kubectl auth whoami` reports `system:admin`, `system:masters`.
No credential contents were read. The public principal inventory in
`railiance-infra/ansible/inventory/ssh_principals.yaml` maps both `agt-*` and
`adm-full` to `tegwick`. ops-warden issues certificates; railiance-infra owns
host principal mapping. A different principal on this same account does not
separate privilege.
## Supervised starting identity and later scoped promotion (T01/T02)
Use separate agent and admin OS identities on both the workstation and server.
Agents must not inherit the admin SSH key/certificate, SSH agent socket, sudo,
container-runtime socket, kubeconfig, OpenBao token or admin home directory.
Moving a file or changing the default context under the same unrestricted
account is insufficient. Keep an independently verified attended admin session
open during cutover; verify recovery before withdrawing the old agent path.
The supervised starting Kubernetes identity is outside `system:masters`, with an
explicit reviewed observation allowlist. Do not simply bind built-in `view`:
ConfigMaps, pod specs and logs can themselves contain credentials. Do not grant
workload edits, arbitrary ConfigMap writes, exec/attach/portforward, proxy,
logs, Secret verbs, token issuance, impersonation, RBAC writes or CSR approval.
Broader action authority is a per-agent autopilot grant earned through evidence,
with cost and risk limits in EUR; it is not unrestricted credential access.
Deployment create/patch authority allows code or mounts to extract credentials.
This is an upstream documented escalation route, not a missing deny rule:
[Kubernetes RBAC good practices](https://kubernetes.io/docs/concepts/security/rbac-good-practices/).
Agents prepare exact changes; in supervised-mode the supervisor approves or runs
them through the privileged path. An agent-controlled GitOps write path must obey
that same gate. Later autopilot may execute scoped actions without individual
approval only through verified policy and budget/risk enforcement. Otherwise it
recreates the bypass. Mode, supervisor, proposal dispositions and verified
outcomes belong to the identified agent's existing records and receipts.
Set k3s's persistent kubeconfig mode to `600` in railiance-enablement and fix
the existing file, but do not mistake that step for OS-account separation.
The final selected launcher/account setup must prove agents cannot regain the
old admin account, including through workstation/Windows interoperability.
No new credential broker or harness implementation is proposed.
Acceptance uses the actual agent process/account, not only admin impersonation:
whoami without masters; denied Secret get/list/watch; denied pod exec, workload
writes, logs, token issuance and impersonation; denied admin-file reads and
sudo; no accessible admin socket/key/token; approved observation succeeds;
attended admin recovery succeeds. Record authorization booleans and file access
results, never credential contents. Negative checks must not attempt to print
an actual secret if access unexpectedly succeeds.
## Secret annotation policy (T03)
`reject-secret-last-applied.yaml` contains a native v1 policy and Deny binding.
It rejects the annotation key even when its value is empty, on CREATE/UPDATE,
cluster-wide. It introduces no controller or workload. Server dry-run on the
verified v1.35.1+k3s1 cluster accepted both objects on September 28:
```text
validatingadmissionpolicy.admissionregistration.k8s.io/reject-secret-last-applied serverside-applied (server dry run)
validatingadmissionpolicybinding.admissionregistration.k8s.io/reject-secret-last-applied serverside-applied (server dry run)
```
Subsequent live native tests passed, but actual ESO refresh failed and required
rollback. Enforcement is now enabled after explicit metadata fixes and successful fresh
refreshes of all 39 ExternalSecrets. The platform owner's pinned revision
`7daf7e9` is authoritative; the original proposal file is retained for history.
Reference: [ValidatingAdmissionPolicy](https://kubernetes.io/docs/reference/access-authn-authz/validating-admission-policy/).
Before any retry, fix and verify the 31 ESO declarations described in the rollout
receipt, then in one attended railiance-platform window: remove existing last-applied
annotations without logging values; persist the manifest in that owner's
deployment path; dry-run and diff; install policy and binding. Use only a
synthetic, non-credential Secret in a scratch namespace to verify clean create
and update succeed, annotated create/update (including empty annotation) fail,
and client-side apply fails. Verify the policy type-check status, then remove
the fixture. Record only names, booleans and counts. Do not use dry-run output
of real Secret objects or print admission errors containing real values.
If the policy interrupts a required write, the attended admin can delete its
binding, fix the writer to use server-side apply/replace, and rebind. This
temporarily reopens annotation leakage and must be recorded. The policy does
not revoke any credential or stop other ways of reading secrets.
## Guidance and deferred rotation (T04/T05)
The September 24 standing notice exists. The raw presence template is now
withdrawn: absent annotations can trigger a dump of the full Secret. Use only
the maintenance helper, which captures and suppresses kubectl output on errors. Claude's recorded guard remains a
stopgap. No equivalent Codex/Grok read-denial hook has been established by this
work; this session has broad kubectl/SSH permissions, so instructions are the
current protection. No claim is made that every Grok installation was inspected.
OpenBao/Vault secret reads belong inside the same attended boundary, regardless
of preapproved command prefixes.
Rotation remains in existing T05 with the founder's September 24 trigger-based
deferral. Do not silently cancel it or open another plan. At final closure,
either execute the rotation in its attended window or explicitly resolve its
existing disposition under the work-record rules. No rotation was performed.
## Bounded implementation evidence
The existing sandbox mechanism passed the synthetic checks in
[the sandbox receipt](../../evidence/2026-09-28-supervised-sandbox-proof.json).
It does not prove interactive agent migration. The per-agent record at
`.kaizen/agents/custodian-codex/supervision.json` keeps this agent supervised,
with no autopilot grant or EUR limits assigned. The descriptive summarizer
`scripts/summarize_agent_supervision.py` cannot authorize or promote an agent.
No eligible scoring sample exists; the failed annotation rollout and recovery
are retained visibly rather than counted as unchanged success.
Current T03 outcome: nine admission checks pass; all 39 ExternalSecrets refreshed
successfully under Deny; the guard is Synced/Healthy. Source fixes and deployment
receipts are in the linked rollout record. The absent-namespace orphan Secret was deleted after explicit founder approval,
using its exact UID precondition; absence and unchanged 3 GiB PVC were verified.
`orphan-secret-deletion.json` now contains the execution disposition. T03 is
complete. Agent-runtime migration remains a separate unfinished task.

View file

@ -0,0 +1,27 @@
{
"reviewed_at": "2026-09-28T14:30:16.548657+00:00",
"resource": "Secret",
"namespace": "platform-pg-drill",
"name": "drill-minio",
"uid": "2fcb66df-d90f-4776-8ea0-8ca1a04bd307",
"created_at": "2026-08-13T11:09:33Z",
"namespace_exists": false,
"pods_in_namespace": 0,
"delete_options": {
"apiVersion": "v1",
"kind": "DeleteOptions",
"preconditions": {
"uid": "2fcb66df-d90f-4776-8ea0-8ca1a04bd307"
}
},
"proposal": "Delete only this orphan drill Secret, using the UID precondition. Do not recreate namespace, delete PVC or alter other resources.",
"reason": "Namespace is absent; API refuses annotation-only metadata update. Secret is an August 13 scratch drill artifact; referencing Deployment has zero Ready replicas and no pods.",
"risk": "Deletion removes the remaining credential copy in this orphan Secret. No Secret value has been inspected or archived.",
"storage": "Bound 3Gi platform-pg-drill-1 PVC and its PV retained unchanged.",
"approval": "Founder explicitly selected: Delete only the orphan Secret",
"executed": true,
"server_dry_run_passed": true,
"executed_at": "2026-09-28T16:07:36.040146+00:00",
"verified_absent": true,
"pvc_unchanged": true
}

View file

@ -0,0 +1,60 @@
#!/usr/bin/env python3
"""Positive/negative admission proof using only a uniquely named synthetic Secret."""
import copy
import json
import subprocess
import uuid
from datetime import datetime, timezone
KEY = "kubectl.kubernetes.io/last-applied-configuration"
def run(args, obj=None):
return subprocess.run(["kubectl", "-n", "whitehat", *args],
input=json.dumps(obj) if obj is not None else None,
capture_output=True, text=True, timeout=30)
def denied(result):
# Do not accept connectivity/RBAC failures as admission-policy success.
return result.returncode != 0 and "Secret last-applied annotations are forbidden" in result.stderr
def main():
name = "cust-0073-proof-" + uuid.uuid4().hex[:12]
obj = {"apiVersion": "v1", "kind": "Secret", "metadata": {"name": name},
"type": "Opaque", "data": {"fixture": "c3ludGhldGlj"}}
report = {"captured_at": datetime.now(timezone.utc).isoformat(), "namespace": "whitehat",
"fixture": name, "synthetic_only": True, "checks": {}}
created = False
try:
result = run(["create", "--field-manager=cust-0073-proof", "-f", "-"], obj)
created = result.returncode == 0
report["checks"]["clean_create_allowed"] = created
if not created:
raise RuntimeError("synthetic create failed")
for label, value in [("empty", ""), ("populated", "synthetic")]:
annotated = copy.deepcopy(obj)
annotated["metadata"]["name"] = name + "-denied"
annotated["metadata"]["annotations"] = {KEY: value}
report["checks"][label + "_annotated_create_denied"] = denied(run(["create", "--dry-run=server", "-f", "-"], annotated))
patch = {"metadata": {"annotations": {KEY: value}}}
report["checks"][label + "_annotated_update_denied"] = denied(run(["patch", "secret", name, "--dry-run=server", "--type=merge", "-p", json.dumps(patch)]))
report["checks"]["client_apply_denied"] = denied(run(["apply", "--dry-run=server", "-f", "-"], obj))
report["checks"]["clean_server_apply_allowed"] = run(["apply", "--server-side", "--field-manager=cust-0073-proof", "-f", "-"], obj).returncode == 0
report["checks"]["clean_update_allowed"] = run(["patch", "secret", name, "--type=merge", "-p", json.dumps({"data": {"fixture": "c3ludGhldGljLXVwZGF0ZQ=="}})]).returncode == 0
except (RuntimeError, subprocess.SubprocessError, OSError):
report["error"] = "proof incomplete; raw output suppressed"
finally:
if created:
try:
report["checks"]["fixture_removed"] = run(["delete", "secret", name, "--wait=true"]).returncode == 0
except (subprocess.SubprocessError, OSError):
report["checks"]["fixture_removed"] = False
report["passed"] = "error" not in report and len(report["checks"]) == 9 and all(report["checks"].values())
print(json.dumps(report, indent=2))
return 0 if report["passed"] else 1
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,27 @@
# Prepared under CUST-WP-0073-T03; not installed.
# Owner: railiance-platform. Clean existing annotations in an attended session
# before binding; otherwise subsequent updates to those Secrets are rejected.
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
name: reject-secret-last-applied
spec:
failurePolicy: Fail
matchConstraints:
resourceRules:
- apiGroups: [""]
apiVersions: ["v1"]
operations: ["CREATE", "UPDATE"]
resources: ["secrets"]
scope: "*"
validations:
- expression: '!has(object.metadata.annotations) || !("kubectl.kubernetes.io/last-applied-configuration" in object.metadata.annotations)'
message: "Secret last-applied annotations are forbidden; use server-side apply or replace."
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
name: reject-secret-last-applied
spec:
policyName: reject-secret-last-applied
validationActions: [Deny]

View file

@ -0,0 +1,91 @@
#!/usr/bin/env python3
"""Bounded CUST-WP-0073-T03 maintenance. Never emit kubectl output/errors.
Run on railiance01 through the supervised admin path. Default is inspection;
--clean removes only the last-applied annotation, leaving Secret data untouched.
"""
import argparse
import json
import re
import subprocess
from datetime import datetime, timezone
KEY = "kubectl.kubernetes.io/last-applied-configuration"
PRESENCE = ('{{ $found := false }}{{ range $key, $_ := .metadata.annotations }}'
'{{ if eq $key "' + KEY + '" }}{{ $found = true }}{{ end }}{{ end }}'
'{{ if $found }}HAS-ANNOTATION{{ else }}clean{{ end }}')
NAME = re.compile(r"^[a-z0-9][a-z0-9.-]*$")
def run(args):
# Even a template error can contain the entire Secret. Never forward it.
result = subprocess.run(["kubectl", *args], capture_output=True, text=True, timeout=30)
if result.returncode:
raise RuntimeError("kubectl operation failed; output suppressed")
return result.stdout.strip()
def inspect(namespace, name):
value = run(["-n", namespace, "get", "secret", name, "-o", "go-template=" + PRESENCE])
if value not in ("clean", "HAS-ANNOTATION"):
raise RuntimeError("unexpected presence result; output suppressed")
return value == "HAS-ANNOTATION"
def maintain(clean=False):
# Custom columns use fixed universally-present identity fields; no annotation
# or data output. Validate before using any returned text as an argument.
identities = run(["get", "secrets", "-A", "--no-headers", "-o",
"custom-columns=NAMESPACE:.metadata.namespace,NAME:.metadata.name"])
rows = []
for line in identities.splitlines():
pair = line.split()
if len(pair) != 2 or not all(NAME.fullmatch(value) for value in pair):
raise RuntimeError("invalid Secret identity output; suppressed")
rows.append(pair)
namespaces = run(["get", "namespaces", "-o", "name"]).splitlines()
if not all(value.startswith("namespace/") and NAME.fullmatch(value.split("/", 1)[1]) for value in namespaces):
raise RuntimeError("invalid namespace inventory; suppressed")
active_namespaces = {value.split("/", 1)[1] for value in namespaces}
report = {"captured_at": datetime.now(timezone.utc).isoformat(),
"mode": "clean" if clean else "inspect", "checked": 0,
"annotated": [], "cleaned": [], "orphaned_namespace": [], "complete": False}
try:
for namespace, name in rows:
if namespace not in active_namespaces:
report["orphaned_namespace"].append(namespace + "/" + name)
continue
report["checked"] += 1
if not inspect(namespace, name):
continue
identity = namespace + "/" + name
report["annotated"].append(identity)
if clean:
# A single JSON patch operation cannot modify credential data.
patch = [{"op": "remove", "path": "/metadata/annotations/" + KEY.replace("/", "~1")}]
run(["-n", namespace, "patch", "secret", name, "--type=json",
"-p", json.dumps(patch)])
if inspect(namespace, name):
raise RuntimeError("annotation still present")
report["cleaned"].append(identity)
report["active_namespace_scan_complete"] = True
report["complete"] = not report["orphaned_namespace"]
except (RuntimeError, subprocess.SubprocessError, OSError):
report["error"] = "maintenance incomplete; raw output suppressed; inspect before retry"
return report
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--clean", action="store_true")
args = parser.parse_args()
try:
report = maintain(args.clean)
except (RuntimeError, subprocess.SubprocessError, OSError):
report = {"complete": False, "error": "inventory failed; raw output suppressed"}
print(json.dumps(report, indent=2))
return 0 if report["complete"] else 1
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,16 @@
{
"source_observation": "2026-09-28T12:30:20Z",
"revisions": {
"/home/worsch/railiance-cluster": "550b50aa94ad0c10f68bbf7eac18d7c89f992c77",
"/home/worsch/state-hub": "e92471df3b415f9692a25eef9470f667c377b468",
"/home/worsch/rail-knative": "cd38dba3653e7fc85d3b2d3a074811b9a7216f25",
"/home/worsch/railiance-enablement": "28baf36ad6399543315288a9dd5038882899a3a5"
},
"active_pod_unsupported_accounting_features": [],
"unsupported_features_checked": [
"pod-level resources",
"overhead",
"restartable init containers"
],
"scope": "Read-only observation; no Secret objects queried; metrics are samples, not performance acceptance."
}

View file

@ -0,0 +1,696 @@
{
"schema": "custodian.allocation-reconcile.v1",
"workplan_id": "CUST-WP-0071-T01",
"captured_at": "2026-09-28T12:30:20Z",
"cluster_observation_schema": "railiance.cluster-resource-observation.v1",
"count_host_and_cluster_cpus_once": true,
"host": {
"owner": "railiance-cluster",
"resource_id": "resource:hosteurope:railiance01",
"same_cpus_as_cluster": true,
"cluster_resource_id": "resource:railiance:reef-railiance:k3s"
},
"capacity_cpu_m": 4000,
"scheduled_request_cpu_m": 3420,
"pending_unscheduled_cpu_m": 0,
"residual_cpu_m": 580,
"not_a_scheduling_guarantee": true,
"workloads": [
{
"namespace": "state-hub",
"workload": "railiance-apps",
"pods": 2,
"cpu_request_m": 260,
"memory_request_bytes": 671088640,
"owner": "state-hub",
"service": "state-hub",
"tenant": "unknown"
},
{
"namespace": "core-hub",
"workload": "rapp-core-hub",
"pods": 3,
"cpu_request_m": 200,
"memory_request_bytes": 939524096,
"owner": "core-hub",
"service": "core-hub",
"tenant": "unknown"
},
{
"namespace": "databases",
"workload": "forgejo-db",
"pods": 1,
"cpu_request_m": 200,
"memory_request_bytes": 536870912,
"owner": "rapp-postgres",
"service": "apps-pg",
"tenant": "unknown"
},
{
"namespace": "sso",
"workload": "net-kingdom-sso-mfa",
"pods": 4,
"cpu_request_m": 150,
"memory_request_bytes": 268435456,
"owner": "net-kingdom",
"service": "keycape-authelia",
"tenant": "unknown"
},
{
"namespace": "knative-serving",
"workload": "knative-serving",
"pods": 4,
"cpu_request_m": 140,
"memory_request_bytes": 377487360,
"owner": "rail-knative",
"service": "knative-serving",
"tenant": "unknown"
},
{
"namespace": "flex-auth",
"workload": "flex-auth",
"pods": 6,
"cpu_request_m": 110,
"memory_request_bytes": 201326592,
"owner": "flex-auth",
"service": "flex-auth",
"tenant": "unknown"
},
{
"namespace": "mfa",
"workload": "net-kingdom-sso-mfa",
"pods": 1,
"cpu_request_m": 110,
"memory_request_bytes": 402653184,
"owner": "net-kingdom",
"service": "lldap-mfa",
"tenant": "unknown"
},
{
"namespace": "reuse",
"workload": "reuse-surface",
"pods": 2,
"cpu_request_m": 110,
"memory_request_bytes": 301989888,
"owner": "reuse-surface",
"service": "reuse-surface",
"tenant": "unknown"
},
{
"namespace": "activity-core",
"workload": "activity-core",
"pods": 10,
"cpu_request_m": 100,
"memory_request_bytes": 268435456,
"owner": "activity-core",
"service": "activity-core",
"tenant": "unknown"
},
{
"namespace": "cnpg-system",
"workload": "cloudnative-pg",
"pods": 1,
"cpu_request_m": 100,
"memory_request_bytes": 104857600,
"owner": "rapp-postgres",
"service": "cloudnative-pg",
"tenant": "unknown"
},
{
"namespace": "coulomb-social",
"workload": "coulomb-social",
"pods": 1,
"cpu_request_m": 100,
"memory_request_bytes": 268435456,
"owner": "coulomb-social",
"service": "coulomb-social",
"tenant": "unknown"
},
{
"namespace": "databases",
"workload": "apps-pg",
"pods": 1,
"cpu_request_m": 100,
"memory_request_bytes": 268435456,
"owner": "rapp-postgres",
"service": "apps-pg",
"tenant": "unknown"
},
{
"namespace": "databases",
"workload": "net-kingdom-pg",
"pods": 1,
"cpu_request_m": 100,
"memory_request_bytes": 268435456,
"owner": "rapp-postgres",
"service": "apps-pg",
"tenant": "unknown"
},
{
"namespace": "databases",
"workload": "platform-pg",
"pods": 1,
"cpu_request_m": 100,
"memory_request_bytes": 268435456,
"owner": "rapp-postgres",
"service": "apps-pg",
"tenant": "unknown"
},
{
"namespace": "databases",
"workload": "platform-pg-2",
"pods": 1,
"cpu_request_m": 100,
"memory_request_bytes": 268435456,
"owner": "rapp-postgres",
"service": "apps-pg",
"tenant": "unknown"
},
{
"namespace": "forgejo",
"workload": "gitea",
"pods": 1,
"cpu_request_m": 100,
"memory_request_bytes": 134217728,
"owner": "railiance-forge",
"service": "forgejo",
"tenant": "unknown"
},
{
"namespace": "kube-system",
"workload": "coredns-7bdb54f89",
"pods": 1,
"cpu_request_m": 100,
"memory_request_bytes": 73400320,
"owner": "railiance-cluster",
"service": "k3s-control-plane",
"tenant": "unknown"
},
{
"namespace": "kube-system",
"workload": "metrics-server-786d997795",
"pods": 1,
"cpu_request_m": 100,
"memory_request_bytes": 73400320,
"owner": "railiance-cluster",
"service": "k3s-control-plane",
"tenant": "unknown"
},
{
"namespace": "openbao",
"workload": "openbao",
"pods": 1,
"cpu_request_m": 100,
"memory_request_bytes": 268435456,
"owner": "railiance-platform",
"service": "openbao",
"tenant": "unknown"
},
{
"namespace": "telemetry",
"workload": "prometheus",
"pods": 1,
"cpu_request_m": 100,
"memory_request_bytes": 536870912,
"owner": "rapp-telemetry",
"service": "prometheus-grafana",
"tenant": "unknown"
},
{
"namespace": "user-engine",
"workload": "user-engine-pg",
"pods": 1,
"cpu_request_m": 100,
"memory_request_bytes": 268435456,
"owner": "user-engine",
"service": "user-engine",
"tenant": "unknown"
},
{
"namespace": "telemetry",
"workload": "grafana",
"pods": 1,
"cpu_request_m": 70,
"memory_request_bytes": 201326592,
"owner": "rapp-telemetry",
"service": "prometheus-grafana",
"tenant": "unknown"
},
{
"namespace": "vergabe-demo-company",
"workload": "vergabe-teilnahme",
"pods": 1,
"cpu_request_m": 60,
"memory_request_bytes": 268435456,
"owner": "railiance-apps",
"service": "vergabe-teilnahme",
"tenant": "unknown"
},
{
"namespace": "argocd",
"workload": "argocd-application-controller",
"pods": 1,
"cpu_request_m": 50,
"memory_request_bytes": 268435456,
"owner": "railiance-enablement",
"service": "argocd",
"tenant": "unknown"
},
{
"namespace": "audit-core",
"workload": "audit-core",
"pods": 1,
"cpu_request_m": 50,
"memory_request_bytes": 67108864,
"owner": "audit-core",
"service": "audit-core",
"tenant": "unknown"
},
{
"namespace": "coulomb",
"workload": "ihp-railiance-probe",
"pods": 1,
"cpu_request_m": 50,
"memory_request_bytes": 134217728,
"owner": "unknown",
"service": "ihp-railiance-probe",
"tenant": "unknown"
},
{
"namespace": "issue-core",
"workload": "issue-core",
"pods": 1,
"cpu_request_m": 50,
"memory_request_bytes": 134217728,
"owner": "issue-core",
"service": "issue-core",
"tenant": "unknown"
},
{
"namespace": "kourier-system",
"workload": "3scale-kourier-gateway",
"pods": 1,
"cpu_request_m": 50,
"memory_request_bytes": 209715200,
"owner": "rail-knative",
"service": "kourier",
"tenant": "unknown"
},
{
"namespace": "target-revenue",
"workload": "target-revenue",
"pods": 1,
"cpu_request_m": 50,
"memory_request_bytes": 268435456,
"owner": "target-revenue",
"service": "target-revenue",
"tenant": "unknown"
},
{
"namespace": "user-engine",
"workload": "user-engine",
"pods": 1,
"cpu_request_m": 50,
"memory_request_bytes": 67108864,
"owner": "user-engine",
"service": "user-engine",
"tenant": "unknown"
},
{
"namespace": "knative-serving",
"workload": "net-kourier-controller",
"pods": 1,
"cpu_request_m": 30,
"memory_request_bytes": 209715200,
"owner": "rail-knative",
"service": "knative-serving",
"tenant": "unknown"
},
{
"namespace": "argocd",
"workload": "argocd-repo-server",
"pods": 1,
"cpu_request_m": 25,
"memory_request_bytes": 134217728,
"owner": "railiance-enablement",
"service": "argocd",
"tenant": "unknown"
},
{
"namespace": "canned-prompts",
"workload": "canned-prompts",
"pods": 1,
"cpu_request_m": 25,
"memory_request_bytes": 100663296,
"owner": "canned-prompts",
"service": "canned-prompts",
"tenant": "unknown"
},
{
"namespace": "email-connect",
"workload": "email-connect",
"pods": 1,
"cpu_request_m": 25,
"memory_request_bytes": 67108864,
"owner": "email-connect",
"service": "email-connect",
"tenant": "unknown"
},
{
"namespace": "openbao",
"workload": "rapp-openbao",
"pods": 1,
"cpu_request_m": 25,
"memory_request_bytes": 33554432,
"owner": "railiance-platform",
"service": "openbao",
"tenant": "unknown"
},
{
"namespace": "rein-aharness",
"workload": "rein-aharness",
"pods": 1,
"cpu_request_m": 25,
"memory_request_bytes": 67108864,
"owner": "rein-aharness",
"service": "rein-aharness",
"tenant": "unknown"
},
{
"namespace": "sbom-nexus",
"workload": "sbom-nexus",
"pods": 1,
"cpu_request_m": 25,
"memory_request_bytes": 67108864,
"owner": "sbom-nexus",
"service": "sbom-nexus",
"tenant": "unknown"
},
{
"namespace": "telemetry",
"workload": "alertmanager",
"pods": 1,
"cpu_request_m": 25,
"memory_request_bytes": 67108864,
"owner": "rapp-telemetry",
"service": "prometheus-grafana",
"tenant": "unknown"
},
{
"namespace": "telemetry",
"workload": "kube-state-metrics",
"pods": 1,
"cpu_request_m": 25,
"memory_request_bytes": 67108864,
"owner": "rapp-telemetry",
"service": "prometheus-grafana",
"tenant": "unknown"
},
{
"namespace": "telemetry",
"workload": "rapp-telemetry",
"pods": 1,
"cpu_request_m": 25,
"memory_request_bytes": 134217728,
"owner": "rapp-telemetry",
"service": "prometheus-grafana",
"tenant": "unknown"
},
{
"namespace": "tenant-engine",
"workload": "tenant-engine",
"pods": 1,
"cpu_request_m": 25,
"memory_request_bytes": 50331648,
"owner": "tenant-engine",
"service": "tenant-engine",
"tenant": "unknown"
},
{
"namespace": "rapp-qonto-egress",
"workload": "qonto-egress-proxy",
"pods": 1,
"cpu_request_m": 20,
"memory_request_bytes": 67108864,
"owner": "rapp-qonto",
"service": "qonto-egress",
"tenant": "tenant:friendly:binky"
},
{
"namespace": "activity-core",
"workload": "actcore-temporal-ui-tls-2-1888679036-1756117428",
"pods": 1,
"cpu_request_m": 10,
"memory_request_bytes": 67108864,
"owner": "activity-core",
"service": "activity-core",
"tenant": "unknown"
},
{
"namespace": "approval-engine",
"workload": "approval-engine",
"pods": 1,
"cpu_request_m": 10,
"memory_request_bytes": 67108864,
"owner": "approval-engine",
"service": "approval-engine",
"tenant": "unknown"
},
{
"namespace": "argocd",
"workload": "argocd-applicationset-controller",
"pods": 1,
"cpu_request_m": 10,
"memory_request_bytes": 67108864,
"owner": "railiance-enablement",
"service": "argocd",
"tenant": "unknown"
},
{
"namespace": "argocd",
"workload": "argocd-redis",
"pods": 1,
"cpu_request_m": 10,
"memory_request_bytes": 33554432,
"owner": "railiance-enablement",
"service": "argocd",
"tenant": "unknown"
},
{
"namespace": "policy-nexus",
"workload": "policy-nexus",
"pods": 1,
"cpu_request_m": 10,
"memory_request_bytes": 33554432,
"owner": "policy-nexus",
"service": "policy-nexus",
"tenant": "unknown"
},
{
"namespace": "bao-notice",
"workload": "bao-notice",
"pods": 1,
"cpu_request_m": 5,
"memory_request_bytes": 16777216,
"owner": "railiance-platform",
"service": "bao-notice",
"tenant": "unknown"
},
{
"namespace": "informed-decision",
"workload": "informed-decision",
"pods": 1,
"cpu_request_m": 5,
"memory_request_bytes": 67108864,
"owner": "informed-decision",
"service": "informed-decision",
"tenant": "unknown"
},
{
"namespace": "cert-manager",
"workload": "cainjector",
"pods": 1,
"cpu_request_m": 0,
"memory_request_bytes": 0,
"owner": "railiance-cluster",
"service": "cert-manager",
"tenant": "unknown"
},
{
"namespace": "cert-manager",
"workload": "cert-manager",
"pods": 1,
"cpu_request_m": 0,
"memory_request_bytes": 0,
"owner": "railiance-cluster",
"service": "cert-manager",
"tenant": "unknown"
},
{
"namespace": "cert-manager",
"workload": "webhook",
"pods": 1,
"cpu_request_m": 0,
"memory_request_bytes": 0,
"owner": "railiance-cluster",
"service": "cert-manager",
"tenant": "unknown"
},
{
"namespace": "databases",
"workload": "state-hub-db",
"pods": 1,
"cpu_request_m": 0,
"memory_request_bytes": 0,
"owner": "rapp-postgres",
"service": "apps-pg",
"tenant": "unknown"
},
{
"namespace": "external-secrets",
"workload": "external-secrets",
"pods": 1,
"cpu_request_m": 0,
"memory_request_bytes": 0,
"owner": "railiance-platform",
"service": "external-secrets",
"tenant": "unknown"
},
{
"namespace": "external-secrets",
"workload": "external-secrets-cert-controller",
"pods": 1,
"cpu_request_m": 0,
"memory_request_bytes": 0,
"owner": "railiance-platform",
"service": "external-secrets",
"tenant": "unknown"
},
{
"namespace": "external-secrets",
"workload": "external-secrets-webhook",
"pods": 1,
"cpu_request_m": 0,
"memory_request_bytes": 0,
"owner": "railiance-platform",
"service": "external-secrets",
"tenant": "unknown"
},
{
"namespace": "forgejo",
"workload": "forgejo-runner",
"pods": 1,
"cpu_request_m": 0,
"memory_request_bytes": 0,
"owner": "railiance-forge",
"service": "forgejo",
"tenant": "unknown"
},
{
"namespace": "kube-system",
"workload": "local-path-provisioner",
"pods": 1,
"cpu_request_m": 0,
"memory_request_bytes": 0,
"owner": "railiance-cluster",
"service": "k3s-control-plane",
"tenant": "unknown"
},
{
"namespace": "kube-system",
"workload": "svclb-traefik-0c8aecaf",
"pods": 1,
"cpu_request_m": 0,
"memory_request_bytes": 0,
"owner": "railiance-cluster",
"service": "k3s-control-plane",
"tenant": "unknown"
},
{
"namespace": "kube-system",
"workload": "traefik",
"pods": 1,
"cpu_request_m": 0,
"memory_request_bytes": 0,
"owner": "railiance-cluster",
"service": "k3s-control-plane",
"tenant": "unknown"
},
{
"namespace": "target-revenue",
"workload": "target-revenue-pg",
"pods": 1,
"cpu_request_m": 0,
"memory_request_bytes": 0,
"owner": "target-revenue",
"service": "target-revenue",
"tenant": "unknown"
}
],
"unknown_namespaces": [],
"zero_request_workloads": [
"cert-manager/cainjector",
"cert-manager/cert-manager",
"cert-manager/webhook",
"databases/state-hub-db",
"external-secrets/external-secrets",
"external-secrets/external-secrets-cert-controller",
"external-secrets/external-secrets-webhook",
"forgejo/forgejo-runner",
"kube-system/local-path-provisioner",
"kube-system/svclb-traefik-0c8aecaf",
"kube-system/traefik",
"target-revenue/target-revenue-pg"
],
"pending_unscheduled": [],
"measurement_gaps": [
"node 239.62.205.92.host.secureserver.net lacks independent region/zone labels"
],
"state_hub_preflight_observation": {
"schema": "state-hub.release-headroom-preflight.v1-input",
"observed_at": "2026-09-28T12:30:20Z",
"freshness_seconds": 0,
"nodes": [
{
"name": "239.62.205.92.host.secureserver.net",
"ready": true,
"unschedulable": false,
"allocatable_cpu_m": 4000,
"allocatable_memory_bytes": 16770076672,
"allocated_cpu_m": 3420,
"allocated_memory_bytes": 9806282752
}
],
"pending_unrelated": []
},
"signal_notes": [
"Host resource:hosteurope:railiance01 and cluster resource:railiance:reef-railiance:k3s are the same 4 vCPU; do not add them.",
"Prometheus namespace_cpu:kube_pod_container_resource_requests:sum omitted Forgejo 100m on 2026-09-11; Kubernetes API is the scheduler-effective source.",
"node-exporter was disabled; host CPU usage is not a reservation and is not treated as spare capacity.",
"Zero-request pods are demand, not zero. Missing metrics are listed as gaps, not zeros.",
"STATE-WP-0091 preflight consumes state_hub_preflight_observation; residual millicores are not admission."
],
"state_hub_preflight": {
"schema": "state-hub.release-headroom-preflight.v1",
"ok": true,
"observed_at": "2026-09-28T12:30:20Z",
"freshness_seconds": 0,
"remaining_cpu_m": 580,
"remaining_memory_bytes": 6963793920,
"pending_unrelated_cpu_m": 0,
"api_surge_cpu_m": 100,
"mcp_surge_cpu_m": 10,
"migrate_cpu_m": 50,
"atomic": true,
"reasons": [],
"notes": [
"Aggregate remaining millicores is not a kube-scheduler guarantee.",
"Preflight does not lower requests and does not start Helm."
],
"hook_order": [
"pre-upgrade migrate job (helm.sh/hook-weight -5)",
"API RollingUpdate maxSurge=1 maxUnavailable=0",
"MCP RollingUpdate maxSurge=1 maxUnavailable=0"
]
}
}

View file

@ -0,0 +1,91 @@
# Railiance allocation reconcile — 2026-09-28T12:30:20Z
CUST-WP-0071-T01. Scheduler-effective requests from the Kubernetes API
via `railiance-cluster` observe (RCLUSTER-WP-0014 / RAIL-BS-WP-0014).
Host and cluster are the same 4 vCPU and are counted once.
- Capacity: 4000m
- Scheduled requests: 3420m
- Pending unscheduled: 0m
- Residual (not a guarantee): 580m
- Unknown namespaces: none
- Zero-request workloads: 12
| Namespace | Workload | Owner | Tenant | CPU request (m) | Pods |
|---|---|---|---|---:|---:|
| state-hub | railiance-apps | state-hub | unknown | 260 | 2 |
| core-hub | rapp-core-hub | core-hub | unknown | 200 | 3 |
| databases | forgejo-db | rapp-postgres | unknown | 200 | 1 |
| sso | net-kingdom-sso-mfa | net-kingdom | unknown | 150 | 4 |
| knative-serving | knative-serving | rail-knative | unknown | 140 | 4 |
| flex-auth | flex-auth | flex-auth | unknown | 110 | 6 |
| mfa | net-kingdom-sso-mfa | net-kingdom | unknown | 110 | 1 |
| reuse | reuse-surface | reuse-surface | unknown | 110 | 2 |
| activity-core | activity-core | activity-core | unknown | 100 | 10 |
| cnpg-system | cloudnative-pg | rapp-postgres | unknown | 100 | 1 |
| coulomb-social | coulomb-social | coulomb-social | unknown | 100 | 1 |
| databases | apps-pg | rapp-postgres | unknown | 100 | 1 |
| databases | net-kingdom-pg | rapp-postgres | unknown | 100 | 1 |
| databases | platform-pg | rapp-postgres | unknown | 100 | 1 |
| databases | platform-pg-2 | rapp-postgres | unknown | 100 | 1 |
| forgejo | gitea | railiance-forge | unknown | 100 | 1 |
| kube-system | coredns-7bdb54f89 | railiance-cluster | unknown | 100 | 1 |
| kube-system | metrics-server-786d997795 | railiance-cluster | unknown | 100 | 1 |
| openbao | openbao | railiance-platform | unknown | 100 | 1 |
| telemetry | prometheus | rapp-telemetry | unknown | 100 | 1 |
| user-engine | user-engine-pg | user-engine | unknown | 100 | 1 |
| telemetry | grafana | rapp-telemetry | unknown | 70 | 1 |
| vergabe-demo-company | vergabe-teilnahme | railiance-apps | unknown | 60 | 1 |
| argocd | argocd-application-controller | railiance-enablement | unknown | 50 | 1 |
| audit-core | audit-core | audit-core | unknown | 50 | 1 |
| coulomb | ihp-railiance-probe | unknown | unknown | 50 | 1 |
| issue-core | issue-core | issue-core | unknown | 50 | 1 |
| kourier-system | 3scale-kourier-gateway | rail-knative | unknown | 50 | 1 |
| target-revenue | target-revenue | target-revenue | unknown | 50 | 1 |
| user-engine | user-engine | user-engine | unknown | 50 | 1 |
| knative-serving | net-kourier-controller | rail-knative | unknown | 30 | 1 |
| argocd | argocd-repo-server | railiance-enablement | unknown | 25 | 1 |
| canned-prompts | canned-prompts | canned-prompts | unknown | 25 | 1 |
| email-connect | email-connect | email-connect | unknown | 25 | 1 |
| openbao | rapp-openbao | railiance-platform | unknown | 25 | 1 |
| rein-aharness | rein-aharness | rein-aharness | unknown | 25 | 1 |
| sbom-nexus | sbom-nexus | sbom-nexus | unknown | 25 | 1 |
| telemetry | alertmanager | rapp-telemetry | unknown | 25 | 1 |
| telemetry | kube-state-metrics | rapp-telemetry | unknown | 25 | 1 |
| telemetry | rapp-telemetry | rapp-telemetry | unknown | 25 | 1 |
| tenant-engine | tenant-engine | tenant-engine | unknown | 25 | 1 |
| rapp-qonto-egress | qonto-egress-proxy | rapp-qonto | tenant:friendly:binky | 20 | 1 |
| activity-core | actcore-temporal-ui-tls-2-1888679036-1756117428 | activity-core | unknown | 10 | 1 |
| approval-engine | approval-engine | approval-engine | unknown | 10 | 1 |
| argocd | argocd-applicationset-controller | railiance-enablement | unknown | 10 | 1 |
| argocd | argocd-redis | railiance-enablement | unknown | 10 | 1 |
| policy-nexus | policy-nexus | policy-nexus | unknown | 10 | 1 |
| bao-notice | bao-notice | railiance-platform | unknown | 5 | 1 |
| informed-decision | informed-decision | informed-decision | unknown | 5 | 1 |
| cert-manager | cainjector | railiance-cluster | unknown | 0 | 1 |
| cert-manager | cert-manager | railiance-cluster | unknown | 0 | 1 |
| cert-manager | webhook | railiance-cluster | unknown | 0 | 1 |
| databases | state-hub-db | rapp-postgres | unknown | 0 | 1 |
| external-secrets | external-secrets | railiance-platform | unknown | 0 | 1 |
| external-secrets | external-secrets-cert-controller | railiance-platform | unknown | 0 | 1 |
| external-secrets | external-secrets-webhook | railiance-platform | unknown | 0 | 1 |
| forgejo | forgejo-runner | railiance-forge | unknown | 0 | 1 |
| kube-system | local-path-provisioner | railiance-cluster | unknown | 0 | 1 |
| kube-system | svclb-traefik-0c8aecaf | railiance-cluster | unknown | 0 | 1 |
| kube-system | traefik | railiance-cluster | unknown | 0 | 1 |
| target-revenue | target-revenue-pg | target-revenue | unknown | 0 | 1 |
## STATE-WP-0091 preflight
- ok: `True`
- remaining_cpu_m: 580
- note: Aggregate remaining millicores is not a kube-scheduler guarantee.
- note: Preflight does not lower requests and does not start Helm.
## Signal notes
- Host resource:hosteurope:railiance01 and cluster resource:railiance:reef-railiance:k3s are the same 4 vCPU; do not add them.
- Prometheus namespace_cpu:kube_pod_container_resource_requests:sum omitted Forgejo 100m on 2026-09-11; Kubernetes API is the scheduler-effective source.
- node-exporter was disabled; host CPU usage is not a reservation and is not treated as spare capacity.
- Zero-request pods are demand, not zero. Missing metrics are listed as gaps, not zeros.
- STATE-WP-0091 preflight consumes state_hub_preflight_observation; residual millicores are not admission.

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,319 @@
[
{
"id": "activity-core/actcore-backup-offsite",
"source": "/home/worsch/activity-core/k8s/railiance/15-externalsecret-backup-offsite.yaml",
"template": {
"metadata": {
"labels": {
"app.kubernetes.io/name": "activity-core",
"app.kubernetes.io/part-of": "activity-core"
},
"annotations": {
"argocd.argoproj.io/sync-wave": "0"
}
}
}
},
{
"id": "activity-core/actcore-forgejo-admin",
"source": "/home/worsch/activity-core/k8s/railiance/15-externalsecret-forgejo-admin.yaml",
"template": {
"metadata": {
"labels": {
"app.kubernetes.io/name": "activity-core",
"app.kubernetes.io/part-of": "activity-core"
},
"annotations": {
"argocd.argoproj.io/sync-wave": "0"
}
}
}
},
{
"id": "activity-core/actcore-issue-core-runtime",
"source": "/home/worsch/activity-core/k8s/railiance/15-externalsecret-issue-core.yaml",
"template": {
"metadata": {
"labels": {
"app.kubernetes.io/name": "activity-core",
"app.kubernetes.io/part-of": "activity-core"
},
"annotations": {
"argocd.argoproj.io/sync-wave": "0"
}
}
}
},
{
"id": "activity-core/actcore-ops-run-worker-tokens",
"source": "/home/worsch/activity-core/k8s/railiance/15-externalsecret-worker-tokens.yaml",
"template": {
"metadata": {
"labels": {
"app.kubernetes.io/name": "activity-core",
"app.kubernetes.io/part-of": "activity-core"
}
}
}
},
{
"id": "audit-core/audit-core-senders",
"source": "/home/worsch/audit-core/deploy/externalsecret-senders.yaml",
"template": {
"metadata": {}
}
},
{
"id": "email-connect/email-connect-runtime",
"source": "/home/worsch/email-connect/deploy/k8s/railiance/externalsecret.yaml",
"template": {
"metadata": {
"labels": {
"app.kubernetes.io/name": "email-connect",
"app.kubernetes.io/part-of": "email-connect"
},
"annotations": {
"argocd.argoproj.io/sync-wave": "0"
}
}
}
},
{
"id": "activity-core/llm-connect-provider-secrets",
"source": "/home/worsch/llm-connect/deploy/k8s/activity-core-llm-connect/externalsecret.yaml",
"template": {
"metadata": {
"labels": {
"app.kubernetes.io/name": "llm-connect",
"app.kubernetes.io/part-of": "railiance-gitops"
}
}
}
},
{
"id": "forgejo/forgejo-mailer",
"source": "/home/worsch/railiance-apps/manifests/forgejo-mailer-externalsecret.yaml",
"template": {
"metadata": {
"labels": {
"app.kubernetes.io/name": "forgejo",
"app.kubernetes.io/part-of": "railiance-apps"
}
}
}
},
{
"id": "reuse/reuse-surface-runtime",
"source": "/home/worsch/railiance-apps/manifests/reuse-surface-runtime-externalsecret.yaml",
"template": {
"metadata": {
"labels": {
"app.kubernetes.io/name": "reuse-surface",
"app.kubernetes.io/part-of": "railiance-apps"
}
}
}
},
{
"id": "core-hub/core-hub-api-token",
"source": "/home/worsch/railiance-platform/argocd/platform-addons/openbao-secretstore/core-hub.externalsecrets.yaml",
"template": {
"metadata": {}
}
},
{
"id": "core-hub/core-hub-runtime-database",
"source": "/home/worsch/railiance-platform/argocd/platform-addons/openbao-secretstore/core-hub.externalsecrets.yaml",
"template": {
"metadata": {}
}
},
{
"id": "core-hub/core-hub-migration-database",
"source": "/home/worsch/railiance-platform/argocd/platform-addons/openbao-secretstore/core-hub.externalsecrets.yaml",
"template": {
"metadata": {}
}
},
{
"id": "core-hub/hub-core-runtime-database",
"source": "/home/worsch/railiance-platform/argocd/platform-addons/openbao-secretstore/core-hub.externalsecrets.yaml",
"template": {
"metadata": {}
}
},
{
"id": "core-hub/hub-core-migration-database",
"source": "/home/worsch/railiance-platform/argocd/platform-addons/openbao-secretstore/core-hub.externalsecrets.yaml",
"template": {
"metadata": {}
}
},
{
"id": "sso/keycape-secrets-engine-approval-client",
"source": "/home/worsch/railiance-platform/argocd/platform-addons/openbao-secretstore/keycape-approval-clients.externalsecrets.yaml",
"template": {
"metadata": {}
}
},
{
"id": "sso/keycape-approval-engine-operator-client",
"source": "/home/worsch/railiance-platform/argocd/platform-addons/openbao-secretstore/keycape-approval-clients.externalsecrets.yaml",
"template": {
"metadata": {}
}
},
{
"id": "sso/keycape-informed-decision-sitting-requester-client",
"source": "/home/worsch/railiance-platform/argocd/platform-addons/openbao-secretstore/sitting-requester.yaml",
"template": {
"metadata": {}
}
},
{
"id": "sso/keycape-secrets-engine-requester-client",
"source": "/home/worsch/railiance-platform/argocd/platform-addons/openbao-secretstore/t03-requester.yaml",
"template": {
"metadata": {}
}
},
{
"id": "approval-engine/approval-engine-audit",
"source": "/home/worsch/railiance-platform/manifests/factory-audit-senders.yaml",
"template": {
"metadata": {
"annotations": {
"railiance.io/credential-change": "CCR-2026-0021",
"railiance.io/admission": "approved"
}
}
}
},
{
"id": "informed-decision/informed-decision-audit",
"source": "/home/worsch/railiance-platform/manifests/factory-audit-senders.yaml",
"template": {
"metadata": {
"annotations": {
"railiance.io/credential-change": "CCR-2026-0022",
"railiance.io/admission": "approved"
}
}
}
},
{
"id": "sso/keycape-factor-read",
"source": "/home/worsch/railiance-platform/manifests/keycape-factor-custody.yaml",
"template": {
"metadata": {
"labels": {
"app.kubernetes.io/part-of": "net-kingdom-sso-mfa"
}
}
}
},
{
"id": "state-hub/state-hub-rename-preflight",
"source": "/home/worsch/railiance-platform/openbao/state-hub-preflight/delivery.yaml",
"template": {
"metadata": {}
}
},
{
"id": "issue-core/issue-core-runtime",
"source": "/home/worsch/rapp-issue-core/manifests/20-secret.yaml",
"template": {
"metadata": {
"labels": {
"app.kubernetes.io/name": "issue-core",
"app.kubernetes.io/part-of": "issue-core"
}
}
}
},
{
"id": "databases/platform-pg-backup-s3",
"source": "/home/worsch/rapp-postgres/helm/platform-pg-backup-s3.externalsecret.yaml",
"template": {
"metadata": {
"labels": {
"app.kubernetes.io/name": "platform-pg",
"app.kubernetes.io/part-of": "railiance-gitops",
"railiance.io/layer": "s3-platform"
}
}
}
},
{
"id": "rapp-qonto/rapp-qonto",
"source": "/home/worsch/rapp-qonto/runtime/knative/externalsecret.yaml",
"template": {
"metadata": {}
}
},
{
"id": "telemetry/telemetry-alert-smtp",
"source": "/home/worsch/rapp-telemetry/acknowledgment/smtp-custody.yaml",
"template": {
"metadata": {}
}
},
{
"id": "telemetry/telemetry-grafana-admin",
"source": "/home/worsch/rapp-telemetry/manifests/custody.yaml",
"template": {
"metadata": {}
}
},
{
"id": "user-engine/user-engine-runtime",
"source": "/home/worsch/rapp-user-engine/manifests/openbao-runtime.yaml",
"template": {
"metadata": {
"labels": {
"app.kubernetes.io/name": "user-engine",
"app.kubernetes.io/part-of": "user-engine"
}
}
}
},
{
"id": "user-engine/identity-provisioner-client",
"source": "/home/worsch/rapp-user-engine/manifests/openbao-runtime.yaml",
"template": {
"metadata": {
"labels": {
"app.kubernetes.io/name": "user-engine",
"app.kubernetes.io/part-of": "user-engine"
}
}
}
},
{
"id": "sso/identity-provisioner-token",
"source": "/home/worsch/rapp-user-engine/manifests/openbao-runtime.yaml",
"template": {
"metadata": {
"labels": {
"app.kubernetes.io/name": "identity-provisioner",
"app.kubernetes.io/part-of": "net-kingdom-sso-mfa"
}
}
}
},
{
"id": "target-revenue/target-revenue-runtime",
"source": "/home/worsch/target-revenue/k8s/railiance/externalsecret.yaml",
"template": {
"metadata": {
"labels": {
"app.kubernetes.io/name": "target-revenue",
"app.kubernetes.io/part-of": "target-revenue"
},
"annotations": {
"argocd.argoproj.io/sync-wave": "0"
}
}
}
}
]

View file

@ -0,0 +1,167 @@
{
"observed_at": "2026-09-28T13:54:07.234385+00:00",
"declarations": 31,
"server_dry_run": "passed",
"server_diff_exit": 1,
"checks": [
{
"id": "activity-core/actcore-backup-offsite",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "activity-core/actcore-forgejo-admin",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "activity-core/actcore-issue-core-runtime",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "activity-core/actcore-ops-run-worker-tokens",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "audit-core/audit-core-senders",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "email-connect/email-connect-runtime",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "activity-core/llm-connect-provider-secrets",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "forgejo/forgejo-mailer",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "reuse/reuse-surface-runtime",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "core-hub/core-hub-api-token",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "core-hub/core-hub-runtime-database",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "core-hub/core-hub-migration-database",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "core-hub/hub-core-runtime-database",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "core-hub/hub-core-migration-database",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "sso/keycape-secrets-engine-approval-client",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "sso/keycape-approval-engine-operator-client",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "sso/keycape-informed-decision-sitting-requester-client",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "sso/keycape-secrets-engine-requester-client",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "approval-engine/approval-engine-audit",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "informed-decision/informed-decision-audit",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "sso/keycape-factor-read",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "state-hub/state-hub-rename-preflight",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "issue-core/issue-core-runtime",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "databases/platform-pg-backup-s3",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "rapp-qonto/rapp-qonto",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "telemetry/telemetry-alert-smtp",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "telemetry/telemetry-grafana-admin",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "user-engine/user-engine-runtime",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "user-engine/identity-provisioner-client",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "sso/identity-provisioner-token",
"only_template_changed": true,
"template_metadata_matches": true
},
{
"id": "target-revenue/target-revenue-runtime",
"only_template_changed": true,
"template_metadata_matches": true
}
],
"no_credential_values_read": true,
"activation": "APPROVED",
"authority": "ADMINISTER @ realm:kubernetes/railiance01",
"authorization": "Founder: Good, go on, after 31-declaration remediation described."
}

View file

@ -0,0 +1,85 @@
[
{
"repo": "audit-core",
"commit": "f0dff91eb53cf4f29bc28ff6804a41aea07abe88",
"status": "applied",
"instance": "railiance01",
"exact_commit_verified": true
},
{
"repo": "email-connect",
"commit": "73702b7e1a1671948b0545ef32d6e0de9cca9c65",
"status": "applied",
"instance": "railiance01",
"exact_commit_verified": true
},
{
"repo": "llm-connect",
"commit": "08850d0aafc82bed457bdbfdb6a050f6f532320c",
"status": "applied",
"instance": "railiance01",
"exact_commit_verified": true
},
{
"repo": "railiance-apps",
"commit": "53dcd0121925d9bc13edc3f07efc7a1775c14917",
"status": "applied",
"instance": "railiance01",
"exact_commit_verified": true
},
{
"repo": "railiance-platform",
"commit": "80e053988fcd7f45c4e297a416e4915d7a73804a",
"status": "applied",
"instance": "railiance01",
"exact_commit_verified": true
},
{
"repo": "rapp-issue-core",
"commit": "171545d42a710491a55b28ba7499d23c5ba657d2",
"status": "applied",
"instance": "railiance01",
"exact_commit_verified": true
},
{
"repo": "rapp-postgres",
"commit": "11c1d489b580c45c612768fc6091c796bde8d77f",
"status": "applied",
"instance": "railiance01",
"exact_commit_verified": true
},
{
"repo": "rapp-qonto",
"commit": "28acdd11e689464057127e51924ee4153195ae34",
"status": "applied",
"instance": "railiance01",
"exact_commit_verified": true
},
{
"repo": "rapp-telemetry",
"commit": "34cfa03de5c298f0b3bbc6413e0f72e30d51d4c4",
"status": "applied",
"instance": "railiance01",
"exact_commit_verified": true
},
{
"repo": "rapp-user-engine",
"commit": "76ca9dbf9d3c53266c15676f8fb4d83dae8a5aa1",
"status": "applied",
"instance": "railiance01",
"exact_commit_verified": true
},
{
"repo": "target-revenue",
"commit": "a3a8a27a8cda32ae3c7b55353ee16a131c50a0a0",
"status": "applied",
"instance": "railiance01",
"exact_commit_verified": true
},
{
"repo": "activity-core",
"commit": "19fc7e4597649b44581dca75bfb46227c552b7fd",
"status": "pushed via documented statehub fix-consistency; PASS with legacy warnings",
"exact_commit_verified": true
}
]

View file

@ -0,0 +1,284 @@
{
"phase": "after-binding",
"started_at": "2026-09-28T14:29:01.530820+00:00",
"checked_at": "2026-09-28T14:29:47.150368+00:00",
"total": 39,
"ready": 39,
"fresh": 39,
"complete": true,
"rows": [
{
"id": "activity-core/actcore-backup-offsite",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:02Z",
"fresh_refresh": true
},
{
"id": "activity-core/actcore-forgejo-admin",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:03Z",
"fresh_refresh": true
},
{
"id": "activity-core/actcore-issue-core-runtime",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:03Z",
"fresh_refresh": true
},
{
"id": "activity-core/actcore-ops-run-worker-tokens",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:03Z",
"fresh_refresh": true
},
{
"id": "activity-core/llm-connect-provider-secrets",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:04Z",
"fresh_refresh": true
},
{
"id": "approval-engine/approval-engine-audit",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:04Z",
"fresh_refresh": true
},
{
"id": "audit-core/audit-core-database",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:04Z",
"fresh_refresh": true
},
{
"id": "audit-core/audit-core-database-migrate",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:04Z",
"fresh_refresh": true
},
{
"id": "audit-core/audit-core-senders",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:05Z",
"fresh_refresh": true
},
{
"id": "canned-prompts/canned-prompts-postgres-migration",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:05Z",
"fresh_refresh": true
},
{
"id": "canned-prompts/canned-prompts-postgres-runtime",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:06Z",
"fresh_refresh": true
},
{
"id": "core-hub/core-hub-api-token",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:07Z",
"fresh_refresh": true
},
{
"id": "core-hub/core-hub-migration-database",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:07Z",
"fresh_refresh": true
},
{
"id": "core-hub/core-hub-runtime-database",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:08Z",
"fresh_refresh": true
},
{
"id": "core-hub/hub-core-migration-database",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:08Z",
"fresh_refresh": true
},
{
"id": "core-hub/hub-core-runtime-database",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:08Z",
"fresh_refresh": true
},
{
"id": "databases/platform-pg-backup-s3",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:09Z",
"fresh_refresh": true
},
{
"id": "email-connect/email-connect-runtime",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:09Z",
"fresh_refresh": true
},
{
"id": "forgejo/forgejo-mailer",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:09Z",
"fresh_refresh": true
},
{
"id": "informed-decision/informed-decision-audit",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:09Z",
"fresh_refresh": true
},
{
"id": "issue-core/issue-core-runtime",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:10Z",
"fresh_refresh": true
},
{
"id": "rapp-qonto/rapp-qonto",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:10Z",
"fresh_refresh": true
},
{
"id": "reuse/reuse-surface-runtime",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:10Z",
"fresh_refresh": true
},
{
"id": "sbom-nexus/sbom-nexus-postgres-migration",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:10Z",
"fresh_refresh": true
},
{
"id": "sbom-nexus/sbom-nexus-postgres-runtime",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:10Z",
"fresh_refresh": true
},
{
"id": "sso/identity-provisioner-token",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:10Z",
"fresh_refresh": true
},
{
"id": "sso/keycape-approval-engine-operator-client",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:11Z",
"fresh_refresh": true
},
{
"id": "sso/keycape-factor-read",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:11Z",
"fresh_refresh": true
},
{
"id": "sso/keycape-informed-decision-sitting-requester-client",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:11Z",
"fresh_refresh": true
},
{
"id": "sso/keycape-secrets-engine-approval-client",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:11Z",
"fresh_refresh": true
},
{
"id": "sso/keycape-secrets-engine-requester-client",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:11Z",
"fresh_refresh": true
},
{
"id": "state-hub/state-hub-rename-preflight",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:12Z",
"fresh_refresh": true
},
{
"id": "target-revenue/target-revenue-runtime",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:12Z",
"fresh_refresh": true
},
{
"id": "telemetry/telemetry-alert-smtp",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:12Z",
"fresh_refresh": true
},
{
"id": "telemetry/telemetry-grafana-admin",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:12Z",
"fresh_refresh": true
},
{
"id": "tenant-engine/tenant-engine-postgres-migration",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:12Z",
"fresh_refresh": true
},
{
"id": "tenant-engine/tenant-engine-postgres-runtime",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:13Z",
"fresh_refresh": true
},
{
"id": "user-engine/identity-provisioner-client",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:13Z",
"fresh_refresh": true
},
{
"id": "user-engine/user-engine-runtime",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:29:14Z",
"fresh_refresh": true
}
]
}

View file

@ -0,0 +1,284 @@
{
"phase": "before-binding",
"started_at": "2026-09-28T14:24:02.452772+00:00",
"checked_at": "2026-09-28T14:24:31.597651+00:00",
"total": 39,
"ready": 39,
"fresh": 39,
"complete": true,
"rows": [
{
"id": "activity-core/actcore-backup-offsite",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:05Z",
"fresh_refresh": true
},
{
"id": "activity-core/actcore-forgejo-admin",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:05Z",
"fresh_refresh": true
},
{
"id": "activity-core/actcore-issue-core-runtime",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:05Z",
"fresh_refresh": true
},
{
"id": "activity-core/actcore-ops-run-worker-tokens",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:05Z",
"fresh_refresh": true
},
{
"id": "activity-core/llm-connect-provider-secrets",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:05Z",
"fresh_refresh": true
},
{
"id": "approval-engine/approval-engine-audit",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:05Z",
"fresh_refresh": true
},
{
"id": "audit-core/audit-core-database",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:05Z",
"fresh_refresh": true
},
{
"id": "audit-core/audit-core-database-migrate",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:05Z",
"fresh_refresh": true
},
{
"id": "audit-core/audit-core-senders",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:05Z",
"fresh_refresh": true
},
{
"id": "canned-prompts/canned-prompts-postgres-migration",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:06Z",
"fresh_refresh": true
},
{
"id": "canned-prompts/canned-prompts-postgres-runtime",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:06Z",
"fresh_refresh": true
},
{
"id": "core-hub/core-hub-api-token",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:06Z",
"fresh_refresh": true
},
{
"id": "core-hub/core-hub-migration-database",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:06Z",
"fresh_refresh": true
},
{
"id": "core-hub/core-hub-runtime-database",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:06Z",
"fresh_refresh": true
},
{
"id": "core-hub/hub-core-migration-database",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:06Z",
"fresh_refresh": true
},
{
"id": "core-hub/hub-core-runtime-database",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:06Z",
"fresh_refresh": true
},
{
"id": "databases/platform-pg-backup-s3",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:06Z",
"fresh_refresh": true
},
{
"id": "email-connect/email-connect-runtime",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:06Z",
"fresh_refresh": true
},
{
"id": "forgejo/forgejo-mailer",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:06Z",
"fresh_refresh": true
},
{
"id": "informed-decision/informed-decision-audit",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:07Z",
"fresh_refresh": true
},
{
"id": "issue-core/issue-core-runtime",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:07Z",
"fresh_refresh": true
},
{
"id": "rapp-qonto/rapp-qonto",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:07Z",
"fresh_refresh": true
},
{
"id": "reuse/reuse-surface-runtime",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:07Z",
"fresh_refresh": true
},
{
"id": "sbom-nexus/sbom-nexus-postgres-migration",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:07Z",
"fresh_refresh": true
},
{
"id": "sbom-nexus/sbom-nexus-postgres-runtime",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:07Z",
"fresh_refresh": true
},
{
"id": "sso/identity-provisioner-token",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:07Z",
"fresh_refresh": true
},
{
"id": "sso/keycape-approval-engine-operator-client",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:07Z",
"fresh_refresh": true
},
{
"id": "sso/keycape-factor-read",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:08Z",
"fresh_refresh": true
},
{
"id": "sso/keycape-informed-decision-sitting-requester-client",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:08Z",
"fresh_refresh": true
},
{
"id": "sso/keycape-secrets-engine-approval-client",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:08Z",
"fresh_refresh": true
},
{
"id": "sso/keycape-secrets-engine-requester-client",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:08Z",
"fresh_refresh": true
},
{
"id": "state-hub/state-hub-rename-preflight",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:08Z",
"fresh_refresh": true
},
{
"id": "target-revenue/target-revenue-runtime",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:08Z",
"fresh_refresh": true
},
{
"id": "telemetry/telemetry-alert-smtp",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:08Z",
"fresh_refresh": true
},
{
"id": "telemetry/telemetry-grafana-admin",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:08Z",
"fresh_refresh": true
},
{
"id": "tenant-engine/tenant-engine-postgres-migration",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:08Z",
"fresh_refresh": true
},
{
"id": "tenant-engine/tenant-engine-postgres-runtime",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:09Z",
"fresh_refresh": true
},
{
"id": "user-engine/identity-provisioner-client",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:09Z",
"fresh_refresh": true
},
{
"id": "user-engine/user-engine-runtime",
"ready": true,
"has_template": true,
"refresh_time": "2026-09-28T14:24:09Z",
"fresh_refresh": true
}
]
}

View file

@ -0,0 +1,31 @@
{
"executed_at": "2026-09-28T16:07:36.040146+00:00",
"authority": "ADMINISTER @ realm:kubernetes/railiance01",
"activation": "APPROVED",
"authorization": "Founder explicitly selected: Delete only the orphan Secret",
"deleted": {
"kind": "Secret",
"namespace": "platform-pg-drill",
"name": "drill-minio",
"uid": "2fcb66df-d90f-4776-8ea0-8ca1a04bd307"
},
"uid_precondition_used": true,
"verified_absent": true,
"pvc_before": {
"uid": "f94df968-92d6-4f52-8e3a-3684ff7b064b",
"resource_version": "46926933",
"volume": "pvc-f94df968-92d6-4f52-8e3a-3684ff7b064b",
"storage": "3Gi",
"phase": "Bound"
},
"pvc_after": {
"uid": "f94df968-92d6-4f52-8e3a-3684ff7b064b",
"resource_version": "46926933",
"volume": "pvc-f94df968-92d6-4f52-8e3a-3684ff7b064b",
"storage": "3Gi",
"phase": "Bound"
},
"pvc_unchanged": true,
"other_resources_mutated": false,
"secret_values_read_or_archived": false
}

View file

@ -0,0 +1,18 @@
{
"captured_at": "2026-09-28T13:01:46.140808+00:00",
"namespace": "whitehat",
"fixture": "cust-0073-proof-7525cc730079",
"synthetic_only": true,
"checks": {
"clean_create_allowed": true,
"empty_annotated_create_denied": true,
"empty_annotated_update_denied": true,
"populated_annotated_create_denied": true,
"populated_annotated_update_denied": true,
"client_apply_denied": true,
"clean_server_apply_allowed": true,
"clean_update_allowed": true,
"fixture_removed": true
},
"passed": true
}

View file

@ -0,0 +1,18 @@
{
"captured_at": "2026-09-28T14:28:57.199281+00:00",
"namespace": "whitehat",
"fixture": "cust-0073-proof-e199ed6810eb",
"synthetic_only": true,
"checks": {
"clean_create_allowed": true,
"empty_annotated_create_denied": true,
"empty_annotated_update_denied": true,
"populated_annotated_create_denied": true,
"populated_annotated_update_denied": true,
"client_apply_denied": true,
"clean_server_apply_allowed": true,
"clean_update_allowed": true,
"fixture_removed": true
},
"passed": true
}

View file

@ -0,0 +1,17 @@
{
"captured_at": "2026-09-28T13:00:29.360819+00:00",
"namespace": "whitehat",
"fixture": "cust-0073-proof-3063da4fd6ff",
"synthetic_only": true,
"checks": {
"clean_create_allowed": true,
"empty_annotated_create_denied": true,
"empty_annotated_update_denied": true,
"populated_annotated_create_denied": true,
"populated_annotated_update_denied": true,
"client_apply_denied": true,
"clean_server_apply_allowed": false,
"fixture_removed": true
},
"passed": false
}

View file

@ -0,0 +1,68 @@
{
"captured_at": "2026-09-28T12:58:10.819938+00:00",
"mode": "clean",
"checked": 257,
"annotated": [
"approval-engine/approval-engine-audit",
"core-hub/core-hub-api-token",
"core-hub/core-hub-migration-database",
"core-hub/core-hub-runtime-database",
"core-hub/hub-core-migration-database",
"core-hub/hub-core-runtime-database",
"informed-decision/informed-decision-audit",
"rapp-qonto/rapp-qonto-runtime",
"reuse/reuse-surface-env",
"sso/authelia-secrets",
"sso/keycape-approval-engine-operator-client",
"sso/keycape-config",
"sso/keycape-factor-read",
"sso/keycape-informed-decision-sitting-requester-client",
"sso/keycape-pi-token",
"sso/keycape-rapp-qonto-client",
"sso/keycape-secrets-engine-approval-client",
"sso/keycape-secrets-engine-requester-client",
"sso/lldap-secrets",
"state-hub/state-hub-env",
"state-hub/state-hub-rename-preflight",
"target-revenue/target-revenue-pg-credentials",
"target-revenue/target-revenue-runtime",
"target-revenue/target-revenue-trf-app-credentials",
"telemetry/telemetry-alert-smtp",
"telemetry/telemetry-grafana-admin",
"user-engine/user-engine-delivery"
],
"cleaned": [
"approval-engine/approval-engine-audit",
"core-hub/core-hub-api-token",
"core-hub/core-hub-migration-database",
"core-hub/core-hub-runtime-database",
"core-hub/hub-core-migration-database",
"core-hub/hub-core-runtime-database",
"informed-decision/informed-decision-audit",
"rapp-qonto/rapp-qonto-runtime",
"reuse/reuse-surface-env",
"sso/authelia-secrets",
"sso/keycape-approval-engine-operator-client",
"sso/keycape-config",
"sso/keycape-factor-read",
"sso/keycape-informed-decision-sitting-requester-client",
"sso/keycape-pi-token",
"sso/keycape-rapp-qonto-client",
"sso/keycape-secrets-engine-approval-client",
"sso/keycape-secrets-engine-requester-client",
"sso/lldap-secrets",
"state-hub/state-hub-env",
"state-hub/state-hub-rename-preflight",
"target-revenue/target-revenue-pg-credentials",
"target-revenue/target-revenue-runtime",
"target-revenue/target-revenue-trf-app-credentials",
"telemetry/telemetry-alert-smtp",
"telemetry/telemetry-grafana-admin",
"user-engine/user-engine-delivery"
],
"orphaned_namespace": [
"platform-pg-drill/drill-minio"
],
"complete": false,
"active_namespace_scan_complete": true
}

View file

@ -0,0 +1,12 @@
{
"captured_at": "2026-09-28T14:24:09.996656+00:00",
"mode": "clean",
"checked": 257,
"annotated": [],
"cleaned": [],
"orphaned_namespace": [
"platform-pg-drill/drill-minio"
],
"complete": false,
"active_namespace_scan_complete": true
}

View file

@ -0,0 +1,16 @@
{
"captured_at": "2026-09-28T12:55:20.775101+00:00",
"mode": "clean",
"checked": 168,
"annotated": [
"approval-engine/approval-engine-audit",
"core-hub/core-hub-api-token",
"platform-pg-drill/drill-minio"
],
"cleaned": [
"approval-engine/approval-engine-audit",
"core-hub/core-hub-api-token"
],
"complete": false,
"error": "maintenance incomplete; raw output suppressed; inspect before retry"
}

View file

@ -0,0 +1,70 @@
{
"captured_at": "2026-09-28T12:52:14.833450+00:00",
"mode": "clean",
"checked": 168,
"annotated": [
"activity-core/actcore-runtime-secret",
"activity-core/llm-connect-provider-secrets",
"approval-engine/approval-engine-audit",
"audit-core/audit-core-senders",
"core-hub/core-hub-api-token",
"core-hub/core-hub-migration-database",
"core-hub/core-hub-runtime-database",
"core-hub/hub-core-migration-database",
"core-hub/hub-core-runtime-database",
"coulomb/ihp-railiance-probe-env",
"databases/net-kingdom-pg-privacyidea-app",
"databases/platform-pg-backup-s3",
"databases/platform-pg-bootstrap",
"databases/state-hub-db-credentials",
"email-connect/email-connect-runtime",
"external-secrets/openbao-audit-core-approle",
"external-secrets/openbao-backup-object-storage-approle",
"external-secrets/openbao-rapp-qonto-approle",
"external-secrets/openbao-sso-user-engine-runtime-approle",
"external-secrets/openbao-user-engine-runtime-approle",
"forgejo/forgejo-mailer",
"forgejo/forgejo-runner-registration",
"informed-decision/informed-decision-audit",
"knative-serving/webhook-certs",
"mfa/privacyidea-auditkeys",
"mfa/privacyidea-config",
"mfa/privacyidea-enckey",
"mfa/privacyidea-trigger-admin",
"openbao/bao-tls",
"platform-pg-drill/drill-minio"
],
"cleaned": [
"activity-core/actcore-runtime-secret",
"activity-core/llm-connect-provider-secrets",
"approval-engine/approval-engine-audit",
"audit-core/audit-core-senders",
"core-hub/core-hub-api-token",
"core-hub/core-hub-migration-database",
"core-hub/core-hub-runtime-database",
"core-hub/hub-core-migration-database",
"core-hub/hub-core-runtime-database",
"coulomb/ihp-railiance-probe-env",
"databases/net-kingdom-pg-privacyidea-app",
"databases/platform-pg-backup-s3",
"databases/platform-pg-bootstrap",
"databases/state-hub-db-credentials",
"email-connect/email-connect-runtime",
"external-secrets/openbao-audit-core-approle",
"external-secrets/openbao-backup-object-storage-approle",
"external-secrets/openbao-rapp-qonto-approle",
"external-secrets/openbao-sso-user-engine-runtime-approle",
"external-secrets/openbao-user-engine-runtime-approle",
"forgejo/forgejo-mailer",
"forgejo/forgejo-runner-registration",
"informed-decision/informed-decision-audit",
"knative-serving/webhook-certs",
"mfa/privacyidea-auditkeys",
"mfa/privacyidea-config",
"mfa/privacyidea-enckey",
"mfa/privacyidea-trigger-admin",
"openbao/bao-tls"
],
"complete": false,
"error": "maintenance incomplete; raw output suppressed; inspect before retry"
}

View file

@ -0,0 +1,18 @@
{
"observed_at": "2026-09-28T14:30:57.341819+00:00",
"application_revision": "7daf7e90675b21c8f9556028e54aa8c0a13fd9f0",
"application_declaration_commit": "db51ec802801ddf85a80bf81980865c9f9239839",
"sync": "Synced",
"health": "Healthy",
"operation_phase": "Succeeded",
"binding_present": true,
"validation_actions": [
"Deny"
],
"policy_type_checking": {},
"policy_observed_generation": 1,
"policy_generation": 1,
"externalsecrets_total": 39,
"externalsecrets_ready": 39,
"externalsecrets_with_template": 39
}

View file

@ -0,0 +1,10 @@
{
"captured_at": "2026-09-28T16:08:03.695223+00:00",
"mode": "inspect",
"checked": 257,
"annotated": [],
"cleaned": [],
"orphaned_namespace": [],
"complete": true,
"active_namespace_scan_complete": true
}

View file

@ -0,0 +1,22 @@
{
"captured_at": "2026-09-28T13:00:47.979858+00:00",
"mode": "clean",
"checked": 257,
"annotated": [
"sso/keycape-approval-engine-operator-client",
"sso/keycape-factor-read",
"sso/keycape-secrets-engine-approval-client",
"state-hub/state-hub-rename-preflight"
],
"cleaned": [
"sso/keycape-approval-engine-operator-client",
"sso/keycape-factor-read",
"sso/keycape-secrets-engine-approval-client",
"state-hub/state-hub-rename-preflight"
],
"orphaned_namespace": [
"platform-pg-drill/drill-minio"
],
"complete": false,
"active_namespace_scan_complete": true
}

View file

@ -0,0 +1,10 @@
{
"observed_at": "2026-09-28T13:17:32.061629+00:00",
"application_revision": "6016f72a8db26df1eed7b7b9f3b36c8a0eb9f3b7",
"sync": "Synced",
"health": "Healthy",
"operation_phase": "Succeeded",
"binding_present": false,
"externalsecrets_total": 39,
"externalsecrets_ready": 39
}

View file

@ -0,0 +1,147 @@
# Secret annotation guard: rollout, failed integration and rollback
CUST-WP-0073-T03, September 28, 2026. The founder authorized continuing the
existing workplans. This receipt records an unsuccessful rollout with recovery;
it is not evidence for promoting the agent to autopilot.
## Source and deployment
The platform owner source added the native policy/binding and safe maintenance
helper in `railiance-platform@800cbfa870661d47bee34c747f04f6145875adf1`.
Application commit `54885ac1589074a68d9607d1be250c84c5c2307a` pinned that revision.
The AppProject allowlist gained only the two admission kinds. Publication used
repo-manager; deployment used manual Argo resource-scoped sync, without syncing
unrelated root changes. The application has no automated sync or finalizer.
Policy type checking passed. The first synthetic proof failed on an SSA field
ownership conflict; its failed receipt is retained. The corrected proof tests
SSA of the same value followed by a clean update. All nine native checks passed:
clean create/update/SSA; annotated create/update rejected, including empty
values; client-side apply rejected; synthetic fixture removed. These checks did
not establish compatibility with existing controllers.
## Actual integration failure
ESO v0.16.1 copied the ExternalSecret source last-applied annotation back onto
its target when no target template existed. Under Deny enforcement, required
Secret refreshes failed (ten ExternalSecrets observed in SecretSyncedError).
31 live ExternalSecrets lacked an explicit template. The implementation is
visible in the [installed-version upstream source](https://github.com/external-secrets/external-secrets/blob/v0.16.1/pkg/controllers/externalsecret/externalsecret_controller_template.go):
without a target template the controller copies source metadata; with one it
uses template metadata. Merely removing annotations from the targets does not
fix this writer behavior.
The binding was deleted promptly to restore refreshes. Failed ExternalSecrets
were explicitly force-refreshed. All 39 became Ready. Source rollback commit
`6016f72a8db26df1eed7b7b9f3b36c8a0eb9f3b7` removes the binding from kustomization
and keeps it in `binding.pending.yaml`. Application commit
`c5d65b0b405be9ce5624f49c6f6df54c12b38735` pins that policy-only revision.
Both were published using repo-manager; the root was synced only for this
Application, then the child synced to its policy-only revision.
Final read-back at 13:17:32 UTC: application Synced/Healthy, operation Succeeded,
binding absent, 39/39 ExternalSecrets Ready. See
`2026-09-28-secret-annotation-rollback.json`. The policy remains installed but
UNBOUND. A normal sync of the pinned source cannot re-enable enforcement.
## Cleanup and limits
The recorded passes removed the annotation from 49 distinct Secrets in active
namespaces. Some were cleaned again after ESO recreated the annotation. This is
not a claim that all remain annotation-free after rollback. The helper changed
only that metadata key, never Secret data. ESO recovery performs its normal
refresh behavior; no credential rotation was performed by this work.
`platform-pg-drill/drill-minio` is orphaned: its namespace is absent, so the API
refuses the metadata patch. A referencing Deployment, a PVC and Service also
remain without that namespace. No orphan was deleted or namespace recreated.
Cluster-wide cleanup is incomplete. Disposition remains under existing T03.
Kubectl subprocess output was captured and suppressed throughout the helper.
The old raw presence template failed on absent annotations; its error was
suppressed rather than exposing a Secret dump. The replacement iterates keys
and handles absent/empty maps. Orientation §6 now requires the safe helper.
## Remaining work in T03
Before re-enabling the strict guard, explicitly set target metadata in the 31
owning ExternalSecret declarations while preserving intended labels/annotations
and all existing data templates. Verify actual controller refresh and clean
resulting target metadata, repeat cleanup and synthetic checks, then verify
ESO refresh with enforcement enabled. No ESO exemption or controller upgrade
is proposed. The owner source changes and orphan disposition remain unfinished;
no additional workplan or task has been created.
Receipts alongside this file: `secret-annotation-cleanup.json`,
`secret-annotation-cleanup-retry.json`, `secret-annotation-cleanup-active.json`,
`secret-annotation-post-binding.json`, `secret-annotation-admission-proof.json`,
`secret-annotation-admission-proof-final.json`, and
`secret-annotation-rollback.json`, all prefixed `2026-09-28-`.
## Corrected rollout — September 28 follow-up
The founder instructed “Good, go on” after the 31-declaration remediation was
identified. Explicit target metadata was added to 31 ExternalSecrets in 23 files
across 12 owning repositories. Source labels and intentional annotations remain;
controller bookkeeping and last-applied are not inherited. Data mappings, data
templates, store references, creation/deletion policies and refresh intervals
were unchanged. Server-side dry-run and semantic comparison verified this for
all 31. A canary refreshed successfully and removed its copied annotation.
All source changes were committed and published. Eleven repositories have exact
primary repo-manager receipts. activity-core's repo-manager registration refused
pre-existing historical workplan IDs; its documented `statehub fix-consistency`
path passed with warnings and pushed the exact metadata commit, without changing
those historical file IDs. See `2026-09-28-eso-metadata-publication.json` and
`2026-09-28-eso-metadata-changes.json`. Required user-engine checks passed (four
tests); telemetry pinned-chart fetch/check and family validation passed (one
pre-existing declaration warning).
Thirty non-Argo-managed ExternalSecrets received metadata-only server-side
apply through the existing SSH admin path. Target Revenue's ExternalSecret used
a selective Argo sync at `a3a8a27a8cda32ae3c7b55353ee16a131c50a0a0`, declared by
platform commit `d2631f6112fca8f374b37aa52bc34400c12c95e1`. The operation result
lists only that ExternalSecret; no migration/bootstrap hook or workload rollout
was run. Its application is Synced and Healthy.
All 39 ExternalSecrets completed fresh successful refreshes before enforcement.
A complete active-namespace scan checked 257 Secrets and found no last-applied
annotations; ESO had removed the formerly inherited metadata. The absent-namespace
orphan remained separately reported.
Guard source `7daf7e90675b21c8f9556028e54aa8c0a13fd9f0` includes `binding.yaml`.
Application commit `db51ec802801ddf85a80bf81980865c9f9239839` pins that source.
Both were published through repo-manager. Selective root/child Argo sync enabled
the binding without unrelated app changes. At 14:30:57 UTC the guard was
Synced/Healthy, the binding was present with Deny, and policy type checking was
clear at observed generation 1. Nine native admission checks passed again.
**All 39 ExternalSecrets then completed fresh successful refreshes under Deny.**
Receipts: `2026-09-28-secret-annotation-enforced.json`,
`2026-09-28-secret-annotation-admission-proof-reenabled.json`, and
`2026-09-28-eso-refresh-{before,after}-binding.json`.
The old rollout failure remains a failed original proposal requiring refinement
and recovery. Successful remediation does not retroactively earn unchanged
execution credit. No agent promotion, credential rotation or interactive-runtime
cutover is claimed.
The remaining orphan is the August 13 Secret
`platform-pg-drill/drill-minio`, UID `2fcb66df-d90f-4776-8ea0-8ca1a04bd307`.
The namespace is absent and has no pods. A reviewable UID-bound proposal to
delete only that Secret is in `docs/changes/CUST-WP-0073/orphan-secret-deletion.json`.
Explicit deletion approval is pending; the bound 3 GiB PVC and all other resources
are outside that proposal. No deletion has occurred.
### Approved orphan cleanup
The founder explicitly approved deleting only the orphan Secret. The API accepted
the DELETE with UID precondition `2fcb66df-d90f-4776-8ea0-8ca1a04bd307`; a fresh
identity inventory verified absence. The 3 GiB PVC retained its exact UID,
resourceVersion, volume and Bound status. No other resources or the namespace
were changed, and no Secret values were read or archived. Receipt:
`2026-09-28-orphan-secret-deletion.json`. This resolves the cleanup exception
and completes CUST-WP-0073-T03; the earlier pending-deletion text is historical.
Final complete cluster-wide scan after deletion: 257 Secrets checked, zero
forbidden annotations, zero orphan exceptions, helper exit 0. Receipt:
`2026-09-28-secret-annotation-final-scan.json`.

View file

@ -0,0 +1,12 @@
{
"captured_at": "2026-09-28T14:29:51.628307+00:00",
"mode": "inspect",
"checked": 257,
"annotated": [],
"cleaned": [],
"orphaned_namespace": [
"platform-pg-drill/drill-minio"
],
"complete": false,
"active_namespace_scan_complete": true
}

View file

@ -0,0 +1,94 @@
# CUST-WP-0071 — allocation review, 2026-09-28
## Recommendation
Keep the accepted Vergabe pilot at 60m CPU / 256Mi memory requests and
1 CPU / 1Gi limits. Keep the already reduced Knative allocations. Propose no
new live allocation change from this sample. This is a provisional pilot
recommendation, not acceptance of representative user performance.
The node has reservation room but little measured processing room at the
snapshot: 3,420m requested of 4,000m, zero pending requests, **3,963m observed
CPU** and 12,075,401,216 bytes observed memory. The 580m reservation residual
must not be called spare processing capacity. Avoid admitting concurrent builds
on the strength of that residual alone.
## Evidence and coverage
- Node verified as `92.205.62.239`, k3s v1.35.1+k3s1.
- `2026-09-28-allocation-reconcile.json` and `.md`: existing collector plus
updated namespace owners; same hardware counted once. State Hub release
preflight passes at this observation, not as a standing admission grant.
- `2026-09-28-cluster-observation.json`: retained source observation, including
instantaneous demand. Collector revisions are recorded in the companion
provenance file. No Secret objects were queried.
- Collector limitation checked against active pods: no pod-level resources,
overhead or restartable init containers were present. Its simplified init
accounting therefore did not encounter these unsupported features today.
This does not certify its accounting for future workloads.
- `2026-09-28-allocation-telemetry.json`: exact PromQL and responses for seven
days, evaluated at five-minute resolution, namespace aggregates. The five
namespaces below each have 2,016 CPU evaluation points. These are evaluation
points, not proof of complete raw scrape coverage or representative traffic.
| Namespace | CPU p95 (m) | CPU peak (m) | Memory peak (MiB) |
|---|---:|---:|---:|
| vergabe-demo-company | 0.52 | 20.10 | 191.14 |
| knative-serving | 7.75 | 8.54 | 273.54 |
| kourier-system | 3.88 | 4.06 | 59.00 |
| forgejo | 1454.35 | 2208.27 | 4511.85 |
| databases | 238.08 | 341.43 | 1155.20 |
Namespace peaks need not be simultaneous and cannot be added into a node peak.
Forgejo includes its runner; the databases row is shared demand, not an estimate
of Vergabe's incremental database cost. Namespace aggregates can hide missing
individual pod series. Peak means the maximum sampled value, not every burst.
Vergabe's throttled-period ratio is 0.000106 (about 0.0106%); the restart counter
query reports zero increase for the namespaces above. Counter evidence is not
proof that deleted or recreated pods never restarted. Forgejo's throttle ratio
is absent; it is not zero. Host CPU history, Vergabe HTTP request/latency series,
and the Forgejo namespace CPU-request recording remain absent in these queries.
Live Vergabe image:
`forgejo.coulomb.social/coulomb/vergabe-teilnahme@sha256:a26444f59c259698159c69ccb96f73dc648a261ece4c86bb2037a9d977870d91`.
One ready replica, 60m/1 CPU and 256Mi/1Gi. No customer data was written.
## Existing work replaces duplicate changes
`RAIL-KNATIVE-WP-0002` finished on September 27. Its T03 verifies that the
railiance-cluster installer now preserves the reduced requests. The stale inbox
warning about the installer reverting them is superseded by that file evidence.
`RAIL-EN-WP-0002` is also finished: ArgoCD resources are already declared and
applied. Neither warrants another task here.
T03 retains review of Forgejo/runner demand, twelve zero-request workloads and
the distinction between release reservations and real CPU contention. There is
no approved new sizing change for T04 to deploy today. T04 must verify the
accepted final recommendation, including an explicit keep decision if supported,
rather than manufacture a resize to satisfy its title.
## Smallest remaining execution
Use the existing T02 for one bounded synthetic pilot session, with product-owner
response/error targets, two concurrent users, document round-trip, edits and
restart evidence. Reuse the invited-pilot fixture; do not create a load-test
service. Link the existing RAPPS-WP-0014-T03 acceptance work rather than duplicate
its data-recovery tasks. The seven-day idle/light-use sample is useful input but
does not replace this session.
T03/T04 then resolve a single allocation recommendation and verify only accepted
changes. Preserve a 160m reservation envelope for the current State Hub
100m API + 10m MCP + 50m migration requests, and account separately for scheduled
maintenance and competing releases. This is a review assumption, not a global
admission policy or evidence that the node has 160m spare execution capacity.
T05 reuses activity-core's durable scheduler: Monday 08:00 Europe/Berlin,
Custodian review ownership, retained reports and State Hub progress delivery.
Retain the exact queries with every report; missing signals stay unknown.
The minimum first report covers keep/investigate decisions above, allocation,
sample coverage and links to these existing tasks. The first scheduled run,
acknowledgment and missed-run/recovery evidence are still required. No weekly
schedule was installed by this review; no unattended receipt is claimed.
No new task, workplan, intake, monitoring service or resource mutation was made.

View file

@ -0,0 +1,11 @@
{
"scenario_type": "cross_owner_wait",
"case_id": "CUST-WP-0073-T02--GLAS-WP-0012",
"obligor_workplan_id": "GLAS-WP-0012",
"beneficiary_workplan_id": "CUST-WP-0073",
"state": "waiting",
"reason": "Hub confirms GLAS-WP-0012 blocked. The existing local profile lacks end-to-end production acceptance; standalone bwrap process proof is insufficient for interactive agent migration. No worker injected or owner task reassigned.",
"flavor": "implementation",
"observer": "the-custodian",
"next_action": "Observe existing GLAS-WP-0012 acceptance receipt before relying on its runtime; verify actual agent admin-path denial under CUST-WP-0073-T02."
}

View file

@ -0,0 +1,20 @@
{
"captured_at": "2026-09-28T12:50:24.073685+00:00",
"workplan_task": "CUST-WP-0073-T02",
"profile": "profile.bwrap-local",
"model_called": false,
"interactive_agent_migrated": false,
"sandbox_id": "0e96c810",
"checks": {
"admin_paths_absent": true,
"admin_environment_absent": true,
"only_loopback_interface": true,
"approved_observation_readable": true,
"proposal_preparation_works": true,
"wrong_consumer_denied": true,
"workspace_removed": true,
"destroyed": true,
"host_source_unchanged": true
},
"passed": true
}

View file

@ -1,8 +1,10 @@
# Namespace → owner/service mapping for CUST-WP-0071-T01.
# Unknown namespaces stay unknown; do not invent tenants.
namespaces:
knative-serving: {owner: rail-kubernetes, service: knative-serving, tenant: unknown}
kourier-system: {owner: rail-kubernetes, service: kourier, tenant: unknown}
knative-serving: {owner: rail-knative, service: knative-serving, tenant: unknown}
kourier-system: {owner: rail-knative, service: kourier, tenant: unknown}
argocd: {owner: railiance-enablement, service: argocd, tenant: unknown}
bao-notice: {owner: railiance-platform, service: bao-notice, tenant: unknown}
kube-system: {owner: railiance-cluster, service: k3s-control-plane, tenant: unknown}
cert-manager: {owner: railiance-cluster, service: cert-manager, tenant: unknown}
external-secrets: {owner: railiance-platform, service: external-secrets, tenant: unknown}