Add independent net-kingdom security progress assessment

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

history/20261011-netkingdom-security-progress-assessment.md

@@ -0,0 +1,166 @@
+# Net-Kingdom security integration: independent progress assessment
+
+- **Timestamp:** 2026-10-11
+- **Author:** Claude Code (Opus 5.5), independent review requested by Bernd
+- **Subject:** the GPT-6.1 Sol goal "consolidate and finish the open railiance-platform /
+  net-kingdom security workplans across access-engine, secrets-engine, ops-mason,
+  ops-warden, key-cape, user-engine, tenant-engine, gate-house, kings-guard, using
+  informed-decisions and policy-nexus guardrails". Reported spend so far: 18h 18m, 16.9M tokens.
+- **Method:** repo workplan scan (frontmatter + task blocks), 3-day git logs, State Hub
+  `/decisions/`. Counts are a snapshot from 2026-10-11 and are approximate (grep-based).
+
+## 1. Short version
+
+The agent is **busy but not converging**. In three days it made about 520 commits across
+the security repos, and the open task count is still about 98. Of those, **about 79 (≈80 %)
+are `wait`**, 39 are explicit human gates, and none uses `wait_until`. Most commit messages
+describe paperwork ("Retain…", "Record…", "Prepare…", "Freeze…packet", "exact historical
+owner-review revisions", "generated projection refresh"). They do not describe a capability
+going live. The binding constraint is no longer agent throughput. It is **a small number
+of founder/operator actions plus live-system steps that no agent can take**. Adding more
+agent time on top of that mostly produces more evidence about the same blockage.
+
+**Recommendation:** pause autonomous breadth. Run one attended "unblock session" against
+the 3–4 decisions that gate most wait tasks. Then turn the repeated evidence and
+consistency work into deterministic tooling (Section 4), and restart agents only on tasks
+whose next step is code or a live change.
+
+## 2. Snapshot
+
+| Repo | Open WPs | Open tasks | of which wait | Commits (3 d) |
+|---|---:|---:|---:|---:|
+| railiance-platform | 9 | 30 | 25 | 88 |
+| net-kingdom | 7 | 17 | 17 | 72 |
+| access-engine | 4 | 14 | 6 | 41 |
+| ops-mason | 5 | 8 | 5 | 87 |
+| ops-warden | 5 | 8 | 7 | 45 |
+| secrets-engine | 5 | 7 | 6 | 21 |
+| user-engine | 5 | 7 | 7 | 28 |
+| key-cape | 2 | 3 | 3 | 29 |
+| tenant-engine | 1 | 2 | 2 | 1 |
+| gate-house | 1 | 1 | 0 | 8 |
+| kings-guard | 1 | 1 | 1 | 3 |
+| policy-nexus | 0 | 0 | 0 | 14 |
+| **Total** | **45** | **~98** | **~79** | **~437 (+ custodian)** |
+
+Changed-file mix over 3 days: md and json dominate everywhere except access-engine (104 go
+files). For example, railiance-platform changed 109 json + 100 md files against 71 py,
+and ops-mason 109 md against 35 py. Only access-engine shows a code-heavy profile.
+
+The hub has 61 open decisions. The decision ids cited most in wait tasks:
+
+| Decision | Tasks citing it | Hub state | Meaning |
+|---|---:|---|---|
+| `89f1058f` ArgoCD owns production go-ahead | 11 | open | One founder go unblocks GitOps lane incl. policy-nexus |
+| `62aa95e4` Attended OpenBao provisioning window | 9 | open | One attended session clears credential lanes |
+| `1591daa1` Trust root / unseal & recovery custody | 4 | open | Founder design decision, gates seal/break-glass work |
+| `caafe4be` secrets-engine-openbao client live go | 4 | **resolved** | Tasks still wait: check whether the remaining blocker is the platform Secret, not the decision |
+| `d426c67e`, `b005b483` | 11, 9 | not found as hub decisions | Probably local/task ids: verify they resolve to something actionable |
+
+So **about 35 wait tasks (~45 %) hang on three open decisions and one resolved one.**
+
+## 3. Diagnosis
+
+1. **Evidence inflation, not progress.** The agent seems to be optimizing for being
+   "verified, retained, exact" at each checkpoint. You are in build mode
+   (see `user_build_mode_security_stance`), so independently reviewed source-acceptance
+   packets for things that are not yet running cost more than they return. One evidence
+   record per *live* state change is enough. Packets about packets are not needed.
+2. **Waiting is being done by an agent.** Each loop re-reads wait tasks, writes
+   "still waiting / prepared request" commits and re-syncs. Observing whether a foreign
+   task or decision has become terminal is a pure function of hub state. Derived wait
+   states (CUST-WP-0074, C-39..41) already exist for exactly this.
+3. **Human gates are scattered across tasks.** 39 `needs_human` tasks, 41 `decision_id`s,
+   but only a handful of real founder questions. The prepared decision packets
+   (`history/20261003-security-gate-packets.md`, `docs/decision-packets-2026-10-03.md`)
+   were not turned into a resolved state. A batch of informed decisions would remove most
+   of the blockage.
+4. **Owner/role disputes are routed as messages and re-checked by agents**
+   (e.g. "activity-core replied (not accepted yet)", "ops-mason has published no stance map",
+   "audit-core must publish…"). These are contract gaps. Each one needs a schema plus an
+   owner, not repeated agent negotiation.
+5. **policy-nexus has no workplan.** It is supposed to be the guardrail place, but it has
+   14 commits and no open plan, and its production go is itself behind `89f1058f`. Until
+   guardrails exist there as machine-checkable policy, agents keep re-deriving them in prose.
+6. **Concurrency is not the lever.** With 80 % of tasks waiting, five parallel workers
+   mostly contend over the same blocked lanes and produce consistency sync churn
+   ("chore(consistency): sync task status from DB [auto]" appears repeatedly).
+
+## 4. Where deterministic functions should replace agents
+
+| Activity the agent currently does | Deterministic replacement | Owner / where |
+|---|---|---|
+| Polling wait tasks, writing "still waiting" records | Hub-derived `waiting-external` / `needs-human` (exists). Add a nightly job that lists *newly unblocked* tasks and emails/posts them. Agents start only on that list. | state-hub (C-39..41) + activity-core trigger |
+| Consistency syncs, index/projection refreshes, "generated projection refresh" commits | `statehub fix-consistency` + generators run in CI or a pre-commit hook. Never as a separate agent commit. | each repo's Makefile / Forgejo Actions |
+| Source-inventory, work-record index, publication.json refresh | `make inventory` that is idempotent and checked by `--check` in CI (policy-nexus already has `source-inventory.json`, so reuse that tool fleet-wide). | policy-nexus `tools/` as shared lib |
+| "Independently reviewed" source-acceptance packets | Test suites + schema validation + signed CI run as the evidence. Human/LLM review only for design changes. | per repo |
+| Checking an owner published a contract (stance map, tenancy.yaml metadata, case-type pin) | JSON-Schema/CUE contract files plus a cross-repo conformance check (`make contract-check`) that fails with the missing field. | net-kingdom canon schemas; reuse-surface |
+| KnownSecurityCase execution (oidc-browser-relying-party etc.) | Already designed as deterministic: ops-mason renders → ops-warden applies. Make it one command per case type, idempotent, with a dry-run diff. | ops-mason, ops-warden |
+| Policy guardrails written as prose in workplans | Policies as code in policy-nexus (OPA/Rego or CUE) evaluated in CI and by access-engine at runtime. | policy-nexus |
+| Credential routing questions | `warden route find` (exists). Agents must call it, not message owners. | ops-warden |
+| Live checks (token capabilities, login proofs, lease revoke) | Scripted smoke tests (`make smoke-<lane>`) run by the operator in the attended window, which emit one machine-readable receipt. | railiance-platform |
+
+What still deserves an agent: design work where intent is unclear (trust-root proposal,
+tenant tier contract), writing the generators and checks above, code in
+access-engine/user-engine/key-cape, and drafting informed-decision packets.
+
+## 5. Intended collaboration (target shape, for re-anchoring the agent)
+
+- **key-cape** (Keycloak) = authentication and issuers. **user-engine** = user lifecycle and
+  self-service (MFA enrollment). **tenant-engine** = tenant/tier/issuer ownership.
+- **access-engine** = authorization decisions and freshness enforcement, fed by
+  **policy-nexus** policies (guardrails as code).
+- **secrets-engine** over OpenBao (railiance-platform hosts it) = secret custody/leases.
+  **ops-warden** = SSH certificates and credential *routing* only.
+- **ops-mason** = renders SecurityCaseType → KnownSecurityCase plans. **ops-warden** /
+  platform GitOps (ArgoCD) apply them.
+- **gate-house** = ingress/edge enforcement. **kings-guard** = detection/guard posture.
+- **net-kingdom** = canon (case types, scenarios, posture). **railiance-platform** = where it runs.
+
+One identity → authz → secrets → case-execution path proven end to end beats nine
+partially accepted lanes.
+
+## 6. Next steps (in order)
+
+1. **Stop the current autonomous loop** after its in-flight commits. Freeze new
+   "record/retain/prepare" packet work.
+2. **Attended unblock session (≈2 h, Bernd + one agent):**
+   - resolve `89f1058f` (ArgoCD production go, incl. policy-nexus);
+   - run the `62aa95e4` OpenBao provisioning window, using scripted smoke tests that emit receipts;
+   - decide `1591daa1` trust root. Accept a build-mode default (single holder, documented
+     re-arm item) instead of a final quorum design;
+   - check the 4 tasks still citing resolved `caafe4be`, and the unknown ids `d426c67e` / `b005b483`.
+   Expected effect: ~35 wait tasks become actionable or cancelable.
+3. **Triage cancel pass (deterministic list, human approve):** cancel or demote to
+   `flavor: residual` every wait task whose blocker is "the next type follows once the
+   first is proven", adoption triggers with no adopter, or an enterprise feature (SAML
+   federation NK-WP-0011, tutorials NK-WP-0009) that is not on the critical path. Target
+   fewer than 40 open tasks.
+4. **Open a policy-nexus workplan** for the guardrails as code (build-mode baseline +
+   re-arm list), consumed by access-engine and CI.
+5. **Build the deterministic layer from Section 4** as one small cross-repo workplan:
+   unblocked-task notifier, `contract-check`, CI-run consistency/inventory, per-lane smoke
+   receipts. This is good agent work: bounded, testable, done once.
+6. **Prove one end-to-end case**: oidc-browser-relying-party as the first KnownSecurityCase
+   (NK-WP-0048-T06) through ops-mason → ops-warden/ArgoCD, with login, revocation and audit
+   receipt. Use it as the template for the rest.
+7. **Restart agents narrowly**: at most 2–3 workers, each pinned to one workplan whose next
+   task is `todo` (not `wait`), each with a clear done-test. Use `/ralph-workplan`-style
+   self-retirement, and stop when a task turns into `wait` instead of documenting the wait.
+
+## 7. Efficiency guardrails for the coding agent's goal text
+
+Add these to the Sol goal:
+
+- "Do not create evidence/packet/index commits that do not accompany a code or live-state change."
+- "If the next step of a task needs a human or another owner, set `wait` with qualifiers once and move on. Do not revisit it until the hub reports it unblocked."
+- "Batch all founder questions into one informed-decision packet per day, with a recommended default."
+- "Prefer writing a script or check that makes a class of tasks deterministic over doing the tasks by hand."
+- "Success metric = open task count going down and live capabilities proven, not commits."
+
+## 8. Caveats
+
+Counts come from grep over workplan files and may include archived or misformatted blocks.
+"Paperwork vs code" is inferred from commit subjects and file extensions, not from reading
+every diff. Some packet work may be legitimately required by canon review gates. Two
+frequently cited ids could not be resolved to hub decisions.