From 844ed5812e50990edcb643e65eee1b50af971e83 Mon Sep 17 00:00:00 2001 From: codex Date: Sun, 11 Oct 2026 20:09:38 +0200 Subject: [PATCH] Add independent net-kingdom security progress assessment Co-Authored-By: Claude Opus 5.5 --- ...netkingdom-security-progress-assessment.md | 166 ++++++++++++++++++ 1 file changed, 166 insertions(+) create mode 100644 history/20261011-netkingdom-security-progress-assessment.md diff --git a/history/20261011-netkingdom-security-progress-assessment.md b/history/20261011-netkingdom-security-progress-assessment.md new file mode 100644 index 00000000..39d6434d --- /dev/null +++ b/history/20261011-netkingdom-security-progress-assessment.md @@ -0,0 +1,166 @@ +# Net-Kingdom security integration: independent progress assessment + +- **Timestamp:** 2026-10-11 +- **Author:** Claude Code (Opus 5.5), independent review requested by Bernd +- **Subject:** the GPT-6.1 Sol goal "consolidate and finish the open railiance-platform / + net-kingdom security workplans across access-engine, secrets-engine, ops-mason, + ops-warden, key-cape, user-engine, tenant-engine, gate-house, kings-guard, using + informed-decisions and policy-nexus guardrails". Reported spend so far: 18h 18m, 16.9M tokens. +- **Method:** repo workplan scan (frontmatter + task blocks), 3-day git logs, State Hub + `/decisions/`. Counts are a snapshot from 2026-10-11 and are approximate (grep-based). + +## 1. Short version + +The agent is **busy but not converging**. In three days it made about 520 commits across +the security repos, and the open task count is still about 98. Of those, **about 79 (≈80 %) +are `wait`**, 39 are explicit human gates, and none uses `wait_until`. Most commit messages +describe paperwork ("Retain…", "Record…", "Prepare…", "Freeze…packet", "exact historical +owner-review revisions", "generated projection refresh"). They do not describe a capability +going live. The binding constraint is no longer agent throughput. It is **a small number +of founder/operator actions plus live-system steps that no agent can take**. Adding more +agent time on top of that mostly produces more evidence about the same blockage. + +**Recommendation:** pause autonomous breadth. Run one attended "unblock session" against +the 3–4 decisions that gate most wait tasks. Then turn the repeated evidence and +consistency work into deterministic tooling (Section 4), and restart agents only on tasks +whose next step is code or a live change. + +## 2. Snapshot + +| Repo | Open WPs | Open tasks | of which wait | Commits (3 d) | +|---|---:|---:|---:|---:| +| railiance-platform | 9 | 30 | 25 | 88 | +| net-kingdom | 7 | 17 | 17 | 72 | +| access-engine | 4 | 14 | 6 | 41 | +| ops-mason | 5 | 8 | 5 | 87 | +| ops-warden | 5 | 8 | 7 | 45 | +| secrets-engine | 5 | 7 | 6 | 21 | +| user-engine | 5 | 7 | 7 | 28 | +| key-cape | 2 | 3 | 3 | 29 | +| tenant-engine | 1 | 2 | 2 | 1 | +| gate-house | 1 | 1 | 0 | 8 | +| kings-guard | 1 | 1 | 1 | 3 | +| policy-nexus | 0 | 0 | 0 | 14 | +| **Total** | **45** | **~98** | **~79** | **~437 (+ custodian)** | + +Changed-file mix over 3 days: md and json dominate everywhere except access-engine (104 go +files). For example, railiance-platform changed 109 json + 100 md files against 71 py, +and ops-mason 109 md against 35 py. Only access-engine shows a code-heavy profile. + +The hub has 61 open decisions. The decision ids cited most in wait tasks: + +| Decision | Tasks citing it | Hub state | Meaning | +|---|---:|---|---| +| `89f1058f` ArgoCD owns production go-ahead | 11 | open | One founder go unblocks GitOps lane incl. policy-nexus | +| `62aa95e4` Attended OpenBao provisioning window | 9 | open | One attended session clears credential lanes | +| `1591daa1` Trust root / unseal & recovery custody | 4 | open | Founder design decision, gates seal/break-glass work | +| `caafe4be` secrets-engine-openbao client live go | 4 | **resolved** | Tasks still wait: check whether the remaining blocker is the platform Secret, not the decision | +| `d426c67e`, `b005b483` | 11, 9 | not found as hub decisions | Probably local/task ids: verify they resolve to something actionable | + +So **about 35 wait tasks (~45 %) hang on three open decisions and one resolved one.** + +## 3. Diagnosis + +1. **Evidence inflation, not progress.** The agent seems to be optimizing for being + "verified, retained, exact" at each checkpoint. You are in build mode + (see `user_build_mode_security_stance`), so independently reviewed source-acceptance + packets for things that are not yet running cost more than they return. One evidence + record per *live* state change is enough. Packets about packets are not needed. +2. **Waiting is being done by an agent.** Each loop re-reads wait tasks, writes + "still waiting / prepared request" commits and re-syncs. Observing whether a foreign + task or decision has become terminal is a pure function of hub state. Derived wait + states (CUST-WP-0074, C-39..41) already exist for exactly this. +3. **Human gates are scattered across tasks.** 39 `needs_human` tasks, 41 `decision_id`s, + but only a handful of real founder questions. The prepared decision packets + (`history/20261003-security-gate-packets.md`, `docs/decision-packets-2026-10-03.md`) + were not turned into a resolved state. A batch of informed decisions would remove most + of the blockage. +4. **Owner/role disputes are routed as messages and re-checked by agents** + (e.g. "activity-core replied (not accepted yet)", "ops-mason has published no stance map", + "audit-core must publish…"). These are contract gaps. Each one needs a schema plus an + owner, not repeated agent negotiation. +5. **policy-nexus has no workplan.** It is supposed to be the guardrail place, but it has + 14 commits and no open plan, and its production go is itself behind `89f1058f`. Until + guardrails exist there as machine-checkable policy, agents keep re-deriving them in prose. +6. **Concurrency is not the lever.** With 80 % of tasks waiting, five parallel workers + mostly contend over the same blocked lanes and produce consistency sync churn + ("chore(consistency): sync task status from DB [auto]" appears repeatedly). + +## 4. Where deterministic functions should replace agents + +| Activity the agent currently does | Deterministic replacement | Owner / where | +|---|---|---| +| Polling wait tasks, writing "still waiting" records | Hub-derived `waiting-external` / `needs-human` (exists). Add a nightly job that lists *newly unblocked* tasks and emails/posts them. Agents start only on that list. | state-hub (C-39..41) + activity-core trigger | +| Consistency syncs, index/projection refreshes, "generated projection refresh" commits | `statehub fix-consistency` + generators run in CI or a pre-commit hook. Never as a separate agent commit. | each repo's Makefile / Forgejo Actions | +| Source-inventory, work-record index, publication.json refresh | `make inventory` that is idempotent and checked by `--check` in CI (policy-nexus already has `source-inventory.json`, so reuse that tool fleet-wide). | policy-nexus `tools/` as shared lib | +| "Independently reviewed" source-acceptance packets | Test suites + schema validation + signed CI run as the evidence. Human/LLM review only for design changes. | per repo | +| Checking an owner published a contract (stance map, tenancy.yaml metadata, case-type pin) | JSON-Schema/CUE contract files plus a cross-repo conformance check (`make contract-check`) that fails with the missing field. | net-kingdom canon schemas; reuse-surface | +| KnownSecurityCase execution (oidc-browser-relying-party etc.) | Already designed as deterministic: ops-mason renders → ops-warden applies. Make it one command per case type, idempotent, with a dry-run diff. | ops-mason, ops-warden | +| Policy guardrails written as prose in workplans | Policies as code in policy-nexus (OPA/Rego or CUE) evaluated in CI and by access-engine at runtime. | policy-nexus | +| Credential routing questions | `warden route find` (exists). Agents must call it, not message owners. | ops-warden | +| Live checks (token capabilities, login proofs, lease revoke) | Scripted smoke tests (`make smoke-`) run by the operator in the attended window, which emit one machine-readable receipt. | railiance-platform | + +What still deserves an agent: design work where intent is unclear (trust-root proposal, +tenant tier contract), writing the generators and checks above, code in +access-engine/user-engine/key-cape, and drafting informed-decision packets. + +## 5. Intended collaboration (target shape, for re-anchoring the agent) + +- **key-cape** (Keycloak) = authentication and issuers. **user-engine** = user lifecycle and + self-service (MFA enrollment). **tenant-engine** = tenant/tier/issuer ownership. +- **access-engine** = authorization decisions and freshness enforcement, fed by + **policy-nexus** policies (guardrails as code). +- **secrets-engine** over OpenBao (railiance-platform hosts it) = secret custody/leases. + **ops-warden** = SSH certificates and credential *routing* only. +- **ops-mason** = renders SecurityCaseType → KnownSecurityCase plans. **ops-warden** / + platform GitOps (ArgoCD) apply them. +- **gate-house** = ingress/edge enforcement. **kings-guard** = detection/guard posture. +- **net-kingdom** = canon (case types, scenarios, posture). **railiance-platform** = where it runs. + +One identity → authz → secrets → case-execution path proven end to end beats nine +partially accepted lanes. + +## 6. Next steps (in order) + +1. **Stop the current autonomous loop** after its in-flight commits. Freeze new + "record/retain/prepare" packet work. +2. **Attended unblock session (≈2 h, Bernd + one agent):** + - resolve `89f1058f` (ArgoCD production go, incl. policy-nexus); + - run the `62aa95e4` OpenBao provisioning window, using scripted smoke tests that emit receipts; + - decide `1591daa1` trust root. Accept a build-mode default (single holder, documented + re-arm item) instead of a final quorum design; + - check the 4 tasks still citing resolved `caafe4be`, and the unknown ids `d426c67e` / `b005b483`. + Expected effect: ~35 wait tasks become actionable or cancelable. +3. **Triage cancel pass (deterministic list, human approve):** cancel or demote to + `flavor: residual` every wait task whose blocker is "the next type follows once the + first is proven", adoption triggers with no adopter, or an enterprise feature (SAML + federation NK-WP-0011, tutorials NK-WP-0009) that is not on the critical path. Target + fewer than 40 open tasks. +4. **Open a policy-nexus workplan** for the guardrails as code (build-mode baseline + + re-arm list), consumed by access-engine and CI. +5. **Build the deterministic layer from Section 4** as one small cross-repo workplan: + unblocked-task notifier, `contract-check`, CI-run consistency/inventory, per-lane smoke + receipts. This is good agent work: bounded, testable, done once. +6. **Prove one end-to-end case**: oidc-browser-relying-party as the first KnownSecurityCase + (NK-WP-0048-T06) through ops-mason → ops-warden/ArgoCD, with login, revocation and audit + receipt. Use it as the template for the rest. +7. **Restart agents narrowly**: at most 2–3 workers, each pinned to one workplan whose next + task is `todo` (not `wait`), each with a clear done-test. Use `/ralph-workplan`-style + self-retirement, and stop when a task turns into `wait` instead of documenting the wait. + +## 7. Efficiency guardrails for the coding agent's goal text + +Add these to the Sol goal: + +- "Do not create evidence/packet/index commits that do not accompany a code or live-state change." +- "If the next step of a task needs a human or another owner, set `wait` with qualifiers once and move on. Do not revisit it until the hub reports it unblocked." +- "Batch all founder questions into one informed-decision packet per day, with a recommended default." +- "Prefer writing a script or check that makes a class of tasks deterministic over doing the tasks by hand." +- "Success metric = open task count going down and live capabilities proven, not commits." + +## 8. Caveats + +Counts come from grep over workplan files and may include archived or misformatted blocks. +"Paperwork vs code" is inferred from commit subjects and file extensions, not from reading +every diff. Some packet work may be legitimately required by canon review gates. Two +frequently cited ids could not be resolved to hub decisions.