the-custodian/workplans/CUST-WP-0079-deterministic-security-flow.md

29 KiB
Raw Blame History

id type title domain repo status owner topic_slug created updated flavor depends_on state_hub_workstream_id
CUST-WP-0079 workplan Deterministic security flow: standing authorizations, flow report, and agent re-engagement infotech the-custodian active claude-code custodian 2026-10-11 2026-10-19 implementation
ab288f92-d364-5c24-9362-aee451f266c3

Follow-up to history/20261011-netkingdom-security-progress-assessment.md. About 80 % of the security estate's open tasks were wait, and agent time went into re-reading waits and producing evidence packets. This plan replaces that loop with (1) standing founder authorizations, (2) deterministic reports and sync, and (3) a narrow, re-scoped coding-agent goal.

Critical path (everything else is residual until promoted): identity (key-cape, user-engine) → authorization (access-engine + policy-nexus guardrails) → secrets (secrets-engine over OpenBao, response-wrapped delivery) → SecurityCase execution (ops-mason → ops-warden / ArgoCD), proven first on oidc-browser-relying-party (NK-WP-0048-T06, KEY-WP-0038).

Founder decision batch and hub/repo decision sync

id: CUST-WP-0079-T01
status: done
priority: high
state_hub_task_id: "618b32bb-3b30-5c31-b054-65b455d62b0b"

2026-10-11: D1, D2, D3 resolved, plus round 2 (standing build-mode change rule; D10 MFA remainder; D4 + D5). Recorded in docs/decisions/2026-10-11-security-unblock-decisions.md. 31 open hub decisions were already resolved in repo decision logs (gate-house, kings-guard, tenant-engine, security-canon, repo-manager) and were synced. Open hub decisions went from 61 to 22. 26 tasks were requalified, including fixes for tasks mislabeled as human gates. 10 workplans were demoted to residual.

Deterministic tooling

id: CUST-WP-0079-T02
status: done
priority: high
state_hub_task_id: "2764581a-ff8f-5a05-90bd-3dc6bd058ae5"

scripts/security_flow_report.py (READY / UNBLOCKED / ATTENDED / STALLED from workplan files), scripts/requalify_tasks.py (apply task requalifications from a JSON plan), scripts/sync_resolved_decisions.py (resolve hub decisions that repo logs already record as resolved or accepted).

Schedule the flow report and decision sync

id: CUST-WP-0079-T03
status: progress
priority: high
state_hub_task_id: "1be0569c-221d-5925-a0ff-fd2af88e52b1"
depends_on: [ACTIVITY-WP-0047]
blocking_reason: "ACTIVITY-WP-0047 now selects existing bounded operations: the prior coding profile cannot run Python or access the Hub. Await runtime-owner source/mount/state/network bindings, activation and first unassisted activity/report/sink/owner receipt."

Run sync_resolved_decisions.py --apply and security_flow_report.py --json daily (activity-core schedule, or a workstation timer until activity-core has the lane). Post a hub message to the coding agent only when UNBLOCKED or READY changed, and to the founder only when ATTENDED is non-empty. Agents must not poll wait tasks themselves.

Implementation (2026-10-11): scripts/security_flow_daily.py runs both commands, includes the-custodian in the default security estate, checkpoints each delivered recipient, and notifies the coding agent only when READY/UNBLOCKED changes. Empty ATTENDED lists never message the founder. User timer units are in infra/security-flow/; make security-flow-check passes nine tests including interrupted delivery and retry. Live hub dry-run succeeds. Schedule activation handoff 117e538e-da06-48b8-ba68-b82300a832e7 to activity-core is attached to case:cross-owner:CUST-WP-0079-T03:activity-core (infra/security-flow/scheduling-handoff.json); no owner acceptance or foreign dependency is invented. Schedule activation remains pending: workstation systemctl --user cannot connect to its user bus (No data available), so checked-in units alone do not establish daily execution.

2026-10-13 owner return: Activity Core accepted the handoff in message 00704319-97b2-469c-977e-6168ef20b9f9. Verified owner record ACTIVITY-WP-0047 (implementation, active) and committed definition activity-definitions/daily-security-flow.md: daily 07:30 Europe/Berlin, misfire skip, fixed python3 scripts/security_flow_daily.py --apply command, correlation CUST-WP-0079-T03, no repository grant. Definition remains enabled: false; source acceptance is complete, activation is not. The existing coordination case now names ACTIVITY-WP-0047 as obligor. T03 remains wait until its claiming rein can reach workstation checkouts/the hub, the owner activates the schedule, and the first unassisted fire proves completed ops_run, report delivery and checkpoint behavior. No scheduler was enabled and no foreign owner task was changed by this reconciliation.

Encode the standing change rule as a policy-nexus guardrail

id: CUST-WP-0079-T04
status: progress
depends_on: [PNEX-WP-0006]
blocking_reason: "Founder approved exact publication packet on 2026-10-19; policy-nexus owner executing PNEX-WP-0006 release. Native/public verification remains required before closure."
priority: high
state_hub_task_id: "5dd52156-09a8-5781-b288-99ae822f59a6"

Canon source written 2026-10-11: canon/standards/build-mode-change-authorization_v0.1.md (founder-decided). Closes when PNEX-WP-0006 publishes it.

Stop agent-authored index and sync commits

id: CUST-WP-0079-T05
status: done
priority: medium
state_hub_task_id: "8a3dc219-8982-5903-9407-e1b549d4544c"

WORK-RECORDS.md, source inventories, publication indexes and "projection refresh" files must be regenerated by a make target in a pre-commit hook or a Forgejo Action, not by agent commits. Start with the security repos. Done when a week of git log shows no standalone index-only commits from agents.

Implementation (2026-10-11): scripts/check_generated_only_commit.py checks staged files or every commit in a revision range. Forgejo workflow generated-commit-guard.yaml rejects standalone WORK-RECORDS, source-inventory, publication-index, projection-refresh, generated briefs and explicit evidence/decision-packet commits regardless of author; a later source commit cannot hide an earlier generated-only commit. make generated-commit-check provides the local hook entry point. The CI suite covers the rejection using a real temporary git repository. Consumer generator adoption and the required week of clean logs remain open; this task is not claimed complete.

The local guard is installed; make generated-commit-hook reproduces that installation and refuses to replace a different hook. The consistency CLI has no no-commit/no-push mode: its first invocation unexpectedly published brief-only ebaa38b, retained as a failed acceptance observation. Its subsequent invocation committed the already reviewed, staged implementation as 6fe7201; that commit includes source code. Future generated-only commits are rejected locally and in CI. The full Custodian suite passes 77 tests.

The default flow report now includes Custodian and activity-core, both required by the renewed critical path. Cross-owner completion can therefore unblock the platform activation task without an agent polling its wait body. Human gates also retain their intervention note in the generated report instead of a blank reason. CLI regression checks cover the actual owner completion transition and the default Custodian lane.

The report also excludes todo/progress tasks with unresolved explicit task dependencies, an owner gate or a human flag from READY. They remain visible under DEPENDENT or ATTENDED, without agent polling of wait bodies. Regression tests prove that owner completion admits the successor and that an unknown predecessor or stale progress label cannot produce readiness. This is queue qualification, not evidence that the corresponding live work has finished.

scripts/install_shared_generated_guard.py now checks or installs the guard in the repository's already configured shared hook directory. It preserves existing hook routing and other hooks, and refuses a differing or redirected pre-commit hook. Real temporary-repository tests prove rejection and source-change acceptance. The missing shared guard was installed for the existing repo-manager hook path; the week's fleet evidence and generator adoption still remain open.

The generated attended list now retains unresolved ordered dependencies. An owner token session following the first SecurityCase can therefore be listed as a future attended act without implying it is ready to run now. CLI regression coverage checks this boundary.

2026-10-12 closeout: make generated-commit-hook now recognizes the existing shared hook and preserves its routing and other owner hooks. A real Git/Make regression test checks repeat installation and rejection of a generated-only commit. Local implementation is complete; T05 remains waiting for fleet generator adoption and the originally required week of clean history. Neither one day of observations nor this local repair satisfies that acceptance condition.

Session closeout reviewed all three open Custodian plans. CUST-WP-0071 and CUST-WP-0073 remain blocked on their existing named owner/founder returns. CUST-WP-0079 is now blocked because all remaining tasks wait on scheduling-owner acceptance, policy publication, fleet/time-based acceptance or attended actions. No tasks or workplans were added, and CUST-WP-0076 remains finished. The cross-repo critical-path goal was paused by the operator; this bounded closeout does not resume it or claim native estate completion.

Attended window #1

id: CUST-WP-0079-T06
status: wait
needs_human: true
blocking_reason: "Owner source rechecked 2026-10-19: RPF-WP-0044-T08 is done and KEY-WP-0040-T01 preview is done. Remaining attended acts are USER-WP-0026-T03 positive readback, USER-WP-0028-T03 U08 recovery, and RPF-WP-0044-T04 OpenBao OIDC/MFA login; completed preview does not authorize factor mutation."
priority: high
state_hub_task_id: "e3da807f-9a6e-513f-82dc-3714a489e7bf"

Re-engage the coding agent with the renewed goal

id: CUST-WP-0079-T07
status: done
priority: high
state_hub_task_id: "91ec924c-b305-5a63-925d-087a3b927d60"

Replace the Sol goal with the renewed text in the assessment history file (section "Renewed goal"). Re-engage now for the READY lane. It does not depend on T06.

Completed 2026-10-11: the renewed critical-path goal is active in thread 01a126e1-76f8-7643-8cf3-e93336539ade; its attached text was read before execution. Work resumed from READY/UNBLOCKED, with three workers pinned to eligible workplans and the accepted build-mode authorization in force. This closes re-engagement only, not the live critical-path goal.

Current owner-source audit — 2026-10-19

Activity Core source remains disabled (daily-security-flow, ACTIVITY-WP-0047 T02/T03 wait), so neither checked-in scheduling nor clean source establishes an unassisted fire. Policy Nexus PNEX-WP-0006 remains blocked for founder-batch publication and its source-inventory gate; its eight guardrail tests pass. No privileged activation or policy publication was attempted.

The fixed seven-day local Git window [2026-10-12T00:00:00+02:00, 2026-10-19T00:00:00+02:00] contains 498 commits across all 15 repositories in security_flow_report.DEFAULT_REPOS. Applying the existing generated-only classifier to each commit found zero standalone generated-only commits. The receipt docs/evidence/2026-10-19-security-flow-week-history.json records repository HEADs and counts. This proves the observed local history window; it does not prove unobserved remote branches or fleet generator adoption. The shared repo-manager pre-commit hook rejects generated-only commits and scans staged secrets, but does not regenerate indexes. T05 therefore remains open for actual Make/hook or Forgejo generator adoption. Reuse the existing State Hub scripts/consistency_check.py work-record rendering functions rather than the auto-committing consistency CLI inside a hook; any integration must preserve unstaged edits and generate from the staged source snapshot.

Scope discovery is now explicit and the three relevant owner checkouts are clean after source-only commits: policy-nexus 19b65d5 adds SCOPE and links it from its existing capability declaration; repo-manager 027b54e documents sync, shared-hook boundaries and the missing generator adoption; activity-core 70ff241 documents the disabled fixed-command consumer and execution boundary. These are source capabilities, not runtime publication or authorization claims.

T06's stale blockers were corrected from owner workplans: the coulombcore check was already complete, and KeyCape's attended TOTP preview completed October 17. Actual U08 recovery remains with USER-WP-0028-T03; a successful preview is not factor-disable/re-enrollment acceptance.

T05 preparation now includes scripts/generate_staged_work_records.py and make generated-work-records / make generated-work-records-check. The adapter reuses the State Hub renderer against staged Markdown blobs, refuses conflicts, symlink sources and unstaged generated-file edits, and stages only the generated index. It never invokes the consistency CLI, hub, commit or push. Seven regression tests cover actual owner rendering, staged-versus-unstaged source isolation, generated-file preservation, read-only checking, idempotency and symlink refusal; these and the existing generated guard tests pass (ten tests). Shared-hook activation and fleet Make target adoption are still outstanding; the adapter alone is not evidence that repository indexes are generated during commits.

Review refinement: generation requires --repo-slug, so worktrees and renamed checkouts retain canonical identity. Each explicit renderer checkout is loaded in a fresh subprocess, preventing cross-checkout module-cache reuse. Tests prove both boundaries. Usage and adoption limitations are documented in docs/staged-work-record-generation.md and the local SCOPE capability; no shared hook routing was changed.

T05 generator adoption — 2026-10-19

T05 is progress and the workplan active because authorized source adoption is underway. The shared repo-manager hook now composes the original generated-only guard, opt-in staged generator and unchanged exact-scope secret scanner. The Custodian's separate local scanner hook was upgraded with the reviewed installer (changed: true); its routing and other hooks were preserved. Opt-in is explicit in .generated-work-records.json, with a dedicated staged Make recipe. Ordinary Makefiles and .env includes are not loaded by the hook. Recipe content remains trusted source, not an arbitrary-code sandbox.

Actual owner commits exercised the hook: activity-core 8a7275c found its index current; policy-nexus c89364d and repo-manager 05046fb regenerated and included WORK-RECORDS with the adoption source. All three checkouts were clean after their commits, and the latter two retained successful secret scans. Subsequent owner work may change those checkouts independently.

Twenty-one adapter/shared-hook/guard tests pass, including preservation of partially staged source and unrelated staged briefs. Real git commit --only exposed Git's temporary-index behavior; the adapter now refuses regeneration in an alternate index before mutation. Pre-generating against normally staged sources and including WORK-RECORDS in the partial commit passes with unrelated staging intact. Instructions are in docs/staged-work-record-generation.md.

This is WORK-RECORDS adoption in three provider repos plus Custodian preparation, not fleet completion. Remaining surfaces include the other security repositories, hub-derived .custodian-brief.md, policy-nexus source inventory (whose authored configuration/dispositions must be retained), and net-kingdom's generated coordination-frame receiver through tools/coordination-frame/render.py. Historic evidence inventories are not recurring generator outputs. T05 stays open until those applicable owner generation paths and fleet acceptance are verified; the independent schedule, publication and attended gates remain open.

Fleet WORK-RECORDS adoption and source cleanup — 2026-10-19

The staged WORK-RECORDS lane is now adopted across every repository in the 15-repository default security report estate, plus the relevant implementation providers. Each adopted repository declares its canonical slug, isolated Make recipe and SCOPE boundary. Real commit hooks checked/rendered the staged index; shared generated-only rejection and existing exact-scope secret scans remained in force. No native runtime, credentials, public policy or schedule activation was performed by this adoption.

All 25 provider checkouts were clean at verification (3 earlier adopters, 11 security-fleet adoptions, 4 isolation providers and 7 sizing providers). Current source revisions:

Repository Commit
net-kingdom 4c7f32a
railiance-platform b06d1f0
access-engine 7eb4e22
secrets-engine 510d292
ops-mason 3457ecf
key-cape 5defa33
user-engine db05c13
tenant-engine f94bb5d
gate-house c7d47f8
kings-guard 5e4942f
rapp-policy-nexus ff41754
activity-core 7490eb8
policy-nexus c89364d
repo-manager 05046fb
ops-warden 6ed0e9e
railiance-cluster 4604faf
railiance-forge 9aa2ac6
railiance-apps a6fe0ab
resource-framework (resource-control) 0754009
rapp-telemetry 6015b54
railiance-telemetry d089325
vergabe-watch (vergabe-teilnahme) 17429f8
railiance-infra 5094c30
glas-harness 579cefd
rein-codex (rein-aharness) fb14c38

The first fifteen provider rows were rechecked directly after the final commit; the additional sizing/isolation rows carry their worker completion receipts and root's independent checkout verification. Custodian's own source adoption was committed separately and uses its reviewed local scanner-hook composition.

Dirty generated indexes were inspected and original snapshots retained under /tmp/cust79-fleet-adoption/<repo>/. Platform, Access and King's Guard originals matched the staged owner renderer exactly. NetKingdom and Secrets originals were stale row subsets of current source; regeneration added the missing source records. User Engine's original reflected its pending source handoff, which was reviewed and committed with the derived index rather than discarded.

Existing unfinished source was inspected and retained: Ops Mason's lock matched its current pyproject (uv lock --check --offline) and all 260 tests passed; its lock is now committed. Rapp Policy Nexus passed its nine packaging tests and Helm lint. USER-WP-0035-T02's pending owner handoff was checked against current caller/receiver/provider/portal code and retained with dated source hashes. EMAIL-IN-0001 is durable in email-connect 994db7d (clean), and NET-IN-0005 is included in 4c7f32a. Both remain owner preparation requests; they do not prove setup-mail capability or runtime delivery. NetKingdom's pending cadence capture receipts were preserved (3 events, 3 receipts, 588 inventory observations); their record still says independent receiver readback is pending. No current process or cadence acceptance is inferred from those historical files.

User Engine's typo-suffixed .claude/settings.local.json. was inspected and preserved inactive at /home/worsch/.local/state/custodian-preserved/user-engine/settings.local.json.inactive-2026-10-19, SHA256 b70496e3978a7e45304ad6e2042f81953bc8d6ac518aa82fcab36c4e5c754243. It was not renamed to active settings or used to grant permissions; the location and hash receipt is committed with the source reconciliation.

State Hub supplied a separate strict rendering-only brief CLI in 09803dc, with identifier follow-up f8f9cd9 verified in publication/projection and published capability declaration eac38b7. Owner validation passed 17 focused tests and 168 consistency tests on rerun; initial combined-suite database teardown errors are retained in the owner/root report. The CLI does not perform Git operations or Hub writes. Attempts to refresh the five pending security briefs failed closed at /repo-goals or /repos/<slug>; no replacement snapshot was written. Their existing generated deltas were reviewed against source, retained with their original October 15–18 timestamps, and committed alongside meaningful adoption source. They are historical orientation, not fresh Hub readback. Originals remain in the preservation snapshot. The brief's deliberate Markdown hard-break whitespace was retained.

T05 remains progress: WORK-RECORDS adoption does not establish automation for hub-derived briefs, Policy Nexus source inventory, or NetKingdom's generated coordination-frame receiver. Those owner generator families and recurring execution remain to be implemented/verified without changing authored source inventory dispositions. T03, T04 and T06 retain their independent schedule, publication and attended acceptance gates. No new owner acceptance is inferred from clean checkouts or the earlier clean seven-day Git window.

State Hub's five older metric snapshots remain intentionally dirty after owner review; they were not included in the 25 clean provider claim. Custodian's inherited NetKingdom assessment receipts also remain outside these owner cleanup commits. Neither source adoption nor clean provider worktrees authorizes deleting unrelated observations.

Local brief policy and preserved metric snapshots — 2026-10-19

The operator selected local generated briefs and removal of tracked snapshots. State Hub's renderer now omits Git operations for brief refreshes, including the ordinary consistency path. The migration preserves the existing local bytes, adds root-anchored ignores and removes only the tracked snapshot. No offline orientation file is deleted. The explicit strict brief CLI remains available; failed fresh reads preserve prior local orientation. This supersedes the earlier plan to couple hub-derived brief snapshots to source commits.

The five State Hub legacy-meter snapshots reviewed in the preceding session are now preserved outside its checkout at /home/worsch/.local/state/custodian-preserved/state-hub/legacy-meter-20261019/. manifest.json records every original path, SHA256 and tracked-baseline hash. The tracked Oct14 baseline was restored and the four untracked copies removed only after exact archive verification. These are historical observations, not scheduled-execution or retirement acceptance. State Hub was clean after this archive and before the local-brief source migration. This updates the earlier intentionally-dirty observation without discarding evidence or committing a generated-only packet. Custodian's inherited NetKingdom receipts remain intact.

T05 acceptance audit — 2026-10-19

T05 is done. The earlier full seven-day local history audit covered 498 commits across the 15-repository default estate with no standalone generated-only commits. Staged WORK-RECORDS generation is adopted in all those repositories and the additional providers. The concrete remaining recurring artifacts were rechecked against tracked source paths, rather than inferred from filenames:

  • NetKingdom generated/coordination-frame/approval-engine-receiver.json now derives from indexed owner sources through the local commit recipe (bde143e). Eleven tests cover the existing compiler plus partial staging, sibling preservation, target edits, symlinks and alternate-index refusal. The hook never rewrites sibling repos. Its subsequent local-brief migration is 1bd869c.
  • Policy Nexus's generated release inventories and manifests already run in the scheduled Forgejo build. Its tracked source-inventory.json and config carry authored dispositions and must not be auto-refreshed as disposable output. 5a6e78f adds a non-destructive staged review utility (65 tests), not a second versioned inventory. Publication remains a separate T04 gate: two newly discovered documents still need explicit dispositions and the founder's public-release batch. This audit does not claim a fresh scheduled release.
  • Brief snapshots are deliberately no longer Git artifacts, per operator choice. State Hub 39b51a2 removes the renderer's commit path; all 185 focused and consistency regression tests pass. Its owner sync returned applied at that exact revision. Local brief preservation was checked with hashes during the provider migration; the files remain available and ignored.

The default-estate path audit found no additional recurring source inventory, publication index or projection refresh output lacking its owner generation path. Dated historical receipts are retained observations, not refresh jobs. This closes generation and local-history acceptance only. T03, T04 and T06 remain waiting on their named runtime/publication/attended gates, so this workplan is blocked rather than finished. No residual work is concealed by T05.

Daily/weekly executor source is committed in Activity Core 4ae00fc, with 57 focused bounded-operation/evidence/definition tests passing. It reuses the existing ADR-007 fixed-command stage, suppresses raw command output, validates trusted bindings before subprocess execution and returns compact hash/count receipts. The selected commands emit no agent ops_run or model call. Both schedules remain disabled. ACTIVITY-WP-0047 records the replacement receipt contract and runtime acceptance gates; the corresponding Custodian handoff and weekly-report documentation now consume that contract. NetKingdom's required consistency pass synced 64/64 bindings with zero automation/assessment failures (27 existing warnings, 5 informational findings); its brief refreshed locally without any brief commit.

Runtime review refinement: Activity Core 0319fe8 retains the concrete proposal in docs/custodian-runtime-binding-proposal.md. It pins the four stdlib owner scripts from Custodian 60d893a (30,169 bytes), source/state/fleet paths, direct Hub service semantics, paused verification and checkpoint-preserving rollback. Direct Kubernetes ConfigMap projection is unsuitable because its symlinks are correctly refused by the executor; image-baked files or a hash-verified init copy are the two existing delivery patterns. Existing worker vs dedicated placement, fleet publisher/freshness and state retention/backup acceptance remain owner inputs. They were requested without enabling schedules or claiming deployment approval. This is source preparation for T03, not a new scheduler or first-fire proof. Activity Core remained clean after the documentation commit.

Owner delivery resumed after interview — 2026-10-19

Decision 8a64ba65-1068-409c-86f8-b9414b29b7b4 selects the existing Activity Core worker with pinned owner scripts and private retained state. T03 is progress for the delegated Activity Core/Repo Manager source packaging and snapshot publisher work. Both definitions remain disabled pending the exact reviewed implementation and existing activation conditions. No second worker/scheduler is needed by this selected design. Routine live changes use the existing standing build-mode rule only when its concrete review, rollback, abort-owner and receipt conditions hold; access/secret scope, spend and public-publication exclusions remain in force.

Policy Nexus 8af92bf resolves the two source-discovery review rows as explicit metadata-pending under existing inventory rules. No publication mappings or foreign canon changed. Its 65 tests,186-source audit (74 published/94 pending), local immutable release verification and currency checks pass. T04 still needs the founder's existing public-release disposition for the build-mode standard and derived JSON; the two unrelated pending documents are not part of that ask.

Exact publication approved; execution pending verification

Founder approved the prepared publication and deployment in async reply call_5157a09329d34746a3fdf857d5a47b8d, item 0. The immutable packet is docs/changes/CUST-WP-0079/policy-publication-r1/PLAN.md, hashlist SHA256 309ef01650b65bc6b28e86daad9b2835a242631cf60e0896e849461bb917504f. The supervision record binds original and approved revision 1. It authorizes image 214b122c3d175563f114552fa1826dac9642433be019c736c3265d0a3ef379f1, paired release hashes, exact hosting changes and rollback to verified live revision 9. Nine rendered resources passed server dry-run; only image/release metadata changes. The owner is executing the approved sequence after fresh preconditions. T04 remains open until public and native verification succeeds.

Activity Core fed119a bakes the pinned commands and enforces required input admission; 3b25aea selects an immutable snapshot through a protected descriptor under a fixed retained mount. The latter has 76 focused passing tests. This avoids a deployment patch for every new snapshot. The existing core-hub publisher has Forge custody, but no accepted file-transport path to Activity Core exists. An opt-in endpoint, exact 15-repository disclosure policy, narrowly selected network access and receiver are being prepared. Existing Forge read authority does not authorize this new downstream disclosure. No new credential mount or live access change has been made; both job definitions remain disabled.