net-kingdom/docs/smooth-bootstrap-guide.md

NET-WP-0018 Smooth Bootstrap Guide

Exercise status: unknown for the procedures in this runbook; no per-procedure successful-run receipt with operator attribution was established in the 2026-09-05 review. See procedure inventory.

Status: draft (initial consolidation for T05) Date: 2026-06-03 Purpose: The single operator guide for a smooth, repeatable NetKingdom security bootstrap. An operator knows what to do, in what order, and what (non-secret) evidence proves each step complete. Covers the full sequence from the T05 spec + inputs from T02 runtime architecture, T03 retrospective + gap matrix, existing UX contracts (operator-journey, user-lifecycle), console lifecycle-guide (incl. 0019 T06-adjacent polish), evidence templates/validators, and make targets.

This replaces piecemeal reliance on separate docs. It makes wrong-order execution visibly hard via "next safe action" and blocked gates. Links to concrete commands, scripts, console subcommands, validate targets, and evidence.

See also:

• docs/NetkingdomRuntimeArchitecture.md (T02 – what exists)
• docs/security-bootstrap-retrospective.md (T03 – what was bumpy, now automated, gaps)
• tools/security-bootstrap-console/security_bootstrap_console.py + make security-bootstrap-* (control surface, evidence, validators)
• Current lifecycle owner routes; the 0019 scripts are retired.
• .local/security-bootstrap.json + console status (current gates)

Pragmatic note (per 0018 Coordination): Track your progress through this guide using State Hub /progress/ (with workstream/task), dated notes in NET-WP-0018 workplan, git, console evidence/validators, /tmp evidence. This feeds future retrospectives.

Overall Model and Principles

From platform architecture and UX contracts:

• Stages: S1 Low-trust assembly → ... → S6 Reopen under custody (see console status).
• Shell / First screen always answers: Current stage, Next safe action, Blocked gates (why), Evidence (non-secret records).
• UI posture (console/web): Calm field notebook; black/white + hi accents; panels; sentence case; no hype. Shows effective access before any save/action. Blocked conditions explicit (e.g., no platform-root for non-king, MFA required for privileged).
• Evidence discipline: All steps produce/require non-secret evidence.json or metadata flags matching exact templates/validators (no secret markers). 12+ bools for user lifecycle (effective preview, no root grant, actor checks, verified identity/claims, reversible, no secrets recorded, etc.).
• Actor classes & previews: Always distinguish (setup operator, platform admin, tenant admin, reviewer, king). Show effective privileges before create/save. Never grant platform-root except via explicit king path.
• Secret boundary: Console/UI never collects/stores secrets. Use password-safe, k8s secrets, or operator memory. Use the existing governed credential-delivery path; legacy automatic admin-secret extraction is withdrawn.
• Reversible where possible; human custody gates explicit.
• Handoff to production readiness: After S6, move to 0017 production items (audit durability, etc. – not duplicated here).

Sequence overview (high-level; details per section):

1. Prerequisites & cluster foundation.
2. Credential bundle / king kit (SOPS/age, custody).
3. PrivacyIDEA bootstrap + realm.
4. LLDAP/bootstrap user (platform-root/king) + MFA self-enroll + verify.
5. KeyCape deployment + client registration + OIDC.
6. OpenBao init/unseal/config (OIDC admin binding via KeyCape).
7. Token cleanup, root disposition, restore drill, escrow/custody.
8. User lifecycle (onboard/lock/offboard/review through the current User Engine and IAM owner contracts).
9. State Hub sync, audit posture, cleanup/rotation.
10. Platform reopen (S6) + handoff.

Each step has: commands/scripts, evidence required, blocked conditions, links to validators/console.

Step 1: Prerequisites, Cluster Foundation, Credential Bundle

Prerequisites:

• Live Railiance/k3s cluster with ingress, cert-manager, NetworkPolicies, operators as per T02.
• Operator access: kubectl to sso/openbao ns; password safe entries (net-kingdom/LLDAP/admin, etc.); age keys/custodian public.
• No live OpenBao init in console (attended only).

Credential bundle / king kit:

• Use make security-bootstrap-king-kit or console king-kit.
• Dedicated king credential (platform-root@lldap, separate from personal).
• MFA: privacyIDEA self-service TOTP.
• Storage: password-safe + offline packet (age encrypted).
• Evidence: custodian_age_*_confirmed, king_credential_ready, mfa_enrolled_confirmed, password_safe_confirmed, storage_classes, custody_packet_prepared.
• Validate: make security-bootstrap-validate-kit or console validate-king-kit.
• Custody roster (for 2of3 target): make security-bootstrap-custody-roster-template, sign, validate.

Blocked if: No king kit or custody approval.

Next safe after: Approve custody mode (console approve-custody-mode or make with ARGS for flags like --mfa-enrolled-confirmed).

Step 2: PrivacyIDEA Bootstrap + Realm

• Repair missing realm/resolver configuration; preserve current owner-managed self-service/MFA policies.
• Use sso-mfa/k8s/privacyidea/repair-realm-live.sh or console action "Repair privacyIDEA realm and resolver". See the configuration-preserving contract; missing onboarding policies use their existing owner procedure.
• Enroll platform-root TOTP in repaired realm (pi-admin for setup).
• Evidence: related to t02 validate (realm healthy).
• Validate: make security-bootstrap-validate-t02 (covers audit/recovery gates incl. this).

Blocked if: Realm not correct for LLDAP users (MFA/self-enroll fails).

See T03 retrospective for past realm drift bumps (now partially automated via runbook + validate).

Step 3: LLDAP / Bootstrap User Creation (platform-root / king)

• Establish or recover platform-root only through the exact owner-reviewed IAM bootstrap/recovery ceremony; ordinary public registration cannot grant platform-root.
• The legacy account-creation and break-glass scripts are retired; see recovery limits. Preserve existing account/password custody pending owner verification.
• Self-enroll TOTP in privacyIDEA.
• Verify MFA state: bash sso-mfa/k8s/privacyidea/check-user-mfa-state.sh platform-root --token-file /absolute/private/diagnostic.jwt after owner-admitted protected input. See the diagnostic contract; the KeyCape workload credential is not a general diagnostic delivery lane. Counts do not verify user existence or an MFA login.
• Verify OIDC/KeyCape path: cd ../keycape && ./verify-openbao-client.sh.
• Groups: net-kingdom-admins for platform-root.
• Evidence: identity_account_created, identity_group_confirmed, mfa_*, oidc_login_verified.
• For non-root trials: follow Step 7 with exact owner scope and actual consumer evidence.

Blocked if: Actor requests net-kingdom-admins without king path; no MFA for privileged.

Console: lifecycle-guide for current owner routes. Legacy execution and inferred-claims commands exit with failure.

Step 4: KeyCape Deployment + Client Registration + OIDC

• KeyCape as lightweight IAM (conforms to IAM Profile v0.2).
• Deploy client config (sso-mfa/k8s/keycape/create-secrets.sh).
• Apply keycape-config Secret, restart KeyCape.
• Register bootstrap clients (netkingdom-bootstrap-console, openbao-admin).
• OpenBao admin redirects: localhost CLI callbacks plus http://127.0.0.1:18200/ui/vault/auth/netkingdom/oidc/callback for the preferred operator-tunneled browser UI login. The former public callback is retained only for the bounded listener rollback window; keycape remains a compatibility auth mount.
• Verify OIDC admin login: platform-root obtains OpenBao platform-admin via KeyCape/MFA.
• Evidence: keycape client gates, openbao_oidc_* , oidc_login_verified.
• Validate related in t02 / console.

Issuer: https://kc.coulomb.social (see T02 for claims: tenant, groups, roles, assurance, etc.).

See T03 for past callback/registration bumps (now gated).

Step 5: OpenBao Init / Unseal / Config + OIDC Admin Binding

Unseal custody model (select first): see docs/openbao-unseal-custody-models.md.

Model	When	Console
sops-held-automation	Default — fast rebuild test cycles	Selectable; gate automation
attended-ceremony	Production trust	Blocked until implemented
auto-unseal-transit	Production HA	Blocked until implemented

make security-bootstrap-openbao-unseal-custody-models
make security-bootstrap-select-openbao-unseal-custody-model MODEL=sops-held-automation

Development strategy: prove automation path first (SSH engine, warden, host CA), then add attended and auto-unseal gates for production profiles.

Console refuses live init in all models — automation runs via creds-bootstrap-agent.sh (sops-held) or future attended/auto-unseal playbooks.

• Preflight: make security-bootstrap-openbao-preflight --run or console.
• Init/unseal: sops-held-automation → sso-mfa/bootstrap/creds-bootstrap-agent.sh; attended-ceremony → human ceremony (planned automation hooks).
• Post-unseal: apply initial config (auth, mounts, policies, audit).
• OIDC auth config against KeyCape (maps claims/groups to policies e.g. net-kingdom-admins → platform-admin).
• Key material handling: trial exposure taint, rotate unseal keys, emergency lockdown, restore drill (snapshot, isolate, verify, destroy).
• Root token disposition: revoked (evidence: root_token_disposition).
• Evidence: openbao_initialized, initial_config_applied, trial_exposed + response_complete, keys_rotated, post_unseal_verified, openbao_oidc_*, restore_drill_passed, etc.
• Validate: multiple t02-related + console gates (taint logic in runbooks).

Blocked if: Not unsealed, no OIDC binding, no restore drill, trial material not responded to.

Console runbooks: Key material compromised, generate new unseal, emergency lock-down, restore drill, token revocation.

See T02 for secret/credential path details; T03 for past claim shape / token issues (now partially automated via gates/runbooks).

Step 6: Token Cleanup, State Hub Sync, Audit Posture, Cleanup/Rotation

• Revoke short-lived/bootstrap tokens (use console revoke helpers or runbooks; no plaintext on CLI).
• State Hub: sync work (POST /progress/, decisions, etc.); ensure .custodian-brief reflects.
• Audit posture: record audit_core_bootstrap_risk_accepted (or production_sink_ready), owner, review_date (2026-07-02), note. Use console approve or metadata update. Validate: console audit_core_posture.
• Cleanup/rotation: review bootstrap-era creds/databases/paths; rotate as needed. Evidence: cleanup_complete.
• Custody packet / roster final.

Evidence: root_token_disposition, audit_core_*, cleanup_complete, etc.

Console: cleanup-evidence-template, validate-cleanup, metadata updates.

See T03 for operator-state / token bumps (now in metadata + validators).

Step 7: User lifecycle through its owners

Follow the current lifecycle routes and retirement notice. User Engine owns ordinary onboarding, account lifecycle and product memberships; the existing identity-provisioner and IAM provider own identity and password setup. Use only admitted clients/tenants. Platform-root bootstrap has a separate owner ceremony and cannot be performed through public registration.

Preview the intended scope, obtain the existing authorization, and retain actual identity/claims/access, denial and exact cleanup receipts. A group list cannot verify issued claims or prove all sessions revoked. Never delete test accounts by a name pattern. Preserve other tenants and global identity state.

lifecycle-flow-template and onboarding-dry-run-template are blank evidence worksheets. Their validators check structure; passing is not native acceptance. The old Make/console execution commands and “User lifecycle dry-run (T06)” runbook are retired. Group administration and independent recovery remain open in NK-WP-0049; do not infer readiness from the earlier 0019 receipts.

Step 8: Platform Reopen + Handoff

• Final gates: all prior evidence + platform_reopened flag.
• Approve custody if needed.
• Console: status shows S6; "Review related workplans".
• Handoff: produce handover checklist (console handover-checklist); transfer to production readiness (audit durability, escrow, etc. per 0017 – not duplicated here).
• Evidence: platform_reopened, review_date, notes.

Validate: related custody/kit + full status.

Step 9: Post-Reopen / Optimization

• Use T03 retrospective + T02 arch + this guide for future drills.
• Address gaps (UE adapters per assessment, full Audit Core correlation, more T08 validators).
• Rehearse rebuild per T09 (T09 complete; see docs/security-bootstrap-rebuild-risk-and-rehearsal.md for risk classification + non-destructive scripted/ns/parallel plan using 0019+creds+T08 validators; scripted/namespace first; use current owner lifecycle contracts and exact trial provenance).

Evidence Summary Table (Core Non-Secret)

• King/custody: age keys, roster, packet, approval.
• Identity/MFA: account/group created, mfa_enrolled, oidc_verified.
• OpenBao: initialized, config_applied, oidc_bound, root_disposition, restore_drill, keys_rotated, taint/response.
• Lifecycle (0019): dry_run_date, actor_class, groups, effective_access_summary, lock_offboard_result, verified, no_secret_material_recorded, prevents, shows_effective_before_save.
• General: cleanup_complete, platform_reopened, audit_core_* (risk accepted or sink ready), metadata_updated_at.

All validated via console validate-* or make targets. Templates in console.

References and Updates

• Full list in T02/T03 docs.
• Console lifecycle-guide, status, web-ui.
• Update this guide + console guide section as T06/T08 work proceeds (e.g., more validators, control surface alignment).
• For web-ui exposure of this guide: see T06.

This guide + the runtime architecture + retrospective turn the first bootstrap into a repeatable, auditable (pragmatically), low-diagnosis path. Use it; record evidence; improve via T07+.

Next after this guide: Align control surface (T06), add tests (T07), integrate validations (T08), assess rebuild risk (T09).

See NET-WP-0018 workplan for full acceptance.