From 4f65e41ac55d2bc652122b76312048efdaf6b698 Mon Sep 17 00:00:00 2001 From: tegwick Date: Sat, 17 Oct 2026 00:28:17 +0200 Subject: [PATCH] fix(identity): retain group provenance and prepare sealed recovery custody Assistant: codex Assistant-Model: gpt-6.1-sol Assistant-Session: 01a1468b-63fa-7710-85dc-8f2e16b3ebfa --- docs/service-group-provenance.md | 114 ++++++ identity-provisioner/Containerfile | 2 +- identity-provisioner/group_provenance.py | 170 +++++++++ identity-provisioner/provisioner.py | 223 +++++++++--- identity-provisioner/server.py | 14 + .../tests/test_directory_contract.py | 2 +- .../tests/test_group_provenance.py | 326 ++++++++++++++++++ .../tests/test_service_groups.py | 17 +- sso-mfa/bootstrap/sops-custody-fd.py | 54 +++ sso-mfa/bootstrap/sops-custody-unlock.sh | 13 +- sso-mfa/bootstrap/sops-private.py | 18 + sso-mfa/k8s/README.md | 13 + ...lated-keycape-native-fixture-manifest.json | 18 +- tests/test_prepare_isolated_keycape.py | 108 ++++++ tests/test_private_manifest_binding.py | 98 ++++++ tests/test_sealed_custody.py | 143 ++++++++ .../isolated-keycape-session.py | 3 + tools/bootstrap-recovery/isolated-keycape.py | 26 ++ .../prepare-isolated-keycape.py | 119 +++++++ .../run-isolated-keycape.sh | 7 +- workplans/NK-WP-0048-security-case-types.md | 119 ++++++- ...-bootstrap-secret-free-recovery-handoff.md | 167 ++++++++- 22 files changed, 1685 insertions(+), 89 deletions(-) create mode 100644 docs/service-group-provenance.md create mode 100644 identity-provisioner/group_provenance.py create mode 100644 identity-provisioner/tests/test_group_provenance.py create mode 100644 sso-mfa/bootstrap/sops-custody-fd.py create mode 100644 tests/test_prepare_isolated_keycape.py create mode 100644 tests/test_private_manifest_binding.py create mode 100644 tests/test_sealed_custody.py create mode 100644 tools/bootstrap-recovery/prepare-isolated-keycape.py diff --git a/docs/service-group-provenance.md b/docs/service-group-provenance.md new file mode 100644 index 0000000..87aa5b4 --- /dev/null +++ b/docs/service-group-provenance.md @@ -0,0 +1,114 @@ +# Durable service group creation provenance + +This source change narrows the existing authenticated identity-provisioner +`/v1/groups/ensure` and `/v1/groups/revoke` endpoints. It introduces no new +principal, credential, policy decision, or rollback authorization. An endpoint's +service token and a journal receipt do not establish an accepted case, caller +session, owner assent, or permission to use a protected execution lane. + +## Request and receipt migration + +Ensure keeps `groups`, `members`, `idempotency_key` and `correlation_id`. +The original correlation is now part of its immutable binding: the original +idempotency key with a different correlation or group/member request is refused. +The response adds `ensure_idempotency_key`, `ensure_correlation_id`, +`created_groups` (name and exact directory id) and `created_memberships` +(name, directory id and username). The owner journal independently retains the +complete prestate and expected postimage. Response inventories are advisory; +caller-supplied inventories are rejected and never authorize undo. + +Revoke retains the original group/member request, its own `idempotency_key` and +`correlation_id`, and optional boolean `delete_if_empty`. It additionally requires +`ensure_idempotency_key` and `ensure_correlation_id` from the original ensure. +Legacy revoke requests fail before directory access. No old deployment inventory +is imported as proof of creation: pre-upgrade groups and memberships cannot be +rolled back automatically without independent owner reconciliation. + +An ensure replay returns its original receipt and never recreates subsequently +removed state. Reusing its key after rollback is refused. Revoke replays return +its original outcome; changing deletion options after completion is refused. +There are at most 16 groups, 256 requested membership entries, 4096 retained +preexisting members per operation, and 256 journal records by default. Records +have no timeout or automatic eviction. Capacity exhaustion stops new operations. + +## Storage policy and local fixture configuration + +The implemented SQLite backend is for local development and disposable tests. +It is not an admitted production storage design. Accepted +[ADR-0015 section3](adr/ADR-0015-netkingdom-railiance-workload-packaging-and-relational-platform.md) +requires production relational storage through CloudNative PG and `rapp-postgres` +using `database-per-consumer`; it explicitly rejects per-workload SQLite-on-a-PVC +as a production pattern. The rail-kubernetes wave-1 contract also does not supply +a generic persistent-storage contract. This source does not authorize adding a +SQLite PVC to identity-provisioner. + +Production group operations remain unadmitted and must stay disabled until the +existing owner paths admit a CNPG-backed implementation of the same journal +semantics, credentials, storage/backup/restore and writer-isolation contracts, +or an explicit governed exception to ADR-0015 identifies the exact scope, +limits, duration, recovery and risk owner. No such backend, exception or native +assent is supplied here. `railiance-platform` owns shared database/custody +governance; `rapp-postgres` owns the package consumer provisioning surface. +The identity-provisioner runtime package/operating assignment must also be +explicitly settled rather than inferred from User Engine's dependency on it. + +For local/disposable tests only, `GROUP_PROVENANCE_DB` selects an absolute, +non-symlink SQLite path in a private test directory owned by the process UID, +with mode0700 for the directory and mode0600 for the database. The local fixture +explicitly initializes a genuinely new empty store with +`GroupProvenance(absolute_path, initialize=True)`; initialization refuses an +existing file. Ordinary opening never recreates a missing store. The completed +disposable fixture uses private container tmpfs and preserves the file across +separate Python processes; that proves process-restart semantics, not production +durability. `/tmp`, `emptyDir`, tmpfs and an image layer do not satisfy native +retention or recovery duties. + +With no configuration, ordinary directory readiness reports +`service_groups: disabled` and group endpoints refuse work. Missing/untrusted or +unavailable configured local storage prevents startup or returns a fixed +`group_provenance` dependency failure without path/database diagnostics. +`service_groups: available` checks capability only: the implementation does not +infer or enforce production admission, policy exceptions, accepted cases or +writer isolation from a filesystem path. + +The existing reference Deployment remains one `Recreate` replica at UID/GID10001 +with `readOnlyRootFilesystem: true`, no provenance volume and an older immutable +image. Image rebuild, native owner review and credential delivery remain separate +acts. Preserve its nonroot/read-only boundaries. Native retention and recovery +must preserve original idempotency bindings, prestate, confirmed creations and +unresolved intents; losing a journal never authorizes replacement initialization +and replay. No arbitrary JSON import, receipt expiry or automatic uncertainty +clearing is introduced. Owner reconciliation needs independent evidence. + +## Safety and remaining native acceptance + +Before each directory mutation, durable intent is committed. A successful +response and exact readback are required before recording a confirmed creation. +A definite failure before any mutation attempt retains a `refused` record and +releases the active fence; its original key still cannot retry. Transport loss, +readback failure after an attempted mutation or a process exit leaves pending/uncertain state; +it survives reopening and blocks new mutations instead of becoming retry success. +Every database connection rechecks the private path and original device/inode; +missing/replaced storage is refused. Every intent update must affect exactly one +matching journal row before execution can continue. +The journal records mutation intent and confirmed outcomes, not an atomic LDAP +transaction or a complete audit emission guarantee. + +Rollback validates every targeted group id and complete member postimage before +its first destructive operation, and verifies each subsequent change. It removes +only memberships confirmed newly added by that ensure. It deletes only groups +confirmed newly created by that ensure, and only while empty. Preexisting groups, +including empty groups, are never deleted. Existing members are never included +in the removal inventory. Detected shared-member drift or group replacement +refuses the operation before any destructive call and retains uncertainty. + +LLDAP exposes no membership-generation token or conditional mutation/CAS here. +Readback cannot detect a remove/re-add ABA sequence or close a read/write race +with an external writer. Native rollback therefore still requires an owner-proven +writer isolation/fencing contract or a provider-side versioned/CAS operation; +these synthetic fixtures do not prove that boundary. A durable SQLite file is +trusted local owner-held state, not a production storage admission, tamper-evident +remote evidence or an authorization broker. Existing caller authentication is unchanged; case/source/principal binding, +per-attempt protected session revocation, independently admitted evidence callbacks, +native storage/backup recovery and native endpoint acceptance remain outstanding +under NK-WP-0048-T14/T16. This source change alone does not finish those tasks. diff --git a/identity-provisioner/Containerfile b/identity-provisioner/Containerfile index c30138f..755d68e 100644 --- a/identity-provisioner/Containerfile +++ b/identity-provisioner/Containerfile @@ -3,7 +3,7 @@ FROM python:3.12-slim@sha256:d764629ce0ddd8c71fd371e9901efb324a95789d2315a47db7e RUN useradd --create-home --uid 10001 provisioner WORKDIR /app COPY --from=lldap /app/lldap_set_password /app/lldap_set_password -COPY provisioner.py password_setup.py server.py /app/ +COPY provisioner.py group_provenance.py password_setup.py server.py /app/ USER 10001:10001 EXPOSE 8080 CMD ["python", "server.py"] diff --git a/identity-provisioner/group_provenance.py b/identity-provisioner/group_provenance.py new file mode 100644 index 0000000..82bff97 --- /dev/null +++ b/identity-provisioner/group_provenance.py @@ -0,0 +1,170 @@ +"""Bounded, owner-private recovery journal; never an authorization source. + +An unresolved write intent is retained indefinitely and blocks further writes. +Only the directory owner may reconcile it using independently obtained evidence. +""" +from __future__ import annotations + +import json +from contextlib import contextmanager +import os +from pathlib import Path +import sqlite3 +import stat +from typing import Any +from urllib.parse import quote + + +class GroupProvenanceUnavailable(RuntimeError): + def __init__(self): + super().__init__("group_provenance_unavailable") + + +class GroupProvenance: + def __init__(self, path: str, *, capacity: int = 256, initialize: bool = False) -> None: + target = Path(path) + if not target.is_absolute() or target.parent.resolve() != target.parent or capacity < 1 or capacity > 4096: + raise ValueError("private absolute provenance path and bounded capacity required") + parent = target.parent.lstat() + if not stat.S_ISDIR(parent.st_mode) or parent.st_uid != os.geteuid() or parent.st_mode & 0o077: + raise ValueError("provenance directory must be owner-private") + # Service startup never creates a replacement for missing durable state. + flags = os.O_RDWR | os.O_NOFOLLOW + if initialize: + flags |= os.O_CREAT | os.O_EXCL + fd = os.open(target, flags, 0o600) + try: + info = os.fstat(fd) + if not stat.S_ISREG(info.st_mode) or info.st_uid != os.geteuid() or info.st_mode & 0o077 or info.st_nlink != 1: + raise ValueError("provenance file must be owner-private and regular") + finally: + os.close(fd) + self.path = str(target) + self._device, self._inode = info.st_dev, info.st_ino + self.capacity = capacity + with self._db() as db: + if initialize: + db.execute("CREATE TABLE group_receipts (key TEXT PRIMARY KEY, binding TEXT NOT NULL, state TEXT NOT NULL, active INTEGER UNIQUE, body TEXT NOT NULL)") + db.execute("PRAGMA user_version=1") + if db.execute("PRAGMA user_version").fetchone()[0] != 1: + raise RuntimeError("unrecognized group provenance schema") + db.execute("SELECT key, binding, state, active, body FROM group_receipts LIMIT 0") + + def _check_file(self): + try: + target = Path(self.path) + parent = target.parent.lstat() + info = target.lstat() + if target.parent.resolve() != target.parent or not stat.S_ISDIR(parent.st_mode) or parent.st_uid != os.geteuid() or parent.st_mode & 0o077: + raise GroupProvenanceUnavailable() + if not stat.S_ISREG(info.st_mode) or info.st_uid != os.geteuid() or info.st_mode & 0o077 or info.st_nlink != 1: + raise GroupProvenanceUnavailable() + if (info.st_dev, info.st_ino) != (self._device, self._inode): + raise GroupProvenanceUnavailable() + except OSError: + raise GroupProvenanceUnavailable() from None + + def check(self): + with self._db() as db: + if db.execute("PRAGMA user_version").fetchone()[0] != 1: + raise GroupProvenanceUnavailable() + db.execute("SELECT key FROM group_receipts LIMIT 0") + + @staticmethod + def _load(body, key, binding): + try: + if not isinstance(body, str) or len(body) > 1048576: + raise GroupProvenanceUnavailable() + value = json.loads(body) + if not isinstance(value, dict) or value.get("key") != key or value.get("binding") != binding: + raise GroupProvenanceUnavailable() + if value.get("state") not in {"pending", "ensured", "revoking", "revoked", "uncertain", "refused"}: + raise GroupProvenanceUnavailable() + if not isinstance(value.get("groups"), dict) or len(value["groups"]) > 16: + raise GroupProvenanceUnavailable() + for field, limit in (("created_groups", 16), ("created_memberships", 256)): + if not isinstance(value.get(field), list) or len(value[field]) > limit or any(not isinstance(item, dict) for item in value[field]): + raise GroupProvenanceUnavailable() + if "intent" not in value or value["intent"] is not None and not isinstance(value["intent"], dict): + raise GroupProvenanceUnavailable() + for group in value["groups"].values(): + if not isinstance(group, dict) or "id" not in group or not all(isinstance(group.get(f), list) and all(isinstance(u, str) for u in group[f]) for f in ("before", "after")): + raise GroupProvenanceUnavailable() + if value["state"] in {"ensured", "revoking", "revoked"} and ( + not isinstance(value.get("requested_groups"), list) or not isinstance(value.get("ensure_correlation_id"), str) + ): + raise GroupProvenanceUnavailable() + return value + except (TypeError, ValueError): + raise GroupProvenanceUnavailable() from None + + @contextmanager + def _db(self): + db = None + try: + self._check_file() + db = sqlite3.connect("file:" + quote(self.path, safe="/") + "?mode=rw", uri=True, timeout=5) + self._check_file() + db.execute("PRAGMA synchronous=FULL") + with db: + yield db + except sqlite3.Error: + raise GroupProvenanceUnavailable() from None + finally: + if db is not None: + db.close() + + def reserve(self, key: str, binding: str) -> tuple[dict[str, Any], bool]: + with self._db() as db: + db.execute("BEGIN IMMEDIATE") + row = db.execute("SELECT binding, body FROM group_receipts WHERE key=?", (key,)).fetchone() + if row: + if row[0] != binding: + raise ValueError("ensure idempotency key is bound to another request") + return self._load(row[1], key, binding), False + if db.execute("SELECT COUNT(*) FROM group_receipts").fetchone()[0] >= self.capacity: + raise RuntimeError("provenance capacity reached; retained records require owner disposition") + receipt = {"key": key, "binding": binding, "state": "pending", "intent": None, + "groups": {}, "created_groups": [], "created_memberships": []} + try: + db.execute("INSERT INTO group_receipts VALUES (?, ?, ?, 1, ?)", + (key, binding, "pending", json.dumps(receipt, sort_keys=True))) + except sqlite3.IntegrityError: + raise RuntimeError("another group operation is unresolved") from None + return receipt, True + + def get(self, key: str, binding: str) -> dict[str, Any]: + with self._db() as db: + row = db.execute("SELECT binding, body FROM group_receipts WHERE key=?", (key,)).fetchone() + if not row or row[0] != binding: + raise ValueError("unknown ensure receipt or request binding mismatch") + return self._load(row[1], key, binding) + + def claim_revoke(self, key: str, binding: str, delete_if_empty: bool) -> dict[str, Any]: + with self._db() as db: + db.execute("BEGIN IMMEDIATE") + row = db.execute("SELECT binding, body FROM group_receipts WHERE key=?", (key,)).fetchone() + if not row or row[0] != binding: + raise ValueError("unknown ensure receipt or request binding mismatch") + receipt = self._load(row[1], key, binding) + if receipt["state"] == "revoked": + if receipt["delete_if_empty"] != delete_if_empty: + raise ValueError("completed rollback options cannot change") + return receipt + if receipt["state"] != "ensured": + raise RuntimeError("group provenance unresolved; independent owner recovery required") + receipt.update(state="revoking", delete_if_empty=delete_if_empty) + try: + db.execute("UPDATE group_receipts SET state='revoking', active=1, body=? WHERE key=?", + (json.dumps(receipt, sort_keys=True), key)) + except sqlite3.IntegrityError: + raise RuntimeError("another group operation is unresolved") from None + return receipt + + def save(self, receipt: dict[str, Any]) -> None: + active = 1 if receipt["state"] in {"pending", "revoking", "uncertain"} else None + with self._db() as db: + written = db.execute("UPDATE group_receipts SET state=?, active=?, body=? WHERE key=? AND binding=?", + (receipt["state"], active, json.dumps(receipt, sort_keys=True), receipt["key"], receipt["binding"])) + if written.rowcount != 1: + raise GroupProvenanceUnavailable() diff --git a/identity-provisioner/provisioner.py b/identity-provisioner/provisioner.py index d97b409..1e66be6 100644 --- a/identity-provisioner/provisioner.py +++ b/identity-provisioner/provisioner.py @@ -4,12 +4,15 @@ from __future__ import annotations from dataclasses import dataclass import json +import hashlib import re import time from typing import Any, Callable from urllib.error import HTTPError, URLError from urllib.request import Request, urlopen +from group_provenance import GroupProvenance, GroupProvenanceUnavailable + @dataclass(frozen=True) class Result: @@ -19,6 +22,14 @@ class Result: resumed: bool +@dataclass(frozen=True) +class GroupResult(Result): + ensure_idempotency_key: str + ensure_correlation_id: str + created_groups: tuple[dict[str, Any], ...] + created_memberships: tuple[dict[str, Any], ...] + + @dataclass(frozen=True) class DriftResult: provider: str @@ -53,7 +64,9 @@ class LLDAPProvisioner: opener: Callable = urlopen, auth_rejected_cooldown: float = 30.0, clock: Callable[[], float] = time.monotonic, + group_provenance: GroupProvenance | None = None, ) -> None: + self.group_provenance = group_provenance self.base_url = base_url.rstrip("/") self.admin_password = admin_password self.opener = opener @@ -217,7 +230,10 @@ mutation Remove($userId: String!, $groupId: Int!) { if exc.code == "auth_rejected": self._auth_rejected_until = now + self.auth_rejected_cooldown raise - return {"status": "ready", "dependency": "directory"} + if self.group_provenance is not None: + self.group_provenance.check() + return {"status": "ready", "dependency": "directory", + "service_groups": "available" if self.group_provenance is not None else "disabled"} def _login(self, timeout: float = 10) -> str: return directory_login( @@ -227,53 +243,156 @@ mutation Remove($userId: String!, $groupId: Int!) { timeout=timeout, ) - def ensure_service_groups(self, payload: dict[str, Any]) -> Result: - """Create the listed service-access groups if absent and add the listed - memberships. Never removes anything and never creates users.""" - groups, members = _service_group_request(payload) - token = self._login() - users, directory_groups = self._directory(token) - known = {str(u.get("id")) for u in users} - unknown = sorted({u for us in members.values() for u in us} - known) - if unknown: - raise ValueError(f"unknown directory users: {unknown}") - existed = {str(g.get("displayName")) for g in directory_groups} - ids = {name: self._ensure_group(token, directory_groups, name) for name in groups} - for group, users_ in members.items(): - for username in users_: - self._add_group(token, username, ids[group]) - for group, users_ in members.items(): - for username in users_: - user = self._user(token, username) or {} - if group not in {str(g.get("displayName")) for g in user.get("groups", ())}: - raise RuntimeError(f"membership of {username!r} in {group!r} did not read back") - return Result("netkingdom-lldap", ",".join(groups), "groups_ensured", set(groups) <= existed) + def _group_store(self) -> GroupProvenance: + store = self.group_provenance + if store is None: + raise GroupProvenanceUnavailable() + store.check() + return store - def revoke_service_groups(self, payload: dict[str, Any]) -> Result: - """Rollback of ensure_service_groups: remove exactly the listed memberships, - and delete a listed group only when delete_if_empty is true and no member - remains.""" + @staticmethod + def _group_binding(groups, members, correlation_id) -> str: + # Original correlation is part of the exact ensure binding. + canonical = {"ensure_correlation_id": correlation_id, "groups": sorted(groups), "members": {g: sorted(members.get(g, [])) for g in sorted(groups)}} + return hashlib.sha256(json.dumps(canonical, sort_keys=True, separators=(",", ":")).encode()).hexdigest() + + @staticmethod + def _group_result(receipt, *, resumed=False) -> GroupResult: + status = "groups_ensured" if receipt["state"] == "ensured" else ( + "groups_revoked_and_deleted" if receipt.get("deleted") else "groups_revoked") + return GroupResult("netkingdom-lldap", ",".join(receipt["requested_groups"]), status, resumed, + receipt["key"], receipt["ensure_correlation_id"], tuple(receipt["created_groups"]), tuple(receipt["created_memberships"])) + + def ensure_service_groups(self, payload: dict[str, Any]) -> GroupResult: + """Additive ensure with durable creation provenance, not new authority. + + Replays return the original result, never recreate removed state. Pending + or ambiguous calls survive restart and require independent owner recovery. + """ groups, members = _service_group_request(payload) + store = self._group_store() + receipt, fresh = store.reserve(payload["idempotency_key"], self._group_binding(groups, members, payload["correlation_id"])) + if not fresh: + if receipt["state"] != "ensured": + raise RuntimeError("group provenance unresolved or already rolled back") + return self._group_result(receipt, resumed=True) + attempted = False + try: + token = self._login() + users, directory_groups = self._directory(token) + known = {str(u.get("id")) for u in users} + if {u for us in members.values() for u in us} - known: + raise ValueError("unknown directory users") + ids = {str(g["displayName"]): int(g["id"]) for g in directory_groups} + receipt["requested_groups"] = sorted(groups) + receipt["ensure_correlation_id"] = payload["correlation_id"] + # Complete prestate is persisted before the first directory mutation. + for name in groups: + gid = ids.get(name) + before = sorted(self._group_members(token, gid)) if gid is not None else [] + if len(before) + sum(len(g["before"]) for g in receipt["groups"].values()) > 4096: + raise ValueError("group prestate exceeds bounded provenance capacity") + receipt["groups"][name] = {"id": gid, "before": before, "after": sorted(set(before) | set(members.get(name, [])))} + store.save(receipt) + for name in groups: + state = receipt["groups"][name] + if state["id"] is None: + receipt["intent"] = {"operation": "create_group", "name": name} + store.save(receipt) + attempted = True + gid = self._ensure_group(token, directory_groups, name) + state["id"] = gid + # A successful creation response plus readback is needed. + checked = {str(g["displayName"]): int(g["id"]) for g in self._directory(token)[1]} + if checked.get(name) != gid or self._group_members(token, gid): + raise RuntimeError("created group readback differs") + receipt["created_groups"].append({"name": name, "id": gid}) + receipt["intent"] = None + store.save(receipt) + for username in members.get(name, []): + if username in state["before"]: + continue + receipt["intent"] = {"operation": "add_member", "name": name, "id": state["id"], "username": username} + store.save(receipt) + # Unlike the lifecycle helper, duplicate/racing additions are + # ambiguous, never accepted as proof that this attempt created it. + attempted = True + result = self._gql(token, "mutation Add($userId: String!, $groupId: Int!) { addUserToGroup(userId: $userId, groupId: $groupId) { ok } }", + {"userId": username, "groupId": state["id"]}) + if result.get("addUserToGroup", {}).get("ok") is not True or username not in self._group_members(token, state["id"]): + raise RuntimeError("membership creation did not read back") + receipt["created_memberships"].append({"name": name, "id": state["id"], "username": username}) + receipt["intent"] = None + store.save(receipt) + self._check_group_postimage(token, receipt) + receipt["state"] = "ensured" + store.save(receipt) + except Exception: + receipt["state"] = "uncertain" if attempted else "refused" + receipt["failure_phase"] = "mutation_attempted" if attempted else "before_mutation" + store.save(receipt) + raise + return self._group_result(receipt) + + def _check_group_postimage(self, token, receipt) -> None: + ids = {str(g["displayName"]): int(g["id"]) for g in self._directory(token)[1]} + for name, state in receipt["groups"].items(): + if ids.get(name) != state["id"] or sorted(self._group_members(token, state["id"])) != state["after"]: + raise RuntimeError("group postimage changed; independent owner reconciliation required") + + def revoke_service_groups(self, payload: dict[str, Any]) -> GroupResult: + """Reverse confirmed creations from the owner journal only. + + Caller-supplied creation inventories confer no rights. This protects + preexisting state and refuses detected drift; native LLDAP has no CAS or + membership-generation token, so external writer isolation is still an + owner acceptance prerequisite (readback alone cannot exclude ABA). + """ + groups, members = _service_group_request(payload) + _required(payload, "ensure_idempotency_key", "ensure_correlation_id") delete_if_empty = payload.get("delete_if_empty", False) if not isinstance(delete_if_empty, bool): raise ValueError("delete_if_empty must be true or false") - token = self._login() - _, directory_groups = self._directory(token) - ids = {str(g.get("displayName")): int(g["id"]) for g in directory_groups} - for group, users_ in members.items(): - if group not in ids: - continue - for username in users_: - user = self._user(token, username) or {} - if group in {str(g.get("displayName")) for g in user.get("groups", ())}: - self._remove_group(token, username, ids[group]) - deleted = [] - if delete_if_empty: - for group in groups: - if group in ids and not self._group_members(token, ids[group]): - self._gql(token, "mutation Del($id: Int!) { deleteGroup(groupId: $id) { ok } }", {"id": ids[group]}) - deleted.append(group) - return Result("netkingdom-lldap", ",".join(groups), "groups_revoked" if not deleted else "groups_revoked_and_deleted", False) + store = self._group_store() + receipt = store.claim_revoke(payload["ensure_idempotency_key"], self._group_binding(groups, members, payload["ensure_correlation_id"]), delete_if_empty) + if receipt["state"] == "revoked": + return self._group_result(receipt, resumed=True) + try: + token = self._login() + # Validate the entire operation before the first destructive call. + self._check_group_postimage(token, receipt) + for edge in receipt["created_memberships"]: + receipt["intent"] = {"operation": "remove_member", **edge} + store.save(receipt) + self._remove_group(token, edge["username"], edge["id"]) + expected = receipt["groups"][edge["name"]]["after"] + expected.remove(edge["username"]) + self._check_group_postimage(token, receipt) + receipt["intent"] = None + store.save(receipt) + receipt["deleted"] = [] + if delete_if_empty: + for group in receipt["created_groups"]: + # Only this attempt's groups, never a preexisting empty group. + self._check_group_postimage(token, receipt) + if self._group_members(token, group["id"]): + raise RuntimeError("created group is no longer empty") + receipt["intent"] = {"operation": "delete_group", **group} + store.save(receipt) + self._gql(token, "mutation Del($id: Int!) { deleteGroup(groupId: $id) { ok } }", {"id": group["id"]}) + if any(int(g["id"]) == group["id"] or g["displayName"] == group["name"] for g in self._directory(token)[1]): + raise RuntimeError("group deletion did not read back") + receipt["deleted"].append(group["name"]) + del receipt["groups"][group["name"]] + receipt["intent"] = None + store.save(receipt) + receipt["state"] = "revoked" + store.save(receipt) + except Exception: + receipt["state"] = "uncertain" + store.save(receipt) + raise + return self._group_result(receipt) def _group_members(self, token: str, group_id: int) -> list[str]: value = self._gql(token, "query G($id: Int!) { group(groupId: $id) { users { id } } }", {"id": group_id}) @@ -437,7 +556,19 @@ def _service_group_request(payload: dict[str, Any]) -> tuple[list[str], dict[str platform-suspension and LLDAP admin groups are reserved and can never be touched through this path. """ + if not isinstance(payload, dict): + raise ValueError("service group request must be an object") _required(payload, "groups", "idempotency_key", "correlation_id") + allowed = {"groups", "members", "idempotency_key", "correlation_id", "delete_if_empty", "ensure_idempotency_key", "ensure_correlation_id"} + if set(payload) - allowed: + raise ValueError("unsupported service group request fields") + if not isinstance(payload["idempotency_key"], str) or len(payload["idempotency_key"]) > 256: + raise ValueError("invalid idempotency key") + if "ensure_idempotency_key" in payload and (not isinstance(payload["ensure_idempotency_key"], str) or not 16 <= len(payload["ensure_idempotency_key"]) <= 256): + raise ValueError("invalid ensure idempotency key") + for key in ("correlation_id", "ensure_correlation_id"): + if key in payload and (not isinstance(payload[key], str) or not 1 <= len(payload[key]) <= 256): + raise ValueError("invalid correlation identifier") groups = payload["groups"] if not isinstance(groups, list) or not groups or len(groups) > 16: raise ValueError("groups must be a list of 1 to 16 names") @@ -446,14 +577,18 @@ def _service_group_request(payload: dict[str, Any]) -> tuple[list[str], dict[str raise ValueError(f"invalid service group name {name!r}") if name.startswith(RESERVED_GROUP_PREFIXES): raise ValueError(f"group {name!r} is reserved") - members = payload.get("members") or {} + members = payload.get("members", {}) if not isinstance(members, dict): raise ValueError("members must map group names to usernames") + total_members = 0 for group, users in members.items(): if group not in groups: raise ValueError(f"members names group {group!r}, which is not in groups") if not isinstance(users, list) or not all(isinstance(u, str) and DIRECTORY_USERNAME.fullmatch(u) for u in users): raise ValueError(f"members of {group!r} must be directory usernames") + total_members += len(users) + if total_members > 256: + raise ValueError("at most 256 service memberships per request") return list(dict.fromkeys(groups)), {g: list(dict.fromkeys(u)) for g, u in members.items()} diff --git a/identity-provisioner/server.py b/identity-provisioner/server.py index 0331795..8d43db8 100644 --- a/identity-provisioner/server.py +++ b/identity-provisioner/server.py @@ -6,6 +6,7 @@ import os from urllib.parse import parse_qs, urlsplit import secrets +from group_provenance import GroupProvenance, GroupProvenanceUnavailable from provisioner import DependencyFailure, LLDAPProvisioner, _directory_username, dispatch from password_setup import LLDAPPasswordSetter, PasswordSetupGrants @@ -23,6 +24,8 @@ class Handler(BaseHTTPRequestHandler): if path.path == "/readyz": try: return self._send(200, self.provisioner.preflight()) + except GroupProvenanceUnavailable: + return self._send(503, {"error": "dependency_unavailable", "dependency": "group_provenance"}) except DependencyFailure as exc: return self._send(503, exc.payload()) except RuntimeError: @@ -53,6 +56,8 @@ class Handler(BaseHTTPRequestHandler): return self._send(404, {"error": "not_found"}) except (ValueError, json.JSONDecodeError) as exc: return self._send(400, {"error": "invalid_request", "message": str(exc)}) + except GroupProvenanceUnavailable: + return self._send(503, {"error": "dependency_unavailable", "dependency": "group_provenance"}) except DependencyFailure as exc: return self._send(503, exc.payload()) except RuntimeError: @@ -109,10 +114,19 @@ class Handler(BaseHTTPRequestHandler): return +def configured_group_provenance(): + path = os.environ.get("GROUP_PROVENANCE_DB", "") + try: + return GroupProvenance(path) if path else None + except (OSError, ValueError, RuntimeError): + raise GroupProvenanceUnavailable() from None + + def main(): Handler.provisioner = LLDAPProvisioner( base_url=os.environ["LLDAP_URL"], admin_password=os.environ["LLDAP_ADMIN_PASSWORD"], + group_provenance=configured_group_provenance(), ) Handler.service_token = os.environ["PROVISIONER_SERVICE_TOKEN"].strip() if not Handler.service_token: diff --git a/identity-provisioner/tests/test_directory_contract.py b/identity-provisioner/tests/test_directory_contract.py index 5f5dfb8..00491ca 100644 --- a/identity-provisioner/tests/test_directory_contract.py +++ b/identity-provisioner/tests/test_directory_contract.py @@ -199,7 +199,7 @@ class DirectoryContractTests(unittest.TestCase): def test_preflight_covers_login_and_directory_read(self): self.assertEqual( - {"status": "ready", "dependency": "directory"}, + {"status": "ready", "dependency": "directory", "service_groups": "disabled"}, self.provisioner.preflight(), ) self.assertEqual(1, self.state.logins) diff --git a/identity-provisioner/tests/test_group_provenance.py b/identity-provisioner/tests/test_group_provenance.py new file mode 100644 index 0000000..1bcb284 --- /dev/null +++ b/identity-provisioner/tests/test_group_provenance.py @@ -0,0 +1,326 @@ +"""Synthetic actual provisioner/HTTP/storage proof; no live LDAP or authority.""" +import concurrent.futures +from dataclasses import asdict +from http.server import ThreadingHTTPServer +import json +import os +import sqlite3 +from pathlib import Path +import tempfile +import threading +import unittest +from unittest.mock import patch +from urllib.error import HTTPError +from urllib.request import Request, urlopen + +from group_provenance import GroupProvenance +from provisioner import LLDAPProvisioner, dispatch +from server import Handler, configured_group_provenance +from test_service_groups import Directory, req + + +def rollback(**kw): + return req(ensure_idempotency_key=req()["idempotency_key"], ensure_correlation_id=req()["correlation_id"], **kw) + + +class ProvenanceTests(unittest.TestCase): + def test_inventory_excludes_preexisting_membership_and_empty_group(self): + d = Directory() + d.members[3].add("tegwick") + result = d.ensure_service_groups(req()) + self.assertEqual([{"name": "forge-coulomb-owners", "id": 4}, {"name": "forge-coulomb-readers", "id": 5}], list(result.created_groups)) + self.assertEqual([{"name": "forge-coulomb-owners", "id": 4, "username": "tegwick"}], list(result.created_memberships)) + d.revoke_service_groups(rollback(delete_if_empty=True)) + self.assertEqual({"tegwick"}, d.members[3]) + self.assertIn("forge-admins", d.groups) + self.assertNotIn("forge-coulomb-owners", d.groups) + + def test_preexisting_empty_group_never_deleted(self): + d = Directory() + d.ensure_service_groups(req()) + d.revoke_service_groups(rollback(delete_if_empty=True)) + self.assertIn("forge-admins", d.groups) + self.assertEqual(set(), d.members[3]) + + def test_unknown_receipt_and_digest_mismatch_before_login(self): + d = Directory() + with self.assertRaises(ValueError): + d.revoke_service_groups(rollback()) + d.ensure_service_groups(req()) + d.calls.clear() + with patch.object(d, "_login", side_effect=AssertionError("must not login")): + with self.assertRaises(ValueError): + d.revoke_service_groups(rollback(members={"forge-admins": ["agent-codex"]})) + with self.assertRaises(ValueError): + d.ensure_service_groups(req(groups=["another-group"], members={})) + self.assertEqual([], d.calls) + + def test_unsigned_inventory_and_legacy_revoke_cannot_mutate(self): + d = Directory() + d.ensure_service_groups(req()) + d.calls.clear() + for payload in [req(), rollback(created_groups=[{"name": "forge-admins", "id": 3}]), rollback(created_memberships=[]), rollback(provenance={})]: + with self.assertRaises(ValueError): + d.revoke_service_groups(payload) + self.assertEqual([], d.calls) + + def test_shared_membership_drift_refuses_all_destructive_calls(self): + d = Directory() + d.ensure_service_groups(req()) + d.members[3].add("agent-codex") + d.calls.clear() + with self.assertRaisesRegex(RuntimeError, "postimage"): + d.revoke_service_groups(rollback(delete_if_empty=True)) + self.assertEqual([], d.calls) + self.assertEqual({"tegwick", "agent-codex"}, d.members[3]) + + def test_replaced_group_identity_refuses_all_destructive_calls(self): + d = Directory() + d.ensure_service_groups(req()) + d.groups["forge-admins"] = 10 + d.members[10] = {"tegwick"} + d.calls.clear() + with self.assertRaisesRegex(RuntimeError, "postimage"): + d.revoke_service_groups(rollback()) + self.assertEqual([], d.calls) + + def test_successful_ensure_replay_never_recreates_later_removed_state(self): + d = Directory() + first = d.ensure_service_groups(req()) + d.members[3].clear() + d.calls.clear() + replay = d.ensure_service_groups(req()) + self.assertTrue(replay.resumed) + self.assertEqual(first.created_memberships, replay.created_memberships) + self.assertEqual([], d.calls) + self.assertEqual(set(), d.members[3]) + + def test_interrupted_add_persists_intent_and_blocks_restart_retry(self): + d = Directory() + original = d._gql + def lost_response(token, query, variables): + result = original(token, query, variables) + if "addUserToGroup" in query: + raise RuntimeError("synthetic lost response") + return result + d._gql = lost_response + with self.assertRaises(RuntimeError): + d.ensure_service_groups(req()) + d.group_provenance = GroupProvenance(d.group_provenance.path) + receipt = d.group_provenance.get(req()["idempotency_key"], d._group_binding(req()["groups"], req()["members"], req()["correlation_id"])) + self.assertEqual("uncertain", receipt["state"]) + self.assertEqual("add_member", receipt["intent"]["operation"]) + self.assertEqual([], receipt["created_memberships"]) + d.calls.clear() + for operation, payload in [(d.ensure_service_groups, req()), (d.revoke_service_groups, rollback()), (d.ensure_service_groups, req(idempotency_key="different-key-is-not-recovery"))]: + with self.assertRaises(RuntimeError): + operation(payload) + self.assertEqual([], d.calls) + + def test_interrupted_rollback_retains_confirmed_inventory_and_pending_intent(self): + d = Directory() + d.ensure_service_groups(req()) + original = d._remove_group + def lost_response(*args): + original(*args) + raise RuntimeError("lost remove response") + d._remove_group = lost_response + with self.assertRaises(RuntimeError): + d.revoke_service_groups(rollback()) + d.group_provenance = GroupProvenance(d.group_provenance.path) + receipt = d.group_provenance.get(req()["idempotency_key"], d._group_binding(req()["groups"], req()["members"], req()["correlation_id"])) + self.assertEqual("uncertain", receipt["state"]) + self.assertEqual("remove_member", receipt["intent"]["operation"]) + self.assertEqual(2, len(receipt["created_memberships"])) + d.calls.clear() + with self.assertRaises(RuntimeError): + d.revoke_service_groups(rollback()) + self.assertEqual([], d.calls) + + def test_unresolved_pre_call_intent_survives_hard_process_loss(self): + d = Directory() + d._gql = lambda *_: (_ for _ in ()).throw(SystemExit("process lost before response")) + with self.assertRaises(SystemExit): + d.ensure_service_groups(req()) + d.group_provenance = GroupProvenance(d.group_provenance.path) + receipt = d.group_provenance.get(req()["idempotency_key"], d._group_binding(req()["groups"], req()["members"], req()["correlation_id"])) + self.assertEqual("pending", receipt["state"]) + self.assertEqual("add_member", receipt["intent"]["operation"]) + with self.assertRaises(RuntimeError): + d.ensure_service_groups(req()) + + def test_missing_config_private_permissions_and_lost_database_fail_closed(self): + with patch.dict(os.environ, {}, clear=True): + self.assertIsNone(configured_group_provenance()) + d = Directory() + path = d.group_provenance.path + with patch.dict(os.environ, {"GROUP_PROVENANCE_DB": path}): + self.assertIsInstance(configured_group_provenance(), GroupProvenance) + os.chmod(path, 0o644) + with self.assertRaises(RuntimeError): + configured_group_provenance() + os.chmod(path, 0o600) + os.unlink(path) + with self.assertRaises(RuntimeError): + configured_group_provenance() + self.assertFalse(Path(path).exists()) + provider = LLDAPProvisioner(base_url="https://invalid", admin_password="synthetic") + with patch.object(provider, "_login", side_effect=AssertionError("no directory")): + with self.assertRaises(RuntimeError): + provider.ensure_service_groups(req()) + with self.assertRaises(RuntimeError): + provider.revoke_service_groups(rollback()) + + def test_missing_intent_row_stops_before_any_directory_effect(self): + d = Directory() + store = d.group_provenance + original_save = store.save + def lose_row(receipt): + if receipt["intent"] is not None: + with sqlite3.connect(store.path) as db: + db.execute("DELETE FROM group_receipts") + original_save(receipt) + store.save = lose_row + with self.assertRaisesRegex(RuntimeError, "group_provenance_unavailable"): + d.ensure_service_groups(req()) + self.assertEqual([], d.calls) + + def test_live_instance_rejects_replaced_database_before_effect(self): + d = Directory() + original = d.group_provenance + replacement = str(Path(d.storage.name) / "replacement.db") + GroupProvenance(replacement, initialize=True) + os.replace(replacement, original.path) + with self.assertRaisesRegex(RuntimeError, "group_provenance_unavailable"): + d.ensure_service_groups(req()) + self.assertEqual([], d.calls) + + def test_failed_before_mutation_is_retained_without_stranding_other_keys(self): + for failure in ("unknown-user", "oversized-prestate", "directory-read-failed"): + d = Directory() + old_members = d._group_members + request = req(members={"forge-admins": ["nobody"]}) if failure == "unknown-user" else req() + if failure == "oversized-prestate": + d._group_members = lambda *_: [f"synthetic{i}" for i in range(4097)] + if failure == "directory-read-failed": + d._group_members = lambda *_: (_ for _ in ()).throw(RuntimeError("read failed")) + with self.assertRaises((ValueError, RuntimeError)): + d.ensure_service_groups(request) + binding = d._group_binding(request["groups"], request["members"], request["correlation_id"]) + receipt = d.group_provenance.get(request["idempotency_key"], binding) + self.assertEqual("refused", receipt["state"]) + self.assertEqual("before_mutation", receipt["failure_phase"]) + self.assertEqual([], d.calls) + d._group_members = old_members + with self.assertRaises(RuntimeError): + d.ensure_service_groups(request) + self.assertEqual("groups_ensured", d.ensure_service_groups(req(idempotency_key="another-valid-operation-key")).status) + + def test_current_correlations_are_bound_to_original_ensure(self): + d = Directory() + d.ensure_service_groups(req()) + d.calls.clear() + with self.assertRaises(ValueError): + d.ensure_service_groups(req(correlation_id="other-case")) + payload = rollback() + payload["ensure_correlation_id"] = "other-case" + with self.assertRaises(ValueError): + d.revoke_service_groups(payload) + self.assertEqual([], d.calls) + + def test_readiness_separates_directory_health_from_group_capability(self): + provider = LLDAPProvisioner(base_url="https://invalid", admin_password="synthetic") + provider._login = lambda **_: "synthetic" + provider._gql = lambda *_args, **_kw: {"groups": []} + self.assertEqual("disabled", provider.preflight()["service_groups"]) + d = Directory() + provider.group_provenance = d.group_provenance + self.assertEqual("available", provider.preflight()["service_groups"]) + os.chmod(d.group_provenance.path, 0o644) + with self.assertRaisesRegex(RuntimeError, "group_provenance_unavailable"): + provider.preflight() + + def test_concurrent_sqlite_reservation_allows_one_original_attempt(self): + d = Directory() + path = d.group_provenance.path + def reserve(_): + return GroupProvenance(path).reserve("synthetic-concurrent-key", "same-binding")[1] + with concurrent.futures.ThreadPoolExecutor(max_workers=8) as pool: + fresh = list(pool.map(reserve, range(24))) + self.assertEqual(1, sum(fresh)) + + def test_bounded_records_never_age_or_evict_into_authority(self): + d = Directory() + store = GroupProvenance(d.group_provenance.path, capacity=1) + receipt, _ = store.reserve("one", "binding") + receipt.update(state="ensured", requested_groups=[], ensure_correlation_id="test") + store.save(receipt) + with self.assertRaisesRegex(RuntimeError, "capacity"): + store.reserve("two", "binding") + self.assertFalse(store.reserve("one", "binding")[1]) + + def test_rollback_replay_and_key_reuse_do_not_recreate(self): + d = Directory() + d.ensure_service_groups(req()) + first = d.revoke_service_groups(rollback(delete_if_empty=True)) + d.calls.clear() + result = d.revoke_service_groups(rollback(delete_if_empty=True)) + self.assertEqual(first.status, result.status) + self.assertTrue(result.resumed) + with self.assertRaises(RuntimeError): + d.ensure_service_groups(req()) + with self.assertRaises(ValueError): + d.revoke_service_groups(rollback(delete_if_empty=False)) + self.assertEqual([], d.calls) + + def test_real_http_handler_reinitialized_from_config_retains_receipt(self): + d = Directory() + path = d.group_provenance.path + def boot(): + with patch.dict(os.environ, {"GROUP_PROVENANCE_DB": path}): + provider = LLDAPProvisioner(base_url="https://synthetic.invalid", admin_password="unused", group_provenance=configured_group_provenance()) + # Only directory transport is synthetic; actual initialized adapter, + # dispatch, HTTP authentication, configuration and SQLite are used. + for name in ("_login", "_directory", "_group_members", "_ensure_group", "_gql", "_remove_group"): + setattr(provider, name, getattr(d, name)) + class LocalHandler(Handler): + provisioner = provider + service_token = "synthetic-service-token" + server = ThreadingHTTPServer(("127.0.0.1", 0), LocalHandler) + thread = threading.Thread(target=server.serve_forever, daemon=True) + thread.start() + return server, thread + def stop(server, thread): + server.shutdown(); server.server_close(); thread.join() + def post(server, endpoint, payload): + request = Request(f"http://127.0.0.1:{server.server_port}/v1/groups/{endpoint}", data=json.dumps(payload).encode(), + headers={"Authorization": "Bearer synthetic-service-token"}) + with urlopen(request) as response: + return json.load(response) + server, thread = boot() + try: + created = post(server, "ensure", req()) + finally: + stop(server, thread) + server, thread = boot() + try: + d.calls.clear() + replay = post(server, "ensure", req()) + self.assertTrue(replay["resumed"]) + self.assertEqual(created["created_memberships"], replay["created_memberships"]) + self.assertEqual([], d.calls) + revoked = post(server, "revoke", rollback(delete_if_empty=True)) + self.assertEqual("groups_revoked_and_deleted", revoked["status"]) + self.assertIn("forge-admins", d.groups) + with self.assertRaises(HTTPError) as error: + post(server, "revoke", req()) + self.assertEqual(400, error.exception.code) + d.calls.clear() + os.chmod(path, 0o644) + with self.assertRaises(HTTPError) as error: + post(server, "ensure", req(idempotency_key="different-invocation-key")) + self.assertEqual(503, error.exception.code) + self.assertEqual({"error": "dependency_unavailable", "dependency": "group_provenance"}, json.load(error.exception)) + self.assertEqual([], d.calls) + finally: + stop(server, thread) diff --git a/identity-provisioner/tests/test_service_groups.py b/identity-provisioner/tests/test_service_groups.py index 07a38ff..c6dd48b 100644 --- a/identity-provisioner/tests/test_service_groups.py +++ b/identity-provisioner/tests/test_service_groups.py @@ -1,4 +1,8 @@ import unittest +import tempfile +from pathlib import Path + +from group_provenance import GroupProvenance from provisioner import LLDAPProvisioner, dispatch @@ -7,6 +11,8 @@ class Directory(LLDAPProvisioner): """In-memory LLDAP: users, groups and memberships.""" def __init__(self): + self.storage = tempfile.TemporaryDirectory() + self.group_provenance = GroupProvenance(str(Path(self.storage.name) / "groups.db"), initialize=True) self.users = {"tegwick", "agent-codex"} self.groups = {"lldap_admin": 1, "tenant:trial:one:users": 2, "forge-admins": 3} self.members = {1: {"tegwick"}, 2: set(), 3: set()} @@ -44,6 +50,9 @@ class Directory(LLDAPProvisioner): return sorted(self.members[group_id]) def _gql(self, token, query, variables): + if "addUserToGroup" in query: + self._add_group(token, variables["userId"], variables["groupId"]) + return {"addUserToGroup": {"ok": True}} if "deleteGroup" in query: self.calls.append(("delete", variables["id"])) name = next(n for n, i in self.groups.items() if i == variables["id"]) @@ -108,18 +117,18 @@ class EnsureTests(unittest.TestCase): class RevokeTests(unittest.TestCase): def test_removes_only_listed_memberships_and_keeps_groups_by_default(self): d = Directory() + d.members[3].add("agent-codex") dispatch(d, "/v1/groups/ensure", req()) - d.members[d.groups["forge-admins"]].add("agent-codex") - r = dispatch(d, "/v1/groups/revoke", req()) + r = dispatch(d, "/v1/groups/revoke", req(ensure_idempotency_key=req()["idempotency_key"], ensure_correlation_id=req()["correlation_id"])) self.assertEqual("groups_revoked", r.status) self.assertEqual({"agent-codex"}, d.members[d.groups["forge-admins"]]) self.assertIn("forge-coulomb-owners", d.groups) def test_deletes_only_empty_listed_groups_when_asked(self): d = Directory() + d.members[3].add("agent-codex") dispatch(d, "/v1/groups/ensure", req()) - d.members[d.groups["forge-admins"]].add("agent-codex") - r = dispatch(d, "/v1/groups/revoke", req(delete_if_empty=True)) + r = dispatch(d, "/v1/groups/revoke", req(delete_if_empty=True, ensure_idempotency_key=req()["idempotency_key"], ensure_correlation_id=req()["correlation_id"])) self.assertEqual("groups_revoked_and_deleted", r.status) self.assertIn("forge-admins", d.groups) self.assertNotIn("forge-coulomb-owners", d.groups) diff --git a/sso-mfa/bootstrap/sops-custody-fd.py b/sso-mfa/bootstrap/sops-custody-fd.py new file mode 100644 index 0000000..6b1643d --- /dev/null +++ b/sso-mfa/bootstrap/sops-custody-fd.py @@ -0,0 +1,54 @@ +#!/usr/bin/env python3 +"""Attended custody through a sealed anonymous Linux descriptor, never a key path.""" +import argparse +import fcntl +import getpass +import os +import re +import subprocess +import sys +import warnings + +SEALS=fcntl.F_SEAL_WRITE|fcntl.F_SEAL_GROW|fcntl.F_SEAL_SHRINK|fcntl.F_SEAL_SEAL + +class Refused(Exception):pass + +def sealed_key(material): + if not isinstance(material,bytes) or not 1<=len(material)<=16384 or not re.fullmatch(b'AGE-SECRET-KEY-1[A-Z0-9]+\n?',material):raise Refused() + fd=os.memfd_create('netkingdom-custody',os.MFD_CLOEXEC|os.MFD_ALLOW_SEALING) + try: + os.fchmod(fd,0o600) + with os.fdopen(os.dup(fd),'wb') as out:out.write(material);out.flush() + os.lseek(fd,0,os.SEEK_SET) + fcntl.fcntl(fd,fcntl.F_ADD_SEALS,SEALS) + if fcntl.fcntl(fd,fcntl.F_GET_SEALS)!=SEALS:raise Refused() + return fd + except BaseException:os.close(fd);raise + +def run(material,recipient,command): + if not re.fullmatch(r'age1[023456789acdefghjklmnpqrstuvwxyz]{58}',recipient) or not command:raise Refused() + fd=sealed_key(material) + try: + result=subprocess.run(['age-keygen','-y',f'/proc/self/fd/{fd}'],pass_fds=(fd,),stdout=subprocess.PIPE,stderr=subprocess.DEVNULL,timeout=10) + if result.returncode or result.stdout.strip()!=recipient.encode():raise Refused() + # The key must not compete with any ambient custody source. Children + # receive only this exact inherited descriptor, with close-on-exec elsewhere. + env={k:v for k,v in os.environ.items() if k not in ('NETKINGDOM_CUSTODY_AGE_KEY_FILE','NETKINGDOM_CUSTODY_AGE_KEY_FD','SOPS_AGE_KEY_FILE','SOPS_AGE_KEY','SOPS_AGE_KEY_CMD')} + env['NETKINGDOM_CUSTODY_AGE_KEY_FD']=str(fd) + return subprocess.run(command,env=env,pass_fds=(fd,)).returncode + finally:os.close(fd) + +def main(): + p=argparse.ArgumentParser(description=__doc__);p.add_argument('--expected-recipient',required=True);p.add_argument('command',nargs=argparse.REMAINDER);args=p.parse_args() + command=args.command[1:] if args.command[:1]==['--'] else args.command + try: + if (not all(os.isatty(fd) for fd in (0,1,2)) or not command + or not re.fullmatch(r'age1[023456789acdefghjklmnpqrstuvwxyz]{58}',args.expected_recipient)):raise Refused() + with warnings.catch_warnings(): + warnings.simplefilter('error',getpass.GetPassWarning) + material=getpass.getpass('Paste custody age private key line (input hidden): ').encode()+b'\n' + return run(material,args.expected_recipient,command) + except Exception: + print('ERROR: sealed custody input or execution refused; private diagnostics suppressed.',file=sys.stderr);return 1 + +if __name__=='__main__':raise SystemExit(main()) diff --git a/sso-mfa/bootstrap/sops-custody-unlock.sh b/sso-mfa/bootstrap/sops-custody-unlock.sh index 9e9fa46..fd88e49 100755 --- a/sso-mfa/bootstrap/sops-custody-unlock.sh +++ b/sso-mfa/bootstrap/sops-custody-unlock.sh @@ -14,6 +14,7 @@ EXPECTED_RECIPIENT="${EXPECTED_AGE_RECIPIENT:-}" SOURCE_FILE="" INPUT_MODE="" CHECK_ONLY=false +SEALED_FD=false COMMAND=() usage() { @@ -27,6 +28,7 @@ Options: --from-stdin Read custody age key material from stdin. --prompt Prompt for one AGE-SECRET-KEY-1... line. --check-only Validate the supplied key and exit. + --sealed-fd Hidden prompt, explicit command, no named key file. -h, --help Show this help. Examples: @@ -83,6 +85,10 @@ while [[ $# -gt 0 ]]; do INPUT_MODE="prompt" shift ;; + --sealed-fd) + SEALED_FD=true + shift + ;; --check-only) CHECK_ONLY=true shift @@ -121,6 +127,11 @@ if [[ "$CHECK_ONLY" == false && ${#COMMAND[@]} -eq 0 && ! -t 0 ]]; then fail "an interactive custody shell requires a terminal; supply an explicit command" fi +if [[ "$SEALED_FD" == true ]]; then + [[ "$INPUT_MODE" == prompt && "$CHECK_ONLY" == false && ${#COMMAND[@]} -gt 0 ]] || fail "sealed descriptor mode requires --prompt and an explicit command" + exec python3 "$SCRIPT_DIR/sops-custody-fd.py" --expected-recipient "$EXPECTED_RECIPIENT" -- "${COMMAND[@]}" +fi + TMP_DIR="$(mktemp -d "${TMPDIR:-/tmp}/netkingdom-sops-age.XXXXXX")" KEY_FILE="$TMP_DIR/keys.txt" cleanup() { @@ -185,7 +196,7 @@ if [[ "$CHECK_ONLY" == true ]]; then fi # The supplied key must not compete with an ambient inline key or key command. -unset SOPS_AGE_KEY SOPS_AGE_KEY_CMD +unset SOPS_AGE_KEY SOPS_AGE_KEY_CMD NETKINGDOM_CUSTODY_AGE_KEY_FD export SOPS_AGE_KEY_FILE="$KEY_FILE" export NETKINGDOM_CUSTODY_AGE_KEY_FILE="$KEY_FILE" diff --git a/sso-mfa/bootstrap/sops-private.py b/sso-mfa/bootstrap/sops-private.py index 21c20b2..c67d4a3 100644 --- a/sso-mfa/bootstrap/sops-private.py +++ b/sso-mfa/bootstrap/sops-private.py @@ -3,6 +3,7 @@ from __future__ import annotations import json +import fcntl import os from pathlib import Path import re @@ -99,7 +100,24 @@ def age_document(source, recipient, *, raw=None): def private_key_fd(): + inherited = os.environ.get('NETKINGDOM_CUSTODY_AGE_KEY_FD', '') name = os.environ.get('NETKINGDOM_CUSTODY_AGE_KEY_FILE', '') + if inherited: + if name or not re.fullmatch(r'[1-9][0-9]{0,8}',inherited) or int(inherited)<3: + raise Refused('ambiguous custody descriptor') + fd=os.dup(int(inherited)) + try: + info=os.fstat(fd) + required=fcntl.F_SEAL_WRITE|fcntl.F_SEAL_GROW|fcntl.F_SEAL_SHRINK|fcntl.F_SEAL_SEAL + if (not stat.S_ISREG(info.st_mode) or info.st_uid!=os.getuid() or info.st_mode&0o077 + or info.st_nlink!=0 or not 0", "safe_result": "private /tmp/nk49-keycape-native-fixture-review-/result.json" }, - "scope": "candidate itself on complete synthetic format/integrity archive fixture; no native PI/LLDAP/Authelia proof, real custody, real factor or real client credential", + "scope": "candidate itself on complete synthetic format/integrity archive fixture; no native PI/LLDAP/Authelia proof, real custody, real factor or real client credential; successor binds sealed-descriptor custody preparation sources; synthetic fixture itself overrides custody with generated fixture key as before", "mutations": [ "conditional creation of exact fresh local containers and local-driver tmpfs volumes", "private synthetic file import to those volumes", diff --git a/tests/test_prepare_isolated_keycape.py b/tests/test_prepare_isolated_keycape.py new file mode 100644 index 0000000..f486a77 --- /dev/null +++ b/tests/test_prepare_isolated_keycape.py @@ -0,0 +1,108 @@ +import copy +import hashlib +import importlib.util +import json +import os +from pathlib import Path +import tempfile +import pty +import subprocess +import unittest +from unittest.mock import patch + +ROOT=Path(__file__).parents[1] +spec=importlib.util.spec_from_file_location('prepare',ROOT/'tools/bootstrap-recovery/prepare-isolated-keycape.py') +p=importlib.util.module_from_spec(spec);spec.loader.exec_module(p) + +class PreparationTests(unittest.TestCase): + def setUp(self): + self.temp=tempfile.TemporaryDirectory();self.addCleanup(self.temp.cleanup) + self.path=Path(self.temp.name)/'ciphertext';self.path.write_bytes(b'synthetic ciphertext');self.path.chmod(0o600) + self.transfer={'schema':'netkingdom.identity-offsite.v1','status':'readback_verified','provider':'Storage Share', + 'folder':'Backup/railiance-backups/net-kingdom-identity','artifact':'net-kingdom-identity-20261016T063159Z-5882771aa2ce.tar.age', + 'ciphertext_sha256':'a'*64,'ciphertext_bytes':len(b'synthetic ciphertext'),'etag':'"synthetic"','fetched_path':str(self.path), + 'independent_owner_retrieval_verified':True} + self.admission={name:'a'*64 for name in ('ciphertext_sha256','manifest_sha256','verifier_sha256','trust_sha256','tls_cert_sha256')} + self.admission.update(issuer='https://issuer.example',consumer=dict(client_id='existing',secret_env='KEYCAPE_EXISTING_SECRET',audience='existing-api',scope='read',subject='svc:existing',tenant='existing',roles='reader',deny_scope='write'), + source_sha256={name:'sha256:'+hashlib.sha256((ROOT/name).read_bytes()).hexdigest() for name in p.SOURCE_PATHS}) + def test_complete_public_bindings_admitted_without_reading_ciphertext(self): + original=Path.read_bytes + def read(path): + self.assertNotEqual(path,self.path) + return original(path) + with patch.object(Path,'read_bytes',read):self.assertTrue(p.validate(self.transfer,self.admission,ROOT)) + def test_oversize_metadata_refused_before_any_content_read(self): + self.path.write_bytes(b'x'*(128*1024+1)) + with patch.object(Path,'read_bytes',side_effect=AssertionError('unbounded read')): + with self.assertRaises(p.Refused):p.read_metadata(self.path) + def test_duplicate_json_keys_refused_at_every_level(self): + for raw in (b'{"issuer":"one","issuer":"two"}',b'{"consumer":{"client_id":"one","client_id":"two"}}'): + with self.subTest(raw=raw),self.assertRaises(p.Refused):p.strict_json(raw) + def test_bounded_metadata_reader_accepts_exact_limit(self): + self.path.write_bytes(b'x'*128) + self.assertEqual(p.read_metadata(self.path,128),b'x'*128) + with self.assertRaises(p.Refused):p.read_metadata(self.path,127) + def test_wrong_archive_binding_refused(self): + self.admission['ciphertext_sha256']='b'*64 + with self.assertRaises(p.Refused):p.validate(self.transfer,self.admission,ROOT) + def test_ciphertext_size_mismatch_refused_without_decryption(self): + self.transfer['ciphertext_bytes']+=1 + with self.assertRaises(p.Refused):p.validate(self.transfer,self.admission,ROOT) + def test_missing_independent_transfer_refused(self): + self.transfer['independent_owner_retrieval_verified']=False + with self.assertRaises(p.Refused):p.validate(self.transfer,self.admission,ROOT) + def test_missing_or_drifted_source_refused(self): + for source in p.SOURCE_PATHS: + bad=copy.deepcopy(self.admission);bad['source_sha256'][source]='0'*64 + with self.subTest(source=source),self.assertRaises(p.Refused):p.validate(self.transfer,bad,ROOT) + bad=copy.deepcopy(self.admission);bad['source_sha256'].pop(next(iter(p.SOURCE_PATHS))) + with self.assertRaises(p.Refused):p.validate(self.transfer,bad,ROOT) + def test_unreviewed_extra_dependency_refused(self): + self.admission['source_sha256']['unreviewed.py']='a'*64 + with self.assertRaises(p.Refused):p.validate(self.transfer,self.admission,ROOT) + def test_ambiguous_or_unsupported_issuer_refused(self): + for issuer in ('http://issuer.example','https://u:p@issuer.example','https://issuer.example:8443','https://issuer.example/path','https://issuer.example?x=1','https://issuer.example#fragment'): + self.admission['issuer']=issuer + with self.subTest(issuer=issuer),self.assertRaises(p.Refused):p.validate(self.transfer,self.admission,ROOT) + def test_missing_consumer_or_overlapping_denial_refused(self): + self.admission['consumer']['deny_scope']='read' + with self.assertRaises(p.Refused):p.validate(self.transfer,self.admission,ROOT) + self.admission['consumer'].pop('roles') + with self.assertRaises(p.Refused):p.validate(self.transfer,self.admission,ROOT) + def test_symlink_private_input_refused_without_read(self): + link=self.path.parent/'link';link.symlink_to(self.path) + with self.assertRaises(p.Refused):p.metadata_path(link) + def test_publicly_readable_private_input_refused(self): + self.path.chmod(0o644) + with self.assertRaises(p.Refused):p.metadata_path(self.path) + def test_bad_admission_digest_never_reads_tls_or_starts_custody(self): + admission=self.path.parent/'admission';admission.write_text(json.dumps(self.admission));admission.chmod(0o600) + argv=['prepare','--transfer-receipt',str(self.path),'--admission',str(admission),'--admission-sha256','b'*64, + '--tls-cert','/never/read-cert','--tls-key','/never/read-key','--trust','/never/read-trust'] + with patch('sys.argv',argv),patch('builtins.print') as output: + self.assertEqual(p.main(),1) + self.assertEqual(output.call_args.args,('ERROR: reviewed metadata/source/dependency preflight refused; no key requested.',)) + def test_actual_launcher_bad_digest_stops_before_hidden_prompt(self): + master,slave=pty.openpty() + try: + args=['bash',str(ROOT/'tools/bootstrap-recovery/run-isolated-keycape.sh'),str(self.path),str(self.path),'b'*64,str(self.path),str(self.path),str(self.path)] + process=subprocess.Popen(args,stdin=slave,stdout=slave,stderr=slave) + process.wait(timeout=5);os.close(slave);slave=None + output=b'' + while True: + try: + block=os.read(master,4096) + if not block:break + output+=block + except OSError:break + self.assertEqual(process.returncode,1) + self.assertIn(b'preflight refused; no key requested.',output) + self.assertNotIn(b'Paste custody',output) + self.assertNotIn(b'Enter only',output) + finally: + if slave is not None:os.close(slave) + os.close(master) + def test_launcher_prepares_before_sealed_descriptor_custody(self): + script=(ROOT/'tools/bootstrap-recovery/run-isolated-keycape.sh').read_text() + self.assertLess(script.index('python3 tools/bootstrap-recovery/prepare-isolated-keycape.py'),script.index('bash sso-mfa/bootstrap/sops-custody-unlock.sh')) + self.assertIn('--sealed-fd --prompt -- python3',script) diff --git a/tests/test_private_manifest_binding.py b/tests/test_private_manifest_binding.py new file mode 100644 index 0000000..c8bd1c9 --- /dev/null +++ b/tests/test_private_manifest_binding.py @@ -0,0 +1,98 @@ +"""Generated archive/key only; actual age authentication + archive restore. + +Docker and issuer behavior are intentionally mocked here, not native evidence. +""" +import base64 +from datetime import datetime,timedelta,timezone +import hashlib +import importlib.util +import json +import os +from pathlib import Path +import subprocess +import time +from unittest.mock import MagicMock,patch +import pytest +from cryptography import x509 +from cryptography.hazmat.primitives import hashes,serialization +from cryptography.hazmat.primitives.asymmetric import rsa +from cryptography.x509.oid import NameOID +from test_identity_bundle import stage,b + +ROOT=Path(__file__).parents[1] +def load(name,path): + spec=importlib.util.spec_from_file_location(name,path);m=importlib.util.module_from_spec(spec);spec.loader.exec_module(m);return m +s=load('session_manifest',ROOT/'tools/bootstrap-recovery/isolated-keycape-session.py') +h=load('helper_manifest',ROOT/'sso-mfa/bootstrap/sops-custody-fd.py') + +@pytest.fixture +def envelope(stage,tmp_path,monkeypatch): + raw=subprocess.run(['age-keygen'],stdout=subprocess.PIPE,stderr=subprocess.DEVNULL,check=True).stdout + key=h.sealed_key(next(line for line in raw.splitlines() if line.startswith(b'AGE-SECRET-KEY-'))+b'\n') + recipient=subprocess.run(['age-keygen','-y',f'/proc/self/fd/{key}'],pass_fds=(key,),stdout=subprocess.PIPE,stderr=subprocess.DEVNULL,check=True).stdout.decode().strip() + profile=b.profile();profile['public_recipient']=recipient;monkeypatch.setattr(b,'profile',lambda:profile);monkeypatch.setattr(s.bundle,'profile',lambda:profile) + artifact=tmp_path/'identity.tar.age';sealed=b.seal(stage,artifact,{'test_only':True}) + signer=rsa.generate_private_key(public_exponent=65537,key_size=2048);name=x509.Name([x509.NameAttribute(NameOID.COMMON_NAME,'synthetic.invalid')]);now=datetime.now(timezone.utc) + cert=(x509.CertificateBuilder().subject_name(name).issuer_name(name).public_key(signer.public_key()).serial_number(1).not_valid_before(now-timedelta(minutes=1)).not_valid_after(now+timedelta(hours=1)).sign(signer,hashes.SHA256())).public_bytes(serialization.Encoding.PEM) + tls={'/recovery-tls/server.pem':cert,'/recovery-tls/trust.pem':cert,'/recovery-tls/server-key.pem':signer.private_bytes(serialization.Encoding.PEM,serialization.PrivateFormat.TraditionalOpenSSL,serialization.NoEncryption())} + transfer=dict(schema='netkingdom.identity-offsite.v1',status='readback_verified',provider='Storage Share',folder='Backup/railiance-backups/net-kingdom-identity',artifact='net-kingdom-identity-20261016T063159Z-5882771aa2ce.tar.age',etag='"synthetic"',ciphertext_sha256=sealed['ciphertext_sha256'],ciphertext_bytes=artifact.stat().st_size,fetched_path=str(artifact),independent_owner_retrieval_verified=True) + admission=dict(ciphertext_sha256=sealed['ciphertext_sha256'],manifest_binding_mode='ciphertext-derived-v1',issuer='https://synthetic.invalid',tls_cert_sha256=hashlib.sha256(cert).hexdigest(),trust_sha256=hashlib.sha256(cert).hexdigest(),verifier_sha256=hashlib.sha256(b'synthetic executable').hexdigest()) + try:yield artifact,key,transfer,admission,tls + finally:os.close(key) + +def execute(envelope,*,wrong_key=False): + artifact,key,transfer,admission,tls=envelope;receipt={};seen={};checkpoints=[] + if wrong_key: + raw=subprocess.run(['age-keygen'],stdout=subprocess.PIPE,stderr=subprocess.DEVNULL,check=True).stdout + key=h.sealed_key(next(line for line in raw.splitlines() if line.startswith(b'AGE-SECRET-KEY-'))+b'\n') + actualrun=s.run + def run(argv,**kwargs):return actualrun(argv,**kwargs) if argv[0]=='age-keygen' else b'' + def native(root,manifest,checked,receipt,**kwargs):seen.update(checked);receipt['status']='synthetic_flow_checked' + execution={'pid':os.getpid(),'sid':os.getsid(0),'pgid':os.getpgrp(),'uid':os.getuid(),'deadline':time.monotonic()+60} + try: + with patch.object(s,'verify_execution'),patch.object(s.local,'preflight',return_value={}),patch.object(s.local.verify.custody,'private_key_fd',side_effect=lambda:os.dup(key)),patch.object(s,'start_reaper',return_value=MagicMock()),patch.object(s,'run',side_effect=run),patch.object(s.adapter,'container_file',return_value=b'synthetic executable'),patch.object(s.adapter,'private_payload',return_value={'files':{'/etc/keycape/config.yaml':base64.b64encode(b'issuer: https://synthetic.invalid\nport: 8080').decode()},'env':{}}),patch.object(s,'provision'),patch.object(s.adapter,'check_jwks'),patch.object(s.adapter,'native_checks',side_effect=native),patch.object(s,'cleanup_all'): + # Key file must be valid at the later public signer construction. + payload_key=rsa.generate_private_key(public_exponent=65537,key_size=2048).private_bytes(serialization.Encoding.PEM,serialization.PrivateFormat.TraditionalOpenSSL,serialization.NoEncryption()) + with patch.object(s.adapter,'private_payload',return_value={'files':{'/etc/keycape/config.yaml':base64.b64encode(b'issuer: https://synthetic.invalid\nport: 8080').decode(),'/etc/keycape/key.pem':base64.b64encode(payload_key).decode()},'env':{}}): + s.execute(transfer,admission,tls,receipt,execution,checkpoint=lambda r:checkpoints.append(json.loads(json.dumps(r)))) + return receipt,seen,checkpoints + finally: + if wrong_key:os.close(key) + +def test_exact_ciphertext_authenticated_restore_derives_private_manifest_without_safe_leak(envelope): + receipt,seen,checkpoints=execute(envelope) + derived=seen['manifest_sha256'];assert len(derived)==64 + safe=json.dumps([receipt,checkpoints]);assert derived not in safe;assert 'manifest_sha256' not in safe + assert receipt['all_manifest_hashes_verified'] is True + assert receipt['temporary_plaintext_cleanup_verified'] is True + +def test_changed_ciphertext_refuses_before_archive_decryption(envelope): + artifact,*_=envelope;artifact.write_bytes(artifact.read_bytes()[:-1]+b'!') + with patch.object(s.adapter,'process') as process,pytest.raises(s.adapter.Refused):execute(envelope) + assert not process.called + +def test_modified_ciphertext_rebound_metadata_still_fails_authenticated_age(envelope): + artifact,key,transfer,admission,tls=envelope + raw=artifact.read_bytes();artifact.write_bytes(raw[:-32]+b'!'*32) + digest=hashlib.sha256(artifact.read_bytes()).hexdigest() + transfer['ciphertext_sha256']=digest;admission['ciphertext_sha256']=digest + with pytest.raises(s.adapter.Refused):execute(envelope) + +def test_wrong_key_refuses_before_decryption_and_never_exposes_manifest(envelope): + with pytest.raises(s.adapter.Refused):execute(envelope,wrong_key=True) + +def test_wrong_explicit_manifest_refuses_after_full_restore(envelope): + admission=envelope[3];admission['manifest_binding_mode']='explicit-digest-v1';admission['manifest_sha256']='0'*64 + with pytest.raises(s.adapter.Refused):execute(envelope) + +@pytest.mark.parametrize('mode',['derived','ciphertext-derived-v2',None,True,{}]) +def test_unknown_binding_mode_refuses_before_effects(envelope,mode): + envelope[3]['manifest_binding_mode']=mode + with patch.object(s.local,'preflight') as preflight,pytest.raises(s.adapter.Refused):execute(envelope) + assert not preflight.called + +def test_conflicting_binding_and_unverified_transfer_refuse(envelope): + admission=envelope[3];transfer=envelope[2];admission['manifest_sha256']='0'*64 + with pytest.raises(s.adapter.Refused):s.adapter.manifest_binding_mode(admission,transfer) + admission.pop('manifest_sha256');transfer['independent_owner_retrieval_verified']=False + with pytest.raises(s.adapter.Refused):s.adapter.manifest_binding_mode(admission,transfer) diff --git a/tests/test_sealed_custody.py b/tests/test_sealed_custody.py new file mode 100644 index 0000000..e69d542 --- /dev/null +++ b/tests/test_sealed_custody.py @@ -0,0 +1,143 @@ +import fcntl +import importlib.util +import json +import os +from pathlib import Path +import signal +import pty +import select +import subprocess +import sys +import tempfile +import time +import unittest +from unittest.mock import patch + +ROOT=Path(__file__).parents[1] +def load(name,path): + spec=importlib.util.spec_from_file_location(name,path);m=importlib.util.module_from_spec(spec);spec.loader.exec_module(m);return m +h=load('fdhelper',ROOT/'sso-mfa/bootstrap/sops-custody-fd.py') +c=load('custody',ROOT/'sso-mfa/bootstrap/sops-private.py') + +class SealedCustodyTests(unittest.TestCase): + @classmethod + def setUpClass(cls): + # Capture generated test key in memory only, never into test output/path. + r=subprocess.run(['age-keygen'],stdout=subprocess.PIPE,stderr=subprocess.DEVNULL,check=True) + cls.material=next(x for x in r.stdout.splitlines() if x.startswith(b'AGE-SECRET-KEY-'))+b'\n' + fd=h.sealed_key(cls.material) + try:cls.recipient=subprocess.run(['age-keygen','-y',f'/proc/self/fd/{fd}'],pass_fds=(fd,),stdout=subprocess.PIPE,stderr=subprocess.DEVNULL,check=True).stdout.decode().strip() + finally:os.close(fd) + def test_sealed_descriptor_private_duplicate_is_cloexec_and_decrypts(self): + encrypted=subprocess.run(['age','-r',self.recipient],input=b'synthetic recovery material',stdout=subprocess.PIPE,stderr=subprocess.DEVNULL,check=True).stdout + original=h.sealed_key(self.material) + try: + with patch.dict(os.environ,{'NETKINGDOM_CUSTODY_AGE_KEY_FD':str(original)},clear=True):fd=c.private_key_fd() + try: + self.assertFalse(os.get_inheritable(fd));self.assertEqual(os.fstat(fd).st_nlink,0) + r=subprocess.run(['age','-d','-i',f'/proc/self/fd/{fd}'],pass_fds=(fd,),input=encrypted,stdout=subprocess.PIPE,stderr=subprocess.DEVNULL) + self.assertEqual(r.returncode,0);self.assertEqual(r.stdout,b'synthetic recovery material') + finally:os.close(fd) + finally:os.close(original) + def test_incomplete_seals_or_named_file_refused(self): + for named in (False,True): + if named: + file=tempfile.TemporaryFile();fd=file.fileno();file.write(self.material);file.flush() + else:fd=os.memfd_create('synthetic-unsealed',os.MFD_CLOEXEC|os.MFD_ALLOW_SEALING);os.write(fd,self.material) + try: + with patch.dict(os.environ,{'NETKINGDOM_CUSTODY_AGE_KEY_FD':str(fd)},clear=True),self.assertRaises((c.Refused,OSError)): + c.private_key_fd() + finally: + if named:file.close() + else:os.close(fd) + def test_ambiguous_sources_refused(self): + with patch.dict(os.environ,{'NETKINGDOM_CUSTODY_AGE_KEY_FD':'3','NETKINGDOM_CUSTODY_AGE_KEY_FILE':'/not/opened'},clear=True),self.assertRaises(c.Refused):c.private_key_fd() + def test_malformed_descriptor_text_refused_before_dup(self): + for value in ('-1','2','03',' 3','3 ','1000000000','3/4'): + with patch.dict(os.environ,{'NETKINGDOM_CUSTODY_AGE_KEY_FD':value},clear=True),patch.object(c.os,'dup') as duplicate: + with self.subTest(value=value),self.assertRaises(c.Refused):c.private_key_fd() + duplicate.assert_not_called() + def test_bad_expected_recipient_refused_before_prompt(self): + with patch('sys.argv',['sealed','--expected-recipient','age1bad','--','true']),patch.object(h.os,'isatty',return_value=True),patch.object(h.getpass,'getpass') as prompt,patch('builtins.print'): + self.assertEqual(h.main(),1);prompt.assert_not_called() + def test_existing_file_mode_still_accepts_private_owner_file(self): + with tempfile.TemporaryDirectory() as temp: + path=Path(temp)/'synthetic-key';path.write_bytes(self.material);path.chmod(0o600) + with patch.dict(os.environ,{'NETKINGDOM_CUSTODY_AGE_KEY_FILE':str(path)},clear=True):fd=c.private_key_fd() + try:self.assertEqual(os.read(fd,16384),self.material) + finally:os.close(fd) + def test_helper_passes_exact_fd_and_clears_ambient_sources(self): + code="""import os,sys +fd=int(os.environ['NETKINGDOM_CUSTODY_AGE_KEY_FD']) +assert os.fstat(fd).st_nlink==0 +assert all(k not in os.environ for k in ('NETKINGDOM_CUSTODY_AGE_KEY_FILE','SOPS_AGE_KEY_FILE','SOPS_AGE_KEY','SOPS_AGE_KEY_CMD')) +""" + with patch.dict(os.environ,{'SOPS_AGE_KEY':'synthetic-competing','NETKINGDOM_CUSTODY_AGE_KEY_FILE':'/not/read'}): + self.assertEqual(h.run(self.material,self.recipient,[sys.executable,'-c',code]),0) + def test_normal_and_error_return_close_parent_fd(self): + actual=h.sealed_key + for status in (0,7): + fds=[] + def capture(material): + fd=actual(material);fds.append(fd);return fd + with patch.object(h,'sealed_key',side_effect=capture):self.assertEqual(h.run(self.material,self.recipient,[sys.executable,'-c',f'raise SystemExit({status})']),status) + with self.assertRaises(OSError):os.fstat(fds[0]) + def test_wrong_recipient_does_not_start_child_and_closes_fd(self): + other=subprocess.run(['age-keygen'],stdout=subprocess.PIPE,stderr=subprocess.DEVNULL,check=True).stdout + other=next(x for x in other.splitlines() if x.startswith(b'AGE-SECRET-KEY-'))+b'\n';fd=h.sealed_key(other) + try:recipient=subprocess.run(['age-keygen','-y',f'/proc/self/fd/{fd}'],pass_fds=(fd,),stdout=subprocess.PIPE,stderr=subprocess.DEVNULL,check=True).stdout.decode().strip() + finally:os.close(fd) + with tempfile.TemporaryDirectory() as temp: + marker=Path(temp)/'child' + with self.assertRaises(h.Refused):h.run(self.material,recipient,[sys.executable,'-c','import pathlib,sys;pathlib.Path(sys.argv[1]).touch()',str(marker)]) + self.assertFalse(marker.exists()) + def test_actual_attended_shell_sealed_mode_hides_key_and_runs_original_fd_consumer(self): + code="import importlib.util,os,sys;spec=importlib.util.spec_from_file_location('c',sys.argv[1]);c=importlib.util.module_from_spec(spec);spec.loader.exec_module(c);fd=c.private_key_fd();assert os.fstat(fd).st_nlink==0;os.close(fd)" + master,slave=pty.openpty() + child=subprocess.Popen(['bash','-x',str(ROOT/'sso-mfa/bootstrap/sops-custody-unlock.sh'),'--expected-recipient',self.recipient,'--sealed-fd','--prompt','--',sys.executable,'-c',code,str(ROOT/'sso-mfa/bootstrap/sops-private.py')],stdin=slave,stdout=slave,stderr=slave) + os.close(slave);captured=b'';sent=False;end=time.monotonic()+5 + try: + while time.monotonic()bundle.LIMIT+4*1024*1024:raise adapter.Refused() root=work/'restored';root.mkdir(mode=0o700) manifest,_=bundle.restore(archive,root) + # Data binding comes from exact reviewed ciphertext, never a new grant. + admission=adapter.private_manifest_admission(admission,manifest) payload=adapter.private_payload(root,manifest,admission['manifest_sha256']) import yaml cfg=yaml.safe_load(base64.b64decode(payload['files']['/etc/keycape/config.yaml'])) diff --git a/tools/bootstrap-recovery/isolated-keycape.py b/tools/bootstrap-recovery/isolated-keycape.py index 32f4643..4cc40e6 100644 --- a/tools/bootstrap-recovery/isolated-keycape.py +++ b/tools/bootstrap-recovery/isolated-keycape.py @@ -34,6 +34,32 @@ def load_bundle(): return module +def manifest_binding_mode(admission,transfer): + """Versioned private-manifest binding; legacy explicit digests stay valid.""" + mode=admission.get('manifest_binding_mode','explicit-digest-v1') + if mode not in ('explicit-digest-v1','ciphertext-derived-v1'):raise Refused() + if (not isinstance(admission.get('ciphertext_sha256'),str) + or not re.fullmatch('[a-f0-9]{64}',admission['ciphertext_sha256']) + or admission['ciphertext_sha256']!=transfer.get('ciphertext_sha256') + or transfer.get('independent_owner_retrieval_verified') is not True):raise Refused() + if mode=='explicit-digest-v1': + if not isinstance(admission.get('manifest_sha256'),str) or not re.fullmatch('[a-f0-9]{64}',admission['manifest_sha256']):raise Refused() + elif 'manifest_sha256' in admission:raise Refused() + return mode + + +def private_manifest_admission(admission,manifest): + # Only called after the fixed ciphertext bytes were hashed, authenticated + # decryption succeeded and bundle.restore checked the complete archive. + # Never return this enriched private object in a safe receipt/checkpoint. + digest=hashlib.sha256(json.dumps(manifest,sort_keys=True,separators=(',',':')).encode()).hexdigest() + mode=admission.get('manifest_binding_mode','explicit-digest-v1') + if mode=='explicit-digest-v1': + if digest!=admission['manifest_sha256']:raise Refused() + elif mode!='ciphertext-derived-v1' or 'manifest_sha256' in admission:raise Refused() + return {**admission,'manifest_sha256':digest} + + def private_payload(root, manifest, expected_manifest_sha256): """Bind the complete validated archive before selecting KeyCape material. diff --git a/tools/bootstrap-recovery/prepare-isolated-keycape.py b/tools/bootstrap-recovery/prepare-isolated-keycape.py new file mode 100644 index 0000000..10a7ef4 --- /dev/null +++ b/tools/bootstrap-recovery/prepare-isolated-keycape.py @@ -0,0 +1,119 @@ +#!/usr/bin/env python3 +"""Validate reviewed public recovery metadata before requesting custody input. + +Never opens TLS material, archive plaintext or any key. This validates bindings, +not owner assent, archive contents or recovery authority. +""" +import argparse +import hashlib +import importlib.util +import json +import os +from pathlib import Path +import re +import stat +from urllib.parse import urlsplit + +ROOT=Path(__file__).resolve().parents[2] +# The executable closure extends the synthetic fixture closure with the attended +# custody entry point and this validator. Tests and historical receipts do not run. +SOURCE_PATHS=tuple(sorted([ + 'sso-mfa/bootstrap/sops-custody-unlock.sh','sso-mfa/bootstrap/sops-private.py','sso-mfa/bootstrap/sops-custody-fd.py', + 'sso-mfa/k8s/privacyidea/key-custody-remote.py', + *('tools/bootstrap-recovery/'+name for name in ( + 'isolated-keycape.py','isolated-keycape-session.py','run-isolated-keycape.sh', + 'prepare-isolated-keycape.py','isolated-lldap.py','identity-bundle.py', + 'identity-backup-profile.json','retrieve-identity-offline.py','identity-offsite.py', + 'verify-identity.py','identity-custody.json','identity-files.py', + 'check-privacyidea-database.py'))])) + +class Refused(Exception):pass + +def require(value): + if not value:raise Refused() + +def metadata_path(path): + path=Path(os.path.abspath(path)) + require(not any(p.is_symlink() for p in (path,*path.parents))) + st=path.stat() + require(stat.S_ISREG(st.st_mode) and st.st_uid==os.getuid() and st.st_nlink==1 and not st.st_mode&0o077) + return path + +def read_metadata(path,limit=128*1024): + path=metadata_path(path) + require(0/dev/null || { printf 'ERROR: required tool unavailable: %s\n' "$tool"; exit 1; } done +python3 tools/bootstrap-recovery/prepare-isolated-keycape.py \ + --transfer-receipt "$1" --admission "$2" --admission-sha256 "$3" \ + --tls-cert "$4" --tls-key "$5" --trust "$6" python3 - <<'PY' import importlib.util from pathlib import Path @@ -29,12 +32,12 @@ print('OK: fixed local target and cached pinned images available.') PY NK49_KC_RUN="$(mktemp -d /tmp/nk49-isolated-keycape-XXXXXXXX)" printf 'Safe evidence directory: %s\n' "$NK49_KC_RUN" -printf '%s\n' 'Owner-reviewed candidate: isolated original issuer/trust and current registered service-client checks only.' +printf '%s\n' 'Isolated original issuer/trust and selected registered service-client checks only.' printf '%s\n' 'No browser, downstream consumer or retained-factor MFA acceptance is claimed.' printf '%s\n' 'Enter only the independently held backup age key at the hidden prompt.' bash sso-mfa/bootstrap/sops-custody-unlock.sh \ --expected-recipient age1zvryunvjhvpkmasskauga2heeg0ztnte9ymgppvjge36ekumk50syr3tsz \ - --prompt -- python3 tools/bootstrap-recovery/isolated-keycape-session.py \ + --sealed-fd --prompt -- python3 tools/bootstrap-recovery/isolated-keycape-session.py \ --transfer-receipt "$1" --admission "$2" --admission-sha256 "$3" \ --tls-cert "$4" --tls-key "$5" --trust "$6" --receipt "$NK49_KC_RUN/recovery.json" printf 'Safe receipt: %s/recovery.json\n' "$NK49_KC_RUN" diff --git a/workplans/NK-WP-0048-security-case-types.md b/workplans/NK-WP-0048-security-case-types.md index 339a65c..a8495da 100644 --- a/workplans/NK-WP-0048-security-case-types.md +++ b/workplans/NK-WP-0048-security-case-types.md @@ -9,7 +9,7 @@ flavor: implementation owner: claude-code topic_slug: netkingdom created: "2026-10-08" -updated: "2026-10-16" +updated: "2026-10-17" depends_on: [] related_repos: - ops-mason @@ -323,7 +323,7 @@ the rationale. The type is `acknowledged`; it becomes `implemented` with T06. ```task id: NK-WP-0048-T06 status: wait -depends_on: [NK-WP-0048-T05, NK-WP-0048-T08, NK-WP-0048-T09, NK-WP-0048-T12, NK-WP-0048-T13, NK-WP-0048-T14, NK-WP-0048-T15, NK-WP-0048-T16, NK-WP-0048-T17, NK-WP-0048-T18, MASON-WP-0010-T03, WARDEN-WP-0044-T03] +depends_on: [NK-WP-0048-T05, NK-WP-0048-T08, NK-WP-0048-T09, NK-WP-0048-T12, NK-WP-0048-T13, NK-WP-0048-T14, NK-WP-0048-T15, NK-WP-0048-T16, NK-WP-0048-T17, NK-WP-0048-T18, NK-WP-0048-T20, MASON-WP-0010-T03, WARDEN-WP-0044-T03] blocking_reason: "Type1.2 architecture has a confirmed source-only founder acknowledgment; canonical1.1 remains unchanged/default and the1.2 reference case stays proposed. Needs explicit version selection/reference-source integration without normal admission bypass, closed owner adapters plus global/native receipt verifier (WARDEN-WP-0044-T03), independently verified six-object structural materialization and CCR42 claims/created-only provenance, exact broker/case/source reconciliation, then separately authorized native mint/delivery and attended registry/consumer verification. Source acknowledgment, a bundle or local tests do not prove implemented/materialized/live status." priority: high state_hub_task_id: "d9b26bcf-a7c6-59a2-92a0-f913d76a203c" @@ -686,24 +686,48 @@ hook; exact pre-existing staged/working brief bytes were restored. The separate canonical integration receipt retains those refusals and projection evidence. Native admission and all other-owner acceptance remain open. -## T14 — Preserve created-only provenance for group ensure and revoke +## T14 — Admit governed group rollback against native owner boundaries ```task id: NK-WP-0048-T14 status: todo +depends_on: [NK-WP-0048-T20] priority: high state_hub_task_id: "fce95440-a25b-5f27-8ef3-a3d182099113" ``` -Owner: NetKingdom. Next action: extend or wrap the existing identity-provisioner -ensure/revoke contract with independently retained prestate and exact newly -created groups/memberships before exposing reversible owner callbacks. Current -ensure returns no membership-creation inventory; current revoke may remove a -membership that predated this case. Completion requires actual helper-backed -positive/refusal fixtures, all validation before mutation, per-attempt evidence -and revocation, readback, ambiguity handling, and rollback that preserves every -pre-existing/shared group and membership. No existing endpoint or service token -is treated as new authorization. +Owner: NetKingdom. Next action: specify a CNPG-backed journal through the +existing `rapp-postgres` database-per-consumer contract, or obtain an explicit +scoped governed exception to accepted ADR-0015 before any production SQLite +proposal. The implemented SQLite backend supplies local/disposable proof only; +a new PVC is not an admitted production path. Prepare the native writer-isolation +contract for the existing identity-provisioner, and bind the +existing protected owner callbacks to the accepted case/source/principal, +per-attempt evidence and session revocation. Identify the actual deployment and +custody owner through the existing ADR-0015 routes before proposing a rollout; +an existing service token, caller-supplied JSON or provenance receipt establishes +no new authorization. + +2026-10-17 refinement: T20 extracts the completed durable provenance source +candidate and disposable LLDAP compatibility proof from this task. The source +returns exact creation inventories, retains prestate/intents across process +restart and refuses detected shared-member/group-identity drift. Its publication +and projection are still pending the coordinator's separate integration receipt. +This narrows T14 without closing its original externally referenced native gate. +The task remains `todo` because preparation of the native contracts can proceed; +no missing owner assent is represented as received. + +Completion still requires independently reviewed/admitted ADR-0015-conforming +CNPG storage (or an explicit governed exception), backup/lost-store recovery and +migration of legacy requests; a native +writer-isolation/fencing or provider CAS/version contract that closes external +writer races and remove/re-add ABA; exact case/source/principal and current +native object bindings for protected callbacks; and actual per-attempt evidence, +revocation and readback/refusal verification through the existing owner paths. +Only independently retained confirmed creations may be undone. Every +pre-existing/shared group and membership must remain preserved; unresolved +intents never expire into retry authority. The disposable fixture supplies none +of these native custody, authorization, owner-acceptance or production claims. ## T15 — Define authenticated recovery for lost registration process state @@ -880,3 +904,74 @@ projection. T19's assigned UUID is retained. The original staged and working bri bytes were restored and independently compared. The exact source, projection and publication checks are in Custodian browser-resume-r1/canonical-integration.json. Native acceptance remains open under T16 and the other named owner tasks. + + +## T20 — Prepare durable group provenance and prove disposable LLDAP compatibility + +```task +id: NK-WP-0048-T20 +status: done +priority: high +state_hub_task_id: "863fcbdb-e246-51ba-87ca-a2f44d01d6f8" +``` + +Completed 2026-10-17, source candidate preparation and bounded synthetic +compatibility proof only. This extracts evidenced implementation/fixture scope +from T14; T14 retains native storage, writer isolation, protected callback and +owner-acceptance gates. Source integration/publication and State Hub projection +are pending the coordinator's separate receipts; no revision or task UUID is +invented here. + +The existing identity-provisioner group ensure/revoke interface now has a +reviewed candidate for a bounded owner-private SQLite journal, proven only in +local/disposable use. The initial storage instructions wrongly suggested a native +persistent-volume path; source review against accepted ADR-0015 corrected them +before publication. Production SQLite-on-a-PVC is not admitted: T14 retains +CNPG/rapp-postgres storage admission or a separately governed exception. +The local journal saves +complete prestate and intent before each attempted directory mutation, inventories +only confirmed new groups/memberships, binds original ensure idempotency and +correlation to the exact request digest, and refuses unsigned caller inventories +or legacy broad revoke requests. Replays never recreate removed state. Missing, +replaced or untrusted storage and missing intent rows stop execution; definite +before-mutation refusals retain evidence without stranding unrelated valid work. +Interrupted writes/readback remain unresolved across reopening. Revoke validates +all targeted native ids and complete member postimages before destructive work, +removes only confirmed creations and never deletes a preexisting empty group. +Server/configuration wiring, image-copy closure and owner storage/request +migration instructions are included. The existing endpoint's caller +authentication remains the existing boundary, not a new grant. + +The candidate was prepared against NetKingdom +`878abc93588e12b6da1bac055cfbb3abb8d7adf1`; exact source patch digest +`sha256:1f3f3ca94c46dadac8f24896053d4d23179de795e50a24298a0cfbcc46cca30e`. +The independent Custodian review passed all 65 provisioner checks. The earlier +62-check candidate, two independent-review findings and subsequent fixes remain +retained rather than overwritten. Seven additional fixture-plan checks exercise +the preparation boundary; these counts overlap the same candidate and do not +measure task-completion percentage or independent acceptance. + +A separately reviewed bounded fixture then exercised the exact candidate against +cached pinned disposable LLDAP and Python images, through actual GraphQL. All 14 +closed compatibility/containment oracles passed: exact creation inventory, +separate-process SQLite reopening/replay/revoke, complete preexisting directory +preservation including empty/shared groups and memberships, refusal of shared +membership drift, retained uncertainty after a lost response following a real +addition, and refusal of unknown/cross-case receipt bindings. The fixture used +only newly generated synthetic users/passwords/key material in private tmpfs, +with a network-none anchor and shared loopback helper, no host ports/mounts, +no image pulls and no production or actual credential contact. The coordinator's +safe receipt reports both exact owned containers absent after cleanup; final +independent cleanup verification remains the coordinator's integration duty. + +Evidence: Custodian +`docs/changes/netkingdom-target-2026-10-16/group-provenance-r1/`, including exact +source base/pre/postimages, patch/manifest, independent review and retained +refinements. Native fixture manifest +`sha256:8a0514f5f14137355972400cd4852f48c892316d28e3abd3c86635d7cb81eef2`; +retained safe result `native-fixture/execution-result.json` (original +`/tmp/nk48-groups-native-review-5pywb19z/result.json`). The source task supplies +no native owner assent, production rollout, protected session/evidence callbacks, +ordinary type admission or consumer login proof. LLDAP has no conditional +membership mutation or generation token here: exact readback cannot exclude +ABA or a concurrent external writer, so T14 remains open. diff --git a/workplans/NK-WP-0049-bootstrap-secret-free-recovery-handoff.md b/workplans/NK-WP-0049-bootstrap-secret-free-recovery-handoff.md index d0c196b..359415e 100644 --- a/workplans/NK-WP-0049-bootstrap-secret-free-recovery-handoff.md +++ b/workplans/NK-WP-0049-bootstrap-secret-free-recovery-handoff.md @@ -8,7 +8,7 @@ status: active owner: codex topic_slug: netkingdom created: "2026-10-12" -updated: "2026-10-16" +updated: "2026-10-17" flavor: implementation depends_on: [] state_hub_workstream_id: "17d315f0-829e-57c7-85cc-3a7914152804" @@ -28,8 +28,8 @@ These are retrospective task records of evidenced work, not additional delivery or newly granted authority. The earlier dated narrative remains in Git at `cc2dd01`; its evidence references are carried into the tasks below. -**Current position:** 22 tasks done; 2 in progress; 1 ready to start (`todo`); -6 waiting on explicit prerequisites. These are task counts, not an effort-based +**Current position, source refinement 2026-10-17:** 24 tasks done; 2 in +progress; 3 ready to start (`todo`); 7 waiting on explicit prerequisites. These are task counts, not an effort-based completion percentage. Assignees are also written in task descriptions because the current Forge projector does not populate the dedicated Hub assignee field. The outage-retrieval tool and restore-packet draft are now prepared; independent outage retrieval and Storage Share publication are accepted; @@ -958,27 +958,150 @@ retained, and scanner policy/exemptions are unchanged. ```task id: NK-WP-0049-T33 -status: todo +status: wait priority: high assignee: codex -depends_on: [NK-WP-0049-T32] +depends_on: [NK-WP-0049-T32, NK-WP-0049-T34, NK-WP-0049-T35, NK-WP-0049-T36] +blocking_reason: "Source custody/preflight preparation is evidenced in T34; await original TLS/trust recovery disposition T35 and selected original registration/private-admission metadata T36 before final reviewed command and existing owner process." state_hub_task_id: "56c8d469-7a10-5d50-a30e-95cbf3cd4756" ``` **Accountable worker:** NetKingdom codex with KeyCape and selected consumer owners. -Prepare the exact archive/manifest, original issuer/TLS/trust and executable -bindings, selected current registration and dependency graph using public metadata. -Use the existing independently held archive-key and owner-controlled TLS custody -paths; never put secret values in the packet. Validate the attended launcher and -its full dependency closure, execution deadline, recovery obligations and safe -receipt path before asking for any owner-held input. Bind approved source revision -and rollback/abort limits through the existing owner process. +This umbrella retains final attempt readiness. Completed custody/preflight source +preparation is extracted into T34; T35 resolves original TLS/trust recovery and +T36 prepares the selected original registration and private admission. This split +records distinct outcomes without counting source preparation as recovery. -**Done when:** reviewed executable command and metadata-only admission packet -are ready for the existing attended custody path; prerequisites and independent -receipt verification are explicit. T18 still owns actual execution, original -consumer continuity and acceptance. T19 retains retained-factor MFA. This task -neither authorizes a production consumer redirect nor creates a new client grant. +After those deliverables, assemble the exact existing six-argument attended +command and admission digest, bind the integrated source/dependency revision and +fixed local target, and record execution deadline, safe receipt location, abort, +rollback and independent receipt verification through the existing owner process. +Keep private manifest derivation inside custody and original TLS/client/key inputs +in the owner's own terminal; the current null fields must refuse execution. +Existing session authorization is retained within scope; this record does not +supply missing owner assent or approve any actual recovery attempt. + +**Done when:** reviewed executable command and complete metadata-only/private +admission packet are ready for the existing attended custody path; prerequisites +and independent receipt verification are explicit. T18 still owns actual +execution, original consumer continuity and acceptance. T19 retains retained-factor +MFA. This task neither authorizes a production consumer redirect nor creates a +new client grant. + +## Deliver attended custody, metadata preflight and private manifest preparation source + +```task +id: NK-WP-0049-T34 +status: done +priority: high +assignee: codex +depends_on: [NK-WP-0049-T32] +state_hub_task_id: "821b3a56-6f36-5812-abc9-5a00e37cb2e7" +``` + +**Accountable worker:** NetKingdom codex; independent Custodian source review. +Completed 2026-10-17 as source preparation. Existing attended custody entry points +now have a reviewed candidate sealed anonymous descriptor mode: hidden prompt, +actual public-recipient verification, strict owned/private/full-seal descriptor +admission, non-inheritable returned duplicate and explicit descriptor passing. +The existing file mode stays compatible, with its original hard-loss limitation; +no improved file-mode cleanup is claimed. Exact parent-loss tests prove descriptor +lifetime without signalling inherited user shell groups. An admitted child retains +its descriptor until it exits; parent loss is not child revocation. + +The candidate checks bounded, duplicate-free public JSON, exact admission digest, +transfer/archive and selected service bindings, typed hashes for all 17 invoked +local source/config dependencies and required Python imports before requesting a +key. TLS and ciphertext preparation uses file metadata only. Legacy explicit +manifest digests remain supported; versioned ciphertext-derived binding computes +the canonical digest privately only after exact reviewed ciphertext, authenticated +age decryption and full existing archive validation. The derived private digest +never enters safe receipts/checkpoints and grants no new consumer authority. + +The coordinator independently passed 111 focused checks. The existing custody +regression suite passed 94 checks; 61 real-SOPS tests skipped because that trusted +runtime is unavailable, so that compatibility coverage remains unverified. Tests +use generated keys/archive material only, including actual age cryptography, +synthetic hidden-prompt and exact parent-loss processes. Manifest-builder tests +mock Docker/issuer behavior and confer no new native proof. Frozen R6/R7 proof, +intermediate test failures and scanner history remain retained. + +Bounded public image readback separately establishes the cached original image's +13670142-byte `/keycape` executable as +`sha256:cd89121393e1465aa76009763602a5b8c75f9dddb5679aea6242623f135b51b4`. +This is executable identity metadata, not recovery or custody acceptance. + +Evidence: Custodian `docs/changes/netkingdom-target-2026-10-16/recovery-attempt-r1/` +(`candidate.patch`, `review.json`, source closure, attempt/admission metadata and +`public-verifier-image-readback.json`). Canonical integration/publication and +projection receipts must be recorded separately when performed; this completed +source-preparation record does not invent them. T33 remains open for complete +attempt readiness; actual recovery/consumer acceptance remains T18 and MFA T19. + +## Establish original KeyCape TLS and trust recovery ownership for outage use + +```task +id: NK-WP-0049-T35 +status: todo +priority: high +assignee: codex +depends_on: [NK-WP-0049-T34] +state_hub_task_id: "1bfd1703-589f-5511-bad5-812d860e4a71" +``` + +**Accountable worker:** NetKingdom codex, coordinating with the source owner of +Ingress/cert-manager/Traefik and the original TLS custody/provider boundary. +**Next action:** identify and record that actual owner's existing recovery path, +then obtain its metadata/disposition through the ordinary owner process. A missing +exact credential-routing catalog match is not evidence that no owner exists. + +Current source declares `kc.coulomb.social`, `sso/kc-tls`, cert-manager +`ClusterIssuer/letsencrypt-prod` and Traefik TLS termination before KeyCape 8080. +The identity backup profile excludes `kc-tls`; the verified archive therefore +establishes no independent recovery of that TLS identity/trust. Determine whether +the exact original certificate/key/trust is independently recoverable during an +issuer/cluster outage, or whether a documented reissue path is acceptable for the +selected continuity claim. Do not read or place the TLS key in work records, use +a fresh self-signed trust substitute, mutate production or claim owner assent. + +**Done when:** named owner/provider, independent outage prerequisites, original +public identity/certificate/trust hashes or accepted reissue disposition, private +custody entry point and verification/abort limits are explicit and reviewed. +Record any unavailable recovery capability as a live owner work record and bind +its dependency/unblock condition; a prose-only gap or untested archive is not +completion. This delivers preparation/disposition, not actual TLS recovery. + +## Prepare selected original service registration and private admission bindings + +```task +id: NK-WP-0049-T36 +status: todo +priority: high +assignee: codex +depends_on: [NK-WP-0049-T34] +state_hub_task_id: "8b72334b-fc27-5707-a189-bad345eaf3bc" +``` + +**Accountable worker:** NetKingdom codex with KeyCape and the selected existing +service consumer's owner. **Next action:** identify that owner and the exact +original registration's non-secret selector metadata using existing source/owner +evidence; preserve independent private custody and never retrieve a client secret. + +Bind reviewed ciphertext `7117b3fe1e4840c86b301c94c85e6b16709e5626bb0884bd0580bde630e30e30`, +its independently verified transfer, source/image/executable closure and an +explicit versioned manifest-binding mode. For the original service registration, +record client identifier, secret environment name, audience, granted scope, +subject, tenant, required roles and denied scope without values or new grants. +Describe the native service-client dependency graph and distinguish unexercised +LLDAP/Authelia/privacyIDEA/downstream flows. Keep private manifest material inside +the existing custody process, never in Git, safe receipts, State Hub or chat. + +**Done when:** the selected existing service owner and exact non-secret registration +bindings are independently established, the private admission builder validates +positive/negative source/archive/selector cases, and its metadata is ready to +combine with T35's original TLS/trust disposition under T33. This task may prepare +registration metadata while T35 proceeds independently; it does not finalize the +whole admission, perform actual recovery, redirect a consumer or confer assent. ## Accept remaining recovery and caller evidence and close the G08 handoff @@ -999,7 +1122,7 @@ Owner boundary: NetKingdom, Platform and affected service owners, with Custodian G08 receiving the final evidence. The completed archive/job milestones remain accepted independently of this final gate. -**Done when:** T02 and T15–T31 have their stated completed or explicitly accepted +**Done when:** T02 and T15–T36 have their stated completed or explicitly accepted handoff outcomes; service authentication/token/audit proof and current/legacy caller dispositions are accepted; the source workplan and projected tasks agree; and G08 has an exact evidence/handoff record. Record actionable residuals as live @@ -1035,3 +1158,11 @@ failures/automation errors; existing warnings and scanner false-positive history remain retained. Custodian identity-restore-adapter-next/canonical-r7-integration.json binds source/projection/native-fixture evidence separately. T18 remains waiting for the actual owner-held attempt and selected consumer acceptance. + + +2026-10-17 source/attempt refinement: T34 records completed attended custody, +preflight/private-manifest source preparation and its bounded tests; T35 and T36 +make original TLS recovery ownership and selected original admission metadata live +work. T33 retains final attempt readiness and its original Hub UUID. This adds no +actual recovery, MFA, native authority or owner assent. Source task counts above +refer to this file; projection has not been claimed for this proposed refinement.