From 5ef016be017885926061d3b6af6c0a24bc7ecceb Mon Sep 17 00:00:00 2001 From: codex Date: Sat, 5 Sep 2026 19:21:06 +0200 Subject: [PATCH] Contain backup upload credentials and prepare provider recovery gates Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a06ecb-456a-71c2-b41e-0755d336e883 --- docs/backup-credential-recovery.md | 67 +++++++++++++++++ docs/forgejo-backup.md | 3 + ...05-backup-provider-recovery-preparation.md | 24 +++++++ lib/railiance-backup-common.sh | 55 ++++++++++---- tests/test_backup_credentials.py | 72 +++++++++++++++++++ ...-0029-backup-credential-default-removal.md | 12 ++++ 6 files changed, 219 insertions(+), 14 deletions(-) create mode 100644 docs/backup-credential-recovery.md create mode 100644 history/2026-09-05-backup-provider-recovery-preparation.md diff --git a/docs/backup-credential-recovery.md b/docs/backup-credential-recovery.md new file mode 100644 index 0000000..3201779 --- /dev/null +++ b/docs/backup-credential-recovery.md @@ -0,0 +1,67 @@ +# WP-0029 provider recovery procedure + +Scope: invalidate the exposed Nextcloud upload predecessor and prove replacement +encrypted upload and offsite recovery under CCR-2026-0004. The route is a +Nextcloud file-drop share, not a platform-admin OpenBao credential. OpenBao +administration does not confer Nextcloud share-owner authority. + +## Required owner inputs + +- Nextcloud share owner/account and the reviewed share-management session. +- A replacement file-drop share/token delivered through protected custody; + never chat, argv, Git or State Hub. Record only custody coordinates. +- Owner-authorized retrieval access for the encrypted offsite artifact. Do not + assume an upload-only file-drop token can list or download backups. +- A protected reference to the predecessor for a bounded rejection probe, or + accepted provider-side revocation evidence. Do not reconstruct it into logs. + +## Execution order + +1. Confirm current OpenBao KV version and preserve every existing field. The + lane contains `AGE_PRIVATE_KEY`; replacing only upload fields must use CAS + and preserve recovery escrow. Never use a partial `kv put` that drops it. +2. Confirm the exact Nextcloud share and all backup consumers. Besides the + workstation helpers, activity-core consumes `NC_WEBDAV_TOKEN` and + `NC_WEBDAV_URL` through ExternalSecret `actcore-backup-offsite`. Coordinate + its next invocation so process-cached credentials cannot produce a false + replacement-success claim. +3. Create the replacement under provider authority. Verify its permissions + match the file-drop lane; no production delete/admin authority is needed. + Seed both upload fields through contained OpenBao CAS and verify delivery + with boolean comparisons. Keep provider secrets out of evidence. +4. Upload a uniquely named age-encrypted artifact with the replacement. Require + HTTP 200/201/204; redirects and network errors are not success. Record only + artifact identity and safe status. A synthetic fixture can prove transport, + but it does not establish Forgejo application recovery. +5. Invalidate the predecessor at the provider. Verify rejection with the old + credential in protected memory against the same bounded operation. A 401/403 + or accepted provider invalidation receipt is evidence; a timeout is not. + Preserve existing retained backup objects and access to their encryption key. +6. Retrieve the newly uploaded encrypted artifact through the separate owner + recovery path, decrypt in a protected temporary directory, and verify its + expected content. For Forgejo restore acceptance, use a fetched real backup + and the owning isolated restore procedure with repository/blob/database + checks. Require the local fetched artifact to exist before invoking the + historical infra drill: that script otherwise creates a fresh production + dump, which cannot prove offsite recovery. +7. Record provider invalidation, replacement KV version, upload/download, + decryption, isolated restore and cleanup receipts. Mark T02 done only after + the agreed recovery checks pass. No receipt may contain the predecessor's + value, fingerprint, length or shape. + +## Recovery-key exposure remains separate + +CCR-2026-0004 records historical exposure of the age recovery key as well as +upload credentials. Upload-share rotation alone must not clear that taint. +Retain old age recovery material until all retained artifacts are accounted for +and any owner-approved re-encryption is complete. Do not silently rotate or +discard the age key as part of this upload-token repair. + +## Prepared transport + +`lib/railiance-backup-common.sh` passes curl credentials and credential-bearing +URLs via stdin configuration, disables curlrc, suppresses backend diagnostics, +requires HTTPS and rejects non-success responses. The Python backend streams +the artifact, refuses redirects and suppresses credential-bearing exception +text. These source changes are preparation; no provider rotation, upload or +restore is claimed by their tests. diff --git a/docs/forgejo-backup.md b/docs/forgejo-backup.md index 1fe2af9..c35847b 100644 --- a/docs/forgejo-backup.md +++ b/docs/forgejo-backup.md @@ -32,6 +32,9 @@ There is no built-in credential fallback. Missing credentials stop execution before any cluster dump. Local encryption dry-runs skip upload authentication. Provider rotation and replacement upload/restore proof are tracked separately in `RPF-WP-0029`; removing the source default does not prove revocation. +The concrete owner procedure is `docs/backup-credential-recovery.md`. Upload +credentials and credential-bearing URLs are passed to curl through stdin, +and backend errors never print those values. Redirects are refused. Decrypt: `~/.config/age/railiance-backup.key` (same key as other Railiance backups). diff --git a/history/2026-09-05-backup-provider-recovery-preparation.md b/history/2026-09-05-backup-provider-recovery-preparation.md new file mode 100644 index 0000000..5c32e7c --- /dev/null +++ b/history/2026-09-05-backup-provider-recovery-preparation.md @@ -0,0 +1,24 @@ +# WP-0029 continuation: provider gate and transport containment + +The user authorized continuing with backup credential invalidation and recovery. +Routing and CCR-2026-0004 identify a Nextcloud file-drop lane plus age recovery +escrow. No provider share-management authority or replacement custody reference +was supplied. Requested only owner/account and custody coordinates, not values. + +Found and fixed credential transport defects in the retained backup helper: +curl argv previously contained the upload credential and credential-bearing URL; +the Python fallback could print them through an exception traceback. Curl now +receives escaped config on stdin with curlrc disabled, both paths require HTTPS, +redirects/non-success responses fail, and backend diagnostics are contained. +The Python fallback streams artifact bytes. Added tests using synthetic markers. + +Prepared `docs/backup-credential-recovery.md`: provider invalidation, CAS-safe +replacement preserving age escrow, activity-core delivery, encrypted upload, +separate owner download, and isolated offsite restore acceptance. The legacy +infra drill's missing-file fallback makes a fresh production dump; a downloaded +artifact must be proven present before it can qualify as offsite recovery. + +No provider change, credential read/provisioning, upload or restore was performed. +The attempted metadata-only taint query lacked caller authentication. Historical +CCR recovery-key exposure remains recorded; no current taint state or rotation +is inferred. T02 remains waiting on the provider inputs and live proof. diff --git a/lib/railiance-backup-common.sh b/lib/railiance-backup-common.sh index d63e76a..c4b627c 100644 --- a/lib/railiance-backup-common.sh +++ b/lib/railiance-backup-common.sh @@ -12,8 +12,8 @@ railiance_backup_load_openbao_lane() { fi command -v bao >/dev/null 2>&1 || return 0 bao kv metadata get "${RAILIANCE_BACKUP_BAO_PATH}" >/dev/null 2>&1 || return 0 - : "${RAILIANCE_BACKUP_NC_TOKEN:=$(bao kv get -field=NC_WEBDAV_TOKEN "${RAILIANCE_BACKUP_BAO_PATH}")}" - : "${RAILIANCE_BACKUP_NC_WEBDAV_URL:=$(bao kv get -field=NC_WEBDAV_URL "${RAILIANCE_BACKUP_BAO_PATH}")}" + : "${RAILIANCE_BACKUP_NC_TOKEN:=$(bao kv get -field=NC_WEBDAV_TOKEN "${RAILIANCE_BACKUP_BAO_PATH}" 2>/dev/null)}" + : "${RAILIANCE_BACKUP_NC_WEBDAV_URL:=$(bao kv get -field=NC_WEBDAV_URL "${RAILIANCE_BACKUP_BAO_PATH}" 2>/dev/null)}" } railiance_backup_require_openbao_lane() { @@ -51,27 +51,54 @@ railiance_backup_require_tools() { railiance_backup_nc_upload() { local file="$1" remote_name="$2" local dest="${RAILIANCE_BACKUP_NC_WEBDAV_URL%/}/${RAILIANCE_BACKUP_NC_PREFIX}/${remote_name}" + if [[ "$dest" != https://* || "$dest" == *$'\n'* || "$dest" == *$'\r'* || + "${RAILIANCE_BACKUP_NC_TOKEN}" == *$'\n'* || "${RAILIANCE_BACKUP_NC_TOKEN}" == *$'\r'* || + "$file" == *$'\n'* || "$file" == *$'\r'* ]]; then + echo "ERROR: invalid backup upload input" >&2 + return 1 + fi if command -v curl >/dev/null 2>&1; then - curl -sf -u "${RAILIANCE_BACKUP_NC_TOKEN}:" -T "$file" "$dest" \ - || { echo "ERROR: Nextcloud upload failed: ${remote_name}" >&2; return 1; } + # Credentials and credential-bearing URLs travel over stdin, never argv. + local config_url config_token config_file status + config_url="${dest//\\/\\\\}"; config_url="${config_url//\"/\\\"}" + config_token="${RAILIANCE_BACKUP_NC_TOKEN//\\/\\\\}"; config_token="${config_token//\"/\\\"}" + config_file="${file//\\/\\\\}"; config_file="${config_file//\"/\\\"}" + status="$(printf 'url = "%s"\nuser = "%s:"\nupload-file = "%s"\n' \ + "$config_url" "$config_token" "$config_file" | \ + curl --disable --silent --fail --proto '=https' --max-time 600 \ + --output /dev/null --write-out '%{http_code}' --config - 2>/dev/null)" \ + || { echo "ERROR: Nextcloud upload failed" >&2; return 1; } + case "$status" in + 200|201|204) ;; + *) echo "ERROR: Nextcloud upload failed" >&2; return 1 ;; + esac return 0 fi # Python fallback (worker image) RAILIANCE_BACKUP_NC_TOKEN="${RAILIANCE_BACKUP_NC_TOKEN}" \ RAILIANCE_BACKUP_UPLOAD_URL="${dest}" \ RAILIANCE_BACKUP_UPLOAD_FILE="${file}" \ - python3 - <<'PY' || { echo "ERROR: Nextcloud upload failed: ${remote_name}" >&2; return 1; } -import os, urllib.request, base64 + python3 - <<'PY' || { echo "ERROR: Nextcloud upload failed" >&2; return 1; } +import os, urllib.request, urllib.error, base64, sys url = os.environ["RAILIANCE_BACKUP_UPLOAD_URL"] path = os.environ["RAILIANCE_BACKUP_UPLOAD_FILE"] token = os.environ["RAILIANCE_BACKUP_NC_TOKEN"] -data = open(path, "rb").read() -req = urllib.request.Request(url, data=data, method="PUT") -req.add_header("Authorization", "Basic " + base64.b64encode(f"{token}:".encode()).decode()) -req.add_header("Content-Type", "application/octet-stream") -with urllib.request.urlopen(req, timeout=600) as resp: - if resp.status not in (200, 201, 204): - raise SystemExit(f"HTTP {resp.status}") +class NoRedirect(urllib.request.HTTPRedirectHandler): + def redirect_request(self, req, fp, code, msg, headers, newurl): + raise urllib.error.HTTPError(req.full_url, code, 'redirect refused', headers, fp) +try: + with open(path, 'rb') as source: + chunks = iter(lambda: source.read(1024 * 1024), b'') + req = urllib.request.Request(url, data=chunks, method="PUT") + req.add_header("Authorization", "Basic " + base64.b64encode(f"{token}:".encode()).decode()) + req.add_header("Content-Type", "application/octet-stream") + req.add_header("Content-Length", str(os.fstat(source.fileno()).st_size)) + with urllib.request.build_opener(NoRedirect()).open(req, timeout=600) as resp: + if resp.status not in (200, 201, 204): + sys.exit(1) +except Exception: + # URLs may themselves contain credentials: never emit exception text. + sys.exit(1) PY } @@ -91,4 +118,4 @@ railiance_backup_record_success() { mkdir -p "$stamp_dir" echo "$ts" > "${stamp_dir}/.last-success" echo "$ts" >> "${stamp_dir}/success-log" -} \ No newline at end of file +} diff --git a/tests/test_backup_credentials.py b/tests/test_backup_credentials.py index 9baa767..8f715db 100644 --- a/tests/test_backup_credentials.py +++ b/tests/test_backup_credentials.py @@ -33,6 +33,78 @@ railiance_backup_require_openbao_lane self.assertEqual(result.returncode, 0) self.assertEqual(result.stdout + result.stderr, "") + def test_curl_receives_credentials_only_on_stdin(self): + result = self.run_shell(''' +RAILIANCE_BACKUP_NC_TOKEN=fixture-secret-marker +RAILIANCE_BACKUP_NC_WEBDAV_URL=https://example.invalid/fixture-secret-marker +curl() { + [[ "$*" != *fixture-secret-marker* ]] || return 9 + local config + config="$(cat)" + [[ "$config" == *'user = "fixture-secret-marker:"'* ]] || return 8 + [[ "$config" == *'url = "https://example.invalid/fixture-secret-marker/forgejo/test.age"'* ]] || return 7 + printf 201 +} +railiance_backup_nc_upload /tmp/test.age test.age +''') + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(result.stdout + result.stderr, '') + + def test_redirect_and_backend_diagnostics_are_not_success_or_output(self): + for status in ['302', '401', '403', '500']: + result = self.run_shell(''' +RAILIANCE_BACKUP_NC_TOKEN=fixture-secret-marker +RAILIANCE_BACKUP_NC_WEBDAV_URL=https://example.invalid/fixture-secret-marker +curl() { cat >/dev/null; echo fixture-secret-marker >&2; printf '%s'; } +railiance_backup_nc_upload /tmp/test.age test.age +''' % status) + self.assertNotEqual(result.returncode, 0) + self.assertEqual(result.stdout, '') + self.assertEqual(result.stderr, 'ERROR: Nextcloud upload failed\n') + + def test_config_line_injection_fails_before_curl(self): + result = self.run_shell(''' +RAILIANCE_BACKUP_NC_TOKEN=$'fixture\\noutput=/tmp/unwanted' +RAILIANCE_BACKUP_NC_WEBDAV_URL=https://example.invalid/upload +curl() { echo backend-was-called; return 1; } +railiance_backup_nc_upload /tmp/test.age test.age +''') + self.assertNotEqual(result.returncode, 0) + self.assertEqual(result.stdout, '') + self.assertEqual(result.stderr, 'ERROR: invalid backup upload input\n') + + def test_plain_http_is_rejected_before_backend(self): + result = self.run_shell(''' +RAILIANCE_BACKUP_NC_TOKEN=fixture +RAILIANCE_BACKUP_NC_WEBDAV_URL=http://example.invalid/upload +curl() { echo backend-was-called; return 1; } +railiance_backup_nc_upload /tmp/test.age test.age +''') + self.assertNotEqual(result.returncode, 0) + self.assertEqual(result.stdout, '') + + def test_python_backend_exception_cannot_disclose_credential_url(self): + result = self.run_shell(''' +RAILIANCE_BACKUP_NC_TOKEN=fixture-secret-marker +RAILIANCE_BACKUP_NC_WEBDAV_URL=https://example.invalid/fixture-secret-marker +command() { + if [[ "$*" == '-v curl' ]]; then return 1; fi + builtin command "$@" +} +python3() { + /usr/bin/python3 -c 'import sys,urllib.request +class FailedOpener: + def open(self, req, **kwargs): + raise RuntimeError(req.full_url) +urllib.request.build_opener=lambda *args: FailedOpener() +exec(sys.stdin.read())' +} +railiance_backup_nc_upload /dev/null test.age +''') + self.assertNotEqual(result.returncode, 0) + self.assertEqual(result.stdout, '') + self.assertEqual(result.stderr, 'ERROR: Nextcloud upload failed\n') + def test_openbao_lane_supplies_missing_credentials(self): result = self.run_shell(''' bao() { diff --git a/workplans/RPF-WP-0029-backup-credential-default-removal.md b/workplans/RPF-WP-0029-backup-credential-default-removal.md index e977055..e546dbf 100644 --- a/workplans/RPF-WP-0029-backup-credential-default-removal.md +++ b/workplans/RPF-WP-0029-backup-credential-default-removal.md @@ -27,6 +27,11 @@ state_hub_task_id: "4b5aefdb-a746-54f9-ba29-ebb840e7848d" Removed the literal upload credential default from tools/cmd/forgejo-backup. Missing governed input now fails before cluster operations with a value-free diagnostic; encryption dry-runs skip upload authentication. Offline tests prove missing-input denial, explicit input, and mocked OpenBao resolution. +2026-09-05 continuation: removed credentials and credential-bearing URLs from +curl argv; curl config travels through stdin with curlrc disabled. Both upload +backends suppress credential-bearing diagnostics, require HTTPS and reject +redirects/non-success status. Added transport containment and failure tests. + ## Invalidate predecessor and prove replacement recovery ```task @@ -38,6 +43,13 @@ state_hub_task_id: "b3f3402f-890b-5781-9b3e-1c9c0d28cea8" Provider-side invalidation and replacement custody need the attended provider owner and CCR-2026-0004 lifecycle procedure. Record only non-secret invalidation, encrypted upload and restore receipts. No provider authority or replacement receipt was available; source removal alone does not close the reported exposure. Never record the predecessor value, fingerprint, length, or shape. +Prepared owner execution procedure: `docs/backup-credential-recovery.md`. +Awaiting the Nextcloud share owner/account and replacement custody coordinates; +OpenBao authority is not provider share-management authority. Activity-core is +also a consumer of this upload lane. Preserve AGE_PRIVATE_KEY and historical +exposure evidence; upload-token rotation cannot clear recovery-key taint. +No provider mutation, backup upload or restore has been run in this continuation. + ## Portfolio review — 2026-09-05 INTENT binding: dependable backup custody and recovery. Retain T02 until the