Contain backup upload credentials and prepare provider recovery gates
Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a06ecb-456a-71c2-b41e-0755d336e883
This commit is contained in:
parent
08a406f2f8
commit
5ef016be01
6 changed files with 219 additions and 14 deletions
67
docs/backup-credential-recovery.md
Normal file
67
docs/backup-credential-recovery.md
Normal file
|
|
@ -0,0 +1,67 @@
|
||||||
|
# WP-0029 provider recovery procedure
|
||||||
|
|
||||||
|
Scope: invalidate the exposed Nextcloud upload predecessor and prove replacement
|
||||||
|
encrypted upload and offsite recovery under CCR-2026-0004. The route is a
|
||||||
|
Nextcloud file-drop share, not a platform-admin OpenBao credential. OpenBao
|
||||||
|
administration does not confer Nextcloud share-owner authority.
|
||||||
|
|
||||||
|
## Required owner inputs
|
||||||
|
|
||||||
|
- Nextcloud share owner/account and the reviewed share-management session.
|
||||||
|
- A replacement file-drop share/token delivered through protected custody;
|
||||||
|
never chat, argv, Git or State Hub. Record only custody coordinates.
|
||||||
|
- Owner-authorized retrieval access for the encrypted offsite artifact. Do not
|
||||||
|
assume an upload-only file-drop token can list or download backups.
|
||||||
|
- A protected reference to the predecessor for a bounded rejection probe, or
|
||||||
|
accepted provider-side revocation evidence. Do not reconstruct it into logs.
|
||||||
|
|
||||||
|
## Execution order
|
||||||
|
|
||||||
|
1. Confirm current OpenBao KV version and preserve every existing field. The
|
||||||
|
lane contains `AGE_PRIVATE_KEY`; replacing only upload fields must use CAS
|
||||||
|
and preserve recovery escrow. Never use a partial `kv put` that drops it.
|
||||||
|
2. Confirm the exact Nextcloud share and all backup consumers. Besides the
|
||||||
|
workstation helpers, activity-core consumes `NC_WEBDAV_TOKEN` and
|
||||||
|
`NC_WEBDAV_URL` through ExternalSecret `actcore-backup-offsite`. Coordinate
|
||||||
|
its next invocation so process-cached credentials cannot produce a false
|
||||||
|
replacement-success claim.
|
||||||
|
3. Create the replacement under provider authority. Verify its permissions
|
||||||
|
match the file-drop lane; no production delete/admin authority is needed.
|
||||||
|
Seed both upload fields through contained OpenBao CAS and verify delivery
|
||||||
|
with boolean comparisons. Keep provider secrets out of evidence.
|
||||||
|
4. Upload a uniquely named age-encrypted artifact with the replacement. Require
|
||||||
|
HTTP 200/201/204; redirects and network errors are not success. Record only
|
||||||
|
artifact identity and safe status. A synthetic fixture can prove transport,
|
||||||
|
but it does not establish Forgejo application recovery.
|
||||||
|
5. Invalidate the predecessor at the provider. Verify rejection with the old
|
||||||
|
credential in protected memory against the same bounded operation. A 401/403
|
||||||
|
or accepted provider invalidation receipt is evidence; a timeout is not.
|
||||||
|
Preserve existing retained backup objects and access to their encryption key.
|
||||||
|
6. Retrieve the newly uploaded encrypted artifact through the separate owner
|
||||||
|
recovery path, decrypt in a protected temporary directory, and verify its
|
||||||
|
expected content. For Forgejo restore acceptance, use a fetched real backup
|
||||||
|
and the owning isolated restore procedure with repository/blob/database
|
||||||
|
checks. Require the local fetched artifact to exist before invoking the
|
||||||
|
historical infra drill: that script otherwise creates a fresh production
|
||||||
|
dump, which cannot prove offsite recovery.
|
||||||
|
7. Record provider invalidation, replacement KV version, upload/download,
|
||||||
|
decryption, isolated restore and cleanup receipts. Mark T02 done only after
|
||||||
|
the agreed recovery checks pass. No receipt may contain the predecessor's
|
||||||
|
value, fingerprint, length or shape.
|
||||||
|
|
||||||
|
## Recovery-key exposure remains separate
|
||||||
|
|
||||||
|
CCR-2026-0004 records historical exposure of the age recovery key as well as
|
||||||
|
upload credentials. Upload-share rotation alone must not clear that taint.
|
||||||
|
Retain old age recovery material until all retained artifacts are accounted for
|
||||||
|
and any owner-approved re-encryption is complete. Do not silently rotate or
|
||||||
|
discard the age key as part of this upload-token repair.
|
||||||
|
|
||||||
|
## Prepared transport
|
||||||
|
|
||||||
|
`lib/railiance-backup-common.sh` passes curl credentials and credential-bearing
|
||||||
|
URLs via stdin configuration, disables curlrc, suppresses backend diagnostics,
|
||||||
|
requires HTTPS and rejects non-success responses. The Python backend streams
|
||||||
|
the artifact, refuses redirects and suppresses credential-bearing exception
|
||||||
|
text. These source changes are preparation; no provider rotation, upload or
|
||||||
|
restore is claimed by their tests.
|
||||||
|
|
@ -32,6 +32,9 @@ There is no built-in credential fallback. Missing credentials stop execution
|
||||||
before any cluster dump. Local encryption dry-runs skip upload authentication.
|
before any cluster dump. Local encryption dry-runs skip upload authentication.
|
||||||
Provider rotation and replacement upload/restore proof are tracked separately
|
Provider rotation and replacement upload/restore proof are tracked separately
|
||||||
in `RPF-WP-0029`; removing the source default does not prove revocation.
|
in `RPF-WP-0029`; removing the source default does not prove revocation.
|
||||||
|
The concrete owner procedure is `docs/backup-credential-recovery.md`. Upload
|
||||||
|
credentials and credential-bearing URLs are passed to curl through stdin,
|
||||||
|
and backend errors never print those values. Redirects are refused.
|
||||||
|
|
||||||
Decrypt: `~/.config/age/railiance-backup.key` (same key as other Railiance backups).
|
Decrypt: `~/.config/age/railiance-backup.key` (same key as other Railiance backups).
|
||||||
|
|
||||||
|
|
|
||||||
24
history/2026-09-05-backup-provider-recovery-preparation.md
Normal file
24
history/2026-09-05-backup-provider-recovery-preparation.md
Normal file
|
|
@ -0,0 +1,24 @@
|
||||||
|
# WP-0029 continuation: provider gate and transport containment
|
||||||
|
|
||||||
|
The user authorized continuing with backup credential invalidation and recovery.
|
||||||
|
Routing and CCR-2026-0004 identify a Nextcloud file-drop lane plus age recovery
|
||||||
|
escrow. No provider share-management authority or replacement custody reference
|
||||||
|
was supplied. Requested only owner/account and custody coordinates, not values.
|
||||||
|
|
||||||
|
Found and fixed credential transport defects in the retained backup helper:
|
||||||
|
curl argv previously contained the upload credential and credential-bearing URL;
|
||||||
|
the Python fallback could print them through an exception traceback. Curl now
|
||||||
|
receives escaped config on stdin with curlrc disabled, both paths require HTTPS,
|
||||||
|
redirects/non-success responses fail, and backend diagnostics are contained.
|
||||||
|
The Python fallback streams artifact bytes. Added tests using synthetic markers.
|
||||||
|
|
||||||
|
Prepared `docs/backup-credential-recovery.md`: provider invalidation, CAS-safe
|
||||||
|
replacement preserving age escrow, activity-core delivery, encrypted upload,
|
||||||
|
separate owner download, and isolated offsite restore acceptance. The legacy
|
||||||
|
infra drill's missing-file fallback makes a fresh production dump; a downloaded
|
||||||
|
artifact must be proven present before it can qualify as offsite recovery.
|
||||||
|
|
||||||
|
No provider change, credential read/provisioning, upload or restore was performed.
|
||||||
|
The attempted metadata-only taint query lacked caller authentication. Historical
|
||||||
|
CCR recovery-key exposure remains recorded; no current taint state or rotation
|
||||||
|
is inferred. T02 remains waiting on the provider inputs and live proof.
|
||||||
|
|
@ -12,8 +12,8 @@ railiance_backup_load_openbao_lane() {
|
||||||
fi
|
fi
|
||||||
command -v bao >/dev/null 2>&1 || return 0
|
command -v bao >/dev/null 2>&1 || return 0
|
||||||
bao kv metadata get "${RAILIANCE_BACKUP_BAO_PATH}" >/dev/null 2>&1 || return 0
|
bao kv metadata get "${RAILIANCE_BACKUP_BAO_PATH}" >/dev/null 2>&1 || return 0
|
||||||
: "${RAILIANCE_BACKUP_NC_TOKEN:=$(bao kv get -field=NC_WEBDAV_TOKEN "${RAILIANCE_BACKUP_BAO_PATH}")}"
|
: "${RAILIANCE_BACKUP_NC_TOKEN:=$(bao kv get -field=NC_WEBDAV_TOKEN "${RAILIANCE_BACKUP_BAO_PATH}" 2>/dev/null)}"
|
||||||
: "${RAILIANCE_BACKUP_NC_WEBDAV_URL:=$(bao kv get -field=NC_WEBDAV_URL "${RAILIANCE_BACKUP_BAO_PATH}")}"
|
: "${RAILIANCE_BACKUP_NC_WEBDAV_URL:=$(bao kv get -field=NC_WEBDAV_URL "${RAILIANCE_BACKUP_BAO_PATH}" 2>/dev/null)}"
|
||||||
}
|
}
|
||||||
|
|
||||||
railiance_backup_require_openbao_lane() {
|
railiance_backup_require_openbao_lane() {
|
||||||
|
|
@ -51,27 +51,54 @@ railiance_backup_require_tools() {
|
||||||
railiance_backup_nc_upload() {
|
railiance_backup_nc_upload() {
|
||||||
local file="$1" remote_name="$2"
|
local file="$1" remote_name="$2"
|
||||||
local dest="${RAILIANCE_BACKUP_NC_WEBDAV_URL%/}/${RAILIANCE_BACKUP_NC_PREFIX}/${remote_name}"
|
local dest="${RAILIANCE_BACKUP_NC_WEBDAV_URL%/}/${RAILIANCE_BACKUP_NC_PREFIX}/${remote_name}"
|
||||||
|
if [[ "$dest" != https://* || "$dest" == *$'\n'* || "$dest" == *$'\r'* ||
|
||||||
|
"${RAILIANCE_BACKUP_NC_TOKEN}" == *$'\n'* || "${RAILIANCE_BACKUP_NC_TOKEN}" == *$'\r'* ||
|
||||||
|
"$file" == *$'\n'* || "$file" == *$'\r'* ]]; then
|
||||||
|
echo "ERROR: invalid backup upload input" >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
if command -v curl >/dev/null 2>&1; then
|
if command -v curl >/dev/null 2>&1; then
|
||||||
curl -sf -u "${RAILIANCE_BACKUP_NC_TOKEN}:" -T "$file" "$dest" \
|
# Credentials and credential-bearing URLs travel over stdin, never argv.
|
||||||
|| { echo "ERROR: Nextcloud upload failed: ${remote_name}" >&2; return 1; }
|
local config_url config_token config_file status
|
||||||
|
config_url="${dest//\\/\\\\}"; config_url="${config_url//\"/\\\"}"
|
||||||
|
config_token="${RAILIANCE_BACKUP_NC_TOKEN//\\/\\\\}"; config_token="${config_token//\"/\\\"}"
|
||||||
|
config_file="${file//\\/\\\\}"; config_file="${config_file//\"/\\\"}"
|
||||||
|
status="$(printf 'url = "%s"\nuser = "%s:"\nupload-file = "%s"\n' \
|
||||||
|
"$config_url" "$config_token" "$config_file" | \
|
||||||
|
curl --disable --silent --fail --proto '=https' --max-time 600 \
|
||||||
|
--output /dev/null --write-out '%{http_code}' --config - 2>/dev/null)" \
|
||||||
|
|| { echo "ERROR: Nextcloud upload failed" >&2; return 1; }
|
||||||
|
case "$status" in
|
||||||
|
200|201|204) ;;
|
||||||
|
*) echo "ERROR: Nextcloud upload failed" >&2; return 1 ;;
|
||||||
|
esac
|
||||||
return 0
|
return 0
|
||||||
fi
|
fi
|
||||||
# Python fallback (worker image)
|
# Python fallback (worker image)
|
||||||
RAILIANCE_BACKUP_NC_TOKEN="${RAILIANCE_BACKUP_NC_TOKEN}" \
|
RAILIANCE_BACKUP_NC_TOKEN="${RAILIANCE_BACKUP_NC_TOKEN}" \
|
||||||
RAILIANCE_BACKUP_UPLOAD_URL="${dest}" \
|
RAILIANCE_BACKUP_UPLOAD_URL="${dest}" \
|
||||||
RAILIANCE_BACKUP_UPLOAD_FILE="${file}" \
|
RAILIANCE_BACKUP_UPLOAD_FILE="${file}" \
|
||||||
python3 - <<'PY' || { echo "ERROR: Nextcloud upload failed: ${remote_name}" >&2; return 1; }
|
python3 - <<'PY' || { echo "ERROR: Nextcloud upload failed" >&2; return 1; }
|
||||||
import os, urllib.request, base64
|
import os, urllib.request, urllib.error, base64, sys
|
||||||
url = os.environ["RAILIANCE_BACKUP_UPLOAD_URL"]
|
url = os.environ["RAILIANCE_BACKUP_UPLOAD_URL"]
|
||||||
path = os.environ["RAILIANCE_BACKUP_UPLOAD_FILE"]
|
path = os.environ["RAILIANCE_BACKUP_UPLOAD_FILE"]
|
||||||
token = os.environ["RAILIANCE_BACKUP_NC_TOKEN"]
|
token = os.environ["RAILIANCE_BACKUP_NC_TOKEN"]
|
||||||
data = open(path, "rb").read()
|
class NoRedirect(urllib.request.HTTPRedirectHandler):
|
||||||
req = urllib.request.Request(url, data=data, method="PUT")
|
def redirect_request(self, req, fp, code, msg, headers, newurl):
|
||||||
req.add_header("Authorization", "Basic " + base64.b64encode(f"{token}:".encode()).decode())
|
raise urllib.error.HTTPError(req.full_url, code, 'redirect refused', headers, fp)
|
||||||
req.add_header("Content-Type", "application/octet-stream")
|
try:
|
||||||
with urllib.request.urlopen(req, timeout=600) as resp:
|
with open(path, 'rb') as source:
|
||||||
if resp.status not in (200, 201, 204):
|
chunks = iter(lambda: source.read(1024 * 1024), b'')
|
||||||
raise SystemExit(f"HTTP {resp.status}")
|
req = urllib.request.Request(url, data=chunks, method="PUT")
|
||||||
|
req.add_header("Authorization", "Basic " + base64.b64encode(f"{token}:".encode()).decode())
|
||||||
|
req.add_header("Content-Type", "application/octet-stream")
|
||||||
|
req.add_header("Content-Length", str(os.fstat(source.fileno()).st_size))
|
||||||
|
with urllib.request.build_opener(NoRedirect()).open(req, timeout=600) as resp:
|
||||||
|
if resp.status not in (200, 201, 204):
|
||||||
|
sys.exit(1)
|
||||||
|
except Exception:
|
||||||
|
# URLs may themselves contain credentials: never emit exception text.
|
||||||
|
sys.exit(1)
|
||||||
PY
|
PY
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -91,4 +118,4 @@ railiance_backup_record_success() {
|
||||||
mkdir -p "$stamp_dir"
|
mkdir -p "$stamp_dir"
|
||||||
echo "$ts" > "${stamp_dir}/.last-success"
|
echo "$ts" > "${stamp_dir}/.last-success"
|
||||||
echo "$ts" >> "${stamp_dir}/success-log"
|
echo "$ts" >> "${stamp_dir}/success-log"
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -33,6 +33,78 @@ railiance_backup_require_openbao_lane
|
||||||
self.assertEqual(result.returncode, 0)
|
self.assertEqual(result.returncode, 0)
|
||||||
self.assertEqual(result.stdout + result.stderr, "")
|
self.assertEqual(result.stdout + result.stderr, "")
|
||||||
|
|
||||||
|
def test_curl_receives_credentials_only_on_stdin(self):
|
||||||
|
result = self.run_shell('''
|
||||||
|
RAILIANCE_BACKUP_NC_TOKEN=fixture-secret-marker
|
||||||
|
RAILIANCE_BACKUP_NC_WEBDAV_URL=https://example.invalid/fixture-secret-marker
|
||||||
|
curl() {
|
||||||
|
[[ "$*" != *fixture-secret-marker* ]] || return 9
|
||||||
|
local config
|
||||||
|
config="$(cat)"
|
||||||
|
[[ "$config" == *'user = "fixture-secret-marker:"'* ]] || return 8
|
||||||
|
[[ "$config" == *'url = "https://example.invalid/fixture-secret-marker/forgejo/test.age"'* ]] || return 7
|
||||||
|
printf 201
|
||||||
|
}
|
||||||
|
railiance_backup_nc_upload /tmp/test.age test.age
|
||||||
|
''')
|
||||||
|
self.assertEqual(result.returncode, 0, result.stderr)
|
||||||
|
self.assertEqual(result.stdout + result.stderr, '')
|
||||||
|
|
||||||
|
def test_redirect_and_backend_diagnostics_are_not_success_or_output(self):
|
||||||
|
for status in ['302', '401', '403', '500']:
|
||||||
|
result = self.run_shell('''
|
||||||
|
RAILIANCE_BACKUP_NC_TOKEN=fixture-secret-marker
|
||||||
|
RAILIANCE_BACKUP_NC_WEBDAV_URL=https://example.invalid/fixture-secret-marker
|
||||||
|
curl() { cat >/dev/null; echo fixture-secret-marker >&2; printf '%s'; }
|
||||||
|
railiance_backup_nc_upload /tmp/test.age test.age
|
||||||
|
''' % status)
|
||||||
|
self.assertNotEqual(result.returncode, 0)
|
||||||
|
self.assertEqual(result.stdout, '')
|
||||||
|
self.assertEqual(result.stderr, 'ERROR: Nextcloud upload failed\n')
|
||||||
|
|
||||||
|
def test_config_line_injection_fails_before_curl(self):
|
||||||
|
result = self.run_shell('''
|
||||||
|
RAILIANCE_BACKUP_NC_TOKEN=$'fixture\\noutput=/tmp/unwanted'
|
||||||
|
RAILIANCE_BACKUP_NC_WEBDAV_URL=https://example.invalid/upload
|
||||||
|
curl() { echo backend-was-called; return 1; }
|
||||||
|
railiance_backup_nc_upload /tmp/test.age test.age
|
||||||
|
''')
|
||||||
|
self.assertNotEqual(result.returncode, 0)
|
||||||
|
self.assertEqual(result.stdout, '')
|
||||||
|
self.assertEqual(result.stderr, 'ERROR: invalid backup upload input\n')
|
||||||
|
|
||||||
|
def test_plain_http_is_rejected_before_backend(self):
|
||||||
|
result = self.run_shell('''
|
||||||
|
RAILIANCE_BACKUP_NC_TOKEN=fixture
|
||||||
|
RAILIANCE_BACKUP_NC_WEBDAV_URL=http://example.invalid/upload
|
||||||
|
curl() { echo backend-was-called; return 1; }
|
||||||
|
railiance_backup_nc_upload /tmp/test.age test.age
|
||||||
|
''')
|
||||||
|
self.assertNotEqual(result.returncode, 0)
|
||||||
|
self.assertEqual(result.stdout, '')
|
||||||
|
|
||||||
|
def test_python_backend_exception_cannot_disclose_credential_url(self):
|
||||||
|
result = self.run_shell('''
|
||||||
|
RAILIANCE_BACKUP_NC_TOKEN=fixture-secret-marker
|
||||||
|
RAILIANCE_BACKUP_NC_WEBDAV_URL=https://example.invalid/fixture-secret-marker
|
||||||
|
command() {
|
||||||
|
if [[ "$*" == '-v curl' ]]; then return 1; fi
|
||||||
|
builtin command "$@"
|
||||||
|
}
|
||||||
|
python3() {
|
||||||
|
/usr/bin/python3 -c 'import sys,urllib.request
|
||||||
|
class FailedOpener:
|
||||||
|
def open(self, req, **kwargs):
|
||||||
|
raise RuntimeError(req.full_url)
|
||||||
|
urllib.request.build_opener=lambda *args: FailedOpener()
|
||||||
|
exec(sys.stdin.read())'
|
||||||
|
}
|
||||||
|
railiance_backup_nc_upload /dev/null test.age
|
||||||
|
''')
|
||||||
|
self.assertNotEqual(result.returncode, 0)
|
||||||
|
self.assertEqual(result.stdout, '')
|
||||||
|
self.assertEqual(result.stderr, 'ERROR: Nextcloud upload failed\n')
|
||||||
|
|
||||||
def test_openbao_lane_supplies_missing_credentials(self):
|
def test_openbao_lane_supplies_missing_credentials(self):
|
||||||
result = self.run_shell('''
|
result = self.run_shell('''
|
||||||
bao() {
|
bao() {
|
||||||
|
|
|
||||||
|
|
@ -27,6 +27,11 @@ state_hub_task_id: "4b5aefdb-a746-54f9-ba29-ebb840e7848d"
|
||||||
|
|
||||||
Removed the literal upload credential default from tools/cmd/forgejo-backup. Missing governed input now fails before cluster operations with a value-free diagnostic; encryption dry-runs skip upload authentication. Offline tests prove missing-input denial, explicit input, and mocked OpenBao resolution.
|
Removed the literal upload credential default from tools/cmd/forgejo-backup. Missing governed input now fails before cluster operations with a value-free diagnostic; encryption dry-runs skip upload authentication. Offline tests prove missing-input denial, explicit input, and mocked OpenBao resolution.
|
||||||
|
|
||||||
|
2026-09-05 continuation: removed credentials and credential-bearing URLs from
|
||||||
|
curl argv; curl config travels through stdin with curlrc disabled. Both upload
|
||||||
|
backends suppress credential-bearing diagnostics, require HTTPS and reject
|
||||||
|
redirects/non-success status. Added transport containment and failure tests.
|
||||||
|
|
||||||
## Invalidate predecessor and prove replacement recovery
|
## Invalidate predecessor and prove replacement recovery
|
||||||
|
|
||||||
```task
|
```task
|
||||||
|
|
@ -38,6 +43,13 @@ state_hub_task_id: "b3f3402f-890b-5781-9b3e-1c9c0d28cea8"
|
||||||
|
|
||||||
Provider-side invalidation and replacement custody need the attended provider owner and CCR-2026-0004 lifecycle procedure. Record only non-secret invalidation, encrypted upload and restore receipts. No provider authority or replacement receipt was available; source removal alone does not close the reported exposure. Never record the predecessor value, fingerprint, length, or shape.
|
Provider-side invalidation and replacement custody need the attended provider owner and CCR-2026-0004 lifecycle procedure. Record only non-secret invalidation, encrypted upload and restore receipts. No provider authority or replacement receipt was available; source removal alone does not close the reported exposure. Never record the predecessor value, fingerprint, length, or shape.
|
||||||
|
|
||||||
|
Prepared owner execution procedure: `docs/backup-credential-recovery.md`.
|
||||||
|
Awaiting the Nextcloud share owner/account and replacement custody coordinates;
|
||||||
|
OpenBao authority is not provider share-management authority. Activity-core is
|
||||||
|
also a consumer of this upload lane. Preserve AGE_PRIVATE_KEY and historical
|
||||||
|
exposure evidence; upload-token rotation cannot clear recovery-key taint.
|
||||||
|
No provider mutation, backup upload or restore has been run in this continuation.
|
||||||
|
|
||||||
## Portfolio review — 2026-09-05
|
## Portfolio review — 2026-09-05
|
||||||
|
|
||||||
INTENT binding: dependable backup custody and recovery. Retain T02 until the
|
INTENT binding: dependable backup custody and recovery. Retain T02 until the
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue