Prepare Forgejo-image-tested portable scanner rollout revision 4
This commit is contained in:
codex
parent
719a7f4164
commit
a4cfd2cecf
22 changed files with 20328 additions and 18 deletions
|
|
@ -27,3 +27,9 @@ The first diagnostic was returned: node/deployment/pod identity, readiness, old
|
|||
The founder's condition report found exactly one missing scanner fragment among171 repositories, at `coulomb/markitect-main.git`. This explains the safe revision2 refusal. [Revision3](native-r3/PLAN.md), packet SHA256 `df1cb48259ec5411a3f8cd24d436078daf09d241c39448454526f0a796798c34`, restores only that old-policy fragment, verifies its dispatcher and all171 active hook sets, preserves the failed receipt, then reuses the sealed unchanged JSON rollout and six source changes. Seven controller and six real filesystem cases passed; the original scanner accepts this preparation and all source preimages remain unchanged. Await exact founder review/execution via `/tmp/forge-scanner-recover-and-rollout.sh`. This new repair is not covered by an inferred retry permission; it is concretely included in the reviewed revision3 action. Repair protection remains installed on later JSON rollback. Existing-history admission and the cause of missing adoption are not established by this repair.
|
||||
|
||||
Founder approval received for exact revision3 packet `df1cb48259ec5411a3f8cd24d436078daf09d241c39448454526f0a796798c34` (User: Approve exact revision 3; request_user_input_async call_d07a070d30a54a538c872888bd5b0e30 item0, 2026-11-02). Attended execution is pending via `bash /tmp/forge-scanner-recover-and-rollout.sh`; the approval receipt is `native-r3-approval.json`. No new review is requested within this approved scope.
|
||||
|
||||
## Revision3 result and proposed revision4
|
||||
|
||||
The founder executed approved revision3: the missing-fragment repair and171-repository coverage verification succeeded; candidate receiver probes passed; hook migration failed before deployment patch. The old policy/template/probes and protected metadata were verified by rollback. The repair remains installed. Exact failed receipts are preserved in `native-r4/`. The pinned Forgejo userspace reproduces the unsupported `chmod --reference` defect; workstation tests had missed it.
|
||||
|
||||
[Revision4](native-r4/PLAN.md), packet SHA256 `64c73100f2347470fda1b8b3194dbf6a8009291d9d80223af463b3c5ecab52fd`, corrects numeric mode handling and exact known group-write restoration, reuses the existing verified immutable ConfigMap, and has a fresh execution/source gate. Eight shell image cases, a four-case171-repository/1539-hook controller run, three controller checks and runtime self-test passed. Candidate scanner/profile and six owner source hashes are unchanged. Await exact revision4 review; no native revision4 action has run. The proposed founder launcher is `/tmp/forge-scanner-rollout-r4.sh`.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,39 @@
|
|||
# Portable scanner rollout, revision 4
|
||||
|
||||
Stable proposal `SYNC-SCANNER-JSON-DIGEST-20261101`, revision 4. Revisions 1–3, exact revision-3 approval and both failed executions remain preserved. Revision 4 has not run natively and requires exact founder review.
|
||||
|
||||
## Verified starting state and defect
|
||||
|
||||
The approved revision-3 run restored the missing old-policy scanner fragment at markitect-main. Its real dispatcher passed clean/private-key/history/quarantine/unavailable-scanner/recovery probes. All 171 fragments and unchanged protected hook metadata were verified. The JSON candidate then passed its native receive probe, but hook migration failed before the deployment patch. Guarded rollback verified the old template/policy, all 171 fragments and old-policy receive probes. The restored protection remains installed.
|
||||
|
||||
A reproducible compatibility defect exists in the reviewed migration script: `chmod --reference` is unavailable in Forgejo 11.0.3-rootless. The exact old migration fails in the published image pinned at `code.forgejo.org/forgejo/forgejo@sha256:691657c7e64af7730a2e885e6d022cf16ae60724931c5392bd61838afadcaeba`, also the pinned guard image in owner source. Workstation tests used a different userspace and missed this defect. Failed native receipts are preserved in this folder; local image tests retain the old-script counterexample.
|
||||
|
||||
The previous attempt left the public candidate binaries and immutable ConfigMap `forgejo-secret-template-619d94208139` staged. They are inert under the currently selected old template. Revision 4 verifies/reuses them rather than creating that existing ConfigMap again.
|
||||
|
||||
## Exact revised action
|
||||
|
||||
Founder launcher: `/tmp/forge-scanner-rollout-r4.sh`, selecting the attended admin context and this packet's exact SHA256 acknowledgement. Agent credentials/authority do not change. Fresh receipt: `/tmp/forge-scanner-rollout-r4-receipt.json`; an existing receipt refuses execution. Earlier receipts are never removed or overwritten.
|
||||
|
||||
1. Verify this sealed packet and unchanged source/binary/probe/input artifacts in revision 2. Keep the exact same six owner-source candidates and candidate adapter SHA256 `d7dbd0ef57312ab9507874bb5fd67c49bef637b6d7dee121e526227307af4bf0`. Recheck node/deployment/pod identity, old guard/template, resources/strategy and all source preimages. Inventory every current repository (maximum 500); all fragments must already have the known old bytes and custody. The completed repair is a required starting state; there is no uncovered-repository exception.
|
||||
2. Snapshot active hook hashes, owners, groups and modes. Supported baseline is uid/gid 1000 and mode 0755 or 0775, as observed in all 1,539 hooks from the failed native execution. Verify the existing target ConfigMap is immutable and has exactly the reviewed hook data; pin its UID against recreation. Do not mutate/create the ConfigMap. Server dry-run the same conditional two-field deployment patch and capture the client ConfigMap diff; raw objects/errors stay withheld.
|
||||
3. Run `runtime_self_test.sh` in private tmpfs. It verifies numeric chmod, stable descriptor-based permission restoration, stat/hash and atomic rename, and cleans up its public fixture. Require success before production hook/deployment changes. This checks actual live userspace, supplementing the pinned-image tests.
|
||||
4. Verify/reuse the previously staged candidate directory with its original marker and exact adapter/Gitleaks hashes. Run the existing native JSON/private-key/history/quarantine/unavailable-scanner/recovery probe before activation.
|
||||
5. Migrate only scanner fragments with the original all-scope prevalidation, known bytes and partial-failure recovery. The two `chmod --reference` calls are replaced by `chmod "$(stat -c %a …)"`, preserving numeric modes in the container's supported syntax. Paths remain bounded to repository scanner fragments. Unknown bytes, symlinks or custody refuse.
|
||||
6. Verify/restore exact baseline hook modes after migration and after pod recreation, including generated/protected hooks whose contents are never overwritten. Kubernetes volume processing may add group-write permission. The only accepted mode difference is baseline 0755 becoming 0775; remove that added bit to restore 0755. A 0775 baseline stays 0775. Require unchanged content hash, uid/gid, regular non-symlink shape and safe parents for every file before the first restoration. For changed modes, open the verified inode, recheck identity/hash/owner, chmod through `/proc/self/fd`, and verify the same final inode/mode. Unknown content, ownership, mode or hook-set changes refuse; never rewrite generated dispatchers or authorization-hook content. The controller also prevalidates protected content before any scanner rewrite.
|
||||
7. Apply the original UID/resourceVersion/before-value JSON patch to append the exact candidate artifact guard and select the new hook ConfigMap. No image, resource, strategy, security-context or credential changes. Wait up to 180 seconds for the correct deployment's Ready pod; repeat migration/inventory to cover known old fragments on repositories created during transition, restore only known group-write additions, then require original protected metadata equality. Stream mode records through stdin rather than placing the large inventory in command arguments.
|
||||
8. Verify all artifact guards, mounted new hook hash, full receiver inventory and post-activation JSON/private-key/history/quarantine/unavailable-scanner/recovery probe. Emit `activation-verified` only after those checks pass.
|
||||
9. After native verification, the agent applies the same six exact owner-source candidates via this folder's `source_followthrough.py`, requiring the approved revision-4 packet and verified revision-4 receipt. The candidates and resulting six hashes are unchanged from revision 2; only the receipt gate changes. Commit/publish through normal hooks, retry eligible held work and continue the CUST-WP-0083 failure audit. UUID/fingerprint/prose findings remain separate adjudication; the scanner exception is not broadened.
|
||||
|
||||
## Recovery and tests
|
||||
|
||||
The original guarded rollback restores old deployment fields/template and known old scanner bytes, with the portable migration and exact baseline mode restoration now used during recovery too. It verifies old artifacts/template, protected metadata and the old-policy real receiver probe. Retain the repaired markitect-main fragment, staged public binaries/ConfigMap and every failed receipt. Unknown concurrent changes are retained and reported as unverified. A failed rollback still requires founder recovery; this is not a guarantee of recovery.
|
||||
|
||||
Tests were run in the pinned Forgejo userspace with no network, host credentials or writable source mounts. Eight shell cases cover the old-script counterexample, apply/rollback with mixed modes, complete-scope refusal, partial failure, symlinks, simulated pod permission additions, unexpected modes and content drift. The actual controller migration/stream framing passed a separate 171-repository/1,539-hook run: apply, post-recreation mode restoration (19 known group-write additions), rollback and protected-content refusal before writes. Three controller tests verify immutable ConfigMap reuse/UID pinning, changed/mutable data refusal and all six original source candidates. The new runtime self-test passed in that image. Original real-Gitleaks source/receiver proof and successful native candidate probe remain applicable because the scanner binary/profile are unchanged. Local tests are not native activation evidence.
|
||||
|
||||
Historical admission, why markitect-main adoption missed its fragment, and complete Forgejo new-repository/write lifecycle coverage remain unproved by this repair. Preserve those limits in the forge-owned class-level followup; no clean-history claim is made.
|
||||
|
||||
## Cost, interruption and authority
|
||||
|
||||
No new infrastructure, service, image or resource commitment. Incremental infrastructure cost EUR 0; founder time/existing compute unpriced. Estimate 20–35 attended minutes. Expected Recreate interruption remains 1–3 minutes; rollback readiness up to another 180 seconds plus bounded commands/probes. Numeric mode restoration takes bounded extra inventory/metadata work (180-second operation limit); interruption estimate is not an availability guarantee.
|
||||
|
||||
Illustrative recovery effort EUR 100 at EUR 100/hour. Complete security/recovery loss remains unpriced and requires explicit founder judgement. Existing filename/digest exception limitations remain; no autopilot authority, risk budget, promotion or unchanged-success credit is implied. Exact review is needed because executable migration/recovery, existing-ConfigMap handling, protected-hook mode restoration and the source receipt gate change. Governing instruction: `docs/agent-autonomy-decision.md`, "Approval is bound to the reviewed proposal revision; changing that revision requires review again."
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
{
|
||||
"result": "passed",
|
||||
"image": "code.forgejo.org/forgejo/forgejo@sha256:691657c7e64af7730a2e885e6d022cf16ae60724931c5392bd61838afadcaeba",
|
||||
"repositories": 171,
|
||||
"hooks": 1539,
|
||||
"cases": [
|
||||
"initial-migration",
|
||||
"post-recreate-mode-restoration",
|
||||
"rollback-mode-restoration",
|
||||
"protected-drift-before-write"
|
||||
],
|
||||
"group_write_additions_reversed": 19,
|
||||
"events": [
|
||||
{
|
||||
"event": "hooks-apply",
|
||||
"repositories": 171,
|
||||
"portable_modes": true,
|
||||
"baseline_modes_restored": true,
|
||||
"group_write_additions_reversed": 0
|
||||
},
|
||||
{
|
||||
"event": "hooks-apply",
|
||||
"repositories": 171,
|
||||
"portable_modes": true,
|
||||
"baseline_modes_restored": true,
|
||||
"group_write_additions_reversed": 19
|
||||
},
|
||||
{
|
||||
"event": "hooks-rollback",
|
||||
"repositories": 171,
|
||||
"portable_modes": true,
|
||||
"baseline_modes_restored": true,
|
||||
"group_write_additions_reversed": 19
|
||||
}
|
||||
],
|
||||
"native_mutation": false
|
||||
}
|
||||
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
|
|
@ -0,0 +1,66 @@
|
|||
#!/bin/sh
|
||||
# Caller supplies a sealed, exact path list and known public before/after bytes.
|
||||
set +vx
|
||||
set -eu
|
||||
PATH=/usr/bin:/bin
|
||||
export PATH
|
||||
area=$1 direction=$2 before_sum=$3 after_sum=$4
|
||||
test "$(id -u)" = 1000
|
||||
for file in before after paths; do test -f "$area/$file" && test ! -L "$area/$file"; done
|
||||
test "$(sha256sum "$area/before" | cut -d ' ' -f 1)" = "$before_sum"
|
||||
test "$(sha256sum "$area/after" | cut -d ' ' -f 1)" = "$after_sum"
|
||||
case "$direction" in apply) ;; rollback) ;; *) exit 35;; esac
|
||||
validate_path() {
|
||||
path=$1
|
||||
case "$path" in /data/git/gitea-repositories/*/*.git/hooks/pre-receive.d/10-custodian-secret-scan) ;; *) exit 36;; esac
|
||||
parent=${path%/*}
|
||||
while test "$parent" != /data; do
|
||||
test -d "$parent" && test ! -L "$parent"
|
||||
parent=${parent%/*}
|
||||
done
|
||||
test -f "$path" && test ! -L "$path"
|
||||
test "$(stat -c %u "$path")" = 1000
|
||||
case "$(stat -c %a "$path")" in 755|775) ;; *) exit 37;; esac
|
||||
current=$(sha256sum "$path" | cut -d ' ' -f 1)
|
||||
if test "$direction" = apply; then test "$current" = "$before_sum"
|
||||
else test "$current" = "$before_sum" || test "$current" = "$after_sum"; fi
|
||||
}
|
||||
# Validate the complete scope before the first write.
|
||||
count=0
|
||||
while IFS= read -r path; do validate_path "$path"; count=$((count+1)); done < "$area/paths"
|
||||
test "$count" -gt 0
|
||||
temporary=
|
||||
restore_partial() {
|
||||
code=$?
|
||||
trap - EXIT HUP INT TERM
|
||||
if test -n "$temporary"; then rm -f -- "$temporary" || code=38; fi
|
||||
# Only our exact replacement may be restored. Unknown drift is retained.
|
||||
if test "$direction" = apply && test "$code" != 0; then
|
||||
while IFS= read -r path; do
|
||||
if test -f "$path" && test ! -L "$path" &&
|
||||
test "$(sha256sum "$path" | cut -d ' ' -f 1)" = "$after_sum"; then
|
||||
staged=$(mktemp "${path%/*}/.custodian-restore-XXXXXX") || { code=38; continue; }
|
||||
if cp "$area/before" "$staged" && chmod "$(stat -c %a "$path")" "$staged" && mv -T -- "$staged" "$path"; then :
|
||||
else rm -f -- "$staged"; code=38; fi
|
||||
fi
|
||||
done < "$area/paths"
|
||||
fi
|
||||
exit "$code"
|
||||
}
|
||||
trap restore_partial EXIT
|
||||
trap 'exit 39' HUP INT TERM
|
||||
while IFS= read -r path; do
|
||||
validate_path "$path"
|
||||
if test "$direction" = rollback; then source="$area/before"; sum=$before_sum; mode=775
|
||||
else source="$area/after"; sum=$after_sum; mode=755; fi
|
||||
if test "$(sha256sum "$path" | cut -d ' ' -f 1)" != "$sum"; then
|
||||
temporary=$(mktemp "${path%/*}/.custodian-replace-XXXXXX")
|
||||
cp "$source" "$temporary"
|
||||
chmod "$(stat -c %a "$path")" "$temporary"
|
||||
test "$(sha256sum "$temporary" | cut -d ' ' -f 1)" = "$sum"
|
||||
mv -T -- "$temporary" "$path"
|
||||
temporary=
|
||||
fi
|
||||
test "$(sha256sum "$path" | cut -d ' ' -f 1)" = "$sum"
|
||||
done < "$area/paths"
|
||||
printf '{"hook_files_verified":%s,"direction":"%s"}\n' "$count" "$direction"
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
{
|
||||
"image": "code.forgejo.org/forgejo/forgejo@sha256:691657c7e64af7730a2e885e6d022cf16ae60724931c5392bd61838afadcaeba",
|
||||
"result": "passed",
|
||||
"cases": [
|
||||
{
|
||||
"case": "old-counterexample",
|
||||
"returncode": 0
|
||||
},
|
||||
{
|
||||
"case": "roundtrip",
|
||||
"returncode": 0
|
||||
},
|
||||
{
|
||||
"case": "complete-scope-refusal",
|
||||
"returncode": 0
|
||||
},
|
||||
{
|
||||
"case": "partial-failure",
|
||||
"returncode": 0
|
||||
},
|
||||
{
|
||||
"case": "symlink-refusal",
|
||||
"returncode": 0
|
||||
},
|
||||
{
|
||||
"case": "fsgroup-modes",
|
||||
"returncode": 0
|
||||
},
|
||||
{
|
||||
"case": "mode-drift",
|
||||
"returncode": 0
|
||||
},
|
||||
{
|
||||
"case": "hash-drift",
|
||||
"returncode": 0
|
||||
}
|
||||
],
|
||||
"native_mutation": false
|
||||
}
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
{
|
||||
"proposal_id": "SYNC-SCANNER-JSON-DIGEST-20261101",
|
||||
"revision": 4,
|
||||
"reused_r2_packet_sha256": "d22b1f555af96c203b7d68efd25e80815ce18ca4e3bf0dd3ab96b6f699d50237",
|
||||
"files": {
|
||||
"PLAN.md": "0800de1fe3307613071d736be8536fa047e8894923a6bea13daee85dd5254c02",
|
||||
"controller-image-tests.json": "4b478fd0651550b2719a0d23e37e1c77d561482941cea18ef364ffe4b6a7e553",
|
||||
"failed-r3-native-receipt.json": "10b0992da2e659cc8adb197fb0ec926702d84000b8fd4ce18c7fae58ddccd38b",
|
||||
"failed-r3-recovery-receipt.json": "b4a17af4a7419e83edbc38295d734d4e662243e7182a8c4039fe6307182185a8",
|
||||
"hooks.sh": "d207b005989b3e1c8205cd0e72490952bfbef634c0f26af15c3cbabdb610c082",
|
||||
"image-tests.json": "f07e340381ac848a67eac789946db7f348aab2e08cb034fab6206723087435e7",
|
||||
"prior-execution-outcome.json": "9af96e2af8e8715b2f542dfea532c714d626dff761a78aba039b5f2d0a273865",
|
||||
"proposal.json": "7e5c1c33e6519abacc84e4420c2a9ad70a84a96fa4c9578e48dbc45322d5055b",
|
||||
"restore_modes.sh": "06dfb60bb2d33d3b5cef0ff5e9298c89328d6bbf6ebd2c98ae78cefeb45910b7",
|
||||
"rollout.py": "e654911c38154fc803f7abb01f410940c9b34627436716b34cea2f8544760889",
|
||||
"run_image_tests.py": "626f8083d726101c7b88a673a608d83cca7cfe8bbac430b2ce58bb96af974254",
|
||||
"runtime-image-test.json": "aeff32499bafb4bacca16d2ad72b824453734c1812cdbd2c0a24be29f8f66721",
|
||||
"runtime_self_test.sh": "8d68268a3f8d6cfce07e723893de7657e431f19000d915403a5d7218d284f269",
|
||||
"source_followthrough.py": "053d1758a788c40612677e8d1dc93e6aefda04d339f1b161d6527ba6a71cfd92",
|
||||
"test_controller.py": "ec8ee5c68b772851a44e0b59b5bca9af6159a1420ecac4f9aa60f2cef7b8de8d",
|
||||
"test_controller_image.py": "88eea8f204987d0b1452b4af66ee2609bf02d2dee9c847ebaee7a0fc944bef84",
|
||||
"test_image.sh": "f82fe8b830cc4cc341d8b50e18e034415a7ad2b77140c7e7dea8a0f56c468f41"
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
{
|
||||
"proposal_id": "SYNC-SCANNER-JSON-DIGEST-20261101",
|
||||
"executed_revision": 3,
|
||||
"packet_sha256": "df1cb48259ec5411a3f8cd24d436078daf09d241c39448454526f0a796798c34",
|
||||
"outcome": "failed",
|
||||
"partial_result": "old-policy fragment repair verified across171; candidate receiver probe passed; hook migration failed before deployment patch",
|
||||
"recovery": "rollback verified old policy/metadata/probes across171; repaired fragment retained",
|
||||
"compatibility_defect": "chmod --reference unsupported in pinned Forgejo image; original migration counterexample reproduced there",
|
||||
"failed_recovery_receipt_sha256": "b4a17af4a7419e83edbc38295d734d4e662243e7182a8c4039fe6307182185a8",
|
||||
"failed_native_receipt_sha256": "10b0992da2e659cc8adb197fb0ec926702d84000b8fd4ce18c7fae58ddccd38b",
|
||||
"new_policy_activated": false,
|
||||
"six_owner_sources_applied": false
|
||||
}
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
{
|
||||
"proposal_id": "SYNC-SCANNER-JSON-DIGEST-20261101",
|
||||
"revision": 4,
|
||||
"original_revision": 1,
|
||||
"reused_r2_packet_sha256": "d22b1f555af96c203b7d68efd25e80815ce18ca4e3bf0dd3ab96b6f699d50237",
|
||||
"target": "railiance01 forgejo/forge; already-covered scanner fragments; same six owner source candidates",
|
||||
"expected_result": "Portable JSON-profile activation with exact protected hook bytes/metadata preserved; completed coverage repair retained",
|
||||
"verification_ref": "PLAN.md, image-tests.json, controller-image-tests.json, runtime-image-test.json and3 controller tests; native runtime self-test/dry-run/probes mandatory",
|
||||
"rollback_ref": "PLAN.md; original guarded rollback using portable migration and exact mode restoration; all old artifacts and failed receipts retained",
|
||||
"cost_estimate_eur": {
|
||||
"additional_infrastructure": 0,
|
||||
"attended_minutes": [
|
||||
20,
|
||||
35
|
||||
],
|
||||
"founder_time": "unpriced"
|
||||
},
|
||||
"risk_estimate_eur": {
|
||||
"complete_credible_loss_bound": null,
|
||||
"illustrative_recovery": 100,
|
||||
"basis": "one recovery hour at EUR100/hour; full security/recovery loss unpriced, supervision required"
|
||||
},
|
||||
"authority": "pending exact founder revision4 review; not natively executed",
|
||||
"same_candidate_binary_sha256": "d7dbd0ef57312ab9507874bb5fd67c49bef637b6d7dee121e526227307af4bf0",
|
||||
"source_candidates": "unchanged six exact candidates from revision2; revision4 receipt gate"
|
||||
}
|
||||
|
|
@ -0,0 +1,48 @@
|
|||
#!/bin/sh
|
||||
# Restore only known group-write additions on unchanged, owned hook inodes.
|
||||
set +vx
|
||||
set -eu
|
||||
export PATH=/usr/bin:/bin
|
||||
records=$1
|
||||
test "$(id -u)" = 1000
|
||||
fail() { printf '{"mode_check":"%s","result":"refused"}\n' "$1"; exit 61; }
|
||||
tab=$(printf '\t')
|
||||
validate() {
|
||||
mode=$1 sum=$2 path=$3
|
||||
case "$mode" in 755|775) ;; *) fail unsupported-baseline-mode;; esac
|
||||
case "$path" in /data/git/gitea-repositories/*/*.git/hooks/*) ;; *) fail path-scope;; esac
|
||||
parent=${path%/*}
|
||||
while test "$parent" != /data; do
|
||||
test -d "$parent" && test ! -L "$parent" || fail parent-shape
|
||||
parent=${parent%/*}
|
||||
done
|
||||
test -f "$path" && test ! -L "$path" || fail file-shape
|
||||
test "$(stat -c %u "$path")" = 1000 && test "$(stat -c %g "$path")" = 1000 || fail file-owner
|
||||
test "$(sha256sum "$path" | cut -d ' ' -f 1)" = "$sum" || fail file-hash
|
||||
current=$(stat -c %a "$path")
|
||||
test "$current" = "$mode" || { test "$mode" = 755 && test "$current" = 775; } || fail unexpected-mode
|
||||
}
|
||||
count=0
|
||||
while IFS="$tab" read -r mode sum path; do validate "$mode" "$sum" "$path"; count=$((count+1)); done < "$records"
|
||||
test "$count" -gt 0 || fail empty-scope
|
||||
changed=0
|
||||
while IFS="$tab" read -r mode sum path; do
|
||||
validate "$mode" "$sum" "$path"
|
||||
if test "$current" != "$mode"; then
|
||||
inode=$(stat -c '%d:%i' "$path")
|
||||
exec 7< "$path"
|
||||
fd=/proc/self/fd/7
|
||||
test "$(stat -L -c '%d:%i' "$fd")" = "$inode" || fail opened-inode
|
||||
test "$(stat -L -c %u "$fd")" = 1000 && test "$(stat -L -c %g "$fd")" = 1000 || fail opened-owner
|
||||
test "$(stat -L -c %a "$fd")" = 775 || fail opened-mode
|
||||
test "$(sha256sum "$fd" | cut -d ' ' -f 1)" = "$sum" || fail opened-hash
|
||||
chmod "$mode" "$fd"
|
||||
test "$(stat -L -c %a "$fd")" = "$mode" || fail restored-mode
|
||||
test -f "$path" && test ! -L "$path" && test "$(stat -c '%d:%i' "$path")" = "$inode" || fail final-inode
|
||||
exec 7<&-
|
||||
changed=$((changed+1))
|
||||
fi
|
||||
validate "$mode" "$sum" "$path"
|
||||
test "$current" = "$mode" || fail final-mode
|
||||
done < "$records"
|
||||
printf '{"hook_modes_verified":%s,"group_write_additions_reversed":%s}\n' "$count" "$changed"
|
||||
194
docs/changes/2026-11-01-scanner-json-digest/native-r4/rollout.py
Normal file
194
docs/changes/2026-11-01-scanner-json-digest/native-r4/rollout.py
Normal file
|
|
@ -0,0 +1,194 @@
|
|||
"""Founder-run r4: portable hook migration, exact mode restoration, existing CM.
|
||||
|
||||
Original r2/r3 code and failed receipts are preserved. Same scanner binary,
|
||||
profile, deployment fields and six owner-source candidates. Acknowledgement
|
||||
binds reviewed bytes; it is not a credential or runtime approval service.
|
||||
"""
|
||||
from pathlib import Path
|
||||
import argparse
|
||||
import base64
|
||||
import copy
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import json
|
||||
import re
|
||||
import shlex
|
||||
import subprocess
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
ORIGINAL = HERE.parent / 'native-r2'
|
||||
OUTPUT = Path('/tmp/forge-scanner-rollout-r4-receipt.json')
|
||||
R2 = 'd22b1f555af96c203b7d68efd25e80815ce18ca4e3bf0dd3ab96b6f699d50237'
|
||||
|
||||
|
||||
def sha(raw): return hashlib.sha256(raw).hexdigest()
|
||||
|
||||
|
||||
def ensure(ok, reason):
|
||||
if not ok: raise RuntimeError(reason)
|
||||
|
||||
|
||||
def support():
|
||||
spec = importlib.util.spec_from_file_location('original_rollout', ORIGINAL / 'rollout.py')
|
||||
m = importlib.util.module_from_spec(spec); spec.loader.exec_module(m)
|
||||
ensure(m.verify_packet() == R2, 'original-reviewed-packet-drift')
|
||||
m.RECEIPT = OUTPUT
|
||||
return m
|
||||
|
||||
|
||||
def verify_packet():
|
||||
raw = (HERE / 'packet.json').read_bytes(); p = json.loads(raw)
|
||||
for name, digest in p['files'].items():
|
||||
ensure(Path(name).name == name and sha((HERE / name).read_bytes()) == digest, 'r4-packet-drift')
|
||||
support()
|
||||
return sha(raw)
|
||||
|
||||
|
||||
def operation(args):
|
||||
if 'exec' in args:
|
||||
for source, label in [(HERE/'runtime_self_test.sh','runtime-tool-self-test'), (HERE/'hooks.sh','hook-migration'), (HERE/'restore_modes.sh','hook-mode-restoration'),
|
||||
(ORIGINAL/'stage.sh','artifact-staging'), (ORIGINAL/'probe.sh','candidate-receive-probe'),
|
||||
(ORIGINAL/'rollback-probe.sh','old-policy-receive-probe')]:
|
||||
if source.read_text() in args: return label
|
||||
if any(isinstance(a,str) and 'custodian-replace-' in a for a in args): return 'hook-migration'
|
||||
if any(isinstance(a,str) and 'opened-inode' in a for a in args): return 'hook-mode-restoration'
|
||||
if any(isinstance(a,str) and 'for repo in /data/git/gitea-repositories/' in a for a in args): return 'hook-inventory'
|
||||
return 'public-artifact-or-template-check'
|
||||
for verb in ('get','patch'):
|
||||
if verb in args: return verb + ':' + args[args.index(verb)+1]
|
||||
return 'reviewed-kubernetes-operation'
|
||||
|
||||
|
||||
def make_runner(m):
|
||||
class Rollout(m.Rollout):
|
||||
def k(self, *args, data=None, timeout=60):
|
||||
label = operation(args)
|
||||
try: q = subprocess.run(['kubectl',*args], input=data, capture_output=True, timeout=timeout)
|
||||
except subprocess.TimeoutExpired:
|
||||
self.event('command-failed', operation=label, category='timeout')
|
||||
raise RuntimeError('kubernetes-timeout:' + label)
|
||||
if q.returncode:
|
||||
category = 'remote-check-nonzero' if b'command terminated with exit code' in q.stderr else 'command-failed'
|
||||
for needle,name in [(b'Forbidden','forbidden'),(b'Unauthorized','unauthorized'),(b'NotFound','not-found')]:
|
||||
if needle in q.stderr: category=name
|
||||
self.event('command-failed', operation=label, category=category)
|
||||
raise RuntimeError('kubernetes-command-failed:' + label)
|
||||
return q.stdout
|
||||
|
||||
def existing_cm(self):
|
||||
cm = json.loads(self.k('-n','forgejo','get','configmap',self.c['new_configmap'],'-o','json'))
|
||||
wanted = json.loads((ORIGINAL/'configmap.json').read_bytes())
|
||||
ensure(cm.get('immutable') is True and cm.get('data') == wanted['data'] and
|
||||
cm['metadata']['name'] == self.c['new_configmap'] and cm['metadata']['namespace'] == 'forgejo', 'existing-public-configmap-drift')
|
||||
uid = cm['metadata']['uid']
|
||||
ensure(not hasattr(self,'cm_uid') or self.cm_uid == uid, 'existing-configmap-identity-drift')
|
||||
self.cm_uid = uid
|
||||
|
||||
def dry_run(self,d):
|
||||
self.existing_cm()
|
||||
patch = m.patch_for(d,self.c)
|
||||
self.k('-n','forgejo','patch','deployment','forge','--type=json','--dry-run=server','-p',json.dumps(patch),'-o','name')
|
||||
diff = subprocess.run(['kubectl','diff','-f','-'], input=(ORIGINAL/'configmap.json').read_bytes(), capture_output=True, timeout=60)
|
||||
ensure(diff.returncode in (0,1),'client-diff-failed; raw diagnostics withheld')
|
||||
self.event('server-dry-run-passed', reused_immutable_configmap=self.c['new_configmap'], configmap_uid=self.cm_uid,
|
||||
client_configmap_diff_exit=diff.returncode, deployment_changes=2, strategy='Recreate', resources_unchanged=True)
|
||||
|
||||
def preflight(self):
|
||||
result = super().preflight()
|
||||
for files in self.receipt['before_hooks'].values():
|
||||
for f in files.values():
|
||||
ensure(f['uid'] == 1000 and f['gid'] == 1000 and f['mode'] in ('755','775'),'unsupported-hook-custody-or-mode')
|
||||
return result
|
||||
|
||||
def migrate(self,pod,direction):
|
||||
c=self.c
|
||||
before=self.inventory(pod,{c['old_hook_sha256'],c['new_hook_sha256']})
|
||||
baseline=self.receipt['before_hooks']
|
||||
reference=copy.deepcopy(before)
|
||||
records=[]
|
||||
expected=c['old_hook_sha256'] if direction=='rollback' else c['new_hook_sha256']
|
||||
for repo,files in baseline.items():
|
||||
ensure(repo in before and set(before[repo]) == set(files),'hook-set-drift')
|
||||
for name,f in files.items():
|
||||
now=before[repo][name]
|
||||
ensure((now['uid'],now['gid'])==(f['uid'],f['gid']),'hook-owner-drift')
|
||||
ensure(now['mode']==f['mode'] or (f['mode']=='755' and now['mode']=='775'),'unexpected-hook-mode-drift')
|
||||
if name!=m.HOOK: ensure(now['sha256']==f['sha256'],'protected-hook-content-drift')
|
||||
reference[repo][name]=dict(f)
|
||||
digest=expected if name==m.HOOK else f['sha256']
|
||||
records.append(f['mode']+'\t'+digest+'\t'+repo+'/hooks/'+name+'\n')
|
||||
paths=''.join(repo+'/hooks/'+m.HOOK+'\n' for repo,files in sorted(before.items())
|
||||
if direction=='rollback' or files[m.HOOK]['sha256']==c['old_hook_sha256'])
|
||||
# One private tmpfs control area; public hooks/paths/modes only.
|
||||
wrapper='''set +vx
|
||||
set -eu
|
||||
umask 077
|
||||
test "$(stat -f -c %t /dev/shm)" = 1021994
|
||||
area=$(mktemp -d /dev/shm/forge-r4-hooks-XXXXXXXX)
|
||||
cleanup() { code=$?; trap - EXIT; rm -f -- "$area/before" "$area/after" "$area/paths" "$area/modes"; rmdir "$area" || code=70; exit "$code"; }
|
||||
trap cleanup EXIT
|
||||
'''
|
||||
for name in ('before','after'):
|
||||
b64=base64.b64encode((ORIGINAL/(name+'-hook')).read_bytes()).decode()
|
||||
wrapper+='printf %s '+shlex.quote(b64)+' | base64 -d > "$area/'+name+'"\n'
|
||||
# Stream records through stdin; never put a large inventory in argv.
|
||||
wrapper+=': > "$area/paths"\nwhile IFS= read -r line; do\n test "$line" = __FORGE_R4_MODE_RECORDS__ && break\n printf "%s\\n" "$line" >> "$area/paths"\ndone\ncat > "$area/modes"\n'
|
||||
if paths:
|
||||
wrapper+='sh -c '+shlex.quote((HERE/'hooks.sh').read_text())+' sh "$area" "$1" "$2" "$3"\n'
|
||||
wrapper+='sh -c '+shlex.quote((HERE/'restore_modes.sh').read_text())+' sh "$area/modes"\n'
|
||||
# shell output contains two JSON lines; only report vetted counters.
|
||||
raw=self.shell(pod,wrapper,(direction,c['old_hook_sha256'],c['new_hook_sha256']),data=(paths+'__FORGE_R4_MODE_RECORDS__\n'+''.join(records)).encode(),timeout=180)
|
||||
counters=[json.loads(line) for line in raw.splitlines() if line]
|
||||
after=self.inventory(pod,{expected})
|
||||
m.protected_equal(reference,after)
|
||||
self.event('hooks-'+direction,repositories=len(after),portable_modes=True,baseline_modes_restored=True,
|
||||
group_write_additions_reversed=sum(x.get('group_write_additions_reversed',0) for x in counters))
|
||||
return after
|
||||
|
||||
def apply(self,binary):
|
||||
d,pod,before=self.preflight()
|
||||
raw=binary.read_bytes();ensure(sha(raw)==self.c['new_adapter_sha256'],'candidate-binary-drift')
|
||||
tools=json.loads(self.shell(pod,(HERE/'runtime_self_test.sh').read_text()))
|
||||
ensure(all(tools.get(k) for k in ('numeric_modes','descriptor_mode_restore','atomic_rename','temporary_test_removed')),'runtime-tool-self-test-failed')
|
||||
self.event('runtime-tool-self-test-passed',checks=tools)
|
||||
self.receipt['native_mutation']=True;self.event('staging-started')
|
||||
try:
|
||||
self.shell(pod,(ORIGINAL/'stage.sh').read_text(),(self.c['new_dir'],self.c['old_dir'],self.c['new_adapter_sha256'],self.c['scanner_sha256'],'SYNC-SCANNER-JSON-DIGEST-20261101-R2'),data=raw)
|
||||
self.shell(pod,self.c['new_guard_command'][2],('check',))
|
||||
self.probe(pod,self.c['new_dir'],before)
|
||||
self.existing_cm()
|
||||
self.migrate(pod,'apply')
|
||||
patch=m.patch_for(self.deployment(),self.c)
|
||||
self.k('-n','forgejo','patch','deployment','forge','--type=json','-p',json.dumps(patch),'-o','name')
|
||||
self.event('deployment-patched')
|
||||
pod=self.ready_pod(self.deployment())
|
||||
after=self.migrate(pod,'apply')
|
||||
m.protected_equal(before,after)
|
||||
self.shell(pod,self.c['new_guard_command'][2],('check',))
|
||||
mounted=self.shell(pod,'cat /opt/forge-secret-template/hooks/pre-receive.d/10-custodian-secret-scan')
|
||||
ensure(sha(mounted)==self.c['new_hook_sha256'],'mounted-template-drift')
|
||||
self.probe(pod,self.c['new_dir'],after)
|
||||
self.event('activation-verified',repositories=len(after),hook_sha256=self.c['new_hook_sha256'],baseline_hook_metadata_preserved=True)
|
||||
except Exception as exc:
|
||||
self.event('activation-failed',reason=str(exc) if isinstance(exc,RuntimeError) else type(exc).__name__)
|
||||
try:self.rollback(before)
|
||||
except Exception as recovery:self.event('rollback-unverified',reason=str(recovery) if isinstance(recovery,RuntimeError) else type(recovery).__name__)
|
||||
raise
|
||||
return Rollout
|
||||
|
||||
|
||||
def main():
|
||||
p=argparse.ArgumentParser(description=__doc__);p.add_argument('--approved-digest',required=True)
|
||||
p.add_argument('--binary',type=Path,default=Path('/tmp/sync-scanner-candidate/receive-secret-scan-candidate'));args=p.parse_args()
|
||||
digest=verify_packet();ensure(args.approved_digest==digest,'exact-founder-acknowledgement-required')
|
||||
m=support();c=json.loads((ORIGINAL/'inputs.json').read_bytes());runner=make_runner(m)(c)
|
||||
runner.receipt.update(revision=4,packet_sha256=digest,reused_r2_packet_sha256=R2,mode='apply')
|
||||
with OUTPUT.open('x') as stream:stream.write(json.dumps(runner.receipt)+'\n')
|
||||
runner.apply(args.binary)
|
||||
|
||||
|
||||
if __name__=='__main__':
|
||||
try:main()
|
||||
except Exception as exc:
|
||||
print(json.dumps({'result':'refused-or-unverified','reason':str(exc) if isinstance(exc,RuntimeError) else type(exc).__name__}))
|
||||
raise SystemExit(1)
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
from pathlib import Path
|
||||
import subprocess,json,time
|
||||
p=Path(__file__).resolve().parent
|
||||
image='code.forgejo.org/forgejo/forgejo@sha256:691657c7e64af7730a2e885e6d022cf16ae60724931c5392bd61838afadcaeba'
|
||||
prefix=['docker','run','--rm','--network=none','--user=1000:1000','--read-only','--cap-drop=ALL','--security-opt=no-new-privileges:true','--tmpfs','/tmp:rw,size=64m,uid=1000,gid=1000','--tmpfs','/data:rw,size=64m,uid=1000,gid=1000','--mount','type=bind,src='+str(p)+',dst=/scripts,readonly','--mount','type=bind,src='+str(p.parent/'native-r2')+',dst=/original,readonly','--entrypoint=/bin/sh',image,'/scripts/test_image.sh']
|
||||
results=[]
|
||||
for case in ['old-counterexample','roundtrip','complete-scope-refusal','partial-failure','symlink-refusal','fsgroup-modes','mode-drift','hash-drift']:
|
||||
q=subprocess.run(prefix+[case],capture_output=True,timeout=60)
|
||||
results.append({'case':case,'returncode':q.returncode})
|
||||
(p/'image-tests.json').write_text(json.dumps({'image':image,'result':'passed' if len(results)==8 and all(x['returncode']==0 for x in results) else 'unverified','cases':results,'native_mutation':False},indent=2)+'\n')
|
||||
print(json.dumps(results[-1]),flush=True)
|
||||
if q.returncode:
|
||||
print('test failed; raw output withheld');raise SystemExit(1)
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
{
|
||||
"numeric_modes": true,
|
||||
"descriptor_mode_restore": true,
|
||||
"atomic_rename": true,
|
||||
"temporary_test_removed": true,
|
||||
"reference_option_supported": false,
|
||||
"image": "code.forgejo.org/forgejo/forgejo@sha256:691657c7e64af7730a2e885e6d022cf16ae60724931c5392bd61838afadcaeba",
|
||||
"native_mutation": false
|
||||
}
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
#!/bin/sh
|
||||
# Disposable public tool fixture in tmpfs, before hosted-hook/deployment changes.
|
||||
set +vx
|
||||
set -eu
|
||||
export PATH=/usr/bin:/bin
|
||||
test "$(id -u)" = 1000
|
||||
test "$(stat -f -c %t /dev/shm)" = 1021994
|
||||
umask 077
|
||||
area=$(mktemp -d /dev/shm/forge-r4-tools-XXXXXXXX)
|
||||
inode=$(stat -c '%d:%i' "$area")
|
||||
cleanup() {
|
||||
code=$?
|
||||
trap - EXIT HUP INT TERM
|
||||
if test -d "$area" && test ! -L "$area" && test "$(stat -c '%d:%i' "$area")" = "$inode"; then
|
||||
rm -f -- "$area/file" "$area/replacement" || code=71
|
||||
rmdir "$area" || code=71
|
||||
else code=71; fi
|
||||
exit "$code"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
trap 'exit 72' HUP INT TERM
|
||||
printf 'public numeric-mode fixture\n' > "$area/file"
|
||||
chmod 775 "$area/file"
|
||||
file_inode=$(stat -c '%d:%i' "$area/file")
|
||||
exec 7< "$area/file"
|
||||
test "$(stat -L -c '%d:%i' /proc/self/fd/7)" = "$file_inode"
|
||||
chmod 755 /proc/self/fd/7
|
||||
test "$(stat -L -c %a /proc/self/fd/7)" = 755
|
||||
test "$(stat -c %a "$area/file")" = 755
|
||||
test "$(sha256sum /proc/self/fd/7 | cut -d ' ' -f 1)" = "$(sha256sum "$area/file" | cut -d ' ' -f 1)"
|
||||
exec 7<&-
|
||||
cp "$area/file" "$area/replacement"
|
||||
mv -T -- "$area/replacement" "$area/file"
|
||||
test -f "$area/file" && test ! -L "$area/file"
|
||||
reference=false
|
||||
if chmod --reference=/dev/null --help >/dev/null 2>&1; then reference=true; fi
|
||||
rm -f -- "$area/file"
|
||||
rmdir "$area"
|
||||
test ! -e "$area"
|
||||
trap - EXIT HUP INT TERM
|
||||
printf '{"numeric_modes":true,"descriptor_mode_restore":true,"atomic_rename":true,"temporary_test_removed":true,"reference_option_supported":%s}\n' "$reference"
|
||||
|
|
@ -0,0 +1,87 @@
|
|||
"""Apply exactly reviewed paired scanner and Apps source after native verification.
|
||||
|
||||
No Git operations or native actions. Requires founder acknowledgement of packet.
|
||||
All six original hashes are checked before writing; differing owner work is refused.
|
||||
"""
|
||||
from pathlib import Path
|
||||
import argparse
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import tempfile
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
SOURCE = HERE.parent / 'native-r2'
|
||||
|
||||
|
||||
def sha(raw): return hashlib.sha256(raw).hexdigest()
|
||||
|
||||
|
||||
def patched(original, diff):
|
||||
before = original.decode().splitlines(keepends=True)
|
||||
result = []; cursor = 0; lines = diff.splitlines(keepends=True); i = 2
|
||||
while i < len(lines):
|
||||
match = re.fullmatch(r'@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@[^\n]*\n?', lines[i])
|
||||
if not match: raise ValueError('malformed-hunk')
|
||||
start = int(match[1]) - 1
|
||||
if start < cursor: raise ValueError('overlapping-hunk')
|
||||
result.extend(before[cursor:start]); cursor = start; i += 1
|
||||
while i < len(lines) and not lines[i].startswith('@@ '):
|
||||
line = lines[i]; tag, content = line[0], line[1:]
|
||||
if tag in (' ', '-'):
|
||||
if cursor >= len(before) or before[cursor] != content: raise ValueError('hunk-preimage-drift')
|
||||
cursor += 1
|
||||
if tag in (' ', '+'): result.append(content)
|
||||
elif tag != '-': raise ValueError('unsupported-patch-line')
|
||||
i += 1
|
||||
result.extend(before[cursor:])
|
||||
return ''.join(result).encode()
|
||||
|
||||
|
||||
def candidates():
|
||||
changes = json.loads((SOURCE / 'scanner-source-changes.json').read_bytes())
|
||||
changes += json.loads((SOURCE / 'apps-source-changes.json').read_bytes())
|
||||
result = []
|
||||
for change in changes:
|
||||
target = Path(change['target']); raw = target.read_bytes()
|
||||
if sha(raw) != change['original_sha256']: raise RuntimeError('owner-source-drift:' + str(target))
|
||||
candidate = ((SOURCE / change['candidate_file']).read_bytes() if 'candidate_file' in change else patched(raw, change['patch']))
|
||||
if sha(candidate) != change['candidate_sha256']: raise RuntimeError('candidate-hash-mismatch')
|
||||
result.append((target, raw, candidate))
|
||||
return result
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--apply', action='store_true')
|
||||
parser.add_argument('--approved-digest')
|
||||
args = parser.parse_args()
|
||||
spec = importlib.util.spec_from_file_location('rollout', HERE / 'rollout.py')
|
||||
module = importlib.util.module_from_spec(spec); spec.loader.exec_module(module)
|
||||
digest = module.verify_packet()
|
||||
prepared = candidates()
|
||||
if args.apply:
|
||||
if args.approved_digest != digest: raise RuntimeError('exact-founder-acknowledgement-required')
|
||||
receipt = json.loads(Path('/tmp/forge-scanner-rollout-r4-receipt.json').read_bytes())
|
||||
if receipt.get('packet_sha256') != digest or receipt['events'][-1]['event'] != 'activation-verified':
|
||||
raise RuntimeError('native-activation-not-verified')
|
||||
for target, original, candidate in prepared:
|
||||
if target.read_bytes() != original: raise RuntimeError('concurrent-owner-source-drift')
|
||||
mode = target.stat().st_mode & 0o777
|
||||
fd, name = tempfile.mkstemp(prefix='.scanner-reviewed-', dir=target.parent)
|
||||
try:
|
||||
with os.fdopen(fd, 'wb') as stream: stream.write(candidate); stream.flush(); os.fsync(stream.fileno())
|
||||
os.chmod(name, mode)
|
||||
os.replace(name, target)
|
||||
finally:
|
||||
if os.path.exists(name): os.unlink(name)
|
||||
print(json.dumps({'result': 'applied' if args.apply else 'preimages-and-candidates-verified', 'files': len(prepared), 'packet_sha256': digest}))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
try: main()
|
||||
except Exception as error:
|
||||
print(json.dumps({'result': 'refused-or-unverified', 'reason': str(error) if isinstance(error, RuntimeError) else type(error).__name__}))
|
||||
raise SystemExit(1)
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
"""Existing immutable ConfigMap and unchanged source preimages; no cluster access."""
|
||||
from pathlib import Path
|
||||
import copy
|
||||
import importlib.util
|
||||
import json
|
||||
import unittest
|
||||
|
||||
HERE=Path(__file__).resolve().parent
|
||||
s=importlib.util.spec_from_file_location('r4',HERE/'rollout.py');r=importlib.util.module_from_spec(s);s.loader.exec_module(r)
|
||||
m=r.support();C=json.loads((r.ORIGINAL/'inputs.json').read_bytes())
|
||||
|
||||
|
||||
class Tests(unittest.TestCase):
|
||||
def runner(self,cm):
|
||||
class Fake(r.make_runner(m)):
|
||||
def k(self,*args,**kw):
|
||||
self.last_command=args
|
||||
return json.dumps(cm).encode()
|
||||
return Fake(C)
|
||||
def cm(self):
|
||||
cm=json.loads((r.ORIGINAL/'configmap.json').read_bytes());cm['metadata']['uid']='known-public-cm-uid';return cm
|
||||
def test_identical_existing_immutable_cm_is_reused_with_uid_pin(self):
|
||||
cm=self.cm();runner=self.runner(cm);runner.existing_cm();runner.existing_cm()
|
||||
self.assertEqual(runner.cm_uid,'known-public-cm-uid')
|
||||
self.assertIn('get',runner.last_command);self.assertNotIn('create',runner.last_command)
|
||||
cm['metadata']['uid']='recreated'
|
||||
with self.assertRaisesRegex(RuntimeError,'identity-drift'):runner.existing_cm()
|
||||
def test_mutable_or_changed_cm_data_refused(self):
|
||||
for key in ('mutable','changed-data','extra-data'):
|
||||
cm=self.cm()
|
||||
if key=='mutable':cm['immutable']=False
|
||||
elif key=='changed-data':cm['data']['hook']='changed'
|
||||
else:cm['data']['unexpected']='public-sentinel'
|
||||
with self.subTest(key=key),self.assertRaisesRegex(RuntimeError,'configmap-drift'):self.runner(cm).existing_cm()
|
||||
def test_same_six_source_candidates_still_match(self):
|
||||
s=importlib.util.spec_from_file_location('follow',HERE/'source_followthrough.py');f=importlib.util.module_from_spec(s);s.loader.exec_module(f)
|
||||
self.assertEqual(len(f.candidates()),6)
|
||||
|
||||
|
||||
if __name__=='__main__':unittest.main()
|
||||
|
|
@ -0,0 +1,70 @@
|
|||
"""Run the actual controller migration/stream framing in pinned Forgejo userspace."""
|
||||
from pathlib import Path
|
||||
import importlib.util
|
||||
import json
|
||||
import subprocess
|
||||
|
||||
HERE=Path(__file__).resolve().parent
|
||||
IMAGE='code.forgejo.org/forgejo/forgejo@sha256:691657c7e64af7730a2e885e6d022cf16ae60724931c5392bd61838afadcaeba'
|
||||
spec=importlib.util.spec_from_file_location('r4',HERE/'rollout.py');r=importlib.util.module_from_spec(spec);spec.loader.exec_module(r)
|
||||
m=r.support();C=json.loads((r.ORIGINAL/'inputs.json').read_bytes())
|
||||
|
||||
|
||||
def main():
|
||||
argv=['docker','run','-d','--rm','--network=none','--user=1000:1000','--read-only','--cap-drop=ALL','--security-opt=no-new-privileges:true',
|
||||
'--tmpfs','/tmp:rw,size=64m,uid=1000,gid=1000','--tmpfs','/data:rw,size=64m,uid=1000,gid=1000',
|
||||
'--mount','type=bind,src='+str(r.ORIGINAL)+',dst=/original,readonly','--entrypoint=/bin/sh',IMAGE,'-c','tail -f /dev/null']
|
||||
created=subprocess.run(argv,capture_output=True,timeout=30)
|
||||
if created.returncode:raise RuntimeError('fixture-container-start-failed')
|
||||
cid=created.stdout.decode().strip();events=[];phase='setup'
|
||||
def shell(script,args=(),data=None,timeout=180):
|
||||
q=subprocess.run(['docker','exec','-i',cid,'/bin/sh','-c',script,'sh',*args],input=data,capture_output=True,timeout=timeout)
|
||||
if q.returncode:raise RuntimeError('fixture-command-failed:'+phase)
|
||||
return q.stdout
|
||||
try:
|
||||
script=r'''set -eu
|
||||
n=0
|
||||
while test "$n" -lt 171; do
|
||||
name=$(printf test%03d "$n")
|
||||
repo=/data/git/gitea-repositories/coulomb/$name.git
|
||||
for hook in pre-receive post-receive update pre-receive.d/gitea post-receive.d/gitea update.d/gitea pre-receive.d/20-authorize post-receive.d/20-notify; do
|
||||
path="$repo/hooks/$hook";mkdir -p "${path%/*}"
|
||||
printf 'unchanged generated hook\n' > "$path";chmod 775 "$path"
|
||||
done
|
||||
file="$repo/hooks/pre-receive.d/10-custodian-secret-scan"
|
||||
cp /original/before-hook "$file";chmod 775 "$file"
|
||||
if test "$n" -lt 3; then chmod 755 "$file"; fi
|
||||
if test "$n" -lt 16; then chmod 755 "$repo/hooks/pre-receive"; fi
|
||||
n=$((n+1))
|
||||
done
|
||||
'''
|
||||
shell(script)
|
||||
class TestRunner(r.make_runner(m)):
|
||||
def shell(self,pod,script,args=(),data=None,timeout=90):return shell(script,args,data,max(timeout,180))
|
||||
def event(self,name,**fields):events.append({'event':name,**fields})
|
||||
runner=TestRunner(C);before=runner.inventory('fixture',{C['old_hook_sha256']});runner.receipt['before_hooks']=before
|
||||
phase='initial-migration';runner.migrate('fixture','apply')
|
||||
phase='simulate-fsgroup';shell("find /data -type f -exec chmod g+w {} +")
|
||||
phase='post-recreate-migration';runner.migrate('fixture','apply')
|
||||
assert events[-1]['group_write_additions_reversed']==19
|
||||
phase='simulate-second-fsgroup';shell("find /data -type f -exec chmod g+w {} +")
|
||||
phase='rollback-migration';runner.migrate('fixture','rollback')
|
||||
after=runner.inventory('fixture',{C['old_hook_sha256']});assert after==before
|
||||
# Unknown protected data must stop before any scanner rewrite.
|
||||
phase='protected-drift';shell("printf 'owner content drift\\n' > /data/git/gitea-repositories/coulomb/test000.git/hooks/pre-receive")
|
||||
try:runner.migrate('fixture','apply')
|
||||
except RuntimeError as exc:assert str(exc)=='protected-hook-content-drift'
|
||||
else:raise AssertionError('protected-drift-not-refused')
|
||||
still=runner.inventory('fixture',{C['old_hook_sha256']});assert len(still)==171
|
||||
result={'result':'passed','image':IMAGE,'repositories':171,'hooks':sum(len(x) for x in before.values()),
|
||||
'cases':['initial-migration','post-recreate-mode-restoration','rollback-mode-restoration','protected-drift-before-write'],
|
||||
'group_write_additions_reversed':19,'events':events,'native_mutation':False}
|
||||
(HERE/'controller-image-tests.json').write_text(json.dumps(result,indent=2)+'\n');print(json.dumps(result))
|
||||
except Exception as exc:
|
||||
(HERE/'controller-image-tests-failed.json').write_text(json.dumps({'result':'unverified','phase':phase,'error':type(exc).__name__,'image':IMAGE,'native_mutation':False},indent=2)+'\n')
|
||||
raise
|
||||
finally:
|
||||
subprocess.run(['docker','stop','--time=0',cid],capture_output=True,timeout=30)
|
||||
|
||||
|
||||
if __name__=='__main__':main()
|
||||
|
|
@ -0,0 +1,88 @@
|
|||
#!/bin/sh
|
||||
# Actual pinned Forgejo image, no host data/config mounts or network.
|
||||
set +vx
|
||||
set -eu
|
||||
case=$1
|
||||
area=/tmp/control
|
||||
mkdir -p "$area"
|
||||
cp /original/before-hook "$area/before"
|
||||
cp /original/after-hook "$area/after"
|
||||
before=$(sha256sum "$area/before" | cut -d ' ' -f 1)
|
||||
after=$(sha256sum "$area/after" | cut -d ' ' -f 1)
|
||||
first=/data/git/gitea-repositories/coulomb/first.git/hooks/pre-receive.d/10-custodian-secret-scan
|
||||
second=/data/git/gitea-repositories/coulomb/second.git/hooks/pre-receive.d/10-custodian-secret-scan
|
||||
for path in "$first" "$second"; do
|
||||
mkdir -p "${path%/*}"
|
||||
cp "$area/before" "$path"
|
||||
chmod 755 "$path"
|
||||
printf '%s\n' "$path" >> "$area/paths"
|
||||
printf 'unchanged generated dispatcher\n' > "${path%/pre-receive.d/*}/pre-receive"
|
||||
chmod 755 "${path%/pre-receive.d/*}/pre-receive"
|
||||
done
|
||||
chmod 775 "$second"
|
||||
protected=${first%/pre-receive.d/*}/pre-receive
|
||||
protected_sum=$(sha256sum "$protected" | cut -d ' ' -f 1)
|
||||
run_hooks() { sh /scripts/hooks.sh "$area" "$1" "$before" "$after"; }
|
||||
if test "$case" = old-counterexample; then
|
||||
if sh /original/hooks.sh "$area" apply "$before" "$after" > /tmp/old.out 2>/tmp/old.err; then exit 80; fi
|
||||
grep -qi 'reference' /tmp/old.err
|
||||
printf '{"case":"old-counterexample","unsupported_reference_reproduced":true}\n'
|
||||
exit 0
|
||||
fi
|
||||
case "$case" in
|
||||
roundtrip)
|
||||
run_hooks apply
|
||||
test "$(stat -c %a "$first")" = 755
|
||||
test "$(stat -c %a "$second")" = 775
|
||||
run_hooks rollback
|
||||
test "$(sha256sum "$first" | cut -d ' ' -f 1)" = "$before"
|
||||
test "$(stat -c %a "$first")" = 755
|
||||
test "$(stat -c %a "$second")" = 775
|
||||
;;
|
||||
complete-scope-refusal)
|
||||
printf 'owner drift\n' > "$second"
|
||||
if run_hooks apply >/dev/null 2>&1; then exit 81; fi
|
||||
test "$(sha256sum "$first" | cut -d ' ' -f 1)" = "$before"
|
||||
;;
|
||||
partial-failure)
|
||||
chmod 555 "${second%/*}"
|
||||
if run_hooks apply >/dev/null 2>&1; then exit 82; fi
|
||||
test "$(sha256sum "$first" | cut -d ' ' -f 1)" = "$before"
|
||||
test "$(stat -c %a "$first")" = 755
|
||||
;;
|
||||
symlink-refusal)
|
||||
rm "$second";ln -s "$first" "$second"
|
||||
if run_hooks apply >/dev/null 2>&1; then exit 83; fi
|
||||
test "$(sha256sum "$first" | cut -d ' ' -f 1)" = "$before"
|
||||
;;
|
||||
fsgroup-modes|mode-drift|hash-drift)
|
||||
run_hooks apply >/dev/null
|
||||
printf '755\t%s\t%s\n' "$after" "$first" > "$area/modes"
|
||||
printf '775\t%s\t%s\n' "$after" "$second" >> "$area/modes"
|
||||
printf '755\t%s\t%s\n' "$protected_sum" "$protected" >> "$area/modes"
|
||||
chmod g+w "$first" "$protected"
|
||||
if test "$case" = mode-drift; then
|
||||
chmod 777 "$second"
|
||||
if sh /scripts/restore_modes.sh "$area/modes" >/dev/null 2>&1; then exit 84; fi
|
||||
test "$(stat -c %a "$first")" = 775
|
||||
elif test "$case" = hash-drift; then
|
||||
printf 'owner content drift\n' > "$protected"
|
||||
if sh /scripts/restore_modes.sh "$area/modes" >/dev/null 2>&1; then exit 85; fi
|
||||
test "$(stat -c %a "$first")" = 775
|
||||
else
|
||||
sh /scripts/restore_modes.sh "$area/modes"
|
||||
test "$(stat -c %a "$first")" = 755
|
||||
test "$(stat -c %a "$second")" = 775
|
||||
test "$(stat -c %a "$protected")" = 755
|
||||
test "$(sha256sum "$protected" | cut -d ' ' -f 1)" = "$protected_sum"
|
||||
run_hooks rollback >/dev/null
|
||||
fi
|
||||
;;
|
||||
*) exit 86;;
|
||||
esac
|
||||
# Hash drift fixture deliberately changes protected content; all others preserve it.
|
||||
if test "$case" != hash-drift; then
|
||||
test "$(sha256sum "$protected" | cut -d ' ' -f 1)" = "$protected_sum"
|
||||
fi
|
||||
test -z "$(find /data -name '.custodian-*' -print -quit)"
|
||||
printf '{"case":"%s","passed":true}\n' "$case"
|
||||
Loading…
Add table
Add a link
Reference in a new issue