diff --git a/.forgejo/workflows/journeys.yaml b/.forgejo/workflows/journeys.yaml index 0367f9b..54dc4be 100644 --- a/.forgejo/workflows/journeys.yaml +++ b/.forgejo/workflows/journeys.yaml @@ -2,13 +2,13 @@ name: Account journey acceptance on: push: branches: [main] - paths: ["src/**", "tests/**", "scripts/run_journey_suites.py", "Makefile", ".forgejo/workflows/journeys.yaml"] + paths: ["src/**", "tests/**", "scripts/run_journey_suites.py", "scripts/check_layer_conformance.py", "pyproject.toml", "Containerfile", "Makefile", ".forgejo/workflows/journeys.yaml"] workflow_dispatch: jobs: journeys: runs-on: ubuntu-latest container: - image: python:3.12-slim@sha256:d764629ce0ddd8c71fd371e9901efb324a95789d2315a47db7e4d27e78f1b0e9 + image: python:3.12-slim@sha256:a6e34c598f2467ed0e9a8d349809fcd8b5c603269512df273a0bb1784edc11b1 steps: - name: Test the exact commit run: | @@ -17,7 +17,8 @@ jobs: python -c 'import os, urllib.request; urllib.request.urlretrieve("https://forgejo.coulomb.social/" + os.environ["GITHUB_REPOSITORY"] + "/archive/" + os.environ["GITHUB_SHA"] + ".tar.gz", "journey-source.tar.gz")' tar xzf journey-source.tar.gz -C journey-source --strip-components=1 cd journey-source - PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=src python -m unittest discover -s tests + python -m pip install --no-cache-dir ".[runtime]" "pytest==8.4.2" + PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=src python -m pytest -q PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=src python scripts/check_layer_conformance.py --report PYTHONDONTWRITEBYTECODE=1 python scripts/run_journey_suites.py --report journey-report.json cat journey-report.json diff --git a/docs/evidence/2026-10-11-journey-ci-repair/REVIEW.md b/docs/evidence/2026-10-11-journey-ci-repair/REVIEW.md new file mode 100644 index 0000000..86860eb --- /dev/null +++ b/docs/evidence/2026-10-11-journey-ci-repair/REVIEW.md @@ -0,0 +1,36 @@ +# Actual journey CI repair — 2026-10-11 + +Exact pushed source `1bb7cacc98479f56f6b1c28c000479efd260a944` has +native journey task 11448 (run 307) failed while its image-build and smoke +tasks passed. Sanitized actual logs show ImportError/ModuleNotFoundError: the +clean Python runner has neither pytest nor jsonschema. The workflow downloaded +the exact source but installed no project or test dependencies; it failed +before mapped journeys ran. Historical failures do not dispose this failure. + +The source fix installs `.[runtime]` and pinned pytest 8.4.2, then runs the +full pytest suite rather than unittest-only discovery. This executes the +pytest-style v0.8 pin/provenance regressions as well as unittest cases. The +accepted-layer detector and mapped journey runner remain mandatory. Workflow +triggers now include dependency, base-image and checker source changes. The +runner uses the same reviewed patched immutable official Python 3.12 base. +No application package input or published image byte changes are implied. + +Clean-container rehearsal uses the exact original archive and the repaired +setup: 317 tests pass and 20 optional integration/absent-sibling-fixture tests +skip; accepted v0.8 exact digest/scope checker passes; all 66 mapped journey +tests pass with zero skips. The journey report remains complete:false with +U02–U09, T02, T04 and T05 unresolved. These automated checks accept no real +user login, MFA, workload admission, grant, token lifetime or native consumer. + +Two local rehearsal setup failures are retained separately: a read-only global +pip destination and a noexec scratch virtualenv preventing compiled-wheel +imports. A writable scratch virtualenv with an explicit executable temporary +mount corrects the harness without weakening tests. Actual native CI failure +was missing dependencies, not these local harness constraints. + +Native CI for the corrected source remains unverified until the owner commits +the workflow, publishes that source and observes its exact new SHA. The +currently published image can retain its original source receipt: this fix +changes workflow/evidence only, not the 48 runtime build inputs. Release +vulnerability disposition and native consumer acceptance stay in the existing +owner gates. No approval or deployment authority is asserted here. diff --git a/docs/evidence/2026-10-11-journey-ci-repair/clean-container-proof.json b/docs/evidence/2026-10-11-journey-ci-repair/clean-container-proof.json new file mode 100644 index 0000000..dfa9656 --- /dev/null +++ b/docs/evidence/2026-10-11-journey-ci-repair/clean-container-proof.json @@ -0,0 +1 @@ +{"original_exit": 1, "original_exception_classes": ["ImportError", "ModuleNotFoundError"], "original_missing_pytest": true, "original_missing_jsonschema": true, "dependency_install_exit": 0, "fixed_checks": [{"argv": ["/tmp/ci-venv/bin/python", "-m", "pytest", "-q"], "exit": 0, "exception_classes": [], "failed_collection_modules": [], "summary": ["317 passed", "20 skipped"]}, {"argv": ["/tmp/ci-venv/bin/python", "scripts/check_layer_conformance.py", "--report"], "exit": 0, "exception_classes": [], "accepted_v08_digest_named": true}, {"argv": ["/tmp/ci-venv/bin/python", "scripts/run_journey_suites.py", "--report", "/tmp/journey-report.json"], "exit": 0, "exception_classes": []}], "journey_summary": {"tests_run": 66, "test_success": true, "skipped": 0, "complete": false, "unresolved_journeys": ["U02", "U03", "U04", "U05", "U06", "U07", "U08", "U09", "T02", "T04", "T05"]}} diff --git a/docs/evidence/2026-10-11-journey-ci-repair/initial-noexec-harness-failure.json b/docs/evidence/2026-10-11-journey-ci-repair/initial-noexec-harness-failure.json new file mode 100644 index 0000000..b685f8b --- /dev/null +++ b/docs/evidence/2026-10-11-journey-ci-repair/initial-noexec-harness-failure.json @@ -0,0 +1 @@ +{"original_exit": 1, "original_exception_classes": ["ImportError", "ModuleNotFoundError"], "original_missing_pytest": true, "original_missing_jsonschema": true, "dependency_install_exit": 0, "fixed_checks": [{"argv": ["/tmp/ci-venv/bin/python", "-m", "pytest", "-q"], "exit": 2, "exception_classes": ["ImportError"], "failed_collection_modules": ["tests/test_custody_adapter.py", "tests/test_custody_bootstrap.py", "tests/test_custody_policy_package.py"], "summary": []}]} diff --git a/docs/evidence/2026-10-11-journey-ci-repair/initial-readonly-harness-failure.json b/docs/evidence/2026-10-11-journey-ci-repair/initial-readonly-harness-failure.json new file mode 100644 index 0000000..2a5123c --- /dev/null +++ b/docs/evidence/2026-10-11-journey-ci-repair/initial-readonly-harness-failure.json @@ -0,0 +1 @@ +{"original_exit": 1, "original_exception_classes": ["ImportError", "ModuleNotFoundError"], "original_missing_pytest": true, "original_missing_jsonschema": true, "dependency_install_exit": 1, "install_error_classes": ["OSError"], "install_failure_flags": {"Read-only file system": true, "No space left on device": false, "Temporary failure in name resolution": false, "Failed to establish a new connection": false, "No matching distribution found": false, "Permission denied": false, "Invalid requirement": false}} diff --git a/docs/evidence/2026-10-11-journey-ci-repair/native-failure.json b/docs/evidence/2026-10-11-journey-ci-repair/native-failure.json new file mode 100644 index 0000000..7dc3a0d --- /dev/null +++ b/docs/evidence/2026-10-11-journey-ci-repair/native-failure.json @@ -0,0 +1,15 @@ +{ + "head_sha": "1bb7cacc98479f56f6b1c28c000479efd260a944", + "task": 11448, + "status": "failure", + "step": "Test the exact commit", + "exception_classes": [ + "ImportError", + "ModuleNotFoundError" + ], + "missing_pytest": true, + "missing_jsonschema": true, + "unittest_suite_failed": true, + "journey_runner_reached": false, + "raw_log_retained": false +} diff --git a/docs/evidence/2026-10-11-journey-ci-repair/receipt.json b/docs/evidence/2026-10-11-journey-ci-repair/receipt.json new file mode 100644 index 0000000..ea68aa0 --- /dev/null +++ b/docs/evidence/2026-10-11-journey-ci-repair/receipt.json @@ -0,0 +1,23 @@ +{ + "original_source_commit": "1bb7cacc98479f56f6b1c28c000479efd260a944", + "original_native_task": 11448, + "original_native_run": 307, + "native_log_handling": "Fetched in process and retained only fixed failure classes/booleans; no raw CI logs, assertions, tokens or trace payloads stored or emitted.", + "source_archive_sha256": "a10641ce1a0ecc11f0686310c23ce39d2fe70787f1eacb559dd0fe8ffc8ddfe1", + "workflow_sha256": "c0e9953aca593e6005632e5b634969f919543758af711adec7f0181cd9944fac", + "workflow_shell_syntax": "passed", + "native_ci_after_fix": "not yet run; root must commit and observe exact new SHA", + "release_image_input_change": false, + "fix_scope": "Install project runtime dependencies and pinned pytest8.4.2; execute full pytest collection; preserve accepted layer detector and mapped journey gate; trigger on dependency/base/checker changes.", + "local_rehearsal_limits": { + "memory": "1GiB", + "cpu": "1core", + "source": "Exact clean git archive copied into ephemeral scratch; no credentials or live target fixtures." + }, + "local_setup_failures": [ + "First rehearsal tried global pip installation under read-only image filesystem; corrected with scratch virtualenv.", + "Scratch virtualenv on default noexec tmpfs could not load compiled dependency wheels; corrected with explicit exec temporary mount, retaining strict full-suite gate." + ], + "external_acceptance_claimed": false, + "native_authentication_or_approval_changed": false +} diff --git a/docs/evidence/2026-10-11-journey-ci-repair/verify-clean-container.py b/docs/evidence/2026-10-11-journey-ci-repair/verify-clean-container.py new file mode 100644 index 0000000..e8c4a03 --- /dev/null +++ b/docs/evidence/2026-10-11-journey-ci-repair/verify-clean-container.py @@ -0,0 +1,27 @@ +import subprocess,sys,os,json,re,shutil +from pathlib import Path +root=Path("/tmp/journey-source");shutil.copytree("/source",root);os.chdir(root) +env=os.environ.copy();env["PYTHONDONTWRITEBYTECODE"]="1";env["PYTHONPATH"]="src" +original=subprocess.run([sys.executable,"-m","unittest","discover","-s","tests"],env=env,capture_output=True,text=True) +text=original.stdout+original.stderr +summary={"original_exit":original.returncode,"original_exception_classes":sorted(set(re.findall(r"\b(?:ModuleNotFoundError|ImportError|AssertionError|RuntimeError|ValueError)\b",text))),"original_missing_pytest":"No module named 'pytest'" in text,"original_missing_jsonschema":"No module named 'jsonschema'" in text} +subprocess.run([sys.executable,"-m","venv","/tmp/ci-venv"],check=True,capture_output=True) +py="/tmp/ci-venv/bin/python" +install=subprocess.run([py,"-m","pip","install","--no-cache-dir",".[runtime]","pytest==8.4.2"],capture_output=True,text=True) +summary["dependency_install_exit"]=install.returncode +if install.returncode: + summary["install_error_classes"]=sorted(set(re.findall(r"\b(?:OSError|PermissionError|RuntimeError|ConnectionError|SSLError|ReadTimeoutError|ProxyError)\b",install.stdout+install.stderr))) + summary["install_failure_flags"]={v:(v in install.stdout+install.stderr) for v in ["Read-only file system","No space left on device","Temporary failure in name resolution","Failed to establish a new connection","No matching distribution found","Permission denied","Invalid requirement"]} + print(json.dumps(summary));raise SystemExit(install.returncode) +checks=[] +for argv in [[py,"-m","pytest","-q"],[py,"scripts/check_layer_conformance.py","--report"],[py,"scripts/run_journey_suites.py","--report","/tmp/journey-report.json"]]: + r=subprocess.run(argv,env=env,capture_output=True,text=True);combined=r.stdout+r.stderr + check={"argv":argv,"exit":r.returncode,"exception_classes":sorted(set(re.findall(r"\b(?:ModuleNotFoundError|ImportError|AssertionError|RuntimeError|ValueError)\b",combined)))} + if "pytest" in argv:check["failed_collection_modules"]=sorted(set(re.findall(r"(?:ERROR collecting |ERROR )(tests/[A-Za-z0-9_/.]+\.py)",combined)));check["summary"]=re.findall(r"(?:\d+ passed|\d+ skipped|\d+ failed|\d+ warnings?)",combined[-1000:]) + if "--report" in argv and "check_layer_conformance.py" in argv[1]:check["accepted_v08_digest_named"]="3d8298560069ef471fba28d2b1245fb2309328f231dad1a667cdf0ce5a5a7c8b" in combined + checks.append(check) + if r.returncode:break +summary["fixed_checks"]=checks +if Path("/tmp/journey-report.json").exists(): + d=json.loads(Path("/tmp/journey-report.json").read_text());summary["journey_summary"]={"tests_run":d["tests_run"],"test_success":d["test_success"],"skipped":d["skipped"],"complete":d["complete"],"unresolved_journeys":[r["id"] for r in d["journeys"] if r["implementation"]!="implemented"]} +print(json.dumps(summary));raise SystemExit(any(c["exit"] for c in checks)) diff --git a/workplans/ADHOC-2026-10-11.md b/workplans/ADHOC-2026-10-11.md index 365dc17..c388def 100644 --- a/workplans/ADHOC-2026-10-11.md +++ b/workplans/ADHOC-2026-10-11.md @@ -1,7 +1,7 @@ --- id: USER-WP-ADHOC-2026-10-11 type: workplan -title: "Refresh vulnerable readback candidate runtime base" +title: "Prepare readback runtime base and journey CI gates" domain: communication repo: user-engine status: finished @@ -10,6 +10,7 @@ owner: codex topic_slug: user-engine created: "2026-10-11" updated: "2026-10-11" +state_hub_workstream_id: "1aac634c-f321-5bf6-b178-4544d7854e8e" --- ## Refresh immutable Python patch base and verify exact application preservation @@ -18,13 +19,32 @@ updated: "2026-10-11" id: USER-WP-ADHOC-2026-10-11-T01 status: done priority: high +state_hub_task_id: "68979c68-6ab7-5675-9961-ddbb31a4c88d" ``` Replace only the Containerfile base digest with the current official Python 3.12 index; retain exact application bytes and resolved Python dependency versions. -The bounded local candidate removes all 3 CRITICAL and44 of 88 HIGH OS findings. +The bounded local candidate removes all 3 CRITICAL and 44 of 88 HIGH OS findings. Retain remaining 44 unfixed HIGH findings in the unfiltered fresh scan. Exact source/image/provenance checks and 20 image / 323 source tests pass. Evidence: `docs/evidence/2026-10-11-runtime-base-refresh.md`. This source remediation finishes no deployment: USER-WP-0026-T03 and USER-WP-0028-T02 retain native consumer acceptance, clean source commit/publication and scoped rollout gates. + +## Repair actual missing-dependency journey CI setup + +```task +id: USER-WP-ADHOC-2026-10-11-T02 +status: done +priority: high +``` + +Correct native failed task 11448 for exact source 1bb7cacc: install project +runtime dependencies and pinned pytest, run the full pytest suite including +v0.8 regressions, preserve layer and mapped-journey gates. A clean-container +rehearsal passes 317 tests with 20 optional skips and all 66 mapped journey tests +with no skips; external journey acceptance remains incomplete. Evidence: +`docs/evidence/2026-10-11-journey-ci-repair/REVIEW.md`. This task completes +source repair and bounded verification; exact new native CI execution, release +risk disposition and real workload acceptance remain USER-WP-0026-T03 / +USER-WP-0028-T02. No runtime build input, live policy or image is changed.