Install runtime and test dependencies in exact-source journey CI
Assistant: codex Assistant-Model: gpt-6.1-sol Assistant-Session: 01a126e1-76f8-7643-8cf3-e93336539ade
This commit is contained in:
parent
1bb7cacc98
commit
ac8b04edc7
9 changed files with 130 additions and 5 deletions
|
|
@ -2,13 +2,13 @@ name: Account journey acceptance
|
|||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths: ["src/**", "tests/**", "scripts/run_journey_suites.py", "Makefile", ".forgejo/workflows/journeys.yaml"]
|
||||
paths: ["src/**", "tests/**", "scripts/run_journey_suites.py", "scripts/check_layer_conformance.py", "pyproject.toml", "Containerfile", "Makefile", ".forgejo/workflows/journeys.yaml"]
|
||||
workflow_dispatch:
|
||||
jobs:
|
||||
journeys:
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: python:3.12-slim@sha256:d764629ce0ddd8c71fd371e9901efb324a95789d2315a47db7e4d27e78f1b0e9
|
||||
image: python:3.12-slim@sha256:a6e34c598f2467ed0e9a8d349809fcd8b5c603269512df273a0bb1784edc11b1
|
||||
steps:
|
||||
- name: Test the exact commit
|
||||
run: |
|
||||
|
|
@ -17,7 +17,8 @@ jobs:
|
|||
python -c 'import os, urllib.request; urllib.request.urlretrieve("https://forgejo.coulomb.social/" + os.environ["GITHUB_REPOSITORY"] + "/archive/" + os.environ["GITHUB_SHA"] + ".tar.gz", "journey-source.tar.gz")'
|
||||
tar xzf journey-source.tar.gz -C journey-source --strip-components=1
|
||||
cd journey-source
|
||||
PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=src python -m unittest discover -s tests
|
||||
python -m pip install --no-cache-dir ".[runtime]" "pytest==8.4.2"
|
||||
PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=src python -m pytest -q
|
||||
PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=src python scripts/check_layer_conformance.py --report
|
||||
PYTHONDONTWRITEBYTECODE=1 python scripts/run_journey_suites.py --report journey-report.json
|
||||
cat journey-report.json
|
||||
|
|
|
|||
36
docs/evidence/2026-10-11-journey-ci-repair/REVIEW.md
Normal file
36
docs/evidence/2026-10-11-journey-ci-repair/REVIEW.md
Normal file
|
|
@ -0,0 +1,36 @@
|
|||
# Actual journey CI repair — 2026-10-11
|
||||
|
||||
Exact pushed source `1bb7cacc98479f56f6b1c28c000479efd260a944` has
|
||||
native journey task 11448 (run 307) failed while its image-build and smoke
|
||||
tasks passed. Sanitized actual logs show ImportError/ModuleNotFoundError: the
|
||||
clean Python runner has neither pytest nor jsonschema. The workflow downloaded
|
||||
the exact source but installed no project or test dependencies; it failed
|
||||
before mapped journeys ran. Historical failures do not dispose this failure.
|
||||
|
||||
The source fix installs `.[runtime]` and pinned pytest 8.4.2, then runs the
|
||||
full pytest suite rather than unittest-only discovery. This executes the
|
||||
pytest-style v0.8 pin/provenance regressions as well as unittest cases. The
|
||||
accepted-layer detector and mapped journey runner remain mandatory. Workflow
|
||||
triggers now include dependency, base-image and checker source changes. The
|
||||
runner uses the same reviewed patched immutable official Python 3.12 base.
|
||||
No application package input or published image byte changes are implied.
|
||||
|
||||
Clean-container rehearsal uses the exact original archive and the repaired
|
||||
setup: 317 tests pass and 20 optional integration/absent-sibling-fixture tests
|
||||
skip; accepted v0.8 exact digest/scope checker passes; all 66 mapped journey
|
||||
tests pass with zero skips. The journey report remains complete:false with
|
||||
U02–U09, T02, T04 and T05 unresolved. These automated checks accept no real
|
||||
user login, MFA, workload admission, grant, token lifetime or native consumer.
|
||||
|
||||
Two local rehearsal setup failures are retained separately: a read-only global
|
||||
pip destination and a noexec scratch virtualenv preventing compiled-wheel
|
||||
imports. A writable scratch virtualenv with an explicit executable temporary
|
||||
mount corrects the harness without weakening tests. Actual native CI failure
|
||||
was missing dependencies, not these local harness constraints.
|
||||
|
||||
Native CI for the corrected source remains unverified until the owner commits
|
||||
the workflow, publishes that source and observes its exact new SHA. The
|
||||
currently published image can retain its original source receipt: this fix
|
||||
changes workflow/evidence only, not the 48 runtime build inputs. Release
|
||||
vulnerability disposition and native consumer acceptance stay in the existing
|
||||
owner gates. No approval or deployment authority is asserted here.
|
||||
|
|
@ -0,0 +1 @@
|
|||
{"original_exit": 1, "original_exception_classes": ["ImportError", "ModuleNotFoundError"], "original_missing_pytest": true, "original_missing_jsonschema": true, "dependency_install_exit": 0, "fixed_checks": [{"argv": ["/tmp/ci-venv/bin/python", "-m", "pytest", "-q"], "exit": 0, "exception_classes": [], "failed_collection_modules": [], "summary": ["317 passed", "20 skipped"]}, {"argv": ["/tmp/ci-venv/bin/python", "scripts/check_layer_conformance.py", "--report"], "exit": 0, "exception_classes": [], "accepted_v08_digest_named": true}, {"argv": ["/tmp/ci-venv/bin/python", "scripts/run_journey_suites.py", "--report", "/tmp/journey-report.json"], "exit": 0, "exception_classes": []}], "journey_summary": {"tests_run": 66, "test_success": true, "skipped": 0, "complete": false, "unresolved_journeys": ["U02", "U03", "U04", "U05", "U06", "U07", "U08", "U09", "T02", "T04", "T05"]}}
|
||||
|
|
@ -0,0 +1 @@
|
|||
{"original_exit": 1, "original_exception_classes": ["ImportError", "ModuleNotFoundError"], "original_missing_pytest": true, "original_missing_jsonschema": true, "dependency_install_exit": 0, "fixed_checks": [{"argv": ["/tmp/ci-venv/bin/python", "-m", "pytest", "-q"], "exit": 2, "exception_classes": ["ImportError"], "failed_collection_modules": ["tests/test_custody_adapter.py", "tests/test_custody_bootstrap.py", "tests/test_custody_policy_package.py"], "summary": []}]}
|
||||
|
|
@ -0,0 +1 @@
|
|||
{"original_exit": 1, "original_exception_classes": ["ImportError", "ModuleNotFoundError"], "original_missing_pytest": true, "original_missing_jsonschema": true, "dependency_install_exit": 1, "install_error_classes": ["OSError"], "install_failure_flags": {"Read-only file system": true, "No space left on device": false, "Temporary failure in name resolution": false, "Failed to establish a new connection": false, "No matching distribution found": false, "Permission denied": false, "Invalid requirement": false}}
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
{
|
||||
"head_sha": "1bb7cacc98479f56f6b1c28c000479efd260a944",
|
||||
"task": 11448,
|
||||
"status": "failure",
|
||||
"step": "Test the exact commit",
|
||||
"exception_classes": [
|
||||
"ImportError",
|
||||
"ModuleNotFoundError"
|
||||
],
|
||||
"missing_pytest": true,
|
||||
"missing_jsonschema": true,
|
||||
"unittest_suite_failed": true,
|
||||
"journey_runner_reached": false,
|
||||
"raw_log_retained": false
|
||||
}
|
||||
23
docs/evidence/2026-10-11-journey-ci-repair/receipt.json
Normal file
23
docs/evidence/2026-10-11-journey-ci-repair/receipt.json
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
{
|
||||
"original_source_commit": "1bb7cacc98479f56f6b1c28c000479efd260a944",
|
||||
"original_native_task": 11448,
|
||||
"original_native_run": 307,
|
||||
"native_log_handling": "Fetched in process and retained only fixed failure classes/booleans; no raw CI logs, assertions, tokens or trace payloads stored or emitted.",
|
||||
"source_archive_sha256": "a10641ce1a0ecc11f0686310c23ce39d2fe70787f1eacb559dd0fe8ffc8ddfe1",
|
||||
"workflow_sha256": "c0e9953aca593e6005632e5b634969f919543758af711adec7f0181cd9944fac",
|
||||
"workflow_shell_syntax": "passed",
|
||||
"native_ci_after_fix": "not yet run; root must commit and observe exact new SHA",
|
||||
"release_image_input_change": false,
|
||||
"fix_scope": "Install project runtime dependencies and pinned pytest8.4.2; execute full pytest collection; preserve accepted layer detector and mapped journey gate; trigger on dependency/base/checker changes.",
|
||||
"local_rehearsal_limits": {
|
||||
"memory": "1GiB",
|
||||
"cpu": "1core",
|
||||
"source": "Exact clean git archive copied into ephemeral scratch; no credentials or live target fixtures."
|
||||
},
|
||||
"local_setup_failures": [
|
||||
"First rehearsal tried global pip installation under read-only image filesystem; corrected with scratch virtualenv.",
|
||||
"Scratch virtualenv on default noexec tmpfs could not load compiled dependency wheels; corrected with explicit exec temporary mount, retaining strict full-suite gate."
|
||||
],
|
||||
"external_acceptance_claimed": false,
|
||||
"native_authentication_or_approval_changed": false
|
||||
}
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
import subprocess,sys,os,json,re,shutil
|
||||
from pathlib import Path
|
||||
root=Path("/tmp/journey-source");shutil.copytree("/source",root);os.chdir(root)
|
||||
env=os.environ.copy();env["PYTHONDONTWRITEBYTECODE"]="1";env["PYTHONPATH"]="src"
|
||||
original=subprocess.run([sys.executable,"-m","unittest","discover","-s","tests"],env=env,capture_output=True,text=True)
|
||||
text=original.stdout+original.stderr
|
||||
summary={"original_exit":original.returncode,"original_exception_classes":sorted(set(re.findall(r"\b(?:ModuleNotFoundError|ImportError|AssertionError|RuntimeError|ValueError)\b",text))),"original_missing_pytest":"No module named 'pytest'" in text,"original_missing_jsonschema":"No module named 'jsonschema'" in text}
|
||||
subprocess.run([sys.executable,"-m","venv","/tmp/ci-venv"],check=True,capture_output=True)
|
||||
py="/tmp/ci-venv/bin/python"
|
||||
install=subprocess.run([py,"-m","pip","install","--no-cache-dir",".[runtime]","pytest==8.4.2"],capture_output=True,text=True)
|
||||
summary["dependency_install_exit"]=install.returncode
|
||||
if install.returncode:
|
||||
summary["install_error_classes"]=sorted(set(re.findall(r"\b(?:OSError|PermissionError|RuntimeError|ConnectionError|SSLError|ReadTimeoutError|ProxyError)\b",install.stdout+install.stderr)))
|
||||
summary["install_failure_flags"]={v:(v in install.stdout+install.stderr) for v in ["Read-only file system","No space left on device","Temporary failure in name resolution","Failed to establish a new connection","No matching distribution found","Permission denied","Invalid requirement"]}
|
||||
print(json.dumps(summary));raise SystemExit(install.returncode)
|
||||
checks=[]
|
||||
for argv in [[py,"-m","pytest","-q"],[py,"scripts/check_layer_conformance.py","--report"],[py,"scripts/run_journey_suites.py","--report","/tmp/journey-report.json"]]:
|
||||
r=subprocess.run(argv,env=env,capture_output=True,text=True);combined=r.stdout+r.stderr
|
||||
check={"argv":argv,"exit":r.returncode,"exception_classes":sorted(set(re.findall(r"\b(?:ModuleNotFoundError|ImportError|AssertionError|RuntimeError|ValueError)\b",combined)))}
|
||||
if "pytest" in argv:check["failed_collection_modules"]=sorted(set(re.findall(r"(?:ERROR collecting |ERROR )(tests/[A-Za-z0-9_/.]+\.py)",combined)));check["summary"]=re.findall(r"(?:\d+ passed|\d+ skipped|\d+ failed|\d+ warnings?)",combined[-1000:])
|
||||
if "--report" in argv and "check_layer_conformance.py" in argv[1]:check["accepted_v08_digest_named"]="3d8298560069ef471fba28d2b1245fb2309328f231dad1a667cdf0ce5a5a7c8b" in combined
|
||||
checks.append(check)
|
||||
if r.returncode:break
|
||||
summary["fixed_checks"]=checks
|
||||
if Path("/tmp/journey-report.json").exists():
|
||||
d=json.loads(Path("/tmp/journey-report.json").read_text());summary["journey_summary"]={"tests_run":d["tests_run"],"test_success":d["test_success"],"skipped":d["skipped"],"complete":d["complete"],"unresolved_journeys":[r["id"] for r in d["journeys"] if r["implementation"]!="implemented"]}
|
||||
print(json.dumps(summary));raise SystemExit(any(c["exit"] for c in checks))
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
id: USER-WP-ADHOC-2026-10-11
|
||||
type: workplan
|
||||
title: "Refresh vulnerable readback candidate runtime base"
|
||||
title: "Prepare readback runtime base and journey CI gates"
|
||||
domain: communication
|
||||
repo: user-engine
|
||||
status: finished
|
||||
|
|
@ -10,6 +10,7 @@ owner: codex
|
|||
topic_slug: user-engine
|
||||
created: "2026-10-11"
|
||||
updated: "2026-10-11"
|
||||
state_hub_workstream_id: "1aac634c-f321-5bf6-b178-4544d7854e8e"
|
||||
---
|
||||
|
||||
## Refresh immutable Python patch base and verify exact application preservation
|
||||
|
|
@ -18,13 +19,32 @@ updated: "2026-10-11"
|
|||
id: USER-WP-ADHOC-2026-10-11-T01
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "68979c68-6ab7-5675-9961-ddbb31a4c88d"
|
||||
```
|
||||
|
||||
Replace only the Containerfile base digest with the current official Python 3.12
|
||||
index; retain exact application bytes and resolved Python dependency versions.
|
||||
The bounded local candidate removes all 3 CRITICAL and44 of 88 HIGH OS findings.
|
||||
The bounded local candidate removes all 3 CRITICAL and 44 of 88 HIGH OS findings.
|
||||
Retain remaining 44 unfixed HIGH findings in the unfiltered fresh scan. Exact
|
||||
source/image/provenance checks and 20 image / 323 source tests pass. Evidence:
|
||||
`docs/evidence/2026-10-11-runtime-base-refresh.md`. This source remediation
|
||||
finishes no deployment: USER-WP-0026-T03 and USER-WP-0028-T02 retain native
|
||||
consumer acceptance, clean source commit/publication and scoped rollout gates.
|
||||
|
||||
## Repair actual missing-dependency journey CI setup
|
||||
|
||||
```task
|
||||
id: USER-WP-ADHOC-2026-10-11-T02
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Correct native failed task 11448 for exact source 1bb7cacc: install project
|
||||
runtime dependencies and pinned pytest, run the full pytest suite including
|
||||
v0.8 regressions, preserve layer and mapped-journey gates. A clean-container
|
||||
rehearsal passes 317 tests with 20 optional skips and all 66 mapped journey tests
|
||||
with no skips; external journey acceptance remains incomplete. Evidence:
|
||||
`docs/evidence/2026-10-11-journey-ci-repair/REVIEW.md`. This task completes
|
||||
source repair and bounded verification; exact new native CI execution, release
|
||||
risk disposition and real workload acceptance remain USER-WP-0026-T03 /
|
||||
USER-WP-0028-T02. No runtime build input, live policy or image is changed.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue