test: isolate ordinary unit runs from ambient credentials and provider I/O
Assistant: codex Assistant-Model: gpt-6.1-sol Assistant-Session: 01a1468b-63fa-7710-85dc-8f2e16b3ebfa
This commit is contained in:
parent
922c26da0f
commit
b593c87983
14 changed files with 592 additions and 6 deletions
3
Makefile
3
Makefile
|
|
@ -1,5 +1,6 @@
|
|||
# ops-warden development and install targets
|
||||
ROOT := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
|
||||
UNIT_PYTHON ?= $(ROOT)/.venv/bin/python
|
||||
PHASE_MEMORY_REPO ?= $(abspath $(ROOT)/../phase-memory)
|
||||
|
||||
.DEFAULT_GOAL := help
|
||||
|
|
@ -43,7 +44,7 @@ verify-memory: check-memory
|
|||
@warden memory status
|
||||
|
||||
test:
|
||||
PYTHONPATH=src:$(PHASE_MEMORY_REPO)/src uv run pytest
|
||||
"$(UNIT_PYTHON)" -I scripts/run_unit_tests.py --phase-memory-root "$(PHASE_MEMORY_REPO)" $(TEST_ARGS)
|
||||
|
||||
lint:
|
||||
uv run ruff check .
|
||||
|
|
|
|||
64
docs/development/safe-unit-tests.md
Normal file
64
docs/development/safe-unit-tests.md
Normal file
|
|
@ -0,0 +1,64 @@
|
|||
# Safe unit-test entry point
|
||||
|
||||
Use `make test`, or `.venv/bin/python -I scripts/run_unit_tests.py`. Dependencies
|
||||
must already be installed; this path does not run uv, synchronize or download.
|
||||
`UNIT_PYTHON` may select another existing interpreter. `PHASE_MEMORY_REPO` remains
|
||||
an explicit local source option. Pass pytest options with `TEST_ARGS`, for example
|
||||
`make test TEST_ARGS='-q tests/test_safe_unit_runner.py'`.
|
||||
|
||||
The launcher creates an empty temporary HOME and temporary fixture root and starts
|
||||
an isolated (`-I`) Python interpreter with a constructed environment. Caller token,
|
||||
proxy, Python, pytest, XDG and Warden configuration variables are never copied or
|
||||
inventoried. Third-party pytest plugin autoload is disabled. Tests can explicitly
|
||||
introduce synthetic variables with monkeypatch, but allowed children always get
|
||||
the original temporary HOME and a fresh environment, never those variables.
|
||||
|
||||
Before pytest import/collection, a Python audit hook and subprocess wrapper:
|
||||
|
||||
- deny file opens outside the selected source roots, installed interpreter/library
|
||||
roots and fixture root; writes through open are limited to the fixture root;
|
||||
- deny sockets, DNS and process creation except the closed fixture inventory;
|
||||
- refuse shell, executable, preexec and inherited-FD overrides outside that inventory;
|
||||
- install the same file/socket/process denial before executing every allowed
|
||||
Python child, including BrowserInputs' exact validator code and reverified copied
|
||||
module hashes. Validator children cannot start further processes.
|
||||
|
||||
The process inventory in `scripts/run_unit_tests.py` is intentionally small:
|
||||
BrowserInputs validation, four metadata-checker scripts with fixed argument shapes,
|
||||
the exact original-intent restart probe, the synthetic containment print, and
|
||||
closed guard-regression probes. Metadata checkers receive only fixture output
|
||||
paths. Only the generated-artifact checker child can run the exact read-only Git
|
||||
log/status queries for this repository's catalog, with global/system config and
|
||||
optional locks disabled, hooks disabled and fsmonitor disabled. Those local Git
|
||||
commands are not provider operations.
|
||||
|
||||
Two reviewed non-Python cases remain: exact-byte, temporary contract-double scripts
|
||||
with a closed argv list, and five exact fixed `printf` commands used by existing
|
||||
transport tests. They run Bash without profile/rc startup and without inherited
|
||||
environment; no arbitrary shell text, provider binary, SSH, kubectl or OpenBao
|
||||
command is permitted. Default unit selection excludes four integration-marked
|
||||
SSH-keygen tests. Selecting them explicitly does not grant their native commands.
|
||||
|
||||
Only `test_desk.py::test_approve_flow_http_dry` receives a network lease. It may
|
||||
bind `127.0.0.1:0` and connect to the endpoint actually allocated to that test's
|
||||
socket. DNS is replaced by numeric fixture resolution. Another loopback service
|
||||
(e.g. the real Hub on port 8000), Unix sockets and remote endpoints stay denied.
|
||||
The lease and registered endpoints are cleared after the test.
|
||||
|
||||
Coverage tests now use the explicit public declaration projection snapshot in
|
||||
`tests/fixtures/workload-coverage`, with provenance hashes. They compare the
|
||||
published source snapshot rather than silently consulting the workstation HOME.
|
||||
Run the existing coverage report separately with an explicit estate root when
|
||||
measuring current estate state. Tests of the acknowledged NetKingdom source use
|
||||
`--netkingdom-root` (default: sibling source checkout), never a HOME symlink. The
|
||||
optional Phase Memory library's one existing public coordination-profile fixture
|
||||
is explicitly readable alongside its source; no runtime memory store is inherited.
|
||||
|
||||
This is an accident guard for trusted unit tests, not a security sandbox for
|
||||
hostile Python, native extensions or hostile installed dependencies. It does not
|
||||
revoke already-open descriptors, prevent every filesystem mutation syscall, or
|
||||
provide OS process/network isolation. Fixed Bash/Git fixture processes rely on
|
||||
the reviewed executable and source/argument shapes; Python audit hooks do not run
|
||||
inside those binaries. Do not run untrusted test code under a privileged runtime.
|
||||
Direct `pytest` bypasses this launcher and is not equivalent validation. No
|
||||
production Warden boundary, grant, catalog, source pin or owner credential changes.
|
||||
283
scripts/run_unit_tests.py
Normal file
283
scripts/run_unit_tests.py
Normal file
|
|
@ -0,0 +1,283 @@
|
|||
"""Run trusted unit tests with no inherited credentials or external I/O.
|
||||
|
||||
This is an accident guard for this suite, not an OS sandbox for hostile Python.
|
||||
Direct pytest bypasses it. Use make test (already installed dependencies only).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import runpy
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
SELF = Path(__file__).resolve()
|
||||
SHELL_FIXTURES = {
|
||||
"printf 'the-secret-value'", "printf 'sekret'", "printf 'new'",
|
||||
"printf '%s' '{\"wrap_info\":{\"token\":\"hvs.WRAP\"}}'",
|
||||
"printf 'not-json'",
|
||||
}
|
||||
JOURNAL_CODE_SHA256 = '69aa18f302fd698104cb2253b4af0c3a077e72a0a427183e3dce1444c4926df9'
|
||||
|
||||
|
||||
class IsolationDenied(PermissionError, subprocess.SubprocessError):
|
||||
"""A unit test attempted an unreviewed external operation."""
|
||||
|
||||
|
||||
def clean_environment(home: Path, temporary: Path) -> dict[str, str]:
|
||||
# Construct, never copy/inventory the caller's environment.
|
||||
return {'HOME': str(home), 'TMPDIR': str(temporary), 'PATH': '/usr/bin:/bin',
|
||||
'LANG': 'C.UTF-8', 'PYTHONDONTWRITEBYTECODE': '1',
|
||||
'PYTEST_DISABLE_PLUGIN_AUTOLOAD': '1'}
|
||||
|
||||
|
||||
def inside(path: Path, roots: tuple[Path, ...]) -> bool:
|
||||
return any(path == root or root in path.parents for root in roots)
|
||||
|
||||
|
||||
class Guard:
|
||||
def __init__(self, temporary: Path, read_roots: list[Path], child: bool = False, git_metadata: bool = False):
|
||||
self.temporary = temporary.resolve()
|
||||
self.read_roots = tuple(p.resolve() for p in read_roots)
|
||||
self.child = child
|
||||
self.git_metadata = git_metadata
|
||||
self.loopback_test = False
|
||||
self.endpoints: set[tuple[str, int]] = set()
|
||||
self.launch = None
|
||||
self.original_popen = subprocess.Popen
|
||||
|
||||
def fixture_path(self, value: str) -> Path:
|
||||
path = Path(value).resolve()
|
||||
if not inside(path, (self.temporary,)):
|
||||
raise IsolationDenied('unit isolation: non-fixture path')
|
||||
return path
|
||||
|
||||
def audit(self, event, args):
|
||||
if event == 'open' and not isinstance(args[0], int):
|
||||
path = Path(os.fsdecode(args[0])).resolve()
|
||||
mode, flags = args[1:3]
|
||||
write = ((isinstance(mode, str) and any(c in mode for c in 'wax+'))
|
||||
or (isinstance(flags, int) and flags & (os.O_WRONLY | os.O_RDWR | os.O_CREAT | os.O_TRUNC | os.O_APPEND)))
|
||||
roots = (self.temporary,) if write else self.read_roots
|
||||
if path != Path('/dev/null') and not inside(path, roots):
|
||||
raise IsolationDenied('unit isolation: file outside reviewed roots')
|
||||
if event == 'subprocess.Popen':
|
||||
if self.launch is None or tuple(args[1]) != self.launch:
|
||||
raise IsolationDenied('unit isolation: unexpected process')
|
||||
if event in {'os.system', 'os.exec', 'os.posix_spawn', 'os.fork', 'os.forkpty'}:
|
||||
raise IsolationDenied('unit isolation: native execution disabled')
|
||||
if event.startswith('socket.'):
|
||||
if not self.loopback_test or self.child:
|
||||
raise IsolationDenied('unit isolation: sockets disabled')
|
||||
if event == 'socket.__new__':
|
||||
if args[1] != socket.AF_INET or args[2] & 15 != socket.SOCK_STREAM:
|
||||
raise IsolationDenied('unit isolation: socket family/type denied')
|
||||
elif event == 'socket.bind':
|
||||
if args[1] != ('127.0.0.1', 0):
|
||||
raise IsolationDenied('unit isolation: only ephemeral fixture bind')
|
||||
elif event == 'socket.connect':
|
||||
if args[1] not in self.endpoints:
|
||||
raise IsolationDenied('unit isolation: only this fixture endpoint')
|
||||
elif event not in {'socket.gethostname'}:
|
||||
raise IsolationDenied('unit isolation: DNS/socket operation denied')
|
||||
|
||||
def python_payload(self, argv):
|
||||
"""Closed subprocess inventory; every allowed Python child installs guard."""
|
||||
from warden import case_inputs
|
||||
if (len(argv) == 5 and argv[:3] == [sys.executable, '-I', '-c']
|
||||
and argv[3] == case_inputs._VALIDATE):
|
||||
directory = self.fixture_path(argv[4])
|
||||
if not directory.name.startswith('warden-case-validator-'):
|
||||
raise IsolationDenied('unit isolation: validator fixture directory differs')
|
||||
# Reverify copied modules immediately before launch, independently of caller.
|
||||
for relative, digest in case_inputs.SOURCE_PINS.items():
|
||||
if hashlib.sha256((directory / relative).read_bytes()).hexdigest() != digest:
|
||||
raise IsolationDenied('unit isolation: validator source pin differs')
|
||||
return {'code': argv[3], 'args': [str(directory)]}
|
||||
if len(argv) == 3 and argv[:2] == [sys.executable, '-c'] and argv[2] == 'print("SYNTHETIC-SECRET")':
|
||||
return {'code': argv[2], 'args': []}
|
||||
if (len(argv) == 6 and argv[:3] == [sys.executable, '-B', '-c']
|
||||
and hashlib.sha256(argv[3].encode()).hexdigest() == JOURNAL_CODE_SHA256):
|
||||
self.fixture_path(argv[4])
|
||||
if len(argv[5]) > 192:
|
||||
raise IsolationDenied('unit isolation: journal source bound')
|
||||
return {'code': argv[3], 'args': argv[4:]}
|
||||
if len(argv) >= 2 and argv[0] in (sys.executable, 'python3'):
|
||||
script = Path(argv[1]).resolve()
|
||||
args = argv[2:]
|
||||
if script == ROOT / 'tests/unit_isolation_probe.py' and len(args) == 1 and args[0] in {'state', 'socket', 'provider', 'home', 'nested'}:
|
||||
return {'script': str(script), 'args': args}
|
||||
if script == ROOT / 'scripts/check_layer_conformance.py' and not args:
|
||||
return {'script': str(script), 'args': args}
|
||||
if script == ROOT / 'scripts/emit_high_risk_paths.py' and args == ['--check']:
|
||||
return {'script': str(script), 'args': args, 'git_metadata': True}
|
||||
if script == ROOT / 'scripts/check_principals_drift.py' and len(args) == 4 and args[::2] == ['--inventory', '--infra']:
|
||||
for value in args[1::2]:
|
||||
self.fixture_path(value)
|
||||
return {'script': str(script), 'args': args}
|
||||
if script == ROOT / 'scripts/build_flex_auth_registry.py':
|
||||
if (len(args) in (3, 5) and Path(args[0]).resolve() == ROOT / 'examples/inventory.seed.yaml'
|
||||
and args[-2] == '-o' and (len(args) == 3 or args[1] == '--zone-resolutions')):
|
||||
self.fixture_path(args[-1])
|
||||
if len(args) == 5:
|
||||
self.fixture_path(args[2])
|
||||
return {'script': str(script), 'args': args}
|
||||
raise IsolationDenied('unit isolation: command not in reviewed subprocess inventory')
|
||||
|
||||
def popen(self, argv, *pos, **kw):
|
||||
if pos or kw.get('executable') or kw.get('preexec_fn') or kw.get('pass_fds') or kw.get('close_fds') is False:
|
||||
raise IsolationDenied('unit isolation: unexpected process options')
|
||||
if kw.pop('shell', False):
|
||||
if self.child or not isinstance(argv, str) or argv not in SHELL_FIXTURES:
|
||||
raise IsolationDenied('unit isolation: unreviewed shell command')
|
||||
return self.spawn(['/bin/bash', '--noprofile', '--norc', '-c', argv], kw)
|
||||
if not isinstance(argv, (list, tuple)):
|
||||
raise IsolationDenied('unit isolation: argv list required')
|
||||
argv = [os.fspath(x) for x in argv]
|
||||
# Never run a real provider: the only shell fixture is an exact-byte double.
|
||||
from warden.doubles import _DOUBLES
|
||||
metadata_commands = (['git', 'log', '-1', '--format=%H %cI', '--', str(ROOT / 'registry/routing/catalog.yaml')],
|
||||
['git', 'status', '--porcelain', '--', str(ROOT / 'registry/routing/catalog.yaml')])
|
||||
if self.child and not (self.git_metadata and argv in metadata_commands):
|
||||
raise IsolationDenied('unit isolation: nested process disabled')
|
||||
candidate = Path(argv[0])
|
||||
if not candidate.is_absolute():
|
||||
import shutil
|
||||
resolved = shutil.which(argv[0], path=kw.get('env', os.environ).get('PATH', ''))
|
||||
candidate = Path(resolved) if resolved else candidate
|
||||
if self.child and self.git_metadata:
|
||||
if Path(kw.get('cwd', '.')).resolve() != ROOT:
|
||||
raise IsolationDenied('unit isolation: metadata cwd differs')
|
||||
command = ['/usr/bin/git', '-c', 'core.fsmonitor=false', '-c', 'core.hooksPath=/dev/null', '--no-pager', *argv[1:]]
|
||||
elif candidate.is_absolute() and inside(candidate.resolve(), (self.temporary,)) and candidate.name in _DOUBLES:
|
||||
if candidate.read_text() != _DOUBLES[candidate.name].script:
|
||||
raise IsolationDenied('unit isolation: double bytes differ')
|
||||
tail = argv[1:]
|
||||
if (candidate.name == 'bao' and tail not in (
|
||||
['kv', 'get', '-field=NPM_AUTH_TOKEN', 'platform/x/y'],
|
||||
['kv', 'get', '-field=API_KEY', 'platform/x/y/z'],
|
||||
['login', '-method=oidc'], ['write', 'secret/x'])) or (candidate.name == 'key-cape' and tail != ['login']):
|
||||
raise IsolationDenied('unit isolation: double arguments differ')
|
||||
command = ['/bin/bash', '--noprofile', '--norc', str(candidate), *tail]
|
||||
else:
|
||||
payload = self.python_payload(argv)
|
||||
payload.update(temporary=str(self.temporary), read_roots=[str(x) for x in self.read_roots])
|
||||
command = [sys.executable, '-I', str(SELF), '--child', json.dumps(payload)]
|
||||
return self.spawn(command, kw)
|
||||
|
||||
def spawn(self, command, kw):
|
||||
cwd = Path(kw.get('cwd') or ROOT).resolve()
|
||||
if not inside(cwd, (ROOT, self.temporary)):
|
||||
raise IsolationDenied('unit isolation: non-fixture working directory')
|
||||
kw['cwd'] = str(cwd)
|
||||
# env supplied by tests is never propagated: even synthetic token mutations
|
||||
# remain in the parent. All approved children have only this fresh environment.
|
||||
kw['env'] = clean_environment(self.temporary / 'home', self.temporary)
|
||||
kw['env'].update(GIT_CONFIG_NOSYSTEM='1', GIT_CONFIG_GLOBAL='/dev/null', GIT_OPTIONAL_LOCKS='0')
|
||||
self.launch = tuple(command)
|
||||
try:
|
||||
return self.original_popen(command, **kw)
|
||||
finally:
|
||||
self.launch = None
|
||||
|
||||
def install(self):
|
||||
sys.addaudithook(self.audit)
|
||||
subprocess.Popen = self.popen
|
||||
guard = self
|
||||
original_socket = socket.socket
|
||||
class FixtureSocket(original_socket):
|
||||
def bind(self, address):
|
||||
super().bind(address)
|
||||
guard.endpoints.add(self.getsockname())
|
||||
socket.socket = FixtureSocket
|
||||
def numeric_addresses(host, port, family=0, type=0, proto=0, flags=0):
|
||||
if not guard.loopback_test or (host, port) not in guard.endpoints:
|
||||
raise IsolationDenied('unit isolation: DNS/external endpoint denied')
|
||||
return [(socket.AF_INET, socket.SOCK_STREAM, 6, '', (host, port))]
|
||||
socket.getaddrinfo = numeric_addresses
|
||||
def local_name(host=''):
|
||||
if not guard.loopback_test or host != '127.0.0.1':
|
||||
raise IsolationDenied('unit isolation: hostname lookup denied')
|
||||
return 'localhost'
|
||||
socket.getfqdn = local_name
|
||||
|
||||
def pytest_configure(self, config):
|
||||
config._unit_guard = self
|
||||
|
||||
# Exact single synthetic HTTP test only; no other test gets a loopback lease.
|
||||
def pytest_runtest_setup(self, item):
|
||||
self.loopback_test = item.nodeid == 'tests/test_desk.py::test_approve_flow_http_dry'
|
||||
self.endpoints.clear()
|
||||
|
||||
def pytest_runtest_teardown(self, item):
|
||||
self.loopback_test = False
|
||||
self.endpoints.clear()
|
||||
|
||||
|
||||
def worker(temporary: Path, arguments: list[str], netkingdom: Path | None, phase: Path | None):
|
||||
sys.dont_write_bytecode = True
|
||||
tempfile.tempdir = str(temporary)
|
||||
paths = [ROOT / 'src']
|
||||
if phase:
|
||||
paths.append(phase / 'src')
|
||||
# Explicit trusted source roots only, never ambient PYTHONPATH/plugins.
|
||||
sys.path[:0] = [str(p) for p in paths]
|
||||
roots = [ROOT, *paths, Path(sys.prefix), Path(sys.base_prefix), temporary]
|
||||
if phase:
|
||||
roots.append(phase / 'tests/fixtures/ops-warden-coordination-profile.json')
|
||||
if netkingdom:
|
||||
roots.append(netkingdom)
|
||||
os.environ['WARDEN_TEST_NETKINGDOM_ROOT'] = str(netkingdom)
|
||||
guard = Guard(temporary, roots)
|
||||
guard.install()
|
||||
# Pytest itself and collection run only after installation.
|
||||
import pytest
|
||||
raise SystemExit(pytest.main(['-p', 'no:cacheprovider', '--basetemp', str(temporary / 'pytest'), *arguments], plugins=[guard]))
|
||||
|
||||
|
||||
def main():
|
||||
if sys.argv[1:2] == ['--child']:
|
||||
payload = json.loads(sys.argv[2])
|
||||
temporary = Path(payload['temporary'])
|
||||
sys.dont_write_bytecode = True
|
||||
tempfile.tempdir = str(temporary)
|
||||
sys.path.insert(0, str(ROOT / 'src'))
|
||||
guard = Guard(temporary, [Path(p) for p in payload['read_roots']], child=True,
|
||||
git_metadata=payload.get('git_metadata', False))
|
||||
guard.install()
|
||||
sys.argv = [payload.get('script', '-c'), *payload['args']]
|
||||
if 'script' in payload:
|
||||
runpy.run_path(payload['script'], run_name='__main__')
|
||||
else:
|
||||
exec(compile(payload['code'], '<reviewed-unit-child>', 'exec'), {'__name__': '__main__'})
|
||||
return
|
||||
if sys.argv[1:2] == ['--worker']:
|
||||
temporary = Path(sys.argv[2])
|
||||
netkingdom = Path(sys.argv[3]) if sys.argv[3] else None
|
||||
phase = Path(sys.argv[4]) if sys.argv[4] else None
|
||||
worker(temporary, sys.argv[5:], netkingdom, phase)
|
||||
return
|
||||
import argparse
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--netkingdom-root', type=Path, default=ROOT.parent / 'net-kingdom')
|
||||
parser.add_argument('--phase-memory-root', type=Path, default=ROOT.parent / 'phase-memory')
|
||||
options, arguments = parser.parse_known_args()
|
||||
netkingdom = options.netkingdom_root.resolve() if options.netkingdom_root.is_dir() else None
|
||||
phase = options.phase_memory_root.resolve() if (options.phase_memory_root / 'src').is_dir() else None
|
||||
with tempfile.TemporaryDirectory(prefix='warden-unit-') as directory:
|
||||
temporary = Path(directory)
|
||||
home = temporary / 'home'
|
||||
home.mkdir()
|
||||
result = subprocess.run([sys.executable, '-I', str(SELF), '--worker', str(temporary),
|
||||
str(netkingdom) if netkingdom else '', str(phase) if phase else '', *arguments],
|
||||
cwd=ROOT, env=clean_environment(home, temporary))
|
||||
raise SystemExit(result.returncode)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
20
tests/fixtures/workload-coverage/provenance.json
vendored
Normal file
20
tests/fixtures/workload-coverage/provenance.json
vendored
Normal file
|
|
@ -0,0 +1,20 @@
|
|||
[
|
||||
{
|
||||
"repo": "rapp-core-hub",
|
||||
"source": "retained public snapshot from warden-1.3-kernel-fixes-r1 test preparation",
|
||||
"source_sha256": "98e81fb1116aa0ac9dc1a3e7aef9dd87c6d2110d98d46fbfca2f76eec8f3a77b",
|
||||
"projection_sha256": "9ce902da0c77809f1094a634244652835b162f59682e9fa6923d8c8aa4ed4a3c"
|
||||
},
|
||||
{
|
||||
"repo": "rapp-issue-core",
|
||||
"source": "retained public snapshot from warden-1.3-kernel-fixes-r1 test preparation",
|
||||
"source_sha256": "0425c9fe84e26b359b2a35fc37ef8fcb781de1aaa6229c59fcd246ee523514a5",
|
||||
"projection_sha256": "a1956deecf2f0fbca44549e5ea5e8f1c4b74ebf91a9482c64c982491af0d923b"
|
||||
},
|
||||
{
|
||||
"repo": "rapp-qonto",
|
||||
"source": "retained public snapshot from warden-1.3-kernel-fixes-r1 test preparation",
|
||||
"source_sha256": "6279dc2f3ab09519dffa1ceeedb2095f1182960e0106109a87aa5c0052446883",
|
||||
"projection_sha256": "15aa9809e8d6a120c516be5ce9adfb72c00382140c171e791083ad117f503b3f"
|
||||
}
|
||||
]
|
||||
15
tests/fixtures/workload-coverage/rapp-core-hub/declarations/rapp.yaml
vendored
Normal file
15
tests/fixtures/workload-coverage/rapp-core-hub/declarations/rapp.yaml
vendored
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
# Public declaration projection for the published coverage snapshot; not live evidence.
|
||||
rapp_id: rapp-core-hub
|
||||
workload_identity:
|
||||
name: core-hub
|
||||
composition:
|
||||
member_repos:
|
||||
- deployables:
|
||||
- core-hub-api
|
||||
- core-hub-migrate
|
||||
- hub-core-candidate
|
||||
- hub-core-candidate-migrate
|
||||
- repository-classification-publisher
|
||||
data_classification: confidential
|
||||
criticality: high
|
||||
readiness_state: verified
|
||||
11
tests/fixtures/workload-coverage/rapp-issue-core/declarations/rapp.yaml
vendored
Normal file
11
tests/fixtures/workload-coverage/rapp-issue-core/declarations/rapp.yaml
vendored
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
# Public declaration projection for the published coverage snapshot; not live evidence.
|
||||
rapp_id: rapp-issue-core
|
||||
workload_identity:
|
||||
name: issue-core
|
||||
composition:
|
||||
member_repos:
|
||||
- deployables:
|
||||
- issue-core
|
||||
data_classification: confidential
|
||||
criticality: high
|
||||
readiness_state: verified
|
||||
11
tests/fixtures/workload-coverage/rapp-qonto/declarations/rapp.yaml
vendored
Normal file
11
tests/fixtures/workload-coverage/rapp-qonto/declarations/rapp.yaml
vendored
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
# Public declaration projection for the published coverage snapshot; not live evidence.
|
||||
rapp_id: rapp-qonto
|
||||
workload_identity:
|
||||
name: qonto
|
||||
composition:
|
||||
member_repos:
|
||||
- deployables:
|
||||
- rapp-qonto
|
||||
data_classification: restricted
|
||||
criticality: critical
|
||||
readiness_state: verified
|
||||
|
|
@ -2,6 +2,7 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
|
@ -12,7 +13,8 @@ from warden.case_plan import CaseError, plan_case
|
|||
from warden.cli import app
|
||||
from warden.routing.catalog import load_catalog
|
||||
|
||||
LIVE_CASE = Path.home() / "net-kingdom/security-cases/forgejo-netkingdom-oidc.yaml"
|
||||
LIVE_CASE = Path(os.environ.get("WARDEN_TEST_NETKINGDOM_ROOT",
|
||||
Path(__file__).resolve().parents[2] / "net-kingdom")) / "security-cases/forgejo-netkingdom-oidc.yaml"
|
||||
|
||||
TYPE = {
|
||||
"apiVersion": "netkingdom.io/security-case-type/v0.1",
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
|
@ -12,7 +13,8 @@ from warden.case_type import check_follow_through, load_type
|
|||
from warden.cli import app
|
||||
from warden.routing.catalog import load_catalog
|
||||
|
||||
LIVE_TYPE = Path.home() / "net-kingdom/security-case-types/oidc-browser-relying-party/type.yaml"
|
||||
LIVE_TYPE = Path(os.environ.get("WARDEN_TEST_NETKINGDOM_ROOT",
|
||||
Path(__file__).resolve().parents[2] / "net-kingdom")) / "security-case-types/oidc-browser-relying-party/type.yaml"
|
||||
|
||||
BASE = {
|
||||
"apiVersion": "netkingdom.io/security-case-type/v0.1",
|
||||
|
|
|
|||
|
|
@ -273,7 +273,11 @@ class TestPepStanceMap:
|
|||
# --- classification coverage (v0.8 §6.4 obligation 3) -------------------------
|
||||
|
||||
def test_published_coverage_equals_measured_coverage():
|
||||
"""The published figure must equal what the repo actually measures.
|
||||
"""The published figure must equal its explicit public declaration snapshot.
|
||||
|
||||
This is a deterministic source check, not a claim about current estate state.
|
||||
Live measurement remains scripts/report_coverage.py with an explicit estate
|
||||
root; tests never treat the developer HOME as an authority or fixture.
|
||||
|
||||
ops-warden asked gate-house for §13.1's Coverage column and its figures are
|
||||
that column's first entries, so their accuracy is ours to hold. The register
|
||||
|
|
@ -295,7 +299,7 @@ def test_published_coverage_equals_measured_coverage():
|
|||
published = yaml.safe_load((repo / "pep-stance.yaml").read_text())[
|
||||
"classification_coverage"
|
||||
]
|
||||
measured = module.measure()
|
||||
measured = module.measure(repo / "tests/fixtures/workload-coverage")
|
||||
|
||||
for population in ("signing_targets", "routing_lanes"):
|
||||
assert published[population] == measured[population], population
|
||||
|
|
|
|||
92
tests/test_safe_unit_runner.py
Normal file
92
tests/test_safe_unit_runner.py
Normal file
|
|
@ -0,0 +1,92 @@
|
|||
"""Exercise installed parent and nested child controls, never real credentials."""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def guarded(request):
|
||||
guard = getattr(request.config, '_unit_guard', None)
|
||||
if guard is None:
|
||||
pytest.skip('requires make test / scripts/run_unit_tests.py')
|
||||
return guard
|
||||
|
||||
|
||||
def test_parent_environment_is_allowlisted_and_home_is_temporary(guarded):
|
||||
# The external regression invocation sets synthetic BAO_TOKEN/VAULT_TOKEN/HOME.
|
||||
assert 'BAO_TOKEN' not in os.environ
|
||||
assert 'VAULT_TOKEN' not in os.environ
|
||||
assert Path(os.environ['TMPDIR']) in Path.home().parents
|
||||
assert 'PYTHONPATH' not in os.environ
|
||||
|
||||
|
||||
@pytest.mark.parametrize('command', [
|
||||
['bao', 'token', 'lookup'], ['ssh', 'railiance01'], ['curl', 'http://127.0.0.1:8000'],
|
||||
[sys.executable, '-c', 'import os; print(os.environ)'],
|
||||
])
|
||||
def test_unexpected_process_refused_before_launch(guarded, command):
|
||||
with pytest.raises(PermissionError, match='unit isolation'):
|
||||
subprocess.run(command, check=True)
|
||||
|
||||
|
||||
def test_parent_socket_denies_even_unregistered_loopback(guarded):
|
||||
with pytest.raises(PermissionError, match='unit isolation'):
|
||||
socket.socket().connect(('127.0.0.1', 8000))
|
||||
|
||||
|
||||
@pytest.mark.parametrize('path', ['/home/synthetic-ambient-unit-fixture/.vault-token',
|
||||
'/root/.kube/config', '/proc/self/environ',
|
||||
'/tmp/warden-synthetic-ambient-home/.vault-token'])
|
||||
def test_ambient_paths_refused_without_opening_them(guarded, path):
|
||||
with pytest.raises(PermissionError, match='unit isolation'):
|
||||
Path(path).read_bytes()
|
||||
|
||||
|
||||
def test_symlink_cannot_escape_fixture_root(guarded, tmp_path):
|
||||
link = tmp_path / 'ambient'
|
||||
link.symlink_to('/home/synthetic-ambient-unit-fixture/.vault-token')
|
||||
with pytest.raises(PermissionError, match='unit isolation'):
|
||||
link.read_bytes()
|
||||
|
||||
|
||||
def test_child_drops_even_synthetic_token_environment(guarded, monkeypatch):
|
||||
monkeypatch.setenv('BAO_TOKEN', 'synthetic-only-never-a-real-token')
|
||||
monkeypatch.setenv('VAULT_TOKEN', 'synthetic-only-never-a-real-token')
|
||||
monkeypatch.setenv('HOME', '/home/synthetic-ambient-unit-fixture')
|
||||
result = subprocess.run([sys.executable, str(ROOT / 'tests/unit_isolation_probe.py'), 'state'],
|
||||
capture_output=True, text=True, check=True, env=dict(os.environ))
|
||||
assert json.loads(result.stdout) == {'bao_token_present': False,
|
||||
'vault_token_present': False, 'home_in_temporary': True}
|
||||
|
||||
|
||||
@pytest.mark.parametrize('mode', ['socket', 'provider', 'home', 'nested'])
|
||||
def test_guard_exists_in_each_allowed_python_child(guarded, mode):
|
||||
result = subprocess.run([sys.executable, str(ROOT / 'tests/unit_isolation_probe.py'), mode],
|
||||
capture_output=True, text=True, check=True)
|
||||
assert result.stdout == 'denied\n'
|
||||
|
||||
|
||||
def test_forged_provider_double_refused(guarded, tmp_path):
|
||||
provider = tmp_path / 'bao'
|
||||
provider.write_text('#!/bin/bash\necho not-the-reviewed-fixture\n')
|
||||
provider.chmod(0o700)
|
||||
with pytest.raises(PermissionError, match='double bytes differ'):
|
||||
subprocess.run([str(provider), 'token', 'lookup'], check=True)
|
||||
|
||||
|
||||
def test_os_open_truncation_respects_read_only_source_root(guarded, tmp_path, monkeypatch):
|
||||
# Synthetic readable source outside the guard's writable subtree.
|
||||
source = tmp_path / 'synthetic-source'
|
||||
source.write_bytes(b'synthetic public source')
|
||||
monkeypatch.setattr(guarded, 'temporary', tmp_path / 'writable')
|
||||
with pytest.raises(PermissionError, match='unit isolation'):
|
||||
descriptor = os.open(source, os.O_RDONLY | os.O_TRUNC)
|
||||
os.close(descriptor)
|
||||
assert source.read_bytes() == b'synthetic public source'
|
||||
|
|
@ -1,5 +1,7 @@
|
|||
"""Explicit lane-to-workload join tests (WARDEN-WP-0032 / RMGR-WP-0010-T06)."""
|
||||
from pathlib import Path
|
||||
import hashlib
|
||||
import json
|
||||
|
||||
import yaml
|
||||
|
||||
|
|
@ -10,7 +12,8 @@ ROOT = Path(__file__).resolve().parents[1]
|
|||
|
||||
|
||||
def test_repo_catalog_uses_only_explicit_workload_references():
|
||||
report = build(ROOT / "registry/routing/catalog.yaml", Path.home())
|
||||
report = build(ROOT / "registry/routing/catalog.yaml",
|
||||
ROOT / "tests/fixtures/workload-coverage")
|
||||
assert report["ok"] is True
|
||||
assert len(report["resolved"]) == 4
|
||||
assert len(report["unknown"]) == 22
|
||||
|
|
@ -67,3 +70,14 @@ def test_invalid_exact_deployable_resolves_unknown(tmp_path):
|
|||
assert not report["resolved"]
|
||||
assert report["unknown"][0]["lane"] == "x"
|
||||
assert "deployable" in report["unknown"][0]["reason"]
|
||||
|
||||
|
||||
def test_public_workload_projection_matches_provenance():
|
||||
base = ROOT / 'tests/fixtures/workload-coverage'
|
||||
provenance = json.loads((base / 'provenance.json').read_text())
|
||||
expected = {'rapp-core-hub', 'rapp-issue-core', 'rapp-qonto'}
|
||||
assert len(provenance) == len(expected)
|
||||
assert {row['repo'] for row in provenance} == expected
|
||||
for row in provenance:
|
||||
projection = base / row['repo'] / 'declarations/rapp.yaml'
|
||||
assert hashlib.sha256(projection.read_bytes()).hexdigest() == row['projection_sha256']
|
||||
|
|
|
|||
29
tests/unit_isolation_probe.py
Normal file
29
tests/unit_isolation_probe.py
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
"""Closed child regression probes; executed only under the unit runner guard."""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
mode = sys.argv[1]
|
||||
if mode == 'state':
|
||||
print(json.dumps({'bao_token_present': 'BAO_TOKEN' in os.environ,
|
||||
'vault_token_present': 'VAULT_TOKEN' in os.environ,
|
||||
'home_in_temporary': Path(os.environ['TMPDIR']) in Path.home().parents}))
|
||||
else:
|
||||
try:
|
||||
if mode == 'socket':
|
||||
socket.socket().connect(('127.0.0.1', 8000))
|
||||
elif mode == 'provider':
|
||||
subprocess.run(['bao', 'token', 'lookup'], check=True)
|
||||
elif mode == 'home':
|
||||
Path('/home/synthetic-ambient-unit-fixture/.vault-token').read_text()
|
||||
elif mode == 'nested':
|
||||
subprocess.run([sys.executable, '-c', 'print("SYNTHETIC-SECRET")'], check=True)
|
||||
else:
|
||||
raise AssertionError('unknown probe')
|
||||
except PermissionError:
|
||||
print('denied')
|
||||
else:
|
||||
raise AssertionError('external operation was not refused')
|
||||
|
|
@ -354,3 +354,41 @@ then use the preserved isolated review runner, not the ambient `make test` path.
|
|||
No credential lookup, live investigation or rotation is authorized by this note.
|
||||
T03 remains progress; contained sessions, authenticated owner provenance,
|
||||
materialization, exact type acknowledgment/pin and first native case remain open.
|
||||
|
||||
|
||||
2026-10-17 safe unit-runner source candidate: `make test` now selects an existing
|
||||
interpreter and `scripts/run_unit_tests.py`, which constructs the environment and
|
||||
empty temporary HOME before pytest collection, denies unexpected process/socket
|
||||
use and ambient file opens, and guards the pinned BrowserInputs child too.
|
||||
Closed fixture exceptions cover the synthetic desk endpoint, metadata checkers,
|
||||
exact-byte contract doubles and fixed transport prints; they confer no provider
|
||||
authority. Explicit public declaration fixtures replace ambient HOME source
|
||||
lookups. See `docs/development/safe-unit-tests.md` and Custodian packet
|
||||
`docs/changes/netkingdom-target-2026-10-17/warden-safe-unit-runner-r1`.
|
||||
Synthetic token, ambient-path, forged-provider and nested-child regressions are
|
||||
retained alongside the initial failed isolation runs and corrected results. This
|
||||
is a source candidate for the existing safe-test handoff, not an OS sandbox or
|
||||
native case acceptance. T03 remains progress; contained sessions, authenticated
|
||||
owner provenance, materialization, exact type acknowledgment/pin and first native
|
||||
case remain open. No credential lookup, live investigation or rotation occurred.
|
||||
|
||||
Candidate validation: ordinary `make test` under synthetic ambient token/HOME
|
||||
controls passes 783 tests with four integration-marked tests deselected. Affected
|
||||
focused checks pass 117 tests; new runner code/regressions pass Ruff. Retained
|
||||
initial failures exposed fixture dependencies and were corrected, not suppressed.
|
||||
|
||||
|
||||
2026-10-17 root follow-through: the original safe-runner source candidate was
|
||||
independently rejected for O_TRUNC open-write misclassification and unchecked
|
||||
public fixture projection hashes. Both counterexamples are retained. The minimal
|
||||
correction treats truncating/appending opens as writes and adds the closed
|
||||
projection-byte provenance regression. Root inspected the two exact corrections
|
||||
and ran ordinary make test on stable corrected source with only synthetic parent
|
||||
HOME/token/Python/pytest sentinels: 785 passed, four deselected. No real credential
|
||||
or native provider action occurred. Initial author failures and rejected review
|
||||
are preserved in the Custodian warden-safe-unit-runner-r1 and
|
||||
warden-safe-unit-runner-review-r1 packets, with root disposition/full result in
|
||||
warden-safe-unit-integration-r1. This completes the known trusted-unit-entry
|
||||
accident-prevention follow-through; direct pytest bypass, native/non-open/preopened
|
||||
file-descriptor limits remain explicit. No OS sandbox, isolated agent assignment,
|
||||
production grant or native T03 acceptance follows. T03 stays progress.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue