test: isolate ordinary unit runs from ambient credentials and provider I/O
Assistant: codex Assistant-Model: gpt-6.1-sol Assistant-Session: 01a1468b-63fa-7710-85dc-8f2e16b3ebfa
This commit is contained in:
parent
922c26da0f
commit
b593c87983
14 changed files with 592 additions and 6 deletions
3
Makefile
3
Makefile
|
|
@ -1,5 +1,6 @@
|
||||||
# ops-warden development and install targets
|
# ops-warden development and install targets
|
||||||
ROOT := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
|
ROOT := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
|
||||||
|
UNIT_PYTHON ?= $(ROOT)/.venv/bin/python
|
||||||
PHASE_MEMORY_REPO ?= $(abspath $(ROOT)/../phase-memory)
|
PHASE_MEMORY_REPO ?= $(abspath $(ROOT)/../phase-memory)
|
||||||
|
|
||||||
.DEFAULT_GOAL := help
|
.DEFAULT_GOAL := help
|
||||||
|
|
@ -43,7 +44,7 @@ verify-memory: check-memory
|
||||||
@warden memory status
|
@warden memory status
|
||||||
|
|
||||||
test:
|
test:
|
||||||
PYTHONPATH=src:$(PHASE_MEMORY_REPO)/src uv run pytest
|
"$(UNIT_PYTHON)" -I scripts/run_unit_tests.py --phase-memory-root "$(PHASE_MEMORY_REPO)" $(TEST_ARGS)
|
||||||
|
|
||||||
lint:
|
lint:
|
||||||
uv run ruff check .
|
uv run ruff check .
|
||||||
|
|
|
||||||
64
docs/development/safe-unit-tests.md
Normal file
64
docs/development/safe-unit-tests.md
Normal file
|
|
@ -0,0 +1,64 @@
|
||||||
|
# Safe unit-test entry point
|
||||||
|
|
||||||
|
Use `make test`, or `.venv/bin/python -I scripts/run_unit_tests.py`. Dependencies
|
||||||
|
must already be installed; this path does not run uv, synchronize or download.
|
||||||
|
`UNIT_PYTHON` may select another existing interpreter. `PHASE_MEMORY_REPO` remains
|
||||||
|
an explicit local source option. Pass pytest options with `TEST_ARGS`, for example
|
||||||
|
`make test TEST_ARGS='-q tests/test_safe_unit_runner.py'`.
|
||||||
|
|
||||||
|
The launcher creates an empty temporary HOME and temporary fixture root and starts
|
||||||
|
an isolated (`-I`) Python interpreter with a constructed environment. Caller token,
|
||||||
|
proxy, Python, pytest, XDG and Warden configuration variables are never copied or
|
||||||
|
inventoried. Third-party pytest plugin autoload is disabled. Tests can explicitly
|
||||||
|
introduce synthetic variables with monkeypatch, but allowed children always get
|
||||||
|
the original temporary HOME and a fresh environment, never those variables.
|
||||||
|
|
||||||
|
Before pytest import/collection, a Python audit hook and subprocess wrapper:
|
||||||
|
|
||||||
|
- deny file opens outside the selected source roots, installed interpreter/library
|
||||||
|
roots and fixture root; writes through open are limited to the fixture root;
|
||||||
|
- deny sockets, DNS and process creation except the closed fixture inventory;
|
||||||
|
- refuse shell, executable, preexec and inherited-FD overrides outside that inventory;
|
||||||
|
- install the same file/socket/process denial before executing every allowed
|
||||||
|
Python child, including BrowserInputs' exact validator code and reverified copied
|
||||||
|
module hashes. Validator children cannot start further processes.
|
||||||
|
|
||||||
|
The process inventory in `scripts/run_unit_tests.py` is intentionally small:
|
||||||
|
BrowserInputs validation, four metadata-checker scripts with fixed argument shapes,
|
||||||
|
the exact original-intent restart probe, the synthetic containment print, and
|
||||||
|
closed guard-regression probes. Metadata checkers receive only fixture output
|
||||||
|
paths. Only the generated-artifact checker child can run the exact read-only Git
|
||||||
|
log/status queries for this repository's catalog, with global/system config and
|
||||||
|
optional locks disabled, hooks disabled and fsmonitor disabled. Those local Git
|
||||||
|
commands are not provider operations.
|
||||||
|
|
||||||
|
Two reviewed non-Python cases remain: exact-byte, temporary contract-double scripts
|
||||||
|
with a closed argv list, and five exact fixed `printf` commands used by existing
|
||||||
|
transport tests. They run Bash without profile/rc startup and without inherited
|
||||||
|
environment; no arbitrary shell text, provider binary, SSH, kubectl or OpenBao
|
||||||
|
command is permitted. Default unit selection excludes four integration-marked
|
||||||
|
SSH-keygen tests. Selecting them explicitly does not grant their native commands.
|
||||||
|
|
||||||
|
Only `test_desk.py::test_approve_flow_http_dry` receives a network lease. It may
|
||||||
|
bind `127.0.0.1:0` and connect to the endpoint actually allocated to that test's
|
||||||
|
socket. DNS is replaced by numeric fixture resolution. Another loopback service
|
||||||
|
(e.g. the real Hub on port 8000), Unix sockets and remote endpoints stay denied.
|
||||||
|
The lease and registered endpoints are cleared after the test.
|
||||||
|
|
||||||
|
Coverage tests now use the explicit public declaration projection snapshot in
|
||||||
|
`tests/fixtures/workload-coverage`, with provenance hashes. They compare the
|
||||||
|
published source snapshot rather than silently consulting the workstation HOME.
|
||||||
|
Run the existing coverage report separately with an explicit estate root when
|
||||||
|
measuring current estate state. Tests of the acknowledged NetKingdom source use
|
||||||
|
`--netkingdom-root` (default: sibling source checkout), never a HOME symlink. The
|
||||||
|
optional Phase Memory library's one existing public coordination-profile fixture
|
||||||
|
is explicitly readable alongside its source; no runtime memory store is inherited.
|
||||||
|
|
||||||
|
This is an accident guard for trusted unit tests, not a security sandbox for
|
||||||
|
hostile Python, native extensions or hostile installed dependencies. It does not
|
||||||
|
revoke already-open descriptors, prevent every filesystem mutation syscall, or
|
||||||
|
provide OS process/network isolation. Fixed Bash/Git fixture processes rely on
|
||||||
|
the reviewed executable and source/argument shapes; Python audit hooks do not run
|
||||||
|
inside those binaries. Do not run untrusted test code under a privileged runtime.
|
||||||
|
Direct `pytest` bypasses this launcher and is not equivalent validation. No
|
||||||
|
production Warden boundary, grant, catalog, source pin or owner credential changes.
|
||||||
283
scripts/run_unit_tests.py
Normal file
283
scripts/run_unit_tests.py
Normal file
|
|
@ -0,0 +1,283 @@
|
||||||
|
"""Run trusted unit tests with no inherited credentials or external I/O.
|
||||||
|
|
||||||
|
This is an accident guard for this suite, not an OS sandbox for hostile Python.
|
||||||
|
Direct pytest bypasses it. Use make test (already installed dependencies only).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
import runpy
|
||||||
|
import socket
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
SELF = Path(__file__).resolve()
|
||||||
|
SHELL_FIXTURES = {
|
||||||
|
"printf 'the-secret-value'", "printf 'sekret'", "printf 'new'",
|
||||||
|
"printf '%s' '{\"wrap_info\":{\"token\":\"hvs.WRAP\"}}'",
|
||||||
|
"printf 'not-json'",
|
||||||
|
}
|
||||||
|
JOURNAL_CODE_SHA256 = '69aa18f302fd698104cb2253b4af0c3a077e72a0a427183e3dce1444c4926df9'
|
||||||
|
|
||||||
|
|
||||||
|
class IsolationDenied(PermissionError, subprocess.SubprocessError):
|
||||||
|
"""A unit test attempted an unreviewed external operation."""
|
||||||
|
|
||||||
|
|
||||||
|
def clean_environment(home: Path, temporary: Path) -> dict[str, str]:
|
||||||
|
# Construct, never copy/inventory the caller's environment.
|
||||||
|
return {'HOME': str(home), 'TMPDIR': str(temporary), 'PATH': '/usr/bin:/bin',
|
||||||
|
'LANG': 'C.UTF-8', 'PYTHONDONTWRITEBYTECODE': '1',
|
||||||
|
'PYTEST_DISABLE_PLUGIN_AUTOLOAD': '1'}
|
||||||
|
|
||||||
|
|
||||||
|
def inside(path: Path, roots: tuple[Path, ...]) -> bool:
|
||||||
|
return any(path == root or root in path.parents for root in roots)
|
||||||
|
|
||||||
|
|
||||||
|
class Guard:
|
||||||
|
def __init__(self, temporary: Path, read_roots: list[Path], child: bool = False, git_metadata: bool = False):
|
||||||
|
self.temporary = temporary.resolve()
|
||||||
|
self.read_roots = tuple(p.resolve() for p in read_roots)
|
||||||
|
self.child = child
|
||||||
|
self.git_metadata = git_metadata
|
||||||
|
self.loopback_test = False
|
||||||
|
self.endpoints: set[tuple[str, int]] = set()
|
||||||
|
self.launch = None
|
||||||
|
self.original_popen = subprocess.Popen
|
||||||
|
|
||||||
|
def fixture_path(self, value: str) -> Path:
|
||||||
|
path = Path(value).resolve()
|
||||||
|
if not inside(path, (self.temporary,)):
|
||||||
|
raise IsolationDenied('unit isolation: non-fixture path')
|
||||||
|
return path
|
||||||
|
|
||||||
|
def audit(self, event, args):
|
||||||
|
if event == 'open' and not isinstance(args[0], int):
|
||||||
|
path = Path(os.fsdecode(args[0])).resolve()
|
||||||
|
mode, flags = args[1:3]
|
||||||
|
write = ((isinstance(mode, str) and any(c in mode for c in 'wax+'))
|
||||||
|
or (isinstance(flags, int) and flags & (os.O_WRONLY | os.O_RDWR | os.O_CREAT | os.O_TRUNC | os.O_APPEND)))
|
||||||
|
roots = (self.temporary,) if write else self.read_roots
|
||||||
|
if path != Path('/dev/null') and not inside(path, roots):
|
||||||
|
raise IsolationDenied('unit isolation: file outside reviewed roots')
|
||||||
|
if event == 'subprocess.Popen':
|
||||||
|
if self.launch is None or tuple(args[1]) != self.launch:
|
||||||
|
raise IsolationDenied('unit isolation: unexpected process')
|
||||||
|
if event in {'os.system', 'os.exec', 'os.posix_spawn', 'os.fork', 'os.forkpty'}:
|
||||||
|
raise IsolationDenied('unit isolation: native execution disabled')
|
||||||
|
if event.startswith('socket.'):
|
||||||
|
if not self.loopback_test or self.child:
|
||||||
|
raise IsolationDenied('unit isolation: sockets disabled')
|
||||||
|
if event == 'socket.__new__':
|
||||||
|
if args[1] != socket.AF_INET or args[2] & 15 != socket.SOCK_STREAM:
|
||||||
|
raise IsolationDenied('unit isolation: socket family/type denied')
|
||||||
|
elif event == 'socket.bind':
|
||||||
|
if args[1] != ('127.0.0.1', 0):
|
||||||
|
raise IsolationDenied('unit isolation: only ephemeral fixture bind')
|
||||||
|
elif event == 'socket.connect':
|
||||||
|
if args[1] not in self.endpoints:
|
||||||
|
raise IsolationDenied('unit isolation: only this fixture endpoint')
|
||||||
|
elif event not in {'socket.gethostname'}:
|
||||||
|
raise IsolationDenied('unit isolation: DNS/socket operation denied')
|
||||||
|
|
||||||
|
def python_payload(self, argv):
|
||||||
|
"""Closed subprocess inventory; every allowed Python child installs guard."""
|
||||||
|
from warden import case_inputs
|
||||||
|
if (len(argv) == 5 and argv[:3] == [sys.executable, '-I', '-c']
|
||||||
|
and argv[3] == case_inputs._VALIDATE):
|
||||||
|
directory = self.fixture_path(argv[4])
|
||||||
|
if not directory.name.startswith('warden-case-validator-'):
|
||||||
|
raise IsolationDenied('unit isolation: validator fixture directory differs')
|
||||||
|
# Reverify copied modules immediately before launch, independently of caller.
|
||||||
|
for relative, digest in case_inputs.SOURCE_PINS.items():
|
||||||
|
if hashlib.sha256((directory / relative).read_bytes()).hexdigest() != digest:
|
||||||
|
raise IsolationDenied('unit isolation: validator source pin differs')
|
||||||
|
return {'code': argv[3], 'args': [str(directory)]}
|
||||||
|
if len(argv) == 3 and argv[:2] == [sys.executable, '-c'] and argv[2] == 'print("SYNTHETIC-SECRET")':
|
||||||
|
return {'code': argv[2], 'args': []}
|
||||||
|
if (len(argv) == 6 and argv[:3] == [sys.executable, '-B', '-c']
|
||||||
|
and hashlib.sha256(argv[3].encode()).hexdigest() == JOURNAL_CODE_SHA256):
|
||||||
|
self.fixture_path(argv[4])
|
||||||
|
if len(argv[5]) > 192:
|
||||||
|
raise IsolationDenied('unit isolation: journal source bound')
|
||||||
|
return {'code': argv[3], 'args': argv[4:]}
|
||||||
|
if len(argv) >= 2 and argv[0] in (sys.executable, 'python3'):
|
||||||
|
script = Path(argv[1]).resolve()
|
||||||
|
args = argv[2:]
|
||||||
|
if script == ROOT / 'tests/unit_isolation_probe.py' and len(args) == 1 and args[0] in {'state', 'socket', 'provider', 'home', 'nested'}:
|
||||||
|
return {'script': str(script), 'args': args}
|
||||||
|
if script == ROOT / 'scripts/check_layer_conformance.py' and not args:
|
||||||
|
return {'script': str(script), 'args': args}
|
||||||
|
if script == ROOT / 'scripts/emit_high_risk_paths.py' and args == ['--check']:
|
||||||
|
return {'script': str(script), 'args': args, 'git_metadata': True}
|
||||||
|
if script == ROOT / 'scripts/check_principals_drift.py' and len(args) == 4 and args[::2] == ['--inventory', '--infra']:
|
||||||
|
for value in args[1::2]:
|
||||||
|
self.fixture_path(value)
|
||||||
|
return {'script': str(script), 'args': args}
|
||||||
|
if script == ROOT / 'scripts/build_flex_auth_registry.py':
|
||||||
|
if (len(args) in (3, 5) and Path(args[0]).resolve() == ROOT / 'examples/inventory.seed.yaml'
|
||||||
|
and args[-2] == '-o' and (len(args) == 3 or args[1] == '--zone-resolutions')):
|
||||||
|
self.fixture_path(args[-1])
|
||||||
|
if len(args) == 5:
|
||||||
|
self.fixture_path(args[2])
|
||||||
|
return {'script': str(script), 'args': args}
|
||||||
|
raise IsolationDenied('unit isolation: command not in reviewed subprocess inventory')
|
||||||
|
|
||||||
|
def popen(self, argv, *pos, **kw):
|
||||||
|
if pos or kw.get('executable') or kw.get('preexec_fn') or kw.get('pass_fds') or kw.get('close_fds') is False:
|
||||||
|
raise IsolationDenied('unit isolation: unexpected process options')
|
||||||
|
if kw.pop('shell', False):
|
||||||
|
if self.child or not isinstance(argv, str) or argv not in SHELL_FIXTURES:
|
||||||
|
raise IsolationDenied('unit isolation: unreviewed shell command')
|
||||||
|
return self.spawn(['/bin/bash', '--noprofile', '--norc', '-c', argv], kw)
|
||||||
|
if not isinstance(argv, (list, tuple)):
|
||||||
|
raise IsolationDenied('unit isolation: argv list required')
|
||||||
|
argv = [os.fspath(x) for x in argv]
|
||||||
|
# Never run a real provider: the only shell fixture is an exact-byte double.
|
||||||
|
from warden.doubles import _DOUBLES
|
||||||
|
metadata_commands = (['git', 'log', '-1', '--format=%H %cI', '--', str(ROOT / 'registry/routing/catalog.yaml')],
|
||||||
|
['git', 'status', '--porcelain', '--', str(ROOT / 'registry/routing/catalog.yaml')])
|
||||||
|
if self.child and not (self.git_metadata and argv in metadata_commands):
|
||||||
|
raise IsolationDenied('unit isolation: nested process disabled')
|
||||||
|
candidate = Path(argv[0])
|
||||||
|
if not candidate.is_absolute():
|
||||||
|
import shutil
|
||||||
|
resolved = shutil.which(argv[0], path=kw.get('env', os.environ).get('PATH', ''))
|
||||||
|
candidate = Path(resolved) if resolved else candidate
|
||||||
|
if self.child and self.git_metadata:
|
||||||
|
if Path(kw.get('cwd', '.')).resolve() != ROOT:
|
||||||
|
raise IsolationDenied('unit isolation: metadata cwd differs')
|
||||||
|
command = ['/usr/bin/git', '-c', 'core.fsmonitor=false', '-c', 'core.hooksPath=/dev/null', '--no-pager', *argv[1:]]
|
||||||
|
elif candidate.is_absolute() and inside(candidate.resolve(), (self.temporary,)) and candidate.name in _DOUBLES:
|
||||||
|
if candidate.read_text() != _DOUBLES[candidate.name].script:
|
||||||
|
raise IsolationDenied('unit isolation: double bytes differ')
|
||||||
|
tail = argv[1:]
|
||||||
|
if (candidate.name == 'bao' and tail not in (
|
||||||
|
['kv', 'get', '-field=NPM_AUTH_TOKEN', 'platform/x/y'],
|
||||||
|
['kv', 'get', '-field=API_KEY', 'platform/x/y/z'],
|
||||||
|
['login', '-method=oidc'], ['write', 'secret/x'])) or (candidate.name == 'key-cape' and tail != ['login']):
|
||||||
|
raise IsolationDenied('unit isolation: double arguments differ')
|
||||||
|
command = ['/bin/bash', '--noprofile', '--norc', str(candidate), *tail]
|
||||||
|
else:
|
||||||
|
payload = self.python_payload(argv)
|
||||||
|
payload.update(temporary=str(self.temporary), read_roots=[str(x) for x in self.read_roots])
|
||||||
|
command = [sys.executable, '-I', str(SELF), '--child', json.dumps(payload)]
|
||||||
|
return self.spawn(command, kw)
|
||||||
|
|
||||||
|
def spawn(self, command, kw):
|
||||||
|
cwd = Path(kw.get('cwd') or ROOT).resolve()
|
||||||
|
if not inside(cwd, (ROOT, self.temporary)):
|
||||||
|
raise IsolationDenied('unit isolation: non-fixture working directory')
|
||||||
|
kw['cwd'] = str(cwd)
|
||||||
|
# env supplied by tests is never propagated: even synthetic token mutations
|
||||||
|
# remain in the parent. All approved children have only this fresh environment.
|
||||||
|
kw['env'] = clean_environment(self.temporary / 'home', self.temporary)
|
||||||
|
kw['env'].update(GIT_CONFIG_NOSYSTEM='1', GIT_CONFIG_GLOBAL='/dev/null', GIT_OPTIONAL_LOCKS='0')
|
||||||
|
self.launch = tuple(command)
|
||||||
|
try:
|
||||||
|
return self.original_popen(command, **kw)
|
||||||
|
finally:
|
||||||
|
self.launch = None
|
||||||
|
|
||||||
|
def install(self):
|
||||||
|
sys.addaudithook(self.audit)
|
||||||
|
subprocess.Popen = self.popen
|
||||||
|
guard = self
|
||||||
|
original_socket = socket.socket
|
||||||
|
class FixtureSocket(original_socket):
|
||||||
|
def bind(self, address):
|
||||||
|
super().bind(address)
|
||||||
|
guard.endpoints.add(self.getsockname())
|
||||||
|
socket.socket = FixtureSocket
|
||||||
|
def numeric_addresses(host, port, family=0, type=0, proto=0, flags=0):
|
||||||
|
if not guard.loopback_test or (host, port) not in guard.endpoints:
|
||||||
|
raise IsolationDenied('unit isolation: DNS/external endpoint denied')
|
||||||
|
return [(socket.AF_INET, socket.SOCK_STREAM, 6, '', (host, port))]
|
||||||
|
socket.getaddrinfo = numeric_addresses
|
||||||
|
def local_name(host=''):
|
||||||
|
if not guard.loopback_test or host != '127.0.0.1':
|
||||||
|
raise IsolationDenied('unit isolation: hostname lookup denied')
|
||||||
|
return 'localhost'
|
||||||
|
socket.getfqdn = local_name
|
||||||
|
|
||||||
|
def pytest_configure(self, config):
|
||||||
|
config._unit_guard = self
|
||||||
|
|
||||||
|
# Exact single synthetic HTTP test only; no other test gets a loopback lease.
|
||||||
|
def pytest_runtest_setup(self, item):
|
||||||
|
self.loopback_test = item.nodeid == 'tests/test_desk.py::test_approve_flow_http_dry'
|
||||||
|
self.endpoints.clear()
|
||||||
|
|
||||||
|
def pytest_runtest_teardown(self, item):
|
||||||
|
self.loopback_test = False
|
||||||
|
self.endpoints.clear()
|
||||||
|
|
||||||
|
|
||||||
|
def worker(temporary: Path, arguments: list[str], netkingdom: Path | None, phase: Path | None):
|
||||||
|
sys.dont_write_bytecode = True
|
||||||
|
tempfile.tempdir = str(temporary)
|
||||||
|
paths = [ROOT / 'src']
|
||||||
|
if phase:
|
||||||
|
paths.append(phase / 'src')
|
||||||
|
# Explicit trusted source roots only, never ambient PYTHONPATH/plugins.
|
||||||
|
sys.path[:0] = [str(p) for p in paths]
|
||||||
|
roots = [ROOT, *paths, Path(sys.prefix), Path(sys.base_prefix), temporary]
|
||||||
|
if phase:
|
||||||
|
roots.append(phase / 'tests/fixtures/ops-warden-coordination-profile.json')
|
||||||
|
if netkingdom:
|
||||||
|
roots.append(netkingdom)
|
||||||
|
os.environ['WARDEN_TEST_NETKINGDOM_ROOT'] = str(netkingdom)
|
||||||
|
guard = Guard(temporary, roots)
|
||||||
|
guard.install()
|
||||||
|
# Pytest itself and collection run only after installation.
|
||||||
|
import pytest
|
||||||
|
raise SystemExit(pytest.main(['-p', 'no:cacheprovider', '--basetemp', str(temporary / 'pytest'), *arguments], plugins=[guard]))
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
if sys.argv[1:2] == ['--child']:
|
||||||
|
payload = json.loads(sys.argv[2])
|
||||||
|
temporary = Path(payload['temporary'])
|
||||||
|
sys.dont_write_bytecode = True
|
||||||
|
tempfile.tempdir = str(temporary)
|
||||||
|
sys.path.insert(0, str(ROOT / 'src'))
|
||||||
|
guard = Guard(temporary, [Path(p) for p in payload['read_roots']], child=True,
|
||||||
|
git_metadata=payload.get('git_metadata', False))
|
||||||
|
guard.install()
|
||||||
|
sys.argv = [payload.get('script', '-c'), *payload['args']]
|
||||||
|
if 'script' in payload:
|
||||||
|
runpy.run_path(payload['script'], run_name='__main__')
|
||||||
|
else:
|
||||||
|
exec(compile(payload['code'], '<reviewed-unit-child>', 'exec'), {'__name__': '__main__'})
|
||||||
|
return
|
||||||
|
if sys.argv[1:2] == ['--worker']:
|
||||||
|
temporary = Path(sys.argv[2])
|
||||||
|
netkingdom = Path(sys.argv[3]) if sys.argv[3] else None
|
||||||
|
phase = Path(sys.argv[4]) if sys.argv[4] else None
|
||||||
|
worker(temporary, sys.argv[5:], netkingdom, phase)
|
||||||
|
return
|
||||||
|
import argparse
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument('--netkingdom-root', type=Path, default=ROOT.parent / 'net-kingdom')
|
||||||
|
parser.add_argument('--phase-memory-root', type=Path, default=ROOT.parent / 'phase-memory')
|
||||||
|
options, arguments = parser.parse_known_args()
|
||||||
|
netkingdom = options.netkingdom_root.resolve() if options.netkingdom_root.is_dir() else None
|
||||||
|
phase = options.phase_memory_root.resolve() if (options.phase_memory_root / 'src').is_dir() else None
|
||||||
|
with tempfile.TemporaryDirectory(prefix='warden-unit-') as directory:
|
||||||
|
temporary = Path(directory)
|
||||||
|
home = temporary / 'home'
|
||||||
|
home.mkdir()
|
||||||
|
result = subprocess.run([sys.executable, '-I', str(SELF), '--worker', str(temporary),
|
||||||
|
str(netkingdom) if netkingdom else '', str(phase) if phase else '', *arguments],
|
||||||
|
cwd=ROOT, env=clean_environment(home, temporary))
|
||||||
|
raise SystemExit(result.returncode)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
main()
|
||||||
20
tests/fixtures/workload-coverage/provenance.json
vendored
Normal file
20
tests/fixtures/workload-coverage/provenance.json
vendored
Normal file
|
|
@ -0,0 +1,20 @@
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"repo": "rapp-core-hub",
|
||||||
|
"source": "retained public snapshot from warden-1.3-kernel-fixes-r1 test preparation",
|
||||||
|
"source_sha256": "98e81fb1116aa0ac9dc1a3e7aef9dd87c6d2110d98d46fbfca2f76eec8f3a77b",
|
||||||
|
"projection_sha256": "9ce902da0c77809f1094a634244652835b162f59682e9fa6923d8c8aa4ed4a3c"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"repo": "rapp-issue-core",
|
||||||
|
"source": "retained public snapshot from warden-1.3-kernel-fixes-r1 test preparation",
|
||||||
|
"source_sha256": "0425c9fe84e26b359b2a35fc37ef8fcb781de1aaa6229c59fcd246ee523514a5",
|
||||||
|
"projection_sha256": "a1956deecf2f0fbca44549e5ea5e8f1c4b74ebf91a9482c64c982491af0d923b"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"repo": "rapp-qonto",
|
||||||
|
"source": "retained public snapshot from warden-1.3-kernel-fixes-r1 test preparation",
|
||||||
|
"source_sha256": "6279dc2f3ab09519dffa1ceeedb2095f1182960e0106109a87aa5c0052446883",
|
||||||
|
"projection_sha256": "15aa9809e8d6a120c516be5ce9adfb72c00382140c171e791083ad117f503b3f"
|
||||||
|
}
|
||||||
|
]
|
||||||
15
tests/fixtures/workload-coverage/rapp-core-hub/declarations/rapp.yaml
vendored
Normal file
15
tests/fixtures/workload-coverage/rapp-core-hub/declarations/rapp.yaml
vendored
Normal file
|
|
@ -0,0 +1,15 @@
|
||||||
|
# Public declaration projection for the published coverage snapshot; not live evidence.
|
||||||
|
rapp_id: rapp-core-hub
|
||||||
|
workload_identity:
|
||||||
|
name: core-hub
|
||||||
|
composition:
|
||||||
|
member_repos:
|
||||||
|
- deployables:
|
||||||
|
- core-hub-api
|
||||||
|
- core-hub-migrate
|
||||||
|
- hub-core-candidate
|
||||||
|
- hub-core-candidate-migrate
|
||||||
|
- repository-classification-publisher
|
||||||
|
data_classification: confidential
|
||||||
|
criticality: high
|
||||||
|
readiness_state: verified
|
||||||
11
tests/fixtures/workload-coverage/rapp-issue-core/declarations/rapp.yaml
vendored
Normal file
11
tests/fixtures/workload-coverage/rapp-issue-core/declarations/rapp.yaml
vendored
Normal file
|
|
@ -0,0 +1,11 @@
|
||||||
|
# Public declaration projection for the published coverage snapshot; not live evidence.
|
||||||
|
rapp_id: rapp-issue-core
|
||||||
|
workload_identity:
|
||||||
|
name: issue-core
|
||||||
|
composition:
|
||||||
|
member_repos:
|
||||||
|
- deployables:
|
||||||
|
- issue-core
|
||||||
|
data_classification: confidential
|
||||||
|
criticality: high
|
||||||
|
readiness_state: verified
|
||||||
11
tests/fixtures/workload-coverage/rapp-qonto/declarations/rapp.yaml
vendored
Normal file
11
tests/fixtures/workload-coverage/rapp-qonto/declarations/rapp.yaml
vendored
Normal file
|
|
@ -0,0 +1,11 @@
|
||||||
|
# Public declaration projection for the published coverage snapshot; not live evidence.
|
||||||
|
rapp_id: rapp-qonto
|
||||||
|
workload_identity:
|
||||||
|
name: qonto
|
||||||
|
composition:
|
||||||
|
member_repos:
|
||||||
|
- deployables:
|
||||||
|
- rapp-qonto
|
||||||
|
data_classification: restricted
|
||||||
|
criticality: critical
|
||||||
|
readiness_state: verified
|
||||||
|
|
@ -2,6 +2,7 @@
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import copy
|
import copy
|
||||||
|
import os
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
@ -12,7 +13,8 @@ from warden.case_plan import CaseError, plan_case
|
||||||
from warden.cli import app
|
from warden.cli import app
|
||||||
from warden.routing.catalog import load_catalog
|
from warden.routing.catalog import load_catalog
|
||||||
|
|
||||||
LIVE_CASE = Path.home() / "net-kingdom/security-cases/forgejo-netkingdom-oidc.yaml"
|
LIVE_CASE = Path(os.environ.get("WARDEN_TEST_NETKINGDOM_ROOT",
|
||||||
|
Path(__file__).resolve().parents[2] / "net-kingdom")) / "security-cases/forgejo-netkingdom-oidc.yaml"
|
||||||
|
|
||||||
TYPE = {
|
TYPE = {
|
||||||
"apiVersion": "netkingdom.io/security-case-type/v0.1",
|
"apiVersion": "netkingdom.io/security-case-type/v0.1",
|
||||||
|
|
|
||||||
|
|
@ -2,6 +2,7 @@
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import copy
|
import copy
|
||||||
|
import os
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
@ -12,7 +13,8 @@ from warden.case_type import check_follow_through, load_type
|
||||||
from warden.cli import app
|
from warden.cli import app
|
||||||
from warden.routing.catalog import load_catalog
|
from warden.routing.catalog import load_catalog
|
||||||
|
|
||||||
LIVE_TYPE = Path.home() / "net-kingdom/security-case-types/oidc-browser-relying-party/type.yaml"
|
LIVE_TYPE = Path(os.environ.get("WARDEN_TEST_NETKINGDOM_ROOT",
|
||||||
|
Path(__file__).resolve().parents[2] / "net-kingdom")) / "security-case-types/oidc-browser-relying-party/type.yaml"
|
||||||
|
|
||||||
BASE = {
|
BASE = {
|
||||||
"apiVersion": "netkingdom.io/security-case-type/v0.1",
|
"apiVersion": "netkingdom.io/security-case-type/v0.1",
|
||||||
|
|
|
||||||
|
|
@ -273,7 +273,11 @@ class TestPepStanceMap:
|
||||||
# --- classification coverage (v0.8 §6.4 obligation 3) -------------------------
|
# --- classification coverage (v0.8 §6.4 obligation 3) -------------------------
|
||||||
|
|
||||||
def test_published_coverage_equals_measured_coverage():
|
def test_published_coverage_equals_measured_coverage():
|
||||||
"""The published figure must equal what the repo actually measures.
|
"""The published figure must equal its explicit public declaration snapshot.
|
||||||
|
|
||||||
|
This is a deterministic source check, not a claim about current estate state.
|
||||||
|
Live measurement remains scripts/report_coverage.py with an explicit estate
|
||||||
|
root; tests never treat the developer HOME as an authority or fixture.
|
||||||
|
|
||||||
ops-warden asked gate-house for §13.1's Coverage column and its figures are
|
ops-warden asked gate-house for §13.1's Coverage column and its figures are
|
||||||
that column's first entries, so their accuracy is ours to hold. The register
|
that column's first entries, so their accuracy is ours to hold. The register
|
||||||
|
|
@ -295,7 +299,7 @@ def test_published_coverage_equals_measured_coverage():
|
||||||
published = yaml.safe_load((repo / "pep-stance.yaml").read_text())[
|
published = yaml.safe_load((repo / "pep-stance.yaml").read_text())[
|
||||||
"classification_coverage"
|
"classification_coverage"
|
||||||
]
|
]
|
||||||
measured = module.measure()
|
measured = module.measure(repo / "tests/fixtures/workload-coverage")
|
||||||
|
|
||||||
for population in ("signing_targets", "routing_lanes"):
|
for population in ("signing_targets", "routing_lanes"):
|
||||||
assert published[population] == measured[population], population
|
assert published[population] == measured[population], population
|
||||||
|
|
|
||||||
92
tests/test_safe_unit_runner.py
Normal file
92
tests/test_safe_unit_runner.py
Normal file
|
|
@ -0,0 +1,92 @@
|
||||||
|
"""Exercise installed parent and nested child controls, never real credentials."""
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
import socket
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def guarded(request):
|
||||||
|
guard = getattr(request.config, '_unit_guard', None)
|
||||||
|
if guard is None:
|
||||||
|
pytest.skip('requires make test / scripts/run_unit_tests.py')
|
||||||
|
return guard
|
||||||
|
|
||||||
|
|
||||||
|
def test_parent_environment_is_allowlisted_and_home_is_temporary(guarded):
|
||||||
|
# The external regression invocation sets synthetic BAO_TOKEN/VAULT_TOKEN/HOME.
|
||||||
|
assert 'BAO_TOKEN' not in os.environ
|
||||||
|
assert 'VAULT_TOKEN' not in os.environ
|
||||||
|
assert Path(os.environ['TMPDIR']) in Path.home().parents
|
||||||
|
assert 'PYTHONPATH' not in os.environ
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize('command', [
|
||||||
|
['bao', 'token', 'lookup'], ['ssh', 'railiance01'], ['curl', 'http://127.0.0.1:8000'],
|
||||||
|
[sys.executable, '-c', 'import os; print(os.environ)'],
|
||||||
|
])
|
||||||
|
def test_unexpected_process_refused_before_launch(guarded, command):
|
||||||
|
with pytest.raises(PermissionError, match='unit isolation'):
|
||||||
|
subprocess.run(command, check=True)
|
||||||
|
|
||||||
|
|
||||||
|
def test_parent_socket_denies_even_unregistered_loopback(guarded):
|
||||||
|
with pytest.raises(PermissionError, match='unit isolation'):
|
||||||
|
socket.socket().connect(('127.0.0.1', 8000))
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize('path', ['/home/synthetic-ambient-unit-fixture/.vault-token',
|
||||||
|
'/root/.kube/config', '/proc/self/environ',
|
||||||
|
'/tmp/warden-synthetic-ambient-home/.vault-token'])
|
||||||
|
def test_ambient_paths_refused_without_opening_them(guarded, path):
|
||||||
|
with pytest.raises(PermissionError, match='unit isolation'):
|
||||||
|
Path(path).read_bytes()
|
||||||
|
|
||||||
|
|
||||||
|
def test_symlink_cannot_escape_fixture_root(guarded, tmp_path):
|
||||||
|
link = tmp_path / 'ambient'
|
||||||
|
link.symlink_to('/home/synthetic-ambient-unit-fixture/.vault-token')
|
||||||
|
with pytest.raises(PermissionError, match='unit isolation'):
|
||||||
|
link.read_bytes()
|
||||||
|
|
||||||
|
|
||||||
|
def test_child_drops_even_synthetic_token_environment(guarded, monkeypatch):
|
||||||
|
monkeypatch.setenv('BAO_TOKEN', 'synthetic-only-never-a-real-token')
|
||||||
|
monkeypatch.setenv('VAULT_TOKEN', 'synthetic-only-never-a-real-token')
|
||||||
|
monkeypatch.setenv('HOME', '/home/synthetic-ambient-unit-fixture')
|
||||||
|
result = subprocess.run([sys.executable, str(ROOT / 'tests/unit_isolation_probe.py'), 'state'],
|
||||||
|
capture_output=True, text=True, check=True, env=dict(os.environ))
|
||||||
|
assert json.loads(result.stdout) == {'bao_token_present': False,
|
||||||
|
'vault_token_present': False, 'home_in_temporary': True}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize('mode', ['socket', 'provider', 'home', 'nested'])
|
||||||
|
def test_guard_exists_in_each_allowed_python_child(guarded, mode):
|
||||||
|
result = subprocess.run([sys.executable, str(ROOT / 'tests/unit_isolation_probe.py'), mode],
|
||||||
|
capture_output=True, text=True, check=True)
|
||||||
|
assert result.stdout == 'denied\n'
|
||||||
|
|
||||||
|
|
||||||
|
def test_forged_provider_double_refused(guarded, tmp_path):
|
||||||
|
provider = tmp_path / 'bao'
|
||||||
|
provider.write_text('#!/bin/bash\necho not-the-reviewed-fixture\n')
|
||||||
|
provider.chmod(0o700)
|
||||||
|
with pytest.raises(PermissionError, match='double bytes differ'):
|
||||||
|
subprocess.run([str(provider), 'token', 'lookup'], check=True)
|
||||||
|
|
||||||
|
|
||||||
|
def test_os_open_truncation_respects_read_only_source_root(guarded, tmp_path, monkeypatch):
|
||||||
|
# Synthetic readable source outside the guard's writable subtree.
|
||||||
|
source = tmp_path / 'synthetic-source'
|
||||||
|
source.write_bytes(b'synthetic public source')
|
||||||
|
monkeypatch.setattr(guarded, 'temporary', tmp_path / 'writable')
|
||||||
|
with pytest.raises(PermissionError, match='unit isolation'):
|
||||||
|
descriptor = os.open(source, os.O_RDONLY | os.O_TRUNC)
|
||||||
|
os.close(descriptor)
|
||||||
|
assert source.read_bytes() == b'synthetic public source'
|
||||||
|
|
@ -1,5 +1,7 @@
|
||||||
"""Explicit lane-to-workload join tests (WARDEN-WP-0032 / RMGR-WP-0010-T06)."""
|
"""Explicit lane-to-workload join tests (WARDEN-WP-0032 / RMGR-WP-0010-T06)."""
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
|
||||||
import yaml
|
import yaml
|
||||||
|
|
||||||
|
|
@ -10,7 +12,8 @@ ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
|
||||||
|
|
||||||
def test_repo_catalog_uses_only_explicit_workload_references():
|
def test_repo_catalog_uses_only_explicit_workload_references():
|
||||||
report = build(ROOT / "registry/routing/catalog.yaml", Path.home())
|
report = build(ROOT / "registry/routing/catalog.yaml",
|
||||||
|
ROOT / "tests/fixtures/workload-coverage")
|
||||||
assert report["ok"] is True
|
assert report["ok"] is True
|
||||||
assert len(report["resolved"]) == 4
|
assert len(report["resolved"]) == 4
|
||||||
assert len(report["unknown"]) == 22
|
assert len(report["unknown"]) == 22
|
||||||
|
|
@ -67,3 +70,14 @@ def test_invalid_exact_deployable_resolves_unknown(tmp_path):
|
||||||
assert not report["resolved"]
|
assert not report["resolved"]
|
||||||
assert report["unknown"][0]["lane"] == "x"
|
assert report["unknown"][0]["lane"] == "x"
|
||||||
assert "deployable" in report["unknown"][0]["reason"]
|
assert "deployable" in report["unknown"][0]["reason"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_public_workload_projection_matches_provenance():
|
||||||
|
base = ROOT / 'tests/fixtures/workload-coverage'
|
||||||
|
provenance = json.loads((base / 'provenance.json').read_text())
|
||||||
|
expected = {'rapp-core-hub', 'rapp-issue-core', 'rapp-qonto'}
|
||||||
|
assert len(provenance) == len(expected)
|
||||||
|
assert {row['repo'] for row in provenance} == expected
|
||||||
|
for row in provenance:
|
||||||
|
projection = base / row['repo'] / 'declarations/rapp.yaml'
|
||||||
|
assert hashlib.sha256(projection.read_bytes()).hexdigest() == row['projection_sha256']
|
||||||
|
|
|
||||||
29
tests/unit_isolation_probe.py
Normal file
29
tests/unit_isolation_probe.py
Normal file
|
|
@ -0,0 +1,29 @@
|
||||||
|
"""Closed child regression probes; executed only under the unit runner guard."""
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
import socket
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
|
||||||
|
mode = sys.argv[1]
|
||||||
|
if mode == 'state':
|
||||||
|
print(json.dumps({'bao_token_present': 'BAO_TOKEN' in os.environ,
|
||||||
|
'vault_token_present': 'VAULT_TOKEN' in os.environ,
|
||||||
|
'home_in_temporary': Path(os.environ['TMPDIR']) in Path.home().parents}))
|
||||||
|
else:
|
||||||
|
try:
|
||||||
|
if mode == 'socket':
|
||||||
|
socket.socket().connect(('127.0.0.1', 8000))
|
||||||
|
elif mode == 'provider':
|
||||||
|
subprocess.run(['bao', 'token', 'lookup'], check=True)
|
||||||
|
elif mode == 'home':
|
||||||
|
Path('/home/synthetic-ambient-unit-fixture/.vault-token').read_text()
|
||||||
|
elif mode == 'nested':
|
||||||
|
subprocess.run([sys.executable, '-c', 'print("SYNTHETIC-SECRET")'], check=True)
|
||||||
|
else:
|
||||||
|
raise AssertionError('unknown probe')
|
||||||
|
except PermissionError:
|
||||||
|
print('denied')
|
||||||
|
else:
|
||||||
|
raise AssertionError('external operation was not refused')
|
||||||
|
|
@ -354,3 +354,41 @@ then use the preserved isolated review runner, not the ambient `make test` path.
|
||||||
No credential lookup, live investigation or rotation is authorized by this note.
|
No credential lookup, live investigation or rotation is authorized by this note.
|
||||||
T03 remains progress; contained sessions, authenticated owner provenance,
|
T03 remains progress; contained sessions, authenticated owner provenance,
|
||||||
materialization, exact type acknowledgment/pin and first native case remain open.
|
materialization, exact type acknowledgment/pin and first native case remain open.
|
||||||
|
|
||||||
|
|
||||||
|
2026-10-17 safe unit-runner source candidate: `make test` now selects an existing
|
||||||
|
interpreter and `scripts/run_unit_tests.py`, which constructs the environment and
|
||||||
|
empty temporary HOME before pytest collection, denies unexpected process/socket
|
||||||
|
use and ambient file opens, and guards the pinned BrowserInputs child too.
|
||||||
|
Closed fixture exceptions cover the synthetic desk endpoint, metadata checkers,
|
||||||
|
exact-byte contract doubles and fixed transport prints; they confer no provider
|
||||||
|
authority. Explicit public declaration fixtures replace ambient HOME source
|
||||||
|
lookups. See `docs/development/safe-unit-tests.md` and Custodian packet
|
||||||
|
`docs/changes/netkingdom-target-2026-10-17/warden-safe-unit-runner-r1`.
|
||||||
|
Synthetic token, ambient-path, forged-provider and nested-child regressions are
|
||||||
|
retained alongside the initial failed isolation runs and corrected results. This
|
||||||
|
is a source candidate for the existing safe-test handoff, not an OS sandbox or
|
||||||
|
native case acceptance. T03 remains progress; contained sessions, authenticated
|
||||||
|
owner provenance, materialization, exact type acknowledgment/pin and first native
|
||||||
|
case remain open. No credential lookup, live investigation or rotation occurred.
|
||||||
|
|
||||||
|
Candidate validation: ordinary `make test` under synthetic ambient token/HOME
|
||||||
|
controls passes 783 tests with four integration-marked tests deselected. Affected
|
||||||
|
focused checks pass 117 tests; new runner code/regressions pass Ruff. Retained
|
||||||
|
initial failures exposed fixture dependencies and were corrected, not suppressed.
|
||||||
|
|
||||||
|
|
||||||
|
2026-10-17 root follow-through: the original safe-runner source candidate was
|
||||||
|
independently rejected for O_TRUNC open-write misclassification and unchecked
|
||||||
|
public fixture projection hashes. Both counterexamples are retained. The minimal
|
||||||
|
correction treats truncating/appending opens as writes and adds the closed
|
||||||
|
projection-byte provenance regression. Root inspected the two exact corrections
|
||||||
|
and ran ordinary make test on stable corrected source with only synthetic parent
|
||||||
|
HOME/token/Python/pytest sentinels: 785 passed, four deselected. No real credential
|
||||||
|
or native provider action occurred. Initial author failures and rejected review
|
||||||
|
are preserved in the Custodian warden-safe-unit-runner-r1 and
|
||||||
|
warden-safe-unit-runner-review-r1 packets, with root disposition/full result in
|
||||||
|
warden-safe-unit-integration-r1. This completes the known trusted-unit-entry
|
||||||
|
accident-prevention follow-through; direct pytest bypass, native/non-open/preopened
|
||||||
|
file-descriptor limits remain explicit. No OS sandbox, isolated agent assignment,
|
||||||
|
production grant or native T03 acceptance follows. T03 stays progress.
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue