feat: import governed sandbox commits and enforce native CLI limits
Some checks are pending
Governed runtime contract / contract (push) Waiting to run
Some checks are pending
Governed runtime contract / contract (push) Waiting to run
Assistant: codex Assistant-Model: gpt-5.6-luna Assistant-Session: 01a07ff8-19d0-7820-b4d0-1353833cb7fc
This commit is contained in:
parent
1429db5ad4
commit
c63caf5568
16 changed files with 1236 additions and 7 deletions
|
|
@ -13,13 +13,13 @@
|
|||
"version": "0.1.0"
|
||||
},
|
||||
{
|
||||
"commit": "02b29af9ca87867e0d398c06139d771a19728b70",
|
||||
"commit": "fb57c0228f41021a9ad30dae3f18aed8335b2445",
|
||||
"distribution": "glas-harness",
|
||||
"source": "../glas-harness",
|
||||
"version": "0.1.0"
|
||||
},
|
||||
{
|
||||
"commit": "5c760100264828865c45a7e4e199d43e4603d57a",
|
||||
"commit": "3e49a98a0e2c4a64539a5ccd674be8cd67ac9845",
|
||||
"distribution": "sandboxer",
|
||||
"source": "../sand-boxer",
|
||||
"version": "0.0.0"
|
||||
|
|
|
|||
40
docs/evidence/2026-09-09-runtime-transfer.json
Normal file
40
docs/evidence/2026-09-09-runtime-transfer.json
Normal file
|
|
@ -0,0 +1,40 @@
|
|||
{
|
||||
"schema": "hfact.runtime-transfer-verification.v1",
|
||||
"date": "2026-09-09",
|
||||
"rein_tests": 290,
|
||||
"glas_tests": 133,
|
||||
"actual_bwrap_owner_execution": true,
|
||||
"task_authoring": "deterministic fixture",
|
||||
"queue": "fixture client; heartbeat/close/replay exercised",
|
||||
"artifact_tests": [
|
||||
"exact commit retained after sandbox destruction",
|
||||
"granted path acceptance",
|
||||
"one commit ancestry",
|
||||
"corrupt and truncated bundle",
|
||||
"symlink refusal",
|
||||
"expanded object limit",
|
||||
"wrong head",
|
||||
"missing capture",
|
||||
"teardown failure",
|
||||
"lease loss",
|
||||
"source changed externally",
|
||||
"ignored cache excluded"
|
||||
],
|
||||
"native_controls": [
|
||||
"profile USD and turn limits reach Claude CLI",
|
||||
"version preflight",
|
||||
"strict terminal cost/token/turn accounting",
|
||||
"limit exhaustion refuses success"
|
||||
],
|
||||
"provider_requests": 0,
|
||||
"paid_factory_attempts": 0,
|
||||
"profile_promoted": false,
|
||||
"protected_runtime_rebuilt": false,
|
||||
"remaining": [
|
||||
"durable daily/total budget reservations and uncertain outcome recovery",
|
||||
"provider cap semantics and conservative EUR treatment",
|
||||
"matching protected runtime pins and exact factory identity/placement",
|
||||
"G0 operating admission",
|
||||
"real model G1 and natural Railiance queue G2"
|
||||
]
|
||||
}
|
||||
54
docs/sandbox-artifact-return.md
Normal file
54
docs/sandbox-artifact-return.md
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
# Sandbox commit return and native CLI limits
|
||||
|
||||
The worker owns the original repository transaction and repository grant. Glas
|
||||
owns sandbox lifetime. For a granted profiled run, the worker supplies a trusted
|
||||
process-local capture callback. Glas invokes it after successful rein cleanup
|
||||
and before sandbox destruction. Callback failure is `artifact_capture` failure;
|
||||
teardown still runs. Artifact bytes are never serialized into gateway evidence.
|
||||
|
||||
The initial transfer supports owner-mediated local namespaces and exactly one
|
||||
local Git commit. It captures at most 128 KiB of bundle data, verifies the exact
|
||||
baseline/head and ancestry in a disposable checkout, limits new objects to
|
||||
1,024 / 2 MiB, rejects symlinks/gitlinks and out-of-grant paths, and fast-forwards
|
||||
the original checkout only after successful teardown and another lease/baseline
|
||||
check. Ignored runtime caches are discarded with the sandbox. Source changes,
|
||||
lease loss, missing or corrupt capture, and failed teardown refuse import.
|
||||
|
||||
Acceptance and external metrics remain in the outer worker. The inner Glas task
|
||||
continues to use `--no-metrics` without an inner repository grant. Terminal-close
|
||||
replay uses the existing durable outbox and does not run the workload again.
|
||||
The repository lock is cooperative; unexpected external changes are detected,
|
||||
not rolled back. A crash or lease loss at the final mutation/receipt boundary
|
||||
still requires the existing recovery process to classify the resulting commit.
|
||||
|
||||
## Native limits
|
||||
|
||||
Glas profile limits `max_budget_usd` and `max_turns` are carried in its generated
|
||||
TaskSpec into the agentic Claude adapter. A supplied task file cannot bypass
|
||||
these profile controls. Positive finite USD amounts and positive integer turn
|
||||
limits are required. A controlled run verifies Claude Code >= 2.1.217, supplies
|
||||
`--max-budget-usd` / `--max-turns`, and requires one successful terminal JSON
|
||||
result with valid cost, turn and token accounting. Limit exhaustion, missing
|
||||
accounting and reported overruns cannot produce a successful run. Bounded USD
|
||||
cost follows runner, Glas evidence and the outer close evidence; result text and
|
||||
artifact bytes do not enter the close outbox. Runs without these optional controls
|
||||
retain their legacy behavior, including its incomplete token accounting.
|
||||
|
||||
This is native per-run control and accounting, not a factory spend grant or a
|
||||
hard EUR ceiling. The [Claude CLI contract](https://code.claude.com/docs/en/cli-reference)
|
||||
is the upstream reference. Before paid admission, verify the protected binary's
|
||||
actual enforcement/overrun semantics, reserve daily/total worst-case costs
|
||||
outside the sandbox, define conservative EUR/USD treatment, and retain unknown
|
||||
outcomes as held reservations. No profile was enabled and no paid request was
|
||||
made by this change. Deploy a newly verified runtime containing the matching
|
||||
rein/Glas source pins before exercising the real model path.
|
||||
|
||||
## Verification
|
||||
|
||||
`tests/test_repository_artifact.py` runs real Git export/import with adversarial
|
||||
artifacts. `REIN_REAL_BWRAP=1 pytest tests/test_repository_artifact_bwrap.py -q`
|
||||
adds actual sand-boxer owner execution, Glas lifecycle and worker acceptance,
|
||||
including response-lost close replay. Its queue and task authoring are fixtures;
|
||||
it proves neither a natural Activity Core claim nor model/provider admission.
|
||||
`tests/test_native_limits.py` exercises control propagation, terminal accounting,
|
||||
invalid/exhausted results and refusal of older CLI versions without inference.
|
||||
|
|
@ -24,6 +24,7 @@ one aggregate `LLMResponse` at the end for interface compatibility.
|
|||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
|
@ -32,6 +33,7 @@ from typing import Any, Callable
|
|||
from llm_connect.claude_code import ClaudeCodeAdapter
|
||||
from llm_connect.exceptions import LLMSubprocessError, LLMTimeoutError
|
||||
from llm_connect.models import LLMResponse, RunConfig
|
||||
from rein_aharness.native_limits import NativeLimitError, terminal_accounting, validate_limits
|
||||
|
||||
from rein_aharness.execution_cancel import (
|
||||
ExecutionCancel,
|
||||
|
|
@ -99,6 +101,8 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
|||
self._workdir = workdir
|
||||
self._on_tool_event = on_tool_event
|
||||
self._cancel = cancel
|
||||
self._native_config = (None, None)
|
||||
self._native_cli_checked = False
|
||||
if isinstance(tool_profile, ToolProfile):
|
||||
self._profile = tool_profile
|
||||
else:
|
||||
|
|
@ -109,6 +113,10 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
|||
return self._profile
|
||||
|
||||
def _build_command(self, config: RunConfig) -> list[str]:
|
||||
budget = config.model_params.get("max_budget_usd")
|
||||
turns = config.model_params.get("max_turns")
|
||||
validate_limits(budget, turns)
|
||||
self._native_config = (budget, turns)
|
||||
cmd = [
|
||||
self._cli_path,
|
||||
"--print",
|
||||
|
|
@ -119,6 +127,12 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
|||
]
|
||||
if self._on_tool_event is not None:
|
||||
cmd += ["--output-format", "stream-json", "--include-hook-events", "--verbose"]
|
||||
elif budget is not None or turns is not None:
|
||||
cmd += ["--output-format", "json"]
|
||||
if budget is not None:
|
||||
cmd += ["--max-budget-usd", str(budget)]
|
||||
if turns is not None:
|
||||
cmd += ["--max-turns", str(turns)]
|
||||
if self._model:
|
||||
cmd.extend(["--model", self._model])
|
||||
return cmd
|
||||
|
|
@ -126,12 +140,23 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
|||
def execute_prompt(self, prompt: str, config: RunConfig) -> LLMResponse:
|
||||
self._preflight_budget(config)
|
||||
cmd = self._build_command(config)
|
||||
if self._native_config != (None, None) and not self._native_cli_checked:
|
||||
check = subprocess.run([self._cli_path, "--version"], capture_output=True, text=True, timeout=10, cwd=self._workdir)
|
||||
version = re.match(r"(\d+)\.(\d+)\.(\d+)(?:\s|$)", check.stdout.strip())
|
||||
if check.returncode or not version or tuple(map(int, version.groups())) < (2, 1, 217):
|
||||
raise NativeLimitError("native limits require verified Claude Code 2.1.217 or newer")
|
||||
self._native_cli_checked = True
|
||||
timeout = config.timeout_seconds or self._config.timeout_seconds
|
||||
if self._on_tool_event is not None:
|
||||
response = self._execute_streaming(cmd, prompt, timeout)
|
||||
else:
|
||||
response = self._execute_blocking(cmd, prompt, timeout)
|
||||
self._consume_budget(config, response)
|
||||
try:
|
||||
self._consume_budget(config, response)
|
||||
except Exception as exc:
|
||||
if self._native_config != (None, None):
|
||||
raise NativeLimitError("controlled Claude run failed token accounting", cost_usd=response.metadata.get("cost_usd")) from exc
|
||||
raise
|
||||
return response
|
||||
|
||||
def _execute_blocking(self, cmd: list[str], prompt: str, timeout: int) -> LLMResponse:
|
||||
|
|
@ -144,7 +169,19 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
|||
cwd=self._workdir,
|
||||
)
|
||||
stdout, stderr = self._wait_for_process(proc, prompt, timeout)
|
||||
usage, cost = {}, None
|
||||
if self._native_config != (None, None):
|
||||
try:
|
||||
envelope = json.loads(stdout)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise NativeLimitError("controlled Claude run returned invalid terminal JSON") from exc
|
||||
usage, cost = terminal_accounting(envelope, max_budget_usd=self._native_config[0], max_turns=self._native_config[1])
|
||||
stdout = envelope.get("result", "")
|
||||
if not isinstance(stdout, str):
|
||||
raise NativeLimitError("controlled Claude run returned invalid result text", cost_usd=cost)
|
||||
if proc.returncode != 0:
|
||||
if self._native_config != (None, None):
|
||||
raise NativeLimitError("controlled Claude CLI exited unsuccessfully", cost_usd=cost)
|
||||
raise LLMSubprocessError(
|
||||
f"claude CLI exited with code {proc.returncode}",
|
||||
return_code=proc.returncode,
|
||||
|
|
@ -153,9 +190,10 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
|||
return LLMResponse(
|
||||
content=stdout,
|
||||
model=self._model or "claude-code-cli",
|
||||
usage={},
|
||||
usage=usage,
|
||||
finish_reason="stop",
|
||||
metadata={
|
||||
"cost_usd": cost,
|
||||
"provider": "claude-code-agentic",
|
||||
"cli_path": self._cli_path,
|
||||
"workdir": str(self._workdir),
|
||||
|
|
@ -174,6 +212,7 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
|||
)
|
||||
text_parts: list[str] = []
|
||||
tool_event_count = 0
|
||||
terminal_results = []
|
||||
|
||||
def reader() -> None:
|
||||
nonlocal tool_event_count
|
||||
|
|
@ -186,6 +225,8 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
|||
event = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
if isinstance(event, dict) and event.get("type") == "result":
|
||||
terminal_results.append(event)
|
||||
self._handle_stream_event(event, text_parts)
|
||||
if _is_tool_event(event):
|
||||
tool_event_count += 1
|
||||
|
|
@ -200,7 +241,14 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
|||
reader_thread.join(timeout=5)
|
||||
stderr = proc.stderr.read() if proc.stderr else ""
|
||||
|
||||
usage, cost = {}, None
|
||||
if self._native_config != (None, None):
|
||||
if reader_thread.is_alive() or len(terminal_results) != 1:
|
||||
raise NativeLimitError("controlled Claude stream requires one complete terminal result")
|
||||
usage, cost = terminal_accounting(terminal_results[0], max_budget_usd=self._native_config[0], max_turns=self._native_config[1])
|
||||
if returncode != 0:
|
||||
if self._native_config != (None, None):
|
||||
raise NativeLimitError("controlled Claude CLI exited unsuccessfully", cost_usd=cost)
|
||||
raise LLMSubprocessError(
|
||||
f"claude CLI exited with code {returncode}",
|
||||
return_code=returncode,
|
||||
|
|
@ -210,7 +258,7 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
|||
return LLMResponse(
|
||||
content="".join(text_parts),
|
||||
model=self._model or "claude-code-cli",
|
||||
usage={},
|
||||
usage=usage,
|
||||
finish_reason="stop",
|
||||
metadata={
|
||||
"provider": "claude-code-agentic",
|
||||
|
|
@ -218,6 +266,7 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
|||
"workdir": str(self._workdir),
|
||||
"tool_profile": self._profile.name,
|
||||
"tool_event_count": tool_event_count,
|
||||
"cost_usd": cost,
|
||||
},
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -544,6 +544,7 @@ def _process_profiled_run_active(
|
|||
client.config,
|
||||
report_to_hub=report_to_hub,
|
||||
cancel=cancel,
|
||||
transaction=active_tx,
|
||||
)
|
||||
safe_evidence = normalise_execution_evidence_for_close(
|
||||
gateway_result["evidence"]
|
||||
|
|
|
|||
|
|
@ -300,6 +300,7 @@ def _cmd_run(args: argparse.Namespace) -> int:
|
|||
"tool_profile": result.tool_profile,
|
||||
"budget_tokens": result.budget_tokens,
|
||||
"tokens_spent": result.tokens_spent,
|
||||
"cost_usd": result.cost_usd,
|
||||
"execution_time_s": round(result.execution_time_s, 3),
|
||||
"reason": result.reason,
|
||||
"model": result.model,
|
||||
|
|
|
|||
|
|
@ -51,6 +51,7 @@ _EVIDENCE_STRING_FIELDS = (
|
|||
_EVIDENCE_NUMBER_FIELDS = (
|
||||
"duration_s",
|
||||
"tokens_spent",
|
||||
"cost_usd",
|
||||
"token_budget",
|
||||
"tool_events_count",
|
||||
)
|
||||
|
|
@ -142,6 +143,7 @@ def execute_profiled_run(
|
|||
request_factory: Callable[..., Any] | None = None,
|
||||
gateway: Callable[[Any], Any] | None = None,
|
||||
cancel: ExecutionCancel | None = None,
|
||||
transaction: Any = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Invoke Glas and return its complete JSON-compatible GatewayResult."""
|
||||
guard = resolve_cancel(cancel)
|
||||
|
|
@ -159,9 +161,14 @@ def execute_profiled_run(
|
|||
request_factory = request_factory or ExecutionRequest
|
||||
gateway = gateway or run_execution
|
||||
|
||||
transfer = None
|
||||
if run.repository_grant is not None:
|
||||
from rein_aharness.repository_artifact import RepositoryArtifactTransfer
|
||||
transfer = RepositoryArtifactTransfer(transaction, run.repository_grant, cancel=guard)
|
||||
|
||||
try:
|
||||
request = request_factory(**_request_kwargs(run, config, report_to_hub))
|
||||
result = gateway(request)
|
||||
result = gateway(request, artifact_capture=transfer.capture) if transfer else gateway(request)
|
||||
raw = result.model_dump(mode="json") if hasattr(result, "model_dump") else result
|
||||
except ExecutionCancelled:
|
||||
raise
|
||||
|
|
@ -178,4 +185,10 @@ def execute_profiled_run(
|
|||
raise GlasExecutionError("Glas gateway returned an invalid GatewayResult")
|
||||
if not isinstance(raw.get("evidence"), dict):
|
||||
raise GlasExecutionError("Glas GatewayResult is missing execution evidence")
|
||||
if transfer is not None and raw["ok"]:
|
||||
evidence = raw["evidence"]
|
||||
if evidence.get("session_cleanup") != "succeeded" or evidence.get("sandbox_destroy") != "succeeded":
|
||||
raise GlasExecutionError("artifact import requires confirmed session cleanup and sandbox teardown")
|
||||
imported = transfer.import_after_teardown(evidence.get("commit_sha"))
|
||||
raw["artifact_import"] = imported
|
||||
return raw
|
||||
|
|
|
|||
90
rein_aharness/native_limits.py
Normal file
90
rein_aharness/native_limits.py
Normal file
|
|
@ -0,0 +1,90 @@
|
|||
"""Claude CLI controls and bounded terminal accounting, not a spend admission grant.
|
||||
|
||||
Daily/total reservation and provider cap semantics remain the operating owner's
|
||||
responsibility. A post-run cost check cannot undo an already charged request.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
import math
|
||||
from typing import Any
|
||||
|
||||
|
||||
def validate_limits(max_budget_usd: Any, max_turns: Any) -> None:
|
||||
if max_budget_usd is not None and (
|
||||
isinstance(max_budget_usd, bool)
|
||||
or not isinstance(max_budget_usd, (int, float))
|
||||
or not math.isfinite(max_budget_usd)
|
||||
or max_budget_usd <= 0
|
||||
):
|
||||
raise ValueError("max_budget_usd must be a finite positive USD amount")
|
||||
if max_turns is not None and (
|
||||
isinstance(max_turns, bool) or not isinstance(max_turns, int) or max_turns <= 0
|
||||
):
|
||||
raise ValueError("max_turns must be a positive integer")
|
||||
|
||||
|
||||
class NativeLimitError(RuntimeError):
|
||||
def __init__(self, message: str, *, cost_usd: float | None = None) -> None:
|
||||
super().__init__(message)
|
||||
self.cost_usd = cost_usd
|
||||
|
||||
|
||||
def terminal_accounting(
|
||||
result: Any, *, max_budget_usd: float | None, max_turns: int | None
|
||||
) -> tuple[dict, float]:
|
||||
if not isinstance(result, dict) or result.get("type") != "result":
|
||||
raise NativeLimitError(
|
||||
"controlled Claude run did not return terminal accounting"
|
||||
)
|
||||
cost = result.get("total_cost_usd")
|
||||
if (
|
||||
isinstance(cost, bool)
|
||||
or not isinstance(cost, (int, float))
|
||||
or not math.isfinite(cost)
|
||||
or cost < 0
|
||||
):
|
||||
raise NativeLimitError("controlled Claude run returned invalid cost accounting")
|
||||
cost = float(cost)
|
||||
if result.get("subtype") != "success" or result.get("is_error") is not False:
|
||||
raise NativeLimitError(
|
||||
"controlled Claude run did not complete successfully", cost_usd=cost
|
||||
)
|
||||
if max_budget_usd is not None and cost > max_budget_usd:
|
||||
raise NativeLimitError(
|
||||
"controlled Claude run exceeded its native USD limit", cost_usd=cost
|
||||
)
|
||||
turns = result.get("num_turns")
|
||||
if (
|
||||
isinstance(turns, bool)
|
||||
or not isinstance(turns, int)
|
||||
or turns < 0
|
||||
or (max_turns is not None and turns > max_turns)
|
||||
):
|
||||
raise NativeLimitError(
|
||||
"controlled Claude run returned invalid turn accounting", cost_usd=cost
|
||||
)
|
||||
usage = result.get("usage")
|
||||
if not isinstance(usage, dict):
|
||||
raise NativeLimitError(
|
||||
"controlled Claude run returned no token accounting", cost_usd=cost
|
||||
)
|
||||
counts = {}
|
||||
for name in (
|
||||
"input_tokens",
|
||||
"output_tokens",
|
||||
"cache_creation_input_tokens",
|
||||
"cache_read_input_tokens",
|
||||
):
|
||||
value = usage.get(name, 0 if name.startswith("cache_") else None)
|
||||
if isinstance(value, bool) or not isinstance(value, int) or value < 0:
|
||||
raise NativeLimitError(
|
||||
"controlled Claude run returned invalid token accounting", cost_usd=cost
|
||||
)
|
||||
counts[name] = value
|
||||
return {
|
||||
"prompt_tokens": counts["input_tokens"]
|
||||
+ counts["cache_creation_input_tokens"]
|
||||
+ counts["cache_read_input_tokens"],
|
||||
"completion_tokens": counts["output_tokens"],
|
||||
"total_tokens": sum(counts.values()),
|
||||
}, cost
|
||||
320
rein_aharness/repository_artifact.py
Normal file
320
rein_aharness/repository_artifact.py
Normal file
|
|
@ -0,0 +1,320 @@
|
|||
"""Bounded sandbox Git return into an already-owned repository transaction.
|
||||
|
||||
Artifact bytes stay process-local and are discarded on any gateway/teardown
|
||||
failure. They never enter GatewayResult, Hub, metrics or the close outbox.
|
||||
The initial contract supports one local commit, matching the factory pilot.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import binascii
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from rein_aharness.execution_cancel import ExecutionCancel, resolve_cancel
|
||||
from rein_aharness.repository_grant import RepositoryGrant
|
||||
from rein_aharness.repository_transaction import (
|
||||
RepositoryTransaction,
|
||||
RepositoryTransactionError,
|
||||
)
|
||||
|
||||
MAX_BUNDLE_BYTES = 128 * 1024
|
||||
MAX_NEW_OBJECT_BYTES = 2 * 1024 * 1024
|
||||
_OID = re.compile(r"[0-9a-f]{40}|[0-9a-f]{64}")
|
||||
|
||||
|
||||
class RepositoryArtifactError(RepositoryTransactionError):
|
||||
"""A sandbox result cannot be safely imported under this transaction."""
|
||||
|
||||
|
||||
# Executed inside the selected sandbox by its owner, with no host path access.
|
||||
# The owner transport separately caps output and wall time. No binary or source
|
||||
# content is placed in command arguments or exception messages.
|
||||
_EXPORT = r"""
|
||||
import base64, json, pathlib, subprocess, sys, tempfile
|
||||
repo, baseline, expected, limit = sys.argv[1:]
|
||||
limit = int(limit)
|
||||
def git(*args):
|
||||
p = subprocess.run(["git", "-C", repo, "-c", "core.hooksPath=/dev/null", *args], capture_output=True, timeout=20)
|
||||
if p.returncode: raise RuntimeError("sandbox artifact Git operation refused")
|
||||
return p.stdout
|
||||
if git("rev-parse", "HEAD").decode().strip() != expected:
|
||||
raise RuntimeError("sandbox artifact head mismatch")
|
||||
if git("status", "--porcelain=v1", "--untracked-files=all").strip():
|
||||
raise RuntimeError("sandbox artifact checkout is not clean")
|
||||
if git("rev-list", "--count", baseline + "..HEAD").strip() != b"1":
|
||||
raise RuntimeError("sandbox artifact must contain one commit")
|
||||
git("merge-base", "--is-ancestor", baseline, "HEAD")
|
||||
with tempfile.TemporaryDirectory(prefix="rein-artifact-") as d:
|
||||
path = pathlib.Path(d) / "return.bundle"
|
||||
git("bundle", "create", str(path), baseline + "..HEAD")
|
||||
if path.stat().st_size > limit: raise RuntimeError("sandbox artifact too large")
|
||||
payload = path.read_bytes()
|
||||
print(json.dumps({"version": "1", "baseline": baseline, "head": expected,
|
||||
"bundle": base64.b64encode(payload).decode("ascii")}))
|
||||
"""
|
||||
|
||||
|
||||
def _git(repo: Path, *args: str) -> bytes:
|
||||
# Do not permit inherited Git directory/object/config overrides to redirect
|
||||
# the transfer. All transport is between owner-selected local paths.
|
||||
env = {k: v for k, v in os.environ.items() if not k.startswith("GIT_")}
|
||||
env.update(
|
||||
GIT_CONFIG_NOSYSTEM="1",
|
||||
GIT_CONFIG_GLOBAL=os.devnull,
|
||||
GIT_TERMINAL_PROMPT="0",
|
||||
GIT_NO_REPLACE_OBJECTS="1",
|
||||
)
|
||||
try:
|
||||
proc = subprocess.run(
|
||||
[
|
||||
"git",
|
||||
"-C",
|
||||
str(repo),
|
||||
"-c",
|
||||
"core.hooksPath=/dev/null",
|
||||
"-c",
|
||||
"core.fsmonitor=false",
|
||||
"-c",
|
||||
"protocol.allow=never",
|
||||
"-c",
|
||||
"protocol.file.allow=always",
|
||||
*args,
|
||||
],
|
||||
env=env,
|
||||
capture_output=True,
|
||||
timeout=30,
|
||||
)
|
||||
except (OSError, subprocess.TimeoutExpired) as exc:
|
||||
raise RepositoryArtifactError(
|
||||
"artifact Git operation unavailable or timed out"
|
||||
) from exc
|
||||
if proc.returncode:
|
||||
raise RepositoryArtifactError("artifact Git validation or import refused")
|
||||
return proc.stdout
|
||||
|
||||
|
||||
class RepositoryArtifactTransfer:
|
||||
"""Trusted worker-owned capture; source mutation waits for successful teardown."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
transaction: RepositoryTransaction,
|
||||
grant: RepositoryGrant,
|
||||
*,
|
||||
cancel: ExecutionCancel | None = None,
|
||||
) -> None:
|
||||
if (
|
||||
not isinstance(transaction, RepositoryTransaction)
|
||||
or not transaction.locked
|
||||
or transaction.baseline is None
|
||||
):
|
||||
raise RepositoryArtifactError(
|
||||
"artifact transfer requires the active repository transaction"
|
||||
)
|
||||
if grant.min_commits != 1 or grant.max_commits != 1 or grant.publish:
|
||||
raise RepositoryArtifactError(
|
||||
"artifact transfer v1 requires exactly one local commit"
|
||||
)
|
||||
self.transaction = transaction
|
||||
self.grant = grant
|
||||
self.cancel = resolve_cancel(cancel)
|
||||
self._bundle: bytes | None = None
|
||||
self._head: str | None = None
|
||||
|
||||
def _check(self) -> None:
|
||||
if self.cancel is not None:
|
||||
self.cancel.check()
|
||||
self.transaction.assert_baseline_unchanged()
|
||||
|
||||
def capture(self, sandbox: Any, summary: Any) -> None:
|
||||
"""Called by Glas after session cleanup, before sandbox destruction."""
|
||||
from glas_harness.transport import transport_from_sandbox
|
||||
|
||||
self._check()
|
||||
if self._bundle is not None:
|
||||
raise RepositoryArtifactError("artifact capture was already completed")
|
||||
head = summary.commit_sha
|
||||
if not isinstance(head, str) or not _OID.fullmatch(head):
|
||||
raise RepositoryArtifactError(
|
||||
"sandbox did not report an exact commit identity"
|
||||
)
|
||||
transport = transport_from_sandbox(sandbox)
|
||||
if transport.kind != "local_namespace":
|
||||
raise RepositoryArtifactError(
|
||||
"artifact transfer v1 requires bounded sandbox owner execution"
|
||||
)
|
||||
result = transport.run(
|
||||
[
|
||||
"python3",
|
||||
"-c",
|
||||
_EXPORT,
|
||||
transport.workspace,
|
||||
self.transaction.baseline.head,
|
||||
head,
|
||||
str(MAX_BUNDLE_BYTES),
|
||||
],
|
||||
timeout=30,
|
||||
)
|
||||
if result.returncode:
|
||||
raise RepositoryArtifactError("sandbox artifact export refused")
|
||||
if len(result.stdout) > (MAX_BUNDLE_BYTES * 4 // 3) + 1024:
|
||||
raise RepositoryArtifactError("sandbox artifact exceeds transport limit")
|
||||
try:
|
||||
payload = json.loads(result.stdout)
|
||||
if (
|
||||
not isinstance(payload, dict)
|
||||
or set(payload) != {"version", "baseline", "head", "bundle"}
|
||||
or payload["version"] != "1"
|
||||
or payload["baseline"] != self.transaction.baseline.head
|
||||
or payload["head"] != head
|
||||
):
|
||||
raise ValueError("artifact envelope mismatch")
|
||||
bundle = base64.b64decode(payload["bundle"], validate=True)
|
||||
if not bundle or len(bundle) > MAX_BUNDLE_BYTES:
|
||||
raise ValueError("artifact byte limit")
|
||||
except (ValueError, TypeError, binascii.Error) as exc:
|
||||
raise RepositoryArtifactError(
|
||||
"sandbox artifact envelope is invalid"
|
||||
) from exc
|
||||
self._bundle, self._head = bundle, head
|
||||
self._check()
|
||||
|
||||
def import_after_teardown(self, reported_head: str | None) -> dict[str, Any]:
|
||||
"""Validate in a disposable checkout, then fast-forward under the live lease."""
|
||||
self._check()
|
||||
if self._bundle is None or self._head is None or reported_head != self._head:
|
||||
raise RepositoryArtifactError(
|
||||
"successful gateway did not return the captured artifact"
|
||||
)
|
||||
baseline = self.transaction.baseline
|
||||
with tempfile.TemporaryDirectory(prefix="rein-artifact-verify-") as temporary:
|
||||
root = Path(temporary)
|
||||
bundle_path = root / "return.bundle"
|
||||
bundle_path.write_bytes(self._bundle)
|
||||
bundle_path.chmod(0o600)
|
||||
staging = root / "checkout"
|
||||
_git(
|
||||
root,
|
||||
"clone",
|
||||
"--quiet",
|
||||
"--no-local",
|
||||
"--no-hardlinks",
|
||||
"--no-checkout",
|
||||
"--",
|
||||
str(baseline.repo_root),
|
||||
str(staging),
|
||||
)
|
||||
_git(staging, "checkout", "--quiet", "--detach", baseline.head)
|
||||
with RepositoryTransaction(staging, state_dir=root / "state") as inspection:
|
||||
_git(staging, "bundle", "verify", str(bundle_path))
|
||||
advertised = (
|
||||
_git(staging, "bundle", "list-heads", str(bundle_path))
|
||||
.decode()
|
||||
.strip()
|
||||
)
|
||||
if advertised != self._head + " HEAD":
|
||||
raise RepositoryArtifactError(
|
||||
"artifact must advertise only the expected HEAD"
|
||||
)
|
||||
_git(
|
||||
staging,
|
||||
"fetch",
|
||||
"--quiet",
|
||||
"--no-tags",
|
||||
"--no-write-fetch-head",
|
||||
str(bundle_path),
|
||||
"HEAD",
|
||||
)
|
||||
if (
|
||||
_git(
|
||||
staging, "rev-list", "--count", f"{baseline.head}..{self._head}"
|
||||
).strip()
|
||||
!= b"1"
|
||||
):
|
||||
raise RepositoryArtifactError(
|
||||
"artifact contains an unexpected commit count"
|
||||
)
|
||||
_git(staging, "merge-base", "--is-ancestor", baseline.head, self._head)
|
||||
# Reject symlinks and gitlinks before checking out untrusted data.
|
||||
changes = _git(
|
||||
staging,
|
||||
"diff",
|
||||
"--raw",
|
||||
"--no-renames",
|
||||
"-z",
|
||||
baseline.head,
|
||||
self._head,
|
||||
"--",
|
||||
).split(b"\0")
|
||||
for header in changes[::2]:
|
||||
if header and header.split()[1] not in (
|
||||
b"100644",
|
||||
b"100755",
|
||||
b"000000",
|
||||
):
|
||||
raise RepositoryArtifactError(
|
||||
"artifact changes an unsupported file mode"
|
||||
)
|
||||
oids = _git(
|
||||
staging,
|
||||
"rev-list",
|
||||
"--objects",
|
||||
"--no-object-names",
|
||||
f"{baseline.head}..{self._head}",
|
||||
).splitlines()
|
||||
if len(oids) > 1024:
|
||||
raise RepositoryArtifactError(
|
||||
"artifact contains too many new objects"
|
||||
)
|
||||
total = 0
|
||||
for oid in oids:
|
||||
total += int(_git(staging, "cat-file", "-s", oid.decode()).strip())
|
||||
if total > MAX_NEW_OBJECT_BYTES:
|
||||
raise RepositoryArtifactError(
|
||||
"artifact expands beyond the accepted byte limit"
|
||||
)
|
||||
_git(
|
||||
staging, "merge", "--ff-only", "--no-edit", "--no-stat", self._head
|
||||
)
|
||||
inspection.validate_acceptance(self.grant.acceptance_policy())
|
||||
self._check()
|
||||
_git(
|
||||
baseline.repo_root,
|
||||
"fetch",
|
||||
"--quiet",
|
||||
"--no-tags",
|
||||
"--no-write-fetch-head",
|
||||
str(bundle_path),
|
||||
"HEAD",
|
||||
)
|
||||
self._check()
|
||||
_git(
|
||||
baseline.repo_root,
|
||||
"merge",
|
||||
"--ff-only",
|
||||
"--no-edit",
|
||||
"--no-stat",
|
||||
self._head,
|
||||
)
|
||||
if self.cancel is not None:
|
||||
self.cancel.check()
|
||||
accepted = self.transaction.validate_acceptance(
|
||||
self.grant.acceptance_policy()
|
||||
)
|
||||
evidence = {
|
||||
"version": "1",
|
||||
"sha256": hashlib.sha256(self._bundle).hexdigest(),
|
||||
"bytes": len(self._bundle),
|
||||
"head": accepted.head,
|
||||
"imported": True,
|
||||
}
|
||||
self._bundle = None
|
||||
return evidence
|
||||
|
|
@ -296,6 +296,15 @@ class RepositoryTransaction:
|
|||
evidence["acceptance"] = self.acceptance.evidence()
|
||||
return evidence
|
||||
|
||||
def assert_baseline_unchanged(self) -> None:
|
||||
"""Refuse artifact import if the locked source no longer matches admission."""
|
||||
if not self.locked or self.baseline is None:
|
||||
raise RepositoryTransactionError("baseline verification requires an active transaction")
|
||||
baseline = self.baseline
|
||||
current = _capture_baseline(baseline.repo_root, baseline.git_common_dir, baseline.repo_id)
|
||||
if current != baseline:
|
||||
raise RepositoryAcceptanceError("baseline-changed", "source changed before artifact import")
|
||||
|
||||
def validate_acceptance(
|
||||
self,
|
||||
policy: RepositoryAcceptancePolicy,
|
||||
|
|
|
|||
|
|
@ -30,6 +30,7 @@ from rein_aharness.repository_transaction import (
|
|||
RepositoryTransactionError,
|
||||
)
|
||||
from rein_aharness.taskspec import TaskSpec
|
||||
from rein_aharness.native_limits import NativeLimitError
|
||||
|
||||
PROMPT_TEMPLATE = """\
|
||||
You are an unattended executor session (agent persona below, if any).
|
||||
|
|
@ -63,6 +64,7 @@ class RunResult:
|
|||
tool_profile: str = ""
|
||||
budget_tokens: int | None = None
|
||||
tokens_spent: int | None = None
|
||||
cost_usd: float | None = None
|
||||
execution_time_s: float = 0.0
|
||||
# Real-time per-tool-call audit events, populated only when run_task is
|
||||
# called with emit_tool_events=True. See adapter.py's module docstring
|
||||
|
|
@ -233,11 +235,14 @@ def _execute_locked_task(
|
|||
timeout_seconds=spec.timeout_seconds,
|
||||
skip_if_exists=False,
|
||||
budget_tracker=budget_tracker,
|
||||
model_params={"max_budget_usd": spec.max_budget_usd, "max_turns": spec.max_turns},
|
||||
)
|
||||
|
||||
started = time.monotonic()
|
||||
cost_usd = None
|
||||
try:
|
||||
response = adapter.execute_prompt(prompt, config)
|
||||
cost_usd = response.metadata.get("cost_usd")
|
||||
session_output = response.content
|
||||
resolved_model = response.model
|
||||
session_ok = True
|
||||
|
|
@ -248,6 +253,8 @@ def _execute_locked_task(
|
|||
reason = f"execution cancelled ({exc.reason})"
|
||||
resolved_model = model
|
||||
except Exception as exc: # adapter / budget failures must still be reported
|
||||
if isinstance(exc, NativeLimitError):
|
||||
cost_usd = exc.cost_usd
|
||||
session_output = ""
|
||||
session_ok = False
|
||||
reason = f"session failed: {exc}"
|
||||
|
|
@ -273,6 +280,7 @@ def _execute_locked_task(
|
|||
|
||||
metric_metadata = {
|
||||
"task_title": spec.title,
|
||||
"cost_usd": cost_usd,
|
||||
"tool_profile": profile.name,
|
||||
"labels": list(spec.labels),
|
||||
"completion_event_type": spec.completion_event_type,
|
||||
|
|
@ -325,6 +333,7 @@ def _execute_locked_task(
|
|||
tool_profile=profile.name,
|
||||
budget_tokens=budget_tokens,
|
||||
tokens_spent=tokens_spent,
|
||||
cost_usd=cost_usd,
|
||||
execution_time_s=execution_time_s,
|
||||
tool_events=collected_events,
|
||||
transaction=transaction_evidence,
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@ import json
|
|||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
from rein_aharness.native_limits import validate_limits
|
||||
from rein_aharness.repository_grant import RepositoryGrant, RepositoryGrantError
|
||||
|
||||
|
||||
|
|
@ -29,6 +30,14 @@ class TaskSpec:
|
|||
completion_event_type: str = "executor_run"
|
||||
timeout_seconds: int = 900
|
||||
repository_grant: RepositoryGrant | None = None
|
||||
max_budget_usd: float | None = None
|
||||
max_turns: int | None = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
try:
|
||||
validate_limits(self.max_budget_usd, self.max_turns)
|
||||
except ValueError as exc:
|
||||
raise TaskSpecError(str(exc)) from exc
|
||||
|
||||
@classmethod
|
||||
def from_file(cls, path: str | Path) -> "TaskSpec":
|
||||
|
|
@ -55,4 +64,6 @@ class TaskSpec:
|
|||
completion_event_type=str(raw.get("completion_event_type", "executor_run")),
|
||||
timeout_seconds=int(raw.get("timeout_seconds", 900)),
|
||||
repository_grant=grant,
|
||||
max_budget_usd=raw.get("max_budget_usd"),
|
||||
max_turns=raw.get("max_turns"),
|
||||
)
|
||||
|
|
|
|||
137
tests/test_native_limits.py
Normal file
137
tests/test_native_limits.py
Normal file
|
|
@ -0,0 +1,137 @@
|
|||
from __future__ import annotations
|
||||
import json
|
||||
import subprocess
|
||||
from unittest.mock import MagicMock, patch
|
||||
import pytest
|
||||
from llm_connect.models import BudgetTracker, RunConfig
|
||||
from rein_aharness.adapter import AgenticClaudeCodeAdapter
|
||||
from rein_aharness.native_limits import NativeLimitError, terminal_accounting
|
||||
from rein_aharness.taskspec import TaskSpec, TaskSpecError
|
||||
from test_adapter import _fake_proc
|
||||
|
||||
|
||||
def terminal(**overrides):
|
||||
return {
|
||||
"type": "result",
|
||||
"subtype": "success",
|
||||
"is_error": False,
|
||||
"result": "done",
|
||||
"num_turns": 2,
|
||||
"total_cost_usd": 0.1,
|
||||
"usage": {"input_tokens": 10, "output_tokens": 2, "cache_read_input_tokens": 3},
|
||||
**overrides,
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("stream", [False, True])
|
||||
def test_native_controls_reach_cli_and_account_usage(tmp_path, stream):
|
||||
adapter = AgenticClaudeCodeAdapter(
|
||||
workdir=tmp_path, on_tool_event=(lambda e: None) if stream else None
|
||||
)
|
||||
config = RunConfig(
|
||||
timeout_seconds=30,
|
||||
model_params={"max_budget_usd": 0.25, "max_turns": 3},
|
||||
budget_tracker=BudgetTracker(total=100),
|
||||
)
|
||||
payload = json.dumps(terminal())
|
||||
proc = _fake_proc([payload + "\n"]) if stream else MagicMock()
|
||||
if not stream:
|
||||
proc.communicate.return_value = (payload, "")
|
||||
proc.returncode = 0
|
||||
with (
|
||||
patch(
|
||||
"rein_aharness.adapter.subprocess.run",
|
||||
return_value=subprocess.CompletedProcess(
|
||||
[], 0, "2.1.263 (Claude Code)\n", ""
|
||||
),
|
||||
),
|
||||
patch("rein_aharness.adapter.subprocess.Popen", return_value=proc) as invoke,
|
||||
):
|
||||
result = adapter.execute_prompt("task", config)
|
||||
argv = invoke.call_args.args[0]
|
||||
assert argv[argv.index("--max-budget-usd") + 1] == "0.25"
|
||||
assert argv[argv.index("--max-turns") + 1] == "3"
|
||||
assert result.usage["total_tokens"] == config.budget_tracker.spent == 15
|
||||
assert result.metadata["cost_usd"] == 0.1
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"changes",
|
||||
[
|
||||
{"total_cost_usd": None},
|
||||
{"total_cost_usd": float("nan")},
|
||||
{"total_cost_usd": True},
|
||||
{"total_cost_usd": 0.3},
|
||||
{"num_turns": 4},
|
||||
{"usage": {}},
|
||||
{"subtype": "error_max_budget_usd", "is_error": True},
|
||||
{"subtype": "error_max_turns", "is_error": True},
|
||||
],
|
||||
)
|
||||
def test_unaccounted_or_exhausted_result_cannot_succeed(changes):
|
||||
with pytest.raises(NativeLimitError):
|
||||
terminal_accounting(terminal(**changes), max_budget_usd=0.25, max_turns=3)
|
||||
|
||||
|
||||
def test_error_preserves_only_bounded_cost():
|
||||
with pytest.raises(NativeLimitError) as caught:
|
||||
terminal_accounting(
|
||||
terminal(subtype="error_max_budget_usd", is_error=True, errors=["secret"]),
|
||||
max_budget_usd=0.25,
|
||||
max_turns=3,
|
||||
)
|
||||
assert caught.value.cost_usd == 0.1 and "secret" not in str(caught.value)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("version", ["2.1.216 (Claude Code)", "unrecognized"])
|
||||
def test_old_cli_refuses_before_prompt(tmp_path, version):
|
||||
adapter = AgenticClaudeCodeAdapter(workdir=tmp_path)
|
||||
with (
|
||||
patch(
|
||||
"rein_aharness.adapter.subprocess.run",
|
||||
return_value=subprocess.CompletedProcess([], 0, version, ""),
|
||||
),
|
||||
patch("rein_aharness.adapter.subprocess.Popen") as invoke,
|
||||
):
|
||||
with pytest.raises(NativeLimitError, match="2.1.217"):
|
||||
adapter.execute_prompt(
|
||||
"task", RunConfig(model_params={"max_budget_usd": 0.25})
|
||||
)
|
||||
invoke.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"limits",
|
||||
[
|
||||
{"max_budget_usd": True},
|
||||
{"max_budget_usd": -1},
|
||||
{"max_budget_usd": float("inf")},
|
||||
{"max_turns": True},
|
||||
{"max_turns": 1.5},
|
||||
],
|
||||
)
|
||||
def test_invalid_task_limits_refuse(tmp_path, limits):
|
||||
with pytest.raises(TaskSpecError):
|
||||
TaskSpec(title="task", description="d", target_repo=tmp_path, **limits)
|
||||
|
||||
|
||||
def test_runner_forwards_task_limits(tmp_path):
|
||||
from test_runner import CommittingAdapter, _make_repo
|
||||
from rein_aharness.runner import run_task
|
||||
|
||||
repo = _make_repo(tmp_path)
|
||||
adapter = CommittingAdapter(repo)
|
||||
result = run_task(
|
||||
TaskSpec(
|
||||
title="task",
|
||||
description="d",
|
||||
target_repo=repo,
|
||||
max_budget_usd=0.25,
|
||||
max_turns=3,
|
||||
),
|
||||
adapter=adapter,
|
||||
report_to_hub=False,
|
||||
write_metrics=False,
|
||||
)
|
||||
assert result.ok
|
||||
assert adapter.configs[0].model_params == {"max_budget_usd": 0.25, "max_turns": 3}
|
||||
312
tests/test_repository_artifact.py
Normal file
312
tests/test_repository_artifact.py
Normal file
|
|
@ -0,0 +1,312 @@
|
|||
"""Actual Git export/import across separate workspaces; no gateway writes source."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from rein_aharness.execution_cancel import ExecutionCancel, ExecutionCancelled
|
||||
from rein_aharness.repository_artifact import (
|
||||
RepositoryArtifactError,
|
||||
RepositoryArtifactTransfer,
|
||||
)
|
||||
from rein_aharness.repository_grant import RepositoryGrant
|
||||
from rein_aharness.repository_transaction import (
|
||||
RepositoryAcceptanceError,
|
||||
RepositoryTransaction,
|
||||
)
|
||||
|
||||
|
||||
def git(repo, *args):
|
||||
return (
|
||||
subprocess.check_output(
|
||||
["git", "-C", str(repo), *args], stderr=subprocess.DEVNULL
|
||||
)
|
||||
.decode()
|
||||
.strip()
|
||||
)
|
||||
|
||||
|
||||
def commit(repo, name="result.txt", content="accepted change\n"):
|
||||
path = repo / name
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(content)
|
||||
git(repo, "add", name)
|
||||
git(
|
||||
repo,
|
||||
"-c",
|
||||
"user.name=Fixture",
|
||||
"-c",
|
||||
"user.email=fixture@example.invalid",
|
||||
"commit",
|
||||
"-qm",
|
||||
"bounded result",
|
||||
)
|
||||
return git(repo, "rev-parse", "HEAD")
|
||||
|
||||
|
||||
def fixture(tmp_path):
|
||||
source = tmp_path / "source"
|
||||
source.mkdir()
|
||||
git(source, "init", "-q")
|
||||
commit(source, "README.md", "baseline\n")
|
||||
sandbox = tmp_path / "sandbox"
|
||||
git(tmp_path, "clone", "--quiet", "--no-local", str(source), str(sandbox))
|
||||
grant = RepositoryGrant.from_mapping(
|
||||
{
|
||||
"version": "1",
|
||||
"allowed_paths": ["result.txt"],
|
||||
"commit_count": {"min": 1, "max": 1},
|
||||
"publish": False,
|
||||
}
|
||||
)
|
||||
return source, sandbox, grant
|
||||
|
||||
|
||||
class Transport:
|
||||
"""Local transport double only; exporter and Git run unmodified."""
|
||||
|
||||
kind = "local_namespace"
|
||||
|
||||
def __init__(self, workspace):
|
||||
self.workspace = str(workspace)
|
||||
|
||||
def run(self, argv, *, timeout):
|
||||
return subprocess.run(
|
||||
argv, cwd=self.workspace, capture_output=True, text=True, timeout=timeout
|
||||
)
|
||||
|
||||
|
||||
def capture(transfer, sandbox, head):
|
||||
with patch(
|
||||
"glas_harness.transport.transport_from_sandbox", return_value=Transport(sandbox)
|
||||
):
|
||||
transfer.capture(object(), SimpleNamespace(commit_sha=head))
|
||||
|
||||
|
||||
def test_commit_survives_sandbox_teardown_and_import_preserves_identity(tmp_path):
|
||||
source, sandbox, grant = fixture(tmp_path)
|
||||
with RepositoryTransaction(source, state_dir=tmp_path / "state") as tx:
|
||||
baseline = tx.baseline.head
|
||||
transfer = RepositoryArtifactTransfer(tx, grant)
|
||||
head = commit(sandbox)
|
||||
capture(transfer, sandbox, head)
|
||||
assert git(source, "rev-parse", "HEAD") == baseline
|
||||
assert not (source / "result.txt").exists()
|
||||
shutil.rmtree(sandbox)
|
||||
result = transfer.import_after_teardown(head)
|
||||
assert result["head"] == head and result["imported"]
|
||||
assert git(source, "rev-parse", "HEAD") == head
|
||||
assert (source / "result.txt").read_text() == "accepted change\n"
|
||||
assert not git(source, "status", "--porcelain")
|
||||
assert tx.acceptance.changed_paths == ("result.txt",)
|
||||
assert "bundle" not in result
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"fault",
|
||||
[
|
||||
"ungranted",
|
||||
"two_commits",
|
||||
"wrong_reported_head",
|
||||
"dirty",
|
||||
"symlink",
|
||||
"expanded_size",
|
||||
],
|
||||
)
|
||||
def test_invalid_artifact_cannot_change_source(tmp_path, fault):
|
||||
source, sandbox, grant = fixture(tmp_path)
|
||||
with RepositoryTransaction(source, state_dir=tmp_path / "state") as tx:
|
||||
baseline = tx.baseline.head
|
||||
transfer = RepositoryArtifactTransfer(tx, grant)
|
||||
head = commit(
|
||||
sandbox,
|
||||
"outside.txt" if fault == "ungranted" else "result.txt",
|
||||
"x" * (3 * 1024 * 1024) if fault == "expanded_size" else "result\n",
|
||||
)
|
||||
if fault == "two_commits":
|
||||
head = commit(sandbox, content="again\n")
|
||||
if fault == "dirty":
|
||||
(sandbox / "untracked.txt").write_text("dirty")
|
||||
if fault == "symlink":
|
||||
git(sandbox, "rm", "result.txt")
|
||||
os.symlink("/tmp/outside", sandbox / "result.txt")
|
||||
git(sandbox, "add", "result.txt")
|
||||
git(
|
||||
sandbox,
|
||||
"-c",
|
||||
"user.name=Fixture",
|
||||
"-c",
|
||||
"user.email=fixture@example.invalid",
|
||||
"commit",
|
||||
"--amend",
|
||||
"--no-edit",
|
||||
"-q",
|
||||
)
|
||||
head = git(sandbox, "rev-parse", "HEAD")
|
||||
with pytest.raises((RepositoryArtifactError, RepositoryAcceptanceError)):
|
||||
capture(transfer, sandbox, head)
|
||||
transfer.import_after_teardown(
|
||||
"f" * 40 if fault == "wrong_reported_head" else head
|
||||
)
|
||||
assert git(source, "rev-parse", "HEAD") == baseline
|
||||
assert not (source / "result.txt").exists()
|
||||
assert not (source / "outside.txt").exists()
|
||||
|
||||
|
||||
def test_lease_loss_after_capture_refuses_import(tmp_path):
|
||||
source, sandbox, grant = fixture(tmp_path)
|
||||
cancel = ExecutionCancel()
|
||||
with RepositoryTransaction(source, state_dir=tmp_path / "state") as tx:
|
||||
transfer = RepositoryArtifactTransfer(tx, grant, cancel=cancel)
|
||||
head = commit(sandbox)
|
||||
capture(transfer, sandbox, head)
|
||||
cancel.cancel("lease-loss")
|
||||
with pytest.raises(ExecutionCancelled):
|
||||
transfer.import_after_teardown(head)
|
||||
assert git(source, "rev-parse", "HEAD") == tx.baseline.head
|
||||
|
||||
|
||||
def test_changed_source_baseline_refuses_import(tmp_path):
|
||||
source, sandbox, grant = fixture(tmp_path)
|
||||
with RepositoryTransaction(source, state_dir=tmp_path / "state") as tx:
|
||||
transfer = RepositoryArtifactTransfer(tx, grant)
|
||||
head = commit(sandbox)
|
||||
capture(transfer, sandbox, head)
|
||||
human_head = commit(source, "human.txt", "new human work\n")
|
||||
with pytest.raises(RepositoryAcceptanceError, match="baseline-changed"):
|
||||
transfer.import_after_teardown(head)
|
||||
assert git(source, "rev-parse", "HEAD") == human_head
|
||||
assert (source / "human.txt").read_text() == "new human work\n"
|
||||
assert not (source / "result.txt").exists()
|
||||
|
||||
|
||||
def test_gateway_failure_after_capture_does_not_import(tmp_path):
|
||||
from rein_aharness.glas_execution import execute_profiled_run
|
||||
from rein_aharness.ops_run_client import OpsRun, OpsRunConfig
|
||||
|
||||
source, sandbox, grant = fixture(tmp_path)
|
||||
run = OpsRun(
|
||||
id="run",
|
||||
activity_definition_id="def",
|
||||
idempotency_key="key",
|
||||
target_repo=str(source),
|
||||
title="fixture",
|
||||
description="fixture",
|
||||
state="claimed",
|
||||
harness_profile_ref="harness.agent-dev-local@1.0.0",
|
||||
repository_grant=grant,
|
||||
)
|
||||
with RepositoryTransaction(source, state_dir=tmp_path / "state") as tx:
|
||||
head = commit(sandbox)
|
||||
|
||||
def gateway(request, *, artifact_capture):
|
||||
with patch(
|
||||
"glas_harness.transport.transport_from_sandbox",
|
||||
return_value=Transport(sandbox),
|
||||
):
|
||||
artifact_capture(object(), SimpleNamespace(commit_sha=head))
|
||||
return {
|
||||
"ok": False,
|
||||
"evidence": {
|
||||
"commit_sha": head,
|
||||
"outcome": "failed",
|
||||
"sandbox_destroy": "failed",
|
||||
},
|
||||
}
|
||||
|
||||
result = execute_profiled_run(
|
||||
run,
|
||||
OpsRunConfig(repo_roots=(str(tmp_path),)),
|
||||
report_to_hub=False,
|
||||
transaction=tx,
|
||||
request_factory=lambda **kw: kw,
|
||||
gateway=gateway,
|
||||
)
|
||||
assert not result["ok"]
|
||||
assert git(source, "rev-parse", "HEAD") == tx.baseline.head
|
||||
|
||||
|
||||
def test_success_without_artifact_capture_refuses(tmp_path):
|
||||
from rein_aharness.glas_execution import execute_profiled_run
|
||||
from rein_aharness.ops_run_client import OpsRun, OpsRunConfig
|
||||
|
||||
source, sandbox, grant = fixture(tmp_path)
|
||||
run = OpsRun(
|
||||
id="run",
|
||||
activity_definition_id="def",
|
||||
idempotency_key="key",
|
||||
target_repo=str(source),
|
||||
title="fixture",
|
||||
description="fixture",
|
||||
state="claimed",
|
||||
harness_profile_ref="harness.agent-dev-local@1.0.0",
|
||||
repository_grant=grant,
|
||||
)
|
||||
with RepositoryTransaction(source, state_dir=tmp_path / "state") as tx:
|
||||
with pytest.raises(RepositoryArtifactError, match="captured artifact"):
|
||||
execute_profiled_run(
|
||||
run,
|
||||
OpsRunConfig(repo_roots=(str(tmp_path),)),
|
||||
transaction=tx,
|
||||
request_factory=lambda **kw: kw,
|
||||
gateway=lambda *a, **kw: {
|
||||
"ok": True,
|
||||
"evidence": {
|
||||
"commit_sha": "a" * 40,
|
||||
"session_cleanup": "succeeded",
|
||||
"sandbox_destroy": "succeeded",
|
||||
},
|
||||
},
|
||||
)
|
||||
assert git(source, "rev-parse", "HEAD") == tx.baseline.head
|
||||
|
||||
|
||||
def test_missing_repository_ownership_refuses():
|
||||
grant = RepositoryGrant.from_mapping(
|
||||
{
|
||||
"version": "1",
|
||||
"allowed_paths": ["result.txt"],
|
||||
"commit_count": {"min": 1, "max": 1},
|
||||
"publish": False,
|
||||
}
|
||||
)
|
||||
with pytest.raises(RepositoryArtifactError, match="active repository transaction"):
|
||||
RepositoryArtifactTransfer(None, grant)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("corruption", ["truncated", "substituted_head"])
|
||||
def test_corrupted_capture_cannot_change_source(tmp_path, corruption):
|
||||
source, sandbox, grant = fixture(tmp_path)
|
||||
with RepositoryTransaction(source, state_dir=tmp_path / "state") as tx:
|
||||
transfer = RepositoryArtifactTransfer(tx, grant)
|
||||
head = commit(sandbox)
|
||||
capture(transfer, sandbox, head)
|
||||
if corruption == "truncated":
|
||||
transfer._bundle = transfer._bundle[:100]
|
||||
else:
|
||||
transfer._head = "f" * 40
|
||||
with pytest.raises(RepositoryArtifactError):
|
||||
transfer.import_after_teardown(transfer._head)
|
||||
assert git(source, "rev-parse", "HEAD") == tx.baseline.head
|
||||
assert not (source / "result.txt").exists()
|
||||
|
||||
|
||||
def test_ignored_sandbox_cache_is_not_imported(tmp_path):
|
||||
source, sandbox, grant = fixture(tmp_path)
|
||||
commit(source, ".gitignore", "cache/\n")
|
||||
git(sandbox, "pull", "--ff-only", "--quiet")
|
||||
(sandbox / "cache").mkdir()
|
||||
(sandbox / "cache" / "runtime.txt").write_text("discarded runtime cache")
|
||||
with RepositoryTransaction(source, state_dir=tmp_path / "state") as tx:
|
||||
transfer = RepositoryArtifactTransfer(tx, grant)
|
||||
head = commit(sandbox)
|
||||
capture(transfer, sandbox, head)
|
||||
transfer.import_after_teardown(head)
|
||||
assert not (source / "cache").exists()
|
||||
assert tx.acceptance.changed_paths == ("result.txt",)
|
||||
171
tests/test_repository_artifact_bwrap.py
Normal file
171
tests/test_repository_artifact_bwrap.py
Normal file
|
|
@ -0,0 +1,171 @@
|
|||
"""Opt-in real sandbox/Glas/worker proof; queue and task authoring are fixtures."""
|
||||
|
||||
from __future__ import annotations
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock
|
||||
import pytest
|
||||
|
||||
from glas_harness.contract import (
|
||||
ExecutionSummary,
|
||||
OperationalReadiness,
|
||||
Rein,
|
||||
ToolResult,
|
||||
)
|
||||
from glas_harness.gateway import run_execution
|
||||
from glas_harness.profiles import ProfileCatalog
|
||||
from glas_harness.transport import transport_from_sandbox
|
||||
from sandboxer.core.manager import SandboxManager
|
||||
from sandboxer.lifecycle.store import SandboxStore
|
||||
from rein_aharness.claim_loop import process_one
|
||||
from rein_aharness.close_outbox import CloseOutbox
|
||||
from rein_aharness.metrics import external_metrics_dir
|
||||
from rein_aharness.ops_run_client import (
|
||||
ActivityCoreOpsClient,
|
||||
OpsRun,
|
||||
OpsRunConfig,
|
||||
OpsRunError,
|
||||
)
|
||||
from test_repository_artifact import commit, git
|
||||
from rein_aharness.repository_grant import RepositoryGrant
|
||||
|
||||
pytestmark = pytest.mark.skipif(
|
||||
os.environ.get("REIN_REAL_BWRAP") != "1", reason="opt-in kernel namespace proof"
|
||||
)
|
||||
|
||||
|
||||
class DeterministicRein(Rein):
|
||||
calls = 0
|
||||
workspace = None
|
||||
head = None
|
||||
|
||||
def __init__(self, source, baseline):
|
||||
self.source, self.baseline = source, baseline
|
||||
|
||||
def start_session(self, profile, inputs, sandbox):
|
||||
transport = transport_from_sandbox(sandbox)
|
||||
self.workspace = Path(transport.workspace)
|
||||
return transport
|
||||
|
||||
def dispatch_tool(self, transport, tool_call):
|
||||
self.calls += 1
|
||||
code = """from pathlib import Path
|
||||
import subprocess
|
||||
Path('result.txt').write_text('sandbox-result-only\\n')
|
||||
def git(*args):
|
||||
p=subprocess.run(['git',*args],check=True,capture_output=True,text=True)
|
||||
return p.stdout.strip()
|
||||
git('add','result.txt')
|
||||
git('-c','user.name=Fixture','-c','user.email=fixture@example.invalid','commit','-qm','bounded result')
|
||||
print(git('rev-parse','HEAD'))
|
||||
"""
|
||||
result = transport.run(["python3", "-c", code], timeout=30)
|
||||
assert result.returncode == 0, result.stderr
|
||||
self.head = result.stdout.strip()
|
||||
assert git(self.source, "rev-parse", "HEAD") == self.baseline
|
||||
assert not (self.source / "result.txt").exists()
|
||||
return ToolResult(ok=True, output="sandbox-result-only", tokens_spent=0)
|
||||
|
||||
def end_session(self, session):
|
||||
return ExecutionSummary(
|
||||
committed=True, commit_sha=self.head, outcome="succeeded", tokens_spent=0
|
||||
)
|
||||
|
||||
def cleanup_session(self, session):
|
||||
assert self.workspace.exists()
|
||||
assert git(self.source, "rev-parse", "HEAD") == self.baseline
|
||||
|
||||
|
||||
def test_real_bwrap_worker_import_and_close_replay(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("XDG_DATA_HOME", str(tmp_path / "data"))
|
||||
monkeypatch.setenv("REIN_AHARNESS_STATE_DIR", str(tmp_path / "state"))
|
||||
monkeypatch.setenv("SANDBOXER_NO_STATE_HUB", "1")
|
||||
source = tmp_path / "source"
|
||||
source.mkdir()
|
||||
git(source, "init", "-q")
|
||||
baseline = commit(source, "README.md", "baseline\n")
|
||||
grant = RepositoryGrant.from_mapping(
|
||||
{
|
||||
"version": "1",
|
||||
"allowed_paths": ["result.txt"],
|
||||
"commit_count": {"min": 1, "max": 1},
|
||||
"publish": False,
|
||||
}
|
||||
)
|
||||
run = OpsRun(
|
||||
id="fixture-run",
|
||||
activity_definition_id="fixture-definition",
|
||||
idempotency_key="fixture-key",
|
||||
target_repo=str(source),
|
||||
title="deterministic fixture",
|
||||
description="no inference",
|
||||
state="claimed",
|
||||
harness_profile_ref="harness.agent-dev-local@1.0.0",
|
||||
repository_grant=grant,
|
||||
claim_owner="fixture-worker",
|
||||
attempt=1,
|
||||
)
|
||||
client = MagicMock(spec=ActivityCoreOpsClient)
|
||||
client.config = OpsRunConfig(
|
||||
worker_id="fixture-worker", lease_seconds=90, repo_roots=(str(tmp_path),)
|
||||
)
|
||||
client.claim.return_value = [run]
|
||||
client.heartbeat.return_value = run
|
||||
client.complete.side_effect = [
|
||||
OpsRunError("response lost", action="complete"),
|
||||
OpsRun(
|
||||
id=run.id,
|
||||
activity_definition_id="fixture-definition",
|
||||
idempotency_key="fixture-key",
|
||||
target_repo=str(source),
|
||||
title=run.title,
|
||||
description=run.description,
|
||||
state="succeeded",
|
||||
close_disposition="reconciled",
|
||||
),
|
||||
]
|
||||
catalog = ProfileCatalog()
|
||||
profile, _ = catalog.resolve(run.harness_profile_ref)
|
||||
catalog.profiles()[(profile.id, profile.version)] = profile.model_copy(
|
||||
update={
|
||||
"operational_readiness": OperationalReadiness(
|
||||
status="ready",
|
||||
reason="isolated test fixture",
|
||||
owner="tests",
|
||||
evidence_ref="test:artifact-bwrap",
|
||||
)
|
||||
}
|
||||
)
|
||||
manager = SandboxManager(store=SandboxStore(tmp_path / "sandboxes.json"))
|
||||
rein = DeterministicRein(source, baseline)
|
||||
monkeypatch.setattr(
|
||||
"glas_harness.gateway.run_execution",
|
||||
lambda request, **kwargs: run_execution(
|
||||
request, catalog=catalog, rein=rein, manager=manager, **kwargs
|
||||
),
|
||||
)
|
||||
outbox = CloseOutbox(state_dir=tmp_path / "state")
|
||||
first = process_one(client, outbox=outbox, report_to_hub=False)
|
||||
assert first.reason == "close evidence remains pending", first
|
||||
assert not rein.workspace.exists()
|
||||
assert git(source, "rev-parse", "HEAD") == rein.head
|
||||
assert (source / "result.txt").read_text() == "sandbox-result-only\n"
|
||||
assert not git(source, "status", "--porcelain")
|
||||
assert outbox.status()["pending"] == 1
|
||||
record = json.loads(
|
||||
(external_metrics_dir(source, "rein-aharness") / "executions.jsonl")
|
||||
.read_text()
|
||||
.strip()
|
||||
)
|
||||
assert record["success"]
|
||||
durable = json.dumps(client.complete.call_args.kwargs["result"])
|
||||
assert rein.head in durable
|
||||
assert "sandbox-result-only" not in durable and '"bundle"' not in durable
|
||||
client.claim.return_value = []
|
||||
second = process_one(client, outbox=outbox, report_to_hub=False)
|
||||
assert second.empty and rein.calls == 1
|
||||
assert client.complete.call_count == 2
|
||||
assert client.heartbeat.call_count >= 1
|
||||
assert git(source, "rev-list", "--count", "HEAD") == "2"
|
||||
assert outbox.status() == {"pending": 0, "delivered": 1, "quarantined": 0}
|
||||
|
|
@ -9,7 +9,7 @@ owner: codex
|
|||
topic_slug: rein-aharness
|
||||
priority: high
|
||||
created: "2026-08-23"
|
||||
updated: "2026-09-04"
|
||||
updated: "2026-09-09"
|
||||
related:
|
||||
- REIN-A-0004
|
||||
- GLAS-IN-0002
|
||||
|
|
@ -623,6 +623,18 @@ deployment, and the next natural claim → active heartbeat → terminal run id
|
|||
will be returned to Activity Core after this change is published and deployed;
|
||||
no real workload will be delayed or expired to manufacture evidence.
|
||||
|
||||
2026-09-09 implementation return: the worker now captures a bounded Git bundle
|
||||
through sandbox owner execution after rein cleanup, then validates and imports
|
||||
the exact one-commit result under its original baseline, grant and lease after
|
||||
successful teardown. The actual bwrap/Glas/worker test passed with a deterministic
|
||||
authoring fixture and response-lost close replay, without duplicate execution.
|
||||
Native profile USD/turn limits now reach Claude CLI controls and require valid
|
||||
terminal accounting; missing/exhausted accounting refuses success. Daily/total
|
||||
reservation, EUR treatment and live provider semantics remain HFACT-WP-0001-T01.
|
||||
The matching rein/Glas code must be rebuilt and admitted in the protected runtime;
|
||||
this local proof does not close live G1/G2 or authorize a model request. See the
|
||||
2026-09-09 runtime-transfer evidence and the owning runtime documentation.
|
||||
|
||||
## Re-prove one governed profiled run and close residuals
|
||||
|
||||
```task
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue