feat: import governed sandbox commits and enforce native CLI limits
Some checks are pending
Governed runtime contract / contract (push) Waiting to run
Some checks are pending
Governed runtime contract / contract (push) Waiting to run
Assistant: codex Assistant-Model: gpt-5.6-luna Assistant-Session: 01a07ff8-19d0-7820-b4d0-1353833cb7fc
This commit is contained in:
parent
1429db5ad4
commit
c63caf5568
16 changed files with 1236 additions and 7 deletions
|
|
@ -13,13 +13,13 @@
|
||||||
"version": "0.1.0"
|
"version": "0.1.0"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"commit": "02b29af9ca87867e0d398c06139d771a19728b70",
|
"commit": "fb57c0228f41021a9ad30dae3f18aed8335b2445",
|
||||||
"distribution": "glas-harness",
|
"distribution": "glas-harness",
|
||||||
"source": "../glas-harness",
|
"source": "../glas-harness",
|
||||||
"version": "0.1.0"
|
"version": "0.1.0"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"commit": "5c760100264828865c45a7e4e199d43e4603d57a",
|
"commit": "3e49a98a0e2c4a64539a5ccd674be8cd67ac9845",
|
||||||
"distribution": "sandboxer",
|
"distribution": "sandboxer",
|
||||||
"source": "../sand-boxer",
|
"source": "../sand-boxer",
|
||||||
"version": "0.0.0"
|
"version": "0.0.0"
|
||||||
|
|
|
||||||
40
docs/evidence/2026-09-09-runtime-transfer.json
Normal file
40
docs/evidence/2026-09-09-runtime-transfer.json
Normal file
|
|
@ -0,0 +1,40 @@
|
||||||
|
{
|
||||||
|
"schema": "hfact.runtime-transfer-verification.v1",
|
||||||
|
"date": "2026-09-09",
|
||||||
|
"rein_tests": 290,
|
||||||
|
"glas_tests": 133,
|
||||||
|
"actual_bwrap_owner_execution": true,
|
||||||
|
"task_authoring": "deterministic fixture",
|
||||||
|
"queue": "fixture client; heartbeat/close/replay exercised",
|
||||||
|
"artifact_tests": [
|
||||||
|
"exact commit retained after sandbox destruction",
|
||||||
|
"granted path acceptance",
|
||||||
|
"one commit ancestry",
|
||||||
|
"corrupt and truncated bundle",
|
||||||
|
"symlink refusal",
|
||||||
|
"expanded object limit",
|
||||||
|
"wrong head",
|
||||||
|
"missing capture",
|
||||||
|
"teardown failure",
|
||||||
|
"lease loss",
|
||||||
|
"source changed externally",
|
||||||
|
"ignored cache excluded"
|
||||||
|
],
|
||||||
|
"native_controls": [
|
||||||
|
"profile USD and turn limits reach Claude CLI",
|
||||||
|
"version preflight",
|
||||||
|
"strict terminal cost/token/turn accounting",
|
||||||
|
"limit exhaustion refuses success"
|
||||||
|
],
|
||||||
|
"provider_requests": 0,
|
||||||
|
"paid_factory_attempts": 0,
|
||||||
|
"profile_promoted": false,
|
||||||
|
"protected_runtime_rebuilt": false,
|
||||||
|
"remaining": [
|
||||||
|
"durable daily/total budget reservations and uncertain outcome recovery",
|
||||||
|
"provider cap semantics and conservative EUR treatment",
|
||||||
|
"matching protected runtime pins and exact factory identity/placement",
|
||||||
|
"G0 operating admission",
|
||||||
|
"real model G1 and natural Railiance queue G2"
|
||||||
|
]
|
||||||
|
}
|
||||||
54
docs/sandbox-artifact-return.md
Normal file
54
docs/sandbox-artifact-return.md
Normal file
|
|
@ -0,0 +1,54 @@
|
||||||
|
# Sandbox commit return and native CLI limits
|
||||||
|
|
||||||
|
The worker owns the original repository transaction and repository grant. Glas
|
||||||
|
owns sandbox lifetime. For a granted profiled run, the worker supplies a trusted
|
||||||
|
process-local capture callback. Glas invokes it after successful rein cleanup
|
||||||
|
and before sandbox destruction. Callback failure is `artifact_capture` failure;
|
||||||
|
teardown still runs. Artifact bytes are never serialized into gateway evidence.
|
||||||
|
|
||||||
|
The initial transfer supports owner-mediated local namespaces and exactly one
|
||||||
|
local Git commit. It captures at most 128 KiB of bundle data, verifies the exact
|
||||||
|
baseline/head and ancestry in a disposable checkout, limits new objects to
|
||||||
|
1,024 / 2 MiB, rejects symlinks/gitlinks and out-of-grant paths, and fast-forwards
|
||||||
|
the original checkout only after successful teardown and another lease/baseline
|
||||||
|
check. Ignored runtime caches are discarded with the sandbox. Source changes,
|
||||||
|
lease loss, missing or corrupt capture, and failed teardown refuse import.
|
||||||
|
|
||||||
|
Acceptance and external metrics remain in the outer worker. The inner Glas task
|
||||||
|
continues to use `--no-metrics` without an inner repository grant. Terminal-close
|
||||||
|
replay uses the existing durable outbox and does not run the workload again.
|
||||||
|
The repository lock is cooperative; unexpected external changes are detected,
|
||||||
|
not rolled back. A crash or lease loss at the final mutation/receipt boundary
|
||||||
|
still requires the existing recovery process to classify the resulting commit.
|
||||||
|
|
||||||
|
## Native limits
|
||||||
|
|
||||||
|
Glas profile limits `max_budget_usd` and `max_turns` are carried in its generated
|
||||||
|
TaskSpec into the agentic Claude adapter. A supplied task file cannot bypass
|
||||||
|
these profile controls. Positive finite USD amounts and positive integer turn
|
||||||
|
limits are required. A controlled run verifies Claude Code >= 2.1.217, supplies
|
||||||
|
`--max-budget-usd` / `--max-turns`, and requires one successful terminal JSON
|
||||||
|
result with valid cost, turn and token accounting. Limit exhaustion, missing
|
||||||
|
accounting and reported overruns cannot produce a successful run. Bounded USD
|
||||||
|
cost follows runner, Glas evidence and the outer close evidence; result text and
|
||||||
|
artifact bytes do not enter the close outbox. Runs without these optional controls
|
||||||
|
retain their legacy behavior, including its incomplete token accounting.
|
||||||
|
|
||||||
|
This is native per-run control and accounting, not a factory spend grant or a
|
||||||
|
hard EUR ceiling. The [Claude CLI contract](https://code.claude.com/docs/en/cli-reference)
|
||||||
|
is the upstream reference. Before paid admission, verify the protected binary's
|
||||||
|
actual enforcement/overrun semantics, reserve daily/total worst-case costs
|
||||||
|
outside the sandbox, define conservative EUR/USD treatment, and retain unknown
|
||||||
|
outcomes as held reservations. No profile was enabled and no paid request was
|
||||||
|
made by this change. Deploy a newly verified runtime containing the matching
|
||||||
|
rein/Glas source pins before exercising the real model path.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
`tests/test_repository_artifact.py` runs real Git export/import with adversarial
|
||||||
|
artifacts. `REIN_REAL_BWRAP=1 pytest tests/test_repository_artifact_bwrap.py -q`
|
||||||
|
adds actual sand-boxer owner execution, Glas lifecycle and worker acceptance,
|
||||||
|
including response-lost close replay. Its queue and task authoring are fixtures;
|
||||||
|
it proves neither a natural Activity Core claim nor model/provider admission.
|
||||||
|
`tests/test_native_limits.py` exercises control propagation, terminal accounting,
|
||||||
|
invalid/exhausted results and refusal of older CLI versions without inference.
|
||||||
|
|
@ -24,6 +24,7 @@ one aggregate `LLMResponse` at the end for interface compatibility.
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import json
|
import json
|
||||||
|
import re
|
||||||
import subprocess
|
import subprocess
|
||||||
import threading
|
import threading
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
@ -32,6 +33,7 @@ from typing import Any, Callable
|
||||||
from llm_connect.claude_code import ClaudeCodeAdapter
|
from llm_connect.claude_code import ClaudeCodeAdapter
|
||||||
from llm_connect.exceptions import LLMSubprocessError, LLMTimeoutError
|
from llm_connect.exceptions import LLMSubprocessError, LLMTimeoutError
|
||||||
from llm_connect.models import LLMResponse, RunConfig
|
from llm_connect.models import LLMResponse, RunConfig
|
||||||
|
from rein_aharness.native_limits import NativeLimitError, terminal_accounting, validate_limits
|
||||||
|
|
||||||
from rein_aharness.execution_cancel import (
|
from rein_aharness.execution_cancel import (
|
||||||
ExecutionCancel,
|
ExecutionCancel,
|
||||||
|
|
@ -99,6 +101,8 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
||||||
self._workdir = workdir
|
self._workdir = workdir
|
||||||
self._on_tool_event = on_tool_event
|
self._on_tool_event = on_tool_event
|
||||||
self._cancel = cancel
|
self._cancel = cancel
|
||||||
|
self._native_config = (None, None)
|
||||||
|
self._native_cli_checked = False
|
||||||
if isinstance(tool_profile, ToolProfile):
|
if isinstance(tool_profile, ToolProfile):
|
||||||
self._profile = tool_profile
|
self._profile = tool_profile
|
||||||
else:
|
else:
|
||||||
|
|
@ -109,6 +113,10 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
||||||
return self._profile
|
return self._profile
|
||||||
|
|
||||||
def _build_command(self, config: RunConfig) -> list[str]:
|
def _build_command(self, config: RunConfig) -> list[str]:
|
||||||
|
budget = config.model_params.get("max_budget_usd")
|
||||||
|
turns = config.model_params.get("max_turns")
|
||||||
|
validate_limits(budget, turns)
|
||||||
|
self._native_config = (budget, turns)
|
||||||
cmd = [
|
cmd = [
|
||||||
self._cli_path,
|
self._cli_path,
|
||||||
"--print",
|
"--print",
|
||||||
|
|
@ -119,6 +127,12 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
||||||
]
|
]
|
||||||
if self._on_tool_event is not None:
|
if self._on_tool_event is not None:
|
||||||
cmd += ["--output-format", "stream-json", "--include-hook-events", "--verbose"]
|
cmd += ["--output-format", "stream-json", "--include-hook-events", "--verbose"]
|
||||||
|
elif budget is not None or turns is not None:
|
||||||
|
cmd += ["--output-format", "json"]
|
||||||
|
if budget is not None:
|
||||||
|
cmd += ["--max-budget-usd", str(budget)]
|
||||||
|
if turns is not None:
|
||||||
|
cmd += ["--max-turns", str(turns)]
|
||||||
if self._model:
|
if self._model:
|
||||||
cmd.extend(["--model", self._model])
|
cmd.extend(["--model", self._model])
|
||||||
return cmd
|
return cmd
|
||||||
|
|
@ -126,12 +140,23 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
||||||
def execute_prompt(self, prompt: str, config: RunConfig) -> LLMResponse:
|
def execute_prompt(self, prompt: str, config: RunConfig) -> LLMResponse:
|
||||||
self._preflight_budget(config)
|
self._preflight_budget(config)
|
||||||
cmd = self._build_command(config)
|
cmd = self._build_command(config)
|
||||||
|
if self._native_config != (None, None) and not self._native_cli_checked:
|
||||||
|
check = subprocess.run([self._cli_path, "--version"], capture_output=True, text=True, timeout=10, cwd=self._workdir)
|
||||||
|
version = re.match(r"(\d+)\.(\d+)\.(\d+)(?:\s|$)", check.stdout.strip())
|
||||||
|
if check.returncode or not version or tuple(map(int, version.groups())) < (2, 1, 217):
|
||||||
|
raise NativeLimitError("native limits require verified Claude Code 2.1.217 or newer")
|
||||||
|
self._native_cli_checked = True
|
||||||
timeout = config.timeout_seconds or self._config.timeout_seconds
|
timeout = config.timeout_seconds or self._config.timeout_seconds
|
||||||
if self._on_tool_event is not None:
|
if self._on_tool_event is not None:
|
||||||
response = self._execute_streaming(cmd, prompt, timeout)
|
response = self._execute_streaming(cmd, prompt, timeout)
|
||||||
else:
|
else:
|
||||||
response = self._execute_blocking(cmd, prompt, timeout)
|
response = self._execute_blocking(cmd, prompt, timeout)
|
||||||
self._consume_budget(config, response)
|
try:
|
||||||
|
self._consume_budget(config, response)
|
||||||
|
except Exception as exc:
|
||||||
|
if self._native_config != (None, None):
|
||||||
|
raise NativeLimitError("controlled Claude run failed token accounting", cost_usd=response.metadata.get("cost_usd")) from exc
|
||||||
|
raise
|
||||||
return response
|
return response
|
||||||
|
|
||||||
def _execute_blocking(self, cmd: list[str], prompt: str, timeout: int) -> LLMResponse:
|
def _execute_blocking(self, cmd: list[str], prompt: str, timeout: int) -> LLMResponse:
|
||||||
|
|
@ -144,7 +169,19 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
||||||
cwd=self._workdir,
|
cwd=self._workdir,
|
||||||
)
|
)
|
||||||
stdout, stderr = self._wait_for_process(proc, prompt, timeout)
|
stdout, stderr = self._wait_for_process(proc, prompt, timeout)
|
||||||
|
usage, cost = {}, None
|
||||||
|
if self._native_config != (None, None):
|
||||||
|
try:
|
||||||
|
envelope = json.loads(stdout)
|
||||||
|
except (TypeError, ValueError) as exc:
|
||||||
|
raise NativeLimitError("controlled Claude run returned invalid terminal JSON") from exc
|
||||||
|
usage, cost = terminal_accounting(envelope, max_budget_usd=self._native_config[0], max_turns=self._native_config[1])
|
||||||
|
stdout = envelope.get("result", "")
|
||||||
|
if not isinstance(stdout, str):
|
||||||
|
raise NativeLimitError("controlled Claude run returned invalid result text", cost_usd=cost)
|
||||||
if proc.returncode != 0:
|
if proc.returncode != 0:
|
||||||
|
if self._native_config != (None, None):
|
||||||
|
raise NativeLimitError("controlled Claude CLI exited unsuccessfully", cost_usd=cost)
|
||||||
raise LLMSubprocessError(
|
raise LLMSubprocessError(
|
||||||
f"claude CLI exited with code {proc.returncode}",
|
f"claude CLI exited with code {proc.returncode}",
|
||||||
return_code=proc.returncode,
|
return_code=proc.returncode,
|
||||||
|
|
@ -153,9 +190,10 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
||||||
return LLMResponse(
|
return LLMResponse(
|
||||||
content=stdout,
|
content=stdout,
|
||||||
model=self._model or "claude-code-cli",
|
model=self._model or "claude-code-cli",
|
||||||
usage={},
|
usage=usage,
|
||||||
finish_reason="stop",
|
finish_reason="stop",
|
||||||
metadata={
|
metadata={
|
||||||
|
"cost_usd": cost,
|
||||||
"provider": "claude-code-agentic",
|
"provider": "claude-code-agentic",
|
||||||
"cli_path": self._cli_path,
|
"cli_path": self._cli_path,
|
||||||
"workdir": str(self._workdir),
|
"workdir": str(self._workdir),
|
||||||
|
|
@ -174,6 +212,7 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
||||||
)
|
)
|
||||||
text_parts: list[str] = []
|
text_parts: list[str] = []
|
||||||
tool_event_count = 0
|
tool_event_count = 0
|
||||||
|
terminal_results = []
|
||||||
|
|
||||||
def reader() -> None:
|
def reader() -> None:
|
||||||
nonlocal tool_event_count
|
nonlocal tool_event_count
|
||||||
|
|
@ -186,6 +225,8 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
||||||
event = json.loads(line)
|
event = json.loads(line)
|
||||||
except json.JSONDecodeError:
|
except json.JSONDecodeError:
|
||||||
continue
|
continue
|
||||||
|
if isinstance(event, dict) and event.get("type") == "result":
|
||||||
|
terminal_results.append(event)
|
||||||
self._handle_stream_event(event, text_parts)
|
self._handle_stream_event(event, text_parts)
|
||||||
if _is_tool_event(event):
|
if _is_tool_event(event):
|
||||||
tool_event_count += 1
|
tool_event_count += 1
|
||||||
|
|
@ -200,7 +241,14 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
||||||
reader_thread.join(timeout=5)
|
reader_thread.join(timeout=5)
|
||||||
stderr = proc.stderr.read() if proc.stderr else ""
|
stderr = proc.stderr.read() if proc.stderr else ""
|
||||||
|
|
||||||
|
usage, cost = {}, None
|
||||||
|
if self._native_config != (None, None):
|
||||||
|
if reader_thread.is_alive() or len(terminal_results) != 1:
|
||||||
|
raise NativeLimitError("controlled Claude stream requires one complete terminal result")
|
||||||
|
usage, cost = terminal_accounting(terminal_results[0], max_budget_usd=self._native_config[0], max_turns=self._native_config[1])
|
||||||
if returncode != 0:
|
if returncode != 0:
|
||||||
|
if self._native_config != (None, None):
|
||||||
|
raise NativeLimitError("controlled Claude CLI exited unsuccessfully", cost_usd=cost)
|
||||||
raise LLMSubprocessError(
|
raise LLMSubprocessError(
|
||||||
f"claude CLI exited with code {returncode}",
|
f"claude CLI exited with code {returncode}",
|
||||||
return_code=returncode,
|
return_code=returncode,
|
||||||
|
|
@ -210,7 +258,7 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
||||||
return LLMResponse(
|
return LLMResponse(
|
||||||
content="".join(text_parts),
|
content="".join(text_parts),
|
||||||
model=self._model or "claude-code-cli",
|
model=self._model or "claude-code-cli",
|
||||||
usage={},
|
usage=usage,
|
||||||
finish_reason="stop",
|
finish_reason="stop",
|
||||||
metadata={
|
metadata={
|
||||||
"provider": "claude-code-agentic",
|
"provider": "claude-code-agentic",
|
||||||
|
|
@ -218,6 +266,7 @@ class AgenticClaudeCodeAdapter(ClaudeCodeAdapter):
|
||||||
"workdir": str(self._workdir),
|
"workdir": str(self._workdir),
|
||||||
"tool_profile": self._profile.name,
|
"tool_profile": self._profile.name,
|
||||||
"tool_event_count": tool_event_count,
|
"tool_event_count": tool_event_count,
|
||||||
|
"cost_usd": cost,
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -544,6 +544,7 @@ def _process_profiled_run_active(
|
||||||
client.config,
|
client.config,
|
||||||
report_to_hub=report_to_hub,
|
report_to_hub=report_to_hub,
|
||||||
cancel=cancel,
|
cancel=cancel,
|
||||||
|
transaction=active_tx,
|
||||||
)
|
)
|
||||||
safe_evidence = normalise_execution_evidence_for_close(
|
safe_evidence = normalise_execution_evidence_for_close(
|
||||||
gateway_result["evidence"]
|
gateway_result["evidence"]
|
||||||
|
|
|
||||||
|
|
@ -300,6 +300,7 @@ def _cmd_run(args: argparse.Namespace) -> int:
|
||||||
"tool_profile": result.tool_profile,
|
"tool_profile": result.tool_profile,
|
||||||
"budget_tokens": result.budget_tokens,
|
"budget_tokens": result.budget_tokens,
|
||||||
"tokens_spent": result.tokens_spent,
|
"tokens_spent": result.tokens_spent,
|
||||||
|
"cost_usd": result.cost_usd,
|
||||||
"execution_time_s": round(result.execution_time_s, 3),
|
"execution_time_s": round(result.execution_time_s, 3),
|
||||||
"reason": result.reason,
|
"reason": result.reason,
|
||||||
"model": result.model,
|
"model": result.model,
|
||||||
|
|
|
||||||
|
|
@ -51,6 +51,7 @@ _EVIDENCE_STRING_FIELDS = (
|
||||||
_EVIDENCE_NUMBER_FIELDS = (
|
_EVIDENCE_NUMBER_FIELDS = (
|
||||||
"duration_s",
|
"duration_s",
|
||||||
"tokens_spent",
|
"tokens_spent",
|
||||||
|
"cost_usd",
|
||||||
"token_budget",
|
"token_budget",
|
||||||
"tool_events_count",
|
"tool_events_count",
|
||||||
)
|
)
|
||||||
|
|
@ -142,6 +143,7 @@ def execute_profiled_run(
|
||||||
request_factory: Callable[..., Any] | None = None,
|
request_factory: Callable[..., Any] | None = None,
|
||||||
gateway: Callable[[Any], Any] | None = None,
|
gateway: Callable[[Any], Any] | None = None,
|
||||||
cancel: ExecutionCancel | None = None,
|
cancel: ExecutionCancel | None = None,
|
||||||
|
transaction: Any = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
"""Invoke Glas and return its complete JSON-compatible GatewayResult."""
|
"""Invoke Glas and return its complete JSON-compatible GatewayResult."""
|
||||||
guard = resolve_cancel(cancel)
|
guard = resolve_cancel(cancel)
|
||||||
|
|
@ -159,9 +161,14 @@ def execute_profiled_run(
|
||||||
request_factory = request_factory or ExecutionRequest
|
request_factory = request_factory or ExecutionRequest
|
||||||
gateway = gateway or run_execution
|
gateway = gateway or run_execution
|
||||||
|
|
||||||
|
transfer = None
|
||||||
|
if run.repository_grant is not None:
|
||||||
|
from rein_aharness.repository_artifact import RepositoryArtifactTransfer
|
||||||
|
transfer = RepositoryArtifactTransfer(transaction, run.repository_grant, cancel=guard)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
request = request_factory(**_request_kwargs(run, config, report_to_hub))
|
request = request_factory(**_request_kwargs(run, config, report_to_hub))
|
||||||
result = gateway(request)
|
result = gateway(request, artifact_capture=transfer.capture) if transfer else gateway(request)
|
||||||
raw = result.model_dump(mode="json") if hasattr(result, "model_dump") else result
|
raw = result.model_dump(mode="json") if hasattr(result, "model_dump") else result
|
||||||
except ExecutionCancelled:
|
except ExecutionCancelled:
|
||||||
raise
|
raise
|
||||||
|
|
@ -178,4 +185,10 @@ def execute_profiled_run(
|
||||||
raise GlasExecutionError("Glas gateway returned an invalid GatewayResult")
|
raise GlasExecutionError("Glas gateway returned an invalid GatewayResult")
|
||||||
if not isinstance(raw.get("evidence"), dict):
|
if not isinstance(raw.get("evidence"), dict):
|
||||||
raise GlasExecutionError("Glas GatewayResult is missing execution evidence")
|
raise GlasExecutionError("Glas GatewayResult is missing execution evidence")
|
||||||
|
if transfer is not None and raw["ok"]:
|
||||||
|
evidence = raw["evidence"]
|
||||||
|
if evidence.get("session_cleanup") != "succeeded" or evidence.get("sandbox_destroy") != "succeeded":
|
||||||
|
raise GlasExecutionError("artifact import requires confirmed session cleanup and sandbox teardown")
|
||||||
|
imported = transfer.import_after_teardown(evidence.get("commit_sha"))
|
||||||
|
raw["artifact_import"] = imported
|
||||||
return raw
|
return raw
|
||||||
|
|
|
||||||
90
rein_aharness/native_limits.py
Normal file
90
rein_aharness/native_limits.py
Normal file
|
|
@ -0,0 +1,90 @@
|
||||||
|
"""Claude CLI controls and bounded terminal accounting, not a spend admission grant.
|
||||||
|
|
||||||
|
Daily/total reservation and provider cap semantics remain the operating owner's
|
||||||
|
responsibility. A post-run cost check cannot undo an already charged request.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
import math
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
def validate_limits(max_budget_usd: Any, max_turns: Any) -> None:
|
||||||
|
if max_budget_usd is not None and (
|
||||||
|
isinstance(max_budget_usd, bool)
|
||||||
|
or not isinstance(max_budget_usd, (int, float))
|
||||||
|
or not math.isfinite(max_budget_usd)
|
||||||
|
or max_budget_usd <= 0
|
||||||
|
):
|
||||||
|
raise ValueError("max_budget_usd must be a finite positive USD amount")
|
||||||
|
if max_turns is not None and (
|
||||||
|
isinstance(max_turns, bool) or not isinstance(max_turns, int) or max_turns <= 0
|
||||||
|
):
|
||||||
|
raise ValueError("max_turns must be a positive integer")
|
||||||
|
|
||||||
|
|
||||||
|
class NativeLimitError(RuntimeError):
|
||||||
|
def __init__(self, message: str, *, cost_usd: float | None = None) -> None:
|
||||||
|
super().__init__(message)
|
||||||
|
self.cost_usd = cost_usd
|
||||||
|
|
||||||
|
|
||||||
|
def terminal_accounting(
|
||||||
|
result: Any, *, max_budget_usd: float | None, max_turns: int | None
|
||||||
|
) -> tuple[dict, float]:
|
||||||
|
if not isinstance(result, dict) or result.get("type") != "result":
|
||||||
|
raise NativeLimitError(
|
||||||
|
"controlled Claude run did not return terminal accounting"
|
||||||
|
)
|
||||||
|
cost = result.get("total_cost_usd")
|
||||||
|
if (
|
||||||
|
isinstance(cost, bool)
|
||||||
|
or not isinstance(cost, (int, float))
|
||||||
|
or not math.isfinite(cost)
|
||||||
|
or cost < 0
|
||||||
|
):
|
||||||
|
raise NativeLimitError("controlled Claude run returned invalid cost accounting")
|
||||||
|
cost = float(cost)
|
||||||
|
if result.get("subtype") != "success" or result.get("is_error") is not False:
|
||||||
|
raise NativeLimitError(
|
||||||
|
"controlled Claude run did not complete successfully", cost_usd=cost
|
||||||
|
)
|
||||||
|
if max_budget_usd is not None and cost > max_budget_usd:
|
||||||
|
raise NativeLimitError(
|
||||||
|
"controlled Claude run exceeded its native USD limit", cost_usd=cost
|
||||||
|
)
|
||||||
|
turns = result.get("num_turns")
|
||||||
|
if (
|
||||||
|
isinstance(turns, bool)
|
||||||
|
or not isinstance(turns, int)
|
||||||
|
or turns < 0
|
||||||
|
or (max_turns is not None and turns > max_turns)
|
||||||
|
):
|
||||||
|
raise NativeLimitError(
|
||||||
|
"controlled Claude run returned invalid turn accounting", cost_usd=cost
|
||||||
|
)
|
||||||
|
usage = result.get("usage")
|
||||||
|
if not isinstance(usage, dict):
|
||||||
|
raise NativeLimitError(
|
||||||
|
"controlled Claude run returned no token accounting", cost_usd=cost
|
||||||
|
)
|
||||||
|
counts = {}
|
||||||
|
for name in (
|
||||||
|
"input_tokens",
|
||||||
|
"output_tokens",
|
||||||
|
"cache_creation_input_tokens",
|
||||||
|
"cache_read_input_tokens",
|
||||||
|
):
|
||||||
|
value = usage.get(name, 0 if name.startswith("cache_") else None)
|
||||||
|
if isinstance(value, bool) or not isinstance(value, int) or value < 0:
|
||||||
|
raise NativeLimitError(
|
||||||
|
"controlled Claude run returned invalid token accounting", cost_usd=cost
|
||||||
|
)
|
||||||
|
counts[name] = value
|
||||||
|
return {
|
||||||
|
"prompt_tokens": counts["input_tokens"]
|
||||||
|
+ counts["cache_creation_input_tokens"]
|
||||||
|
+ counts["cache_read_input_tokens"],
|
||||||
|
"completion_tokens": counts["output_tokens"],
|
||||||
|
"total_tokens": sum(counts.values()),
|
||||||
|
}, cost
|
||||||
320
rein_aharness/repository_artifact.py
Normal file
320
rein_aharness/repository_artifact.py
Normal file
|
|
@ -0,0 +1,320 @@
|
||||||
|
"""Bounded sandbox Git return into an already-owned repository transaction.
|
||||||
|
|
||||||
|
Artifact bytes stay process-local and are discarded on any gateway/teardown
|
||||||
|
failure. They never enter GatewayResult, Hub, metrics or the close outbox.
|
||||||
|
The initial contract supports one local commit, matching the factory pilot.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import base64
|
||||||
|
import binascii
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import tempfile
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from rein_aharness.execution_cancel import ExecutionCancel, resolve_cancel
|
||||||
|
from rein_aharness.repository_grant import RepositoryGrant
|
||||||
|
from rein_aharness.repository_transaction import (
|
||||||
|
RepositoryTransaction,
|
||||||
|
RepositoryTransactionError,
|
||||||
|
)
|
||||||
|
|
||||||
|
MAX_BUNDLE_BYTES = 128 * 1024
|
||||||
|
MAX_NEW_OBJECT_BYTES = 2 * 1024 * 1024
|
||||||
|
_OID = re.compile(r"[0-9a-f]{40}|[0-9a-f]{64}")
|
||||||
|
|
||||||
|
|
||||||
|
class RepositoryArtifactError(RepositoryTransactionError):
|
||||||
|
"""A sandbox result cannot be safely imported under this transaction."""
|
||||||
|
|
||||||
|
|
||||||
|
# Executed inside the selected sandbox by its owner, with no host path access.
|
||||||
|
# The owner transport separately caps output and wall time. No binary or source
|
||||||
|
# content is placed in command arguments or exception messages.
|
||||||
|
_EXPORT = r"""
|
||||||
|
import base64, json, pathlib, subprocess, sys, tempfile
|
||||||
|
repo, baseline, expected, limit = sys.argv[1:]
|
||||||
|
limit = int(limit)
|
||||||
|
def git(*args):
|
||||||
|
p = subprocess.run(["git", "-C", repo, "-c", "core.hooksPath=/dev/null", *args], capture_output=True, timeout=20)
|
||||||
|
if p.returncode: raise RuntimeError("sandbox artifact Git operation refused")
|
||||||
|
return p.stdout
|
||||||
|
if git("rev-parse", "HEAD").decode().strip() != expected:
|
||||||
|
raise RuntimeError("sandbox artifact head mismatch")
|
||||||
|
if git("status", "--porcelain=v1", "--untracked-files=all").strip():
|
||||||
|
raise RuntimeError("sandbox artifact checkout is not clean")
|
||||||
|
if git("rev-list", "--count", baseline + "..HEAD").strip() != b"1":
|
||||||
|
raise RuntimeError("sandbox artifact must contain one commit")
|
||||||
|
git("merge-base", "--is-ancestor", baseline, "HEAD")
|
||||||
|
with tempfile.TemporaryDirectory(prefix="rein-artifact-") as d:
|
||||||
|
path = pathlib.Path(d) / "return.bundle"
|
||||||
|
git("bundle", "create", str(path), baseline + "..HEAD")
|
||||||
|
if path.stat().st_size > limit: raise RuntimeError("sandbox artifact too large")
|
||||||
|
payload = path.read_bytes()
|
||||||
|
print(json.dumps({"version": "1", "baseline": baseline, "head": expected,
|
||||||
|
"bundle": base64.b64encode(payload).decode("ascii")}))
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def _git(repo: Path, *args: str) -> bytes:
|
||||||
|
# Do not permit inherited Git directory/object/config overrides to redirect
|
||||||
|
# the transfer. All transport is between owner-selected local paths.
|
||||||
|
env = {k: v for k, v in os.environ.items() if not k.startswith("GIT_")}
|
||||||
|
env.update(
|
||||||
|
GIT_CONFIG_NOSYSTEM="1",
|
||||||
|
GIT_CONFIG_GLOBAL=os.devnull,
|
||||||
|
GIT_TERMINAL_PROMPT="0",
|
||||||
|
GIT_NO_REPLACE_OBJECTS="1",
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
proc = subprocess.run(
|
||||||
|
[
|
||||||
|
"git",
|
||||||
|
"-C",
|
||||||
|
str(repo),
|
||||||
|
"-c",
|
||||||
|
"core.hooksPath=/dev/null",
|
||||||
|
"-c",
|
||||||
|
"core.fsmonitor=false",
|
||||||
|
"-c",
|
||||||
|
"protocol.allow=never",
|
||||||
|
"-c",
|
||||||
|
"protocol.file.allow=always",
|
||||||
|
*args,
|
||||||
|
],
|
||||||
|
env=env,
|
||||||
|
capture_output=True,
|
||||||
|
timeout=30,
|
||||||
|
)
|
||||||
|
except (OSError, subprocess.TimeoutExpired) as exc:
|
||||||
|
raise RepositoryArtifactError(
|
||||||
|
"artifact Git operation unavailable or timed out"
|
||||||
|
) from exc
|
||||||
|
if proc.returncode:
|
||||||
|
raise RepositoryArtifactError("artifact Git validation or import refused")
|
||||||
|
return proc.stdout
|
||||||
|
|
||||||
|
|
||||||
|
class RepositoryArtifactTransfer:
|
||||||
|
"""Trusted worker-owned capture; source mutation waits for successful teardown."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
transaction: RepositoryTransaction,
|
||||||
|
grant: RepositoryGrant,
|
||||||
|
*,
|
||||||
|
cancel: ExecutionCancel | None = None,
|
||||||
|
) -> None:
|
||||||
|
if (
|
||||||
|
not isinstance(transaction, RepositoryTransaction)
|
||||||
|
or not transaction.locked
|
||||||
|
or transaction.baseline is None
|
||||||
|
):
|
||||||
|
raise RepositoryArtifactError(
|
||||||
|
"artifact transfer requires the active repository transaction"
|
||||||
|
)
|
||||||
|
if grant.min_commits != 1 or grant.max_commits != 1 or grant.publish:
|
||||||
|
raise RepositoryArtifactError(
|
||||||
|
"artifact transfer v1 requires exactly one local commit"
|
||||||
|
)
|
||||||
|
self.transaction = transaction
|
||||||
|
self.grant = grant
|
||||||
|
self.cancel = resolve_cancel(cancel)
|
||||||
|
self._bundle: bytes | None = None
|
||||||
|
self._head: str | None = None
|
||||||
|
|
||||||
|
def _check(self) -> None:
|
||||||
|
if self.cancel is not None:
|
||||||
|
self.cancel.check()
|
||||||
|
self.transaction.assert_baseline_unchanged()
|
||||||
|
|
||||||
|
def capture(self, sandbox: Any, summary: Any) -> None:
|
||||||
|
"""Called by Glas after session cleanup, before sandbox destruction."""
|
||||||
|
from glas_harness.transport import transport_from_sandbox
|
||||||
|
|
||||||
|
self._check()
|
||||||
|
if self._bundle is not None:
|
||||||
|
raise RepositoryArtifactError("artifact capture was already completed")
|
||||||
|
head = summary.commit_sha
|
||||||
|
if not isinstance(head, str) or not _OID.fullmatch(head):
|
||||||
|
raise RepositoryArtifactError(
|
||||||
|
"sandbox did not report an exact commit identity"
|
||||||
|
)
|
||||||
|
transport = transport_from_sandbox(sandbox)
|
||||||
|
if transport.kind != "local_namespace":
|
||||||
|
raise RepositoryArtifactError(
|
||||||
|
"artifact transfer v1 requires bounded sandbox owner execution"
|
||||||
|
)
|
||||||
|
result = transport.run(
|
||||||
|
[
|
||||||
|
"python3",
|
||||||
|
"-c",
|
||||||
|
_EXPORT,
|
||||||
|
transport.workspace,
|
||||||
|
self.transaction.baseline.head,
|
||||||
|
head,
|
||||||
|
str(MAX_BUNDLE_BYTES),
|
||||||
|
],
|
||||||
|
timeout=30,
|
||||||
|
)
|
||||||
|
if result.returncode:
|
||||||
|
raise RepositoryArtifactError("sandbox artifact export refused")
|
||||||
|
if len(result.stdout) > (MAX_BUNDLE_BYTES * 4 // 3) + 1024:
|
||||||
|
raise RepositoryArtifactError("sandbox artifact exceeds transport limit")
|
||||||
|
try:
|
||||||
|
payload = json.loads(result.stdout)
|
||||||
|
if (
|
||||||
|
not isinstance(payload, dict)
|
||||||
|
or set(payload) != {"version", "baseline", "head", "bundle"}
|
||||||
|
or payload["version"] != "1"
|
||||||
|
or payload["baseline"] != self.transaction.baseline.head
|
||||||
|
or payload["head"] != head
|
||||||
|
):
|
||||||
|
raise ValueError("artifact envelope mismatch")
|
||||||
|
bundle = base64.b64decode(payload["bundle"], validate=True)
|
||||||
|
if not bundle or len(bundle) > MAX_BUNDLE_BYTES:
|
||||||
|
raise ValueError("artifact byte limit")
|
||||||
|
except (ValueError, TypeError, binascii.Error) as exc:
|
||||||
|
raise RepositoryArtifactError(
|
||||||
|
"sandbox artifact envelope is invalid"
|
||||||
|
) from exc
|
||||||
|
self._bundle, self._head = bundle, head
|
||||||
|
self._check()
|
||||||
|
|
||||||
|
def import_after_teardown(self, reported_head: str | None) -> dict[str, Any]:
|
||||||
|
"""Validate in a disposable checkout, then fast-forward under the live lease."""
|
||||||
|
self._check()
|
||||||
|
if self._bundle is None or self._head is None or reported_head != self._head:
|
||||||
|
raise RepositoryArtifactError(
|
||||||
|
"successful gateway did not return the captured artifact"
|
||||||
|
)
|
||||||
|
baseline = self.transaction.baseline
|
||||||
|
with tempfile.TemporaryDirectory(prefix="rein-artifact-verify-") as temporary:
|
||||||
|
root = Path(temporary)
|
||||||
|
bundle_path = root / "return.bundle"
|
||||||
|
bundle_path.write_bytes(self._bundle)
|
||||||
|
bundle_path.chmod(0o600)
|
||||||
|
staging = root / "checkout"
|
||||||
|
_git(
|
||||||
|
root,
|
||||||
|
"clone",
|
||||||
|
"--quiet",
|
||||||
|
"--no-local",
|
||||||
|
"--no-hardlinks",
|
||||||
|
"--no-checkout",
|
||||||
|
"--",
|
||||||
|
str(baseline.repo_root),
|
||||||
|
str(staging),
|
||||||
|
)
|
||||||
|
_git(staging, "checkout", "--quiet", "--detach", baseline.head)
|
||||||
|
with RepositoryTransaction(staging, state_dir=root / "state") as inspection:
|
||||||
|
_git(staging, "bundle", "verify", str(bundle_path))
|
||||||
|
advertised = (
|
||||||
|
_git(staging, "bundle", "list-heads", str(bundle_path))
|
||||||
|
.decode()
|
||||||
|
.strip()
|
||||||
|
)
|
||||||
|
if advertised != self._head + " HEAD":
|
||||||
|
raise RepositoryArtifactError(
|
||||||
|
"artifact must advertise only the expected HEAD"
|
||||||
|
)
|
||||||
|
_git(
|
||||||
|
staging,
|
||||||
|
"fetch",
|
||||||
|
"--quiet",
|
||||||
|
"--no-tags",
|
||||||
|
"--no-write-fetch-head",
|
||||||
|
str(bundle_path),
|
||||||
|
"HEAD",
|
||||||
|
)
|
||||||
|
if (
|
||||||
|
_git(
|
||||||
|
staging, "rev-list", "--count", f"{baseline.head}..{self._head}"
|
||||||
|
).strip()
|
||||||
|
!= b"1"
|
||||||
|
):
|
||||||
|
raise RepositoryArtifactError(
|
||||||
|
"artifact contains an unexpected commit count"
|
||||||
|
)
|
||||||
|
_git(staging, "merge-base", "--is-ancestor", baseline.head, self._head)
|
||||||
|
# Reject symlinks and gitlinks before checking out untrusted data.
|
||||||
|
changes = _git(
|
||||||
|
staging,
|
||||||
|
"diff",
|
||||||
|
"--raw",
|
||||||
|
"--no-renames",
|
||||||
|
"-z",
|
||||||
|
baseline.head,
|
||||||
|
self._head,
|
||||||
|
"--",
|
||||||
|
).split(b"\0")
|
||||||
|
for header in changes[::2]:
|
||||||
|
if header and header.split()[1] not in (
|
||||||
|
b"100644",
|
||||||
|
b"100755",
|
||||||
|
b"000000",
|
||||||
|
):
|
||||||
|
raise RepositoryArtifactError(
|
||||||
|
"artifact changes an unsupported file mode"
|
||||||
|
)
|
||||||
|
oids = _git(
|
||||||
|
staging,
|
||||||
|
"rev-list",
|
||||||
|
"--objects",
|
||||||
|
"--no-object-names",
|
||||||
|
f"{baseline.head}..{self._head}",
|
||||||
|
).splitlines()
|
||||||
|
if len(oids) > 1024:
|
||||||
|
raise RepositoryArtifactError(
|
||||||
|
"artifact contains too many new objects"
|
||||||
|
)
|
||||||
|
total = 0
|
||||||
|
for oid in oids:
|
||||||
|
total += int(_git(staging, "cat-file", "-s", oid.decode()).strip())
|
||||||
|
if total > MAX_NEW_OBJECT_BYTES:
|
||||||
|
raise RepositoryArtifactError(
|
||||||
|
"artifact expands beyond the accepted byte limit"
|
||||||
|
)
|
||||||
|
_git(
|
||||||
|
staging, "merge", "--ff-only", "--no-edit", "--no-stat", self._head
|
||||||
|
)
|
||||||
|
inspection.validate_acceptance(self.grant.acceptance_policy())
|
||||||
|
self._check()
|
||||||
|
_git(
|
||||||
|
baseline.repo_root,
|
||||||
|
"fetch",
|
||||||
|
"--quiet",
|
||||||
|
"--no-tags",
|
||||||
|
"--no-write-fetch-head",
|
||||||
|
str(bundle_path),
|
||||||
|
"HEAD",
|
||||||
|
)
|
||||||
|
self._check()
|
||||||
|
_git(
|
||||||
|
baseline.repo_root,
|
||||||
|
"merge",
|
||||||
|
"--ff-only",
|
||||||
|
"--no-edit",
|
||||||
|
"--no-stat",
|
||||||
|
self._head,
|
||||||
|
)
|
||||||
|
if self.cancel is not None:
|
||||||
|
self.cancel.check()
|
||||||
|
accepted = self.transaction.validate_acceptance(
|
||||||
|
self.grant.acceptance_policy()
|
||||||
|
)
|
||||||
|
evidence = {
|
||||||
|
"version": "1",
|
||||||
|
"sha256": hashlib.sha256(self._bundle).hexdigest(),
|
||||||
|
"bytes": len(self._bundle),
|
||||||
|
"head": accepted.head,
|
||||||
|
"imported": True,
|
||||||
|
}
|
||||||
|
self._bundle = None
|
||||||
|
return evidence
|
||||||
|
|
@ -296,6 +296,15 @@ class RepositoryTransaction:
|
||||||
evidence["acceptance"] = self.acceptance.evidence()
|
evidence["acceptance"] = self.acceptance.evidence()
|
||||||
return evidence
|
return evidence
|
||||||
|
|
||||||
|
def assert_baseline_unchanged(self) -> None:
|
||||||
|
"""Refuse artifact import if the locked source no longer matches admission."""
|
||||||
|
if not self.locked or self.baseline is None:
|
||||||
|
raise RepositoryTransactionError("baseline verification requires an active transaction")
|
||||||
|
baseline = self.baseline
|
||||||
|
current = _capture_baseline(baseline.repo_root, baseline.git_common_dir, baseline.repo_id)
|
||||||
|
if current != baseline:
|
||||||
|
raise RepositoryAcceptanceError("baseline-changed", "source changed before artifact import")
|
||||||
|
|
||||||
def validate_acceptance(
|
def validate_acceptance(
|
||||||
self,
|
self,
|
||||||
policy: RepositoryAcceptancePolicy,
|
policy: RepositoryAcceptancePolicy,
|
||||||
|
|
|
||||||
|
|
@ -30,6 +30,7 @@ from rein_aharness.repository_transaction import (
|
||||||
RepositoryTransactionError,
|
RepositoryTransactionError,
|
||||||
)
|
)
|
||||||
from rein_aharness.taskspec import TaskSpec
|
from rein_aharness.taskspec import TaskSpec
|
||||||
|
from rein_aharness.native_limits import NativeLimitError
|
||||||
|
|
||||||
PROMPT_TEMPLATE = """\
|
PROMPT_TEMPLATE = """\
|
||||||
You are an unattended executor session (agent persona below, if any).
|
You are an unattended executor session (agent persona below, if any).
|
||||||
|
|
@ -63,6 +64,7 @@ class RunResult:
|
||||||
tool_profile: str = ""
|
tool_profile: str = ""
|
||||||
budget_tokens: int | None = None
|
budget_tokens: int | None = None
|
||||||
tokens_spent: int | None = None
|
tokens_spent: int | None = None
|
||||||
|
cost_usd: float | None = None
|
||||||
execution_time_s: float = 0.0
|
execution_time_s: float = 0.0
|
||||||
# Real-time per-tool-call audit events, populated only when run_task is
|
# Real-time per-tool-call audit events, populated only when run_task is
|
||||||
# called with emit_tool_events=True. See adapter.py's module docstring
|
# called with emit_tool_events=True. See adapter.py's module docstring
|
||||||
|
|
@ -233,11 +235,14 @@ def _execute_locked_task(
|
||||||
timeout_seconds=spec.timeout_seconds,
|
timeout_seconds=spec.timeout_seconds,
|
||||||
skip_if_exists=False,
|
skip_if_exists=False,
|
||||||
budget_tracker=budget_tracker,
|
budget_tracker=budget_tracker,
|
||||||
|
model_params={"max_budget_usd": spec.max_budget_usd, "max_turns": spec.max_turns},
|
||||||
)
|
)
|
||||||
|
|
||||||
started = time.monotonic()
|
started = time.monotonic()
|
||||||
|
cost_usd = None
|
||||||
try:
|
try:
|
||||||
response = adapter.execute_prompt(prompt, config)
|
response = adapter.execute_prompt(prompt, config)
|
||||||
|
cost_usd = response.metadata.get("cost_usd")
|
||||||
session_output = response.content
|
session_output = response.content
|
||||||
resolved_model = response.model
|
resolved_model = response.model
|
||||||
session_ok = True
|
session_ok = True
|
||||||
|
|
@ -248,6 +253,8 @@ def _execute_locked_task(
|
||||||
reason = f"execution cancelled ({exc.reason})"
|
reason = f"execution cancelled ({exc.reason})"
|
||||||
resolved_model = model
|
resolved_model = model
|
||||||
except Exception as exc: # adapter / budget failures must still be reported
|
except Exception as exc: # adapter / budget failures must still be reported
|
||||||
|
if isinstance(exc, NativeLimitError):
|
||||||
|
cost_usd = exc.cost_usd
|
||||||
session_output = ""
|
session_output = ""
|
||||||
session_ok = False
|
session_ok = False
|
||||||
reason = f"session failed: {exc}"
|
reason = f"session failed: {exc}"
|
||||||
|
|
@ -273,6 +280,7 @@ def _execute_locked_task(
|
||||||
|
|
||||||
metric_metadata = {
|
metric_metadata = {
|
||||||
"task_title": spec.title,
|
"task_title": spec.title,
|
||||||
|
"cost_usd": cost_usd,
|
||||||
"tool_profile": profile.name,
|
"tool_profile": profile.name,
|
||||||
"labels": list(spec.labels),
|
"labels": list(spec.labels),
|
||||||
"completion_event_type": spec.completion_event_type,
|
"completion_event_type": spec.completion_event_type,
|
||||||
|
|
@ -325,6 +333,7 @@ def _execute_locked_task(
|
||||||
tool_profile=profile.name,
|
tool_profile=profile.name,
|
||||||
budget_tokens=budget_tokens,
|
budget_tokens=budget_tokens,
|
||||||
tokens_spent=tokens_spent,
|
tokens_spent=tokens_spent,
|
||||||
|
cost_usd=cost_usd,
|
||||||
execution_time_s=execution_time_s,
|
execution_time_s=execution_time_s,
|
||||||
tool_events=collected_events,
|
tool_events=collected_events,
|
||||||
transaction=transaction_evidence,
|
transaction=transaction_evidence,
|
||||||
|
|
|
||||||
|
|
@ -11,6 +11,7 @@ import json
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
|
from rein_aharness.native_limits import validate_limits
|
||||||
from rein_aharness.repository_grant import RepositoryGrant, RepositoryGrantError
|
from rein_aharness.repository_grant import RepositoryGrant, RepositoryGrantError
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -29,6 +30,14 @@ class TaskSpec:
|
||||||
completion_event_type: str = "executor_run"
|
completion_event_type: str = "executor_run"
|
||||||
timeout_seconds: int = 900
|
timeout_seconds: int = 900
|
||||||
repository_grant: RepositoryGrant | None = None
|
repository_grant: RepositoryGrant | None = None
|
||||||
|
max_budget_usd: float | None = None
|
||||||
|
max_turns: int | None = None
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
try:
|
||||||
|
validate_limits(self.max_budget_usd, self.max_turns)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise TaskSpecError(str(exc)) from exc
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def from_file(cls, path: str | Path) -> "TaskSpec":
|
def from_file(cls, path: str | Path) -> "TaskSpec":
|
||||||
|
|
@ -55,4 +64,6 @@ class TaskSpec:
|
||||||
completion_event_type=str(raw.get("completion_event_type", "executor_run")),
|
completion_event_type=str(raw.get("completion_event_type", "executor_run")),
|
||||||
timeout_seconds=int(raw.get("timeout_seconds", 900)),
|
timeout_seconds=int(raw.get("timeout_seconds", 900)),
|
||||||
repository_grant=grant,
|
repository_grant=grant,
|
||||||
|
max_budget_usd=raw.get("max_budget_usd"),
|
||||||
|
max_turns=raw.get("max_turns"),
|
||||||
)
|
)
|
||||||
|
|
|
||||||
137
tests/test_native_limits.py
Normal file
137
tests/test_native_limits.py
Normal file
|
|
@ -0,0 +1,137 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
import json
|
||||||
|
import subprocess
|
||||||
|
from unittest.mock import MagicMock, patch
|
||||||
|
import pytest
|
||||||
|
from llm_connect.models import BudgetTracker, RunConfig
|
||||||
|
from rein_aharness.adapter import AgenticClaudeCodeAdapter
|
||||||
|
from rein_aharness.native_limits import NativeLimitError, terminal_accounting
|
||||||
|
from rein_aharness.taskspec import TaskSpec, TaskSpecError
|
||||||
|
from test_adapter import _fake_proc
|
||||||
|
|
||||||
|
|
||||||
|
def terminal(**overrides):
|
||||||
|
return {
|
||||||
|
"type": "result",
|
||||||
|
"subtype": "success",
|
||||||
|
"is_error": False,
|
||||||
|
"result": "done",
|
||||||
|
"num_turns": 2,
|
||||||
|
"total_cost_usd": 0.1,
|
||||||
|
"usage": {"input_tokens": 10, "output_tokens": 2, "cache_read_input_tokens": 3},
|
||||||
|
**overrides,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("stream", [False, True])
|
||||||
|
def test_native_controls_reach_cli_and_account_usage(tmp_path, stream):
|
||||||
|
adapter = AgenticClaudeCodeAdapter(
|
||||||
|
workdir=tmp_path, on_tool_event=(lambda e: None) if stream else None
|
||||||
|
)
|
||||||
|
config = RunConfig(
|
||||||
|
timeout_seconds=30,
|
||||||
|
model_params={"max_budget_usd": 0.25, "max_turns": 3},
|
||||||
|
budget_tracker=BudgetTracker(total=100),
|
||||||
|
)
|
||||||
|
payload = json.dumps(terminal())
|
||||||
|
proc = _fake_proc([payload + "\n"]) if stream else MagicMock()
|
||||||
|
if not stream:
|
||||||
|
proc.communicate.return_value = (payload, "")
|
||||||
|
proc.returncode = 0
|
||||||
|
with (
|
||||||
|
patch(
|
||||||
|
"rein_aharness.adapter.subprocess.run",
|
||||||
|
return_value=subprocess.CompletedProcess(
|
||||||
|
[], 0, "2.1.263 (Claude Code)\n", ""
|
||||||
|
),
|
||||||
|
),
|
||||||
|
patch("rein_aharness.adapter.subprocess.Popen", return_value=proc) as invoke,
|
||||||
|
):
|
||||||
|
result = adapter.execute_prompt("task", config)
|
||||||
|
argv = invoke.call_args.args[0]
|
||||||
|
assert argv[argv.index("--max-budget-usd") + 1] == "0.25"
|
||||||
|
assert argv[argv.index("--max-turns") + 1] == "3"
|
||||||
|
assert result.usage["total_tokens"] == config.budget_tracker.spent == 15
|
||||||
|
assert result.metadata["cost_usd"] == 0.1
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"changes",
|
||||||
|
[
|
||||||
|
{"total_cost_usd": None},
|
||||||
|
{"total_cost_usd": float("nan")},
|
||||||
|
{"total_cost_usd": True},
|
||||||
|
{"total_cost_usd": 0.3},
|
||||||
|
{"num_turns": 4},
|
||||||
|
{"usage": {}},
|
||||||
|
{"subtype": "error_max_budget_usd", "is_error": True},
|
||||||
|
{"subtype": "error_max_turns", "is_error": True},
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_unaccounted_or_exhausted_result_cannot_succeed(changes):
|
||||||
|
with pytest.raises(NativeLimitError):
|
||||||
|
terminal_accounting(terminal(**changes), max_budget_usd=0.25, max_turns=3)
|
||||||
|
|
||||||
|
|
||||||
|
def test_error_preserves_only_bounded_cost():
|
||||||
|
with pytest.raises(NativeLimitError) as caught:
|
||||||
|
terminal_accounting(
|
||||||
|
terminal(subtype="error_max_budget_usd", is_error=True, errors=["secret"]),
|
||||||
|
max_budget_usd=0.25,
|
||||||
|
max_turns=3,
|
||||||
|
)
|
||||||
|
assert caught.value.cost_usd == 0.1 and "secret" not in str(caught.value)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("version", ["2.1.216 (Claude Code)", "unrecognized"])
|
||||||
|
def test_old_cli_refuses_before_prompt(tmp_path, version):
|
||||||
|
adapter = AgenticClaudeCodeAdapter(workdir=tmp_path)
|
||||||
|
with (
|
||||||
|
patch(
|
||||||
|
"rein_aharness.adapter.subprocess.run",
|
||||||
|
return_value=subprocess.CompletedProcess([], 0, version, ""),
|
||||||
|
),
|
||||||
|
patch("rein_aharness.adapter.subprocess.Popen") as invoke,
|
||||||
|
):
|
||||||
|
with pytest.raises(NativeLimitError, match="2.1.217"):
|
||||||
|
adapter.execute_prompt(
|
||||||
|
"task", RunConfig(model_params={"max_budget_usd": 0.25})
|
||||||
|
)
|
||||||
|
invoke.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"limits",
|
||||||
|
[
|
||||||
|
{"max_budget_usd": True},
|
||||||
|
{"max_budget_usd": -1},
|
||||||
|
{"max_budget_usd": float("inf")},
|
||||||
|
{"max_turns": True},
|
||||||
|
{"max_turns": 1.5},
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_invalid_task_limits_refuse(tmp_path, limits):
|
||||||
|
with pytest.raises(TaskSpecError):
|
||||||
|
TaskSpec(title="task", description="d", target_repo=tmp_path, **limits)
|
||||||
|
|
||||||
|
|
||||||
|
def test_runner_forwards_task_limits(tmp_path):
|
||||||
|
from test_runner import CommittingAdapter, _make_repo
|
||||||
|
from rein_aharness.runner import run_task
|
||||||
|
|
||||||
|
repo = _make_repo(tmp_path)
|
||||||
|
adapter = CommittingAdapter(repo)
|
||||||
|
result = run_task(
|
||||||
|
TaskSpec(
|
||||||
|
title="task",
|
||||||
|
description="d",
|
||||||
|
target_repo=repo,
|
||||||
|
max_budget_usd=0.25,
|
||||||
|
max_turns=3,
|
||||||
|
),
|
||||||
|
adapter=adapter,
|
||||||
|
report_to_hub=False,
|
||||||
|
write_metrics=False,
|
||||||
|
)
|
||||||
|
assert result.ok
|
||||||
|
assert adapter.configs[0].model_params == {"max_budget_usd": 0.25, "max_turns": 3}
|
||||||
312
tests/test_repository_artifact.py
Normal file
312
tests/test_repository_artifact.py
Normal file
|
|
@ -0,0 +1,312 @@
|
||||||
|
"""Actual Git export/import across separate workspaces; no gateway writes source."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
from types import SimpleNamespace
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from rein_aharness.execution_cancel import ExecutionCancel, ExecutionCancelled
|
||||||
|
from rein_aharness.repository_artifact import (
|
||||||
|
RepositoryArtifactError,
|
||||||
|
RepositoryArtifactTransfer,
|
||||||
|
)
|
||||||
|
from rein_aharness.repository_grant import RepositoryGrant
|
||||||
|
from rein_aharness.repository_transaction import (
|
||||||
|
RepositoryAcceptanceError,
|
||||||
|
RepositoryTransaction,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def git(repo, *args):
|
||||||
|
return (
|
||||||
|
subprocess.check_output(
|
||||||
|
["git", "-C", str(repo), *args], stderr=subprocess.DEVNULL
|
||||||
|
)
|
||||||
|
.decode()
|
||||||
|
.strip()
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def commit(repo, name="result.txt", content="accepted change\n"):
|
||||||
|
path = repo / name
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
path.write_text(content)
|
||||||
|
git(repo, "add", name)
|
||||||
|
git(
|
||||||
|
repo,
|
||||||
|
"-c",
|
||||||
|
"user.name=Fixture",
|
||||||
|
"-c",
|
||||||
|
"user.email=fixture@example.invalid",
|
||||||
|
"commit",
|
||||||
|
"-qm",
|
||||||
|
"bounded result",
|
||||||
|
)
|
||||||
|
return git(repo, "rev-parse", "HEAD")
|
||||||
|
|
||||||
|
|
||||||
|
def fixture(tmp_path):
|
||||||
|
source = tmp_path / "source"
|
||||||
|
source.mkdir()
|
||||||
|
git(source, "init", "-q")
|
||||||
|
commit(source, "README.md", "baseline\n")
|
||||||
|
sandbox = tmp_path / "sandbox"
|
||||||
|
git(tmp_path, "clone", "--quiet", "--no-local", str(source), str(sandbox))
|
||||||
|
grant = RepositoryGrant.from_mapping(
|
||||||
|
{
|
||||||
|
"version": "1",
|
||||||
|
"allowed_paths": ["result.txt"],
|
||||||
|
"commit_count": {"min": 1, "max": 1},
|
||||||
|
"publish": False,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return source, sandbox, grant
|
||||||
|
|
||||||
|
|
||||||
|
class Transport:
|
||||||
|
"""Local transport double only; exporter and Git run unmodified."""
|
||||||
|
|
||||||
|
kind = "local_namespace"
|
||||||
|
|
||||||
|
def __init__(self, workspace):
|
||||||
|
self.workspace = str(workspace)
|
||||||
|
|
||||||
|
def run(self, argv, *, timeout):
|
||||||
|
return subprocess.run(
|
||||||
|
argv, cwd=self.workspace, capture_output=True, text=True, timeout=timeout
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def capture(transfer, sandbox, head):
|
||||||
|
with patch(
|
||||||
|
"glas_harness.transport.transport_from_sandbox", return_value=Transport(sandbox)
|
||||||
|
):
|
||||||
|
transfer.capture(object(), SimpleNamespace(commit_sha=head))
|
||||||
|
|
||||||
|
|
||||||
|
def test_commit_survives_sandbox_teardown_and_import_preserves_identity(tmp_path):
|
||||||
|
source, sandbox, grant = fixture(tmp_path)
|
||||||
|
with RepositoryTransaction(source, state_dir=tmp_path / "state") as tx:
|
||||||
|
baseline = tx.baseline.head
|
||||||
|
transfer = RepositoryArtifactTransfer(tx, grant)
|
||||||
|
head = commit(sandbox)
|
||||||
|
capture(transfer, sandbox, head)
|
||||||
|
assert git(source, "rev-parse", "HEAD") == baseline
|
||||||
|
assert not (source / "result.txt").exists()
|
||||||
|
shutil.rmtree(sandbox)
|
||||||
|
result = transfer.import_after_teardown(head)
|
||||||
|
assert result["head"] == head and result["imported"]
|
||||||
|
assert git(source, "rev-parse", "HEAD") == head
|
||||||
|
assert (source / "result.txt").read_text() == "accepted change\n"
|
||||||
|
assert not git(source, "status", "--porcelain")
|
||||||
|
assert tx.acceptance.changed_paths == ("result.txt",)
|
||||||
|
assert "bundle" not in result
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"fault",
|
||||||
|
[
|
||||||
|
"ungranted",
|
||||||
|
"two_commits",
|
||||||
|
"wrong_reported_head",
|
||||||
|
"dirty",
|
||||||
|
"symlink",
|
||||||
|
"expanded_size",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_invalid_artifact_cannot_change_source(tmp_path, fault):
|
||||||
|
source, sandbox, grant = fixture(tmp_path)
|
||||||
|
with RepositoryTransaction(source, state_dir=tmp_path / "state") as tx:
|
||||||
|
baseline = tx.baseline.head
|
||||||
|
transfer = RepositoryArtifactTransfer(tx, grant)
|
||||||
|
head = commit(
|
||||||
|
sandbox,
|
||||||
|
"outside.txt" if fault == "ungranted" else "result.txt",
|
||||||
|
"x" * (3 * 1024 * 1024) if fault == "expanded_size" else "result\n",
|
||||||
|
)
|
||||||
|
if fault == "two_commits":
|
||||||
|
head = commit(sandbox, content="again\n")
|
||||||
|
if fault == "dirty":
|
||||||
|
(sandbox / "untracked.txt").write_text("dirty")
|
||||||
|
if fault == "symlink":
|
||||||
|
git(sandbox, "rm", "result.txt")
|
||||||
|
os.symlink("/tmp/outside", sandbox / "result.txt")
|
||||||
|
git(sandbox, "add", "result.txt")
|
||||||
|
git(
|
||||||
|
sandbox,
|
||||||
|
"-c",
|
||||||
|
"user.name=Fixture",
|
||||||
|
"-c",
|
||||||
|
"user.email=fixture@example.invalid",
|
||||||
|
"commit",
|
||||||
|
"--amend",
|
||||||
|
"--no-edit",
|
||||||
|
"-q",
|
||||||
|
)
|
||||||
|
head = git(sandbox, "rev-parse", "HEAD")
|
||||||
|
with pytest.raises((RepositoryArtifactError, RepositoryAcceptanceError)):
|
||||||
|
capture(transfer, sandbox, head)
|
||||||
|
transfer.import_after_teardown(
|
||||||
|
"f" * 40 if fault == "wrong_reported_head" else head
|
||||||
|
)
|
||||||
|
assert git(source, "rev-parse", "HEAD") == baseline
|
||||||
|
assert not (source / "result.txt").exists()
|
||||||
|
assert not (source / "outside.txt").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_lease_loss_after_capture_refuses_import(tmp_path):
|
||||||
|
source, sandbox, grant = fixture(tmp_path)
|
||||||
|
cancel = ExecutionCancel()
|
||||||
|
with RepositoryTransaction(source, state_dir=tmp_path / "state") as tx:
|
||||||
|
transfer = RepositoryArtifactTransfer(tx, grant, cancel=cancel)
|
||||||
|
head = commit(sandbox)
|
||||||
|
capture(transfer, sandbox, head)
|
||||||
|
cancel.cancel("lease-loss")
|
||||||
|
with pytest.raises(ExecutionCancelled):
|
||||||
|
transfer.import_after_teardown(head)
|
||||||
|
assert git(source, "rev-parse", "HEAD") == tx.baseline.head
|
||||||
|
|
||||||
|
|
||||||
|
def test_changed_source_baseline_refuses_import(tmp_path):
|
||||||
|
source, sandbox, grant = fixture(tmp_path)
|
||||||
|
with RepositoryTransaction(source, state_dir=tmp_path / "state") as tx:
|
||||||
|
transfer = RepositoryArtifactTransfer(tx, grant)
|
||||||
|
head = commit(sandbox)
|
||||||
|
capture(transfer, sandbox, head)
|
||||||
|
human_head = commit(source, "human.txt", "new human work\n")
|
||||||
|
with pytest.raises(RepositoryAcceptanceError, match="baseline-changed"):
|
||||||
|
transfer.import_after_teardown(head)
|
||||||
|
assert git(source, "rev-parse", "HEAD") == human_head
|
||||||
|
assert (source / "human.txt").read_text() == "new human work\n"
|
||||||
|
assert not (source / "result.txt").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_gateway_failure_after_capture_does_not_import(tmp_path):
|
||||||
|
from rein_aharness.glas_execution import execute_profiled_run
|
||||||
|
from rein_aharness.ops_run_client import OpsRun, OpsRunConfig
|
||||||
|
|
||||||
|
source, sandbox, grant = fixture(tmp_path)
|
||||||
|
run = OpsRun(
|
||||||
|
id="run",
|
||||||
|
activity_definition_id="def",
|
||||||
|
idempotency_key="key",
|
||||||
|
target_repo=str(source),
|
||||||
|
title="fixture",
|
||||||
|
description="fixture",
|
||||||
|
state="claimed",
|
||||||
|
harness_profile_ref="harness.agent-dev-local@1.0.0",
|
||||||
|
repository_grant=grant,
|
||||||
|
)
|
||||||
|
with RepositoryTransaction(source, state_dir=tmp_path / "state") as tx:
|
||||||
|
head = commit(sandbox)
|
||||||
|
|
||||||
|
def gateway(request, *, artifact_capture):
|
||||||
|
with patch(
|
||||||
|
"glas_harness.transport.transport_from_sandbox",
|
||||||
|
return_value=Transport(sandbox),
|
||||||
|
):
|
||||||
|
artifact_capture(object(), SimpleNamespace(commit_sha=head))
|
||||||
|
return {
|
||||||
|
"ok": False,
|
||||||
|
"evidence": {
|
||||||
|
"commit_sha": head,
|
||||||
|
"outcome": "failed",
|
||||||
|
"sandbox_destroy": "failed",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
result = execute_profiled_run(
|
||||||
|
run,
|
||||||
|
OpsRunConfig(repo_roots=(str(tmp_path),)),
|
||||||
|
report_to_hub=False,
|
||||||
|
transaction=tx,
|
||||||
|
request_factory=lambda **kw: kw,
|
||||||
|
gateway=gateway,
|
||||||
|
)
|
||||||
|
assert not result["ok"]
|
||||||
|
assert git(source, "rev-parse", "HEAD") == tx.baseline.head
|
||||||
|
|
||||||
|
|
||||||
|
def test_success_without_artifact_capture_refuses(tmp_path):
|
||||||
|
from rein_aharness.glas_execution import execute_profiled_run
|
||||||
|
from rein_aharness.ops_run_client import OpsRun, OpsRunConfig
|
||||||
|
|
||||||
|
source, sandbox, grant = fixture(tmp_path)
|
||||||
|
run = OpsRun(
|
||||||
|
id="run",
|
||||||
|
activity_definition_id="def",
|
||||||
|
idempotency_key="key",
|
||||||
|
target_repo=str(source),
|
||||||
|
title="fixture",
|
||||||
|
description="fixture",
|
||||||
|
state="claimed",
|
||||||
|
harness_profile_ref="harness.agent-dev-local@1.0.0",
|
||||||
|
repository_grant=grant,
|
||||||
|
)
|
||||||
|
with RepositoryTransaction(source, state_dir=tmp_path / "state") as tx:
|
||||||
|
with pytest.raises(RepositoryArtifactError, match="captured artifact"):
|
||||||
|
execute_profiled_run(
|
||||||
|
run,
|
||||||
|
OpsRunConfig(repo_roots=(str(tmp_path),)),
|
||||||
|
transaction=tx,
|
||||||
|
request_factory=lambda **kw: kw,
|
||||||
|
gateway=lambda *a, **kw: {
|
||||||
|
"ok": True,
|
||||||
|
"evidence": {
|
||||||
|
"commit_sha": "a" * 40,
|
||||||
|
"session_cleanup": "succeeded",
|
||||||
|
"sandbox_destroy": "succeeded",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
assert git(source, "rev-parse", "HEAD") == tx.baseline.head
|
||||||
|
|
||||||
|
|
||||||
|
def test_missing_repository_ownership_refuses():
|
||||||
|
grant = RepositoryGrant.from_mapping(
|
||||||
|
{
|
||||||
|
"version": "1",
|
||||||
|
"allowed_paths": ["result.txt"],
|
||||||
|
"commit_count": {"min": 1, "max": 1},
|
||||||
|
"publish": False,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
with pytest.raises(RepositoryArtifactError, match="active repository transaction"):
|
||||||
|
RepositoryArtifactTransfer(None, grant)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("corruption", ["truncated", "substituted_head"])
|
||||||
|
def test_corrupted_capture_cannot_change_source(tmp_path, corruption):
|
||||||
|
source, sandbox, grant = fixture(tmp_path)
|
||||||
|
with RepositoryTransaction(source, state_dir=tmp_path / "state") as tx:
|
||||||
|
transfer = RepositoryArtifactTransfer(tx, grant)
|
||||||
|
head = commit(sandbox)
|
||||||
|
capture(transfer, sandbox, head)
|
||||||
|
if corruption == "truncated":
|
||||||
|
transfer._bundle = transfer._bundle[:100]
|
||||||
|
else:
|
||||||
|
transfer._head = "f" * 40
|
||||||
|
with pytest.raises(RepositoryArtifactError):
|
||||||
|
transfer.import_after_teardown(transfer._head)
|
||||||
|
assert git(source, "rev-parse", "HEAD") == tx.baseline.head
|
||||||
|
assert not (source / "result.txt").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_ignored_sandbox_cache_is_not_imported(tmp_path):
|
||||||
|
source, sandbox, grant = fixture(tmp_path)
|
||||||
|
commit(source, ".gitignore", "cache/\n")
|
||||||
|
git(sandbox, "pull", "--ff-only", "--quiet")
|
||||||
|
(sandbox / "cache").mkdir()
|
||||||
|
(sandbox / "cache" / "runtime.txt").write_text("discarded runtime cache")
|
||||||
|
with RepositoryTransaction(source, state_dir=tmp_path / "state") as tx:
|
||||||
|
transfer = RepositoryArtifactTransfer(tx, grant)
|
||||||
|
head = commit(sandbox)
|
||||||
|
capture(transfer, sandbox, head)
|
||||||
|
transfer.import_after_teardown(head)
|
||||||
|
assert not (source / "cache").exists()
|
||||||
|
assert tx.acceptance.changed_paths == ("result.txt",)
|
||||||
171
tests/test_repository_artifact_bwrap.py
Normal file
171
tests/test_repository_artifact_bwrap.py
Normal file
|
|
@ -0,0 +1,171 @@
|
||||||
|
"""Opt-in real sandbox/Glas/worker proof; queue and task authoring are fixtures."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import MagicMock
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from glas_harness.contract import (
|
||||||
|
ExecutionSummary,
|
||||||
|
OperationalReadiness,
|
||||||
|
Rein,
|
||||||
|
ToolResult,
|
||||||
|
)
|
||||||
|
from glas_harness.gateway import run_execution
|
||||||
|
from glas_harness.profiles import ProfileCatalog
|
||||||
|
from glas_harness.transport import transport_from_sandbox
|
||||||
|
from sandboxer.core.manager import SandboxManager
|
||||||
|
from sandboxer.lifecycle.store import SandboxStore
|
||||||
|
from rein_aharness.claim_loop import process_one
|
||||||
|
from rein_aharness.close_outbox import CloseOutbox
|
||||||
|
from rein_aharness.metrics import external_metrics_dir
|
||||||
|
from rein_aharness.ops_run_client import (
|
||||||
|
ActivityCoreOpsClient,
|
||||||
|
OpsRun,
|
||||||
|
OpsRunConfig,
|
||||||
|
OpsRunError,
|
||||||
|
)
|
||||||
|
from test_repository_artifact import commit, git
|
||||||
|
from rein_aharness.repository_grant import RepositoryGrant
|
||||||
|
|
||||||
|
pytestmark = pytest.mark.skipif(
|
||||||
|
os.environ.get("REIN_REAL_BWRAP") != "1", reason="opt-in kernel namespace proof"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class DeterministicRein(Rein):
|
||||||
|
calls = 0
|
||||||
|
workspace = None
|
||||||
|
head = None
|
||||||
|
|
||||||
|
def __init__(self, source, baseline):
|
||||||
|
self.source, self.baseline = source, baseline
|
||||||
|
|
||||||
|
def start_session(self, profile, inputs, sandbox):
|
||||||
|
transport = transport_from_sandbox(sandbox)
|
||||||
|
self.workspace = Path(transport.workspace)
|
||||||
|
return transport
|
||||||
|
|
||||||
|
def dispatch_tool(self, transport, tool_call):
|
||||||
|
self.calls += 1
|
||||||
|
code = """from pathlib import Path
|
||||||
|
import subprocess
|
||||||
|
Path('result.txt').write_text('sandbox-result-only\\n')
|
||||||
|
def git(*args):
|
||||||
|
p=subprocess.run(['git',*args],check=True,capture_output=True,text=True)
|
||||||
|
return p.stdout.strip()
|
||||||
|
git('add','result.txt')
|
||||||
|
git('-c','user.name=Fixture','-c','user.email=fixture@example.invalid','commit','-qm','bounded result')
|
||||||
|
print(git('rev-parse','HEAD'))
|
||||||
|
"""
|
||||||
|
result = transport.run(["python3", "-c", code], timeout=30)
|
||||||
|
assert result.returncode == 0, result.stderr
|
||||||
|
self.head = result.stdout.strip()
|
||||||
|
assert git(self.source, "rev-parse", "HEAD") == self.baseline
|
||||||
|
assert not (self.source / "result.txt").exists()
|
||||||
|
return ToolResult(ok=True, output="sandbox-result-only", tokens_spent=0)
|
||||||
|
|
||||||
|
def end_session(self, session):
|
||||||
|
return ExecutionSummary(
|
||||||
|
committed=True, commit_sha=self.head, outcome="succeeded", tokens_spent=0
|
||||||
|
)
|
||||||
|
|
||||||
|
def cleanup_session(self, session):
|
||||||
|
assert self.workspace.exists()
|
||||||
|
assert git(self.source, "rev-parse", "HEAD") == self.baseline
|
||||||
|
|
||||||
|
|
||||||
|
def test_real_bwrap_worker_import_and_close_replay(tmp_path, monkeypatch):
|
||||||
|
monkeypatch.setenv("XDG_DATA_HOME", str(tmp_path / "data"))
|
||||||
|
monkeypatch.setenv("REIN_AHARNESS_STATE_DIR", str(tmp_path / "state"))
|
||||||
|
monkeypatch.setenv("SANDBOXER_NO_STATE_HUB", "1")
|
||||||
|
source = tmp_path / "source"
|
||||||
|
source.mkdir()
|
||||||
|
git(source, "init", "-q")
|
||||||
|
baseline = commit(source, "README.md", "baseline\n")
|
||||||
|
grant = RepositoryGrant.from_mapping(
|
||||||
|
{
|
||||||
|
"version": "1",
|
||||||
|
"allowed_paths": ["result.txt"],
|
||||||
|
"commit_count": {"min": 1, "max": 1},
|
||||||
|
"publish": False,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
run = OpsRun(
|
||||||
|
id="fixture-run",
|
||||||
|
activity_definition_id="fixture-definition",
|
||||||
|
idempotency_key="fixture-key",
|
||||||
|
target_repo=str(source),
|
||||||
|
title="deterministic fixture",
|
||||||
|
description="no inference",
|
||||||
|
state="claimed",
|
||||||
|
harness_profile_ref="harness.agent-dev-local@1.0.0",
|
||||||
|
repository_grant=grant,
|
||||||
|
claim_owner="fixture-worker",
|
||||||
|
attempt=1,
|
||||||
|
)
|
||||||
|
client = MagicMock(spec=ActivityCoreOpsClient)
|
||||||
|
client.config = OpsRunConfig(
|
||||||
|
worker_id="fixture-worker", lease_seconds=90, repo_roots=(str(tmp_path),)
|
||||||
|
)
|
||||||
|
client.claim.return_value = [run]
|
||||||
|
client.heartbeat.return_value = run
|
||||||
|
client.complete.side_effect = [
|
||||||
|
OpsRunError("response lost", action="complete"),
|
||||||
|
OpsRun(
|
||||||
|
id=run.id,
|
||||||
|
activity_definition_id="fixture-definition",
|
||||||
|
idempotency_key="fixture-key",
|
||||||
|
target_repo=str(source),
|
||||||
|
title=run.title,
|
||||||
|
description=run.description,
|
||||||
|
state="succeeded",
|
||||||
|
close_disposition="reconciled",
|
||||||
|
),
|
||||||
|
]
|
||||||
|
catalog = ProfileCatalog()
|
||||||
|
profile, _ = catalog.resolve(run.harness_profile_ref)
|
||||||
|
catalog.profiles()[(profile.id, profile.version)] = profile.model_copy(
|
||||||
|
update={
|
||||||
|
"operational_readiness": OperationalReadiness(
|
||||||
|
status="ready",
|
||||||
|
reason="isolated test fixture",
|
||||||
|
owner="tests",
|
||||||
|
evidence_ref="test:artifact-bwrap",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
)
|
||||||
|
manager = SandboxManager(store=SandboxStore(tmp_path / "sandboxes.json"))
|
||||||
|
rein = DeterministicRein(source, baseline)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
"glas_harness.gateway.run_execution",
|
||||||
|
lambda request, **kwargs: run_execution(
|
||||||
|
request, catalog=catalog, rein=rein, manager=manager, **kwargs
|
||||||
|
),
|
||||||
|
)
|
||||||
|
outbox = CloseOutbox(state_dir=tmp_path / "state")
|
||||||
|
first = process_one(client, outbox=outbox, report_to_hub=False)
|
||||||
|
assert first.reason == "close evidence remains pending", first
|
||||||
|
assert not rein.workspace.exists()
|
||||||
|
assert git(source, "rev-parse", "HEAD") == rein.head
|
||||||
|
assert (source / "result.txt").read_text() == "sandbox-result-only\n"
|
||||||
|
assert not git(source, "status", "--porcelain")
|
||||||
|
assert outbox.status()["pending"] == 1
|
||||||
|
record = json.loads(
|
||||||
|
(external_metrics_dir(source, "rein-aharness") / "executions.jsonl")
|
||||||
|
.read_text()
|
||||||
|
.strip()
|
||||||
|
)
|
||||||
|
assert record["success"]
|
||||||
|
durable = json.dumps(client.complete.call_args.kwargs["result"])
|
||||||
|
assert rein.head in durable
|
||||||
|
assert "sandbox-result-only" not in durable and '"bundle"' not in durable
|
||||||
|
client.claim.return_value = []
|
||||||
|
second = process_one(client, outbox=outbox, report_to_hub=False)
|
||||||
|
assert second.empty and rein.calls == 1
|
||||||
|
assert client.complete.call_count == 2
|
||||||
|
assert client.heartbeat.call_count >= 1
|
||||||
|
assert git(source, "rev-list", "--count", "HEAD") == "2"
|
||||||
|
assert outbox.status() == {"pending": 0, "delivered": 1, "quarantined": 0}
|
||||||
|
|
@ -9,7 +9,7 @@ owner: codex
|
||||||
topic_slug: rein-aharness
|
topic_slug: rein-aharness
|
||||||
priority: high
|
priority: high
|
||||||
created: "2026-08-23"
|
created: "2026-08-23"
|
||||||
updated: "2026-09-04"
|
updated: "2026-09-09"
|
||||||
related:
|
related:
|
||||||
- REIN-A-0004
|
- REIN-A-0004
|
||||||
- GLAS-IN-0002
|
- GLAS-IN-0002
|
||||||
|
|
@ -623,6 +623,18 @@ deployment, and the next natural claim → active heartbeat → terminal run id
|
||||||
will be returned to Activity Core after this change is published and deployed;
|
will be returned to Activity Core after this change is published and deployed;
|
||||||
no real workload will be delayed or expired to manufacture evidence.
|
no real workload will be delayed or expired to manufacture evidence.
|
||||||
|
|
||||||
|
2026-09-09 implementation return: the worker now captures a bounded Git bundle
|
||||||
|
through sandbox owner execution after rein cleanup, then validates and imports
|
||||||
|
the exact one-commit result under its original baseline, grant and lease after
|
||||||
|
successful teardown. The actual bwrap/Glas/worker test passed with a deterministic
|
||||||
|
authoring fixture and response-lost close replay, without duplicate execution.
|
||||||
|
Native profile USD/turn limits now reach Claude CLI controls and require valid
|
||||||
|
terminal accounting; missing/exhausted accounting refuses success. Daily/total
|
||||||
|
reservation, EUR treatment and live provider semantics remain HFACT-WP-0001-T01.
|
||||||
|
The matching rein/Glas code must be rebuilt and admitted in the protected runtime;
|
||||||
|
this local proof does not close live G1/G2 or authorize a model request. See the
|
||||||
|
2026-09-09 runtime-transfer evidence and the owning runtime documentation.
|
||||||
|
|
||||||
## Re-prove one governed profiled run and close residuals
|
## Re-prove one governed profiled run and close residuals
|
||||||
|
|
||||||
```task
|
```task
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue