Bootstrap rein-openweights: minimal agentic loop over OpenRouter (REIN-OW-WP-0001)
Implements T01-T04: - tools.py: green-commit-only-equivalent tool surface (read/write/edit/ glob/grep + git status/diff/log/add+commit), path-traversal guarded. - openrouter_client.py: own minimal chat-completions client — llm-connect's OpenRouterAdapter takes a single prompt string and never surfaces tool_calls, so it can't drive a multi-turn tool-calling loop without a breaking change to its frozen Core ABC. llm-connect stays an optional dependency (pyproject.toml), not load-bearing. - loop.py: plan -> tool call -> observe -> repeat, budget- and turn-bounded, tool errors reported back to the model instead of crashing the loop. - credentials.py: own OpenBao AppRole/ambient-token acquisition, per glas-harness ADR-002 (Option B) — glas-harness does not broker this. - runner.py/hub.py: commit-verified success criterion + State Hub progress/token reporting, mirroring rein-aharness's model. 26 tests, all mocked at the httpx/subprocess boundary — no real OpenRouter or OpenBao calls made. T05 (Forgejo repo creation) stays open, deferred to the operator. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
ac5407a83e
commit
4ee612140b
21 changed files with 1207 additions and 36 deletions
5
.gitignore
vendored
Normal file
5
.gitignore
vendored
Normal file
|
|
@ -0,0 +1,5 @@
|
|||
.venv/
|
||||
__pycache__/
|
||||
*.pyc
|
||||
*.egg-info/
|
||||
.pytest_cache/
|
||||
30
INTENT.md
30
INTENT.md
|
|
@ -14,11 +14,11 @@ repeat, with a bounded tool policy) without building one from scratch per
|
|||
occasion.
|
||||
|
||||
rein-openweights exists to be that harness: one agentic loop, model
|
||||
supplied via OpenRouter (through `llm-connect`), usable anywhere a
|
||||
rein-aharness-equivalent session is wanted but a frontier-vendor
|
||||
dependency is not — cost-sensitive workloads, offline/degraded-network
|
||||
tolerance, or simply comparing open-weight model capability against
|
||||
frontier baselines on the same task.
|
||||
supplied via OpenRouter, usable anywhere a rein-aharness-equivalent
|
||||
session is wanted but a frontier-vendor dependency is not —
|
||||
cost-sensitive workloads, offline/degraded-network tolerance, or simply
|
||||
comparing open-weight model capability against frontier baselines on
|
||||
the same task.
|
||||
|
||||
## Governing principle
|
||||
|
||||
|
|
@ -32,8 +32,11 @@ owns the open-weight-model-specific agentic loop and tool execution.
|
|||
## What it must never become
|
||||
|
||||
- **Not a model router.** Choosing which OpenRouter model to use for a
|
||||
given task, pricing, and fallback stays in `llm-connect`. This repo
|
||||
consumes `llm-connect`, it does not reimplement provider routing.
|
||||
given task, pricing, and fallback is a caller/config concern (`--model`),
|
||||
not logic this repo owns. `llm-connect` remains available as an
|
||||
optional dependency for future needs (structured-JSON side calls,
|
||||
diagnostics/replay) but is not load-bearing for the base agentic loop
|
||||
— see glas-harness ADR-002 and `src/rein_openweights/openrouter_client.py`.
|
||||
- **Not a second harness framework.** Session semantics, tool policy
|
||||
schema, sandbox consumption, and audit reporting are glas-harness's
|
||||
contract (`GLAS-WP-0001-T01`) — this repo implements it, not forks it.
|
||||
|
|
@ -44,7 +47,12 @@ owns the open-weight-model-specific agentic loop and tool execution.
|
|||
|
||||
## Status
|
||||
|
||||
Charter + local scaffold only. See `workplans/REIN-OW-WP-0001-bootstrap.md`
|
||||
for the first concrete milestone (minimal tool-use loop against one
|
||||
OpenRouter open-weight model, same starter tool surface as
|
||||
rein-aharness's `green-commit-only` profile).
|
||||
Minimal agentic loop implemented and unit-tested (26 tests, all mocked
|
||||
at the network/subprocess boundary): tool surface, OpenRouter client,
|
||||
credential acquisition, commit-verified success criterion, State Hub
|
||||
reporting. See `workplans/REIN-OW-WP-0001-bootstrap.md`. Not yet
|
||||
executed against a real OpenRouter model or a real OpenBao instance —
|
||||
those are human-triggered follow-ups (real API cost, real credentials).
|
||||
No glas-harness adapter (`reins/rein_openweights.py`, mirroring
|
||||
`reins/rein_aharness.py`) yet — that's a `GLAS-WP-0001` follow-up once
|
||||
this CLI is considered stable enough to shell out to.
|
||||
|
|
|
|||
34
README.md
34
README.md
|
|
@ -1,11 +1,37 @@
|
|||
# rein-openweights
|
||||
|
||||
An agentic tool-use harness that drives **current open-weight models via
|
||||
OpenRouter** (through `llm-connect`), as an alternative to the
|
||||
frontier-model-vendor harnesses (Claude Code, Grok CLI, Codex/GPT CLI).
|
||||
OpenRouter**, as an alternative to the frontier-model-vendor harnesses
|
||||
(Claude Code, Grok CLI, Codex/GPT CLI).
|
||||
|
||||
Part of the glas-harness "rein" family — see
|
||||
[glas-harness ADR-001](../glas-harness/docs/adr/ADR-001-rein-harness-family.md)
|
||||
and this repo's [workplans/REIN-OW-WP-0001](workplans/REIN-OW-WP-0001-bootstrap.md).
|
||||
[glas-harness ADR-001](../glas-harness/docs/adr/ADR-001-rein-harness-family.md),
|
||||
[ADR-002](../glas-harness/docs/adr/ADR-002-credential-brokering-and-composable-reins.md)
|
||||
(credential ownership), and this repo's
|
||||
[workplans/REIN-OW-WP-0001](workplans/REIN-OW-WP-0001-bootstrap.md).
|
||||
|
||||
Uses its own minimal OpenRouter client (`src/rein_openweights/openrouter_client.py`),
|
||||
not `llm-connect`'s `OpenRouterAdapter` — that adapter's `execute_prompt(prompt: str,
|
||||
config)` takes a single prompt string and never surfaces `message.tool_calls`, so it
|
||||
can't drive a multi-turn tool-calling loop without a breaking change to llm-connect's
|
||||
frozen Core ABC. `llm-connect` stays an optional dependency (see `pyproject.toml`),
|
||||
not load-bearing for the base loop.
|
||||
|
||||
Not yet pushed to Forgejo — local scaffold only.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
python3 -m venv .venv && source .venv/bin/activate
|
||||
pip install -e .
|
||||
|
||||
# Credential: either export OPENROUTER_API_KEY directly, or set
|
||||
# REIN_OPENWEIGHTS_APPROLE_DIR / REIN_OPENWEIGHTS_OPENROUTER_KV_PATH for
|
||||
# OpenBao-based acquisition (see src/rein_openweights/credentials.py).
|
||||
export OPENROUTER_API_KEY=sk-...
|
||||
|
||||
rein-openweights run --task-file examples/task-hello-sandbox.json --no-hub
|
||||
```
|
||||
|
||||
Tests: `python3 -m pytest tests/ -q` (26 tests, no network/OpenBao calls —
|
||||
everything mocked at the `httpx`/`bao` subprocess boundary).
|
||||
|
|
|
|||
8
examples/task-hello-sandbox.json
Normal file
8
examples/task-hello-sandbox.json
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
{
|
||||
"title": "Sandbox smoke: NOTES.md",
|
||||
"description": "Create a file NOTES.md in the repository root containing a two-sentence description of what this sandbox repository is for (read README.md first). Commit it with message 'rein-openweights smoke: add NOTES.md'.",
|
||||
"target_repo": "~/executor-sandbox",
|
||||
"agent": "rein-openweights",
|
||||
"labels": ["executor", "smoke"],
|
||||
"timeout_seconds": 600
|
||||
}
|
||||
34
pyproject.toml
Normal file
34
pyproject.toml
Normal file
|
|
@ -0,0 +1,34 @@
|
|||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "rein-openweights"
|
||||
version = "0.1.0"
|
||||
description = "Agentic tool-use harness driving current open-weight models via OpenRouter"
|
||||
requires-python = ">=3.11"
|
||||
dependencies = [
|
||||
"httpx>=0.27",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
rein-openweights = "rein_openweights.cli:main"
|
||||
|
||||
[tool.uv.sources]
|
||||
llm-connect = { path = "../llm-connect", editable = true }
|
||||
|
||||
[project.optional-dependencies]
|
||||
# llm-connect stays optional per glas-harness ADR-002: the base agentic
|
||||
# loop uses its own OpenRouter client and budget tracker (see
|
||||
# openrouter_client.py, budget.py) so it never becomes load-bearing.
|
||||
# This extra is only for future opt-in use of llm-connect's diagnostics/
|
||||
# replay/structured-JSON helpers.
|
||||
llm-connect = ["llm-connect"]
|
||||
dev = ["pytest>=8"]
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["src/rein_openweights"]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
pythonpath = ["src"]
|
||||
0
src/rein_openweights/__init__.py
Normal file
0
src/rein_openweights/__init__.py
Normal file
32
src/rein_openweights/budget.py
Normal file
32
src/rein_openweights/budget.py
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
"""Minimal token budget tracker.
|
||||
|
||||
llm-connect ships a near-identical `BudgetTracker`; rein-openweights
|
||||
vendors its own so the base agentic loop never requires llm-connect as
|
||||
a hard dependency (glas-harness ADR-002: llm-connect stays optional,
|
||||
not load-bearing here).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
class BudgetExceededError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
class BudgetTracker:
|
||||
def __init__(self, total: int) -> None:
|
||||
if total <= 0:
|
||||
raise ValueError(f"BudgetTracker total must be positive, got {total}")
|
||||
self.total = total
|
||||
self.spent = 0
|
||||
|
||||
def remaining(self) -> int:
|
||||
return max(0, self.total - self.spent)
|
||||
|
||||
def consume(self, tokens: int) -> None:
|
||||
new_spent = self.spent + tokens
|
||||
if new_spent > self.total:
|
||||
raise BudgetExceededError(
|
||||
f"Token budget exceeded: {new_spent} tokens used, cap is {self.total}"
|
||||
)
|
||||
self.spent = new_spent
|
||||
41
src/rein_openweights/cli.py
Normal file
41
src/rein_openweights/cli.py
Normal file
|
|
@ -0,0 +1,41 @@
|
|||
"""rein-openweights CLI."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
from dataclasses import asdict
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(prog="rein-openweights")
|
||||
sub = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
run = sub.add_parser("run", help="Run one task via the OpenRouter agentic loop")
|
||||
run.add_argument("--task-file", required=True)
|
||||
run.add_argument("--model", default=None)
|
||||
run.add_argument("--max-turns", type=int, default=20)
|
||||
run.add_argument("--budget-tokens", type=int, default=60_000)
|
||||
run.add_argument("--no-hub", action="store_true", help="Skip hub reporting")
|
||||
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
if args.command == "run":
|
||||
from rein_openweights.runner import DEFAULT_MODEL, run_task
|
||||
|
||||
result = run_task(
|
||||
args.task_file,
|
||||
model=args.model or DEFAULT_MODEL,
|
||||
max_turns=args.max_turns,
|
||||
budget_tokens=args.budget_tokens,
|
||||
report_to_hub=not args.no_hub,
|
||||
)
|
||||
print(json.dumps(asdict(result), indent=2))
|
||||
return 0 if result.ok else 1
|
||||
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
75
src/rein_openweights/credentials.py
Normal file
75
src/rein_openweights/credentials.py
Normal file
|
|
@ -0,0 +1,75 @@
|
|||
"""OpenRouter credential acquisition via OpenBao.
|
||||
|
||||
Per glas-harness ADR-002 (Option B): rein-openweights acquires its own
|
||||
credential, consistent with rein-aharness's principle that the harness
|
||||
is the only credential holder. glas-harness never brokers this.
|
||||
|
||||
Mirrors rein-aharness's mailscan.py pattern: AppRole (unattended lane)
|
||||
first, then an ambient bao token (operator lane), then a single kv read.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
_KV_PATH_ENV = "REIN_OPENWEIGHTS_OPENROUTER_KV_PATH"
|
||||
_DEFAULT_KV_PATH = "reins/rein-openweights/openrouter"
|
||||
_KV_FIELD = "api_key"
|
||||
_BAO_TIMEOUT = 30
|
||||
|
||||
|
||||
class CredentialError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
def _bao(*args: str, env: dict[str, str] | None = None) -> str:
|
||||
result = subprocess.run(
|
||||
["bao", *args],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=_BAO_TIMEOUT,
|
||||
env=env or os.environ.copy(),
|
||||
)
|
||||
if result.returncode != 0:
|
||||
raise CredentialError(f"bao {args[0]} failed: {result.stderr.strip()[:200]}")
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
def _acquire_token() -> str | None:
|
||||
approle_dir = os.environ.get("REIN_OPENWEIGHTS_APPROLE_DIR")
|
||||
if not approle_dir:
|
||||
return None
|
||||
role_id_file = Path(approle_dir) / "role_id"
|
||||
secret_id_file = Path(approle_dir) / "secret_id"
|
||||
if not (role_id_file.is_file() and secret_id_file.is_file()):
|
||||
return None
|
||||
return _bao(
|
||||
"write",
|
||||
"-field=token",
|
||||
"auth/approle/login",
|
||||
f"role_id={role_id_file.read_text().strip()}",
|
||||
f"secret_id={secret_id_file.read_text().strip()}",
|
||||
)
|
||||
|
||||
|
||||
def resolve_openrouter_api_key() -> str | None:
|
||||
"""Resolution order: explicit env var, then OpenBao (AppRole or ambient token)."""
|
||||
explicit = os.environ.get("OPENROUTER_API_KEY")
|
||||
if explicit:
|
||||
return explicit
|
||||
|
||||
kv_path = os.environ.get(_KV_PATH_ENV, _DEFAULT_KV_PATH)
|
||||
try:
|
||||
token = _acquire_token()
|
||||
except CredentialError:
|
||||
return None
|
||||
|
||||
env = os.environ.copy()
|
||||
if token:
|
||||
env["BAO_TOKEN"] = token
|
||||
try:
|
||||
return _bao("kv", "get", f"-field={_KV_FIELD}", kv_path, env=env)
|
||||
except CredentialError:
|
||||
return None
|
||||
65
src/rein_openweights/hub.py
Normal file
65
src/rein_openweights/hub.py
Normal file
|
|
@ -0,0 +1,65 @@
|
|||
"""Custodian State Hub reporting (REST, no MCP) — mirrors rein-aharness's hub.py."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
_DEFAULT_URL = "http://127.0.0.1:8000"
|
||||
_TIMEOUT = 10.0
|
||||
|
||||
|
||||
def _base_url() -> str:
|
||||
return os.environ.get("STATE_HUB_URL", _DEFAULT_URL).rstrip("/")
|
||||
|
||||
|
||||
def post_progress_event(
|
||||
summary: str,
|
||||
event_type: str,
|
||||
detail: dict[str, Any],
|
||||
task_id: str | None = None,
|
||||
) -> bool:
|
||||
payload: dict[str, Any] = {
|
||||
"summary": summary,
|
||||
"event_type": event_type,
|
||||
"detail": detail,
|
||||
"author": "agt-rein-openweights",
|
||||
}
|
||||
if task_id:
|
||||
payload["task_id"] = task_id
|
||||
try:
|
||||
resp = httpx.post(f"{_base_url()}/progress/", json=payload, timeout=_TIMEOUT)
|
||||
resp.raise_for_status()
|
||||
return True
|
||||
except httpx.HTTPError:
|
||||
return False
|
||||
|
||||
|
||||
def post_token_event(
|
||||
repo: str,
|
||||
tokens: int,
|
||||
*,
|
||||
budget: int | None = None,
|
||||
ok: bool = True,
|
||||
detail: dict[str, Any] | None = None,
|
||||
) -> bool:
|
||||
payload: dict[str, Any] = {
|
||||
"repo": repo,
|
||||
"tokens": tokens,
|
||||
"source": "rein-openweights",
|
||||
"ok": ok,
|
||||
}
|
||||
if budget is not None:
|
||||
payload["budget"] = budget
|
||||
if detail:
|
||||
payload["detail"] = detail
|
||||
try:
|
||||
resp = httpx.post(f"{_base_url()}/token-events/upsert", json=payload, timeout=_TIMEOUT)
|
||||
if resp.status_code >= 400:
|
||||
resp = httpx.post(f"{_base_url()}/token-events/", json=payload, timeout=_TIMEOUT)
|
||||
resp.raise_for_status()
|
||||
return True
|
||||
except httpx.HTTPError:
|
||||
return False
|
||||
75
src/rein_openweights/loop.py
Normal file
75
src/rein_openweights/loop.py
Normal file
|
|
@ -0,0 +1,75 @@
|
|||
"""Minimal agentic tool-use loop: plan -> tool call -> observe -> repeat."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from rein_openweights.budget import BudgetExceededError, BudgetTracker
|
||||
from rein_openweights.openrouter_client import OpenRouterClient
|
||||
from rein_openweights.tools import call_tool, openai_tool_schemas
|
||||
|
||||
_SYSTEM_PROMPT = (
|
||||
"You are a bounded coding agent. You may only act through the provided "
|
||||
"tools. When the task is complete, call git_add_commit with a "
|
||||
"descriptive message, then reply with a final message containing the "
|
||||
"word DONE and no further tool calls."
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class LoopResult:
|
||||
turns: int
|
||||
finished: bool
|
||||
transcript: list[dict[str, Any]] = field(default_factory=list)
|
||||
error: str | None = None
|
||||
|
||||
|
||||
def run_loop(
|
||||
client: OpenRouterClient,
|
||||
repo_root: Path,
|
||||
title: str,
|
||||
description: str,
|
||||
*,
|
||||
max_turns: int = 20,
|
||||
budget: BudgetTracker | None = None,
|
||||
) -> LoopResult:
|
||||
messages: list[dict[str, Any]] = [
|
||||
{"role": "system", "content": _SYSTEM_PROMPT},
|
||||
{"role": "user", "content": f"# {title}\n\n{description}"},
|
||||
]
|
||||
tools = openai_tool_schemas()
|
||||
|
||||
for turn in range(1, max_turns + 1):
|
||||
data = client.chat(messages, tools=tools)
|
||||
|
||||
if budget is not None:
|
||||
usage = data.get("usage", {})
|
||||
try:
|
||||
budget.consume(usage.get("total_tokens", 0))
|
||||
except BudgetExceededError as exc:
|
||||
return LoopResult(turns=turn, finished=False, transcript=messages, error=str(exc))
|
||||
|
||||
message = data["choices"][0]["message"]
|
||||
messages.append(message)
|
||||
|
||||
tool_calls = message.get("tool_calls") or []
|
||||
if not tool_calls:
|
||||
finished = "DONE" in (message.get("content") or "")
|
||||
return LoopResult(turns=turn, finished=finished, transcript=messages)
|
||||
|
||||
for call in tool_calls:
|
||||
fn = call["function"]
|
||||
name = fn["name"]
|
||||
try:
|
||||
args = json.loads(fn.get("arguments") or "{}")
|
||||
result = call_tool(repo_root, name, args)
|
||||
except Exception as exc: # noqa: BLE001 - report failure to the model, keep looping
|
||||
result = f"error: {exc}"
|
||||
messages.append(
|
||||
{"role": "tool", "tool_call_id": call.get("id", name), "content": result}
|
||||
)
|
||||
|
||||
return LoopResult(turns=max_turns, finished=False, transcript=messages, error="max_turns exceeded")
|
||||
64
src/rein_openweights/openrouter_client.py
Normal file
64
src/rein_openweights/openrouter_client.py
Normal file
|
|
@ -0,0 +1,64 @@
|
|||
"""Minimal, independent OpenRouter chat-completions client.
|
||||
|
||||
Deliberately does not depend on llm-connect's `OpenRouterAdapter`: its
|
||||
`execute_prompt(prompt: str, config)` takes a single prompt string,
|
||||
builds a fixed [system, user] message pair itself, and never surfaces
|
||||
`message.tool_calls` in its response — it cannot drive a multi-turn
|
||||
tool-calling loop without a breaking change to llm-connect's frozen
|
||||
Core `LLMAdapter` ABC (`execute_prompt` signature). Rather than force
|
||||
that change onto every llm-connect consumer, rein-openweights owns this
|
||||
thin transport directly. See glas-harness ADR-002.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
_DEFAULT_API_BASE = "https://openrouter.ai/api/v1"
|
||||
|
||||
|
||||
class OpenRouterError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
class OpenRouterClient:
|
||||
def __init__(
|
||||
self,
|
||||
api_key: str,
|
||||
model: str,
|
||||
api_base: str = _DEFAULT_API_BASE,
|
||||
timeout: float = 120.0,
|
||||
) -> None:
|
||||
self.api_key = api_key
|
||||
self.model = model
|
||||
self.api_base = api_base.rstrip("/")
|
||||
self.timeout = timeout
|
||||
|
||||
def chat(
|
||||
self,
|
||||
messages: list[dict[str, Any]],
|
||||
tools: list[dict[str, Any]] | None = None,
|
||||
temperature: float = 0.2,
|
||||
max_tokens: int = 4096,
|
||||
) -> dict[str, Any]:
|
||||
payload: dict[str, Any] = {
|
||||
"model": self.model,
|
||||
"messages": messages,
|
||||
"temperature": temperature,
|
||||
"max_tokens": max_tokens,
|
||||
}
|
||||
if tools:
|
||||
payload["tools"] = tools
|
||||
payload["tool_choice"] = "auto"
|
||||
|
||||
response = httpx.post(
|
||||
f"{self.api_base}/chat/completions",
|
||||
json=payload,
|
||||
headers={"Authorization": f"Bearer {self.api_key}"},
|
||||
timeout=self.timeout,
|
||||
)
|
||||
if response.status_code >= 400:
|
||||
raise OpenRouterError(f"OpenRouter {response.status_code}: {response.text[:300]}")
|
||||
return response.json()
|
||||
93
src/rein_openweights/runner.py
Normal file
93
src/rein_openweights/runner.py
Normal file
|
|
@ -0,0 +1,93 @@
|
|||
"""rein-openweights run — bounded agentic session, commit-verified.
|
||||
|
||||
Success criterion mirrors rein-aharness: a local git commit landing is
|
||||
the sole success signal. Reports a State Hub progress + token event on
|
||||
completion unless --no-hub is set.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from dataclasses import asdict, dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from rein_openweights import hub
|
||||
from rein_openweights.budget import BudgetTracker
|
||||
from rein_openweights.credentials import resolve_openrouter_api_key
|
||||
from rein_openweights.loop import run_loop
|
||||
from rein_openweights.openrouter_client import OpenRouterClient
|
||||
from rein_openweights.taskspec import TaskSpec
|
||||
|
||||
DEFAULT_MODEL = "meta-llama/llama-3.1-70b-instruct"
|
||||
|
||||
|
||||
@dataclass
|
||||
class RunResult:
|
||||
ok: bool
|
||||
committed: bool
|
||||
commit_sha: str
|
||||
turns: int
|
||||
tokens_spent: int
|
||||
reason: str = ""
|
||||
|
||||
|
||||
def _git_head(repo: Path) -> str:
|
||||
proc = subprocess.run(
|
||||
["git", "-C", str(repo), "rev-parse", "HEAD"], capture_output=True, text=True
|
||||
)
|
||||
return proc.stdout.strip() if proc.returncode == 0 else ""
|
||||
|
||||
|
||||
def run_task(
|
||||
task_file: str,
|
||||
*,
|
||||
model: str = DEFAULT_MODEL,
|
||||
max_turns: int = 20,
|
||||
budget_tokens: int = 60_000,
|
||||
api_key: str | None = None,
|
||||
report_to_hub: bool = True,
|
||||
) -> RunResult:
|
||||
task = TaskSpec.load(task_file)
|
||||
repo = Path(task.target_repo).expanduser().resolve()
|
||||
head_before = _git_head(repo)
|
||||
|
||||
key = api_key or resolve_openrouter_api_key()
|
||||
if not key:
|
||||
result = RunResult(
|
||||
ok=False, committed=False, commit_sha="", turns=0, tokens_spent=0,
|
||||
reason="no OpenRouter credential resolved",
|
||||
)
|
||||
_report(result, task, report_to_hub)
|
||||
return result
|
||||
|
||||
client = OpenRouterClient(api_key=key, model=model)
|
||||
budget = BudgetTracker(total=budget_tokens)
|
||||
|
||||
loop_result = run_loop(
|
||||
client, repo, task.title, task.description, max_turns=max_turns, budget=budget
|
||||
)
|
||||
|
||||
head_after = _git_head(repo)
|
||||
committed = bool(head_after) and head_after != head_before
|
||||
|
||||
result = RunResult(
|
||||
ok=committed,
|
||||
committed=committed,
|
||||
commit_sha=head_after,
|
||||
turns=loop_result.turns,
|
||||
tokens_spent=budget.spent,
|
||||
reason=loop_result.error or ("" if committed else "no commit produced"),
|
||||
)
|
||||
_report(result, task, report_to_hub)
|
||||
return result
|
||||
|
||||
|
||||
def _report(result: RunResult, task: TaskSpec, report_to_hub: bool) -> None:
|
||||
if not report_to_hub:
|
||||
return
|
||||
hub.post_progress_event(
|
||||
summary=f"rein-openweights run {'ok' if result.ok else 'failed'}: {task.title}",
|
||||
event_type="executor_run",
|
||||
detail=asdict(result),
|
||||
)
|
||||
hub.post_token_event(repo=task.target_repo, tokens=result.tokens_spent, ok=result.ok)
|
||||
29
src/rein_openweights/taskspec.py
Normal file
29
src/rein_openweights/taskspec.py
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
"""TaskSpec — local JSON task definition, mirrors rein-aharness's taskspec.py."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
@dataclass
|
||||
class TaskSpec:
|
||||
title: str
|
||||
description: str
|
||||
target_repo: str
|
||||
agent: str = "rein-openweights"
|
||||
labels: list[str] = field(default_factory=list)
|
||||
timeout_seconds: int = 600
|
||||
|
||||
@classmethod
|
||||
def load(cls, path: str) -> TaskSpec:
|
||||
data = json.loads(Path(path).expanduser().read_text())
|
||||
return cls(
|
||||
title=data["title"],
|
||||
description=data["description"],
|
||||
target_repo=data["target_repo"],
|
||||
agent=data.get("agent", "rein-openweights"),
|
||||
labels=data.get("labels", []),
|
||||
timeout_seconds=data.get("timeout_seconds", 600),
|
||||
)
|
||||
187
src/rein_openweights/tools.py
Normal file
187
src/rein_openweights/tools.py
Normal file
|
|
@ -0,0 +1,187 @@
|
|||
"""Tool surface — mirrors rein-aharness's `green-commit-only` allow-list.
|
||||
|
||||
Read/Write/Edit/Glob/Grep + local git status/diff/log/add+commit. No
|
||||
push, no network, no shell escape hatch — the same bounded surface, just
|
||||
invoked by a different rein.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any, Callable
|
||||
|
||||
|
||||
class ToolPolicyError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
def _resolve(repo_root: Path, rel_path: str) -> Path:
|
||||
repo_root = repo_root.resolve()
|
||||
candidate = (repo_root / rel_path).resolve()
|
||||
if candidate != repo_root and repo_root not in candidate.parents:
|
||||
raise ToolPolicyError(f"path escapes repo root: {rel_path}")
|
||||
return candidate
|
||||
|
||||
|
||||
def read_file(repo_root: Path, path: str) -> str:
|
||||
return _resolve(repo_root, path).read_text()
|
||||
|
||||
|
||||
def write_file(repo_root: Path, path: str, content: str) -> str:
|
||||
target = _resolve(repo_root, path)
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_text(content)
|
||||
return f"wrote {len(content)} bytes to {path}"
|
||||
|
||||
|
||||
def edit_file(repo_root: Path, path: str, old: str, new: str) -> str:
|
||||
target = _resolve(repo_root, path)
|
||||
text = target.read_text()
|
||||
if old not in text:
|
||||
raise ToolPolicyError(f"old string not found in {path}")
|
||||
target.write_text(text.replace(old, new, 1))
|
||||
return f"replaced 1 occurrence in {path}"
|
||||
|
||||
|
||||
def glob_files(repo_root: Path, pattern: str) -> str:
|
||||
repo_root = repo_root.resolve()
|
||||
matches = sorted(str(p.relative_to(repo_root)) for p in repo_root.rglob(pattern) if p.is_file())
|
||||
return "\n".join(matches) or "(no matches)"
|
||||
|
||||
|
||||
def grep_files(repo_root: Path, pattern: str, path: str = ".") -> str:
|
||||
base = _resolve(repo_root, path)
|
||||
hits: list[str] = []
|
||||
for file in sorted(base.rglob("*")):
|
||||
if not file.is_file() or ".git" in file.parts:
|
||||
continue
|
||||
try:
|
||||
text = file.read_text()
|
||||
except (UnicodeDecodeError, OSError):
|
||||
continue
|
||||
for lineno, line in enumerate(text.splitlines(), start=1):
|
||||
if pattern in line:
|
||||
hits.append(f"{file.relative_to(repo_root)}:{lineno}:{line}")
|
||||
return "\n".join(hits) or "(no matches)"
|
||||
|
||||
|
||||
def _git(repo_root: Path, *args: str) -> str:
|
||||
proc = subprocess.run(["git", "-C", str(repo_root), *args], capture_output=True, text=True)
|
||||
if proc.returncode != 0:
|
||||
raise ToolPolicyError(f"git {' '.join(args)} failed: {proc.stderr.strip()}")
|
||||
return proc.stdout
|
||||
|
||||
|
||||
def git_status(repo_root: Path) -> str:
|
||||
return _git(repo_root, "status", "--short") or "(clean)"
|
||||
|
||||
|
||||
def git_diff(repo_root: Path) -> str:
|
||||
return _git(repo_root, "diff") or "(no diff)"
|
||||
|
||||
|
||||
def git_log(repo_root: Path, max_count: int = 10) -> str:
|
||||
return _git(repo_root, "log", f"-{max_count}", "--oneline")
|
||||
|
||||
|
||||
def git_add_commit(repo_root: Path, message: str) -> str:
|
||||
_git(repo_root, "add", "-A")
|
||||
return _git(repo_root, "commit", "-m", message)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Tool:
|
||||
name: str
|
||||
description: str
|
||||
parameters: dict[str, Any]
|
||||
fn: Callable[..., str]
|
||||
|
||||
|
||||
TOOLS: list[Tool] = [
|
||||
Tool(
|
||||
"read_file",
|
||||
"Read a file's contents.",
|
||||
{"type": "object", "properties": {"path": {"type": "string"}}, "required": ["path"]},
|
||||
read_file,
|
||||
),
|
||||
Tool(
|
||||
"write_file",
|
||||
"Write (overwrite) a file's contents, creating parent dirs as needed.",
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {"path": {"type": "string"}, "content": {"type": "string"}},
|
||||
"required": ["path", "content"],
|
||||
},
|
||||
write_file,
|
||||
),
|
||||
Tool(
|
||||
"edit_file",
|
||||
"Replace the first occurrence of `old` with `new` in a file.",
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"path": {"type": "string"},
|
||||
"old": {"type": "string"},
|
||||
"new": {"type": "string"},
|
||||
},
|
||||
"required": ["path", "old", "new"],
|
||||
},
|
||||
edit_file,
|
||||
),
|
||||
Tool(
|
||||
"glob_files",
|
||||
"List files under the repo matching a glob pattern (e.g. '**/*.py').",
|
||||
{"type": "object", "properties": {"pattern": {"type": "string"}}, "required": ["pattern"]},
|
||||
glob_files,
|
||||
),
|
||||
Tool(
|
||||
"grep_files",
|
||||
"Search file contents under `path` for a literal substring.",
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {"pattern": {"type": "string"}, "path": {"type": "string"}},
|
||||
"required": ["pattern"],
|
||||
},
|
||||
grep_files,
|
||||
),
|
||||
Tool("git_status", "Show `git status --short`.", {"type": "object", "properties": {}}, git_status),
|
||||
Tool("git_diff", "Show `git diff`.", {"type": "object", "properties": {}}, git_diff),
|
||||
Tool(
|
||||
"git_log",
|
||||
"Show recent commit log (oneline).",
|
||||
{"type": "object", "properties": {"max_count": {"type": "integer"}}},
|
||||
git_log,
|
||||
),
|
||||
Tool(
|
||||
"git_add_commit",
|
||||
"Stage all changes (`git add -A`) and commit with a message. "
|
||||
"Call this once the task is complete.",
|
||||
{"type": "object", "properties": {"message": {"type": "string"}}, "required": ["message"]},
|
||||
git_add_commit,
|
||||
),
|
||||
]
|
||||
|
||||
TOOLS_BY_NAME = {t.name: t for t in TOOLS}
|
||||
|
||||
|
||||
def openai_tool_schemas() -> list[dict[str, Any]]:
|
||||
return [
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": t.name,
|
||||
"description": t.description,
|
||||
"parameters": t.parameters,
|
||||
},
|
||||
}
|
||||
for t in TOOLS
|
||||
]
|
||||
|
||||
|
||||
def call_tool(repo_root: Path, name: str, args: dict[str, Any]) -> str:
|
||||
tool = TOOLS_BY_NAME.get(name)
|
||||
if tool is None:
|
||||
raise ToolPolicyError(f"unknown tool: {name}")
|
||||
return tool.fn(repo_root, **args)
|
||||
54
tests/test_credentials.py
Normal file
54
tests/test_credentials.py
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
from unittest.mock import patch
|
||||
|
||||
from rein_openweights.credentials import resolve_openrouter_api_key
|
||||
|
||||
|
||||
def test_explicit_env_var_short_circuits(monkeypatch):
|
||||
monkeypatch.setenv("OPENROUTER_API_KEY", "sk-explicit")
|
||||
assert resolve_openrouter_api_key() == "sk-explicit"
|
||||
|
||||
|
||||
def test_falls_back_to_bao_kv_when_no_approle(monkeypatch):
|
||||
monkeypatch.delenv("OPENROUTER_API_KEY", raising=False)
|
||||
monkeypatch.delenv("REIN_OPENWEIGHTS_APPROLE_DIR", raising=False)
|
||||
|
||||
with patch("rein_openweights.credentials._bao", return_value="sk-from-vault") as bao:
|
||||
key = resolve_openrouter_api_key()
|
||||
|
||||
assert key == "sk-from-vault"
|
||||
bao.assert_called_once()
|
||||
assert bao.call_args.args[0] == "kv"
|
||||
|
||||
|
||||
def test_approle_lane_exchanges_token_before_kv_read(monkeypatch, tmp_path):
|
||||
monkeypatch.delenv("OPENROUTER_API_KEY", raising=False)
|
||||
role_id = tmp_path / "role_id"
|
||||
secret_id = tmp_path / "secret_id"
|
||||
role_id.write_text("role-123")
|
||||
secret_id.write_text("secret-456")
|
||||
monkeypatch.setenv("REIN_OPENWEIGHTS_APPROLE_DIR", str(tmp_path))
|
||||
|
||||
calls = []
|
||||
|
||||
def fake_bao(*args, env=None):
|
||||
calls.append(args)
|
||||
if args[0] == "write":
|
||||
return "vault-token"
|
||||
return "sk-from-vault"
|
||||
|
||||
with patch("rein_openweights.credentials._bao", side_effect=fake_bao):
|
||||
key = resolve_openrouter_api_key()
|
||||
|
||||
assert key == "sk-from-vault"
|
||||
assert calls[0][0] == "write"
|
||||
assert calls[1][0] == "kv"
|
||||
|
||||
|
||||
def test_returns_none_when_bao_fails(monkeypatch):
|
||||
monkeypatch.delenv("OPENROUTER_API_KEY", raising=False)
|
||||
monkeypatch.delenv("REIN_OPENWEIGHTS_APPROLE_DIR", raising=False)
|
||||
|
||||
from rein_openweights.credentials import CredentialError
|
||||
|
||||
with patch("rein_openweights.credentials._bao", side_effect=CredentialError("boom")):
|
||||
assert resolve_openrouter_api_key() is None
|
||||
118
tests/test_loop.py
Normal file
118
tests/test_loop.py
Normal file
|
|
@ -0,0 +1,118 @@
|
|||
import subprocess
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
from rein_openweights.budget import BudgetTracker
|
||||
from rein_openweights.loop import run_loop
|
||||
|
||||
|
||||
def _init_repo(tmp_path):
|
||||
subprocess.run(["git", "init", "-q"], cwd=tmp_path, check=True)
|
||||
subprocess.run(["git", "config", "user.email", "t@example.com"], cwd=tmp_path, check=True)
|
||||
subprocess.run(["git", "config", "user.name", "t"], cwd=tmp_path, check=True)
|
||||
subprocess.run(["git", "commit", "-q", "--allow-empty", "-m", "init"], cwd=tmp_path, check=True)
|
||||
return tmp_path
|
||||
|
||||
|
||||
def _response(message, total_tokens=100):
|
||||
return {
|
||||
"choices": [{"message": message}],
|
||||
"usage": {"total_tokens": total_tokens},
|
||||
}
|
||||
|
||||
|
||||
def test_run_loop_executes_tool_call_then_finishes(tmp_path):
|
||||
repo = _init_repo(tmp_path)
|
||||
client = MagicMock()
|
||||
client.chat.side_effect = [
|
||||
_response(
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": None,
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "call_1",
|
||||
"function": {
|
||||
"name": "write_file",
|
||||
"arguments": '{"path": "NOTES.md", "content": "hi"}',
|
||||
},
|
||||
}
|
||||
],
|
||||
}
|
||||
),
|
||||
_response(
|
||||
{
|
||||
"role": "assistant",
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "call_2",
|
||||
"function": {
|
||||
"name": "git_add_commit",
|
||||
"arguments": '{"message": "add notes"}',
|
||||
},
|
||||
}
|
||||
],
|
||||
}
|
||||
),
|
||||
_response({"role": "assistant", "content": "All set. DONE"}),
|
||||
]
|
||||
|
||||
result = run_loop(client, repo, "t", "d", budget=BudgetTracker(total=10_000))
|
||||
|
||||
assert result.finished is True
|
||||
assert result.turns == 3
|
||||
assert (repo / "NOTES.md").read_text() == "hi"
|
||||
log = subprocess.run(
|
||||
["git", "-C", str(repo), "log", "--oneline"], capture_output=True, text=True
|
||||
).stdout
|
||||
assert "add notes" in log
|
||||
|
||||
|
||||
def test_run_loop_stops_when_budget_exceeded(tmp_path):
|
||||
repo = _init_repo(tmp_path)
|
||||
client = MagicMock()
|
||||
client.chat.return_value = _response({"role": "assistant", "content": "still going"}, total_tokens=100)
|
||||
|
||||
result = run_loop(client, repo, "t", "d", budget=BudgetTracker(total=50))
|
||||
|
||||
assert result.error is not None
|
||||
assert "budget" in result.error.lower()
|
||||
|
||||
|
||||
def test_run_loop_stops_at_max_turns(tmp_path):
|
||||
repo = _init_repo(tmp_path)
|
||||
client = MagicMock()
|
||||
client.chat.return_value = _response(
|
||||
{
|
||||
"role": "assistant",
|
||||
"tool_calls": [
|
||||
{"id": "c", "function": {"name": "git_status", "arguments": "{}"}}
|
||||
],
|
||||
}
|
||||
)
|
||||
|
||||
result = run_loop(client, repo, "t", "d", max_turns=2)
|
||||
|
||||
assert result.turns == 2
|
||||
assert result.error == "max_turns exceeded"
|
||||
|
||||
|
||||
def test_run_loop_reports_tool_error_without_crashing(tmp_path):
|
||||
repo = _init_repo(tmp_path)
|
||||
client = MagicMock()
|
||||
client.chat.side_effect = [
|
||||
_response(
|
||||
{
|
||||
"role": "assistant",
|
||||
"tool_calls": [
|
||||
{"id": "c", "function": {"name": "read_file", "arguments": '{"path": "missing.txt"}'}}
|
||||
],
|
||||
}
|
||||
),
|
||||
_response({"role": "assistant", "content": "done anyway DONE"}),
|
||||
]
|
||||
|
||||
result = run_loop(client, repo, "t", "d")
|
||||
|
||||
assert result.finished is True
|
||||
tool_messages = [m for m in result.transcript if m.get("role") == "tool"]
|
||||
assert "error" in tool_messages[0]["content"]
|
||||
43
tests/test_openrouter_client.py
Normal file
43
tests/test_openrouter_client.py
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from rein_openweights.openrouter_client import OpenRouterClient, OpenRouterError
|
||||
|
||||
|
||||
def test_chat_sends_bearer_auth_and_model():
|
||||
client = OpenRouterClient(api_key="sk-test", model="meta-llama/llama-3.1-70b-instruct")
|
||||
fake_response = MagicMock(status_code=200)
|
||||
fake_response.json.return_value = {"choices": [{"message": {"content": "hi"}}]}
|
||||
|
||||
with patch("rein_openweights.openrouter_client.httpx.post", return_value=fake_response) as post:
|
||||
data = client.chat([{"role": "user", "content": "hello"}])
|
||||
|
||||
assert data["choices"][0]["message"]["content"] == "hi"
|
||||
_, kwargs = post.call_args
|
||||
assert kwargs["headers"]["Authorization"] == "Bearer sk-test"
|
||||
assert kwargs["json"]["model"] == "meta-llama/llama-3.1-70b-instruct"
|
||||
assert "tools" not in kwargs["json"]
|
||||
|
||||
|
||||
def test_chat_includes_tools_when_provided():
|
||||
client = OpenRouterClient(api_key="sk-test", model="m")
|
||||
fake_response = MagicMock(status_code=200)
|
||||
fake_response.json.return_value = {"choices": [{"message": {}}]}
|
||||
tools = [{"type": "function", "function": {"name": "read_file"}}]
|
||||
|
||||
with patch("rein_openweights.openrouter_client.httpx.post", return_value=fake_response) as post:
|
||||
client.chat([{"role": "user", "content": "x"}], tools=tools)
|
||||
|
||||
_, kwargs = post.call_args
|
||||
assert kwargs["json"]["tools"] == tools
|
||||
assert kwargs["json"]["tool_choice"] == "auto"
|
||||
|
||||
|
||||
def test_chat_raises_on_http_error():
|
||||
client = OpenRouterClient(api_key="sk-test", model="m")
|
||||
fake_response = MagicMock(status_code=401, text="unauthorized")
|
||||
|
||||
with patch("rein_openweights.openrouter_client.httpx.post", return_value=fake_response):
|
||||
with pytest.raises(OpenRouterError, match="401"):
|
||||
client.chat([{"role": "user", "content": "x"}])
|
||||
75
tests/test_runner.py
Normal file
75
tests/test_runner.py
Normal file
|
|
@ -0,0 +1,75 @@
|
|||
import json
|
||||
import subprocess
|
||||
from unittest.mock import patch
|
||||
|
||||
from rein_openweights.loop import LoopResult
|
||||
from rein_openweights.runner import run_task
|
||||
|
||||
|
||||
def _init_repo(tmp_path):
|
||||
subprocess.run(["git", "init", "-q"], cwd=tmp_path, check=True)
|
||||
subprocess.run(["git", "config", "user.email", "t@example.com"], cwd=tmp_path, check=True)
|
||||
subprocess.run(["git", "config", "user.name", "t"], cwd=tmp_path, check=True)
|
||||
subprocess.run(["git", "commit", "-q", "--allow-empty", "-m", "init"], cwd=tmp_path, check=True)
|
||||
return tmp_path
|
||||
|
||||
|
||||
def _task_file(tmp_path, repo):
|
||||
task = tmp_path / "task.json"
|
||||
task.write_text(
|
||||
json.dumps({"title": "t", "description": "d", "target_repo": str(repo)})
|
||||
)
|
||||
return str(task)
|
||||
|
||||
|
||||
def test_run_task_ok_when_loop_produces_a_commit(tmp_path):
|
||||
repo = _init_repo(tmp_path)
|
||||
task_file = _task_file(tmp_path, repo)
|
||||
|
||||
def fake_run_loop(client, repo_root, title, description, *, max_turns, budget):
|
||||
subprocess.run(["git", "-C", str(repo_root), "commit", "-q", "--allow-empty", "-m", "task"])
|
||||
budget.consume(500)
|
||||
return LoopResult(turns=2, finished=True)
|
||||
|
||||
with (
|
||||
patch("rein_openweights.runner.resolve_openrouter_api_key", return_value="sk-test"),
|
||||
patch("rein_openweights.runner.run_loop", side_effect=fake_run_loop),
|
||||
patch("rein_openweights.runner.hub.post_progress_event", return_value=True),
|
||||
patch("rein_openweights.runner.hub.post_token_event", return_value=True),
|
||||
):
|
||||
result = run_task(task_file)
|
||||
|
||||
assert result.ok is True
|
||||
assert result.committed is True
|
||||
assert result.tokens_spent == 500
|
||||
|
||||
|
||||
def test_run_task_fails_when_no_commit_produced(tmp_path):
|
||||
repo = _init_repo(tmp_path)
|
||||
task_file = _task_file(tmp_path, repo)
|
||||
|
||||
with (
|
||||
patch("rein_openweights.runner.resolve_openrouter_api_key", return_value="sk-test"),
|
||||
patch(
|
||||
"rein_openweights.runner.run_loop",
|
||||
return_value=LoopResult(turns=1, finished=True),
|
||||
),
|
||||
patch("rein_openweights.runner.hub.post_progress_event", return_value=True),
|
||||
patch("rein_openweights.runner.hub.post_token_event", return_value=True),
|
||||
):
|
||||
result = run_task(task_file)
|
||||
|
||||
assert result.ok is False
|
||||
assert result.committed is False
|
||||
assert result.reason == "no commit produced"
|
||||
|
||||
|
||||
def test_run_task_fails_fast_without_credential(tmp_path):
|
||||
repo = _init_repo(tmp_path)
|
||||
task_file = _task_file(tmp_path, repo)
|
||||
|
||||
with patch("rein_openweights.runner.resolve_openrouter_api_key", return_value=None):
|
||||
result = run_task(task_file, report_to_hub=False)
|
||||
|
||||
assert result.ok is False
|
||||
assert "credential" in result.reason
|
||||
121
tests/test_tools.py
Normal file
121
tests/test_tools.py
Normal file
|
|
@ -0,0 +1,121 @@
|
|||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from rein_openweights.tools import (
|
||||
ToolPolicyError,
|
||||
call_tool,
|
||||
edit_file,
|
||||
git_add_commit,
|
||||
git_diff,
|
||||
git_log,
|
||||
git_status,
|
||||
glob_files,
|
||||
grep_files,
|
||||
openai_tool_schemas,
|
||||
read_file,
|
||||
write_file,
|
||||
)
|
||||
|
||||
|
||||
def _init_repo(tmp_path):
|
||||
subprocess.run(["git", "init", "-q"], cwd=tmp_path, check=True)
|
||||
subprocess.run(["git", "config", "user.email", "t@example.com"], cwd=tmp_path, check=True)
|
||||
subprocess.run(["git", "config", "user.name", "t"], cwd=tmp_path, check=True)
|
||||
subprocess.run(["git", "commit", "-q", "--allow-empty", "-m", "init"], cwd=tmp_path, check=True)
|
||||
return tmp_path
|
||||
|
||||
|
||||
def test_write_then_read_file(tmp_path):
|
||||
repo = _init_repo(tmp_path)
|
||||
write_file(repo, "a.txt", "hello")
|
||||
assert read_file(repo, "a.txt") == "hello"
|
||||
|
||||
|
||||
def test_edit_file_replaces_first_occurrence(tmp_path):
|
||||
repo = _init_repo(tmp_path)
|
||||
write_file(repo, "a.txt", "foo foo")
|
||||
edit_file(repo, "a.txt", "foo", "bar")
|
||||
assert read_file(repo, "a.txt") == "bar foo"
|
||||
|
||||
|
||||
def test_edit_file_missing_old_raises(tmp_path):
|
||||
repo = _init_repo(tmp_path)
|
||||
write_file(repo, "a.txt", "hello")
|
||||
with pytest.raises(ToolPolicyError, match="old string not found"):
|
||||
edit_file(repo, "a.txt", "nope", "x")
|
||||
|
||||
|
||||
def test_path_traversal_is_blocked(tmp_path):
|
||||
repo = _init_repo(tmp_path)
|
||||
with pytest.raises(ToolPolicyError, match="escapes repo root"):
|
||||
read_file(repo, "../outside.txt")
|
||||
|
||||
|
||||
def test_glob_files(tmp_path):
|
||||
repo = _init_repo(tmp_path)
|
||||
write_file(repo, "src/a.py", "1")
|
||||
write_file(repo, "src/b.py", "2")
|
||||
write_file(repo, "src/c.txt", "3")
|
||||
result = glob_files(repo, "**/*.py")
|
||||
assert "src/a.py" in result
|
||||
assert "src/b.py" in result
|
||||
assert "src/c.txt" not in result
|
||||
|
||||
|
||||
def test_grep_files(tmp_path):
|
||||
repo = _init_repo(tmp_path)
|
||||
write_file(repo, "a.txt", "hello world\nneedle here\n")
|
||||
result = grep_files(repo, "needle")
|
||||
assert "a.txt:2:needle here" in result
|
||||
|
||||
|
||||
def test_grep_files_no_matches(tmp_path):
|
||||
repo = _init_repo(tmp_path)
|
||||
write_file(repo, "a.txt", "hello")
|
||||
assert grep_files(repo, "zzz") == "(no matches)"
|
||||
|
||||
|
||||
def test_git_status_diff_log_and_commit(tmp_path):
|
||||
repo = _init_repo(tmp_path)
|
||||
write_file(repo, "a.txt", "hello")
|
||||
assert "a.txt" in git_status(repo)
|
||||
git_add_commit(repo, "add a.txt")
|
||||
assert git_status(repo) == "(clean)"
|
||||
assert "add a.txt" in git_log(repo)
|
||||
|
||||
|
||||
def test_git_diff_after_edit(tmp_path):
|
||||
repo = _init_repo(tmp_path)
|
||||
write_file(repo, "a.txt", "hello")
|
||||
git_add_commit(repo, "add a.txt")
|
||||
edit_file(repo, "a.txt", "hello", "goodbye")
|
||||
assert "goodbye" in git_diff(repo)
|
||||
|
||||
|
||||
def test_call_tool_dispatches_by_name(tmp_path):
|
||||
repo = _init_repo(tmp_path)
|
||||
call_tool(repo, "write_file", {"path": "a.txt", "content": "x"})
|
||||
assert read_file(repo, "a.txt") == "x"
|
||||
|
||||
|
||||
def test_call_tool_unknown_name_raises(tmp_path):
|
||||
repo = _init_repo(tmp_path)
|
||||
with pytest.raises(ToolPolicyError, match="unknown tool"):
|
||||
call_tool(repo, "delete_everything", {})
|
||||
|
||||
|
||||
def test_openai_tool_schemas_cover_all_tools():
|
||||
schemas = openai_tool_schemas()
|
||||
names = {s["function"]["name"] for s in schemas}
|
||||
assert names == {
|
||||
"read_file",
|
||||
"write_file",
|
||||
"edit_file",
|
||||
"glob_files",
|
||||
"grep_files",
|
||||
"git_status",
|
||||
"git_diff",
|
||||
"git_log",
|
||||
"git_add_commit",
|
||||
}
|
||||
|
|
@ -12,54 +12,72 @@ to implement the glas-harness rein contract once
|
|||
|
||||
## Task: Pick the starter open-weight model + OpenRouter path
|
||||
|
||||
Confirm `llm-connect` already supports the intended model class (check
|
||||
its existing OpenRouter integration, used today by rein-aharness's
|
||||
mail-triage/brief-daily paths for structured JSON — but not for a
|
||||
multi-turn tool-use loop). Identify gaps: streaming tool-call support,
|
||||
function-calling schema compatibility for the chosen open-weight model.
|
||||
Checked `llm-connect`'s existing OpenRouter integration
|
||||
(`OpenRouterAdapter.execute_prompt`): it takes a single prompt string,
|
||||
builds a fixed `[system, user]` message pair itself, and never surfaces
|
||||
`message.tool_calls` in its response — it cannot drive a multi-turn
|
||||
tool-calling loop without a breaking change to llm-connect's frozen
|
||||
Core `LLMAdapter` ABC. Decision: rein-openweights uses its own minimal
|
||||
OpenRouter client (`openrouter_client.py`) for the agentic loop instead;
|
||||
`tools`/`tool_choice` already pass through llm-connect's own
|
||||
`OPENAI_CHAT_PASSTHROUGH_FIELDS` allow-list, confirming the request
|
||||
shape is standard OpenAI-compatible function-calling, just not
|
||||
round-tripped by that adapter today. Starter model is a `--model`-configurable
|
||||
default (`meta-llama/llama-3.1-70b-instruct`), not hardcoded.
|
||||
|
||||
```task
|
||||
id: REIN-OW-WP-0001-T01
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
## Task: Minimal agentic loop
|
||||
|
||||
Implement plan → tool call → observe → repeat with a hard tool allow-list
|
||||
matching rein-aharness's `green-commit-only` (Read/Write/Edit/Glob/Grep +
|
||||
local git add/commit/status/log/diff, no push/network/shell), bounded by
|
||||
a turn/token budget (reuse `llm-connect`'s `BudgetTracker` the way
|
||||
rein-aharness's runner does).
|
||||
Implemented in `loop.py` + `tools.py`: plan → tool call → observe →
|
||||
repeat with a hard tool allow-list matching rein-aharness's
|
||||
`green-commit-only` (`read_file`/`write_file`/`edit_file`/`glob_files`/
|
||||
`grep_files` + `git_status`/`git_diff`/`git_log`/`git_add_commit`, no
|
||||
push/network/shell), bounded by `max_turns` and a token budget. Budget
|
||||
tracker is vendored locally (`budget.py`), not `llm-connect`'s —
|
||||
consistent with T01/ADR-002 keeping llm-connect optional. Path-traversal
|
||||
guarded (`_resolve`); tool errors are reported back to the model as a
|
||||
`role: tool` message rather than crashing the loop. 26 tests, all
|
||||
mocked at the `httpx`/subprocess boundary.
|
||||
|
||||
```task
|
||||
id: REIN-OW-WP-0001-T02
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
## Task: Success criterion + reporting
|
||||
|
||||
Mirror rein-aharness's model: a local git commit is the sole success
|
||||
signal, State Hub progress/token event on completion, metrics written in
|
||||
a comparable format so the two reins are measurable side by side.
|
||||
Implemented in `runner.py`/`hub.py`: a local git commit is the sole
|
||||
success signal (`RunResult.ok == committed`), State Hub progress event
|
||||
(`executor_run`) and token event posted on completion via a hub module
|
||||
mirroring rein-aharness's, `--no-hub` to skip. Not yet exercised against
|
||||
a live State Hub call in this pass (mocked in tests) — same caveat as
|
||||
rein-aharness/glas-harness's own not-yet-live-run notes elsewhere in
|
||||
this rein family.
|
||||
|
||||
```task
|
||||
id: REIN-OW-WP-0001-T03
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
```
|
||||
|
||||
## Task: Credential brokering
|
||||
|
||||
Resolve per glas-harness `GLAS-WP-0001-T06`: OpenRouter/llm-connect
|
||||
credential acquisition either brokered by glas-harness or done directly
|
||||
here via OpenBao/ops-warden. Do not hardcode a choice before that ADR
|
||||
addendum lands.
|
||||
Resolved by glas-harness `docs/adr/ADR-002-credential-brokering-and-composable-reins.md`
|
||||
(Option B): rein-openweights acquires its own OpenRouter credential.
|
||||
Implemented in `credentials.py`, mirroring rein-aharness's mailscan.py
|
||||
OpenBao AppRole/ambient-token pattern (explicit env var → AppRole
|
||||
exchange → ambient token → single `kv get`). glas-harness does not
|
||||
broker this.
|
||||
|
||||
```task
|
||||
id: REIN-OW-WP-0001-T04
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
```
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue