diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..e1736e9 --- /dev/null +++ b/.gitignore @@ -0,0 +1,5 @@ +.venv/ +__pycache__/ +*.pyc +*.egg-info/ +.pytest_cache/ diff --git a/INTENT.md b/INTENT.md index 84c93b0..16a7cff 100644 --- a/INTENT.md +++ b/INTENT.md @@ -14,11 +14,11 @@ repeat, with a bounded tool policy) without building one from scratch per occasion. rein-openweights exists to be that harness: one agentic loop, model -supplied via OpenRouter (through `llm-connect`), usable anywhere a -rein-aharness-equivalent session is wanted but a frontier-vendor -dependency is not — cost-sensitive workloads, offline/degraded-network -tolerance, or simply comparing open-weight model capability against -frontier baselines on the same task. +supplied via OpenRouter, usable anywhere a rein-aharness-equivalent +session is wanted but a frontier-vendor dependency is not — +cost-sensitive workloads, offline/degraded-network tolerance, or simply +comparing open-weight model capability against frontier baselines on +the same task. ## Governing principle @@ -32,8 +32,11 @@ owns the open-weight-model-specific agentic loop and tool execution. ## What it must never become - **Not a model router.** Choosing which OpenRouter model to use for a - given task, pricing, and fallback stays in `llm-connect`. This repo - consumes `llm-connect`, it does not reimplement provider routing. + given task, pricing, and fallback is a caller/config concern (`--model`), + not logic this repo owns. `llm-connect` remains available as an + optional dependency for future needs (structured-JSON side calls, + diagnostics/replay) but is not load-bearing for the base agentic loop + — see glas-harness ADR-002 and `src/rein_openweights/openrouter_client.py`. - **Not a second harness framework.** Session semantics, tool policy schema, sandbox consumption, and audit reporting are glas-harness's contract (`GLAS-WP-0001-T01`) — this repo implements it, not forks it. @@ -44,7 +47,12 @@ owns the open-weight-model-specific agentic loop and tool execution. ## Status -Charter + local scaffold only. See `workplans/REIN-OW-WP-0001-bootstrap.md` -for the first concrete milestone (minimal tool-use loop against one -OpenRouter open-weight model, same starter tool surface as -rein-aharness's `green-commit-only` profile). +Minimal agentic loop implemented and unit-tested (26 tests, all mocked +at the network/subprocess boundary): tool surface, OpenRouter client, +credential acquisition, commit-verified success criterion, State Hub +reporting. See `workplans/REIN-OW-WP-0001-bootstrap.md`. Not yet +executed against a real OpenRouter model or a real OpenBao instance — +those are human-triggered follow-ups (real API cost, real credentials). +No glas-harness adapter (`reins/rein_openweights.py`, mirroring +`reins/rein_aharness.py`) yet — that's a `GLAS-WP-0001` follow-up once +this CLI is considered stable enough to shell out to. diff --git a/README.md b/README.md index e5e738f..c544e59 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,37 @@ # rein-openweights An agentic tool-use harness that drives **current open-weight models via -OpenRouter** (through `llm-connect`), as an alternative to the -frontier-model-vendor harnesses (Claude Code, Grok CLI, Codex/GPT CLI). +OpenRouter**, as an alternative to the frontier-model-vendor harnesses +(Claude Code, Grok CLI, Codex/GPT CLI). Part of the glas-harness "rein" family — see -[glas-harness ADR-001](../glas-harness/docs/adr/ADR-001-rein-harness-family.md) -and this repo's [workplans/REIN-OW-WP-0001](workplans/REIN-OW-WP-0001-bootstrap.md). +[glas-harness ADR-001](../glas-harness/docs/adr/ADR-001-rein-harness-family.md), +[ADR-002](../glas-harness/docs/adr/ADR-002-credential-brokering-and-composable-reins.md) +(credential ownership), and this repo's +[workplans/REIN-OW-WP-0001](workplans/REIN-OW-WP-0001-bootstrap.md). + +Uses its own minimal OpenRouter client (`src/rein_openweights/openrouter_client.py`), +not `llm-connect`'s `OpenRouterAdapter` — that adapter's `execute_prompt(prompt: str, +config)` takes a single prompt string and never surfaces `message.tool_calls`, so it +can't drive a multi-turn tool-calling loop without a breaking change to llm-connect's +frozen Core ABC. `llm-connect` stays an optional dependency (see `pyproject.toml`), +not load-bearing for the base loop. Not yet pushed to Forgejo — local scaffold only. + +## Usage + +```bash +python3 -m venv .venv && source .venv/bin/activate +pip install -e . + +# Credential: either export OPENROUTER_API_KEY directly, or set +# REIN_OPENWEIGHTS_APPROLE_DIR / REIN_OPENWEIGHTS_OPENROUTER_KV_PATH for +# OpenBao-based acquisition (see src/rein_openweights/credentials.py). +export OPENROUTER_API_KEY=sk-... + +rein-openweights run --task-file examples/task-hello-sandbox.json --no-hub +``` + +Tests: `python3 -m pytest tests/ -q` (26 tests, no network/OpenBao calls — +everything mocked at the `httpx`/`bao` subprocess boundary). diff --git a/examples/task-hello-sandbox.json b/examples/task-hello-sandbox.json new file mode 100644 index 0000000..c8cf4d4 --- /dev/null +++ b/examples/task-hello-sandbox.json @@ -0,0 +1,8 @@ +{ + "title": "Sandbox smoke: NOTES.md", + "description": "Create a file NOTES.md in the repository root containing a two-sentence description of what this sandbox repository is for (read README.md first). Commit it with message 'rein-openweights smoke: add NOTES.md'.", + "target_repo": "~/executor-sandbox", + "agent": "rein-openweights", + "labels": ["executor", "smoke"], + "timeout_seconds": 600 +} diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 0000000..4f2de2b --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,34 @@ +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[project] +name = "rein-openweights" +version = "0.1.0" +description = "Agentic tool-use harness driving current open-weight models via OpenRouter" +requires-python = ">=3.11" +dependencies = [ + "httpx>=0.27", +] + +[project.scripts] +rein-openweights = "rein_openweights.cli:main" + +[tool.uv.sources] +llm-connect = { path = "../llm-connect", editable = true } + +[project.optional-dependencies] +# llm-connect stays optional per glas-harness ADR-002: the base agentic +# loop uses its own OpenRouter client and budget tracker (see +# openrouter_client.py, budget.py) so it never becomes load-bearing. +# This extra is only for future opt-in use of llm-connect's diagnostics/ +# replay/structured-JSON helpers. +llm-connect = ["llm-connect"] +dev = ["pytest>=8"] + +[tool.hatch.build.targets.wheel] +packages = ["src/rein_openweights"] + +[tool.pytest.ini_options] +testpaths = ["tests"] +pythonpath = ["src"] diff --git a/src/rein_openweights/__init__.py b/src/rein_openweights/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/rein_openweights/budget.py b/src/rein_openweights/budget.py new file mode 100644 index 0000000..0b09caf --- /dev/null +++ b/src/rein_openweights/budget.py @@ -0,0 +1,32 @@ +"""Minimal token budget tracker. + +llm-connect ships a near-identical `BudgetTracker`; rein-openweights +vendors its own so the base agentic loop never requires llm-connect as +a hard dependency (glas-harness ADR-002: llm-connect stays optional, +not load-bearing here). +""" + +from __future__ import annotations + + +class BudgetExceededError(RuntimeError): + pass + + +class BudgetTracker: + def __init__(self, total: int) -> None: + if total <= 0: + raise ValueError(f"BudgetTracker total must be positive, got {total}") + self.total = total + self.spent = 0 + + def remaining(self) -> int: + return max(0, self.total - self.spent) + + def consume(self, tokens: int) -> None: + new_spent = self.spent + tokens + if new_spent > self.total: + raise BudgetExceededError( + f"Token budget exceeded: {new_spent} tokens used, cap is {self.total}" + ) + self.spent = new_spent diff --git a/src/rein_openweights/cli.py b/src/rein_openweights/cli.py new file mode 100644 index 0000000..6279de0 --- /dev/null +++ b/src/rein_openweights/cli.py @@ -0,0 +1,41 @@ +"""rein-openweights CLI.""" + +from __future__ import annotations + +import argparse +import json +import sys +from dataclasses import asdict + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(prog="rein-openweights") + sub = parser.add_subparsers(dest="command", required=True) + + run = sub.add_parser("run", help="Run one task via the OpenRouter agentic loop") + run.add_argument("--task-file", required=True) + run.add_argument("--model", default=None) + run.add_argument("--max-turns", type=int, default=20) + run.add_argument("--budget-tokens", type=int, default=60_000) + run.add_argument("--no-hub", action="store_true", help="Skip hub reporting") + + args = parser.parse_args(argv) + + if args.command == "run": + from rein_openweights.runner import DEFAULT_MODEL, run_task + + result = run_task( + args.task_file, + model=args.model or DEFAULT_MODEL, + max_turns=args.max_turns, + budget_tokens=args.budget_tokens, + report_to_hub=not args.no_hub, + ) + print(json.dumps(asdict(result), indent=2)) + return 0 if result.ok else 1 + + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/src/rein_openweights/credentials.py b/src/rein_openweights/credentials.py new file mode 100644 index 0000000..fbe3313 --- /dev/null +++ b/src/rein_openweights/credentials.py @@ -0,0 +1,75 @@ +"""OpenRouter credential acquisition via OpenBao. + +Per glas-harness ADR-002 (Option B): rein-openweights acquires its own +credential, consistent with rein-aharness's principle that the harness +is the only credential holder. glas-harness never brokers this. + +Mirrors rein-aharness's mailscan.py pattern: AppRole (unattended lane) +first, then an ambient bao token (operator lane), then a single kv read. +""" + +from __future__ import annotations + +import os +import subprocess +from pathlib import Path + +_KV_PATH_ENV = "REIN_OPENWEIGHTS_OPENROUTER_KV_PATH" +_DEFAULT_KV_PATH = "reins/rein-openweights/openrouter" +_KV_FIELD = "api_key" +_BAO_TIMEOUT = 30 + + +class CredentialError(RuntimeError): + pass + + +def _bao(*args: str, env: dict[str, str] | None = None) -> str: + result = subprocess.run( + ["bao", *args], + capture_output=True, + text=True, + timeout=_BAO_TIMEOUT, + env=env or os.environ.copy(), + ) + if result.returncode != 0: + raise CredentialError(f"bao {args[0]} failed: {result.stderr.strip()[:200]}") + return result.stdout.strip() + + +def _acquire_token() -> str | None: + approle_dir = os.environ.get("REIN_OPENWEIGHTS_APPROLE_DIR") + if not approle_dir: + return None + role_id_file = Path(approle_dir) / "role_id" + secret_id_file = Path(approle_dir) / "secret_id" + if not (role_id_file.is_file() and secret_id_file.is_file()): + return None + return _bao( + "write", + "-field=token", + "auth/approle/login", + f"role_id={role_id_file.read_text().strip()}", + f"secret_id={secret_id_file.read_text().strip()}", + ) + + +def resolve_openrouter_api_key() -> str | None: + """Resolution order: explicit env var, then OpenBao (AppRole or ambient token).""" + explicit = os.environ.get("OPENROUTER_API_KEY") + if explicit: + return explicit + + kv_path = os.environ.get(_KV_PATH_ENV, _DEFAULT_KV_PATH) + try: + token = _acquire_token() + except CredentialError: + return None + + env = os.environ.copy() + if token: + env["BAO_TOKEN"] = token + try: + return _bao("kv", "get", f"-field={_KV_FIELD}", kv_path, env=env) + except CredentialError: + return None diff --git a/src/rein_openweights/hub.py b/src/rein_openweights/hub.py new file mode 100644 index 0000000..d541f09 --- /dev/null +++ b/src/rein_openweights/hub.py @@ -0,0 +1,65 @@ +"""Custodian State Hub reporting (REST, no MCP) — mirrors rein-aharness's hub.py.""" + +from __future__ import annotations + +import os +from typing import Any + +import httpx + +_DEFAULT_URL = "http://127.0.0.1:8000" +_TIMEOUT = 10.0 + + +def _base_url() -> str: + return os.environ.get("STATE_HUB_URL", _DEFAULT_URL).rstrip("/") + + +def post_progress_event( + summary: str, + event_type: str, + detail: dict[str, Any], + task_id: str | None = None, +) -> bool: + payload: dict[str, Any] = { + "summary": summary, + "event_type": event_type, + "detail": detail, + "author": "agt-rein-openweights", + } + if task_id: + payload["task_id"] = task_id + try: + resp = httpx.post(f"{_base_url()}/progress/", json=payload, timeout=_TIMEOUT) + resp.raise_for_status() + return True + except httpx.HTTPError: + return False + + +def post_token_event( + repo: str, + tokens: int, + *, + budget: int | None = None, + ok: bool = True, + detail: dict[str, Any] | None = None, +) -> bool: + payload: dict[str, Any] = { + "repo": repo, + "tokens": tokens, + "source": "rein-openweights", + "ok": ok, + } + if budget is not None: + payload["budget"] = budget + if detail: + payload["detail"] = detail + try: + resp = httpx.post(f"{_base_url()}/token-events/upsert", json=payload, timeout=_TIMEOUT) + if resp.status_code >= 400: + resp = httpx.post(f"{_base_url()}/token-events/", json=payload, timeout=_TIMEOUT) + resp.raise_for_status() + return True + except httpx.HTTPError: + return False diff --git a/src/rein_openweights/loop.py b/src/rein_openweights/loop.py new file mode 100644 index 0000000..a91ef1c --- /dev/null +++ b/src/rein_openweights/loop.py @@ -0,0 +1,75 @@ +"""Minimal agentic tool-use loop: plan -> tool call -> observe -> repeat.""" + +from __future__ import annotations + +import json +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any + +from rein_openweights.budget import BudgetExceededError, BudgetTracker +from rein_openweights.openrouter_client import OpenRouterClient +from rein_openweights.tools import call_tool, openai_tool_schemas + +_SYSTEM_PROMPT = ( + "You are a bounded coding agent. You may only act through the provided " + "tools. When the task is complete, call git_add_commit with a " + "descriptive message, then reply with a final message containing the " + "word DONE and no further tool calls." +) + + +@dataclass +class LoopResult: + turns: int + finished: bool + transcript: list[dict[str, Any]] = field(default_factory=list) + error: str | None = None + + +def run_loop( + client: OpenRouterClient, + repo_root: Path, + title: str, + description: str, + *, + max_turns: int = 20, + budget: BudgetTracker | None = None, +) -> LoopResult: + messages: list[dict[str, Any]] = [ + {"role": "system", "content": _SYSTEM_PROMPT}, + {"role": "user", "content": f"# {title}\n\n{description}"}, + ] + tools = openai_tool_schemas() + + for turn in range(1, max_turns + 1): + data = client.chat(messages, tools=tools) + + if budget is not None: + usage = data.get("usage", {}) + try: + budget.consume(usage.get("total_tokens", 0)) + except BudgetExceededError as exc: + return LoopResult(turns=turn, finished=False, transcript=messages, error=str(exc)) + + message = data["choices"][0]["message"] + messages.append(message) + + tool_calls = message.get("tool_calls") or [] + if not tool_calls: + finished = "DONE" in (message.get("content") or "") + return LoopResult(turns=turn, finished=finished, transcript=messages) + + for call in tool_calls: + fn = call["function"] + name = fn["name"] + try: + args = json.loads(fn.get("arguments") or "{}") + result = call_tool(repo_root, name, args) + except Exception as exc: # noqa: BLE001 - report failure to the model, keep looping + result = f"error: {exc}" + messages.append( + {"role": "tool", "tool_call_id": call.get("id", name), "content": result} + ) + + return LoopResult(turns=max_turns, finished=False, transcript=messages, error="max_turns exceeded") diff --git a/src/rein_openweights/openrouter_client.py b/src/rein_openweights/openrouter_client.py new file mode 100644 index 0000000..9088dca --- /dev/null +++ b/src/rein_openweights/openrouter_client.py @@ -0,0 +1,64 @@ +"""Minimal, independent OpenRouter chat-completions client. + +Deliberately does not depend on llm-connect's `OpenRouterAdapter`: its +`execute_prompt(prompt: str, config)` takes a single prompt string, +builds a fixed [system, user] message pair itself, and never surfaces +`message.tool_calls` in its response — it cannot drive a multi-turn +tool-calling loop without a breaking change to llm-connect's frozen +Core `LLMAdapter` ABC (`execute_prompt` signature). Rather than force +that change onto every llm-connect consumer, rein-openweights owns this +thin transport directly. See glas-harness ADR-002. +""" + +from __future__ import annotations + +from typing import Any + +import httpx + +_DEFAULT_API_BASE = "https://openrouter.ai/api/v1" + + +class OpenRouterError(RuntimeError): + pass + + +class OpenRouterClient: + def __init__( + self, + api_key: str, + model: str, + api_base: str = _DEFAULT_API_BASE, + timeout: float = 120.0, + ) -> None: + self.api_key = api_key + self.model = model + self.api_base = api_base.rstrip("/") + self.timeout = timeout + + def chat( + self, + messages: list[dict[str, Any]], + tools: list[dict[str, Any]] | None = None, + temperature: float = 0.2, + max_tokens: int = 4096, + ) -> dict[str, Any]: + payload: dict[str, Any] = { + "model": self.model, + "messages": messages, + "temperature": temperature, + "max_tokens": max_tokens, + } + if tools: + payload["tools"] = tools + payload["tool_choice"] = "auto" + + response = httpx.post( + f"{self.api_base}/chat/completions", + json=payload, + headers={"Authorization": f"Bearer {self.api_key}"}, + timeout=self.timeout, + ) + if response.status_code >= 400: + raise OpenRouterError(f"OpenRouter {response.status_code}: {response.text[:300]}") + return response.json() diff --git a/src/rein_openweights/runner.py b/src/rein_openweights/runner.py new file mode 100644 index 0000000..7e10d98 --- /dev/null +++ b/src/rein_openweights/runner.py @@ -0,0 +1,93 @@ +"""rein-openweights run — bounded agentic session, commit-verified. + +Success criterion mirrors rein-aharness: a local git commit landing is +the sole success signal. Reports a State Hub progress + token event on +completion unless --no-hub is set. +""" + +from __future__ import annotations + +import subprocess +from dataclasses import asdict, dataclass +from pathlib import Path + +from rein_openweights import hub +from rein_openweights.budget import BudgetTracker +from rein_openweights.credentials import resolve_openrouter_api_key +from rein_openweights.loop import run_loop +from rein_openweights.openrouter_client import OpenRouterClient +from rein_openweights.taskspec import TaskSpec + +DEFAULT_MODEL = "meta-llama/llama-3.1-70b-instruct" + + +@dataclass +class RunResult: + ok: bool + committed: bool + commit_sha: str + turns: int + tokens_spent: int + reason: str = "" + + +def _git_head(repo: Path) -> str: + proc = subprocess.run( + ["git", "-C", str(repo), "rev-parse", "HEAD"], capture_output=True, text=True + ) + return proc.stdout.strip() if proc.returncode == 0 else "" + + +def run_task( + task_file: str, + *, + model: str = DEFAULT_MODEL, + max_turns: int = 20, + budget_tokens: int = 60_000, + api_key: str | None = None, + report_to_hub: bool = True, +) -> RunResult: + task = TaskSpec.load(task_file) + repo = Path(task.target_repo).expanduser().resolve() + head_before = _git_head(repo) + + key = api_key or resolve_openrouter_api_key() + if not key: + result = RunResult( + ok=False, committed=False, commit_sha="", turns=0, tokens_spent=0, + reason="no OpenRouter credential resolved", + ) + _report(result, task, report_to_hub) + return result + + client = OpenRouterClient(api_key=key, model=model) + budget = BudgetTracker(total=budget_tokens) + + loop_result = run_loop( + client, repo, task.title, task.description, max_turns=max_turns, budget=budget + ) + + head_after = _git_head(repo) + committed = bool(head_after) and head_after != head_before + + result = RunResult( + ok=committed, + committed=committed, + commit_sha=head_after, + turns=loop_result.turns, + tokens_spent=budget.spent, + reason=loop_result.error or ("" if committed else "no commit produced"), + ) + _report(result, task, report_to_hub) + return result + + +def _report(result: RunResult, task: TaskSpec, report_to_hub: bool) -> None: + if not report_to_hub: + return + hub.post_progress_event( + summary=f"rein-openweights run {'ok' if result.ok else 'failed'}: {task.title}", + event_type="executor_run", + detail=asdict(result), + ) + hub.post_token_event(repo=task.target_repo, tokens=result.tokens_spent, ok=result.ok) diff --git a/src/rein_openweights/taskspec.py b/src/rein_openweights/taskspec.py new file mode 100644 index 0000000..6351327 --- /dev/null +++ b/src/rein_openweights/taskspec.py @@ -0,0 +1,29 @@ +"""TaskSpec — local JSON task definition, mirrors rein-aharness's taskspec.py.""" + +from __future__ import annotations + +import json +from dataclasses import dataclass, field +from pathlib import Path + + +@dataclass +class TaskSpec: + title: str + description: str + target_repo: str + agent: str = "rein-openweights" + labels: list[str] = field(default_factory=list) + timeout_seconds: int = 600 + + @classmethod + def load(cls, path: str) -> TaskSpec: + data = json.loads(Path(path).expanduser().read_text()) + return cls( + title=data["title"], + description=data["description"], + target_repo=data["target_repo"], + agent=data.get("agent", "rein-openweights"), + labels=data.get("labels", []), + timeout_seconds=data.get("timeout_seconds", 600), + ) diff --git a/src/rein_openweights/tools.py b/src/rein_openweights/tools.py new file mode 100644 index 0000000..29cfc83 --- /dev/null +++ b/src/rein_openweights/tools.py @@ -0,0 +1,187 @@ +"""Tool surface — mirrors rein-aharness's `green-commit-only` allow-list. + +Read/Write/Edit/Glob/Grep + local git status/diff/log/add+commit. No +push, no network, no shell escape hatch — the same bounded surface, just +invoked by a different rein. +""" + +from __future__ import annotations + +import subprocess +from dataclasses import dataclass +from pathlib import Path +from typing import Any, Callable + + +class ToolPolicyError(RuntimeError): + pass + + +def _resolve(repo_root: Path, rel_path: str) -> Path: + repo_root = repo_root.resolve() + candidate = (repo_root / rel_path).resolve() + if candidate != repo_root and repo_root not in candidate.parents: + raise ToolPolicyError(f"path escapes repo root: {rel_path}") + return candidate + + +def read_file(repo_root: Path, path: str) -> str: + return _resolve(repo_root, path).read_text() + + +def write_file(repo_root: Path, path: str, content: str) -> str: + target = _resolve(repo_root, path) + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text(content) + return f"wrote {len(content)} bytes to {path}" + + +def edit_file(repo_root: Path, path: str, old: str, new: str) -> str: + target = _resolve(repo_root, path) + text = target.read_text() + if old not in text: + raise ToolPolicyError(f"old string not found in {path}") + target.write_text(text.replace(old, new, 1)) + return f"replaced 1 occurrence in {path}" + + +def glob_files(repo_root: Path, pattern: str) -> str: + repo_root = repo_root.resolve() + matches = sorted(str(p.relative_to(repo_root)) for p in repo_root.rglob(pattern) if p.is_file()) + return "\n".join(matches) or "(no matches)" + + +def grep_files(repo_root: Path, pattern: str, path: str = ".") -> str: + base = _resolve(repo_root, path) + hits: list[str] = [] + for file in sorted(base.rglob("*")): + if not file.is_file() or ".git" in file.parts: + continue + try: + text = file.read_text() + except (UnicodeDecodeError, OSError): + continue + for lineno, line in enumerate(text.splitlines(), start=1): + if pattern in line: + hits.append(f"{file.relative_to(repo_root)}:{lineno}:{line}") + return "\n".join(hits) or "(no matches)" + + +def _git(repo_root: Path, *args: str) -> str: + proc = subprocess.run(["git", "-C", str(repo_root), *args], capture_output=True, text=True) + if proc.returncode != 0: + raise ToolPolicyError(f"git {' '.join(args)} failed: {proc.stderr.strip()}") + return proc.stdout + + +def git_status(repo_root: Path) -> str: + return _git(repo_root, "status", "--short") or "(clean)" + + +def git_diff(repo_root: Path) -> str: + return _git(repo_root, "diff") or "(no diff)" + + +def git_log(repo_root: Path, max_count: int = 10) -> str: + return _git(repo_root, "log", f"-{max_count}", "--oneline") + + +def git_add_commit(repo_root: Path, message: str) -> str: + _git(repo_root, "add", "-A") + return _git(repo_root, "commit", "-m", message) + + +@dataclass +class Tool: + name: str + description: str + parameters: dict[str, Any] + fn: Callable[..., str] + + +TOOLS: list[Tool] = [ + Tool( + "read_file", + "Read a file's contents.", + {"type": "object", "properties": {"path": {"type": "string"}}, "required": ["path"]}, + read_file, + ), + Tool( + "write_file", + "Write (overwrite) a file's contents, creating parent dirs as needed.", + { + "type": "object", + "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, + "required": ["path", "content"], + }, + write_file, + ), + Tool( + "edit_file", + "Replace the first occurrence of `old` with `new` in a file.", + { + "type": "object", + "properties": { + "path": {"type": "string"}, + "old": {"type": "string"}, + "new": {"type": "string"}, + }, + "required": ["path", "old", "new"], + }, + edit_file, + ), + Tool( + "glob_files", + "List files under the repo matching a glob pattern (e.g. '**/*.py').", + {"type": "object", "properties": {"pattern": {"type": "string"}}, "required": ["pattern"]}, + glob_files, + ), + Tool( + "grep_files", + "Search file contents under `path` for a literal substring.", + { + "type": "object", + "properties": {"pattern": {"type": "string"}, "path": {"type": "string"}}, + "required": ["pattern"], + }, + grep_files, + ), + Tool("git_status", "Show `git status --short`.", {"type": "object", "properties": {}}, git_status), + Tool("git_diff", "Show `git diff`.", {"type": "object", "properties": {}}, git_diff), + Tool( + "git_log", + "Show recent commit log (oneline).", + {"type": "object", "properties": {"max_count": {"type": "integer"}}}, + git_log, + ), + Tool( + "git_add_commit", + "Stage all changes (`git add -A`) and commit with a message. " + "Call this once the task is complete.", + {"type": "object", "properties": {"message": {"type": "string"}}, "required": ["message"]}, + git_add_commit, + ), +] + +TOOLS_BY_NAME = {t.name: t for t in TOOLS} + + +def openai_tool_schemas() -> list[dict[str, Any]]: + return [ + { + "type": "function", + "function": { + "name": t.name, + "description": t.description, + "parameters": t.parameters, + }, + } + for t in TOOLS + ] + + +def call_tool(repo_root: Path, name: str, args: dict[str, Any]) -> str: + tool = TOOLS_BY_NAME.get(name) + if tool is None: + raise ToolPolicyError(f"unknown tool: {name}") + return tool.fn(repo_root, **args) diff --git a/tests/test_credentials.py b/tests/test_credentials.py new file mode 100644 index 0000000..31851f9 --- /dev/null +++ b/tests/test_credentials.py @@ -0,0 +1,54 @@ +from unittest.mock import patch + +from rein_openweights.credentials import resolve_openrouter_api_key + + +def test_explicit_env_var_short_circuits(monkeypatch): + monkeypatch.setenv("OPENROUTER_API_KEY", "sk-explicit") + assert resolve_openrouter_api_key() == "sk-explicit" + + +def test_falls_back_to_bao_kv_when_no_approle(monkeypatch): + monkeypatch.delenv("OPENROUTER_API_KEY", raising=False) + monkeypatch.delenv("REIN_OPENWEIGHTS_APPROLE_DIR", raising=False) + + with patch("rein_openweights.credentials._bao", return_value="sk-from-vault") as bao: + key = resolve_openrouter_api_key() + + assert key == "sk-from-vault" + bao.assert_called_once() + assert bao.call_args.args[0] == "kv" + + +def test_approle_lane_exchanges_token_before_kv_read(monkeypatch, tmp_path): + monkeypatch.delenv("OPENROUTER_API_KEY", raising=False) + role_id = tmp_path / "role_id" + secret_id = tmp_path / "secret_id" + role_id.write_text("role-123") + secret_id.write_text("secret-456") + monkeypatch.setenv("REIN_OPENWEIGHTS_APPROLE_DIR", str(tmp_path)) + + calls = [] + + def fake_bao(*args, env=None): + calls.append(args) + if args[0] == "write": + return "vault-token" + return "sk-from-vault" + + with patch("rein_openweights.credentials._bao", side_effect=fake_bao): + key = resolve_openrouter_api_key() + + assert key == "sk-from-vault" + assert calls[0][0] == "write" + assert calls[1][0] == "kv" + + +def test_returns_none_when_bao_fails(monkeypatch): + monkeypatch.delenv("OPENROUTER_API_KEY", raising=False) + monkeypatch.delenv("REIN_OPENWEIGHTS_APPROLE_DIR", raising=False) + + from rein_openweights.credentials import CredentialError + + with patch("rein_openweights.credentials._bao", side_effect=CredentialError("boom")): + assert resolve_openrouter_api_key() is None diff --git a/tests/test_loop.py b/tests/test_loop.py new file mode 100644 index 0000000..4897b6e --- /dev/null +++ b/tests/test_loop.py @@ -0,0 +1,118 @@ +import subprocess +from unittest.mock import MagicMock + +from rein_openweights.budget import BudgetTracker +from rein_openweights.loop import run_loop + + +def _init_repo(tmp_path): + subprocess.run(["git", "init", "-q"], cwd=tmp_path, check=True) + subprocess.run(["git", "config", "user.email", "t@example.com"], cwd=tmp_path, check=True) + subprocess.run(["git", "config", "user.name", "t"], cwd=tmp_path, check=True) + subprocess.run(["git", "commit", "-q", "--allow-empty", "-m", "init"], cwd=tmp_path, check=True) + return tmp_path + + +def _response(message, total_tokens=100): + return { + "choices": [{"message": message}], + "usage": {"total_tokens": total_tokens}, + } + + +def test_run_loop_executes_tool_call_then_finishes(tmp_path): + repo = _init_repo(tmp_path) + client = MagicMock() + client.chat.side_effect = [ + _response( + { + "role": "assistant", + "content": None, + "tool_calls": [ + { + "id": "call_1", + "function": { + "name": "write_file", + "arguments": '{"path": "NOTES.md", "content": "hi"}', + }, + } + ], + } + ), + _response( + { + "role": "assistant", + "tool_calls": [ + { + "id": "call_2", + "function": { + "name": "git_add_commit", + "arguments": '{"message": "add notes"}', + }, + } + ], + } + ), + _response({"role": "assistant", "content": "All set. DONE"}), + ] + + result = run_loop(client, repo, "t", "d", budget=BudgetTracker(total=10_000)) + + assert result.finished is True + assert result.turns == 3 + assert (repo / "NOTES.md").read_text() == "hi" + log = subprocess.run( + ["git", "-C", str(repo), "log", "--oneline"], capture_output=True, text=True + ).stdout + assert "add notes" in log + + +def test_run_loop_stops_when_budget_exceeded(tmp_path): + repo = _init_repo(tmp_path) + client = MagicMock() + client.chat.return_value = _response({"role": "assistant", "content": "still going"}, total_tokens=100) + + result = run_loop(client, repo, "t", "d", budget=BudgetTracker(total=50)) + + assert result.error is not None + assert "budget" in result.error.lower() + + +def test_run_loop_stops_at_max_turns(tmp_path): + repo = _init_repo(tmp_path) + client = MagicMock() + client.chat.return_value = _response( + { + "role": "assistant", + "tool_calls": [ + {"id": "c", "function": {"name": "git_status", "arguments": "{}"}} + ], + } + ) + + result = run_loop(client, repo, "t", "d", max_turns=2) + + assert result.turns == 2 + assert result.error == "max_turns exceeded" + + +def test_run_loop_reports_tool_error_without_crashing(tmp_path): + repo = _init_repo(tmp_path) + client = MagicMock() + client.chat.side_effect = [ + _response( + { + "role": "assistant", + "tool_calls": [ + {"id": "c", "function": {"name": "read_file", "arguments": '{"path": "missing.txt"}'}} + ], + } + ), + _response({"role": "assistant", "content": "done anyway DONE"}), + ] + + result = run_loop(client, repo, "t", "d") + + assert result.finished is True + tool_messages = [m for m in result.transcript if m.get("role") == "tool"] + assert "error" in tool_messages[0]["content"] diff --git a/tests/test_openrouter_client.py b/tests/test_openrouter_client.py new file mode 100644 index 0000000..02194d6 --- /dev/null +++ b/tests/test_openrouter_client.py @@ -0,0 +1,43 @@ +from unittest.mock import MagicMock, patch + +import pytest + +from rein_openweights.openrouter_client import OpenRouterClient, OpenRouterError + + +def test_chat_sends_bearer_auth_and_model(): + client = OpenRouterClient(api_key="sk-test", model="meta-llama/llama-3.1-70b-instruct") + fake_response = MagicMock(status_code=200) + fake_response.json.return_value = {"choices": [{"message": {"content": "hi"}}]} + + with patch("rein_openweights.openrouter_client.httpx.post", return_value=fake_response) as post: + data = client.chat([{"role": "user", "content": "hello"}]) + + assert data["choices"][0]["message"]["content"] == "hi" + _, kwargs = post.call_args + assert kwargs["headers"]["Authorization"] == "Bearer sk-test" + assert kwargs["json"]["model"] == "meta-llama/llama-3.1-70b-instruct" + assert "tools" not in kwargs["json"] + + +def test_chat_includes_tools_when_provided(): + client = OpenRouterClient(api_key="sk-test", model="m") + fake_response = MagicMock(status_code=200) + fake_response.json.return_value = {"choices": [{"message": {}}]} + tools = [{"type": "function", "function": {"name": "read_file"}}] + + with patch("rein_openweights.openrouter_client.httpx.post", return_value=fake_response) as post: + client.chat([{"role": "user", "content": "x"}], tools=tools) + + _, kwargs = post.call_args + assert kwargs["json"]["tools"] == tools + assert kwargs["json"]["tool_choice"] == "auto" + + +def test_chat_raises_on_http_error(): + client = OpenRouterClient(api_key="sk-test", model="m") + fake_response = MagicMock(status_code=401, text="unauthorized") + + with patch("rein_openweights.openrouter_client.httpx.post", return_value=fake_response): + with pytest.raises(OpenRouterError, match="401"): + client.chat([{"role": "user", "content": "x"}]) diff --git a/tests/test_runner.py b/tests/test_runner.py new file mode 100644 index 0000000..7c0608d --- /dev/null +++ b/tests/test_runner.py @@ -0,0 +1,75 @@ +import json +import subprocess +from unittest.mock import patch + +from rein_openweights.loop import LoopResult +from rein_openweights.runner import run_task + + +def _init_repo(tmp_path): + subprocess.run(["git", "init", "-q"], cwd=tmp_path, check=True) + subprocess.run(["git", "config", "user.email", "t@example.com"], cwd=tmp_path, check=True) + subprocess.run(["git", "config", "user.name", "t"], cwd=tmp_path, check=True) + subprocess.run(["git", "commit", "-q", "--allow-empty", "-m", "init"], cwd=tmp_path, check=True) + return tmp_path + + +def _task_file(tmp_path, repo): + task = tmp_path / "task.json" + task.write_text( + json.dumps({"title": "t", "description": "d", "target_repo": str(repo)}) + ) + return str(task) + + +def test_run_task_ok_when_loop_produces_a_commit(tmp_path): + repo = _init_repo(tmp_path) + task_file = _task_file(tmp_path, repo) + + def fake_run_loop(client, repo_root, title, description, *, max_turns, budget): + subprocess.run(["git", "-C", str(repo_root), "commit", "-q", "--allow-empty", "-m", "task"]) + budget.consume(500) + return LoopResult(turns=2, finished=True) + + with ( + patch("rein_openweights.runner.resolve_openrouter_api_key", return_value="sk-test"), + patch("rein_openweights.runner.run_loop", side_effect=fake_run_loop), + patch("rein_openweights.runner.hub.post_progress_event", return_value=True), + patch("rein_openweights.runner.hub.post_token_event", return_value=True), + ): + result = run_task(task_file) + + assert result.ok is True + assert result.committed is True + assert result.tokens_spent == 500 + + +def test_run_task_fails_when_no_commit_produced(tmp_path): + repo = _init_repo(tmp_path) + task_file = _task_file(tmp_path, repo) + + with ( + patch("rein_openweights.runner.resolve_openrouter_api_key", return_value="sk-test"), + patch( + "rein_openweights.runner.run_loop", + return_value=LoopResult(turns=1, finished=True), + ), + patch("rein_openweights.runner.hub.post_progress_event", return_value=True), + patch("rein_openweights.runner.hub.post_token_event", return_value=True), + ): + result = run_task(task_file) + + assert result.ok is False + assert result.committed is False + assert result.reason == "no commit produced" + + +def test_run_task_fails_fast_without_credential(tmp_path): + repo = _init_repo(tmp_path) + task_file = _task_file(tmp_path, repo) + + with patch("rein_openweights.runner.resolve_openrouter_api_key", return_value=None): + result = run_task(task_file, report_to_hub=False) + + assert result.ok is False + assert "credential" in result.reason diff --git a/tests/test_tools.py b/tests/test_tools.py new file mode 100644 index 0000000..6ba42ea --- /dev/null +++ b/tests/test_tools.py @@ -0,0 +1,121 @@ +import subprocess + +import pytest + +from rein_openweights.tools import ( + ToolPolicyError, + call_tool, + edit_file, + git_add_commit, + git_diff, + git_log, + git_status, + glob_files, + grep_files, + openai_tool_schemas, + read_file, + write_file, +) + + +def _init_repo(tmp_path): + subprocess.run(["git", "init", "-q"], cwd=tmp_path, check=True) + subprocess.run(["git", "config", "user.email", "t@example.com"], cwd=tmp_path, check=True) + subprocess.run(["git", "config", "user.name", "t"], cwd=tmp_path, check=True) + subprocess.run(["git", "commit", "-q", "--allow-empty", "-m", "init"], cwd=tmp_path, check=True) + return tmp_path + + +def test_write_then_read_file(tmp_path): + repo = _init_repo(tmp_path) + write_file(repo, "a.txt", "hello") + assert read_file(repo, "a.txt") == "hello" + + +def test_edit_file_replaces_first_occurrence(tmp_path): + repo = _init_repo(tmp_path) + write_file(repo, "a.txt", "foo foo") + edit_file(repo, "a.txt", "foo", "bar") + assert read_file(repo, "a.txt") == "bar foo" + + +def test_edit_file_missing_old_raises(tmp_path): + repo = _init_repo(tmp_path) + write_file(repo, "a.txt", "hello") + with pytest.raises(ToolPolicyError, match="old string not found"): + edit_file(repo, "a.txt", "nope", "x") + + +def test_path_traversal_is_blocked(tmp_path): + repo = _init_repo(tmp_path) + with pytest.raises(ToolPolicyError, match="escapes repo root"): + read_file(repo, "../outside.txt") + + +def test_glob_files(tmp_path): + repo = _init_repo(tmp_path) + write_file(repo, "src/a.py", "1") + write_file(repo, "src/b.py", "2") + write_file(repo, "src/c.txt", "3") + result = glob_files(repo, "**/*.py") + assert "src/a.py" in result + assert "src/b.py" in result + assert "src/c.txt" not in result + + +def test_grep_files(tmp_path): + repo = _init_repo(tmp_path) + write_file(repo, "a.txt", "hello world\nneedle here\n") + result = grep_files(repo, "needle") + assert "a.txt:2:needle here" in result + + +def test_grep_files_no_matches(tmp_path): + repo = _init_repo(tmp_path) + write_file(repo, "a.txt", "hello") + assert grep_files(repo, "zzz") == "(no matches)" + + +def test_git_status_diff_log_and_commit(tmp_path): + repo = _init_repo(tmp_path) + write_file(repo, "a.txt", "hello") + assert "a.txt" in git_status(repo) + git_add_commit(repo, "add a.txt") + assert git_status(repo) == "(clean)" + assert "add a.txt" in git_log(repo) + + +def test_git_diff_after_edit(tmp_path): + repo = _init_repo(tmp_path) + write_file(repo, "a.txt", "hello") + git_add_commit(repo, "add a.txt") + edit_file(repo, "a.txt", "hello", "goodbye") + assert "goodbye" in git_diff(repo) + + +def test_call_tool_dispatches_by_name(tmp_path): + repo = _init_repo(tmp_path) + call_tool(repo, "write_file", {"path": "a.txt", "content": "x"}) + assert read_file(repo, "a.txt") == "x" + + +def test_call_tool_unknown_name_raises(tmp_path): + repo = _init_repo(tmp_path) + with pytest.raises(ToolPolicyError, match="unknown tool"): + call_tool(repo, "delete_everything", {}) + + +def test_openai_tool_schemas_cover_all_tools(): + schemas = openai_tool_schemas() + names = {s["function"]["name"] for s in schemas} + assert names == { + "read_file", + "write_file", + "edit_file", + "glob_files", + "grep_files", + "git_status", + "git_diff", + "git_log", + "git_add_commit", + } diff --git a/workplans/REIN-OW-WP-0001-bootstrap.md b/workplans/REIN-OW-WP-0001-bootstrap.md index a9e0d39..5b71f88 100644 --- a/workplans/REIN-OW-WP-0001-bootstrap.md +++ b/workplans/REIN-OW-WP-0001-bootstrap.md @@ -12,54 +12,72 @@ to implement the glas-harness rein contract once ## Task: Pick the starter open-weight model + OpenRouter path -Confirm `llm-connect` already supports the intended model class (check -its existing OpenRouter integration, used today by rein-aharness's -mail-triage/brief-daily paths for structured JSON — but not for a -multi-turn tool-use loop). Identify gaps: streaming tool-call support, -function-calling schema compatibility for the chosen open-weight model. +Checked `llm-connect`'s existing OpenRouter integration +(`OpenRouterAdapter.execute_prompt`): it takes a single prompt string, +builds a fixed `[system, user]` message pair itself, and never surfaces +`message.tool_calls` in its response — it cannot drive a multi-turn +tool-calling loop without a breaking change to llm-connect's frozen +Core `LLMAdapter` ABC. Decision: rein-openweights uses its own minimal +OpenRouter client (`openrouter_client.py`) for the agentic loop instead; +`tools`/`tool_choice` already pass through llm-connect's own +`OPENAI_CHAT_PASSTHROUGH_FIELDS` allow-list, confirming the request +shape is standard OpenAI-compatible function-calling, just not +round-tripped by that adapter today. Starter model is a `--model`-configurable +default (`meta-llama/llama-3.1-70b-instruct`), not hardcoded. ```task id: REIN-OW-WP-0001-T01 -status: todo +status: done priority: high ``` ## Task: Minimal agentic loop -Implement plan → tool call → observe → repeat with a hard tool allow-list -matching rein-aharness's `green-commit-only` (Read/Write/Edit/Glob/Grep + -local git add/commit/status/log/diff, no push/network/shell), bounded by -a turn/token budget (reuse `llm-connect`'s `BudgetTracker` the way -rein-aharness's runner does). +Implemented in `loop.py` + `tools.py`: plan → tool call → observe → +repeat with a hard tool allow-list matching rein-aharness's +`green-commit-only` (`read_file`/`write_file`/`edit_file`/`glob_files`/ +`grep_files` + `git_status`/`git_diff`/`git_log`/`git_add_commit`, no +push/network/shell), bounded by `max_turns` and a token budget. Budget +tracker is vendored locally (`budget.py`), not `llm-connect`'s — +consistent with T01/ADR-002 keeping llm-connect optional. Path-traversal +guarded (`_resolve`); tool errors are reported back to the model as a +`role: tool` message rather than crashing the loop. 26 tests, all +mocked at the `httpx`/subprocess boundary. ```task id: REIN-OW-WP-0001-T02 -status: todo +status: done priority: high ``` ## Task: Success criterion + reporting -Mirror rein-aharness's model: a local git commit is the sole success -signal, State Hub progress/token event on completion, metrics written in -a comparable format so the two reins are measurable side by side. +Implemented in `runner.py`/`hub.py`: a local git commit is the sole +success signal (`RunResult.ok == committed`), State Hub progress event +(`executor_run`) and token event posted on completion via a hub module +mirroring rein-aharness's, `--no-hub` to skip. Not yet exercised against +a live State Hub call in this pass (mocked in tests) — same caveat as +rein-aharness/glas-harness's own not-yet-live-run notes elsewhere in +this rein family. ```task id: REIN-OW-WP-0001-T03 -status: todo +status: done priority: medium ``` ## Task: Credential brokering -Resolve per glas-harness `GLAS-WP-0001-T06`: OpenRouter/llm-connect -credential acquisition either brokered by glas-harness or done directly -here via OpenBao/ops-warden. Do not hardcode a choice before that ADR -addendum lands. +Resolved by glas-harness `docs/adr/ADR-002-credential-brokering-and-composable-reins.md` +(Option B): rein-openweights acquires its own OpenRouter credential. +Implemented in `credentials.py`, mirroring rein-aharness's mailscan.py +OpenBao AppRole/ambient-token pattern (explicit env var → AppRole +exchange → ambient token → single `kv get`). glas-harness does not +broker this. ```task id: REIN-OW-WP-0001-T04 -status: todo +status: done priority: medium ```