docs: assess current capabilities against harness intent
All checks were successful
ci / validate (push) Successful in 2m32s
All checks were successful
ci / validate (push) Successful in 2m32s
Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a0726e-5232-73f2-aaca-2c05ceb62efb
This commit is contained in:
parent
5867842b94
commit
f8942c42e8
2 changed files with 252 additions and 95 deletions
211
SCOPE.md
211
SCOPE.md
|
|
@ -1,115 +1,136 @@
|
|||
# SCOPE
|
||||
|
||||
## One-liner
|
||||
Assessed 2026-09-06 against source revision `5867842` and the recorded owner
|
||||
proofs. Detailed comparison: [scope versus intent assessment](history/2026-09-06-102533-scope-vs-intent-assessment.md).
|
||||
|
||||
`glas-harness` is a profile-driven control-plane library and CLI that resolves
|
||||
one governed execution request to a concrete rein/model/sandbox constellation,
|
||||
coordinates the outer lifecycle, and returns normalized evidence. It consumes
|
||||
`sand-boxer`; it does not provide a sandbox runtime itself.
|
||||
## Current product
|
||||
|
||||
## Verified capabilities
|
||||
`glas-harness` is a Python library and CLI for routing one execution request
|
||||
through an explicit harness profile to a concrete rein. It coordinates sandbox
|
||||
creation, one rein session, result collection and teardown. It consumes
|
||||
sand-boxer for isolation and the selected rein for its agent loop.
|
||||
|
||||
The repository currently implements:
|
||||
It is not yet the general-purpose, persistent agent harness service described
|
||||
by the maturity target in `INTENT.md`. No committed profile is `ready`.
|
||||
|
||||
- strict Pydantic models for execution requests, profiles, rein descriptors,
|
||||
lifecycle results, and compact evidence (`src/glas_harness/contract.py`);
|
||||
- deterministic, fail-closed profile and rein-registry resolution, including
|
||||
compatibility, duplicate, status, capability, and inline-secret checks
|
||||
(`src/glas_harness/profiles.py`);
|
||||
- one bounded outer gateway lifecycle: resolve, create sandbox, derive the
|
||||
returned transport, start and dispatch one rein session, summarize, clean up,
|
||||
destroy the sandbox, and report normalized evidence
|
||||
(`src/glas_harness/gateway.py`);
|
||||
- two CLI-backed rein adapters, for `rein-aharness` and `rein-openweights`;
|
||||
- transport enforcement for SSH workspaces and same-host namespace descriptors.
|
||||
Unsupported, incomplete, ambiguous, unavailable, or unauthorized transports
|
||||
fail closed without falling back to the caller checkout;
|
||||
- explicit governed actor validation (`adm`, `agt`, or `atm`) before sandbox
|
||||
creation, keeping queue `worker_id` values outside the actor field;
|
||||
- one invocation channel: the local `glas-harness run` CLI;
|
||||
- a packaged, versioned profile/rein catalog and `glas-harness profiles`
|
||||
validation command that reports catalog enablement separately from typed
|
||||
operational readiness; and
|
||||
- compact State Hub progress evidence that excludes prompts, model/tool output,
|
||||
provider bodies, and credentials.
|
||||
## Implemented capabilities
|
||||
|
||||
The unit and boundary suite exercises these contracts and the Forgejo workflow
|
||||
installs the package, runs the suite, and validates the packaged catalog.
|
||||
- Contract version `1.0`: typed requests, profiles, rein descriptors, sessions,
|
||||
tool results and execution evidence. Unknown model fields are rejected.
|
||||
- Packaged YAML profile and rein catalogs, with deterministic resolution,
|
||||
contract/status/capability checks, inline-secret screening and duplicate-key
|
||||
rejection, including nested mappings and merge collisions. An unversioned
|
||||
reference resolves only when unique; multiple versions require an exact pin.
|
||||
- Independent catalog enablement and operational readiness. Unknown,
|
||||
incompatible, disabled, ambiguous and known-blocked selections refuse before
|
||||
sandbox creation. `unverified` permits a labeled proof attempt; it is not a
|
||||
successful readiness check.
|
||||
- A synchronous outer lifecycle: resolve, create, start, dispatch `run_task`,
|
||||
summarize, clean up and destroy. Manager initialization failures return
|
||||
normalized creation-failure evidence. Cleanup and destruction are attempted
|
||||
after session failures; this is not a durable recovery/retry service.
|
||||
- Two concrete CLI adapters: `ReinAharness` and `ReinOpenWeights`. Each session
|
||||
retains its own parsed outcome, model and usage even when an adapter is
|
||||
reused. The rein owns the inner tool loop and tool-policy enforcement.
|
||||
- Same-host commands through `SandboxManager.execute`, preserving the exact
|
||||
create-time actor/project/run identity, profile credential references,
|
||||
bounded stdin and command timeout. No direct consumer `nsenter` or host
|
||||
checkout execution fallback. Remote commands use validated SSH reachability
|
||||
and the returned workspace.
|
||||
- Private, exclusive task-file creation under sandbox Git metadata, followed by
|
||||
removal through the execution transport. Prompts travel over stdin rather
|
||||
than shell arguments.
|
||||
- Governed actor types (`adm`, `agt`, `atm`) and opaque assignment, role, duty,
|
||||
goal and resource-envelope references. Attribution is not caller
|
||||
authentication or workforce authorization.
|
||||
- One channel implementation, `CLIChannel`, exposed by `glas-harness run`;
|
||||
`glas-harness profiles` validates and lists the catalog. Python callers can
|
||||
use `run_execution` directly. No HTTP/MCP service is implemented.
|
||||
- Direct-caller output/errors plus compact State Hub progress evidence.
|
||||
Ordinary execution failures are summarized for Hub reporting; raw tool
|
||||
output and event payloads are not included in that report. Reporting is
|
||||
best-effort HTTP, with no durable outbox, retry or delivery guarantee.
|
||||
- Acceptance tooling for owner-boundary probes, runtime inventory and a real
|
||||
local-rein proof. The latter validates artifact content and a single commit
|
||||
before teardown; having the runner does not establish a passing real run.
|
||||
|
||||
## Operational capability status
|
||||
Token/turn limits are passed to supporting rein adapters. Transport timeouts
|
||||
bound individual commands; Glas does not implement aggregate wall-clock or
|
||||
cross-session token accounting. Credential references are labels handed to the
|
||||
owner, not credentials or an authorization grant. Secret screening is not a
|
||||
universal redaction guarantee for arbitrary caller/provider data.
|
||||
|
||||
| Surface | Current verified status |
|
||||
|---|---|
|
||||
| Profile/catalog resolution | Implemented and CI-validated |
|
||||
| Unknown, invalid, or known-blocked profile refusal | Implemented before sandbox creation |
|
||||
| Sandbox create/destroy lifecycle | Implemented; live failure paths prove teardown |
|
||||
| Original-checkout isolation | Non-secret owner-execution proof passed: source absent, mutation refused, sandbox-only commit and complete cleanup |
|
||||
| SSH transport construction | Implemented and unit-tested; no current positive post-hardening live proof |
|
||||
| Same-host bwrap execution | Owner-mediated boundary and stdin task cleanup are live-proven; both local profiles remain `blocked` pending runtime, credential delivery, and explicit egress acceptance |
|
||||
| `rein-aharness` adapter | Implemented; local selection is blocked before creation, while the remote profile remains an explicitly labeled unverified proof path |
|
||||
| `rein-openweights` adapter | Implemented; provider credential repair was proven, but current local selection remains blocked pending its runtime, credential, and egress acceptance |
|
||||
| CLI channel | Implemented |
|
||||
| State Hub evidence | Implemented as compact progress evidence, not a complete session/tool audit service |
|
||||
## Operational evidence and limits
|
||||
|
||||
An `enabled` profile means only that catalog policy permits selection. Typed
|
||||
operational readiness separately records `ready`, `unverified`, or `blocked`.
|
||||
Known-blocked profiles refuse before sandbox creation; unverified profiles may
|
||||
run only as labeled proof attempts, with the state preserved in evidence.
|
||||
There is currently no committed `ready` profile.
|
||||
| Surface | Evidence available | Current limit |
|
||||
|---|---|---|
|
||||
| Catalog and adapters | 110 tests passed at the latest code review; packaged catalog validation passed | Tests do not prove provider authentication or deployed runtime readiness |
|
||||
| Same-host owner boundary | Non-secret live proof: source absent, mutation denied, sandbox-only commit, private task removal and workspace destruction | Deterministic dispatch, not a real model session |
|
||||
| Standalone runtime and private state | Owner proof of rein CLI startup with pinned read-only Python runtime and private HOME/XDG/TMP state | Temporary proof artifact; pinned Claude executable and production selection still pending |
|
||||
| Provider egress | Owner extension and persisted-manager proofs of provider TLS reachability, undeclared host/direct-IP denial and cleanup | Destination control, not HTTP-path or TLS-SNI filtering; not a complete real-rein proof |
|
||||
| Credential transport | Owner synthetic-key proof of delivery, redaction, next-command absence, wrong-project denial and teardown | Real Anthropic lane inactive; custody metadata confirmation does not verify its value or provider authentication |
|
||||
| Remote SSH | Construction, quoting and refusal tests | No positive post-hardening live execution proof |
|
||||
| Tool visibility | Claude stream events parsed after subprocess completion when enabled; open-weight events unavailable | No live gateway event stream or complete cross-rein tool audit |
|
||||
|
||||
## In scope
|
||||
Runtime mounting, credential brokering and egress enforcement above are
|
||||
**sand-boxer/credential-owner capabilities consumed by Glas**, not implementations
|
||||
owned by this repository. Proof evidence is in [docs/evidence](docs/evidence/)
|
||||
and the owner returns in [GLAS-WP-0012](workplans/GLAS-WP-0012-first-local-profile-production-proof.md).
|
||||
Older provider successes from before transport hardening do not prove current
|
||||
sandbox-contained execution.
|
||||
|
||||
- The versioned Glas execution contract and profile/rein catalogs.
|
||||
- Exact profile selection and refusal; no implicit governed rein or model.
|
||||
- Outer execution lifecycle and teardown coordination with sand-boxer.
|
||||
- Reachability-derived command transport and workspace selection.
|
||||
- Rein adapters that translate the stable outer contract to backend CLIs.
|
||||
- Actor attribution and organizational reference propagation without assuming
|
||||
workforce authority.
|
||||
- Direct-caller results and compact, value-safe execution evidence.
|
||||
- Channel interfaces that translate invocations into the same execution
|
||||
request; only the CLI implementation exists today.
|
||||
## Committed profile readiness
|
||||
|
||||
## Out of scope
|
||||
| Profile | Rein | Readiness |
|
||||
|---|---|---|
|
||||
| `harness.agent-dev-local@1.0.0` | rein-aharness / Claude Code | `blocked` |
|
||||
| `harness.agent-dev-openweights-local@1.0.0` | rein-openweights / OpenRouter | `blocked` |
|
||||
| `harness.agent-dev@1.0.0` | rein-aharness / remote agent-dev | `unverified` |
|
||||
|
||||
- Sandbox profiles, placement, provisioning, owner-mediated execution, and
|
||||
teardown implementation — `sand-boxer`.
|
||||
- A rein's internal agent loop, provider client, tool enforcement, and
|
||||
credential acquisition — the selected rein.
|
||||
- Scheduling, claiming, retries, and task sourcing — `activity-core` or another
|
||||
channel owner.
|
||||
- Roles, duties, assignments, goals, resource envelopes, and authorization —
|
||||
their workforce/leadership systems.
|
||||
- Model-provider authority and inference routing — providers and `llm-connect`.
|
||||
- Secret storage or credential brokering.
|
||||
- End-to-end validation and product code generation — `wise-validator` and
|
||||
`snuggle-inventor`.
|
||||
- State Hub work-record authority.
|
||||
The first local Claude acceptance is tracked by `GLAS-WP-0012`. Authorization
|
||||
contract resolution and policy publication have progressed upstream, but real
|
||||
owner verification, service deployment/identity, lane activation, runtime
|
||||
pinning and combined acceptance remain gates. No readiness promotion follows
|
||||
from those partial proofs.
|
||||
|
||||
## Intended but not implemented
|
||||
## Ownership boundaries
|
||||
|
||||
The broader vision in `INTENT.md` includes long-lived/resumable sessions,
|
||||
gateway-owned tool catalogs and policy checks, memory and skills services,
|
||||
additional chat/email/MCP channels, cron surfaces, bounded subagent delegation,
|
||||
and richer session/tool audit streams. None is a current repository capability.
|
||||
The present implementation is a bounded execution router over two reins, not
|
||||
yet Coulomb's general-purpose agent harness service.
|
||||
Glas owns the execution contract, catalogs, outer lifecycle, adapters,
|
||||
transport selection, channel translation and normalized evidence.
|
||||
|
||||
## Active gaps
|
||||
Other owners retain:
|
||||
|
||||
- `GLAS-WP-0012` / `GLAS-IN-0002`: prove the first local rein runtime,
|
||||
governed credential delivery, and explicit egress through the completed
|
||||
owner-execution adapter. The open-weight portion remains a separate live
|
||||
requirement in `GLAS-IN-0002`. See `docs/local-profile-acceptance.md`.
|
||||
- `GLAS-IN-0003`: grandfather the pre-canon `GLAS-0001` identifiers.
|
||||
- `GLAS-IN-0004`: align the documented ad hoc workplan convention with the
|
||||
identifier canon.
|
||||
- Sandbox provisioning, placement, runtime environment, exec implementation,
|
||||
network enforcement and destruction: sand-boxer.
|
||||
- Inner agent loop, provider client and tool enforcement: the selected rein;
|
||||
inference routing where applicable: llm-connect/provider owners.
|
||||
- Secret custody and governed credential delivery: OpenBao, secrets-engine and
|
||||
the sandbox owner; identity/authorization: their owning services.
|
||||
- Scheduling, worker allocation and work initiation: activity-core and
|
||||
workforce/leadership systems. Glas only carries their references.
|
||||
- Work-record authority, tunnels, certificates and production hosting: their
|
||||
existing owners, not this library.
|
||||
- Product validation workflows and code-generation pipelines: wise-validator
|
||||
and snuggle-inventor. Repository acceptance scripts test Glas integration;
|
||||
they do not replace those services.
|
||||
|
||||
## Getting oriented
|
||||
## Intent gaps and live work
|
||||
|
||||
- Intent and gap assessment: `INTENT.md`,
|
||||
`history/2026-08-23-scope-vs-intent-assessment.md`
|
||||
- Contract: `docs/harness-contract.md`, `docs/execution-profiles.md`
|
||||
- Architecture: `docs/adr/ADR-001-rein-harness-family.md`
|
||||
- Live work: `workplans/`, `docs/intakes/residuals.md`
|
||||
- Agent workflow: `AGENTS.md`
|
||||
Long-lived/resumable sessions, memory/skills services, bounded subagent
|
||||
spawn/join, additional channels, general tool-backend plugins, per-tool gateway
|
||||
policy/elevation and durable session/tool audit remain unimplemented. They are
|
||||
vision gaps, not implicitly approved workplans. The Channel and Rein interfaces
|
||||
are an extension foundation, not a complete plugin platform.
|
||||
|
||||
Live operational work remains in `GLAS-WP-0012` and `GLAS-IN-0002`; the latter
|
||||
also retains the independent open-weight acceptance requirement. Identifier
|
||||
intakes `GLAS-IN-0003` and `GLAS-IN-0004` remain open in
|
||||
[the residual register](docs/intakes/residuals.md); their historical statements
|
||||
need owner reconciliation, not an assumption that every old warning still
|
||||
occurs. The ad hoc ID-registration issue was encountered again in the latest
|
||||
review and resolved for that task, not for the fleet convention.
|
||||
|
||||
See [INTENT.md](INTENT.md) for direction, the
|
||||
[contract](docs/harness-contract.md) and
|
||||
[profile documentation](docs/execution-profiles.md) for interfaces, and
|
||||
[workplans](workplans/) for accepted work.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue