From f8942c42e8ab5bac0314cc50870967af41fa290d Mon Sep 17 00:00:00 2001 From: tegwick Date: Sun, 6 Sep 2026 10:28:09 +0200 Subject: [PATCH] docs: assess current capabilities against harness intent Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a0726e-5232-73f2-aaca-2c05ceb62efb --- SCOPE.md | 211 ++++++++++-------- ...09-06-102533-scope-vs-intent-assessment.md | 136 +++++++++++ 2 files changed, 252 insertions(+), 95 deletions(-) create mode 100644 history/2026-09-06-102533-scope-vs-intent-assessment.md diff --git a/SCOPE.md b/SCOPE.md index 2b1bfc6..c0ba7f7 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -1,115 +1,136 @@ # SCOPE -## One-liner +Assessed 2026-09-06 against source revision `5867842` and the recorded owner +proofs. Detailed comparison: [scope versus intent assessment](history/2026-09-06-102533-scope-vs-intent-assessment.md). -`glas-harness` is a profile-driven control-plane library and CLI that resolves -one governed execution request to a concrete rein/model/sandbox constellation, -coordinates the outer lifecycle, and returns normalized evidence. It consumes -`sand-boxer`; it does not provide a sandbox runtime itself. +## Current product -## Verified capabilities +`glas-harness` is a Python library and CLI for routing one execution request +through an explicit harness profile to a concrete rein. It coordinates sandbox +creation, one rein session, result collection and teardown. It consumes +sand-boxer for isolation and the selected rein for its agent loop. -The repository currently implements: +It is not yet the general-purpose, persistent agent harness service described +by the maturity target in `INTENT.md`. No committed profile is `ready`. -- strict Pydantic models for execution requests, profiles, rein descriptors, - lifecycle results, and compact evidence (`src/glas_harness/contract.py`); -- deterministic, fail-closed profile and rein-registry resolution, including - compatibility, duplicate, status, capability, and inline-secret checks - (`src/glas_harness/profiles.py`); -- one bounded outer gateway lifecycle: resolve, create sandbox, derive the - returned transport, start and dispatch one rein session, summarize, clean up, - destroy the sandbox, and report normalized evidence - (`src/glas_harness/gateway.py`); -- two CLI-backed rein adapters, for `rein-aharness` and `rein-openweights`; -- transport enforcement for SSH workspaces and same-host namespace descriptors. - Unsupported, incomplete, ambiguous, unavailable, or unauthorized transports - fail closed without falling back to the caller checkout; -- explicit governed actor validation (`adm`, `agt`, or `atm`) before sandbox - creation, keeping queue `worker_id` values outside the actor field; -- one invocation channel: the local `glas-harness run` CLI; -- a packaged, versioned profile/rein catalog and `glas-harness profiles` - validation command that reports catalog enablement separately from typed - operational readiness; and -- compact State Hub progress evidence that excludes prompts, model/tool output, - provider bodies, and credentials. +## Implemented capabilities -The unit and boundary suite exercises these contracts and the Forgejo workflow -installs the package, runs the suite, and validates the packaged catalog. +- Contract version `1.0`: typed requests, profiles, rein descriptors, sessions, + tool results and execution evidence. Unknown model fields are rejected. +- Packaged YAML profile and rein catalogs, with deterministic resolution, + contract/status/capability checks, inline-secret screening and duplicate-key + rejection, including nested mappings and merge collisions. An unversioned + reference resolves only when unique; multiple versions require an exact pin. +- Independent catalog enablement and operational readiness. Unknown, + incompatible, disabled, ambiguous and known-blocked selections refuse before + sandbox creation. `unverified` permits a labeled proof attempt; it is not a + successful readiness check. +- A synchronous outer lifecycle: resolve, create, start, dispatch `run_task`, + summarize, clean up and destroy. Manager initialization failures return + normalized creation-failure evidence. Cleanup and destruction are attempted + after session failures; this is not a durable recovery/retry service. +- Two concrete CLI adapters: `ReinAharness` and `ReinOpenWeights`. Each session + retains its own parsed outcome, model and usage even when an adapter is + reused. The rein owns the inner tool loop and tool-policy enforcement. +- Same-host commands through `SandboxManager.execute`, preserving the exact + create-time actor/project/run identity, profile credential references, + bounded stdin and command timeout. No direct consumer `nsenter` or host + checkout execution fallback. Remote commands use validated SSH reachability + and the returned workspace. +- Private, exclusive task-file creation under sandbox Git metadata, followed by + removal through the execution transport. Prompts travel over stdin rather + than shell arguments. +- Governed actor types (`adm`, `agt`, `atm`) and opaque assignment, role, duty, + goal and resource-envelope references. Attribution is not caller + authentication or workforce authorization. +- One channel implementation, `CLIChannel`, exposed by `glas-harness run`; + `glas-harness profiles` validates and lists the catalog. Python callers can + use `run_execution` directly. No HTTP/MCP service is implemented. +- Direct-caller output/errors plus compact State Hub progress evidence. + Ordinary execution failures are summarized for Hub reporting; raw tool + output and event payloads are not included in that report. Reporting is + best-effort HTTP, with no durable outbox, retry or delivery guarantee. +- Acceptance tooling for owner-boundary probes, runtime inventory and a real + local-rein proof. The latter validates artifact content and a single commit + before teardown; having the runner does not establish a passing real run. -## Operational capability status +Token/turn limits are passed to supporting rein adapters. Transport timeouts +bound individual commands; Glas does not implement aggregate wall-clock or +cross-session token accounting. Credential references are labels handed to the +owner, not credentials or an authorization grant. Secret screening is not a +universal redaction guarantee for arbitrary caller/provider data. -| Surface | Current verified status | -|---|---| -| Profile/catalog resolution | Implemented and CI-validated | -| Unknown, invalid, or known-blocked profile refusal | Implemented before sandbox creation | -| Sandbox create/destroy lifecycle | Implemented; live failure paths prove teardown | -| Original-checkout isolation | Non-secret owner-execution proof passed: source absent, mutation refused, sandbox-only commit and complete cleanup | -| SSH transport construction | Implemented and unit-tested; no current positive post-hardening live proof | -| Same-host bwrap execution | Owner-mediated boundary and stdin task cleanup are live-proven; both local profiles remain `blocked` pending runtime, credential delivery, and explicit egress acceptance | -| `rein-aharness` adapter | Implemented; local selection is blocked before creation, while the remote profile remains an explicitly labeled unverified proof path | -| `rein-openweights` adapter | Implemented; provider credential repair was proven, but current local selection remains blocked pending its runtime, credential, and egress acceptance | -| CLI channel | Implemented | -| State Hub evidence | Implemented as compact progress evidence, not a complete session/tool audit service | +## Operational evidence and limits -An `enabled` profile means only that catalog policy permits selection. Typed -operational readiness separately records `ready`, `unverified`, or `blocked`. -Known-blocked profiles refuse before sandbox creation; unverified profiles may -run only as labeled proof attempts, with the state preserved in evidence. -There is currently no committed `ready` profile. +| Surface | Evidence available | Current limit | +|---|---|---| +| Catalog and adapters | 110 tests passed at the latest code review; packaged catalog validation passed | Tests do not prove provider authentication or deployed runtime readiness | +| Same-host owner boundary | Non-secret live proof: source absent, mutation denied, sandbox-only commit, private task removal and workspace destruction | Deterministic dispatch, not a real model session | +| Standalone runtime and private state | Owner proof of rein CLI startup with pinned read-only Python runtime and private HOME/XDG/TMP state | Temporary proof artifact; pinned Claude executable and production selection still pending | +| Provider egress | Owner extension and persisted-manager proofs of provider TLS reachability, undeclared host/direct-IP denial and cleanup | Destination control, not HTTP-path or TLS-SNI filtering; not a complete real-rein proof | +| Credential transport | Owner synthetic-key proof of delivery, redaction, next-command absence, wrong-project denial and teardown | Real Anthropic lane inactive; custody metadata confirmation does not verify its value or provider authentication | +| Remote SSH | Construction, quoting and refusal tests | No positive post-hardening live execution proof | +| Tool visibility | Claude stream events parsed after subprocess completion when enabled; open-weight events unavailable | No live gateway event stream or complete cross-rein tool audit | -## In scope +Runtime mounting, credential brokering and egress enforcement above are +**sand-boxer/credential-owner capabilities consumed by Glas**, not implementations +owned by this repository. Proof evidence is in [docs/evidence](docs/evidence/) +and the owner returns in [GLAS-WP-0012](workplans/GLAS-WP-0012-first-local-profile-production-proof.md). +Older provider successes from before transport hardening do not prove current +sandbox-contained execution. -- The versioned Glas execution contract and profile/rein catalogs. -- Exact profile selection and refusal; no implicit governed rein or model. -- Outer execution lifecycle and teardown coordination with sand-boxer. -- Reachability-derived command transport and workspace selection. -- Rein adapters that translate the stable outer contract to backend CLIs. -- Actor attribution and organizational reference propagation without assuming - workforce authority. -- Direct-caller results and compact, value-safe execution evidence. -- Channel interfaces that translate invocations into the same execution - request; only the CLI implementation exists today. +## Committed profile readiness -## Out of scope +| Profile | Rein | Readiness | +|---|---|---| +| `harness.agent-dev-local@1.0.0` | rein-aharness / Claude Code | `blocked` | +| `harness.agent-dev-openweights-local@1.0.0` | rein-openweights / OpenRouter | `blocked` | +| `harness.agent-dev@1.0.0` | rein-aharness / remote agent-dev | `unverified` | -- Sandbox profiles, placement, provisioning, owner-mediated execution, and - teardown implementation — `sand-boxer`. -- A rein's internal agent loop, provider client, tool enforcement, and - credential acquisition — the selected rein. -- Scheduling, claiming, retries, and task sourcing — `activity-core` or another - channel owner. -- Roles, duties, assignments, goals, resource envelopes, and authorization — - their workforce/leadership systems. -- Model-provider authority and inference routing — providers and `llm-connect`. -- Secret storage or credential brokering. -- End-to-end validation and product code generation — `wise-validator` and - `snuggle-inventor`. -- State Hub work-record authority. +The first local Claude acceptance is tracked by `GLAS-WP-0012`. Authorization +contract resolution and policy publication have progressed upstream, but real +owner verification, service deployment/identity, lane activation, runtime +pinning and combined acceptance remain gates. No readiness promotion follows +from those partial proofs. -## Intended but not implemented +## Ownership boundaries -The broader vision in `INTENT.md` includes long-lived/resumable sessions, -gateway-owned tool catalogs and policy checks, memory and skills services, -additional chat/email/MCP channels, cron surfaces, bounded subagent delegation, -and richer session/tool audit streams. None is a current repository capability. -The present implementation is a bounded execution router over two reins, not -yet Coulomb's general-purpose agent harness service. +Glas owns the execution contract, catalogs, outer lifecycle, adapters, +transport selection, channel translation and normalized evidence. -## Active gaps +Other owners retain: -- `GLAS-WP-0012` / `GLAS-IN-0002`: prove the first local rein runtime, - governed credential delivery, and explicit egress through the completed - owner-execution adapter. The open-weight portion remains a separate live - requirement in `GLAS-IN-0002`. See `docs/local-profile-acceptance.md`. -- `GLAS-IN-0003`: grandfather the pre-canon `GLAS-0001` identifiers. -- `GLAS-IN-0004`: align the documented ad hoc workplan convention with the - identifier canon. +- Sandbox provisioning, placement, runtime environment, exec implementation, + network enforcement and destruction: sand-boxer. +- Inner agent loop, provider client and tool enforcement: the selected rein; + inference routing where applicable: llm-connect/provider owners. +- Secret custody and governed credential delivery: OpenBao, secrets-engine and + the sandbox owner; identity/authorization: their owning services. +- Scheduling, worker allocation and work initiation: activity-core and + workforce/leadership systems. Glas only carries their references. +- Work-record authority, tunnels, certificates and production hosting: their + existing owners, not this library. +- Product validation workflows and code-generation pipelines: wise-validator + and snuggle-inventor. Repository acceptance scripts test Glas integration; + they do not replace those services. -## Getting oriented +## Intent gaps and live work -- Intent and gap assessment: `INTENT.md`, - `history/2026-08-23-scope-vs-intent-assessment.md` -- Contract: `docs/harness-contract.md`, `docs/execution-profiles.md` -- Architecture: `docs/adr/ADR-001-rein-harness-family.md` -- Live work: `workplans/`, `docs/intakes/residuals.md` -- Agent workflow: `AGENTS.md` +Long-lived/resumable sessions, memory/skills services, bounded subagent +spawn/join, additional channels, general tool-backend plugins, per-tool gateway +policy/elevation and durable session/tool audit remain unimplemented. They are +vision gaps, not implicitly approved workplans. The Channel and Rein interfaces +are an extension foundation, not a complete plugin platform. + +Live operational work remains in `GLAS-WP-0012` and `GLAS-IN-0002`; the latter +also retains the independent open-weight acceptance requirement. Identifier +intakes `GLAS-IN-0003` and `GLAS-IN-0004` remain open in +[the residual register](docs/intakes/residuals.md); their historical statements +need owner reconciliation, not an assumption that every old warning still +occurs. The ad hoc ID-registration issue was encountered again in the latest +review and resolved for that task, not for the fleet convention. + +See [INTENT.md](INTENT.md) for direction, the +[contract](docs/harness-contract.md) and +[profile documentation](docs/execution-profiles.md) for interfaces, and +[workplans](workplans/) for accepted work. diff --git a/history/2026-09-06-102533-scope-vs-intent-assessment.md b/history/2026-09-06-102533-scope-vs-intent-assessment.md new file mode 100644 index 0000000..6c33e7c --- /dev/null +++ b/history/2026-09-06-102533-scope-vs-intent-assessment.md @@ -0,0 +1,136 @@ +# Scope versus intent assessment + +Timestamp: **2026-09-06 10:25:33 CEST (UTC+02:00)**. +Baseline: Glas `5867842`; INTENT.md dated 2026-08-20. + +## Assessment + +Glas implements the nucleus that INTENT.md explicitly identifies: a versioned, +profile-driven execution router over concrete reins. It supplies a useful +library/CLI boundary, strict catalog selection, sandbox lifecycle coordination +and compact results. It does not yet satisfy the larger promise of Coulomb's +default unified agent runtime. + +The most immediate gap is operational acceptance: no committed profile is +ready, and no post-hardening real local model run has satisfied the full +acceptance contract. The broader feature gaps remain substantial but should +not distract from closing that first usable execution path. + +## Evidence and method + +Reviewed SCOPE.md and INTENT.md against: + +- `src/glas_harness/{contract,profiles,gateway,transport,hub,cli}.py`, both rein + adapters and the channel interface/CLI implementation; +- all three committed profiles and both rein registry descriptors; +- the test suite and the latest review result: 110 passing tests, catalog + validation passing at code commit `863fab7`, with final record sync at + `5867842`; no tests or live credential/model runs were repeated for this + documentation-only assessment; +- `scripts/prove-owner-boundary.py`, `inspect-local-runtime.py` and + `prove-local-profile.py`, plus recorded boundary, egress and credential + transport evidence in `docs/evidence/`; +- GLAS-WP-0012, the residual register and the prior 2026-08-23 assessment; +- recorded owner updates at secrets-engine `7b4b9e3` / `083bee7` and + FLEX-WP-0021; these show implementation/publication, not deployment; and +- `.forgejo/workflows/ci-smoke.yaml` workflow intent (package installation, tests, + catalog validation). No fresh remote CI receipt was checked in this review. + +Source behavior, historical proof and operational readiness are treated as +separate evidence levels. Owner mechanisms are not counted as Glas-owned +implementations. This is an assessment snapshot, not a live backlog authority. + +## Alignment with INTENT.md + +| Intent area | Assessment | Remaining gap / boundary | +|---|---|---| +| Unified harness API | Partial | Versioned request and synchronous single-run lifecycle exist. No resume, persistent session store, turn API or recovery service. | +| Profile catalog | Strong within the nucleus | Explicit constellation, compatibility and readiness checks work. Runtime artifact provenance is not fully encoded by the Glas schema; readiness still needs host-scoped evidence. | +| Sandbox consumption | Implemented boundary, partial operational acceptance | Local owner execution and remote SSH are implemented. Local non-secret proofs pass; real local rein acceptance and positive remote proof remain outstanding. | +| Tool orchestration and policy | Delegated/minimal | One opaque `run_task` invokes a rein. Tool policy is passed to it. No general Glas tool catalog, per-tool authorization or configurable elevated escape path. | +| Limits | Partial | Per-command timeout and backend token/turn parameters exist. No aggregate execution deadline, shared resource accounting or subagent budget enforcement. | +| Memory, skills and identity documents | Absent as Glas services | No store, synchronization, retrieval or lifecycle API. A rein's own features do not establish shared Glas capability. | +| Channels and extension platform | Foundation only | Channel/Rein contracts and CLI exist. No chat/email/MCP bridge, generic tool-backend or memory plugin contract, discovery/lifecycle service or hosted gateway. | +| Subagents | Absent | No spawn/join, bounded context, parent-child identity or delegated sandbox lifecycle contract. | +| Observability | Partial | Run evidence, attribution and optional captured Claude events exist. No live cross-rein event stream, durable audit outbox or guaranteed reporting. Open-weight tool events remain unavailable. | +| Workforce separation | Aligned | Opaque organizational references are preserved; scheduling/allocation remain upstream. Actor enum validation does not authenticate an initiator or authorize an assignment. | +| Sibling ownership | Aligned in code | Provisioning, secrets, scheduling, product validation and generation are not embedded. Integration tests do not turn Glas into wise-validator. | +| Default production agent runtime | Not achieved | No ready profile, production service surface or proven workstation-independent consumer deployment. | + +## Progress since the previous assessment + +The August assessment's missing owner-execution path is no longer current. +Glas replaced direct namespace entry and host task writes with sand-boxer exec; +non-secret source isolation, identity, task cleanup and teardown passed. +Sand-boxer separately proved runtime mounting/private state, controlled egress +and synthetic credential delivery. The real-rein acceptance runner now checks +committed/worktree artifact content before teardown. + +Local code reviews also fixed shared adapter result state, normalized manager +startup failures and rejected duplicate YAML keys rather than silently +accepting overwritten routes or limits. These improve the nucleus without +adding long-lived harness features or proving a provider workload. + +Credential custody is confirmed only through metadata. Gate-house settled the +approval-claim/decision split, secrets-engine implemented it, and flex-auth +published the lifecycle policy. Real decision verification, deployment, +production service identity and lane activation remain distinct prerequisites. + +## Gaps and disposition + +1. **First usable local profile — highest operational priority.** Continue + GLAS-WP-0012 with SAND-WP-0015 and SECRETS-WP-0009 owner returns. Require + pinned Claude startup, real protected delivery, provider scope/budget inputs, + combined artifact/identity/cleanup acceptance and a versioned profile review. + Do not promote either blocked profile based on synthetic proofs. +2. **Other execution routes — independently unproven.** GLAS-IN-0002 retains + open-weight acceptance. The remote profile remains unverified; its lack of a + current positive proof must remain visible and must not be treated as a + ready fallback. +3. **Broader harness features — consumer-backed design needed.** Memory, + skills, persistent sessions, subagents, additional channels and generic tool + plugins are INTENT deltas, not defects in the bounded router contract. + Establish specific consumers and acceptance criteria before new workplans. +4. **Operational governance and audit — partial.** External authorization is + not supplied by Glas attribution. Durable audit delivery, complete tool + visibility and aggregate limits need explicit requirements if consumers + depend on them; existing progress reporting cannot imply those guarantees. +5. **Record conventions — existing owner work.** GLAS-IN-0003/0004 remain + recorded as open. The latest ad hoc task needed a registration workaround; + successful synchronization of that task does not resolve the convention. + Legacy-bootstrap intake wording is historical and needs owner reconciliation + against the current migrated workplan before claiming a current defect. + +No new implementation workplan is created by this assessment. Existing +operational blockers retain their live records; vision gaps remain proposals. + +## INTENT.md details that need reconciliation + +INTENT is intentionally aspirational, but some integration details are stale +or ambiguous even as a target: + +- The sandbox example describes an HTTP create call and consumer fields + `harness/session_id/actor`. Current Glas uses the Python manager with + `actor/project/run_id`, then owner exec locally or SSH remotely. It does not + implement generic exec/read/write/edit tool APIs or recreate transitions. +- The direct-exec integration question is answered for the current nucleus: + local commands use owner exec; remote commands use SSH. Mirror versus + remote-canonical policy is not exposed as a general Glas tool API. +- Cron appears among harness concerns while scheduling is explicitly excluded. + A future invocation adapter can consume scheduled work; ownership of when + work runs must stay upstream. +- Assigning model API keys broadly to llm-connect does not describe the current + native workload lane: OpenBao custody, secrets-engine delivery and owner + execution participate, and rein-openweights has its own provider client. +- "Observable by default" is an aspiration beyond best-effort progress posts; + complete audit delivery and per-tool control are not present guarantees. + +INTENT.md is left unchanged: narrowing or revising its target is a separate +product decision. SCOPE.md now states the implemented behavior and limits. + +## Result + +SCOPE.md now distinguishes implemented router behavior, owner proof evidence, +profile readiness, intended features and external ownership. The appropriate +capability claim is a bounded execution-router library/CLI with hardened +sandbox integration and pending real-workload acceptance.