docs: assess current capabilities against harness intent
All checks were successful
ci / validate (push) Successful in 2m32s
All checks were successful
ci / validate (push) Successful in 2m32s
Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a0726e-5232-73f2-aaca-2c05ceb62efb
This commit is contained in:
parent
5867842b94
commit
f8942c42e8
2 changed files with 252 additions and 95 deletions
136
history/2026-09-06-102533-scope-vs-intent-assessment.md
Normal file
136
history/2026-09-06-102533-scope-vs-intent-assessment.md
Normal file
|
|
@ -0,0 +1,136 @@
|
|||
# Scope versus intent assessment
|
||||
|
||||
Timestamp: **2026-09-06 10:25:33 CEST (UTC+02:00)**.
|
||||
Baseline: Glas `5867842`; INTENT.md dated 2026-08-20.
|
||||
|
||||
## Assessment
|
||||
|
||||
Glas implements the nucleus that INTENT.md explicitly identifies: a versioned,
|
||||
profile-driven execution router over concrete reins. It supplies a useful
|
||||
library/CLI boundary, strict catalog selection, sandbox lifecycle coordination
|
||||
and compact results. It does not yet satisfy the larger promise of Coulomb's
|
||||
default unified agent runtime.
|
||||
|
||||
The most immediate gap is operational acceptance: no committed profile is
|
||||
ready, and no post-hardening real local model run has satisfied the full
|
||||
acceptance contract. The broader feature gaps remain substantial but should
|
||||
not distract from closing that first usable execution path.
|
||||
|
||||
## Evidence and method
|
||||
|
||||
Reviewed SCOPE.md and INTENT.md against:
|
||||
|
||||
- `src/glas_harness/{contract,profiles,gateway,transport,hub,cli}.py`, both rein
|
||||
adapters and the channel interface/CLI implementation;
|
||||
- all three committed profiles and both rein registry descriptors;
|
||||
- the test suite and the latest review result: 110 passing tests, catalog
|
||||
validation passing at code commit `863fab7`, with final record sync at
|
||||
`5867842`; no tests or live credential/model runs were repeated for this
|
||||
documentation-only assessment;
|
||||
- `scripts/prove-owner-boundary.py`, `inspect-local-runtime.py` and
|
||||
`prove-local-profile.py`, plus recorded boundary, egress and credential
|
||||
transport evidence in `docs/evidence/`;
|
||||
- GLAS-WP-0012, the residual register and the prior 2026-08-23 assessment;
|
||||
- recorded owner updates at secrets-engine `7b4b9e3` / `083bee7` and
|
||||
FLEX-WP-0021; these show implementation/publication, not deployment; and
|
||||
- `.forgejo/workflows/ci-smoke.yaml` workflow intent (package installation, tests,
|
||||
catalog validation). No fresh remote CI receipt was checked in this review.
|
||||
|
||||
Source behavior, historical proof and operational readiness are treated as
|
||||
separate evidence levels. Owner mechanisms are not counted as Glas-owned
|
||||
implementations. This is an assessment snapshot, not a live backlog authority.
|
||||
|
||||
## Alignment with INTENT.md
|
||||
|
||||
| Intent area | Assessment | Remaining gap / boundary |
|
||||
|---|---|---|
|
||||
| Unified harness API | Partial | Versioned request and synchronous single-run lifecycle exist. No resume, persistent session store, turn API or recovery service. |
|
||||
| Profile catalog | Strong within the nucleus | Explicit constellation, compatibility and readiness checks work. Runtime artifact provenance is not fully encoded by the Glas schema; readiness still needs host-scoped evidence. |
|
||||
| Sandbox consumption | Implemented boundary, partial operational acceptance | Local owner execution and remote SSH are implemented. Local non-secret proofs pass; real local rein acceptance and positive remote proof remain outstanding. |
|
||||
| Tool orchestration and policy | Delegated/minimal | One opaque `run_task` invokes a rein. Tool policy is passed to it. No general Glas tool catalog, per-tool authorization or configurable elevated escape path. |
|
||||
| Limits | Partial | Per-command timeout and backend token/turn parameters exist. No aggregate execution deadline, shared resource accounting or subagent budget enforcement. |
|
||||
| Memory, skills and identity documents | Absent as Glas services | No store, synchronization, retrieval or lifecycle API. A rein's own features do not establish shared Glas capability. |
|
||||
| Channels and extension platform | Foundation only | Channel/Rein contracts and CLI exist. No chat/email/MCP bridge, generic tool-backend or memory plugin contract, discovery/lifecycle service or hosted gateway. |
|
||||
| Subagents | Absent | No spawn/join, bounded context, parent-child identity or delegated sandbox lifecycle contract. |
|
||||
| Observability | Partial | Run evidence, attribution and optional captured Claude events exist. No live cross-rein event stream, durable audit outbox or guaranteed reporting. Open-weight tool events remain unavailable. |
|
||||
| Workforce separation | Aligned | Opaque organizational references are preserved; scheduling/allocation remain upstream. Actor enum validation does not authenticate an initiator or authorize an assignment. |
|
||||
| Sibling ownership | Aligned in code | Provisioning, secrets, scheduling, product validation and generation are not embedded. Integration tests do not turn Glas into wise-validator. |
|
||||
| Default production agent runtime | Not achieved | No ready profile, production service surface or proven workstation-independent consumer deployment. |
|
||||
|
||||
## Progress since the previous assessment
|
||||
|
||||
The August assessment's missing owner-execution path is no longer current.
|
||||
Glas replaced direct namespace entry and host task writes with sand-boxer exec;
|
||||
non-secret source isolation, identity, task cleanup and teardown passed.
|
||||
Sand-boxer separately proved runtime mounting/private state, controlled egress
|
||||
and synthetic credential delivery. The real-rein acceptance runner now checks
|
||||
committed/worktree artifact content before teardown.
|
||||
|
||||
Local code reviews also fixed shared adapter result state, normalized manager
|
||||
startup failures and rejected duplicate YAML keys rather than silently
|
||||
accepting overwritten routes or limits. These improve the nucleus without
|
||||
adding long-lived harness features or proving a provider workload.
|
||||
|
||||
Credential custody is confirmed only through metadata. Gate-house settled the
|
||||
approval-claim/decision split, secrets-engine implemented it, and flex-auth
|
||||
published the lifecycle policy. Real decision verification, deployment,
|
||||
production service identity and lane activation remain distinct prerequisites.
|
||||
|
||||
## Gaps and disposition
|
||||
|
||||
1. **First usable local profile — highest operational priority.** Continue
|
||||
GLAS-WP-0012 with SAND-WP-0015 and SECRETS-WP-0009 owner returns. Require
|
||||
pinned Claude startup, real protected delivery, provider scope/budget inputs,
|
||||
combined artifact/identity/cleanup acceptance and a versioned profile review.
|
||||
Do not promote either blocked profile based on synthetic proofs.
|
||||
2. **Other execution routes — independently unproven.** GLAS-IN-0002 retains
|
||||
open-weight acceptance. The remote profile remains unverified; its lack of a
|
||||
current positive proof must remain visible and must not be treated as a
|
||||
ready fallback.
|
||||
3. **Broader harness features — consumer-backed design needed.** Memory,
|
||||
skills, persistent sessions, subagents, additional channels and generic tool
|
||||
plugins are INTENT deltas, not defects in the bounded router contract.
|
||||
Establish specific consumers and acceptance criteria before new workplans.
|
||||
4. **Operational governance and audit — partial.** External authorization is
|
||||
not supplied by Glas attribution. Durable audit delivery, complete tool
|
||||
visibility and aggregate limits need explicit requirements if consumers
|
||||
depend on them; existing progress reporting cannot imply those guarantees.
|
||||
5. **Record conventions — existing owner work.** GLAS-IN-0003/0004 remain
|
||||
recorded as open. The latest ad hoc task needed a registration workaround;
|
||||
successful synchronization of that task does not resolve the convention.
|
||||
Legacy-bootstrap intake wording is historical and needs owner reconciliation
|
||||
against the current migrated workplan before claiming a current defect.
|
||||
|
||||
No new implementation workplan is created by this assessment. Existing
|
||||
operational blockers retain their live records; vision gaps remain proposals.
|
||||
|
||||
## INTENT.md details that need reconciliation
|
||||
|
||||
INTENT is intentionally aspirational, but some integration details are stale
|
||||
or ambiguous even as a target:
|
||||
|
||||
- The sandbox example describes an HTTP create call and consumer fields
|
||||
`harness/session_id/actor`. Current Glas uses the Python manager with
|
||||
`actor/project/run_id`, then owner exec locally or SSH remotely. It does not
|
||||
implement generic exec/read/write/edit tool APIs or recreate transitions.
|
||||
- The direct-exec integration question is answered for the current nucleus:
|
||||
local commands use owner exec; remote commands use SSH. Mirror versus
|
||||
remote-canonical policy is not exposed as a general Glas tool API.
|
||||
- Cron appears among harness concerns while scheduling is explicitly excluded.
|
||||
A future invocation adapter can consume scheduled work; ownership of when
|
||||
work runs must stay upstream.
|
||||
- Assigning model API keys broadly to llm-connect does not describe the current
|
||||
native workload lane: OpenBao custody, secrets-engine delivery and owner
|
||||
execution participate, and rein-openweights has its own provider client.
|
||||
- "Observable by default" is an aspiration beyond best-effort progress posts;
|
||||
complete audit delivery and per-tool control are not present guarantees.
|
||||
|
||||
INTENT.md is left unchanged: narrowing or revising its target is a separate
|
||||
product decision. SCOPE.md now states the implemented behavior and limits.
|
||||
|
||||
## Result
|
||||
|
||||
SCOPE.md now distinguishes implemented router behavior, owner proof evidence,
|
||||
profile readiness, intended features and external ownership. The appropriate
|
||||
capability claim is a bounded execution-router library/CLI with hardened
|
||||
sandbox integration and pending real-workload acceptance.
|
||||
Loading…
Add table
Add a link
Reference in a new issue