Assistant: codex Assistant-Model: gpt-5.6-sol Assistant-Session: 01a0233b-178d-7162-b92f-31a31ea8ca9b
This commit is contained in:
parent
a224e71555
commit
cac605abd7
19 changed files with 339 additions and 41 deletions
18
README.md
18
README.md
|
|
@ -17,7 +17,8 @@ assignment / channel / activity
|
|||
|
||||
## Profiles
|
||||
|
||||
List and validate every committed executable profile:
|
||||
List and validate every committed catalog profile, including its independently
|
||||
declared runtime readiness:
|
||||
|
||||
```bash
|
||||
glas-harness profiles
|
||||
|
|
@ -32,19 +33,26 @@ Current constellations include:
|
|||
open-weight route, local bwrap sandbox.
|
||||
- `harness.agent-dev@1.0.0`: `rein-aharness`, remote agent-dev sandbox.
|
||||
|
||||
Run a task through an explicit profile:
|
||||
The two local profiles are currently `blocked` by the missing sand-boxer
|
||||
owner-execution/runtime path. The remote profile is `unverified`: it may be
|
||||
used only as an explicitly labeled proof attempt. No committed profile is
|
||||
currently declared `ready`.
|
||||
|
||||
Start a bounded proof attempt through the currently unverified remote profile:
|
||||
|
||||
```bash
|
||||
glas-harness run \
|
||||
--harness-profile harness.agent-dev-local@1.0.0 \
|
||||
--harness-profile harness.agent-dev@1.0.0 \
|
||||
--repo /absolute/path/to/repo \
|
||||
--title "Bounded change" \
|
||||
--description "Make the requested change and commit it"
|
||||
```
|
||||
|
||||
There is no governed default rein, model, sandbox, or tool profile. Unknown,
|
||||
disabled, incompatible, and ambiguous selections are refused before sandbox
|
||||
creation.
|
||||
disabled, incompatible, ambiguous, and operationally blocked selections are
|
||||
refused before sandbox creation. An unverified selection is allowed so a
|
||||
bounded proof can establish or disprove readiness, and its evidence retains
|
||||
that state.
|
||||
|
||||
After creation, execution uses only the reachability-derived sandbox workspace:
|
||||
same-host namespaces are entered with `nsenter`, while remote workspaces are
|
||||
|
|
|
|||
20
SCOPE.md
20
SCOPE.md
|
|
@ -28,7 +28,8 @@ The repository currently implements:
|
|||
creation, keeping queue `worker_id` values outside the actor field;
|
||||
- one invocation channel: the local `glas-harness run` CLI;
|
||||
- a packaged, versioned profile/rein catalog and `glas-harness profiles`
|
||||
validation command; and
|
||||
validation command that reports catalog enablement separately from typed
|
||||
operational readiness; and
|
||||
- compact State Hub progress evidence that excludes prompts, model/tool output,
|
||||
provider bodies, and credentials.
|
||||
|
||||
|
|
@ -40,20 +41,21 @@ installs the package, runs the suite, and validates the packaged catalog.
|
|||
| Surface | Current verified status |
|
||||
|---|---|
|
||||
| Profile/catalog resolution | Implemented and CI-validated |
|
||||
| Unknown or invalid profile refusal | Implemented before sandbox creation |
|
||||
| Unknown, invalid, or known-blocked profile refusal | Implemented before sandbox creation |
|
||||
| Sandbox create/destroy lifecycle | Implemented; live failure paths prove teardown |
|
||||
| Original-checkout isolation | Enforced by code: it is never a post-create execution fallback |
|
||||
| SSH transport construction | Implemented and unit-tested; no current positive post-hardening live proof |
|
||||
| Same-host bwrap execution | Fail-closed, not operational: consumer `nsenter` is denied and the rein runtime is absent inside the sandbox |
|
||||
| `rein-aharness` adapter | Implemented; current local profile reaches sandbox creation and fails at session start on the bwrap owner boundary |
|
||||
| Same-host bwrap execution | Fail-closed, not operational: both local profiles are declared `blocked` and refused before creation; the underlying proof found consumer `nsenter` denied and no in-sandbox rein runtime |
|
||||
| `rein-aharness` adapter | Implemented; local selection is blocked before creation, while the remote profile remains an explicitly labeled unverified proof path |
|
||||
| `rein-openweights` adapter | Implemented; provider credential repair was proven, but current local sandbox execution has the same unresolved owner boundary |
|
||||
| CLI channel | Implemented |
|
||||
| State Hub evidence | Implemented as compact progress evidence, not a complete session/tool audit service |
|
||||
|
||||
An `enabled` profile currently means that catalog policy permits selection. It
|
||||
does not prove that its rein executable, credential delivery, network egress,
|
||||
and sandbox transport are operational on a particular host. That semantics is
|
||||
an open governance gap tracked in `GLAS-IN-0005`.
|
||||
An `enabled` profile means only that catalog policy permits selection. Typed
|
||||
operational readiness separately records `ready`, `unverified`, or `blocked`.
|
||||
Known-blocked profiles refuse before sandbox creation; unverified profiles may
|
||||
run only as labeled proof attempts, with the state preserved in evidence.
|
||||
There is currently no committed `ready` profile.
|
||||
|
||||
## In scope
|
||||
|
||||
|
|
@ -101,8 +103,6 @@ yet Coulomb's general-purpose agent harness service.
|
|||
- `GLAS-IN-0003`: grandfather the pre-canon `GLAS-0001` identifiers.
|
||||
- `GLAS-IN-0004`: align the documented ad hoc workplan convention with the
|
||||
identifier canon.
|
||||
- `GLAS-IN-0005`: make profile `enabled` status and operational readiness
|
||||
truthful and testable.
|
||||
|
||||
## Getting oriented
|
||||
|
||||
|
|
|
|||
|
|
@ -16,6 +16,7 @@
|
|||
| workplan | GLAS-WP-0005 | blocked | — | workplans/GLAS-WP-0005-sandbox-boundary-runtime-hardening.md |
|
||||
| workplan | GLAS-WP-0006 | finished | — | workplans/GLAS-WP-0006-consumer-actor-validation.md |
|
||||
| workplan | GLAS-WP-0007 | finished | — | workplans/GLAS-WP-0007-capability-scope-truth.md |
|
||||
| workplan | GLAS-WP-0008 | finished | — | workplans/GLAS-WP-0008-profile-operational-readiness.md |
|
||||
| task | GLAS-0001-T01 | done | — | workplans/GLAS-0001-statehub-bootstrap.md |
|
||||
| task | GLAS-0001-T02 | done | — | workplans/GLAS-0001-statehub-bootstrap.md |
|
||||
| task | GLAS-0001-T03 | done | — | workplans/GLAS-0001-statehub-bootstrap.md |
|
||||
|
|
@ -49,8 +50,11 @@
|
|||
| task | GLAS-WP-0006-T01 | done | — | workplans/GLAS-WP-0006-consumer-actor-validation.md |
|
||||
| task | GLAS-WP-0007-T01 | done | — | workplans/GLAS-WP-0007-capability-scope-truth.md |
|
||||
| task | GLAS-WP-0007-T02 | done | — | workplans/GLAS-WP-0007-capability-scope-truth.md |
|
||||
| task | GLAS-WP-0008-T01 | done | — | workplans/GLAS-WP-0008-profile-operational-readiness.md |
|
||||
| task | GLAS-WP-0008-T02 | done | — | workplans/GLAS-WP-0008-profile-operational-readiness.md |
|
||||
| task | GLAS-WP-0008-T03 | done | — | workplans/GLAS-WP-0008-profile-operational-readiness.md |
|
||||
| intake | GLAS-IN-0001 | done | blue | docs/intakes/residuals.md |
|
||||
| intake | GLAS-IN-0002 | todo | red | docs/intakes/residuals.md |
|
||||
| intake | GLAS-IN-0003 | todo | green | docs/intakes/residuals.md |
|
||||
| intake | GLAS-IN-0004 | todo | green | docs/intakes/residuals.md |
|
||||
| intake | GLAS-IN-0005 | todo | red | docs/intakes/residuals.md |
|
||||
| intake | GLAS-IN-0005 | done | red | docs/intakes/residuals.md |
|
||||
|
|
|
|||
|
|
@ -20,7 +20,10 @@ ExecutionEvidence -> assignment/audit consumers
|
|||
|
||||
Each YAML document under `profiles/` declares:
|
||||
|
||||
- stable `id`, semantic `version`, exact `contract_version`, and status;
|
||||
- stable `id`, semantic `version`, exact `contract_version`, and catalog
|
||||
enablement status;
|
||||
- typed operational readiness (`ready`, `unverified`, or `blocked`) with its
|
||||
reason, owner, and evidence reference;
|
||||
- rein registry ID and required capabilities;
|
||||
- sand-boxer `sandbox_profile`;
|
||||
- rein-enforced `tool_profile`;
|
||||
|
|
@ -35,6 +38,20 @@ enabled revision exists. Inline secrets and token-looking values are refused.
|
|||
|
||||
Use `glas-harness profiles` as the catalog/packaging validation command.
|
||||
|
||||
Catalog enablement and operational readiness answer different questions:
|
||||
|
||||
- `status: enabled` permits deterministic selection after schema and rein
|
||||
compatibility checks;
|
||||
- `operational_readiness.status: ready` requires a positive `evidence_ref`;
|
||||
- `unverified` permits a bounded proof attempt but is carried into execution
|
||||
evidence and must not be described as ready; and
|
||||
- `blocked` requires an owner plus evidence reference and is refused during
|
||||
resolution, before sandbox creation.
|
||||
|
||||
Profiles that predate the field default conservatively to `unverified`. The
|
||||
catalog command lists all enabled profiles and labels readiness; it does not
|
||||
filter blocked profiles or call them executable.
|
||||
|
||||
Catalog validation proves schema/compatibility, not transport installation. An
|
||||
execution host must install `glas-harness` with sand-boxer support and make each
|
||||
selected rein plus its dependencies available *inside* the sandbox transport,
|
||||
|
|
@ -44,6 +61,11 @@ also needs its documented `rein-aharness[llm]`/sibling `llm-connect` adapter
|
|||
dependency in that environment. Missing runtime dependencies fail closed at
|
||||
session start and still trigger sandbox teardown.
|
||||
|
||||
The committed local profiles are currently declared `blocked` by
|
||||
`GLAS-IN-0002`. The remote profile is `unverified` because there is no positive
|
||||
post-hardening execution proof. There is currently no committed `ready`
|
||||
profile.
|
||||
|
||||
## Consumer request
|
||||
|
||||
See `examples/execution-request.json`. Consumers know only a profile reference,
|
||||
|
|
@ -59,7 +81,7 @@ The current exact-profile path does not interpret resource envelopes or choose
|
|||
profiles from requirements. A future requirements resolver belongs at this
|
||||
Glas boundary, must be deterministic and explainable, and must refuse ambiguity.
|
||||
|
||||
## Two equivalent selections
|
||||
## Two parallel local selections
|
||||
|
||||
For the same task and organizational references, switch only:
|
||||
|
||||
|
|
@ -71,9 +93,11 @@ harness.agent-dev-openweights-local@1.0.0
|
|||
-> rein-openweights / qwen/qwen-2.5-72b-instruct / profile.bwrap-local
|
||||
```
|
||||
|
||||
Both use `green-commit-only` and return `GatewayResult` / `ExecutionEvidence`.
|
||||
This proves interchangeability at the Glas boundary; it does not claim the
|
||||
models have identical quality, price, or provider behavior.
|
||||
Both declare `green-commit-only` and the same outer `GatewayResult` /
|
||||
`ExecutionEvidence` contract. They are interchangeable as catalog shapes, but
|
||||
both are currently operationally blocked and refused before sandbox creation.
|
||||
This does not claim current runtime availability or identical model quality,
|
||||
price, or provider behavior.
|
||||
|
||||
## Ownership handoff
|
||||
|
||||
|
|
|
|||
|
|
@ -32,6 +32,13 @@ The gateway returns `GatewayResult` with:
|
|||
Raw prompts, model output, tool output, and credential material are excluded
|
||||
from the State Hub detail.
|
||||
|
||||
`HarnessProfile.status` governs catalog selection. The separate
|
||||
`OperationalReadiness` model records runtime status, a reason, its owner, and
|
||||
an evidence reference. `ready` requires positive evidence; `blocked` requires
|
||||
an owner and evidence and is refused during resolution before sandbox
|
||||
creation; `unverified` permits only a labeled proof attempt. The selected state
|
||||
is copied into `ResolvedExecutionContext` and `ExecutionEvidence`.
|
||||
|
||||
## Rein lifecycle
|
||||
|
||||
```python
|
||||
|
|
@ -99,7 +106,8 @@ remain rein-local under ADR-002.
|
|||
|
||||
The gateway returns evidence for every normal refusal/failure path:
|
||||
|
||||
- `resolution`: unknown, disabled, ambiguous, incompatible, or unsafe profile;
|
||||
- `resolution`: unknown, disabled, ambiguous, incompatible, operationally
|
||||
blocked, or unsafe profile;
|
||||
- `sandbox_create`;
|
||||
- `session_start`;
|
||||
- `execution` (including a rein-declared unsuccessful result);
|
||||
|
|
|
|||
|
|
@ -147,7 +147,7 @@ id: GLAS-IN-0005
|
|||
kind: intake
|
||||
title: "Separate profile selection enablement from runtime readiness"
|
||||
lane: red
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
owner: glas-harness
|
||||
repo: glas-harness
|
||||
|
|
@ -168,3 +168,9 @@ description: |
|
|||
fallback or claim readiness from lifecycle-only proof.
|
||||
state_hub_intake_id: "01a02dce-b761-75e6-861a-8abf8b8e0072"
|
||||
```
|
||||
|
||||
Completed by `GLAS-WP-0008` on 2026-08-23. Catalog enablement is now separate
|
||||
from typed operational readiness. Both local profiles are `blocked` by
|
||||
`GLAS-IN-0002` and refuse before sandbox creation; the remote profile is
|
||||
`unverified` and may be used only for a labeled bounded proof attempt. Catalog
|
||||
and execution evidence expose the state, and no host fallback was added.
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
{
|
||||
"harness_profile_ref": "harness.agent-dev-local@1.0.0",
|
||||
"harness_profile_ref": "harness.agent-dev@1.0.0",
|
||||
"repo": "/absolute/path/to/target-repo",
|
||||
"title": "Implement the bounded assignment",
|
||||
"description": "Apply the approved change, run focused checks, and commit the result.",
|
||||
|
|
|
|||
|
|
@ -68,10 +68,10 @@ not a positive local execution path.
|
|||
`GLAS-IN-0002` owns the sand-boxer contract for command execution, mounted or
|
||||
imaged rein runtime, explicit model egress, credential delivery, and teardown
|
||||
evidence. `GLAS-WP-0005-T05` remains waiting on it.
|
||||
2. **Profile readiness semantics — high.** A catalog-valid `enabled` profile is
|
||||
currently presented alongside profiles that cannot dispatch on the deployed
|
||||
runtime. `GLAS-IN-0005` requires an explicit readiness/enablement contract,
|
||||
including whether unproven profiles must be disabled or separately labeled.
|
||||
2. **Profile readiness semantics — resolved 2026-08-23.** `GLAS-WP-0008`
|
||||
separated catalog enablement from typed `ready`, `unverified`, and `blocked`
|
||||
runtime states. Known blockers now refuse before sandbox creation, while a
|
||||
labeled unverified profile remains available for a bounded proof attempt.
|
||||
|
||||
### Record-governance gaps
|
||||
|
||||
|
|
@ -92,13 +92,14 @@ or claim them as capabilities.
|
|||
## Recommended sequence
|
||||
|
||||
1. Complete `GLAS-IN-0002` and capture one positive, source-isolated rein run.
|
||||
2. Resolve `GLAS-IN-0005` so catalog output distinguishes selectable schema
|
||||
from deployable runtime capability.
|
||||
2. Use the implemented readiness state in catalog output and evidence; promote
|
||||
a profile to `ready` only with a positive evidence reference.
|
||||
3. Reassess whether the remote profile is an operational recovery path or an
|
||||
unproven declaration; retain it only with evidence and owned runtime setup.
|
||||
4. Add broader harness pillars only when a real channel/consumer supplies the
|
||||
boundary and evidence requirements.
|
||||
|
||||
Until steps 1–2 complete, describe Glas as a secure, fail-closed execution
|
||||
router nucleus—not yet the default or unified agent harness service promised by
|
||||
the maturity target in `INTENT.md`.
|
||||
Until step 1 completes and at least one profile has positive runtime evidence,
|
||||
describe Glas as a secure, fail-closed execution router nucleus—not yet the
|
||||
default or unified agent harness service promised by the maturity target in
|
||||
`INTENT.md`.
|
||||
|
|
|
|||
|
|
@ -2,6 +2,11 @@ id: harness.agent-dev-local
|
|||
version: "1.0.0"
|
||||
contract_version: "1.0"
|
||||
status: enabled
|
||||
operational_readiness:
|
||||
status: blocked
|
||||
reason: sand-boxer owner-mediated bwrap execution and in-sandbox rein runtime are unavailable
|
||||
owner: sand-boxer
|
||||
evidence_ref: GLAS-IN-0002
|
||||
rein:
|
||||
id: rein-aharness
|
||||
required_capabilities:
|
||||
|
|
|
|||
|
|
@ -2,6 +2,11 @@ id: harness.agent-dev-openweights-local
|
|||
version: "1.0.0"
|
||||
contract_version: "1.0"
|
||||
status: enabled
|
||||
operational_readiness:
|
||||
status: blocked
|
||||
reason: sand-boxer owner-mediated bwrap execution and in-sandbox rein runtime are unavailable
|
||||
owner: sand-boxer
|
||||
evidence_ref: GLAS-IN-0002
|
||||
rein:
|
||||
id: rein-openweights
|
||||
required_capabilities:
|
||||
|
|
|
|||
|
|
@ -2,6 +2,11 @@ id: harness.agent-dev
|
|||
version: "1.0.0"
|
||||
contract_version: "1.0"
|
||||
status: enabled
|
||||
operational_readiness:
|
||||
status: unverified
|
||||
reason: no positive remote execution proof exists after transport hardening
|
||||
owner: glas-harness
|
||||
evidence_ref: GLAS-WP-0005
|
||||
rein:
|
||||
id: rein-aharness
|
||||
required_capabilities:
|
||||
|
|
|
|||
|
|
@ -30,7 +30,7 @@ def main(argv: list[str] | None = None) -> int:
|
|||
run.add_argument("--project", default="glas-harness")
|
||||
run.add_argument("--no-hub", action="store_true", help="Skip the gateway's own hub reporting")
|
||||
|
||||
profiles = sub.add_parser("profiles", help="Validate and list executable Glas profiles")
|
||||
profiles = sub.add_parser("profiles", help="Validate and list Glas catalog profiles")
|
||||
profiles.add_argument("--json", action="store_true", help="Emit machine-readable JSON")
|
||||
|
||||
args = parser.parse_args(argv)
|
||||
|
|
@ -76,6 +76,7 @@ def main(argv: list[str] | None = None) -> int:
|
|||
print(
|
||||
f"{row['profile']['id']}@{row['profile']['version']}\t"
|
||||
f"rein={row['rein_id']}@{row['rein_version']}\t"
|
||||
f"readiness={row['operational_readiness']['status']}\t"
|
||||
f"model={row['model']['model']}\t"
|
||||
f"sandbox={row['sandbox_profile']}"
|
||||
)
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ import re
|
|||
from abc import ABC, abstractmethod
|
||||
from typing import Any, Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
|
||||
|
||||
CONTRACT_VERSION = "1.0"
|
||||
_SEMVER = re.compile(r"^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:[-+][0-9A-Za-z.-]+)?$")
|
||||
|
|
@ -54,11 +54,31 @@ class ReinSelection(ContractModel):
|
|||
required_capabilities: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class OperationalReadiness(ContractModel):
|
||||
"""Evidence-backed runtime status, separate from catalog selection policy."""
|
||||
|
||||
status: Literal["ready", "unverified", "blocked"] = "unverified"
|
||||
reason: str = Field(default="no operational proof declared", min_length=1)
|
||||
owner: str | None = Field(default=None, min_length=1)
|
||||
evidence_ref: str | None = Field(default=None, min_length=1)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_evidence(self) -> OperationalReadiness:
|
||||
if self.status == "ready" and self.evidence_ref is None:
|
||||
raise ValueError("ready operational status requires evidence_ref")
|
||||
if self.status == "blocked" and (self.owner is None or self.evidence_ref is None):
|
||||
raise ValueError("blocked operational status requires owner and evidence_ref")
|
||||
return self
|
||||
|
||||
|
||||
class HarnessProfile(ContractModel):
|
||||
id: str = Field(pattern=r"^harness\.[a-z0-9][a-z0-9._-]*$")
|
||||
version: str
|
||||
contract_version: str
|
||||
status: Literal["enabled", "disabled"] = "enabled"
|
||||
operational_readiness: OperationalReadiness = Field(
|
||||
default_factory=OperationalReadiness
|
||||
)
|
||||
rein: ReinSelection
|
||||
sandbox_profile: str = Field(pattern=r"^profile\.[a-z0-9][a-z0-9._-]*$")
|
||||
tool_profile: str = Field(min_length=1)
|
||||
|
|
@ -152,6 +172,9 @@ class ResolvedExecutionContext(ContractModel):
|
|||
profile: HarnessProfileRef
|
||||
rein_id: str
|
||||
rein_version: str
|
||||
operational_readiness: OperationalReadiness = Field(
|
||||
default_factory=OperationalReadiness
|
||||
)
|
||||
sandbox_profile: str
|
||||
tool_profile: str
|
||||
model: ModelRoute
|
||||
|
|
@ -166,6 +189,7 @@ class ExecutionEvidence(ContractModel):
|
|||
target_repo: str
|
||||
contract_version: str = CONTRACT_VERSION
|
||||
profile_ref: str | None = None
|
||||
profile_readiness: OperationalReadiness | None = None
|
||||
rein_id: str | None = None
|
||||
rein_version: str | None = None
|
||||
model_route: str | None = None
|
||||
|
|
|
|||
|
|
@ -84,6 +84,7 @@ def run_execution(
|
|||
try:
|
||||
consumer = _consumer_from_request(request)
|
||||
profile, descriptor = catalog.resolve(request.harness_profile_ref)
|
||||
catalog.require_operational(profile)
|
||||
selected_rein = rein or catalog.build_rein(profile, descriptor)
|
||||
except Exception as exc:
|
||||
outcome = "refused"
|
||||
|
|
@ -292,6 +293,7 @@ def _build_result(
|
|||
target_repo=request.repo,
|
||||
contract_version=CONTRACT_VERSION,
|
||||
profile_ref=str(profile.ref) if profile else None,
|
||||
profile_readiness=profile.operational_readiness if profile else None,
|
||||
rein_id=descriptor.id if descriptor else None,
|
||||
rein_version=descriptor.version if descriptor else None,
|
||||
model_route=profile.model.route if profile else None,
|
||||
|
|
|
|||
|
|
@ -37,6 +37,10 @@ class IncompatibleProfileError(ProfileError):
|
|||
pass
|
||||
|
||||
|
||||
class OperationallyBlockedProfileError(ProfileError):
|
||||
pass
|
||||
|
||||
|
||||
_SENSITIVE_KEY = re.compile(
|
||||
r"(^|_)(api_key|password|passwd|secret|secret_value|token_value|private_key)$",
|
||||
re.IGNORECASE,
|
||||
|
|
@ -186,12 +190,25 @@ class ProfileCatalog:
|
|||
profile=profile.ref,
|
||||
rein_id=descriptor.id,
|
||||
rein_version=descriptor.version,
|
||||
operational_readiness=profile.operational_readiness,
|
||||
sandbox_profile=profile.sandbox_profile,
|
||||
tool_profile=profile.tool_profile,
|
||||
model=profile.model,
|
||||
limits=profile.limits,
|
||||
)
|
||||
|
||||
def require_operational(self, profile: HarnessProfile) -> None:
|
||||
"""Refuse known runtime blockers while allowing labeled proof attempts."""
|
||||
|
||||
readiness = profile.operational_readiness
|
||||
if readiness.status != "blocked":
|
||||
return
|
||||
raise OperationallyBlockedProfileError(
|
||||
f"profile {profile.ref} operational readiness is blocked: "
|
||||
f"{readiness.reason} (owner={readiness.owner}; "
|
||||
f"evidence={readiness.evidence_ref})"
|
||||
)
|
||||
|
||||
def build_rein(self, profile: HarnessProfile, descriptor: ReinDescriptor) -> Rein:
|
||||
module_name, class_name = descriptor.handler.split(":", 1)
|
||||
module = importlib.import_module(module_name)
|
||||
|
|
|
|||
|
|
@ -12,6 +12,19 @@ def test_profiles_command_lists_committed_constellations(capsys) -> None:
|
|||
|
||||
rows = json.loads(capsys.readouterr().out)
|
||||
assert {row["rein_id"] for row in rows} == {"rein-aharness", "rein-openweights"}
|
||||
assert {row["operational_readiness"]["status"] for row in rows} == {
|
||||
"blocked",
|
||||
"unverified",
|
||||
}
|
||||
|
||||
|
||||
def test_profiles_text_output_labels_runtime_readiness(capsys) -> None:
|
||||
assert main(["profiles"]) == 0
|
||||
|
||||
output = capsys.readouterr().out
|
||||
assert "readiness=blocked" in output
|
||||
assert "readiness=unverified" in output
|
||||
assert "executable" not in output
|
||||
|
||||
|
||||
def test_execution_request_example_matches_contract() -> None:
|
||||
|
|
@ -19,7 +32,7 @@ def test_execution_request_example_matches_contract() -> None:
|
|||
|
||||
request = ExecutionRequest.model_validate_json(fixture.read_text())
|
||||
|
||||
assert request.harness_profile_ref == "harness.agent-dev-local@1.0.0"
|
||||
assert request.harness_profile_ref == "harness.agent-dev@1.0.0"
|
||||
assert request.assignment_ref == "role-assignment:agent-7:42"
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -6,12 +6,14 @@ from sandboxer.models import Reachability, SandboxState, SandboxStatus
|
|||
from glas_harness.contract import (
|
||||
ExecutionRequest,
|
||||
ExecutionSummary,
|
||||
OperationalReadiness,
|
||||
Rein,
|
||||
SandboxHandle,
|
||||
ToolCall,
|
||||
ToolResult,
|
||||
)
|
||||
from glas_harness.gateway import run_execution, run_task_through_rein
|
||||
from glas_harness.profiles import ProfileCatalog
|
||||
|
||||
|
||||
PROFILE = "harness.agent-dev-local@1.0.0"
|
||||
|
|
@ -83,12 +85,29 @@ def _request(*, profile: str = PROFILE, report_to_hub: bool = False) -> Executio
|
|||
)
|
||||
|
||||
|
||||
def _catalog_with_readiness(status: str = "ready") -> ProfileCatalog:
|
||||
catalog = ProfileCatalog()
|
||||
profile, _ = catalog.resolve(PROFILE)
|
||||
readiness = OperationalReadiness(
|
||||
status=status,
|
||||
reason="test-only readiness state",
|
||||
owner="tests",
|
||||
evidence_ref="test:gateway-readiness",
|
||||
)
|
||||
catalog.profiles()[(profile.id, profile.version)] = profile.model_copy(
|
||||
update={"operational_readiness": readiness}
|
||||
)
|
||||
return catalog
|
||||
|
||||
|
||||
def test_run_execution_creates_and_destroys_sandbox() -> None:
|
||||
manager = MagicMock()
|
||||
manager.create.return_value = _fake_status()
|
||||
rein = _FakeRein()
|
||||
|
||||
result = run_execution(_request(), rein=rein, manager=manager)
|
||||
result = run_execution(
|
||||
_request(), catalog=_catalog_with_readiness(), rein=rein, manager=manager
|
||||
)
|
||||
|
||||
assert rein.calls == [
|
||||
"start_session",
|
||||
|
|
@ -100,6 +119,7 @@ def test_run_execution_creates_and_destroys_sandbox() -> None:
|
|||
manager.destroy.assert_called_once_with("sbx1")
|
||||
assert result.ok is True
|
||||
assert result.evidence.profile_ref == PROFILE
|
||||
assert result.evidence.profile_readiness.status == "ready"
|
||||
assert result.evidence.rein_id == "rein-aharness"
|
||||
assert result.evidence.commit_sha == "deadbeef"
|
||||
assert result.evidence.tokens_spent == 123
|
||||
|
|
@ -116,7 +136,9 @@ def test_run_execution_normalizes_execution_failure_and_tears_down() -> None:
|
|||
raise RuntimeError("boom")
|
||||
|
||||
rein = _FailingRein()
|
||||
result = run_execution(_request(), rein=rein, manager=manager)
|
||||
result = run_execution(
|
||||
_request(), catalog=_catalog_with_readiness(), rein=rein, manager=manager
|
||||
)
|
||||
|
||||
assert result.ok is False
|
||||
assert result.evidence.outcome == "failed"
|
||||
|
|
@ -139,6 +161,35 @@ def test_run_execution_refuses_unknown_profile_before_sandbox() -> None:
|
|||
manager.create.assert_not_called()
|
||||
|
||||
|
||||
def test_run_execution_refuses_blocked_profile_before_sandbox() -> None:
|
||||
manager = MagicMock()
|
||||
|
||||
result = run_execution(_request(), rein=_FakeRein(), manager=manager)
|
||||
|
||||
assert result.ok is False
|
||||
assert result.evidence.outcome == "refused"
|
||||
assert result.evidence.failure_stage == "resolution"
|
||||
assert result.evidence.profile_readiness.status == "blocked"
|
||||
assert "GLAS-IN-0002" in (result.evidence.error or "")
|
||||
manager.create.assert_not_called()
|
||||
|
||||
|
||||
def test_run_execution_allows_labeled_unverified_proof_attempt() -> None:
|
||||
manager = MagicMock()
|
||||
manager.create.return_value = _fake_status()
|
||||
|
||||
result = run_execution(
|
||||
_request(),
|
||||
catalog=_catalog_with_readiness("unverified"),
|
||||
rein=_FakeRein(),
|
||||
manager=manager,
|
||||
)
|
||||
|
||||
assert result.ok is True
|
||||
assert result.evidence.profile_readiness.status == "unverified"
|
||||
manager.create.assert_called_once()
|
||||
|
||||
|
||||
def test_run_execution_refuses_worker_identifier_before_sandbox() -> None:
|
||||
manager = MagicMock()
|
||||
request = _request().model_copy(update={"actor": "rein-aharness@railiance01"})
|
||||
|
|
@ -160,7 +211,12 @@ def test_hub_receives_normalized_evidence_without_raw_output() -> None:
|
|||
manager.create.return_value = _fake_status()
|
||||
|
||||
with patch("glas_harness.gateway.hub.post_progress_event", return_value=True) as post:
|
||||
result = run_execution(_request(report_to_hub=True), rein=_FakeRein(), manager=manager)
|
||||
result = run_execution(
|
||||
_request(report_to_hub=True),
|
||||
catalog=_catalog_with_readiness(),
|
||||
rein=_FakeRein(),
|
||||
manager=manager,
|
||||
)
|
||||
|
||||
post.assert_called_once()
|
||||
detail = post.call_args.kwargs["detail"]
|
||||
|
|
@ -181,7 +237,10 @@ def test_hub_failure_detail_excludes_raw_provider_error() -> None:
|
|||
|
||||
with patch("glas_harness.gateway.hub.post_progress_event", return_value=True) as post:
|
||||
result = run_execution(
|
||||
_request(report_to_hub=True), rein=_FailingRein(), manager=manager
|
||||
_request(report_to_hub=True),
|
||||
catalog=_catalog_with_readiness(),
|
||||
rein=_FailingRein(),
|
||||
manager=manager,
|
||||
)
|
||||
|
||||
detail = post.call_args.kwargs["detail"]
|
||||
|
|
@ -201,6 +260,7 @@ def test_wrapper_requires_and_reports_harness_profile() -> None:
|
|||
description="d",
|
||||
rein=_FakeRein(),
|
||||
manager=manager,
|
||||
catalog=_catalog_with_readiness(),
|
||||
report_to_hub=False,
|
||||
)
|
||||
|
||||
|
|
@ -213,7 +273,12 @@ def test_teardown_failure_is_visible_in_evidence() -> None:
|
|||
manager.create.return_value = _fake_status()
|
||||
manager.destroy.side_effect = RuntimeError("cannot teardown")
|
||||
|
||||
result = run_execution(_request(), rein=_FakeRein(), manager=manager)
|
||||
result = run_execution(
|
||||
_request(),
|
||||
catalog=_catalog_with_readiness(),
|
||||
rein=_FakeRein(),
|
||||
manager=manager,
|
||||
)
|
||||
|
||||
assert result.ok is False
|
||||
assert result.evidence.failure_stage == "teardown"
|
||||
|
|
|
|||
|
|
@ -71,6 +71,14 @@ def test_committed_catalog_resolves_both_constellations() -> None:
|
|||
"rein-aharness",
|
||||
"rein-openweights",
|
||||
}
|
||||
assert {
|
||||
(context.profile.id, context.operational_readiness.status)
|
||||
for context in contexts
|
||||
} == {
|
||||
("harness.agent-dev-local", "blocked"),
|
||||
("harness.agent-dev-openweights-local", "blocked"),
|
||||
("harness.agent-dev", "unverified"),
|
||||
}
|
||||
profile, descriptor = catalog.resolve(
|
||||
"harness.agent-dev-openweights-local@1.0.0"
|
||||
)
|
||||
|
|
@ -89,6 +97,41 @@ def test_committed_catalog_resolves_both_constellations() -> None:
|
|||
assert rein.budget_tokens == 60000
|
||||
|
||||
|
||||
def test_ready_profile_requires_an_evidence_reference(tmp_path) -> None:
|
||||
profiles = tmp_path / "profiles"
|
||||
reins = tmp_path / "reins"
|
||||
profiles.mkdir()
|
||||
reins.mkdir()
|
||||
_write(
|
||||
profiles,
|
||||
"profile.yaml",
|
||||
_profile(
|
||||
extra="""operational_readiness:
|
||||
status: ready
|
||||
reason: proof completed
|
||||
owner: tests
|
||||
"""
|
||||
),
|
||||
)
|
||||
_write(reins, "rein.yaml", _rein())
|
||||
|
||||
with pytest.raises(ProfileError, match="ready operational status requires evidence_ref"):
|
||||
ProfileCatalog(profiles, reins).profiles()
|
||||
|
||||
|
||||
def test_undeclared_readiness_defaults_to_unverified(tmp_path) -> None:
|
||||
profiles = tmp_path / "profiles"
|
||||
reins = tmp_path / "reins"
|
||||
profiles.mkdir()
|
||||
reins.mkdir()
|
||||
_write(profiles, "profile.yaml", _profile())
|
||||
_write(reins, "rein.yaml", _rein())
|
||||
|
||||
context = ProfileCatalog(profiles, reins).resolve_context("harness.test@1.0.0")
|
||||
|
||||
assert context.operational_readiness.status == "unverified"
|
||||
assert context.operational_readiness.reason == "no operational proof declared"
|
||||
|
||||
def test_unknown_profile_fails_closed() -> None:
|
||||
with pytest.raises(UnknownProfileError, match="unknown harness profile"):
|
||||
ProfileCatalog().resolve("harness.missing@1.0.0")
|
||||
|
|
|
|||
67
workplans/GLAS-WP-0008-profile-operational-readiness.md
Normal file
67
workplans/GLAS-WP-0008-profile-operational-readiness.md
Normal file
|
|
@ -0,0 +1,67 @@
|
|||
---
|
||||
id: GLAS-WP-0008
|
||||
type: workplan
|
||||
title: "Make profile operational readiness explicit"
|
||||
domain: infotech
|
||||
repo: glas-harness
|
||||
status: finished
|
||||
owner: codex
|
||||
topic_slug: profile-operational-readiness
|
||||
created: "2026-08-23"
|
||||
updated: "2026-08-23"
|
||||
---
|
||||
|
||||
# Make profile operational readiness explicit
|
||||
|
||||
## Define the readiness contract
|
||||
|
||||
```task
|
||||
id: GLAS-WP-0008-T01
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Add a typed operational-readiness state that is distinct from catalog
|
||||
enablement. Expose it in resolved contexts and execution evidence without
|
||||
changing rein selection or introducing a fallback.
|
||||
|
||||
**Completed 2026-08-23:** added the backward-compatible
|
||||
`OperationalReadiness` model and propagated it through resolved catalog
|
||||
contexts and compact execution evidence. Undeclared legacy profiles default to
|
||||
`unverified`; positive `ready` claims require an evidence reference.
|
||||
|
||||
## Declare profile states and enforce known blockers
|
||||
|
||||
```task
|
||||
id: GLAS-WP-0008-T02
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Declare the current evidence-backed state of every committed profile. Refuse a
|
||||
known-blocked profile at resolution before sandbox creation; permit an
|
||||
unverified profile only as an explicitly labeled bounded proof attempt.
|
||||
|
||||
**Completed 2026-08-23:** both bwrap-local profiles now declare `blocked` with
|
||||
owner `sand-boxer` and evidence `GLAS-IN-0002`; the remote profile declares
|
||||
`unverified`. Gateway resolution rejects blocked profiles before sandbox
|
||||
creation while retaining unverified attempts and their label in evidence.
|
||||
|
||||
## Verify and publish the clarified behavior
|
||||
|
||||
```task
|
||||
id: GLAS-WP-0008-T03
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Cover the schema, catalog, CLI, gateway refusal, evidence, and compatibility
|
||||
path with tests. Update scope and execution-profile documentation, close
|
||||
`GLAS-IN-0005`, synchronize State Hub, publish, and report the behavior change
|
||||
to known consumers.
|
||||
|
||||
**Completed 2026-08-23:** CLI text/JSON report readiness without calling the
|
||||
catalog executable, documentation and examples reflect the current states,
|
||||
and the 63-test suite plus packaged catalog validation pass. `GLAS-IN-0005` is
|
||||
absorbed by this workplan. The remaining local runtime dependency was already
|
||||
handed off as live residual `GLAS-IN-0002`; no new residual was created.
|
||||
Loading…
Add table
Add a link
Reference in a new issue