diff --git a/README.md b/README.md index 4c94d37..95e8dad 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,8 @@ assignment / channel / activity ## Profiles -List and validate every committed executable profile: +List and validate every committed catalog profile, including its independently +declared runtime readiness: ```bash glas-harness profiles @@ -32,19 +33,26 @@ Current constellations include: open-weight route, local bwrap sandbox. - `harness.agent-dev@1.0.0`: `rein-aharness`, remote agent-dev sandbox. -Run a task through an explicit profile: +The two local profiles are currently `blocked` by the missing sand-boxer +owner-execution/runtime path. The remote profile is `unverified`: it may be +used only as an explicitly labeled proof attempt. No committed profile is +currently declared `ready`. + +Start a bounded proof attempt through the currently unverified remote profile: ```bash glas-harness run \ - --harness-profile harness.agent-dev-local@1.0.0 \ + --harness-profile harness.agent-dev@1.0.0 \ --repo /absolute/path/to/repo \ --title "Bounded change" \ --description "Make the requested change and commit it" ``` There is no governed default rein, model, sandbox, or tool profile. Unknown, -disabled, incompatible, and ambiguous selections are refused before sandbox -creation. +disabled, incompatible, ambiguous, and operationally blocked selections are +refused before sandbox creation. An unverified selection is allowed so a +bounded proof can establish or disprove readiness, and its evidence retains +that state. After creation, execution uses only the reachability-derived sandbox workspace: same-host namespaces are entered with `nsenter`, while remote workspaces are diff --git a/SCOPE.md b/SCOPE.md index 32f0898..00e4e64 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -28,7 +28,8 @@ The repository currently implements: creation, keeping queue `worker_id` values outside the actor field; - one invocation channel: the local `glas-harness run` CLI; - a packaged, versioned profile/rein catalog and `glas-harness profiles` - validation command; and + validation command that reports catalog enablement separately from typed + operational readiness; and - compact State Hub progress evidence that excludes prompts, model/tool output, provider bodies, and credentials. @@ -40,20 +41,21 @@ installs the package, runs the suite, and validates the packaged catalog. | Surface | Current verified status | |---|---| | Profile/catalog resolution | Implemented and CI-validated | -| Unknown or invalid profile refusal | Implemented before sandbox creation | +| Unknown, invalid, or known-blocked profile refusal | Implemented before sandbox creation | | Sandbox create/destroy lifecycle | Implemented; live failure paths prove teardown | | Original-checkout isolation | Enforced by code: it is never a post-create execution fallback | | SSH transport construction | Implemented and unit-tested; no current positive post-hardening live proof | -| Same-host bwrap execution | Fail-closed, not operational: consumer `nsenter` is denied and the rein runtime is absent inside the sandbox | -| `rein-aharness` adapter | Implemented; current local profile reaches sandbox creation and fails at session start on the bwrap owner boundary | +| Same-host bwrap execution | Fail-closed, not operational: both local profiles are declared `blocked` and refused before creation; the underlying proof found consumer `nsenter` denied and no in-sandbox rein runtime | +| `rein-aharness` adapter | Implemented; local selection is blocked before creation, while the remote profile remains an explicitly labeled unverified proof path | | `rein-openweights` adapter | Implemented; provider credential repair was proven, but current local sandbox execution has the same unresolved owner boundary | | CLI channel | Implemented | | State Hub evidence | Implemented as compact progress evidence, not a complete session/tool audit service | -An `enabled` profile currently means that catalog policy permits selection. It -does not prove that its rein executable, credential delivery, network egress, -and sandbox transport are operational on a particular host. That semantics is -an open governance gap tracked in `GLAS-IN-0005`. +An `enabled` profile means only that catalog policy permits selection. Typed +operational readiness separately records `ready`, `unverified`, or `blocked`. +Known-blocked profiles refuse before sandbox creation; unverified profiles may +run only as labeled proof attempts, with the state preserved in evidence. +There is currently no committed `ready` profile. ## In scope @@ -101,8 +103,6 @@ yet Coulomb's general-purpose agent harness service. - `GLAS-IN-0003`: grandfather the pre-canon `GLAS-0001` identifiers. - `GLAS-IN-0004`: align the documented ad hoc workplan convention with the identifier canon. -- `GLAS-IN-0005`: make profile `enabled` status and operational readiness - truthful and testable. ## Getting oriented diff --git a/WORK-RECORDS.md b/WORK-RECORDS.md index 7f8295c..717d463 100644 --- a/WORK-RECORDS.md +++ b/WORK-RECORDS.md @@ -16,6 +16,7 @@ | workplan | GLAS-WP-0005 | blocked | — | workplans/GLAS-WP-0005-sandbox-boundary-runtime-hardening.md | | workplan | GLAS-WP-0006 | finished | — | workplans/GLAS-WP-0006-consumer-actor-validation.md | | workplan | GLAS-WP-0007 | finished | — | workplans/GLAS-WP-0007-capability-scope-truth.md | +| workplan | GLAS-WP-0008 | finished | — | workplans/GLAS-WP-0008-profile-operational-readiness.md | | task | GLAS-0001-T01 | done | — | workplans/GLAS-0001-statehub-bootstrap.md | | task | GLAS-0001-T02 | done | — | workplans/GLAS-0001-statehub-bootstrap.md | | task | GLAS-0001-T03 | done | — | workplans/GLAS-0001-statehub-bootstrap.md | @@ -49,8 +50,11 @@ | task | GLAS-WP-0006-T01 | done | — | workplans/GLAS-WP-0006-consumer-actor-validation.md | | task | GLAS-WP-0007-T01 | done | — | workplans/GLAS-WP-0007-capability-scope-truth.md | | task | GLAS-WP-0007-T02 | done | — | workplans/GLAS-WP-0007-capability-scope-truth.md | +| task | GLAS-WP-0008-T01 | done | — | workplans/GLAS-WP-0008-profile-operational-readiness.md | +| task | GLAS-WP-0008-T02 | done | — | workplans/GLAS-WP-0008-profile-operational-readiness.md | +| task | GLAS-WP-0008-T03 | done | — | workplans/GLAS-WP-0008-profile-operational-readiness.md | | intake | GLAS-IN-0001 | done | blue | docs/intakes/residuals.md | | intake | GLAS-IN-0002 | todo | red | docs/intakes/residuals.md | | intake | GLAS-IN-0003 | todo | green | docs/intakes/residuals.md | | intake | GLAS-IN-0004 | todo | green | docs/intakes/residuals.md | -| intake | GLAS-IN-0005 | todo | red | docs/intakes/residuals.md | +| intake | GLAS-IN-0005 | done | red | docs/intakes/residuals.md | diff --git a/docs/execution-profiles.md b/docs/execution-profiles.md index d46e837..68eefc2 100644 --- a/docs/execution-profiles.md +++ b/docs/execution-profiles.md @@ -20,7 +20,10 @@ ExecutionEvidence -> assignment/audit consumers Each YAML document under `profiles/` declares: -- stable `id`, semantic `version`, exact `contract_version`, and status; +- stable `id`, semantic `version`, exact `contract_version`, and catalog + enablement status; +- typed operational readiness (`ready`, `unverified`, or `blocked`) with its + reason, owner, and evidence reference; - rein registry ID and required capabilities; - sand-boxer `sandbox_profile`; - rein-enforced `tool_profile`; @@ -35,6 +38,20 @@ enabled revision exists. Inline secrets and token-looking values are refused. Use `glas-harness profiles` as the catalog/packaging validation command. +Catalog enablement and operational readiness answer different questions: + +- `status: enabled` permits deterministic selection after schema and rein + compatibility checks; +- `operational_readiness.status: ready` requires a positive `evidence_ref`; +- `unverified` permits a bounded proof attempt but is carried into execution + evidence and must not be described as ready; and +- `blocked` requires an owner plus evidence reference and is refused during + resolution, before sandbox creation. + +Profiles that predate the field default conservatively to `unverified`. The +catalog command lists all enabled profiles and labels readiness; it does not +filter blocked profiles or call them executable. + Catalog validation proves schema/compatibility, not transport installation. An execution host must install `glas-harness` with sand-boxer support and make each selected rein plus its dependencies available *inside* the sandbox transport, @@ -44,6 +61,11 @@ also needs its documented `rein-aharness[llm]`/sibling `llm-connect` adapter dependency in that environment. Missing runtime dependencies fail closed at session start and still trigger sandbox teardown. +The committed local profiles are currently declared `blocked` by +`GLAS-IN-0002`. The remote profile is `unverified` because there is no positive +post-hardening execution proof. There is currently no committed `ready` +profile. + ## Consumer request See `examples/execution-request.json`. Consumers know only a profile reference, @@ -59,7 +81,7 @@ The current exact-profile path does not interpret resource envelopes or choose profiles from requirements. A future requirements resolver belongs at this Glas boundary, must be deterministic and explainable, and must refuse ambiguity. -## Two equivalent selections +## Two parallel local selections For the same task and organizational references, switch only: @@ -71,9 +93,11 @@ harness.agent-dev-openweights-local@1.0.0 -> rein-openweights / qwen/qwen-2.5-72b-instruct / profile.bwrap-local ``` -Both use `green-commit-only` and return `GatewayResult` / `ExecutionEvidence`. -This proves interchangeability at the Glas boundary; it does not claim the -models have identical quality, price, or provider behavior. +Both declare `green-commit-only` and the same outer `GatewayResult` / +`ExecutionEvidence` contract. They are interchangeable as catalog shapes, but +both are currently operationally blocked and refused before sandbox creation. +This does not claim current runtime availability or identical model quality, +price, or provider behavior. ## Ownership handoff diff --git a/docs/harness-contract.md b/docs/harness-contract.md index 6724872..736d6ba 100644 --- a/docs/harness-contract.md +++ b/docs/harness-contract.md @@ -32,6 +32,13 @@ The gateway returns `GatewayResult` with: Raw prompts, model output, tool output, and credential material are excluded from the State Hub detail. +`HarnessProfile.status` governs catalog selection. The separate +`OperationalReadiness` model records runtime status, a reason, its owner, and +an evidence reference. `ready` requires positive evidence; `blocked` requires +an owner and evidence and is refused during resolution before sandbox +creation; `unverified` permits only a labeled proof attempt. The selected state +is copied into `ResolvedExecutionContext` and `ExecutionEvidence`. + ## Rein lifecycle ```python @@ -99,7 +106,8 @@ remain rein-local under ADR-002. The gateway returns evidence for every normal refusal/failure path: -- `resolution`: unknown, disabled, ambiguous, incompatible, or unsafe profile; +- `resolution`: unknown, disabled, ambiguous, incompatible, operationally + blocked, or unsafe profile; - `sandbox_create`; - `session_start`; - `execution` (including a rein-declared unsuccessful result); diff --git a/docs/intakes/residuals.md b/docs/intakes/residuals.md index 1d5a2e9..5727347 100644 --- a/docs/intakes/residuals.md +++ b/docs/intakes/residuals.md @@ -147,7 +147,7 @@ id: GLAS-IN-0005 kind: intake title: "Separate profile selection enablement from runtime readiness" lane: red -status: todo +status: done priority: high owner: glas-harness repo: glas-harness @@ -168,3 +168,9 @@ description: | fallback or claim readiness from lifecycle-only proof. state_hub_intake_id: "01a02dce-b761-75e6-861a-8abf8b8e0072" ``` + +Completed by `GLAS-WP-0008` on 2026-08-23. Catalog enablement is now separate +from typed operational readiness. Both local profiles are `blocked` by +`GLAS-IN-0002` and refuse before sandbox creation; the remote profile is +`unverified` and may be used only for a labeled bounded proof attempt. Catalog +and execution evidence expose the state, and no host fallback was added. diff --git a/examples/execution-request.json b/examples/execution-request.json index 8d82592..24148fd 100644 --- a/examples/execution-request.json +++ b/examples/execution-request.json @@ -1,5 +1,5 @@ { - "harness_profile_ref": "harness.agent-dev-local@1.0.0", + "harness_profile_ref": "harness.agent-dev@1.0.0", "repo": "/absolute/path/to/target-repo", "title": "Implement the bounded assignment", "description": "Apply the approved change, run focused checks, and commit the result.", diff --git a/history/2026-08-23-scope-vs-intent-assessment.md b/history/2026-08-23-scope-vs-intent-assessment.md index 92f7c69..4e58204 100644 --- a/history/2026-08-23-scope-vs-intent-assessment.md +++ b/history/2026-08-23-scope-vs-intent-assessment.md @@ -68,10 +68,10 @@ not a positive local execution path. `GLAS-IN-0002` owns the sand-boxer contract for command execution, mounted or imaged rein runtime, explicit model egress, credential delivery, and teardown evidence. `GLAS-WP-0005-T05` remains waiting on it. -2. **Profile readiness semantics — high.** A catalog-valid `enabled` profile is - currently presented alongside profiles that cannot dispatch on the deployed - runtime. `GLAS-IN-0005` requires an explicit readiness/enablement contract, - including whether unproven profiles must be disabled or separately labeled. +2. **Profile readiness semantics — resolved 2026-08-23.** `GLAS-WP-0008` + separated catalog enablement from typed `ready`, `unverified`, and `blocked` + runtime states. Known blockers now refuse before sandbox creation, while a + labeled unverified profile remains available for a bounded proof attempt. ### Record-governance gaps @@ -92,13 +92,14 @@ or claim them as capabilities. ## Recommended sequence 1. Complete `GLAS-IN-0002` and capture one positive, source-isolated rein run. -2. Resolve `GLAS-IN-0005` so catalog output distinguishes selectable schema - from deployable runtime capability. +2. Use the implemented readiness state in catalog output and evidence; promote + a profile to `ready` only with a positive evidence reference. 3. Reassess whether the remote profile is an operational recovery path or an unproven declaration; retain it only with evidence and owned runtime setup. 4. Add broader harness pillars only when a real channel/consumer supplies the boundary and evidence requirements. -Until steps 1–2 complete, describe Glas as a secure, fail-closed execution -router nucleus—not yet the default or unified agent harness service promised by -the maturity target in `INTENT.md`. +Until step 1 completes and at least one profile has positive runtime evidence, +describe Glas as a secure, fail-closed execution router nucleus—not yet the +default or unified agent harness service promised by the maturity target in +`INTENT.md`. diff --git a/profiles/harness.agent-dev-local.yaml b/profiles/harness.agent-dev-local.yaml index a4f0f95..433a31c 100644 --- a/profiles/harness.agent-dev-local.yaml +++ b/profiles/harness.agent-dev-local.yaml @@ -2,6 +2,11 @@ id: harness.agent-dev-local version: "1.0.0" contract_version: "1.0" status: enabled +operational_readiness: + status: blocked + reason: sand-boxer owner-mediated bwrap execution and in-sandbox rein runtime are unavailable + owner: sand-boxer + evidence_ref: GLAS-IN-0002 rein: id: rein-aharness required_capabilities: diff --git a/profiles/harness.agent-dev-openweights-local.yaml b/profiles/harness.agent-dev-openweights-local.yaml index e919da7..df9e797 100644 --- a/profiles/harness.agent-dev-openweights-local.yaml +++ b/profiles/harness.agent-dev-openweights-local.yaml @@ -2,6 +2,11 @@ id: harness.agent-dev-openweights-local version: "1.0.0" contract_version: "1.0" status: enabled +operational_readiness: + status: blocked + reason: sand-boxer owner-mediated bwrap execution and in-sandbox rein runtime are unavailable + owner: sand-boxer + evidence_ref: GLAS-IN-0002 rein: id: rein-openweights required_capabilities: diff --git a/profiles/harness.agent-dev.yaml b/profiles/harness.agent-dev.yaml index 8b4086a..a4428a9 100644 --- a/profiles/harness.agent-dev.yaml +++ b/profiles/harness.agent-dev.yaml @@ -2,6 +2,11 @@ id: harness.agent-dev version: "1.0.0" contract_version: "1.0" status: enabled +operational_readiness: + status: unverified + reason: no positive remote execution proof exists after transport hardening + owner: glas-harness + evidence_ref: GLAS-WP-0005 rein: id: rein-aharness required_capabilities: diff --git a/src/glas_harness/cli.py b/src/glas_harness/cli.py index d3344ae..99d5a21 100644 --- a/src/glas_harness/cli.py +++ b/src/glas_harness/cli.py @@ -30,7 +30,7 @@ def main(argv: list[str] | None = None) -> int: run.add_argument("--project", default="glas-harness") run.add_argument("--no-hub", action="store_true", help="Skip the gateway's own hub reporting") - profiles = sub.add_parser("profiles", help="Validate and list executable Glas profiles") + profiles = sub.add_parser("profiles", help="Validate and list Glas catalog profiles") profiles.add_argument("--json", action="store_true", help="Emit machine-readable JSON") args = parser.parse_args(argv) @@ -76,6 +76,7 @@ def main(argv: list[str] | None = None) -> int: print( f"{row['profile']['id']}@{row['profile']['version']}\t" f"rein={row['rein_id']}@{row['rein_version']}\t" + f"readiness={row['operational_readiness']['status']}\t" f"model={row['model']['model']}\t" f"sandbox={row['sandbox_profile']}" ) diff --git a/src/glas_harness/contract.py b/src/glas_harness/contract.py index f386276..3c727e9 100644 --- a/src/glas_harness/contract.py +++ b/src/glas_harness/contract.py @@ -11,7 +11,7 @@ import re from abc import ABC, abstractmethod from typing import Any, Literal -from pydantic import BaseModel, ConfigDict, Field, field_validator +from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator CONTRACT_VERSION = "1.0" _SEMVER = re.compile(r"^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:[-+][0-9A-Za-z.-]+)?$") @@ -54,11 +54,31 @@ class ReinSelection(ContractModel): required_capabilities: dict[str, Any] = Field(default_factory=dict) +class OperationalReadiness(ContractModel): + """Evidence-backed runtime status, separate from catalog selection policy.""" + + status: Literal["ready", "unverified", "blocked"] = "unverified" + reason: str = Field(default="no operational proof declared", min_length=1) + owner: str | None = Field(default=None, min_length=1) + evidence_ref: str | None = Field(default=None, min_length=1) + + @model_validator(mode="after") + def validate_evidence(self) -> OperationalReadiness: + if self.status == "ready" and self.evidence_ref is None: + raise ValueError("ready operational status requires evidence_ref") + if self.status == "blocked" and (self.owner is None or self.evidence_ref is None): + raise ValueError("blocked operational status requires owner and evidence_ref") + return self + + class HarnessProfile(ContractModel): id: str = Field(pattern=r"^harness\.[a-z0-9][a-z0-9._-]*$") version: str contract_version: str status: Literal["enabled", "disabled"] = "enabled" + operational_readiness: OperationalReadiness = Field( + default_factory=OperationalReadiness + ) rein: ReinSelection sandbox_profile: str = Field(pattern=r"^profile\.[a-z0-9][a-z0-9._-]*$") tool_profile: str = Field(min_length=1) @@ -152,6 +172,9 @@ class ResolvedExecutionContext(ContractModel): profile: HarnessProfileRef rein_id: str rein_version: str + operational_readiness: OperationalReadiness = Field( + default_factory=OperationalReadiness + ) sandbox_profile: str tool_profile: str model: ModelRoute @@ -166,6 +189,7 @@ class ExecutionEvidence(ContractModel): target_repo: str contract_version: str = CONTRACT_VERSION profile_ref: str | None = None + profile_readiness: OperationalReadiness | None = None rein_id: str | None = None rein_version: str | None = None model_route: str | None = None diff --git a/src/glas_harness/gateway.py b/src/glas_harness/gateway.py index 4669329..44714c4 100644 --- a/src/glas_harness/gateway.py +++ b/src/glas_harness/gateway.py @@ -84,6 +84,7 @@ def run_execution( try: consumer = _consumer_from_request(request) profile, descriptor = catalog.resolve(request.harness_profile_ref) + catalog.require_operational(profile) selected_rein = rein or catalog.build_rein(profile, descriptor) except Exception as exc: outcome = "refused" @@ -292,6 +293,7 @@ def _build_result( target_repo=request.repo, contract_version=CONTRACT_VERSION, profile_ref=str(profile.ref) if profile else None, + profile_readiness=profile.operational_readiness if profile else None, rein_id=descriptor.id if descriptor else None, rein_version=descriptor.version if descriptor else None, model_route=profile.model.route if profile else None, diff --git a/src/glas_harness/profiles.py b/src/glas_harness/profiles.py index eac3ff2..d8c0ba1 100644 --- a/src/glas_harness/profiles.py +++ b/src/glas_harness/profiles.py @@ -37,6 +37,10 @@ class IncompatibleProfileError(ProfileError): pass +class OperationallyBlockedProfileError(ProfileError): + pass + + _SENSITIVE_KEY = re.compile( r"(^|_)(api_key|password|passwd|secret|secret_value|token_value|private_key)$", re.IGNORECASE, @@ -186,12 +190,25 @@ class ProfileCatalog: profile=profile.ref, rein_id=descriptor.id, rein_version=descriptor.version, + operational_readiness=profile.operational_readiness, sandbox_profile=profile.sandbox_profile, tool_profile=profile.tool_profile, model=profile.model, limits=profile.limits, ) + def require_operational(self, profile: HarnessProfile) -> None: + """Refuse known runtime blockers while allowing labeled proof attempts.""" + + readiness = profile.operational_readiness + if readiness.status != "blocked": + return + raise OperationallyBlockedProfileError( + f"profile {profile.ref} operational readiness is blocked: " + f"{readiness.reason} (owner={readiness.owner}; " + f"evidence={readiness.evidence_ref})" + ) + def build_rein(self, profile: HarnessProfile, descriptor: ReinDescriptor) -> Rein: module_name, class_name = descriptor.handler.split(":", 1) module = importlib.import_module(module_name) diff --git a/tests/test_cli.py b/tests/test_cli.py index 2654bd6..d20d7eb 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -12,6 +12,19 @@ def test_profiles_command_lists_committed_constellations(capsys) -> None: rows = json.loads(capsys.readouterr().out) assert {row["rein_id"] for row in rows} == {"rein-aharness", "rein-openweights"} + assert {row["operational_readiness"]["status"] for row in rows} == { + "blocked", + "unverified", + } + + +def test_profiles_text_output_labels_runtime_readiness(capsys) -> None: + assert main(["profiles"]) == 0 + + output = capsys.readouterr().out + assert "readiness=blocked" in output + assert "readiness=unverified" in output + assert "executable" not in output def test_execution_request_example_matches_contract() -> None: @@ -19,7 +32,7 @@ def test_execution_request_example_matches_contract() -> None: request = ExecutionRequest.model_validate_json(fixture.read_text()) - assert request.harness_profile_ref == "harness.agent-dev-local@1.0.0" + assert request.harness_profile_ref == "harness.agent-dev@1.0.0" assert request.assignment_ref == "role-assignment:agent-7:42" diff --git a/tests/test_gateway.py b/tests/test_gateway.py index 7eac49a..09f10c4 100644 --- a/tests/test_gateway.py +++ b/tests/test_gateway.py @@ -6,12 +6,14 @@ from sandboxer.models import Reachability, SandboxState, SandboxStatus from glas_harness.contract import ( ExecutionRequest, ExecutionSummary, + OperationalReadiness, Rein, SandboxHandle, ToolCall, ToolResult, ) from glas_harness.gateway import run_execution, run_task_through_rein +from glas_harness.profiles import ProfileCatalog PROFILE = "harness.agent-dev-local@1.0.0" @@ -83,12 +85,29 @@ def _request(*, profile: str = PROFILE, report_to_hub: bool = False) -> Executio ) +def _catalog_with_readiness(status: str = "ready") -> ProfileCatalog: + catalog = ProfileCatalog() + profile, _ = catalog.resolve(PROFILE) + readiness = OperationalReadiness( + status=status, + reason="test-only readiness state", + owner="tests", + evidence_ref="test:gateway-readiness", + ) + catalog.profiles()[(profile.id, profile.version)] = profile.model_copy( + update={"operational_readiness": readiness} + ) + return catalog + + def test_run_execution_creates_and_destroys_sandbox() -> None: manager = MagicMock() manager.create.return_value = _fake_status() rein = _FakeRein() - result = run_execution(_request(), rein=rein, manager=manager) + result = run_execution( + _request(), catalog=_catalog_with_readiness(), rein=rein, manager=manager + ) assert rein.calls == [ "start_session", @@ -100,6 +119,7 @@ def test_run_execution_creates_and_destroys_sandbox() -> None: manager.destroy.assert_called_once_with("sbx1") assert result.ok is True assert result.evidence.profile_ref == PROFILE + assert result.evidence.profile_readiness.status == "ready" assert result.evidence.rein_id == "rein-aharness" assert result.evidence.commit_sha == "deadbeef" assert result.evidence.tokens_spent == 123 @@ -116,7 +136,9 @@ def test_run_execution_normalizes_execution_failure_and_tears_down() -> None: raise RuntimeError("boom") rein = _FailingRein() - result = run_execution(_request(), rein=rein, manager=manager) + result = run_execution( + _request(), catalog=_catalog_with_readiness(), rein=rein, manager=manager + ) assert result.ok is False assert result.evidence.outcome == "failed" @@ -139,6 +161,35 @@ def test_run_execution_refuses_unknown_profile_before_sandbox() -> None: manager.create.assert_not_called() +def test_run_execution_refuses_blocked_profile_before_sandbox() -> None: + manager = MagicMock() + + result = run_execution(_request(), rein=_FakeRein(), manager=manager) + + assert result.ok is False + assert result.evidence.outcome == "refused" + assert result.evidence.failure_stage == "resolution" + assert result.evidence.profile_readiness.status == "blocked" + assert "GLAS-IN-0002" in (result.evidence.error or "") + manager.create.assert_not_called() + + +def test_run_execution_allows_labeled_unverified_proof_attempt() -> None: + manager = MagicMock() + manager.create.return_value = _fake_status() + + result = run_execution( + _request(), + catalog=_catalog_with_readiness("unverified"), + rein=_FakeRein(), + manager=manager, + ) + + assert result.ok is True + assert result.evidence.profile_readiness.status == "unverified" + manager.create.assert_called_once() + + def test_run_execution_refuses_worker_identifier_before_sandbox() -> None: manager = MagicMock() request = _request().model_copy(update={"actor": "rein-aharness@railiance01"}) @@ -160,7 +211,12 @@ def test_hub_receives_normalized_evidence_without_raw_output() -> None: manager.create.return_value = _fake_status() with patch("glas_harness.gateway.hub.post_progress_event", return_value=True) as post: - result = run_execution(_request(report_to_hub=True), rein=_FakeRein(), manager=manager) + result = run_execution( + _request(report_to_hub=True), + catalog=_catalog_with_readiness(), + rein=_FakeRein(), + manager=manager, + ) post.assert_called_once() detail = post.call_args.kwargs["detail"] @@ -181,7 +237,10 @@ def test_hub_failure_detail_excludes_raw_provider_error() -> None: with patch("glas_harness.gateway.hub.post_progress_event", return_value=True) as post: result = run_execution( - _request(report_to_hub=True), rein=_FailingRein(), manager=manager + _request(report_to_hub=True), + catalog=_catalog_with_readiness(), + rein=_FailingRein(), + manager=manager, ) detail = post.call_args.kwargs["detail"] @@ -201,6 +260,7 @@ def test_wrapper_requires_and_reports_harness_profile() -> None: description="d", rein=_FakeRein(), manager=manager, + catalog=_catalog_with_readiness(), report_to_hub=False, ) @@ -213,7 +273,12 @@ def test_teardown_failure_is_visible_in_evidence() -> None: manager.create.return_value = _fake_status() manager.destroy.side_effect = RuntimeError("cannot teardown") - result = run_execution(_request(), rein=_FakeRein(), manager=manager) + result = run_execution( + _request(), + catalog=_catalog_with_readiness(), + rein=_FakeRein(), + manager=manager, + ) assert result.ok is False assert result.evidence.failure_stage == "teardown" diff --git a/tests/test_profiles.py b/tests/test_profiles.py index 161569e..3c426a9 100644 --- a/tests/test_profiles.py +++ b/tests/test_profiles.py @@ -71,6 +71,14 @@ def test_committed_catalog_resolves_both_constellations() -> None: "rein-aharness", "rein-openweights", } + assert { + (context.profile.id, context.operational_readiness.status) + for context in contexts + } == { + ("harness.agent-dev-local", "blocked"), + ("harness.agent-dev-openweights-local", "blocked"), + ("harness.agent-dev", "unverified"), + } profile, descriptor = catalog.resolve( "harness.agent-dev-openweights-local@1.0.0" ) @@ -89,6 +97,41 @@ def test_committed_catalog_resolves_both_constellations() -> None: assert rein.budget_tokens == 60000 +def test_ready_profile_requires_an_evidence_reference(tmp_path) -> None: + profiles = tmp_path / "profiles" + reins = tmp_path / "reins" + profiles.mkdir() + reins.mkdir() + _write( + profiles, + "profile.yaml", + _profile( + extra="""operational_readiness: + status: ready + reason: proof completed + owner: tests +""" + ), + ) + _write(reins, "rein.yaml", _rein()) + + with pytest.raises(ProfileError, match="ready operational status requires evidence_ref"): + ProfileCatalog(profiles, reins).profiles() + + +def test_undeclared_readiness_defaults_to_unverified(tmp_path) -> None: + profiles = tmp_path / "profiles" + reins = tmp_path / "reins" + profiles.mkdir() + reins.mkdir() + _write(profiles, "profile.yaml", _profile()) + _write(reins, "rein.yaml", _rein()) + + context = ProfileCatalog(profiles, reins).resolve_context("harness.test@1.0.0") + + assert context.operational_readiness.status == "unverified" + assert context.operational_readiness.reason == "no operational proof declared" + def test_unknown_profile_fails_closed() -> None: with pytest.raises(UnknownProfileError, match="unknown harness profile"): ProfileCatalog().resolve("harness.missing@1.0.0") diff --git a/workplans/GLAS-WP-0008-profile-operational-readiness.md b/workplans/GLAS-WP-0008-profile-operational-readiness.md new file mode 100644 index 0000000..85e2b77 --- /dev/null +++ b/workplans/GLAS-WP-0008-profile-operational-readiness.md @@ -0,0 +1,67 @@ +--- +id: GLAS-WP-0008 +type: workplan +title: "Make profile operational readiness explicit" +domain: infotech +repo: glas-harness +status: finished +owner: codex +topic_slug: profile-operational-readiness +created: "2026-08-23" +updated: "2026-08-23" +--- + +# Make profile operational readiness explicit + +## Define the readiness contract + +```task +id: GLAS-WP-0008-T01 +status: done +priority: high +``` + +Add a typed operational-readiness state that is distinct from catalog +enablement. Expose it in resolved contexts and execution evidence without +changing rein selection or introducing a fallback. + +**Completed 2026-08-23:** added the backward-compatible +`OperationalReadiness` model and propagated it through resolved catalog +contexts and compact execution evidence. Undeclared legacy profiles default to +`unverified`; positive `ready` claims require an evidence reference. + +## Declare profile states and enforce known blockers + +```task +id: GLAS-WP-0008-T02 +status: done +priority: high +``` + +Declare the current evidence-backed state of every committed profile. Refuse a +known-blocked profile at resolution before sandbox creation; permit an +unverified profile only as an explicitly labeled bounded proof attempt. + +**Completed 2026-08-23:** both bwrap-local profiles now declare `blocked` with +owner `sand-boxer` and evidence `GLAS-IN-0002`; the remote profile declares +`unverified`. Gateway resolution rejects blocked profiles before sandbox +creation while retaining unverified attempts and their label in evidence. + +## Verify and publish the clarified behavior + +```task +id: GLAS-WP-0008-T03 +status: done +priority: high +``` + +Cover the schema, catalog, CLI, gateway refusal, evidence, and compatibility +path with tests. Update scope and execution-profile documentation, close +`GLAS-IN-0005`, synchronize State Hub, publish, and report the behavior change +to known consumers. + +**Completed 2026-08-23:** CLI text/JSON report readiness without calling the +catalog executable, documentation and examples reflect the current states, +and the 63-test suite plus packaged catalog validation pass. `GLAS-IN-0005` is +absorbed by this workplan. The remaining local runtime dependency was already +handed off as live residual `GLAS-IN-0002`; no new residual was created.