Normalize Glas execution evidence
Assistant: codex Assistant-Model: gpt-5.6-sol Assistant-Session: 01a028de-e2c8-7732-8521-46a7fc5db82f
This commit is contained in:
parent
b933cf52c8
commit
7e81545b24
13 changed files with 528 additions and 33 deletions
|
|
@ -2,16 +2,15 @@
|
|||
# Custodian Brief — activity-core
|
||||
|
||||
**Domain:** infotech
|
||||
**Last synced:** 2026-08-22 21:02 UTC
|
||||
**Last synced:** 2026-08-22 21:11 UTC
|
||||
**State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)*
|
||||
|
||||
## Active Workstreams
|
||||
|
||||
### Adopt the Glas profile-driven execution contract
|
||||
Progress: 3/5 done | workplan_id: `256dad13-28b4-5361-ab8a-7d1373a5d14b`
|
||||
Progress: 4/5 done | workplan_id: `256dad13-28b4-5361-ab8a-7d1373a5d14b`
|
||||
|
||||
**Open tasks:**
|
||||
- ! Record normalized execution evidence `48ff9bca`
|
||||
- ! Prove on one definition `e71423f9`
|
||||
|
||||
### Production automation execution reliability cleanup
|
||||
|
|
|
|||
|
|
@ -198,7 +198,7 @@
|
|||
| task | ACTIVITY-WP-0032-T01 | done | — | workplans/ACTIVITY-WP-0032-glas-profile-execution-contract.md |
|
||||
| task | ACTIVITY-WP-0032-T02 | done | — | workplans/ACTIVITY-WP-0032-glas-profile-execution-contract.md |
|
||||
| task | ACTIVITY-WP-0032-T03 | done | — | workplans/ACTIVITY-WP-0032-glas-profile-execution-contract.md |
|
||||
| task | ACTIVITY-WP-0032-T04 | wait | — | workplans/ACTIVITY-WP-0032-glas-profile-execution-contract.md |
|
||||
| task | ACTIVITY-WP-0032-T04 | done | — | workplans/ACTIVITY-WP-0032-glas-profile-execution-contract.md |
|
||||
| task | ACTIVITY-WP-0032-T05 | wait | — | workplans/ACTIVITY-WP-0032-glas-profile-execution-contract.md |
|
||||
| task | ACTIVITY-WP-0033-T01 | done | — | workplans/ACTIVITY-WP-0033-sbom-catchup-retry-boundary.md |
|
||||
| task | ACTIVITY-WP-0033-T02 | done | — | workplans/ACTIVITY-WP-0033-sbom-catchup-retry-boundary.md |
|
||||
|
|
|
|||
|
|
@ -67,6 +67,33 @@ workplan task file. Not an issue-core or Forgejo ticket.
|
|||
{ "result": { "path": "briefs/…", "ok": true }, "worker_id": "…" }
|
||||
```
|
||||
|
||||
Glas consumers submit its `GatewayResult` directly. Activity Core stores the
|
||||
safe evidence envelope and drops direct-caller output:
|
||||
|
||||
```json
|
||||
{
|
||||
"worker_id": "glas-worker@railiance01",
|
||||
"result": {
|
||||
"ok": true,
|
||||
"evidence": {
|
||||
"request_id": "…",
|
||||
"contract_version": "1.0",
|
||||
"profile_ref": "harness.agent-dev@1.0.0",
|
||||
"rein_id": "rein-aharness",
|
||||
"rein_version": "0.1.0",
|
||||
"resolved_model": "claude-sonnet-4-6",
|
||||
"sandbox_profile": "profile.bwrap-local",
|
||||
"outcome": "succeeded",
|
||||
"duration_s": 12.4,
|
||||
"tokens_spent": 321,
|
||||
"tool_events_count": 0,
|
||||
"tool_events_completeness": "unavailable"
|
||||
},
|
||||
"tool_output": "dropped before persistence"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{ "error": "llm timeout", "worker_id": "…", "reopen": false }
|
||||
```
|
||||
|
|
@ -74,6 +101,15 @@ workplan task file. Not an issue-core or Forgejo ticket.
|
|||
If `reopen: true` and `attempt < max_attempts` (env `OPS_RUN_MAX_ATTEMPTS`, default 3),
|
||||
state returns to `open`; else `failed`.
|
||||
|
||||
Both completion and failure normalize through the same allowlist. The persisted
|
||||
key is `result.execution_evidence`; it contains the Glas 1.0 constellation,
|
||||
outcome, bounded measurements, commit/artifact references, and allowlisted
|
||||
organizational refs. Zero stays zero and unsupported event visibility stays
|
||||
`unavailable`; absent measurements remain absent. `tool_output`, `tool_error`,
|
||||
prompts, messages, provider responses, credential fields, unknown nested blobs,
|
||||
and undeclared refs are never stored. Read projections normalize historic rows
|
||||
again before returning them.
|
||||
|
||||
## Emit path
|
||||
|
||||
On `emit_tasks` (when `OPS_RUN_QUEUE_ENABLED` is truthy, **default true**):
|
||||
|
|
@ -100,13 +136,16 @@ Never requires Forgejo or issue-core for the claim path.
|
|||
| ----- | ------ |
|
||||
| `ops_runs[]` | matched by `triggering_event_id == run_id` (or contains), else time window |
|
||||
| `artifacts[]` | from `ops_runs.result` (`path`, `head_after`, `target_repo`) |
|
||||
| `ops_runs[].execution_evidence` | compact Glas profile/rein/model/sandbox/outcome evidence |
|
||||
| Forgejo URL | `FORGEJO_WEB_BASE` (default `https://forgejo.coulomb.social`) + org + repo + path + ref |
|
||||
|
||||
Run detail: `GET /ops/automations/{id}/runs/{run_id}` and
|
||||
`/ops/ui/automations/{id}/runs/{run_id}`.
|
||||
|
||||
Executor `result` should include at least: `ok`, `path`, `head_after`,
|
||||
`target_repo`, `committed`. No prompts or raw model output.
|
||||
`target_repo`, `committed`. A Glas-backed executor should submit its full
|
||||
`GatewayResult`; Activity Core extracts the safe envelope. No prompts or raw
|
||||
model output are persisted or returned.
|
||||
|
||||
## Auth
|
||||
|
||||
|
|
|
|||
|
|
@ -465,6 +465,15 @@ curl -sS -X POST "http://localhost:8010/ops-runs/expire-leases" \
|
|||
| `OPS_RUN_SLA_HOURS` | `1` | Stuck threshold in status |
|
||||
| `ACTIVITY_CORE_WORKER_TOKEN` | unset | Harness claim auth |
|
||||
|
||||
Glas-backed workers may return the Glas 1.0 `GatewayResult` on complete or
|
||||
fail. Activity Core persists only `result.execution_evidence` plus the existing
|
||||
compact artifact fields; direct tool/model output and unknown provider blobs
|
||||
are dropped. The run detail API and UI expose profile, rein, model, sandbox,
|
||||
outcome, failure stage, duration, token counts, and commit/artifact references
|
||||
from that safe envelope. If those fields are missing, inspect the executor
|
||||
payload before assuming Glas ran; do not paste raw provider output into
|
||||
`ops_runs.result`.
|
||||
|
||||
**Railiance rollout (T07):** full checklist with image import, migrate job,
|
||||
smoke trigger, claim test, and dual-path residual:
|
||||
|
||||
|
|
|
|||
180
src/activity_core/glas_evidence.py
Normal file
180
src/activity_core/glas_evidence.py
Normal file
|
|
@ -0,0 +1,180 @@
|
|||
"""Compact, non-secret Glas execution evidence for ops_run results.
|
||||
|
||||
Glas owns the versioned evidence envelope. Activity Core accepts that envelope
|
||||
at its completion boundary, keeps only contract fields needed by operators, and
|
||||
never persists direct-caller ``tool_output`` / ``tool_error`` or provider blobs.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from typing import Any
|
||||
|
||||
from activity_core.glas_profile import normalise_execution_refs
|
||||
|
||||
_LEGACY_RESULT_FIELDS = frozenset(
|
||||
{
|
||||
"ok",
|
||||
"approach",
|
||||
"path",
|
||||
"report",
|
||||
"date",
|
||||
"wrote",
|
||||
"committed",
|
||||
"pushed",
|
||||
"skipped_existing",
|
||||
"head_after",
|
||||
"target_repo",
|
||||
"collection_candidates",
|
||||
"reason",
|
||||
"error",
|
||||
}
|
||||
)
|
||||
_EVIDENCE_STRING_FIELDS = frozenset(
|
||||
{
|
||||
"request_id",
|
||||
"correlation_id",
|
||||
"actor",
|
||||
"project",
|
||||
"target_repo",
|
||||
"contract_version",
|
||||
"profile_ref",
|
||||
"rein_id",
|
||||
"rein_version",
|
||||
"model_route",
|
||||
"resolved_model",
|
||||
"sandbox_profile",
|
||||
"sandbox_id",
|
||||
"tool_profile",
|
||||
"outcome",
|
||||
"failure_stage",
|
||||
"error",
|
||||
"started_at",
|
||||
"finished_at",
|
||||
"commit_sha",
|
||||
"tool_events_completeness",
|
||||
}
|
||||
)
|
||||
_EVIDENCE_NUMBER_FIELDS = frozenset(
|
||||
{"duration_s", "tokens_spent", "token_budget", "tool_events_count"}
|
||||
)
|
||||
_EVIDENCE_ENUMS = {
|
||||
"outcome": frozenset({"succeeded", "failed", "refused"}),
|
||||
"failure_stage": frozenset(
|
||||
{
|
||||
"resolution",
|
||||
"sandbox_create",
|
||||
"session_start",
|
||||
"execution",
|
||||
"session_end",
|
||||
"teardown",
|
||||
}
|
||||
),
|
||||
"tool_events_completeness": frozenset({"complete", "partial", "unavailable"}),
|
||||
}
|
||||
|
||||
|
||||
def _compact_scalar(value: Any, *, max_length: int = 2000) -> Any | None:
|
||||
if isinstance(value, bool):
|
||||
return value
|
||||
if (
|
||||
isinstance(value, (int, float))
|
||||
and not isinstance(value, bool)
|
||||
and math.isfinite(value)
|
||||
):
|
||||
return value
|
||||
if isinstance(value, str):
|
||||
return value[:max_length]
|
||||
return None
|
||||
|
||||
|
||||
def _compact_legacy_value(value: Any) -> Any | None:
|
||||
scalar = _compact_scalar(value)
|
||||
if scalar is not None:
|
||||
return scalar
|
||||
if isinstance(value, list):
|
||||
compact = [_compact_scalar(item, max_length=500) for item in value[:20]]
|
||||
return [item for item in compact if item is not None]
|
||||
return None
|
||||
|
||||
|
||||
def normalise_execution_evidence(raw: Any) -> dict[str, Any]:
|
||||
"""Return the safe subset of Glas ``ExecutionEvidence`` contract 1.0."""
|
||||
if not isinstance(raw, dict):
|
||||
return {}
|
||||
|
||||
evidence: dict[str, Any] = {}
|
||||
for key in _EVIDENCE_STRING_FIELDS:
|
||||
value = raw.get(key)
|
||||
if isinstance(value, str):
|
||||
allowed = _EVIDENCE_ENUMS.get(key)
|
||||
if allowed is not None and value not in allowed:
|
||||
continue
|
||||
evidence[key] = value[:2000] if key == "error" else value[:500]
|
||||
|
||||
for key in _EVIDENCE_NUMBER_FIELDS:
|
||||
value = raw.get(key)
|
||||
if (
|
||||
isinstance(value, (int, float))
|
||||
and not isinstance(value, bool)
|
||||
and math.isfinite(value)
|
||||
and value >= 0
|
||||
):
|
||||
evidence[key] = value
|
||||
|
||||
artifacts = raw.get("artifacts")
|
||||
if isinstance(artifacts, list):
|
||||
evidence["artifacts"] = [
|
||||
item[:1000]
|
||||
for item in artifacts[:20]
|
||||
if isinstance(item, str) and item
|
||||
]
|
||||
|
||||
refs = normalise_execution_refs(raw.get("refs"))
|
||||
if refs:
|
||||
evidence["refs"] = refs
|
||||
return evidence
|
||||
|
||||
|
||||
def _normalise_artifact_urls(raw: Any) -> list[dict[str, str]]:
|
||||
if not isinstance(raw, list):
|
||||
return []
|
||||
urls: list[dict[str, str]] = []
|
||||
for item in raw[:10]:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
url = item.get("url")
|
||||
if not isinstance(url, str) or not url.startswith("https://"):
|
||||
continue
|
||||
urls.append(
|
||||
{
|
||||
"kind": str(item.get("kind") or "link")[:80],
|
||||
"label": str(item.get("label") or "artifact")[:120],
|
||||
"url": url[:2000],
|
||||
}
|
||||
)
|
||||
return urls
|
||||
|
||||
|
||||
def normalise_ops_result(raw: Any) -> dict[str, Any]:
|
||||
"""Allowlist legacy completion facts plus normalized Glas evidence."""
|
||||
if not isinstance(raw, dict):
|
||||
return {}
|
||||
|
||||
result: dict[str, Any] = {}
|
||||
for key in _LEGACY_RESULT_FIELDS:
|
||||
value = _compact_legacy_value(raw.get(key))
|
||||
if value is not None:
|
||||
result[key] = value
|
||||
|
||||
artifact_urls = _normalise_artifact_urls(raw.get("artifact_urls"))
|
||||
if artifact_urls:
|
||||
result["artifact_urls"] = artifact_urls
|
||||
|
||||
raw_evidence = raw.get("execution_evidence")
|
||||
if not isinstance(raw_evidence, dict):
|
||||
raw_evidence = raw.get("evidence")
|
||||
evidence = normalise_execution_evidence(raw_evidence)
|
||||
if evidence:
|
||||
result["execution_evidence"] = evidence
|
||||
return result
|
||||
|
|
@ -123,9 +123,13 @@ def normalise_execution_refs(raw: Any) -> dict[str, Any]:
|
|||
value = raw.get(key)
|
||||
if key in _LIST_REF_KEYS:
|
||||
if isinstance(value, (list, tuple)):
|
||||
items = [str(item).strip() for item in value if str(item).strip()]
|
||||
items = [
|
||||
item.strip()[:500]
|
||||
for item in value[:20]
|
||||
if isinstance(item, str) and item.strip()
|
||||
]
|
||||
if items:
|
||||
refs[key] = items
|
||||
elif isinstance(value, str) and value.strip():
|
||||
refs[key] = value.strip()
|
||||
refs[key] = value.strip()[:500]
|
||||
return refs
|
||||
|
|
|
|||
|
|
@ -15,6 +15,7 @@ from activity_core.glas_profile import (
|
|||
normalise_execution_refs,
|
||||
resolve_execution_selector,
|
||||
)
|
||||
from activity_core.glas_evidence import normalise_ops_result
|
||||
from activity_core.orm import OpsRun
|
||||
from activity_core.rules.models import TaskSpec
|
||||
|
||||
|
|
@ -98,7 +99,7 @@ def ops_run_to_dict(row: OpsRun) -> dict[str, Any]:
|
|||
"approach_hint": row.approach_hint,
|
||||
"harness_profile_ref": row.harness_profile_ref,
|
||||
"execution_refs": dict(row.execution_refs or {}),
|
||||
"result": dict(row.result or {}),
|
||||
"result": normalise_ops_result(row.result),
|
||||
"created_at": row.created_at.isoformat() if row.created_at else None,
|
||||
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
|
||||
}
|
||||
|
|
@ -278,7 +279,7 @@ async def complete_ops_run(
|
|||
now = datetime.now(timezone.utc)
|
||||
row.state = "succeeded"
|
||||
row.lease_until = None
|
||||
row.result = dict(result or {})
|
||||
row.result = normalise_ops_result(result)
|
||||
row.updated_at = now
|
||||
return row
|
||||
|
||||
|
|
@ -298,7 +299,7 @@ async def fail_ops_run(
|
|||
if row.state != "claimed" or row.claim_owner != worker_id:
|
||||
return None
|
||||
now = datetime.now(timezone.utc)
|
||||
payload = dict(result or {})
|
||||
payload = normalise_ops_result(result)
|
||||
if error:
|
||||
payload["error"] = error[:2000]
|
||||
row.result = payload
|
||||
|
|
|
|||
|
|
@ -15,6 +15,7 @@ from urllib.parse import quote
|
|||
from sqlalchemy import Select, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from activity_core.glas_evidence import normalise_ops_result
|
||||
from activity_core.orm import ActivityRun, OpsRun, TaskSpawnLog
|
||||
|
||||
# Default public Forgejo web base (no trailing slash). Override with FORGEJO_WEB_BASE.
|
||||
|
|
@ -136,13 +137,33 @@ def artifacts_from_ops_result(
|
|||
|
||||
|
||||
def _ops_summary(row: OpsRun) -> dict[str, Any]:
|
||||
result = dict(row.result or {})
|
||||
# Drop bulky keys if ever present
|
||||
for bad in ("prompt", "raw_output", "messages", "token"):
|
||||
result.pop(bad, None)
|
||||
# Normalize again on read so historic rows cannot leak pre-allowlist blobs.
|
||||
result = normalise_ops_result(row.result)
|
||||
artifacts = artifacts_from_ops_result(
|
||||
result, target_repo=row.target_repo, title=row.title
|
||||
)
|
||||
execution_evidence = dict(result.get("execution_evidence") or {})
|
||||
compact_result = {
|
||||
k: result[k]
|
||||
for k in (
|
||||
"ok",
|
||||
"approach",
|
||||
"path",
|
||||
"date",
|
||||
"wrote",
|
||||
"committed",
|
||||
"pushed",
|
||||
"skipped_existing",
|
||||
"head_after",
|
||||
"target_repo",
|
||||
"collection_candidates",
|
||||
"reason",
|
||||
"error",
|
||||
)
|
||||
if k in result
|
||||
}
|
||||
if execution_evidence:
|
||||
compact_result["execution_evidence"] = execution_evidence
|
||||
return {
|
||||
"id": str(row.id),
|
||||
"state": row.state,
|
||||
|
|
@ -155,26 +176,10 @@ def _ops_summary(row: OpsRun) -> dict[str, Any]:
|
|||
"approach_hint": row.approach_hint,
|
||||
"harness_profile_ref": row.harness_profile_ref,
|
||||
"execution_refs": dict(row.execution_refs or {}),
|
||||
"execution_evidence": execution_evidence,
|
||||
"created_at": row.created_at.isoformat() if row.created_at else None,
|
||||
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
|
||||
"result": {
|
||||
k: result[k]
|
||||
for k in (
|
||||
"ok",
|
||||
"approach",
|
||||
"path",
|
||||
"date",
|
||||
"wrote",
|
||||
"committed",
|
||||
"pushed",
|
||||
"skipped_existing",
|
||||
"head_after",
|
||||
"target_repo",
|
||||
"collection_candidates",
|
||||
"reason",
|
||||
)
|
||||
if k in result
|
||||
},
|
||||
"result": compact_result,
|
||||
"artifacts": artifacts,
|
||||
}
|
||||
|
||||
|
|
|
|||
115
tests/test_glas_evidence.py
Normal file
115
tests/test_glas_evidence.py
Normal file
|
|
@ -0,0 +1,115 @@
|
|||
"""Normalized Glas evidence boundary (ACTIVITY-WP-0032-T04)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
from activity_core.glas_evidence import (
|
||||
normalise_execution_evidence,
|
||||
normalise_ops_result,
|
||||
)
|
||||
|
||||
|
||||
def _gateway_result() -> dict:
|
||||
return {
|
||||
"ok": True,
|
||||
"evidence": {
|
||||
"request_id": "request-1",
|
||||
"correlation_id": "corr-1",
|
||||
"actor": "agt",
|
||||
"project": "activity-core",
|
||||
"target_repo": "activity-core",
|
||||
"contract_version": "1.0",
|
||||
"profile_ref": "harness.agent-dev@1.0.0",
|
||||
"rein_id": "rein-aharness",
|
||||
"rein_version": "0.1.0",
|
||||
"model_route": "claude-code-cli",
|
||||
"resolved_model": "claude-sonnet-4-6",
|
||||
"sandbox_profile": "profile.bwrap-local",
|
||||
"sandbox_id": "sandbox-1",
|
||||
"tool_profile": "green-commit-only",
|
||||
"outcome": "succeeded",
|
||||
"started_at": "2026-08-22T20:00:00Z",
|
||||
"finished_at": "2026-08-22T20:00:05Z",
|
||||
"duration_s": 5.0,
|
||||
"tokens_spent": 0,
|
||||
"token_budget": 60000,
|
||||
"commit_sha": "deadbeef",
|
||||
"artifacts": ["README.md"],
|
||||
"tool_events_count": 0,
|
||||
"tool_events_completeness": "unavailable",
|
||||
"refs": {
|
||||
"assignment_ref": "assignment:42",
|
||||
"goal_refs": ["goal:42@1", {"credential": "must-drop"}],
|
||||
"api_key": "must-drop",
|
||||
},
|
||||
"raw_prompt": "must-drop",
|
||||
"provider_response": {"must": "drop"},
|
||||
},
|
||||
"tool_output": "sensitive direct model output",
|
||||
"tool_error": "sensitive provider exception",
|
||||
"provider_blob": {"credential": "must-drop"},
|
||||
}
|
||||
|
||||
|
||||
def test_normalises_gateway_result_and_drops_direct_output() -> None:
|
||||
result = normalise_ops_result(_gateway_result())
|
||||
|
||||
assert result["ok"] is True
|
||||
evidence = result["execution_evidence"]
|
||||
assert evidence["profile_ref"] == "harness.agent-dev@1.0.0"
|
||||
assert evidence["rein_id"] == "rein-aharness"
|
||||
assert evidence["tokens_spent"] == 0
|
||||
assert evidence["tool_events_count"] == 0
|
||||
assert evidence["tool_events_completeness"] == "unavailable"
|
||||
assert evidence["refs"] == {
|
||||
"assignment_ref": "assignment:42",
|
||||
"goal_refs": ["goal:42@1"],
|
||||
}
|
||||
|
||||
serialized = json.dumps(result)
|
||||
for forbidden in (
|
||||
"tool_output",
|
||||
"tool_error",
|
||||
"provider_blob",
|
||||
"raw_prompt",
|
||||
"provider_response",
|
||||
"credential",
|
||||
):
|
||||
assert forbidden not in serialized
|
||||
|
||||
|
||||
def test_rejects_invalid_enums_and_non_finite_measurements() -> None:
|
||||
evidence = normalise_execution_evidence(
|
||||
{
|
||||
"outcome": "maybe",
|
||||
"failure_stage": "provider-internal",
|
||||
"duration_s": float("nan"),
|
||||
"tokens_spent": -1,
|
||||
"tool_events_completeness": "unknown-ish",
|
||||
}
|
||||
)
|
||||
|
||||
assert evidence == {}
|
||||
|
||||
|
||||
def test_keeps_legacy_artifact_fields_but_not_unknown_blobs() -> None:
|
||||
result = normalise_ops_result(
|
||||
{
|
||||
"ok": True,
|
||||
"path": "briefs/report.md",
|
||||
"head_after": "deadbeef",
|
||||
"target_repo": "activity-core",
|
||||
"artifact_urls": [
|
||||
{"kind": "report", "label": "Report", "url": "https://example.test/a"},
|
||||
{"kind": "unsafe", "label": "Unsafe", "url": "file:///tmp/key"},
|
||||
],
|
||||
"messages": [{"role": "user", "content": "secret"}],
|
||||
}
|
||||
)
|
||||
|
||||
assert result["path"] == "briefs/report.md"
|
||||
assert result["artifact_urls"] == [
|
||||
{"kind": "report", "label": "Report", "url": "https://example.test/a"}
|
||||
]
|
||||
assert "messages" not in result
|
||||
|
|
@ -148,6 +148,13 @@ class TestExecutionRefs:
|
|||
|
||||
assert refs == {"resource_envelope_refs": ["ok"]}
|
||||
|
||||
def test_drops_non_string_items_in_reference_lists(self) -> None:
|
||||
refs = normalise_execution_refs(
|
||||
{"goal_refs": ["goal:1@1", {"credential": "must-drop"}, 17]}
|
||||
)
|
||||
|
||||
assert refs == {"goal_refs": ["goal:1@1"]}
|
||||
|
||||
@pytest.mark.parametrize("bad", [None, "refs", ["a"], 5])
|
||||
def test_non_dict_becomes_empty(self, bad) -> None:
|
||||
assert normalise_execution_refs(bad) == {}
|
||||
|
|
|
|||
|
|
@ -225,6 +225,47 @@ async def test_claim_and_complete_roundtrip(monkeypatch: pytest.MonkeyPatch) ->
|
|||
assert open_run.result["path"] == "briefs/x.md"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_complete_persists_only_normalized_glas_evidence() -> None:
|
||||
from activity_core import ops_run_queue as oq
|
||||
|
||||
row = MagicMock()
|
||||
row.state = "claimed"
|
||||
row.claim_owner = "worker-1"
|
||||
row.result = {}
|
||||
session = AsyncMock()
|
||||
session.get = AsyncMock(return_value=row)
|
||||
|
||||
done = await oq.complete_ops_run(
|
||||
session,
|
||||
uuid.uuid4(),
|
||||
worker_id="worker-1",
|
||||
result={
|
||||
"ok": True,
|
||||
"evidence": {
|
||||
"request_id": "request-1",
|
||||
"profile_ref": "harness.agent-dev@1.0.0",
|
||||
"rein_id": "rein-aharness",
|
||||
"outcome": "succeeded",
|
||||
"duration_s": 0,
|
||||
},
|
||||
"tool_output": "must not persist",
|
||||
},
|
||||
)
|
||||
|
||||
assert done is row
|
||||
assert row.result == {
|
||||
"ok": True,
|
||||
"execution_evidence": {
|
||||
"request_id": "request-1",
|
||||
"profile_ref": "harness.agent-dev@1.0.0",
|
||||
"rein_id": "rein-aharness",
|
||||
"outcome": "succeeded",
|
||||
"duration_s": 0,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fail_reopen_under_max_attempts(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
from activity_core import ops_run_queue as oq
|
||||
|
|
@ -268,6 +309,44 @@ async def test_fail_permanent_at_max_attempts(monkeypatch: pytest.MonkeyPatch) -
|
|||
assert row.state == "failed"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fail_persists_redacted_failure_evidence() -> None:
|
||||
from activity_core import ops_run_queue as oq
|
||||
|
||||
row = MagicMock()
|
||||
row.state = "claimed"
|
||||
row.claim_owner = "worker-1"
|
||||
row.attempt = 1
|
||||
row.result = {}
|
||||
session = AsyncMock()
|
||||
session.get = AsyncMock(return_value=row)
|
||||
|
||||
failed = await oq.fail_ops_run(
|
||||
session,
|
||||
uuid.uuid4(),
|
||||
worker_id="worker-1",
|
||||
error="execution refused",
|
||||
result={
|
||||
"ok": False,
|
||||
"evidence": {
|
||||
"request_id": "request-failed",
|
||||
"profile_ref": "harness.agent-dev@1.0.0",
|
||||
"rein_id": "rein-aharness",
|
||||
"outcome": "failed",
|
||||
"failure_stage": "execution",
|
||||
"error": "execution failed; inspect direct caller error",
|
||||
"duration_s": 3.5,
|
||||
},
|
||||
"tool_error": "raw provider failure must not persist",
|
||||
},
|
||||
)
|
||||
|
||||
assert failed is row
|
||||
assert row.result["error"] == "execution refused"
|
||||
assert row.result["execution_evidence"]["failure_stage"] == "execution"
|
||||
assert "tool_error" not in row.result
|
||||
|
||||
|
||||
def test_label_filter_any_vs_all() -> None:
|
||||
"""Document labels_mode semantics used by claim_ops_runs."""
|
||||
row_labels = {"automated", "research-brief"}
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@ from datetime import datetime, timedelta, timezone
|
|||
from unittest.mock import MagicMock
|
||||
|
||||
from activity_core.run_artifacts import (
|
||||
_ops_summary,
|
||||
artifacts_from_ops_result,
|
||||
build_forgejo_blob_url,
|
||||
match_ops_runs_to_activity_run,
|
||||
|
|
@ -88,3 +89,42 @@ def test_match_ops_runs_window_fallback() -> None:
|
|||
matched = match_ops_runs_to_activity_run(ar, [far, near])
|
||||
assert near in matched
|
||||
assert far not in matched
|
||||
|
||||
|
||||
def test_ops_summary_exposes_compact_execution_constellation() -> None:
|
||||
now = datetime(2026, 8, 22, 20, 0, tzinfo=timezone.utc)
|
||||
row = MagicMock()
|
||||
row.id = uuid.uuid4()
|
||||
row.state = "succeeded"
|
||||
row.title = "Profiled run"
|
||||
row.target_repo = "activity-core"
|
||||
row.claim_owner = "glas-worker@railiance01"
|
||||
row.attempt = 1
|
||||
row.triggering_event_id = "event-1"
|
||||
row.source_id = "profiled"
|
||||
row.approach_hint = "legacy-only"
|
||||
row.harness_profile_ref = "harness.agent-dev@1.0.0"
|
||||
row.execution_refs = {"goal_refs": ["goal:42@1"]}
|
||||
row.created_at = now
|
||||
row.updated_at = now
|
||||
row.result = {
|
||||
"ok": True,
|
||||
"execution_evidence": {
|
||||
"profile_ref": "harness.agent-dev@1.0.0",
|
||||
"rein_id": "rein-aharness",
|
||||
"rein_version": "0.1.0",
|
||||
"resolved_model": "claude-sonnet-4-6",
|
||||
"sandbox_profile": "profile.bwrap-local",
|
||||
"outcome": "succeeded",
|
||||
"duration_s": 5.2,
|
||||
},
|
||||
"raw_output": "must-drop-even-for-historic-rows",
|
||||
}
|
||||
|
||||
summary = _ops_summary(row)
|
||||
|
||||
assert summary["execution_evidence"]["rein_id"] == "rein-aharness"
|
||||
assert summary["result"]["execution_evidence"]["profile_ref"] == (
|
||||
"harness.agent-dev@1.0.0"
|
||||
)
|
||||
assert "raw_output" not in str(summary)
|
||||
|
|
|
|||
|
|
@ -198,7 +198,7 @@ the local integration stack. Python compilation and `git diff --check` passed.
|
|||
|
||||
```task
|
||||
id: ACTIVITY-WP-0032-T04
|
||||
status: wait
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "48ff9bca-6251-59ff-8b6f-6b0d79252624"
|
||||
```
|
||||
|
|
@ -209,6 +209,23 @@ Depends on T02. Glas emits normalized, non-secret execution evidence. Land it in
|
|||
rein. Keep the field allowlist discipline from ACTIVITY-WP-0031: no provider
|
||||
blobs, no credential material in run artefacts.
|
||||
|
||||
Done 2026-08-22 against Glas contract 1.0. Both queue completion and failure
|
||||
normalize the returned `GatewayResult` before persistence. Activity Core keeps
|
||||
the declared `ExecutionEvidence` constellation, outcome/failure stage, bounded
|
||||
measurements, commit/artifact references, and allowlisted organizational refs
|
||||
under `ops_runs.result.execution_evidence`. It preserves meaningful zero values
|
||||
and explicit `unavailable`, while dropping direct `tool_output`, `tool_error`,
|
||||
prompts, messages, provider responses, unknown nested blobs, and undeclared
|
||||
refs. Historic results are normalized again at read time.
|
||||
|
||||
The per-run API/UI projection now includes `execution_evidence`, so operators
|
||||
can see profile, rein, resolved model, sandbox, outcome, duration, tokens, and
|
||||
artifact facts without provider payloads. Tests cover success, failure,
|
||||
redaction, invalid enums/measurements, unknown-versus-zero semantics, legacy
|
||||
artifact compatibility, and the run summary. Verification: 71 focused tests
|
||||
and 433 repository tests passed; the live NATS-to-Temporal bridge test remained
|
||||
deselected because it requires the local integration stack.
|
||||
|
||||
## Prove on one definition
|
||||
|
||||
```task
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue