test-driver/research/findings/F-0003-isolation-needs-canaries.md

---
id: F-0003
type: framework-finding
class: FRAMEWORK_LIMITATION
status: resolved
discovered: "2026-08-22"
discovered_by: TD-WP-0002-T06
workplan: TD-WP-0002
task: TD-WP-0002-T06
resolved: "2026-08-23"
resolved_in: TD-WP-0002-T10
---

# F-0003 — Actor isolation is only observable if the scenario plants canaries

## Observation

Writing `td://self/actor-isolation` out-of-band exposed something the concept
model does not acknowledge: **an isolation violation leaves no trace in ordinary
evidence.**

The self-test only detects a shared memory store because the test harness first
gives each actor a distinguishable private secret and then looks for it in the
wrong places. Remove that instrumentation and a run in which Alice, Bob and Carol
share one memory dict produces an Evidence Pack indistinguishable from a correct
one — same observations, same verdicts, same everything.

So the guarantee holds in the reference scenario *because it was instrumented to
be checkable*, not because runs generally reveal it.

## Why it matters

`INTENT.md` and the Concept Model treat actor isolation as a property of the
framework. As implemented it is a property of a *scenario that was written to
expose it*. Every ordinary scenario — the ones users will actually write —
carries no canaries, and for those the framework's central multi-user claim is
asserted rather than verified.

This matters more once actors are agentic (T07). A shared session, a cached HTTP
client, or a module-level default is exactly how isolation breaks in practice,
and none of those announce themselves.

## A second, related limitation

`td://self/oracle-independence` has a mirror-image problem. The framework
*prevents* an actor from collecting S2/S3 evidence — `Runner` raises
`CollectorIndependenceError` — so no real run can produce the violating artefact.
The check is therefore exercised against a hand-built pack.

That is the right architecture and a weak self-test at once: **a guarantee
enforced by construction cannot be verified by observing real runs.** The suite
now asserts both halves — that the door is locked, and that we would notice
someone coming through the window — but the second half tests the check, not the
system.

Worth stating plainly rather than counting four green self-tests as four
equivalent proofs. They are not equivalent.

## Resolution (T10)

Candidate 3 — **both**.

Every `Actor` is seeded at construction with an automatic private marker
(`Actor.canary`), and `Runner._isolation_violations` examines every actor on
**every** scenario, recording the verdict as S3 evidence. An isolation violation
now leaves a trace whether or not anyone thought to look for one.

Two permanent regressions stand behind it:

- `tests/test_reference_scenario.py::test_every_run_records_a_verdict_on_isolation`
- `tests/selfverification/test_checks_can_fail.py::test_a_leak_is_caught_without_the_test_planting_anything`
  — an actor holding another's marker is caught by the ordinary run, with the
  harness planting nothing.

The second limitation stands as recorded: oracle independence is enforced by
construction, so it is verified once rather than per run. That asymmetry is real
and is now stated in the fitness map rather than hidden behind two equally green
self-tests.

This finding closed the first Concept–Implementation Fitness Loop — see
`history/2026-08-23-td-wp-0002-gate-review.md` § 4.

## Candidate resolutions considered

1. **Automatic canaries** — the runtime seeds each actor with a unique private
   marker at construction and the isolation check runs on every Evidence Pack,
   making isolation continuously observable rather than specially tested.
   Cheap, and the obvious first move.
2. **Accept and document** — isolation is enforced by construction, and the
   self-test covers the construction rather than each run. Honest, but leaves the
   multi-user claim resting on code review.
3. **Both** — canaries for observability, plus an explicit statement that some
   guarantees are structural and are verified once rather than per run.

Carried to `TD-WP-0002-T10`, where it should be answered together with the wider
question of which guarantees are structural and which are observed. Reassess
after T07, when agentic actors make the failure mode concrete rather than
theoretical.