test-driver/research/findings
tegwick 5734b280c6 T06: out-of-band self-verification
Checks written as plain functions over a serialized Evidence Pack, outside the
framework - no Oracle, no Runner, no Verdict aggregation. 12 tests that they
hold, 12 that they can fail. All four td://self identifiers covered.

The substantive check is verdict reproducibility from S3 evidence alone,
asserted on failing runs as well as passing ones.

F-0003 (open): actor isolation leaves no trace in ordinary evidence - the
self-test catches a shared memory store only because the harness plants
per-actor canaries. Isolation is currently a property of a scenario written
to expose it, not of runs in general. The mirror-image case is noted too: a
guarantee enforced by construction cannot be verified by observing real runs,
so four green self-tests are not four equivalent proofs. Carried to T10.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 1629012@bnt-lap001
Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
2026-08-22 23:38:23 +02:00
..
F-0001-milestone-sequence-drift.md T01: establish canonical milestone sequence, record F-0001 2026-08-22 23:07:31 +02:00
F-0002-scenario-coverage-gap.md T05: the lab and its labelled mutation catalogue 2026-08-22 23:31:22 +02:00
F-0003-isolation-needs-canaries.md T06: out-of-band self-verification 2026-08-22 23:38:23 +02:00
README.md T01: establish canonical milestone sequence, record F-0001 2026-08-22 23:07:31 +02:00

Framework Findings

Findings about test-driver itself, kept strictly separate from findings about any system under test. Classes are defined in docs/TestDriverImprovementLoop.md § 9:

PRODUCT_DEFECT · TEST_DEFECT · MECHANICAL_ADAPTATION · SEMANTIC_CHANGE · CONCEPT_DRIFT · FRAMEWORK_LIMITATION · EVIDENCE_FAILURE · UNNECESSARY_COMPLEXITY

File naming: F-NNNN-<slug>.md. Identifiers are stable and never reused.

Each finding carries frontmatter (id, class, status, dates, originating workplan/task) and, when resolved, states which of the three §5 resolution paths was taken. A finding that leaves an open question records it under "Residual" and names the task that will answer it — residuals are not parked in prose.

The wider research control plane (hypotheses/, experiments/, concepts/, decisions/) is built in TD-WP-0002-T03.