test-driver/docs/TestDriverConceptModel.md

Test Driver Concept Model

Current scope, 2026-09-28: TD-WP-0003 removes speculative lifecycle scoring and declared Temperature. See settlement and compatibility.

Status: v0.1 Draft
Framework: test-driver

1. Purpose

test-driver is a use-case-driven verification framework for software systems that are developed under increasingly fluid, fast-moving and agentic development conditions.

Its core premise is that verification should evolve together with the behavior and implementation it protects:

Tests begin fluid and exploratory when software is changing quickly, adapt with agentic assistance while behavior stabilizes, and crystallize into deterministic test code when implementation and expected behavior become sufficiently mature.

The framework is intended to support, from one coherent conceptual model:

• integration testing,
• end-to-end and user-journey testing,
• multi-user interaction testing,
• authorization and security testing,
• resilience and failure testing,
• later, scale and performance testing.

The framework treats a test not primarily as source code, but as a verification asset that connects intended behavior to observed implementation behavior over time.

---

2. Core Principles

2.1 Intent precedes implementation

Tests ultimately protect intended behavior, not incidental implementation details.

A user-facing capability should therefore be described first in terms of purpose, actors, expected outcomes, claims and invariants rather than UI clicks or protocol calls.

2.2 Use cases are the primary behavioral source

The core source object is a UseCase.

A use case describes purposeful behavior involving one or more actors and a system under test. Integration, journey, multi-user and security tests are projections or transformations of that same use case rather than unrelated test suites.

2.3 Agenticity is a means, not the source of truth

Agents are particularly useful for:

• interpreting intent,
• navigating unstable interaction surfaces,
• discovering alternative paths,
• generating scenario variations,
• exploring adversarial behavior,
• adapting tests to legitimate mechanical changes,
• investigating and minimizing failures.

The ultimate verdict should, wherever possible, come from explicit and independent oracles rather than from an actor agent judging its own success.

2.4 Exploration precedes crystallization

Agentic tests should not remain agentic merely because they started that way.

As behavior and implementation become stable, agentic realizations are progressively hardened and finally crystallized into deterministic tests.

2.5–2.6 Evidence, not lifecycle scores

Retain observations and verdicts. Asset selection and retirement remain explicit maintainer choices; there is no energy score or automated retirement policy.

2.7 Implementation is not the truth

When a test and an implementation disagree, the framework must not automatically adapt the test to the implementation.

The disagreement may indicate:

1. an implementation defect,
2. an intentional requirement change,
3. a defective or outdated test,
4. an ambiguous condition requiring investigation.

---

3. Conceptual Layers

The concept model is organized into five layers:

1. Intent — what should be true.
2. Situation — under what circumstances it should be true.
3. Variation — how behavior should be explored.
4. Execution — what actually happened.
5. Evolution — how verification changes over time.

---

4. Intent Layer

4.1 UseCase

A UseCase describes purposeful interaction with a system from the perspective of one or more actors.

A use case should primarily describe:

• intent,
• participating actor roles,
• preconditions,
• relevant outcomes,
• behavioral claims,
• invariants.

It should avoid encoding unnecessary implementation mechanics.

Example:

Alice wants to share document D with Bob so that Bob can read it while Carol remains unable to access it.

4.2 Claim

A Claim is an expected behavior that should hold within a use case or scenario.

Examples:

Bob can read document D.
Bob cannot modify document D.
The sharing operation appears in the audit history.

Claims are typically local to a use case or capability.

4.3 Invariant

An Invariant is a property expected to remain true across many scenarios, implementations or use cases.

Examples:

Tenant isolation must be preserved.
Unauthorized actors must not obtain document contents.
Read permission must not imply write permission.
Audit records must not be silently omitted.

Security requirements are often best represented as invariants.

---

5. Situation Layer

5.1 Actor

An Actor is an independently acting user, system or external participant.

An actor has an epistemic and execution boundary containing at least:

• identity,
• role,
• tenant or organizational context,
• credentials,
• session,
• permissions,
• private memory,
• known resources,
• available interaction surfaces,
• allowed communication channels.

Actors in multi-user tests must not implicitly share knowledge unless the use case explicitly allows it.

5.2 Cast

A Cast is the set of actors participating in a scenario.

Example:

Alice — owner
Bob — intended recipient
Carol — unrelated user

5.3 World

A World is the relevant initial state in which a scenario begins.

It may contain:

• users and identities,
• tenants,
• resources,
• permissions,
• configuration,
• external services,
• clocks,
• feature flags,
• fixtures,
• dependency state.

5.4 Surface

A Surface is a mechanism through which an actor interacts with the system.

Examples:

• browser UI,
• HTTP API,
• CLI,
• mobile UI,
• message queue,
• email,
• webhook,
• external integration API.

5.5 Scenario

A Scenario is a concrete realization of a use case.

Conceptually:

Scenario = UseCase + Cast + World + Schedule + Surfaces + Variant

A scenario determines enough context for one or more actors to execute purposeful behavior.

5.6 Schedule

A Schedule defines causal and temporal relationships between actions.

Instead of assuming every scenario is a linear sequence, the framework should support a causal graph including relationships such as:

• must happen before,
• caused by,
• wait until,
• concurrent with,
• within time T,
• must never occur during interval T.

Schedules are central to multi-user and concurrency testing.

---

6. Variation Layer

6.1 Lens

A Lens is the verification perspective applied to a use case.

Initial lens canon:

Integration Lens

Do participating components communicate correctly?

Journey Lens

Can an actor achieve the intended outcome through the complete system?

Interaction Lens

Does the system remain correct when multiple independent actors interact with shared state?

Security Lens

Do claims and invariants survive unauthorized, manipulated, adversarial or logically invalid behavior?

Resilience Lens

Does the use case remain valid under component, dependency or network failures?

Scale Lens

Does the behavior remain correct and useful under realistic populations and load?

6.2 Variant

A Variant is an alternative realization of a scenario that preserves enough identity with its originating use case to be meaningfully compared.

Variants may differ in:

• actor roles,
• data,
• sequence,
• timing,
• interaction surface,
• environment,
• dependency state.

6.3 Mutation

A Mutation is a deliberate transformation of a scenario intended to probe robustness, correctness or security.

Initial mutation canon:

actor-substitution
resource-substitution
tenant-substitution
sequence-reordering
step-skipping
replay
repetition
concurrency
surface-substitution
invalid-state
privilege-mutation
dependency-failure
time-shift

Security scenarios should preferentially be derived as transformations of ordinary business use cases rather than maintained as a disconnected testing universe.

---

7. Execution Layer

7.1 Driver

A Driver exposes an interaction surface to an actor or deterministic test implementation.

Examples:

BrowserDriver
HttpDriver
CliDriver
MailDriver
QueueDriver

Drivers should support both agentic and deterministic realizations where practical.

7.2 Semantic Action

A Semantic Action describes a meaningful action independently of the mechanics needed to perform it.

Examples:

open_resource(R)
grant_access(Bob, READ)
revoke_access(Bob)
approve_invoice(I)

Semantic actions form an important bridge between agentic exploration and deterministic crystallization.

An agent may initially discover how a semantic action is performed through an unstable interface. Later the driver may provide a stable deterministic implementation of the same semantic action.

7.3 Actor Runtime

The Actor Runtime executes actor behavior while preserving actor isolation.

It is responsible for:

• identity/session isolation,
• agent or deterministic execution,
• memory isolation,
• surface access,
• action recording,
• evidence correlation.

7.4 Orchestrator

The Orchestrator coordinates:

• actors,
• scenario state,
• barriers,
• schedules,
• clocks,
• world setup and teardown,
• run lifecycle.

The orchestrator may know the complete world state even when individual actors do not.

7.5 Observer

An Observer captures facts about what happened without being the actor responsible for producing the behavior.

Observers may inspect:

• externally visible outcomes,
• API responses,
• domain state,
• logs,
• traces,
• metrics,
• audit records,
• messages,
• UI state.

7.6 Observation

An Observation is a captured fact produced during a run.

7.7 Evidence

Evidence is persisted observation material supporting later judgment, diagnosis and lineage.

A run should be able to produce an evidence pack containing, where applicable:

• run identifier,
• use-case and scenario version,
• system/component versions,
• environment,
• actor identities and roles,
• agent/runtime versions,
• random seeds,
• action timeline,
• screenshots or UI traces,
• requests and responses,
• domain-state observations,
• logs,
• metrics,
• distributed traces,
• oracle evaluations,
• verdict.

7.8 Oracle

An Oracle evaluates evidence against a claim or invariant.

Initial oracle categories:

• outcome oracle,
• state oracle,
• authorization oracle,
• invariant oracle,
• interaction oracle,
• temporal oracle,
• negative temporal oracle,
• audit oracle,
• telemetry oracle,
• semantic oracle.

Deterministic oracles are preferred where possible.

7.9 Verdict

A Verdict summarizes the result of an oracle or scenario execution.

Initial verdict canon:

PASS
FAIL
SUSPICIOUS
INCONCLUSIVE

The non-Boolean states are important for exploratory and agentic testing where uncertainty may be real and should not be hidden.

7.10 Finding

A Finding is a meaningful detected discrepancy, defect, risk or unexpected behavior that warrants persistence and possible investigation.

Findings may later generate new or hardened verification assets.

7.11 Investigator

An Investigator attempts to understand a finding and may:

• reproduce it,
• minimize the scenario,
• identify relevant evidence,
• distinguish product defect from test defect,
• propose a deterministic regression test.

The investigator may itself be agentic.

---

8. Verification Asset

A Verification Asset is the central managed object of test-driver.

It represents something the framework has reason to verify over time.

A verification asset connects:

UseCase / Requirement
        |
        v
Verification Asset
        |
        +--> Agentic Scenario
        |
        +--> Adaptive Scenario
        |
        +--> Hardened Scenario
        |
        +--> Deterministic Test Implementation

The identity of the verification asset should survive changes in its test implementation.

A verification asset may contain:

id: document-share-read
usecase: document/share

protects:
  claims:
    - recipient-can-read
    - recipient-cannot-write
  invariants:
    - tenant-isolation
    - least-privilege

maturity: adaptive

execution:
  mode: agentic
  surfaces:
    - browser

lineage:
  created_from: usecase

---

9. Evolution Layer

9.1 Test Maturity

Test-driver uses a progressive maturity model:

T0 Exploratory
T1 Agentic
T2 Adaptive
T3 Specified
T4 Hardened
T5 Deterministic

T0 — Exploratory

The framework knows the intent but little about the stable realization.

T1 — Agentic

An agent can pursue the intended outcome through available surfaces.

T2 — Adaptive

The scenario has acquired partial structure and can adapt to legitimate implementation change.

T3 — Specified

Expected behavior, actions and oracles are increasingly explicit.

T4 — Hardened

The scenario is tightly constrained and should require little interpretation.

T5 — Deterministic

The verification executes as conventional deterministic test code without agentic involvement.

9.2 Use-Case Maturity

Use-case maturity is independent from test maturity.

Initial model:

U0 Hypothesis
U1 Emerging
U2 Established
U3 Stable
U4 Contractual

A stable use case may temporarily require agentic tests when a new implementation is introduced.

9.3 Measured stability

Declared Temperature was removed on 2026-09-28 (F-0008). Crystallization uses observed trajectory stability through assess_stability, not a temperature label.

9.4 Adaptation

Adaptation modifies a test realization while attempting to preserve the intent being verified.

Adaptations must be classified.

Initial adaptation canon:

A0 selector
A1 navigation
A2 protocol
A3 data-shape
A4 workflow
A5 semantic
A6 invariant

A0-A2 may commonly be eligible for automatic adaptation.

A4-A6 should normally produce review, investigation or an explicit requirement change rather than silent mutation.

9.5 Hardening

Hardening reduces ambiguity and increases explicitness.

Hardening may introduce:

• explicit preconditions,
• explicit semantic actions,
• stronger oracles,
• controlled fixtures,
• stable selectors or APIs,
• bounded timing,
• deterministic data,
• reduced agent discretion.

9.6 Crystallization

Crystallization is the transition from learned agentic or adaptive execution into deterministic verification.

Canonical lifecycle:

Explore
  -> Discover
  -> Reproduce
  -> Minimize
  -> Specify
  -> Harden
  -> Crystallize
  -> Deterministic Regression

Crystallization is not necessarily permanent. A major implementation change may temporarily move an asset back toward adaptive or agentic execution.

9.7–9.8 Removed lifecycle scores

Energy and Confidence are removed from the current concept set. Neither was used in a decision across the three reference use cases. H-005 is dormant-indefinite; there is no capture-only energy subsystem.

9.9 Lineage

Lineage records why a verification asset exists and how it evolved.

Example:

UseCase
   |
   +--> exploratory Test A
           |
           +--> Finding F17
                   |
                   +--> Regression Test A.1
                           |
                           +--> deterministic implementation

Lineage should make it possible to answer:

Why does this test exist?

9.10 Asset removal

Maintainers may remove obsolete tests with an explicit review of the protected claims. No computed Retirement concept or policy is implemented or scheduled.

---

10. Triadic Verification

The framework continuously reconciles three evolving systems:

Use Cases
    <->
Implementations
    <->
Verification Assets

This is called Triadic Verification.

A verification asset is valid only insofar as it continues to meaningfully connect intended behavior with observed implementation behavior.

When one side changes, the framework evaluates whether the other relationships remain valid.

---

11–12. Removed planning abstractions

Metabolism and Campaign were removed from the current concept set on 2026-09-28. The experiments choose explicit scenario and mutation lists. No evidence justifies a planner or score-driven scheduling; these are not deferred implementation promises.

---

13. Reference Scenario

The first reference scenario for test-driver should intentionally exercise multi-user state and security semantics.

Use case

Alice owns resource R. Alice grants Bob read access. Bob can access R. Carol cannot access R. Alice revokes Bob's access. Bob can no longer access R.

Actors

Alice — owner
Bob — recipient
Carol — unrelated user

Claims

Bob can read R after grant.
Bob cannot modify R with read-only permission.
Bob cannot read R after revocation.

Invariants

Carol never gains access to R.
Tenant isolation remains intact.
Authorization must be consistent across surfaces.

Derived security mutations

Carol attempts direct access to R.
Bob attempts write access.
Bob substitutes another resource identifier.
Bob retries access immediately after revocation.
Bob attempts access through another surface.
Grant and revoke are executed concurrently.

This scenario is intended as the first vertical slice because it exercises:

• multiple actors,
• identity isolation,
• shared state,
• authorization,
• positive and negative behavior,
• temporal transitions,
• mutation,
• deterministic oracles,
• eventual crystallization.

---

14. Initial Architecture Boundaries

The initial implementation should preserve the following logical components:

UseCase Parser
      |
      v
Scenario Planner
      |
      v
Actor Runtime
      |
      v
Drivers
      |
      v
System Under Test

Observers --> Evidence --> Oracle Engine --> Verdict

Metadata Store:
  maturity
  lineage

The conceptual components may initially share a process or codebase. Their separation is semantic before it is architectural.

---

15. Initial Development Sequence

Superseded. This section previously carried its own M0–M3 sequence, which conflicted with the sequences in INTENT.md and TestDriverInitialMilestones.md. The canonical milestone sequence is TestDriverInitialMilestones.md (M0–M10). Recorded as framework finding F-0001 (CONCEPT_DRIFT), 2026-08-22.

The former M0–M3 corresponded to canonical M1, M4, M5–M7 and M9 respectively. See the mapping table in INTENT.md § Initial Milestones.

Execution order is not the canonical order — see workplans/TD-WP-0002-vertical-spike-crystallization.md.

---

16. Canonical Concept Set v0.1

The initial test-driver vocabulary is:

UseCase
Claim
Invariant
Actor
Cast
World
Surface
Scenario
Schedule
Lens
Variant
Mutation
Driver
SemanticAction
ActorRuntime
Orchestrator
Observer
Observation
Evidence
Oracle
Verdict
Finding
Investigator
VerificationAsset
Maturity
Adaptation
Hardening
Crystallization
Lineage
TriadicVerification

---

17. Summary

The defining model of test-driver is:

                 INTENT
                   |
                UseCase
                   |
                   v
           Verification Asset
                   |
        +----------+----------+
        |          |          |
      Lens      Mutation   Schedule
        |          |          |
        +----------+----------+
                   |
                Scenario
                   |
           Actors + Drivers
                   |
                   v
             System Under Test
                   |
                   v
                Evidence
                   |
                Oracles
                   |
                   v
                Verdict
                   |
                Finding

Repeated stable trajectories --> crystallization --> deterministic regression

The fundamental promise is:

test-driver keeps verification useful while software changes rapidly, then turns learned behavior into cheap deterministic confidence as the system stabilizes.