test-driver/docs/TestDriverConceptModel.md

Test Driver Concept Model

Status: v0.1 Draft
Framework: test-driver

1. Purpose

test-driver is a use-case-driven verification framework for software systems that are developed under increasingly fluid, fast-moving and agentic development conditions.

Its core premise is that verification should evolve together with the behavior and implementation it protects:

Tests begin fluid and exploratory when software is changing quickly, adapt with agentic assistance while behavior stabilizes, and crystallize into deterministic test code when implementation and expected behavior become sufficiently mature.

The framework is intended to support, from one coherent conceptual model:

• integration testing,
• end-to-end and user-journey testing,
• multi-user interaction testing,
• authorization and security testing,
• resilience and failure testing,
• later, scale and performance testing.

The framework treats a test not primarily as source code, but as a verification asset that connects intended behavior to observed implementation behavior over time.

---

2. Core Principles

2.1 Intent precedes implementation

Tests ultimately protect intended behavior, not incidental implementation details.

A user-facing capability should therefore be described first in terms of purpose, actors, expected outcomes, claims and invariants rather than UI clicks or protocol calls.

2.2 Use cases are the primary behavioral source

The core source object is a UseCase.

A use case describes purposeful behavior involving one or more actors and a system under test. Integration, journey, multi-user and security tests are projections or transformations of that same use case rather than unrelated test suites.

2.3 Agenticity is a means, not the source of truth

Agents are particularly useful for:

• interpreting intent,
• navigating unstable interaction surfaces,
• discovering alternative paths,
• generating scenario variations,
• exploring adversarial behavior,
• adapting tests to legitimate mechanical changes,
• investigating and minimizing failures.

The ultimate verdict should, wherever possible, come from explicit and independent oracles rather than from an actor agent judging its own success.

2.4 Exploration precedes crystallization

Agentic tests should not remain agentic merely because they started that way.

As behavior and implementation become stable, agentic realizations are progressively hardened and finally crystallized into deterministic tests.

2.5 Useful tests accumulate energy

Verification assets gain energy when they demonstrate value, for example by catching genuine failures or protecting important behavior.

They lose energy when they repeatedly become invalid, require unnecessary adaptation, become flaky, duplicate stronger verification or protect behavior that is no longer relevant.

2.6 Verification assets may die

Tests are not immortal repository artifacts.

When a verification asset loses relevance and reaches sufficiently low energy, it may be removed from active campaigns, archived or retired.

2.7 Implementation is not the truth

When a test and an implementation disagree, the framework must not automatically adapt the test to the implementation.

The disagreement may indicate:

1. an implementation defect,
2. an intentional requirement change,
3. a defective or outdated test,
4. an ambiguous condition requiring investigation.

---

3. Conceptual Layers

The concept model is organized into five layers:

1. Intent — what should be true.
2. Situation — under what circumstances it should be true.
3. Variation — how behavior should be explored.
4. Execution — what actually happened.
5. Evolution — how verification changes over time.

---

4. Intent Layer

4.1 UseCase

A UseCase describes purposeful interaction with a system from the perspective of one or more actors.

A use case should primarily describe:

• intent,
• participating actor roles,
• preconditions,
• relevant outcomes,
• behavioral claims,
• invariants.

It should avoid encoding unnecessary implementation mechanics.

Example:

Alice wants to share document D with Bob so that Bob can read it while Carol remains unable to access it.

4.2 Claim

A Claim is an expected behavior that should hold within a use case or scenario.

Examples:

Bob can read document D.
Bob cannot modify document D.
The sharing operation appears in the audit history.

Claims are typically local to a use case or capability.

4.3 Invariant

An Invariant is a property expected to remain true across many scenarios, implementations or use cases.

Examples:

Tenant isolation must be preserved.
Unauthorized actors must not obtain document contents.
Read permission must not imply write permission.
Audit records must not be silently omitted.

Security requirements are often best represented as invariants.

---

5. Situation Layer

5.1 Actor

An Actor is an independently acting user, system or external participant.

An actor has an epistemic and execution boundary containing at least:

• identity,
• role,
• tenant or organizational context,
• credentials,
• session,
• permissions,
• private memory,
• known resources,
• available interaction surfaces,
• allowed communication channels.

Actors in multi-user tests must not implicitly share knowledge unless the use case explicitly allows it.

5.2 Cast

A Cast is the set of actors participating in a scenario.

Example:

Alice — owner
Bob — intended recipient
Carol — unrelated user

5.3 World

A World is the relevant initial state in which a scenario begins.

It may contain:

• users and identities,
• tenants,
• resources,
• permissions,
• configuration,
• external services,
• clocks,
• feature flags,
• fixtures,
• dependency state.

5.4 Surface

A Surface is a mechanism through which an actor interacts with the system.

Examples:

• browser UI,
• HTTP API,
• CLI,
• mobile UI,
• message queue,
• email,
• webhook,
• external integration API.

5.5 Scenario

A Scenario is a concrete realization of a use case.

Conceptually:

Scenario = UseCase + Cast + World + Schedule + Surfaces + Variant

A scenario determines enough context for one or more actors to execute purposeful behavior.

5.6 Schedule

A Schedule defines causal and temporal relationships between actions.

Instead of assuming every scenario is a linear sequence, the framework should support a causal graph including relationships such as:

• must happen before,
• caused by,
• wait until,
• concurrent with,
• within time T,
• must never occur during interval T.

Schedules are central to multi-user and concurrency testing.

---

6. Variation Layer

6.1 Lens

A Lens is the verification perspective applied to a use case.

Initial lens canon:

Integration Lens

Do participating components communicate correctly?

Journey Lens

Can an actor achieve the intended outcome through the complete system?

Interaction Lens

Does the system remain correct when multiple independent actors interact with shared state?

Security Lens

Do claims and invariants survive unauthorized, manipulated, adversarial or logically invalid behavior?

Resilience Lens

Does the use case remain valid under component, dependency or network failures?

Scale Lens

Does the behavior remain correct and useful under realistic populations and load?

6.2 Variant

A Variant is an alternative realization of a scenario that preserves enough identity with its originating use case to be meaningfully compared.

Variants may differ in:

• actor roles,
• data,
• sequence,
• timing,
• interaction surface,
• environment,
• dependency state.

6.3 Mutation

A Mutation is a deliberate transformation of a scenario intended to probe robustness, correctness or security.

Initial mutation canon:

actor-substitution
resource-substitution
tenant-substitution
sequence-reordering
step-skipping
replay
repetition
concurrency
surface-substitution
invalid-state
privilege-mutation
dependency-failure
time-shift

Security scenarios should preferentially be derived as transformations of ordinary business use cases rather than maintained as a disconnected testing universe.

---

7. Execution Layer

7.1 Driver

A Driver exposes an interaction surface to an actor or deterministic test implementation.

Examples:

BrowserDriver
HttpDriver
CliDriver
MailDriver
QueueDriver

Drivers should support both agentic and deterministic realizations where practical.

7.2 Semantic Action

A Semantic Action describes a meaningful action independently of the mechanics needed to perform it.

Examples:

open_resource(R)
grant_access(Bob, READ)
revoke_access(Bob)
approve_invoice(I)

Semantic actions form an important bridge between agentic exploration and deterministic crystallization.

An agent may initially discover how a semantic action is performed through an unstable interface. Later the driver may provide a stable deterministic implementation of the same semantic action.

7.3 Actor Runtime

The Actor Runtime executes actor behavior while preserving actor isolation.

It is responsible for:

• identity/session isolation,
• agent or deterministic execution,
• memory isolation,
• surface access,
• action recording,
• evidence correlation.

7.4 Orchestrator

The Orchestrator coordinates:

• actors,
• scenario state,
• barriers,
• schedules,
• clocks,
• world setup and teardown,
• run lifecycle.

The orchestrator may know the complete world state even when individual actors do not.

7.5 Observer

An Observer captures facts about what happened without being the actor responsible for producing the behavior.

Observers may inspect:

• externally visible outcomes,
• API responses,
• domain state,
• logs,
• traces,
• metrics,
• audit records,
• messages,
• UI state.

7.6 Observation

An Observation is a captured fact produced during a run.

7.7 Evidence

Evidence is persisted observation material supporting later judgment, diagnosis and lineage.

A run should be able to produce an evidence pack containing, where applicable:

• run identifier,
• use-case and scenario version,
• system/component versions,
• environment,
• actor identities and roles,
• agent/runtime versions,
• random seeds,
• action timeline,
• screenshots or UI traces,
• requests and responses,
• domain-state observations,
• logs,
• metrics,
• distributed traces,
• oracle evaluations,
• verdict.

7.8 Oracle

An Oracle evaluates evidence against a claim or invariant.

Initial oracle categories:

• outcome oracle,
• state oracle,
• authorization oracle,
• invariant oracle,
• interaction oracle,
• temporal oracle,
• negative temporal oracle,
• audit oracle,
• telemetry oracle,
• semantic oracle.

Deterministic oracles are preferred where possible.

7.9 Verdict

A Verdict summarizes the result of an oracle or scenario execution.

Initial verdict canon:

PASS
FAIL
SUSPICIOUS
INCONCLUSIVE

The non-Boolean states are important for exploratory and agentic testing where uncertainty may be real and should not be hidden.

7.10 Finding

A Finding is a meaningful detected discrepancy, defect, risk or unexpected behavior that warrants persistence and possible investigation.

Findings may later generate new or hardened verification assets.

7.11 Investigator

An Investigator attempts to understand a finding and may:

• reproduce it,
• minimize the scenario,
• identify relevant evidence,
• distinguish product defect from test defect,
• propose a deterministic regression test.

The investigator may itself be agentic.

---

8. Verification Asset

A Verification Asset is the central managed object of test-driver.

It represents something the framework has reason to verify over time.

A verification asset connects:

UseCase / Requirement
        |
        v
Verification Asset
        |
        +--> Agentic Scenario
        |
        +--> Adaptive Scenario
        |
        +--> Hardened Scenario
        |
        +--> Deterministic Test Implementation

The identity of the verification asset should survive changes in its test implementation.

A verification asset may contain:

id: document-share-read
usecase: document/share

protects:
  claims:
    - recipient-can-read
    - recipient-cannot-write
  invariants:
    - tenant-isolation
    - least-privilege

maturity: adaptive
temperature: warm

energy:
  value: 72

execution:
  mode: agentic
  surfaces:
    - browser

lineage:
  created_from: usecase

---

9. Evolution Layer

9.1 Test Maturity

Test-driver uses a progressive maturity model:

T0 Exploratory
T1 Agentic
T2 Adaptive
T3 Specified
T4 Hardened
T5 Deterministic

T0 — Exploratory

The framework knows the intent but little about the stable realization.

T1 — Agentic

An agent can pursue the intended outcome through available surfaces.

T2 — Adaptive

The scenario has acquired partial structure and can adapt to legitimate implementation change.

T3 — Specified

Expected behavior, actions and oracles are increasingly explicit.

T4 — Hardened

The scenario is tightly constrained and should require little interpretation.

T5 — Deterministic

The verification executes as conventional deterministic test code without agentic involvement.

9.2 Use-Case Maturity

Use-case maturity is independent from test maturity.

Initial model:

U0 Hypothesis
U1 Emerging
U2 Established
U3 Stable
U4 Contractual

A stable use case may temporarily require agentic tests when a new implementation is introduced.

9.3 Temperature

Temperature represents implementation or capability fluidity.

Initial model:

HOT   actively being invented
WARM  frequently changing
COOL  stabilizing
COLD  mature or contractual

Temperature influences the preferred execution strategy:

Temperature	Preferred verification mode
HOT	exploratory and agentic
WARM	adaptive with emerging deterministic coverage
COOL	hardened regression with occasional exploration
COLD	predominantly deterministic

A capability may cool as it stabilizes and heat up again during major redesign or migration.

9.4 Adaptation

Adaptation modifies a test realization while attempting to preserve the intent being verified.

Adaptations must be classified.

Initial adaptation canon:

A0 selector
A1 navigation
A2 protocol
A3 data-shape
A4 workflow
A5 semantic
A6 invariant

A0-A2 may commonly be eligible for automatic adaptation.

A4-A6 should normally produce review, investigation or an explicit requirement change rather than silent mutation.

9.5 Hardening

Hardening reduces ambiguity and increases explicitness.

Hardening may introduce:

• explicit preconditions,
• explicit semantic actions,
• stronger oracles,
• controlled fixtures,
• stable selectors or APIs,
• bounded timing,
• deterministic data,
• reduced agent discretion.

9.6 Crystallization

Crystallization is the transition from learned agentic or adaptive execution into deterministic verification.

Canonical lifecycle:

Explore
  -> Discover
  -> Reproduce
  -> Minimize
  -> Specify
  -> Harden
  -> Crystallize
  -> Deterministic Regression

Crystallization is not necessarily permanent. A major implementation change may temporarily move an asset back toward adaptive or agentic execution.

9.7 Energy

Energy expresses the current value of retaining, maintaining and executing a verification asset.

Energy should be derived from observable events rather than being an unexplained score.

Illustrative initial event model:

Event	Energy change
Detects confirmed product defect	+25
Detects confirmed security defect	+40
Prevents regression after previous defect	+20
Exercises recently changed relevant capability	+2
Requires harmless mechanical adaptation	-2
Requires workflow adaptation	-10
Test itself was wrong	-20
False positive or flaky failure	-15
Duplicates stronger existing verification	-20
Protected use case is deprecated	-100

Energy history should be retained:

energy:
  value: 83
  history:
    - event: defect-detected
      delta: 25
      finding: TD-143
    - event: navigation-adaptation
      delta: -2

Energy is not synonymous with correctness.

It is better interpreted as:

the current value of spending verification attention and resources on this asset.

9.8 Confidence

Confidence represents how strongly the framework currently believes that a verification asset faithfully represents the intended behavior it claims to protect.

Confidence should remain conceptually separate from energy.

A high-energy test may still have low confidence if its behavior is poorly specified.

9.9 Lineage

Lineage records why a verification asset exists and how it evolved.

Example:

UseCase
   |
   +--> exploratory Test A
           |
           +--> Finding F17
                   |
                   +--> Regression Test A.1
                           |
                           +--> deterministic implementation

Lineage should make it possible to answer:

Why does this test exist?

9.10 Retirement

A verification asset may move through:

active
  -> low-frequency
  -> archival
  -> retired

Energy reaching zero may trigger retirement consideration, but critical security, regulatory or contractual invariants may define a retirement floor or prohibition.

---

10. Triadic Verification

The framework continuously reconciles three evolving systems:

Use Cases
    <->
Implementations
    <->
Verification Assets

This is called Triadic Verification.

A verification asset is valid only insofar as it continues to meaningfully connect intended behavior with observed implementation behavior.

When one side changes, the framework evaluates whether the other relationships remain valid.

---

11. Test Metabolism

Energy, temperature, maturity and execution cost together create a Test Metabolism.

The framework should preferentially spend verification resources where they are most valuable.

A future campaign planner may consider:

Priority =
    Test Energy
  x Changed-System Proximity
  x Use-Case Criticality
  x Risk
  x Time Since Last Execution

The precise formula is intentionally deferred.

The conceptual requirement is that test selection should be dynamic rather than assuming every historical test has equal present value.

---

12. Campaign

A Campaign is a strategy for selecting and executing verification assets and scenario variants.

Initial campaign examples:

PR smoke
regression
release qualification
authorization
tenant isolation
concurrency
resilience
exploratory

Campaigns manage combinatorial explosion by selecting relevant slices of:

actors x roles x states x data x surfaces x schedules x mutations

---

13. Reference Scenario

The first reference scenario for test-driver should intentionally exercise multi-user state and security semantics.

Use case

Alice owns resource R. Alice grants Bob read access. Bob can access R. Carol cannot access R. Alice revokes Bob's access. Bob can no longer access R.

Actors

Alice — owner
Bob — recipient
Carol — unrelated user

Claims

Bob can read R after grant.
Bob cannot modify R with read-only permission.
Bob cannot read R after revocation.

Invariants

Carol never gains access to R.
Tenant isolation remains intact.
Authorization must be consistent across surfaces.

Derived security mutations

Carol attempts direct access to R.
Bob attempts write access.
Bob substitutes another resource identifier.
Bob retries access immediately after revocation.
Bob attempts access through another surface.
Grant and revoke are executed concurrently.

This scenario is intended as the first vertical slice because it exercises:

• multiple actors,
• identity isolation,
• shared state,
• authorization,
• positive and negative behavior,
• temporal transitions,
• mutation,
• deterministic oracles,
• eventual crystallization.

---

14. Initial Architecture Boundaries

The initial implementation should preserve the following logical components:

UseCase Parser
      |
      v
Scenario Planner
      |
      v
Actor Runtime
      |
      v
Drivers
      |
      v
System Under Test

Observers --> Evidence --> Oracle Engine --> Verdict

Metadata Store:
  maturity
  temperature
  energy
  confidence
  lineage

The conceptual components may initially share a process or codebase. Their separation is semantic before it is architectural.

---

15. Initial Development Sequence

M0 — Deterministic semantic scenario runner

Implement:

• use cases,
• actors,
• worlds,
• semantic actions,
• observations,
• oracles,
• verdicts.

Execute the reference multi-user scenario without AI.

M1 — Agentic driver

Allow an actor agent to realize a semantic action or use-case subgoal through a browser or API surface while preserving actor isolation and evidence capture.

M2 — Adaptation and crystallization

Implement:

• adaptation classification,
• successful-path persistence,
• hardening,
• generation of deterministic test candidates.

M3 — Living verification

Introduce:

• energy,
• temperature,
• lineage,
• retirement,
• mutation campaigns,
• adaptive test selection.

---

16. Canonical Concept Set v0.1

The initial test-driver vocabulary is:

UseCase
Claim
Invariant
Actor
Cast
World
Surface
Scenario
Schedule
Lens
Variant
Mutation
Driver
SemanticAction
ActorRuntime
Orchestrator
Observer
Observation
Evidence
Oracle
Verdict
Finding
Investigator
VerificationAsset
Maturity
Temperature
Adaptation
Hardening
Crystallization
Energy
Confidence
Lineage
Retirement
Campaign
TriadicVerification
TestMetabolism

---

17. Summary

The defining model of test-driver is:

                 INTENT
                   |
                UseCase
                   |
                   v
           Verification Asset
                   |
        +----------+----------+
        |          |          |
      Lens      Mutation   Schedule
        |          |          |
        +----------+----------+
                   |
                Scenario
                   |
           Actors + Drivers
                   |
                   v
             System Under Test
                   |
                   v
                Evidence
                   |
                Oracles
                   |
                   v
                Verdict
                   |
             Finding / Value
                   |
                   v
                 Energy

HOT ------> WARM ------> COOL ------> COLD
 |            |            |           |
agentic     adaptive     hardened   deterministic
 +-------------- crystallization ------>

The fundamental promise is:

test-driver keeps verification useful while software changes rapidly, then turns learned behavior into cheap deterministic confidence as the system stabilizes.