Register with State Hub, persist concept assessment, seed first workplan
- history/2026-08-22-concept-assessment-swot.md: SWOT assessment of the concept corpus with recommendations for the first workplan - statehub register: infotech domain, TD-WP prefix, generated AGENTS.md, .custodian-brief.md and TD-WP-0001 bootstrap workplan - .repo-classification.yaml: category research, domain infotech - SCOPE.md rewritten with real repo boundaries - TD-WP-0002: vertical spike reordering M0-M10 into one end-to-end thread that can falsify the crystallization thesis early - commit previously untracked INTENT.md and docs/ Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 1629012@bnt-lap001 Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
This commit is contained in:
parent
b471bed707
commit
7249c6a403
13 changed files with 3900 additions and 0 deletions
61
workplans/TD-WP-0001-statehub-bootstrap.md
Normal file
61
workplans/TD-WP-0001-statehub-bootstrap.md
Normal file
|
|
@ -0,0 +1,61 @@
|
|||
---
|
||||
id: TD-WP-0001
|
||||
type: workplan
|
||||
title: "Bootstrap State Hub integration"
|
||||
domain: infotech
|
||||
repo: test-driver
|
||||
status: ready
|
||||
owner: codex
|
||||
topic_slug: custodian
|
||||
created: "2026-08-22"
|
||||
updated: "2026-08-22"
|
||||
---
|
||||
|
||||
# Bootstrap State Hub integration
|
||||
|
||||
Use-case-driven verification framework for integration, end-to-end, multi-user, authorization and security testing that matures tests from agentic exploration into deterministic regression.
|
||||
|
||||
## Review Generated Integration Files
|
||||
|
||||
```task
|
||||
id: TD-WP-0001-T01
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Review `INTENT.md`, `SCOPE.md`, `AGENTS.md`, and `.custodian-brief.md`.
|
||||
Replace generated placeholders with repo-specific facts where needed.
|
||||
|
||||
## Verify Local Developer Workflow
|
||||
|
||||
```task
|
||||
id: TD-WP-0001-T02
|
||||
status: wait
|
||||
priority: high
|
||||
```
|
||||
|
||||
Identify the repo's install, test, lint, build, and run commands. Add or refine
|
||||
those commands in the agent instructions so future coding sessions can verify
|
||||
changes confidently.
|
||||
|
||||
## Seed First Real Workplan
|
||||
|
||||
```task
|
||||
id: TD-WP-0001-T03
|
||||
status: done
|
||||
priority: medium
|
||||
```
|
||||
|
||||
Create the first implementation workplan for the repository's most important
|
||||
next change. After workplan file updates, run the sync locally from this repo
|
||||
checkout:
|
||||
|
||||
```bash
|
||||
statehub fix-consistency
|
||||
```
|
||||
|
||||
Blocked until the stack exists: no code, no dependency manifest and no test
|
||||
runner are present yet. Unblocks with TD-WP-0002-T04 (deterministic semantic
|
||||
kernel), which introduces the first Python package and pytest configuration.
|
||||
|
||||
Seeded workplan: `workplans/TD-WP-0002-vertical-spike-crystallization.md`.
|
||||
275
workplans/TD-WP-0002-vertical-spike-crystallization.md
Normal file
275
workplans/TD-WP-0002-vertical-spike-crystallization.md
Normal file
|
|
@ -0,0 +1,275 @@
|
|||
---
|
||||
id: TD-WP-0002
|
||||
type: workplan
|
||||
title: "Vertical spike: falsify the crystallization thesis"
|
||||
domain: infotech
|
||||
repo: test-driver
|
||||
status: proposed
|
||||
owner: codex
|
||||
topic_slug: custodian
|
||||
created: "2026-08-22"
|
||||
updated: "2026-08-22"
|
||||
---
|
||||
|
||||
# Vertical spike: falsify the crystallization thesis
|
||||
|
||||
## Why this workplan exists
|
||||
|
||||
The concept corpus (`INTENT.md`, `docs/`) describes eleven milestones that build
|
||||
four complete layers — research control plane, deterministic kernel, lab,
|
||||
self-verification — before the central thesis is exercised even once. The repo
|
||||
currently holds ~3,000 lines of theory and zero lines of executable code.
|
||||
|
||||
This workplan inverts that order. It drives **one thin thread end to end**
|
||||
through every layer of the model, so that the thesis can be supported or
|
||||
falsified cheaply and early:
|
||||
|
||||
```
|
||||
one use case → deterministic kernel → labelled mutation lab
|
||||
→ agentic realisation of one semantic action
|
||||
→ adaptation classification → one crystallization
|
||||
```
|
||||
|
||||
Reasoning and the full assessment behind this sequencing:
|
||||
`history/2026-08-22-concept-assessment-swot.md`.
|
||||
|
||||
## Success gate
|
||||
|
||||
The spike succeeds when, in one coherent run:
|
||||
|
||||
1. agentic realisation recovers from lab mutations **M01** (moved/renamed
|
||||
control) and **M02** (restructured DOM) with **zero** changes to claims or
|
||||
invariants;
|
||||
2. lab mutation **M05** (authorization defect) is reported as a **Product
|
||||
Finding**, not adapted to;
|
||||
3. one verification asset crystallizes into a deterministic test that runs with
|
||||
**no model involvement** and preserves the relevant oracles;
|
||||
4. **False Adaptation Rate = 0** across the labelled mutation set.
|
||||
|
||||
Failing this gate early and cheaply is a valid and valuable outcome. Passing it
|
||||
is worth more than any other ten items on the M0–M10 roadmap.
|
||||
|
||||
## Explicitly deferred
|
||||
|
||||
Not in this workplan, by decision rather than omission — each is cheap to build,
|
||||
satisfying to build, and impossible to validate at this stage:
|
||||
|
||||
- Energy scoring, decay, and campaign selection (raw immutable `EnergyEvent`s are
|
||||
recorded from the first run; **no** scoring logic is implemented);
|
||||
- Temperature and Confidence as computed values;
|
||||
- Test Metabolism, Campaigns, Retirement floors;
|
||||
- Lineage graph storage beyond a parent pointer;
|
||||
- the five agentic roles (Builder / Experimenter / Critic / Auditor / Maintainer);
|
||||
- any surface beyond one HTTP API and one browser UI.
|
||||
|
||||
## Constraints
|
||||
|
||||
- Stack: Python, pytest, Playwright, Pydantic/dataclasses, YAML, SQLite.
|
||||
One process, one database, one browser engine, one application under test.
|
||||
- Novelty belongs in the verification model, never in the infrastructure.
|
||||
- No LLM judgment where a deterministic oracle is available (Concept Model §2.3).
|
||||
|
||||
---
|
||||
|
||||
## Reconcile milestone numbering and commit the concept corpus
|
||||
|
||||
```task
|
||||
id: TD-WP-0002-T01
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Three documents carry three different milestone sequences: `INTENT.md` (M0–M3),
|
||||
`docs/TestDriverInitialMilestones.md` (M0–M10), and
|
||||
`docs/TestDriverConceptModel.md` §15 (a third M0–M3). This is a CONCEPT_DRIFT
|
||||
finding by the project's own taxonomy, present before any code exists.
|
||||
|
||||
Pick one canonical sequence, mark the others superseded in place, and commit
|
||||
`INTENT.md` and `docs/` to git (currently untracked). Record the drift as the
|
||||
first entry in the framework findings log — the project should catch this on
|
||||
itself before it catches anything else.
|
||||
|
||||
## Decide intent provenance and the classification mechanism on paper
|
||||
|
||||
```task
|
||||
id: TD-WP-0002-T02
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Two design questions gate everything downstream. Answer both in a written design
|
||||
note plus a State Hub decision record, **before** writing kernel code.
|
||||
|
||||
**(a) The classifier.** What *deterministic evidence* separates "the button
|
||||
moved" from "Bob can still read after revoke"? Specify the signal, not the
|
||||
intent. If the honest answer is "a model decides", that violates Concept Model
|
||||
§2.3 and must be resolved now rather than discovered at M6.
|
||||
|
||||
**(b) Intent provenance.** If an agent writes the implementation, an agent writes
|
||||
the use case, and an agent realises the test, then "oracles independent from
|
||||
actors" is procedurally true but epistemically thin. Record whether use cases are
|
||||
human-authored, spec-derived, or agent-generated, and what independence guarantee
|
||||
survives in each case.
|
||||
|
||||
Deliverable: `docs/TestDriverClassificationDesign.md` + one recorded decision.
|
||||
|
||||
## Minimal research control plane
|
||||
|
||||
```task
|
||||
id: TD-WP-0002-T03
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
|
||||
Timebox: half a day. Five hypothesis files (H-001…H-005) each with an explicit
|
||||
falsification condition, one Concept ↔ Implementation Fitness Map, and a stable
|
||||
identifier convention. Plain files under `research/`.
|
||||
|
||||
No CLI, no register tooling, no schema. The control plane is overhead until there
|
||||
are readings to record.
|
||||
|
||||
## Deterministic semantic kernel
|
||||
|
||||
```task
|
||||
id: TD-WP-0002-T04
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Minimal executable representations of UseCase, Actor, World, Scenario,
|
||||
SemanticAction, Observation, Evidence, Oracle, Verdict, VerificationAsset, Run.
|
||||
One deterministic driver (HTTP or direct adapter).
|
||||
|
||||
Run the reference use case end to end:
|
||||
|
||||
```
|
||||
Alice owns resource R. Alice grants Bob READ. Bob can read R.
|
||||
Carol cannot read R. Alice revokes Bob. Bob can no longer read R.
|
||||
```
|
||||
|
||||
Exit: Alice, Bob and Carol hold genuinely isolated identities and sessions; all
|
||||
outcomes are judged by independent deterministic oracles; a run emits a
|
||||
structured Evidence Pack; the scenario replays from known initial state.
|
||||
|
||||
Emit raw `EnergyEvent` records from this point onward. Implement no scoring.
|
||||
|
||||
## Test-driver lab with labelled ground truth
|
||||
|
||||
```task
|
||||
id: TD-WP-0002-T05
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
A deliberately small application: users, tenants, auth, resources, sharing,
|
||||
read/write permissions, revoke, audit history, HTTP API, minimal browser UI.
|
||||
|
||||
Build **15–20 labelled mutations**, not the six sketched in the milestones doc.
|
||||
Six cannot support any statement about precision or recall. Each mutation is a
|
||||
reproducible toggle carrying a recorded expected classification
|
||||
(`MECHANICAL` / `SEMANTIC` / `DEFECT`), with baseline and mutated versions
|
||||
carrying explicit version identifiers.
|
||||
|
||||
The lab is the measuring instrument for every claim the framework makes — a weak
|
||||
lab caps the credibility of all downstream results. It is also potentially the
|
||||
project's first publishable artefact in its own right.
|
||||
|
||||
## Out-of-band ground truth for self-verification
|
||||
|
||||
```task
|
||||
id: TD-WP-0002-T06
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
|
||||
Write the actor-isolation and oracle-independence checks as **plain pytest**
|
||||
against observable behaviour, outside the framework. Using test-driver to verify
|
||||
that test-driver's oracles are independent is a system certifying itself.
|
||||
|
||||
Exit: deliberately breaking actor isolation fails the out-of-band check;
|
||||
allowing an actor to determine its own verdict fails oracle-independence;
|
||||
each failure produces a Framework Finding.
|
||||
|
||||
## Agentic realisation of one semantic action, fully instrumented
|
||||
|
||||
```task
|
||||
id: TD-WP-0002-T07
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Browser driver plus one agentic actor runtime, realising exactly one semantic
|
||||
action — `grant_access(Bob, R, READ)` — from intent, against the lab UI. Strict
|
||||
per-actor context isolation; bounded navigation and tool permissions; full
|
||||
action and evidence recording; agent/model/configuration identity captured in
|
||||
evidence. Oracles stay deterministic.
|
||||
|
||||
**Instrument cost and nondeterminism from the very first run:** tokens, wall
|
||||
time, retry count, and run-to-run variance. If agentic realisation costs more per
|
||||
run than simply asking an agent to rewrite the broken test, the crystallization
|
||||
argument is an aesthetic preference rather than a value proposition. This data is
|
||||
free to collect from run one and impossible to backfill.
|
||||
|
||||
## Adaptation detection and the defect-vs-adaptation classifier
|
||||
|
||||
```task
|
||||
id: TD-WP-0002-T08
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
The centrepiece. Implement the mechanism designed in T02 and run it across the
|
||||
full labelled mutation set from T05.
|
||||
|
||||
Classification path: `IMPLEMENTATION CHANGE` / `INTENT CHANGE` /
|
||||
`PRODUCT DEFECT` / `AMBIGUOUS`, with escalation for semantic changes and
|
||||
ambiguity. Preserve semantic action identity across adaptation; record adaptation
|
||||
history on the verification asset; never modify claims or invariants to
|
||||
accommodate an observed behaviour.
|
||||
|
||||
Measure: Mechanical Recovery Rate, classification precision/recall, and
|
||||
**False Adaptation Rate**. The last is the project's existential safety metric —
|
||||
one publicly normalised authorization defect kills the thesis permanently. Target
|
||||
is zero across the set, and a non-zero result is a stop-and-redesign signal, not
|
||||
a tuning exercise.
|
||||
|
||||
## Crystallize one asset into deterministic regression
|
||||
|
||||
```task
|
||||
id: TD-WP-0002-T09
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Semantic action trajectory capture, stable-realisation detection, deterministic
|
||||
candidate generation, candidate comparison against the existing oracle set, and
|
||||
lineage from the agentic ancestor to the deterministic descendant. Manual
|
||||
acceptance step for now.
|
||||
|
||||
Emit ordinary pytest/Playwright code. Output that drops into a CI system which
|
||||
already exists lets a user adopt the result without adopting the framework.
|
||||
|
||||
Exit: the generated test runs with zero agentic involvement, preserves the
|
||||
relevant claims and oracles, retains visible lineage, and measurably costs less
|
||||
to execute than the agentic ancestor.
|
||||
|
||||
## Gate review and first compression pass
|
||||
|
||||
```task
|
||||
id: TD-WP-0002-T10
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
|
||||
Evaluate the four success-gate criteria against collected evidence and write the
|
||||
result up in `history/` regardless of outcome.
|
||||
|
||||
Then run the compression questions from Milestone 10 while the spike is still
|
||||
small: which concepts have no supporting evidence, which abstractions have no
|
||||
conceptual justification, which metadata never informed a decision, what did the
|
||||
experiments falsify. Mark rejected concepts as rejected rather than silently
|
||||
retaining them, and update the Concept ↔ Implementation Fitness Map.
|
||||
|
||||
Convert at least one framework finding discovered during T04–T09 into a permanent
|
||||
self-verification or deterministic regression — that closes the first genuine
|
||||
Concept–Implementation Fitness Loop.
|
||||
Loading…
Add table
Add a link
Reference in a new issue