the-custodian/canon/architecture/adr-001-workplans-as-repo-artefacts.md

255 lines
8.8 KiB
Markdown
Raw Normal View History

---
id: ADR-001
type: architecture-decision-record
title: "Workplans and Work Items Are Repository Artefacts"
status: accepted
decided_by: Bernd Worsch
date: "2026-02-28"
tags: ["architecture", "state-hub", "workplans", "source-of-truth", "rebuild-principle"]
---
# ADR-001: Workplans and Work Items Are Repository Artefacts
## Status
Accepted.
## Context
During early State Hub development (v0.1v0.4), workstreams and tasks were
created directly in the PostgreSQL database via MCP bootstrap tools
(`create_workstream`, `create_task`). This made the database the **origin** of
work items — not a cache or index. The pattern was convenient for rapid
bootstrapping but is architecturally wrong for a system built on the values of
auditability, reversibility, and local-first sovereignty.
The trigger for formalising this decision was the creation of the v0.5
workplan ("Dynamic Domains & Multi-Repo") directly in the state-hub database
without a corresponding file artefact in any repository.
## Decision
**Workplans and work items MUST originate as Markdown files in the repository
that owns them.** The Custodian State Hub indexes and caches those artefacts
but is never their origin.
Formally: the state-hub must (theoretically, given sufficient compute and
time) be able to **rebuild its full representation** of repositories, their
workplans, tasks, decisions, and dependencies by reading only the files in the
registered repositories. No information that matters for coordination should
exist solely in the database.
### Corollaries
1. **Repository is authoritative.** A workplan file is the canonical record.
The state-hub database row is a materialized cache of that file.
2. **Database is disposable.** Dropping and re-creating the database from
registered repository files must produce an equivalent state. The database
is an operational convenience, not a primary store.
3. **MCP bootstrap tools become index/sync tools.** `create_workstream` and
`create_task` are acceptable as convenience wrappers only if they write the
file first and then register the row. Using them to write DB-only records
violates this ADR.
4. **The rebuild principle implies a sync mechanism.** There must be a defined
path (`make sync-workplans` or equivalent) by which the state-hub reads
workplan files from registered repositories and upserts its database state.
## Workplan File Convention
Each workplan lives in a `workplans/` directory in the repository that owns
the work. The owning repository is identified by domain.
### Location
```
<repo-root>/workplans/<id>-<slug>.md
```
Examples:
- `the-custodian/workplans/CUST-WP-0005-dynamic-domains.md`
- `railiance/workplans/RAIL-WP-0001-three-phoenix.md`
### Frontmatter Schema
```yaml
---
id: CUST-WP-0005 # human-readable workplan ID, unique per repo
type: workplan
title: "State Hub v0.5 — Dynamic Domains & Multi-Repo"
domain: custodian # must match a registered domain slug
status: active # active | completed | archived
owner: custodian
topic_slug: custodian # maps to a state-hub Topic slug
created: "2026-02-28"
updated: "2026-02-28"
---
```
### Task Items
Tasks are embedded in the workplan file as headed sections. Each task
section carries its own YAML block:
```markdown
## P1.1 — Create `domains` table + Alembic migration
```task
id: CUST-WP-0005-T001
status: todo
priority: high
```
Task description prose here.
```
The state-hub parses these embedded task blocks during ingestion and upserts
rows in the `tasks` table. The `id` field is the stable external key; the
state-hub UUID is internal and opaque.
### Decision Items
Decisions are separate files or embedded sections following the same pattern,
using `type: decision` in frontmatter.
## Rebuild Principle
The rebuild sequence for a clean state-hub:
1. `make migrate` — create schema
2. `make seed-domains` — insert domain rows (domains.yaml in canon/)
3. For each registered repository: `make sync-workplans REPO=<slug>` — parse
workplan files and upsert workstreams, tasks, decisions
4. `make sync-progress` — replay progress events from episodic memory logs
After step 4 the database must be functionally equivalent to the live state.
## Consequences
### Immediate
- The v0.5 and v0.3 workplans created DB-first in this session are **legacy
records** that violate this ADR. Remediation: write the corresponding
workplan files, then mark the DB rows as `source: db-legacy` until a sync
mechanism can reconcile them.
- The state-hub CLAUDE.md design-boundary note must be updated: the MCP
bootstrap tools are permitted only as write-through tools (file + DB), never
as DB-only tools.
### Medium Term
- A `make sync-workplans` command must be implemented as part of the
managed-repos / contribution-tracking infrastructure (see v0.3 workplan).
- The `managed_repos` table is the prerequisite: the state-hub must know which
repositories to scan.
- Workplan file format must be versioned and parsed by a dedicated loader
(`state-hub/scripts/sync_workplans.py`).
### Long Term
- When the state-hub grows to cover multiple users or teams, this principle
ensures that no coordination state can be lost by a database failure.
Every repository is its own resilient shard of the coordination graph.
- This is the foundation for the "transgenerational" property: workplans in
git survive database migrations, cloud provider changes, and system
rebuilds.
## Alternatives Considered
**Database-first with export:** Create in DB, export to files on demand.
Rejected: export is easily skipped and files become secondary/stale.
**Files-only, no database:** Parse files on every query.
Rejected: impractical at scale; the database is a necessary cache for
cross-repo aggregation and real-time dashboard queries.
**Hybrid with explicit sync flag:** Mark some records as "db-authoritative"
and others as "file-authoritative."
Rejected: introduces ambiguity about which records matter; violates the
"single source of truth" principle.
## Workplan Closure Protocol
When a workplan is about to be marked `finished`, the responsible agent
MUST perform a closure review before writing the status change. This prevents
the stale-task accumulation that this ADR was designed to make detectable.
### Steps
1. **Query all non-done tasks** in the workplan via
`GET /tasks/?workplan_id=<uuid>` (legacy alias: `workstream_id`; filter for
`todo`, `in_progress`, `blocked`).
2. **Classify each task** into one of three outcomes:
| Outcome | Action |
|---------|--------|
| **Done** — work was completed, DB record just wasn't updated | `PATCH /tasks/{id}/ {"status": "done"}` |
| **Cancelled** — dropped, superseded, or out of scope | `PATCH /tasks/{id}/ {"status": "cancelled", "blocking_reason": "<why>"}` |
| **Carry-forward** — genuinely unfinished, belongs in the next run | Leave open; note in closure review; trigger new workplan |
3. **Append a `## Closure Review` section** to the workplan file:
```markdown
## Closure Review — YYYY-MM-DD
**Outcome:** All tasks completed / N tasks carried forward / N tasks dropped.
### Completed (DB updated)
- TASK-ID — title
### Cancelled (dropped)
| Task | Reason |
|------|--------|
| TASK-ID — title | Superseded by X |
### Carried forward
| Task | Target workplan |
|------|----------------|
| TASK-ID — title | CUST-WP-XXXX |
```
4. **If any tasks are carried forward**: do not mark the workplan
`finished` yet. Create the new workplan file (or amend an existing active
one), then close the current workplan.
5. **Update the workplan frontmatter** `status: finished` and `updated:` date.
6. **Mark the workplan `finished`** in the state hub via MCP or API (`update_workplan_status`).
### Daily Stale-Task Cleanup
As a safety net for cases where the closure review was skipped or incomplete,
a cleanup script cancels any surviving open tasks in completed/archived
workstreams:
```bash
cd ~/the-custodian/state-hub
make cleanup-stale # run immediately
# or add to cron:
# 0 3 * * * cd ~/the-custodian/state-hub && make cleanup-stale
```
The script (`scripts/cleanup_stale_tasks.py`) emits a `cleanup` progress event
recording which tasks were cancelled and in which workstreams. Tasks cancelled
by the cleanup carry a `blocking_reason` noting they should be verified against
the workplan file.
The closure review is the primary mechanism; the cleanup is the fallback. If
the cleanup regularly cancels tasks, it signals that closure reviews are being
skipped — that is the process failure to address, not just the stale tasks.
## Related
- Custodian Constitution v0.1 §2 (Powers) — canon changes require review gate
- ADR-000 (forthcoming) — overall Custodian architecture principles
- State Hub v0.3 workplan — `sync_workplans.py` is a Phase 4 deliverable
- `canon/values/foundational_values_v0.1.md` — Local-first, Auditability,
Reversibility