diff --git a/canon/architecture/adr-001-workplans-as-repo-artefacts.md b/canon/architecture/adr-001-workplans-as-repo-artefacts.md index a6c8696..6e0083b 100644 --- a/canon/architecture/adr-001-workplans-as-repo-artefacts.md +++ b/canon/architecture/adr-001-workplans-as-repo-artefacts.md @@ -4,255 +4,244 @@ type: architecture-decision-record title: "Workplans and Work Items Are Repository Artefacts" status: accepted owner: the-custodian -revision: "accepted-1" -last_reviewed: "2026-02-28" +revision: "accepted-2" +last_reviewed: "2026-08-31" review_interval: 6m decided_by: Bernd Worsch date: "2026-02-28" -tags: ["architecture", "state-hub", "workplans", "source-of-truth", "rebuild-principle"] +tags: ["architecture", "state-hub", "workplans", "source-of-truth", "rebuild-principle", "forgejo", "projection"] --- # ADR-001: Workplans and Work Items Are Repository Artefacts ## Status -Accepted. +Accepted 2026-02-28. + +Amended 2026-08-31 to distinguish file-backed work records from hub-native +records, identify the Forge default branch as the central projection baseline, +and align identity, lifecycle, reconciliation, and closure with ADR-007, +ADR-010, ADR-011, ADR-012, and the work-record standards. The central decision +is unchanged. ## Context -During early State Hub development (v0.1–v0.4), workstreams and tasks were -created directly in the PostgreSQL database via MCP bootstrap tools -(`create_workstream`, `create_task`). This made the database the **origin** of -work items — not a cache or index. The pattern was convenient for rapid -bootstrapping but is architecturally wrong for a system built on the values of -auditability, reversibility, and local-first sovereignty. +During early State Hub development, workstreams and tasks were created directly +in PostgreSQL through bootstrap APIs. This made a database the origin of durable +work rather than a projection of repository-owned artefacts. The pattern was +convenient, but it made work difficult to audit, review, recover, and carry +between generations of tooling. -The trigger for formalising this decision was the creation of the v0.5 -workplan ("Dynamic Domains & Multi-Repo") directly in the state-hub database -without a corresponding file artefact in any repository. +The original decision correctly moved durable workplans and their work items +into repositories. It overstated the consequence, however, by saying that the +entire Hub database and every item that matters for coordination must be +reconstructible from repository files. Later decisions established two +different persistence classes: + +- **file-backed records**, whose durable meaning originates in a repository; +- **hub-native records**, such as append-only progress and runtime facts, whose + durable meaning originates in the Hub. + +Later decisions also established that a central projection reads the pushed +default branch in Forgejo. An arbitrary workstation checkout is a workspace, +not the shared baseline. Unpushed local work may be represented only as an +explicit preliminary overlay. ## Decision -**Workplans and work items MUST originate as Markdown files in the repository -that owns them.** The Custodian State Hub indexes and caches those artefacts -but is never their origin. +### 1. File-backed work originates in the owning repository -Formally: the state-hub must (theoretically, given sufficient compute and -time) be able to **rebuild its full representation** of repositories, their -workplans, tasks, decisions, and dependencies by reading only the files in the -registered repositories. No information that matters for coordination should -exist solely in the database. +Workplans and their file-backed work items **MUST originate as repository +artefacts in the repository that owns the work**. The Hub indexes and projects +those artefacts but is not their origin. -### Corollaries +This includes the workplan, embedded tasks, declared dependencies, residual +handoffs, and durable governed decision artefacts when those are represented as +files. No authoritative field of a file-backed record may exist solely in a Hub +database. -1. **Repository is authoritative.** A workplan file is the canonical record. - The state-hub database row is a materialized cache of that file. +The owning repository is identified by repository identity, not merely by a +domain or topic. Domains and topics classify work; they do not own its source. -2. **Database is disposable.** Dropping and re-creating the database from - registered repository files must produce an equivalent state. The database - is an operational convenience, not a primary store. +### 2. Authority is explicit per persistence class -3. **MCP bootstrap tools become index/sync tools.** `create_workstream` and - `create_task` are acceptable as convenience wrappers only if they write the - file first and then register the row. Using them to write DB-only records - violates this ADR. +Every record contract must declare whether the record is file-backed or +hub-native. The same record must not be writable as authoritative in both +places. -4. **The rebuild principle implies a sync mechanism.** There must be a defined - path (`make sync-workplans` or equivalent) by which the state-hub reads - workplan files from registered repositories and upserts its database state. +- For a **file-backed record**, the repository artefact is authoritative and + the Hub row is derived state. +- For a **hub-native record**, the Hub is authoritative. Progress events, + messages, token events, run history, and similar runtime facts are not made + fictional repository records merely to satisfy a rebuild slogan. -## Workplan File Convention +The word `decision` is used by more than one subsystem. Architecture decisions +and other governed decision artefacts remain files. A runtime decision event may +be hub-native only where its schema says so explicitly; it does not replace the +governed artefact. -Each workplan lives in a `workplans/` directory in the repository that owns -the work. The owning repository is identified by domain. +This replaces the original blanket rejection of a hybrid architecture. The +architecture is hybrid **by declared record class**, never ambiguous within one +record. -### Location +### 3. Forge is the central projection baseline -``` -/workplans/-.md -``` +For the shared central view, the source is the pushed default-branch state held +by Forgejo, as decided by ADR-012. Every projected record must be attributable +to its source repository, path, and commit. -Examples: -- `the-custodian/workplans/CUST-WP-0005-dynamic-domains.md` -- `railiance/workplans/RAIL-WP-0001-three-phoenix.md` +A working copy remains the authoring workspace. Unpushed work may appear in the +Hub only as an attributed preliminary overlay. It must not silently replace or +be presented as the Forge-derived baseline. When its commit reaches Forge, the +baseline absorbs it and the overlay retires. -### Frontmatter Schema +### 4. Mutations of file-backed state are file-first -```yaml ---- -id: CUST-WP-0005 # human-readable workplan ID, unique per repo -type: workplan -title: "State Hub v0.5 — Dynamic Domains & Multi-Repo" -domain: custodian # must match a registered domain slug -status: active # active | completed | archived -owner: custodian -topic_slug: custodian # maps to a state-hub Topic slug -created: "2026-02-28" -updated: "2026-02-28" ---- -``` +Creating or changing a file-backed work record means changing its repository +artefact first. A convenience tool or API is conformant only when it performs a +governed repository mutation and leaves a reviewable file and commit. Writing +only the projected database row is not a durable update. -### Task Items +Direct Hub APIs remain valid for hub-native records. They may also provide +diagnostics or propose repository patches, but a successful database PATCH is +not evidence that a file-backed status changed. -Tasks are embedded in the workplan file as headed sections. Each task -section carries its own YAML block: +### 5. The rebuild guarantee applies to the projection, not the whole database -```markdown -## P1.1 — Create `domains` table + Alembic migration +Given a repository and a specific Forge commit, the system must be able to +reconcile that repository's file-derived projection so that it is equivalent to +the records derived from that commit. Reconciliation must be idempotent, +verifiable, and scoped per repository; a fleet operation is iteration over the +same per-repository operation. -```task -id: CUST-WP-0005-T001 -status: todo -priority: high -``` +Reconciliation retires file-derived records that no longer derive. It does not +delete hub-native history attached to them. It must refuse and report a +repository whose records exist only in the projection until those records have +an explicit disposition. A Forge rebuild does not reconstruct preliminary +overlays and must disclose their retirement before proceeding. -Task description prose here. -``` +Therefore the **file-derived projection is disposable**. The database as a +whole is not disposable when it also contains hub-native facts. -The state-hub parses these embedded task blocks during ingestion and upserts -rows in the `tasks` table. The `id` field is the stable external key; the -state-hub UUID is internal and opaque. +### 6. Identity and lifecycle are delegated contracts -### Decision Items +This ADR does not define a second identity or lifecycle schema. -Decisions are separate files or embedded sections following the same pattern, -using `type: decision` in frontmatter. +- Workplan and task identity follow ADR-007 as amended by ADR-011: global + identity is namespace-aware, and derivable Hub identifiers are governed by + that contract. Existing historical identifiers are grandfathered according + to its migration rules. +- Work-record kinds, locations, lifecycle values, and residual handling follow + `canon/standards/work-record-types_v0.1.md` and + `canon/standards/workplan-terminology-fleet_v0.1.md`. -## Rebuild Principle +New normative text uses `workplan` and `work record`. `workstream` remains only +as a metered compatibility term for legacy database and API surfaces. -The rebuild sequence for a clean state-hub: +### 7. Reconciliation belongs at the repository boundary -1. `make migrate` — create schema -2. `make seed-domains` — insert domain rows (domains.yaml in canon/) -3. For each registered repository: `make sync-workplans REPO=` — parse - workplan files and upsert workstreams, tasks, decisions -4. `make sync-progress` — replay progress events from episodic memory logs +Repo Manager owns discovery, parsing, identity checks, and reconciliation for +file-backed records. State Hub owns the projection and hub-native records. The +implementation may distribute fetch and parse work, but it must preserve that +authority boundary. -After step 4 the database must be functionally equivalent to the live state. +Legacy `create_workstream`, `create_task`, workstation-driven +`sync-workplans`, and similarly named commands are not normative interfaces. +They are conformant only if their current implementation satisfies decisions +3 and 4; otherwise they are transitional or retired. + +## Workplan closure + +A workplan cannot be marked `finished` merely by changing its Hub row. Before +closure, the responsible worker must: + +1. review the source file and the projected view for unfinished tasks; +2. record completed and cancelled outcomes in the source task blocks using the + canonical task lifecycle; +3. turn every actionable carry-forward item into a live work record with + `origin: residual` and an `origin_ref`, or into another explicitly linked + workplan; +4. set the source workplan to `finished`, update it, and commit the result; +5. publish the commit to Forge for the shared baseline and verify that the + projection reconciles to that commit. + +A local reconciliation before push may expose preliminary state, but it does +not complete step 5. + +Automated stale-task cleanup must report file/projection drift. It may not +silently cancel a file-backed task only in the database. An automated repair may +change such a task only through the same governed repository mutation and +commit path as any other file-backed change. Cleanup of hub-native records is +governed by their own retention contract. ## Consequences -### Immediate +### Positive -- The v0.5 and v0.3 workplans created DB-first in this session are **legacy - records** that violate this ADR. Remediation: write the corresponding - workplan files, then mark the DB rows as `source: db-legacy` until a sync - mechanism can reconcile them. +- Durable work remains inspectable, reviewable, and recoverable through Git. +- The central view has an exact repository and commit provenance instead of + reflecting whichever workstation synced most recently. +- Reconciliation can be exercised per repository without destroying progress + history or other hub-native evidence. +- Multiple contributors share a published baseline while retaining an honest + representation of preliminary work. +- Service ownership is clearer: repositories own file-backed truth; the Hub + owns runtime facts and projection services. -- The state-hub CLAUDE.md design-boundary note must be updated: the MCP - bootstrap tools are permitted only as write-through tools (file + DB), never - as DB-only tools. +### Negative -### Medium Term +- File-backed changes require a repository mutation and normally a push before + they become shared baseline state. +- Forge availability affects projection freshness, although it does not prevent + local authoring. +- Preliminary, stale, and retired projection states must be visible in APIs and + user interfaces. +- Git merge conflicts become the explicit conflict mechanism for simultaneous + edits to the same authoritative artefact. +- Tools that previously corrected only database state must be changed to emit a + diagnostic or perform a governed repository update. -- A `make sync-workplans` command must be implemented as part of the - managed-repos / contribution-tracking infrastructure (see v0.3 workplan). +## Migration -- The `managed_repos` table is the prerequisite: the state-hub must know which - repositories to scan. +No bulk rename, UUID rewrite, or historical file rewrite is required by this +amendment. Existing identifiers and legacy terminology retain the grandfathering +rules of ADR-007, ADR-011, and the terminology standard. -- Workplan file format must be versioned and parsed by a dedicated loader - (`state-hub/scripts/sync_workplans.py`). +Implementations must audit and retire DB-first creation, closure, and stale-task +cleanup paths. Existing file-backed projection rows without a Forge source +commit are migration state: they must be matched to a repository artefact or +explicitly dispositioned before reset. Forge-derived reconciliation and +preliminary overlays are implemented under ADR-012 rather than duplicated here. -### Long Term +The `accepted-1` publication remains the immutable historical revision. This +document is `accepted-2`. -- When the state-hub grows to cover multiple users or teams, this principle - ensures that no coordination state can be lost by a database failure. - Every repository is its own resilient shard of the coordination graph. +## Alternatives considered -- This is the foundation for the "transgenerational" property: workplans in - git survive database migrations, cloud provider changes, and system - rebuilds. +**Database-first with export.** Rejected. Export can be skipped, making the +reviewable artefact secondary and stale. -## Alternatives Considered +**Files only, with no projection database.** Rejected. Cross-repository queries, +runtime views, and append-only operational facts need indexed services. -**Database-first with export:** Create in DB, export to files on demand. -Rejected: export is easily skipped and files become secondary/stale. +**Treat every Hub record as reconstructible from Git.** Rejected. This either +loses runtime truth during rebuild or creates artificial files whose only +purpose is mirroring a database. -**Files-only, no database:** Parse files on every query. -Rejected: impractical at scale; the database is a necessary cache for -cross-repo aggregation and real-time dashboard queries. - -**Hybrid with explicit sync flag:** Mark some records as "db-authoritative" -and others as "file-authoritative." -Rejected: introduces ambiguity about which records matter; violates the -"single source of truth" principle. - -## Workplan Closure Protocol - -When a workplan is about to be marked `finished`, the responsible agent -MUST perform a closure review before writing the status change. This prevents -the stale-task accumulation that this ADR was designed to make detectable. - -### Steps - -1. **Query all non-done tasks** in the workplan via - `GET /tasks/?workplan_id=` (legacy alias: `workstream_id`; filter for - `todo`, `in_progress`, `blocked`). - -2. **Classify each task** into one of three outcomes: - - | Outcome | Action | - |---------|--------| - | **Done** — work was completed, DB record just wasn't updated | `PATCH /tasks/{id}/ {"status": "done"}` | - | **Cancelled** — dropped, superseded, or out of scope | `PATCH /tasks/{id}/ {"status": "cancelled", "blocking_reason": ""}` | - | **Carry-forward** — genuinely unfinished, belongs in the next run | Leave open; note in closure review; trigger new workplan | - -3. **Append a `## Closure Review` section** to the workplan file: - - ```markdown - ## Closure Review — YYYY-MM-DD - - **Outcome:** All tasks completed / N tasks carried forward / N tasks dropped. - - ### Completed (DB updated) - - TASK-ID — title - - ### Cancelled (dropped) - | Task | Reason | - |------|--------| - | TASK-ID — title | Superseded by X | - - ### Carried forward - | Task | Target workplan | - |------|----------------| - | TASK-ID — title | CUST-WP-XXXX | - ``` - -4. **If any tasks are carried forward**: do not mark the workplan - `finished` yet. Create the new workplan file (or amend an existing active - one), then close the current workplan. - -5. **Update the workplan frontmatter** `status: finished` and `updated:` date. - -6. **Mark the workplan `finished`** in the state hub via MCP or API (`update_workplan_status`). - -### Daily Stale-Task Cleanup - -As a safety net for cases where the closure review was skipped or incomplete, -a cleanup script cancels any surviving open tasks in completed/archived -workstreams: - -```bash -cd ~/the-custodian/state-hub -make cleanup-stale # run immediately -# or add to cron: -# 0 3 * * * cd ~/the-custodian/state-hub && make cleanup-stale -``` - -The script (`scripts/cleanup_stale_tasks.py`) emits a `cleanup` progress event -recording which tasks were cancelled and in which workstreams. Tasks cancelled -by the cleanup carry a `blocking_reason` noting they should be verified against -the workplan file. - -The closure review is the primary mechanism; the cleanup is the fallback. If -the cleanup regularly cancels tasks, it signals that closure reviews are being -skipped — that is the process failure to address, not just the stale tasks. +**Let working copies push authoritative projection rows.** Rejected by ADR-012. +It makes the baseline depend on the last workstation to reconcile and cannot be +audited against a shared commit. ## Related -- Custodian Constitution v0.1 §2 (Powers) — canon changes require review gate -- ADR-000 (forthcoming) — overall Custodian architecture principles -- State Hub v0.3 workplan — `sync_workplans.py` is a Phase 4 deliverable -- `canon/values/foundational_values_v0.1.md` — Local-first, Auditability, - Reversibility +- ADR-003 — materialized and derived state +- ADR-005 — cross-repository work ownership +- ADR-007 — workplan identity and repository worker topology +- ADR-010 — Hub authority and local cache model +- ADR-011 — namespace-aware federation and reconciliation limits +- ADR-012 — Forge projection source and preliminary overlay +- `canon/standards/work-record-types_v0.1.md` — work-record kinds, lifecycle, + residuals, and reconciliation +- `canon/standards/workplan-terminology-fleet_v0.1.md` — canonical terminology +- `canon/values/foundational_values_v0.1.md` — local-first operation, + auditability, and reversibility diff --git a/canon/architecture/adr-003-materialized-derived-state.md b/canon/architecture/adr-003-materialized-derived-state.md index cdd15f4..ad3c46d 100644 --- a/canon/architecture/adr-003-materialized-derived-state.md +++ b/canon/architecture/adr-003-materialized-derived-state.md @@ -4,8 +4,8 @@ type: architecture-decision-record title: "Materialized Derived State with Fingerprint Invalidation for Repo-Sourced Data" status: accepted owner: the-custodian -revision: "accepted-1" -last_reviewed: "2026-03-20" +revision: "accepted-2" +last_reviewed: "2026-08-31" review_interval: 6m decided_by: Bernd Worsch date: "2026-03-20" diff --git a/canon/architecture/adr-007-workplan-identity-and-repo-worker-topology.md b/canon/architecture/adr-007-workplan-identity-and-repo-worker-topology.md index bab998c..dfb4769 100644 --- a/canon/architecture/adr-007-workplan-identity-and-repo-worker-topology.md +++ b/canon/architecture/adr-007-workplan-identity-and-repo-worker-topology.md @@ -4,8 +4,8 @@ type: architecture-decision-record title: "Workplan Identity Uniqueness, Single Registrar, and Repo Worker Topology" status: accepted owner: the-custodian -revision: "accepted-1" -last_reviewed: "2026-08-17" +revision: "accepted-2" +last_reviewed: "2026-08-31" review_interval: 6m decided_by: Bernd Worsch date: "2026-08-16" diff --git a/canon/architecture/adr-010-hub-authority-and-local-cache-model.md b/canon/architecture/adr-010-hub-authority-and-local-cache-model.md index 37568e3..04e4184 100644 --- a/canon/architecture/adr-010-hub-authority-and-local-cache-model.md +++ b/canon/architecture/adr-010-hub-authority-and-local-cache-model.md @@ -4,8 +4,8 @@ type: architecture-decision-record title: "Hub Authority, Local Cache, and the Two Kinds of Hub Data" status: proposed owner: the-custodian -revision: "draft-1" -last_reviewed: "2026-08-17" +revision: "draft-2" +last_reviewed: "2026-08-31" review_interval: 6m decided_by: Bernd Worsch date: "2026-08-17" diff --git a/canon/architecture/coulomb-estate_v0.1.md b/canon/architecture/coulomb-estate_v0.1.md index ef3c3b6..565a92e 100644 --- a/canon/architecture/coulomb-estate_v0.1.md +++ b/canon/architecture/coulomb-estate_v0.1.md @@ -3,9 +3,9 @@ id: coulomb-estate-architecture title: "Coulomb estate architecture" status: proposed owner: the-custodian -revision: "draft-2" +revision: "draft-3" version: "0.1" -last_reviewed: "2026-08-19" +last_reviewed: "2026-08-31" review_interval: 6m --- @@ -174,26 +174,28 @@ already in §4 and the ADRs in §9. ## 9. Architecture Decisions -Estate ADRs in `the-custodian/canon/architecture/`. Publication ids -still need repo prefixes (WP-0003). Status is the source front-matter. +Estate ADRs in `the-custodian/canon/architecture/`. Status is the source +front-matter. | Id | Status | Decision | | --- | --- | --- | -| `CUST-ADR-001` | accepted | Workplans and tasks originate as repo files; the hub is a read model. | +| `CUST-ADR-001` | accepted | File-backed work originates in repositories; the Hub projects it and separately owns declared Hub-native records. | | `CUST-ADR-002` | accepted | Custodian agent runtime design. | | `CUST-ADR-003` | accepted | Hub state is derived and fingerprint-invalidated. | | `CUST-ADR-004` | accepted | Connectivity-first network posture. | | `CUST-ADR-005` | accepted | Cross-repo work lives in a `prj-*` repo. | | `CUST-ADR-006` | accepted | Three canons; import, do not redefine. | -| `CUST-ADR-007` | accepted | Globally unique workplan ids; one registrar. | +| `CUST-ADR-007` | accepted | Namespace-aware work-record identity, deterministic Hub ids, and repository worker topology. | | `adr-008` | superseded | Relocated to NetKingdom Tenancy Posture. | | `CUST-ADR-010` | proposed | Two kinds of hub data; local cache vs authority. | | `CUST-ADR-011` | proposed | Federated namespaces and reconciliation limits. | +| `CUST-ADR-012` | accepted | Forge is the projection source; unpushed working-copy state is preliminary. | Related and published elsewhere: policy-nexus ADR-0001; Tenancy Posture -and IAM Profile v0.3; railiance-master ADR-0001–0008; activity-core -ACT-ADR-001–005; ops-warden ADR-0001–0005; railiance-platform -RPLAT-ADR-0001–0003. +and IAM Profile v0.3 plus the current NetKingdom security standards; +NetKingdom ADR-0006–0008 and 0010–0015; railiance-master ADR-0001–0009; +activity-core ACT-ADR-001–007; ops-warden ADR-0001–0005 and 0007–0010; +railiance-platform RPLAT-ADR-0001–0003. Unresolved WP-0003 conflicts (hosts/infra duplicate ADR-003/004, coulomb-social ADR-0002 partial supersession) are **not** listed as @@ -210,8 +212,9 @@ mechanical checks (`fix-consistency`, `make currency`). ## 11. Risks and Technical Debt -N/A in this revision. Known residual: this workstation cannot mint hub -UUIDs (ADR-007 registrar). Markitect `arc42-v1` is not registered yet. +N/A in this revision. Known residuals are the Forge-derived reset and +preliminary-overlay implementation under ADR-012. Markitect `arc42-v1` is not +registered yet. ---