diff --git a/.claude/rules/agents.md b/.claude/rules/agents.md new file mode 100644 index 0000000..0e8a5d9 --- /dev/null +++ b/.claude/rules/agents.md @@ -0,0 +1,20 @@ +## Kaizen Agents + +Specialized agent personas available on demand via the state-hub MCP. + +**Discover:** `list_kaizen_agents()` — returns all agents with name, description, category +**Load:** `get_kaizen_agent("tdd-workflow")` — returns full instructions; read and follow them + +Common agents: + +| Agent | Category | When to use | +|-------|----------|-------------| +| `tdd-workflow` | testing | Step-by-step TDD8 workflow for any feature | +| `code-refactoring` | quality | Code quality analysis and safe refactoring | +| `test-maintenance` | testing | Diagnose and fix failing tests | +| `requirements-engineering` | process | Prevent interface/mock mismatches upfront | +| `keepaTodofile` | process | Maintain TODO.md during work | +| `project-management` | process | Track status, determine next steps | +| `datamodel-optimization` | quality | Optimize dataclasses and data structures | + +All 17 agents: call `list_kaizen_agents()` for the full list. diff --git a/.claude/rules/architecture.md b/.claude/rules/architecture.md new file mode 100644 index 0000000..7c2a645 --- /dev/null +++ b/.claude/rules/architecture.md @@ -0,0 +1,8 @@ +## Architecture + + + +## Quick Reference + +`~/state-hub/mcp_server/TOOLS.md` — MCP tool reference diff --git a/.claude/rules/credential-routing.md b/.claude/rules/credential-routing.md new file mode 100644 index 0000000..bb4cf67 --- /dev/null +++ b/.claude/rules/credential-routing.md @@ -0,0 +1,50 @@ +# Credential and access routing + +**Audience:** Codex, Claude Code, Grok, and custodian agents that call **llm-connect** +for inference. Run this check **before** requesting secrets, API keys, SSH access, +login tokens, or database passwords — in any repo, not only `ops-warden`. + +ops-warden **issues SSH certificates only** (`warden sign`, `cert_command`). Every +other credential need belongs to another subsystem. **Do not** message +`ops-warden` on State Hub expecting a secret value; the reply is a pointer, not a key. + +### Lookup (do this first) + +```bash +warden route find "" --json +warden route show --json +``` + +Requires the `warden` CLI from `~/ops-warden` (`uv tool install .` or `uv run warden`). + +| Agent runtime | How to orient | +| --- | --- | +| **Codex / Grok** (shell, HTTP State Hub) | `warden route` commands above; inbox `to_agent=shard-wiki` is for coordination, not secret vending | +| **Claude Code** (MCP when available) | `get_domain_summary("custodian")` for workstreams; **still** use `warden route` for credential ownership | +| **llm-connect** (inference service) | Never put secret retrieval in prompts; route custody to OpenBao/operator paths surfaced by `warden route` | + +### Quick routing table + +| I need… | Owner | ops-warden executes? | +| --- | --- | --- | +| SSH cert (`adm`/`agt`/`atm`) | ops-warden | **Yes** — `warden sign` | +| API key, DB password, provider token | OpenBao (`railiance-platform`) | No — route only | +| Login / OIDC / MFA | key-cape / Keycloak | No — route only | +| Authorization decision | flex-auth | No — route only | +| activity-core → issue-core emission | activity-core + issue-core | No — `warden route show activity-core-issue-sink` | +| SSH tunnel | ops-bridge (+ `cert_command` from warden) | No — route only | + +### Anti-patterns (do not do these) + +- `POST /messages/` to `ops-warden` asking for `ISSUE_CORE_API_KEY`, `OPENROUTER_API_KEY`, etc. +- Inventing `warden secret`, `warden login`, `warden bao`, `warden tunnel` — they do not exist +- Pasting secrets into Git, State Hub, workplans, logs, or chat + +### Other capabilities (reuse-surface) + +Non-credential capabilities are usually discovered through **reuse-surface** federation +(`reuse-surface` registry / `capability.*` indexes). Credential routing is inlined in +every repo's agent instructions because it is high-frequency, high-risk, and easy to +get wrong. + +**Canon:** `~/ops-warden/wiki/CredentialRouting.md` · catalog `~/ops-warden/registry/routing/catalog.yaml` \ No newline at end of file diff --git a/.claude/rules/first-session.md b/.claude/rules/first-session.md new file mode 100644 index 0000000..0763ae3 --- /dev/null +++ b/.claude/rules/first-session.md @@ -0,0 +1,38 @@ +## First Session Protocol + +Triggered when `get_domain_summary("consumer")` shows **no workstreams**. +The project is registered but work has not yet been structured. + +**Step 1 — Read, don't write** +- `~/the-custodian/canon/projects/consumer/project_charter_v0.1.md` — purpose, scope +- `~/the-custodian/canon/projects/consumer/roadmap_v0.1.md` — planned phases +- Scan repo root: README, directory structure, existing code or docs + +**Step 2 — Survey in-progress work** +Look for TODOs, open branches, half-finished files. Note done vs. started but incomplete. + +**Step 3 — Propose workstreams to Bernd** +Propose 1–3 workstreams — each a coherent strand, weeks to months, anchored to a +roadmap phase. **Wait for approval before creating.** + +**Step 4 — Create workplan file first, then DB record (ADR-001)** +``` +workplans/SHARD-WP-NNNN-.md ← write this first +``` +Then register in the hub: +``` +create_workstream(topic_id="4c2e5315-2cb9-447c-9d16-a39bdb0aabd0", title="...", owner="...", description="...") +create_task(workstream_id="", title="...", priority="high|medium|low") +``` + +**Step 5 — Record the setup** +``` +add_progress_event( + summary="First session: structured consumer into N workstreams, M tasks", + event_type="milestone", + topic_id="4c2e5315-2cb9-447c-9d16-a39bdb0aabd0", + detail={"workstreams": [...], "tasks_created": M} +) +``` + + diff --git a/.claude/rules/repo-boundary.md b/.claude/rules/repo-boundary.md new file mode 100644 index 0000000..3ea5992 --- /dev/null +++ b/.claude/rules/repo-boundary.md @@ -0,0 +1,8 @@ +## Repo boundary + +This repo owns **shard-wiki** only. It does not own: + + diff --git a/.claude/rules/repo-identity.md b/.claude/rules/repo-identity.md new file mode 100644 index 0000000..fdbf9ff --- /dev/null +++ b/.claude/rules/repo-identity.md @@ -0,0 +1,5 @@ +**Purpose:** Git-based Markdown wiki orchestrator and federation layer. Python (src/ layout, hatchling, pytest). Early-stage: scaffold + INTENT.md defined, domain model not yet implemented. See INTENT.md for authoritative scope. + +**Domain:** consumer +**Repo slug:** shard-wiki +**Topic ID:** 4c2e5315-2cb9-447c-9d16-a39bdb0aabd0 diff --git a/.claude/rules/session-protocol.md b/.claude/rules/session-protocol.md new file mode 100644 index 0000000..fd3e771 --- /dev/null +++ b/.claude/rules/session-protocol.md @@ -0,0 +1,85 @@ +## Session Protocol + +Dev Hub (State Hub API): http://127.0.0.1:8000 +MCP server name in `~/.claude.json`: `dev-hub` + +**Step 1 — Orient** + +Read the offline-safe brief first — it works without a live hub connection: +```bash +cat .custodian-brief.md +``` +Then call the MCP tool for richer cross-domain context when MCP tools are exposed: +``` +get_domain_summary("consumer") +``` +If MCP tools are unavailable in the current agent session, use the REST API: +```bash +curl -s "http://127.0.0.1:8000/state/summary" | python3 -m json.tool +``` +If the hub is offline: `cd ~/state-hub && make api` + +**Step 2 — Check inbox** +With MCP tools: +``` +get_messages(to_agent="shard-wiki", unread_only=True) +``` +Mark read with `mark_message_read(message_id)`. Reply or act on coordination +requests before proceeding. + +Without MCP tools: +```bash +curl -s "http://127.0.0.1:8000/messages/?to_agent=shard-wiki&unread_only=true" \ + | python3 -m json.tool +curl -s -X PATCH "http://127.0.0.1:8000/messages//read" \ + -H "Content-Type: application/json" -d '{}' +``` + +**Step 3 — Scan workplans** +```bash +ls workplans/ +``` +For each file with `status: ready`, `active`, or `blocked`, note pending +`wait`/`todo`/`progress` tasks. + +**Step 4 — Present brief** + +1. **Active workstreams** for `consumer` — title, task counts, blocking decisions +2. **Pending tasks** from `workplans/` + any `[repo:shard-wiki]` hub tasks +3. **Goal guidance** — if `goal_guidance` in summary: + - `needs_workplan`: surface as top action — *"Repo goal '{title}' has no workplan yet"* + - `alignment_warnings`: flag if active work is not aligned with current goal +4. **Suggested next action** — highest-priority open item +5. **SBOM status** — flag if `last_sbom_at` is unset for this repo + +If no workstreams: follow First Session Protocol (`first-session.md`). + +**During work:** `record_decision()` · `add_progress_event()` · `resolve_decision()` + +> State Hub is a *read model*. Bootstrap tools (`create_workstream`, `create_task`) +> are First Session Protocol only. Work structure belongs in repo files (ADR-001). + +**Session close:** +With MCP tools: +``` +add_progress_event(summary="...", topic_id="4c2e5315-2cb9-447c-9d16-a39bdb0aabd0", workstream_id="") +``` +Without MCP tools: +```bash +curl -s -X POST http://127.0.0.1:8000/progress/ \ + -H "Content-Type: application/json" \ + -d '{"topic_id":"4c2e5315-2cb9-447c-9d16-a39bdb0aabd0","workstream_id":"","event_type":"note","summary":"what changed","author":"codex"}' +``` +If workplan files were modified, ensure the local copy is up to date first: +```bash +git -C pull --ff-only +cd ~/state-hub && make fix-consistency REPO=shard-wiki +``` +For repos where implementation runs on a remote machine (e.g. CoulombCore), +use the combined target which pulls before fixing: +```bash +cd ~/state-hub && make fix-consistency-remote REPO=shard-wiki +``` +**C-15** (DB task ahead of file) is normal in multi-machine workflows — writeback +will sync the file to match DB. **C-16** (repo behind remote) blocks all writes +until you pull — intentional to prevent clobbering remote progress. diff --git a/.claude/rules/stack-and-commands.md b/.claude/rules/stack-and-commands.md new file mode 100644 index 0000000..dc53ac6 --- /dev/null +++ b/.claude/rules/stack-and-commands.md @@ -0,0 +1,19 @@ +## Stack + + +- **Language:** +- **Key deps:** + +## Dev Commands + +```bash +# TODO: Fill in the standard commands for this repo + +# Install dependencies + +# Run tests + +# Lint / type check + +# Build / package (if applicable) +``` diff --git a/.claude/rules/workplan-convention.md b/.claude/rules/workplan-convention.md new file mode 100644 index 0000000..adb9d4e --- /dev/null +++ b/.claude/rules/workplan-convention.md @@ -0,0 +1,40 @@ +## Workplan Convention (ADR-001) + +File location: `workplans/SHARD-WP-NNNN-.md` +ID prefix: `SHARD-WP-` + +Work items originate as files in this repo **before** being registered in the hub. + +Canonical workplan/workstream frontmatter statuses are: +`proposed`, `ready`, `active`, `blocked`, `backlog`, `finished`, `archived`. +Use `proposed` for a newly drafted plan, `ready` after review against current +repo state, and `finished` when implementation is complete. `stalled` and +`needs_review` are derived health labels, not stored statuses. + +Closed workplans may be moved to `workplans/archived/` with a completion-date +prefix: `YYMMDD-SHARD-WP-NNNN-.md`. The frontmatter id remains +unchanged; the prefix is only for quick visual reference. + +Small opportunistic tasks discovered during another session use **Ad Hoc Tasks**: +`workplans/ADHOC-YYYY-MM-DD.md`, workstream slug `adhoc-YYYY-MM-DD`, and task ids +`ADHOC-YYYY-MM-DD-T01`, `T02`, etc. Use adhocs only for low-risk work completed +directly. Promote anything requiring analysis, design, approval, dependencies, or +multiple planned phases into a normal workplan. + +Ecosystem todos from other agents arrive as `[repo:shard-wiki]` hub tasks — +visible at session start. Pick one up by creating the workplan file, then registering +the workstream. + +Task blocks use this shape: + +```task +id: SHARD-WP-NNNN-T01 +status: wait | todo | progress | done | cancel +priority: high | medium | low +state_hub_task_id: "" # written by fix-consistency — do not edit +``` + +Status progression is `todo` → `progress` → `done`; use `wait` for waiting or +blocked work and `cancel` for stopped work. + + diff --git a/.forgejo/workflows/ci-smoke.yaml b/.forgejo/workflows/ci-smoke.yaml new file mode 100644 index 0000000..bd44c56 --- /dev/null +++ b/.forgejo/workflows/ci-smoke.yaml @@ -0,0 +1,29 @@ +# Canonical CI smoke template (tier 1 routing drill). +# Copy to: .forgejo/workflows/ci-smoke.yaml in consumer repos. +name: CI Smoke + +on: + push: + branches: + - main + workflow_dispatch: + +jobs: + host-smoke: + runs-on: self-hosted + steps: + - name: Routing probe (host runner) + run: | + set -eu + echo "repository=${GITHUB_REPOSITORY:-unknown}" + echo "sha=${GITHUB_SHA:-unknown}" + echo "runner=${RUNNER_NAME:-unknown}" + uname -a + + container-smoke: + runs-on: ubuntu-latest + steps: + - name: Routing probe (container label) + run: | + set -eu + echo "container-smoke ok for ${GITHUB_REPOSITORY:-unknown}" \ No newline at end of file diff --git a/.repo-classification.yaml b/.repo-classification.yaml new file mode 100644 index 0000000..0437b21 --- /dev/null +++ b/.repo-classification.yaml @@ -0,0 +1,17 @@ +repo_classification: + standard: Repo Classification Standard + version: '1.0' + classified_at: '2026-06-22' + classified_by: agent + category: project + domain: consumer + secondary_domains: [] + capability_tags: + - knowledge + - documentation + business_stake: + - product + - experience + business_mechanics: + - coordination + - operation diff --git a/AGENTS.md b/AGENTS.md index ff504ab..a1f55be 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,62 +1,219 @@ -# AGENTS.md +# shard-wiki — Agent Instructions -Guidance for agents working in `shard-wiki`. +## Repo Identity -## Read First +**Purpose:** Git-based Markdown wiki orchestrator and federation layer. Python (src/ layout, hatchling, pytest). Early-stage: scaffold + INTENT.md defined, domain model not yet implemented. See INTENT.md for authoritative scope. -1. `INTENT.md` — aspiration and boundaries (stable; architectural changes are rare). -2. `SCOPE.md` — what we are achieving now and current maturity. -3. `.custodian-brief.md` — State Hub snapshot (generated; do not edit manually). +**Domain:** consumer +**Repo slug:** shard-wiki +**Topic ID:** `4c2e5315-2cb9-447c-9d16-a39bdb0aabd0` +**Workplan prefix:** `SHARD-WP-` -## Documentation Layout +--- -This repo follows the CoulombSocial / HelixForge / MarkiTect documentation -layout (recommendation, not strict law). Efficient retrieval by purpose: +## State Hub Integration -| Path | Purpose | -|------|---------| -| `INTENT.md` | Aspiration and boundaries | -| `SCOPE.md` | Top-level view of current achievement; closes gap to INTENT | -| `research/` | Exploration results (`yymmdd-` prefix on files or subdirs) | -| `demand/` | Inbound requests not yet reviewed into spec or workplans | -| `spec/` | Implementation guardrails (PRD, TSD, use cases, architecture) | -| `workplans/` | State Hub–registered implementation tasks | -| `docs/` | Stakeholder documentation (users, developers, humans, agents) | -| `wiki/` | Perspective-free interconnected knowledge (wiki UI when connected) | -| `issues/` | Mirror of relevant open tickets when ticket systems are in use | -| `history/` | Archived material (`yymmdd-` prefix); out of scope for daily work | +The Custodian State Hub tracks work across all domains. Interact via HTTP REST — +there is no MCP server for Codex agents. -**Mode of operation:** close SCOPE → INTENT while learning; refine both as needed. +| Context | URL | +|---------|-----| +| Local workstation | `http://127.0.0.1:8000` | +| Remote via tunnel | `http://127.0.0.1:18000` | -## Domain Vocabulary - -Honor terms from `INTENT.md`: shard, root entity, adapter contract, projection, -overlay, coordination journal, shard modes. Do not invent parallel vocabulary. - -## Build And Test +### Orient at session start ```bash -pip install -e ".[dev]" -pytest -ruff check -ruff format +# Offline brief — works without hub connection +cat .custodian-brief.md + +# Active workstreams for this domain +curl -s "http://127.0.0.1:8000/workstreams/?topic_id=4c2e5315-2cb9-447c-9d16-a39bdb0aabd0&status=active" \ + | python3 -m json.tool + +# Check inbox +curl -s "http://127.0.0.1:8000/messages/?to_agent=shard-wiki&unread_only=true" \ + | python3 -m json.tool ``` -## State Hub +Mark a message read: +```bash +curl -s -X PATCH "http://127.0.0.1:8000/messages//read" \ + -H "Content-Type: application/json" -d '{}' +``` -Workplans register with State Hub. After workplan changes: +### Log progress (required at session close) ```bash -cd ~/state-hub && make fix-consistency REPO=shard-wiki +curl -s -X POST http://127.0.0.1:8000/progress/ \ + -H "Content-Type: application/json" \ + -d '{ + "summary": "what was done", + "event_type": "note", + "author": "codex", + "workstream_id": "", + "task_id": "" + }' ``` -Finished or canceled workplans move to `history/` with a `yymmdd-` archive prefix. +Omit `workstream_id` / `task_id` when not applicable. -## Where To Put New Material +### Update task status -- Exploratory analysis → `research/yymmdd-/` -- Raw feature ask or external requirement → `demand/` -- Reviewed design ready to guide code → `spec/` -- Implementation tasks → `workplans/` -- User/dev/agent how-to → `docs/` -- Collaborative unstructured notes → `wiki/` \ No newline at end of file +```bash +curl -s -X PATCH "http://127.0.0.1:8000/tasks/" \ + -H "Content-Type: application/json" \ + -d '{"status": "progress"}' +# values: wait | todo | progress | done | cancel +``` + +### Flag a task for human review + +```bash +curl -s -X PATCH "http://127.0.0.1:8000/tasks/" \ + -H "Content-Type: application/json" \ + -d '{"needs_human": true, "intervention_note": "reason"}' +``` + +--- + +## Session Protocol + +**Start:** +1. `cat .custodian-brief.md` — domain goal and open workstreams (offline-safe) +2. Check inbox: `GET /messages/?to_agent=shard-wiki&unread_only=true`; mark read +3. Scan workplans: `ls workplans/` — note `status: ready`, `active`, or `blocked` files and open tasks +4. Check human-needed tasks: `GET /tasks/?needs_human=true` + +**During work:** +- Update task statuses in workplan files as tasks progress +- Record significant decisions via `POST /decisions/` + +**Close:** +1. Update workplan file task statuses to reflect progress +2. Log: `POST /progress/` with a summary of what changed +3. Note for the custodian operator: after workplan file changes, run from + `~/state-hub`: + ```bash + make fix-consistency REPO=shard-wiki + ``` + This syncs task status from files into the hub DB. + +--- + +## Credential and access routing + +**Audience:** Codex, Claude Code, Grok, and custodian agents that call **llm-connect** +for inference. Run this check **before** requesting secrets, API keys, SSH access, +login tokens, or database passwords — in any repo, not only `ops-warden`. + +ops-warden **issues SSH certificates only** (`warden sign`, `cert_command`). Every +other credential need belongs to another subsystem. **Do not** message +`ops-warden` on State Hub expecting a secret value; the reply is a pointer, not a key. + +### Lookup (do this first) + +```bash +warden route find "" --json +warden route show --json +``` + +Requires the `warden` CLI from `~/ops-warden` (`uv tool install .` or `uv run warden`). + +| Agent runtime | How to orient | +| --- | --- | +| **Codex / Grok** (shell, HTTP State Hub) | `warden route` commands above; inbox `to_agent=shard-wiki` is for coordination, not secret vending | +| **Claude Code** (MCP when available) | `get_domain_summary("custodian")` for workstreams; **still** use `warden route` for credential ownership | +| **llm-connect** (inference service) | Never put secret retrieval in prompts; route custody to OpenBao/operator paths surfaced by `warden route` | + +### Quick routing table + +| I need… | Owner | ops-warden executes? | +| --- | --- | --- | +| SSH cert (`adm`/`agt`/`atm`) | ops-warden | **Yes** — `warden sign` | +| API key, DB password, provider token | OpenBao (`railiance-platform`) | No — route only | +| Login / OIDC / MFA | key-cape / Keycloak | No — route only | +| Authorization decision | flex-auth | No — route only | +| activity-core → issue-core emission | activity-core + issue-core | No — `warden route show activity-core-issue-sink` | +| SSH tunnel | ops-bridge (+ `cert_command` from warden) | No — route only | + +### Anti-patterns (do not do these) + +- `POST /messages/` to `ops-warden` asking for `ISSUE_CORE_API_KEY`, `OPENROUTER_API_KEY`, etc. +- Inventing `warden secret`, `warden login`, `warden bao`, `warden tunnel` — they do not exist +- Pasting secrets into Git, State Hub, workplans, logs, or chat + +### Other capabilities (reuse-surface) + +Non-credential capabilities are usually discovered through **reuse-surface** federation +(`reuse-surface` registry / `capability.*` indexes). Credential routing is inlined in +every repo's agent instructions because it is high-frequency, high-risk, and easy to +get wrong. + +**Canon:** `~/ops-warden/wiki/CredentialRouting.md` · catalog `~/ops-warden/registry/routing/catalog.yaml` + + + + +--- + +## Workplan Convention (ADR-001) + +Work items originate as files in this repo — not in the hub. The hub is a +read/cache/index layer that rebuilds from files. + +**File location:** `workplans/SHARD-WP-NNNN-.md` + +**Archived location:** finished workplans may move to +`workplans/archived/YYMMDD-SHARD-WP-NNNN-.md`. The `YYMMDD` prefix is +the completion/archive date; the frontmatter `id` does not change. + +**Ad Hoc Tasks:** small opportunistic fixes discovered during a session use +`workplans/ADHOC-YYYY-MM-DD.md` with task ids `ADHOC-YYYY-MM-DD-T01`, etc. Use +this only for low-risk work completed directly; create a normal workplan for +anything needing analysis, design, approval, dependencies, or multiple phases. + +**Frontmatter:** + +```yaml +--- +id: SHARD-WP-NNNN +type: workplan +title: "..." +domain: consumer +repo: shard-wiki +status: proposed | ready | active | blocked | backlog | finished | archived +owner: codex +topic_slug: ... +created: "YYYY-MM-DD" +updated: "YYYY-MM-DD" +state_hub_workstream_id: "" # written by fix-consistency — do not edit +--- +``` + +Use `proposed` for a new draft, `ready` after review against current repo +state, and `finished` after implementation. `stalled` and `needs_review` are +derived health labels, not frontmatter statuses. + +**Task block format** (one per `##` section): + +``` +## Task Title + +` ` `task +id: SHARD-WP-NNNN-T01 +status: wait | todo | progress | done | cancel +priority: high | medium | low +state_hub_task_id: "" # written by fix-consistency — do not edit +` ` ` + +Task description text. +``` + +Status progression: `todo` → `progress` → `done`; use `wait` for waiting/blocked work and `cancel` for stopped work. + +To create a new workplan: +1. Write the file following the format above +2. Notify the custodian operator to run `make fix-consistency REPO=shard-wiki` + (or send a message to the hub agent via `POST /messages/`) diff --git a/CLAUDE.md b/CLAUDE.md index 1807be9..ecfac84 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,53 +1,12 @@ -# CLAUDE.md +# shard-wiki — Claude Code Instructions -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. - -## Repository status - -This is an **early-stage Python repository**. The package scaffold (`src/shard_wiki/`, `tests/`, `pyproject.toml`) exists with only smoke tests — the domain model is not yet implemented. Read `INTENT.md` (aspiration), `SCOPE.md` (current achievement), and `AGENTS.md` (layout and conventions) before designing anything. Close the gap from SCOPE to INTENT via `research/`, `spec/`, and `workplans/`. - -## What this project is - -`shard-wiki` is a **Git-based Markdown wiki orchestrator and federation layer**, not a wiki engine. It lets multiple heterogeneous wiki-shaped page stores (**shards**) attach to a shared root entity and be presented as a **union of pages**, while preserving each shard's separate storage, provenance, capabilities, and history. - -The core job is orchestration across backends — Git repos, repo subdirectories (`wiki/`), Gitea wikis, local folders, Obsidian vaults, WebDAV/Nextcloud directories, Coulomb spaces — never replacing or homogenizing them. - -## Core domain model (the concepts code must honor) - -These abstractions come from `INTENT.md` and define the architecture. New code should map onto them rather than inventing parallel vocabulary: - -- **Shard** — an independently meaningful page store attached to a root entity. Shards have *sovereignty*: their own backend, capabilities, limits, history, and identity model. Not all shards are Git-native. -- **Root entity / information space** — the joined space that shards attach to. Each information space should have a **Git-addressable coordination layer** (history, patches, review, backup, reconciliation) even when individual shards are not Git-native. -- **Shard adapter contract** — the versioned interface a backend implements to participate. Adapters are **capability-aware**: the core must model explicitly which operations a shard supports (read, write, diff, merge, lock, version, publish, accept patches) rather than assuming uniformity. -- **Wiki page model** — a stable, versioned, Markdown-first but backend-neutral representation of pages, paths, links, metadata, revisions. -- **Projection** — a lazy, cache-like local view of remote/external shard content. Prefer lazy projection over eager copying. -- **Overlay** — a non-destructive local edit against a remote, read-only, or capability-limited shard, representable as drafts/patches/commits/merge requests *before* destructive application ("overlay before mutation"). -- **Coordination journal** — the Git-backed record of change flows for an information space. -- **Shard modes** — read-only, write-through, mirrored, projected, cached, canonical. - -## Design constraints to enforce in code - -These are hard boundaries from `INTENT.md`; treat violations as design bugs: - -- **Mechanism over policy.** Provide primitives for federation, sync, overlays, patching, conflict detection, projection, reconciliation. Do *not* hard-code one editorial/sync/conflict/canonical-source policy — keep those configurable. -- **Union without erasure.** Always preserve provenance: which shard a page came from, its freshness, whether it is cached, whether it has overlays, whether it diverges from an equivalent page elsewhere. Never hide authorship, conflicts, freshness, or backend limitations. -- **No silent remote mutation.** Do not mutate remote systems without explicit adapter support and user intent. -- **Graceful degradation.** Limited backends must still be usable as read-only/cache/projection/backup/patch targets. -- **Not a file-sync daemon.** Synchronization is wiki-page-semantic, not generic file mirroring. - -`INTENT.md` has a "Stability Note": changes that redefine what a shard is, Git's role, how root entities are modeled, or whether this is an orchestrator vs. an engine are **architectural changes** and should be rare and deliberate. - -## Build, test, run - -Python with a `src/` layout, built via hatchling, tested with pytest. Tests run against the source tree directly (`pythonpath = ["src"]` in `pyproject.toml`), so no install/editable step is required to run them. - -```bash -pip install -e ".[dev]" # one-time: install dev tooling (pytest, pytest-cov, ruff) -pytest # run the full test suite -pytest tests/test_package.py::test_version_is_exposed # run a single test -pytest --cov # run with coverage -ruff check # lint -ruff format # format -``` - -Note: the system `pytest` is 7.4.x; `minversion` in `pyproject.toml` is pinned to `7.0` to match. Bump it if a newer pytest is installed into the dev environment. +@SCOPE.md +@.claude/rules/repo-identity.md +@.claude/rules/session-protocol.md +@.claude/rules/first-session.md +@.claude/rules/workplan-convention.md +@.claude/rules/stack-and-commands.md +@.claude/rules/architecture.md +@.claude/rules/repo-boundary.md +@.claude/rules/credential-routing.md +@.claude/rules/agents.md diff --git a/SCOPE.md b/SCOPE.md index 5dc4d0d..5d79b31 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -17,7 +17,7 @@ Learnings update both SCOPE and INTENT where necessary. | Layer | State | |-------|-------| -| Code | Foundation slice implemented (SHARD-WP-0007): `provenance` + `policy` leaves, `model` (Identity/Placement/Span/Page/CapabilityProfile), `adapters` (contract + FolderAdapter + conformance suite), `coordination` (event-sourced DecisionLog), `union` (resolution + chorus, overlay-aware), `InformationSpace` orchestrator. Write path added (SHARD-WP-0008): writable adapter, overlay engine (draft→patch→apply-under-drift), edit() unifies write-through + overlay-before-mutation. Native engine implemented (SHARD-WP-0014): `engine` (kernel + typed-extension runtime + per-shard activation [ADR-0001] + capability-profile-from-extensions + EngineShardAdapter + the `ext.struct` built-in) — an engine shard attaches to an InformationSpace as a canonical-mode shard. Git-backed coordination log (SHARD-WP-0009): `DecisionLog` storage factored behind an `EventStore`; `GitEventStore` makes the log git-addressable (each space a ref, append = immutable CAS-guarded commit), a per-space `AppendAuthority` (lease) gives a single-writer total order with re-grantable HA hand-off, cross-process read-your-writes verified, and a verbatim one-time importer (`migrate_space`/JSONL) replays in-memory logs into git; `InformationSpace.git_backed(...)` wires it. Derived views (SHARD-WP-0010): `views` (wikilink + red-link model, BackLinks, RecentChanges, AllPages/SiteMap) — recomputable, provenance-carrying, presentation-free, exposed via `InformationSpace.backlinks/recent_changes/all_pages/site_map`. 152 tests green, ~97% coverage | +| Code | Foundation slice implemented (SHARD-WP-0007): `provenance` + `policy` leaves, `model` (Identity/Placement/Span/Page/CapabilityProfile), `adapters` (contract + FolderAdapter + conformance suite), `coordination` (event-sourced DecisionLog), `union` (resolution + chorus, overlay-aware), `InformationSpace` orchestrator. Write path added (SHARD-WP-0008): writable adapter, overlay engine (draft→patch→apply-under-drift), edit() unifies write-through + overlay-before-mutation. Native engine implemented (SHARD-WP-0014): `engine` (kernel + typed-extension runtime + per-shard activation [ADR-0001] + capability-profile-from-extensions + EngineShardAdapter + the `ext.struct` built-in) — an engine shard attaches to an InformationSpace as a canonical-mode shard. Git-backed coordination log (SHARD-WP-0009): `DecisionLog` storage factored behind an `EventStore`; `GitEventStore` makes the log git-addressable (each space a ref, append = immutable CAS-guarded commit), a per-space `AppendAuthority` (lease) gives a single-writer total order with re-grantable HA hand-off, cross-process read-your-writes verified, and a verbatim one-time importer (`migrate_space`/JSONL) replays in-memory logs into git; `InformationSpace.git_backed(...)` wires it. Derived views (SHARD-WP-0010): `views` (wikilink + red-link model, BackLinks, RecentChanges, AllPages/SiteMap) — recomputable, provenance-carrying, presentation-free, exposed via `InformationSpace.backlinks/recent_changes/all_pages/site_map`. Incremental-first derived tier (SHARD-WP-0011): `incremental` (indexed equivalence via MinHash/LSH blocking + verify, change-driven delta maintenance with retraction/propagation, Merkle-style digest + self-healing I-2 consistency-checker, `UnionIndex` routed behind `InformationSpace.all_pages` with rebuild as explicit fallback). Second adapter (SHARD-WP-0012): `GitShardAdapter` — git-IS-store substrate (read=tracked *.md, write=commit, current_rev=per-path sha for drift, adopted git-native history), passes conformance, works across folder+git shards in union/overlay/edit with no core change (capability-as-data proven on a second substrate). 196 tests green, ~97% coverage | | Intent | `INTENT.md` established; authorization-in-core amendments drafted | | Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis + TiddlyWiki + ikiwiki + Quip + MojoMojo + Oddmuse + UseModWiki deep dives & shard-spectrum synthesis (`research/260614-*/`) | | Demand | NetKingdom integration asks captured, not yet negotiated | diff --git a/src/shard_wiki/adapters/__init__.py b/src/shard_wiki/adapters/__init__.py index 915678e..d469670 100644 --- a/src/shard_wiki/adapters/__init__.py +++ b/src/shard_wiki/adapters/__init__.py @@ -9,10 +9,13 @@ from shard_wiki.adapters.conformance import ( ) from shard_wiki.adapters.contract import CONTRACT_VERSION, ShardAdapter from shard_wiki.adapters.folder import FolderAdapter +from shard_wiki.adapters.git import GitShardAdapter, PageRevision __all__ = [ "ShardAdapter", "FolderAdapter", + "GitShardAdapter", + "PageRevision", "CONTRACT_VERSION", "Check", "ConformanceReport", diff --git a/src/shard_wiki/adapters/git.py b/src/shard_wiki/adapters/git.py new file mode 100644 index 0000000..a9d5ada --- /dev/null +++ b/src/shard_wiki/adapters/git.py @@ -0,0 +1,180 @@ +"""GitShardAdapter — a second substrate: git-as-store (SHARD-WP-0012; TSD §A.3 git-IS-store). + +The home case where **git is the store *and* the journal**. Tracked ``*.md`` paths are the page +keys; the working-tree file is the body; a page's ``source_rev`` is the **commit sha of the last +commit touching its path** (per-path, so an edit to one page never drifts another). The declared +profile is *git-IS-store ⟹ substrate=git ∧ history=git-native* — the implication rule the +capability model enforces (§6.5), validated at registration like any other binding. + +This adapter adds **no core changes**: it implements the same :class:`ShardAdapter` contract the +folder adapter does, proving "write an adapter + declare a verified profile" is the whole cost of a +new substrate (capability-as-data, I-3). Built on the ``git`` CLI via subprocess — zero new deps. +""" + +from __future__ import annotations + +import os +import subprocess +from collections.abc import Iterable +from dataclasses import dataclass +from pathlib import Path + +from shard_wiki.adapters.contract import ShardAdapter +from shard_wiki.model import ( + AccessGrant, + Addressing, + AttachmentMode, + CapabilityProfile, + ContentOpacity, + History, + Identity, + MergeModel, + NativeQuery, + NotSupported, + OperationalEnvelope, + Page, + Placement, + Substrate, + Translation, + Verb, + WriteGranularity, +) +from shard_wiki.provenance import Liveness, ProvenanceEnvelope, Staleness + +__all__ = ["GitShardAdapter", "PageRevision"] + + +@dataclass(frozen=True, slots=True) +class PageRevision: + """One adopted git-native revision of a page: the commit sha and its subject line.""" + + sha: str + message: str + +_GIT_IDENTITY = { + "GIT_AUTHOR_NAME": "shard-wiki", + "GIT_AUTHOR_EMAIL": "shard@shard-wiki", + "GIT_COMMITTER_NAME": "shard-wiki", + "GIT_COMMITTER_EMAIL": "shard@shard-wiki", +} + + +class GitShardAdapter(ShardAdapter): + """A shard whose store is a git repo: keys are tracked ``*.md`` paths, revs are commit shas.""" + + def __init__(self, shard_id: str, repo_path: str | Path, writable: bool = False) -> None: + self._shard_id = shard_id + self._repo = Path(repo_path) + self._writable = writable + self._repo.mkdir(parents=True, exist_ok=True) + if not (self._repo / ".git").exists(): + self._git("init", "--quiet") + + @property + def shard_id(self) -> str: + return self._shard_id + + def profile(self) -> CapabilityProfile: + # VERSION is always available — a git-IS-store has git-native history to adopt (§A.5), + # read-only or not. WRITE (= commit, PER_PAGE) is added only in writable mode. + verbs = {Verb.READ, Verb.VERSION} + granularity = WriteGranularity.NONE + if self._writable: + verbs |= {Verb.WRITE} + granularity = WriteGranularity.PER_PAGE + return CapabilityProfile( + substrate=Substrate.GIT, + attachment_mode=AttachmentMode.GIT_IS_STORE, + write_granularity=granularity, + content_opacity=ContentOpacity.TRANSPARENT, + operational_envelope=OperationalEnvelope.LOCAL_UNBOUNDED, + access_grant=AccessGrant.OPEN, + liveness=Liveness.STATIC, + history=History.GIT_NATIVE, # git-is-store ⟹ git-native (§6.5) + merge_model=MergeModel.GIT_TEXT, + addressing=Addressing.PATH, + native_query=NativeQuery.NONE, + translation=Translation.NATIVE, + supported_verbs=frozenset(verbs), + ).validate() + + def write(self, key: str, body: str) -> Page: + """Write = **commit**: stage the file and commit it (skip a no-op so no empty commit), + returning the page at the new sha. Drift detection rides on ``current_rev`` = that sha.""" + if not self._writable: + raise NotSupported(f"{type(self).__name__} is read-only") + rel = f"{key}.md" + path = self._path_for(key) + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(body, encoding="utf-8") + self._git("add", "--", rel) + if self._run("diff", "--cached", "--quiet").returncode != 0: # staged changes present + self._git("commit", "-m", f"write {rel}", env=_GIT_IDENTITY) + return self.read(key) + + def keys(self) -> Iterable[str]: + out = self._git("ls-files", "*.md").decode() + for line in out.splitlines(): + yield line[: -len(".md")] if line.endswith(".md") else line + + def read(self, key: str) -> Page: + path = self._path_for(key) + if not path.is_file(): + raise KeyError(key) + rev = self.current_rev(key) + return Page( + identity=Identity(self._shard_id, key), + body=path.read_text(encoding="utf-8"), + envelope=ProvenanceEnvelope( + source_shard=self._shard_id, + liveness=Liveness.STATIC, + staleness=Staleness.FRESH, + source_rev=rev, + lineage="git-native", + ), + placements=(Placement(self._shard_id, f"{key}.md"),), + ) + + def current_rev(self, key: str) -> str | None: + """The sha of the last commit touching ``key``'s path (per-path drift token), or None.""" + rel = f"{key}.md" + if not self._path_for(key).is_file(): + return None + sha = self._git("log", "-1", "--format=%H", "--", rel).decode().strip() + return sha or None + + def history(self, key: str) -> tuple[PageRevision, ...]: + """Adopt git-native history (§A.5): the commit list for ``key``'s path, newest-first. + + VERSION-gated; raises ``KeyError`` for an unknown page. Each revision is a commit sha + + subject — the native log surfaced through the contract, not re-implemented. + """ + if not self.profile().supports(Verb.VERSION): + raise NotSupported(f"{type(self).__name__} does not support version") + if not self._path_for(key).is_file(): + raise KeyError(key) + out = self._git("log", "--format=%H%x00%s", "--", f"{key}.md").decode() + revisions = [] + for line in out.splitlines(): + sha, _, message = line.partition("\x00") + revisions.append(PageRevision(sha=sha, message=message)) + return tuple(revisions) + + # -- git plumbing -------------------------------------------------------- + + def _path_for(self, key: str) -> Path: + return self._repo / f"{key}.md" + + def _git(self, *args: str, stdin: bytes | None = None, env: dict | None = None) -> bytes: + return self._run(*args, stdin=stdin, env=env, check=True).stdout + + def _run( + self, *args: str, stdin: bytes | None = None, env: dict | None = None, check: bool = False + ) -> subprocess.CompletedProcess: + return subprocess.run( + ["git", "-C", str(self._repo), *args], + input=stdin, + capture_output=True, + env={**os.environ, **(env or {})}, + check=check, + ) diff --git a/src/shard_wiki/incremental/__init__.py b/src/shard_wiki/incremental/__init__.py index 419c7a9..a3e8178 100644 --- a/src/shard_wiki/incremental/__init__.py +++ b/src/shard_wiki/incremental/__init__.py @@ -22,6 +22,13 @@ from shard_wiki.incremental.minhash import ( jaccard, shingles, ) +from shard_wiki.incremental.union_index import UnionIndex +from shard_wiki.incremental.verification import ( + ConsistencyChecker, + ConsistencyReport, + derived_digest, + region_digest, +) __all__ = [ "shingles", @@ -31,4 +38,9 @@ __all__ = [ "EquivalenceEdge", "EquivalenceIndex", "normalized_title", + "derived_digest", + "region_digest", + "ConsistencyReport", + "ConsistencyChecker", + "UnionIndex", ] diff --git a/src/shard_wiki/incremental/equivalence.py b/src/shard_wiki/incremental/equivalence.py index d59d961..e6852c9 100644 --- a/src/shard_wiki/incremental/equivalence.py +++ b/src/shard_wiki/incremental/equivalence.py @@ -134,8 +134,21 @@ class EquivalenceIndex: def unbind(self, a: Identity, b: Identity) -> None: self._curator_edges.discard(_pair(a, b)) + def set_curator_edges(self, edges: Iterable[tuple[Identity, Identity]]) -> None: + """Replace all curator edges at once (re-syncing from the decision-log fold).""" + self._curator_edges = {_pair(a, b) for a, b in edges if a != b} + # -- queries ------------------------------------------------------------- + def identities(self) -> frozenset[Identity]: + """All identities currently present in the index.""" + return frozenset(self._entries) + + def fingerprint(self, identity: Identity) -> str | None: + """The content fingerprint indexed for ``identity`` (None if absent) — a digest leaf.""" + entry = self._entries.get(identity) + return entry.fingerprint if entry is not None else None + def edges(self) -> frozenset[frozenset[Identity]]: """All equivalence edges (content + curator) among currently present identities.""" present = self._entries.keys() diff --git a/src/shard_wiki/incremental/union_index.py b/src/shard_wiki/incremental/union_index.py new file mode 100644 index 0000000..e5b0ec3 --- /dev/null +++ b/src/shard_wiki/incremental/union_index.py @@ -0,0 +1,91 @@ +"""UnionIndex — the maintained derived tier wired behind resolution + views (SHARD-WP-0011 T4). + +Wraps a :class:`UnionGraph` + decision log with an incrementally maintained +:class:`EquivalenceIndex`. Content equivalence is kept fresh by deltas (``note_change`` / +``note_removed``); curator bindings are re-synced live from the log fold. A full :meth:`rebuild` +is the bounded fallback. :meth:`verify` runs the I-2 consistency-checker over the live source. + +Consumer-visible results are unchanged — equivalence groups are exposed in the same string form the +decision-log fold uses, a *superset* that additionally collapses genuine content duplicates — only +freshness and cost differ (recompute-on-read becomes change-driven). +""" + +from __future__ import annotations + +from shard_wiki.coordination import DecisionLog +from shard_wiki.incremental.equivalence import EquivalenceIndex +from shard_wiki.incremental.verification import ( + ConsistencyChecker, + ConsistencyReport, + derived_digest, +) +from shard_wiki.model import Identity, Page +from shard_wiki.union import UnionGraph + +__all__ = ["UnionIndex"] + + +def _identity(token: str) -> Identity: + shard, _, key = token.partition(":") + return Identity(shard, key) + + +class UnionIndex: + """An incrementally maintained equivalence index over a union, with a rebuild fallback.""" + + def __init__(self, union: UnionGraph, log: DecisionLog, space: str) -> None: + self._union = union + self._log = log + self._space = space + self._eq = EquivalenceIndex() + self.rebuild() + + def rebuild(self) -> None: + """The bounded fallback: re-derive the whole index from current union pages + bindings.""" + self._eq.build(self._union.iter_pages()) + self._sync_curator() + + def note_change(self, page: Page) -> None: + """Change-driven update for one added/edited page (the operational path).""" + self._eq.update(page) + + def note_removed(self, identity: Identity) -> None: + self._eq.remove(identity) + + def _sync_curator(self) -> None: + """Re-sync curator equivalence from the live decision-log fold (cheap, always correct).""" + groups = self._log.fold(self._space).equivalence_groups + edges: list[tuple[Identity, Identity]] = [] + for group in groups: + members = [_identity(m) for m in group] + edges.extend((members[0], other) for other in members[1:]) + self._eq.set_curator_edges(edges) + + def equivalence_groups(self) -> tuple[frozenset[str], ...]: + """Equivalence groups in decision-log string form (curator ∪ content), for the views.""" + self._sync_curator() + return tuple( + frozenset(str(identity) for identity in group) for group in self._eq.groups() + ) + + def digest(self) -> str: + """The Merkle-style digest of the maintained derived tier (I-2).""" + self._sync_curator() + return derived_digest(self._eq) + + def verify(self) -> ConsistencyReport: + """Check the maintained index against a from-scratch fold of the live source; self-heal.""" + self._sync_curator() + checker = ConsistencyChecker( + self._eq, + pages=lambda: list(self._union.iter_pages()), + curator_edges=self._curator_pairs, + ) + return checker.check_and_repair() + + def _curator_pairs(self) -> list[tuple[Identity, Identity]]: + pairs: list[tuple[Identity, Identity]] = [] + for group in self._log.fold(self._space).equivalence_groups: + members = [_identity(m) for m in group] + pairs.extend((members[0], other) for other in members[1:]) + return pairs diff --git a/src/shard_wiki/incremental/verification.py b/src/shard_wiki/incremental/verification.py new file mode 100644 index 0000000..4c31f34 --- /dev/null +++ b/src/shard_wiki/incremental/verification.py @@ -0,0 +1,112 @@ +"""I-2 verification — digest + background consistency-checker (SHARD-WP-0011 T3). + +``derived = f(canonical)`` is made *verified*, not asserted. A **Merkle-style digest** summarizes +the derived tier (each identity's content fingerprint + its incident equivalence edges as a leaf, +order-independently combined into a root) so two derived states are equal iff their digests match. +A **consistency-checker** recomputes the authoritative fold from the current source, compares it to +the maintained index over a (sampled) region, and on mismatch performs a **scoped recompute** of +just the affected identities — self-healing drift from a missed delta or corrupted state. + +The digest is a pure function of index state, so it is "maintained alongside deltas" for free and +is stable under equivalent event orders (leaves are sorted before combination). +""" + +from __future__ import annotations + +import hashlib +from collections.abc import Callable, Iterable +from dataclasses import dataclass + +from shard_wiki.incremental.equivalence import EquivalenceIndex +from shard_wiki.model import Identity, Page + +__all__ = ["region_digest", "derived_digest", "ConsistencyReport", "ConsistencyChecker"] + +CuratorEdges = Iterable[tuple[Identity, Identity]] + + +def _leaf(index: EquivalenceIndex, identity: Identity) -> str: + """A digest leaf for one identity: its fingerprint + its incident edges (as sorted peers).""" + fingerprint = index.fingerprint(identity) or "∅" + peers = sorted( + str(other) + for edge in index.edges() + if identity in edge + for other in edge + if other != identity + ) + payload = f"{identity}|{fingerprint}|{','.join(peers)}" + return hashlib.blake2b(payload.encode("utf-8"), digest_size=16).hexdigest() + + +def region_digest(index: EquivalenceIndex, identities: Iterable[Identity]) -> str: + """A Merkle-style root over the given identities' leaves (order-independent).""" + leaves = sorted(_leaf(index, identity) for identity in identities) + root = hashlib.blake2b(digest_size=16) + for leaf in leaves: + root.update(leaf.encode("utf-8")) + return root.hexdigest() + + +def derived_digest(index: EquivalenceIndex) -> str: + """The digest of the whole maintained derived tier.""" + return region_digest(index, index.identities()) + + +@dataclass(frozen=True, slots=True) +class ConsistencyReport: + """Outcome of a consistency check: what was examined, whether it drifted, and if it healed.""" + + checked: int + drifted: bool + repaired: bool + healthy: bool + + +class ConsistencyChecker: + """Compares the maintained index against an authoritative rebuild and repairs drift in place.""" + + def __init__( + self, + index: EquivalenceIndex, + pages: Callable[[], Iterable[Page]], + curator_edges: Callable[[], CuratorEdges] = lambda: (), + ) -> None: + self._index = index + self._pages = pages + self._curator = curator_edges + + def _authoritative(self) -> EquivalenceIndex: + expected = EquivalenceIndex( + num_bands=self._index.num_bands, threshold=self._index.threshold + ) + expected.build(list(self._pages()), list(self._curator())) + return expected + + def check_and_repair(self, sample: Iterable[Identity] | None = None) -> ConsistencyReport: + """Verify the (sampled) region against a from-scratch fold; scoped-recompute on mismatch.""" + source = {p.identity: p for p in self._pages()} + expected = self._authoritative() + region = ( + set(sample) + if sample is not None + else set(source) | set(self._index.identities()) + ) + + drifted = region_digest(self._index, region) != region_digest(expected, region) + if not drifted: + return ConsistencyReport(len(region), drifted=False, repaired=False, healthy=True) + + self._repair(region, source) + healthy = region_digest(self._index, region) == region_digest(expected, region) + return ConsistencyReport(len(region), drifted=True, repaired=True, healthy=healthy) + + def _repair(self, region: set[Identity], source: dict[Identity, Page]) -> None: + """Scoped recompute: reconcile each affected identity to the current source.""" + present = self._index.identities() + for identity in region: + page = source.get(identity) + if page is not None: + self._index.update(page) if identity in present else self._index.add(page) + elif identity in present: + self._index.remove(identity) diff --git a/src/shard_wiki/space.py b/src/shard_wiki/space.py index 32c4532..5f0a87a 100644 --- a/src/shard_wiki/space.py +++ b/src/shard_wiki/space.py @@ -20,6 +20,7 @@ from shard_wiki.coordination import ( Overlay, OverlayEngine, ) +from shard_wiki.incremental import ConsistencyReport, UnionIndex from shard_wiki.model import Page from shard_wiki.policy import DEFAULT_POLICY, Policy from shard_wiki.union import Resolution, UnionGraph @@ -51,6 +52,8 @@ class InformationSpace: self.log = DecisionLog(store) self.union = UnionGraph(space_id, log=self.log, policy=policy) self.overlays = OverlayEngine(space_id, self.log) + self._index: UnionIndex | None = None # maintained derived tier, built lazily + self._index_stale = True @classmethod def git_backed( @@ -67,6 +70,7 @@ class InformationSpace: """Attach a shard — only if it passes conformance (verified profile, I-3/§6.6).""" assert_conformant(adapter) self.union.attach(adapter) + self._index_stale = True def alias(self, name: str, target: str, actor: str | None = None) -> None: """Record a coordination-canonical alias (``name`` → ``"shard:key"``) in the log.""" @@ -101,7 +105,29 @@ class InformationSpace: write-through-capable target fast-forwards (write-through); a read-only target keeps the draft as local truth (I-5: overlay before mutation, always).""" overlay = self.overlay(name, body, actor=actor) - return self.apply_overlay(overlay.overlay_id) + result = self.apply_overlay(overlay.overlay_id) + self._index_stale = True # the applied edit changes the derived tier + return result + + # --- maintained derived tier (SHARD-WP-0011): incremental-first, rebuild as fallback --- + + @property + def index(self) -> UnionIndex: + """The maintained equivalence index (built lazily; rebuilt when the union has changed).""" + if self._index is None: + self._index = UnionIndex(self.union, self.log, self.space_id) + elif self._index_stale: + self._index.rebuild() # bounded fallback after a mutation + self._index_stale = False + return self._index + + def reindex(self) -> None: + """Force a full rebuild of the maintained derived tier (the explicit fallback path).""" + self.index.rebuild() + + def verify_index(self) -> ConsistencyReport: + """Run the I-2 consistency-checker over the maintained tier; self-heal any drift.""" + return self.index.verify() # --- derived views (SHARD-WP-0010): recomputable, provenance-carrying, presentation-free --- @@ -114,8 +140,8 @@ class InformationSpace: return recent_changes(self.union, self.log, self.space_id, limit=limit) def all_pages(self) -> tuple[AllPagesEntry, ...]: - """The union's distinct pages, chorus/equivalence-collapsed with divergence noted.""" - return all_pages(self.union) + """The union's distinct pages, collapsed via the maintained equivalence index.""" + return all_pages(self.union, equivalence_groups=self.index.equivalence_groups()) def site_map(self) -> SiteMapNode: """The union namespace tree built from page placements.""" diff --git a/src/shard_wiki/views/allpages.py b/src/shard_wiki/views/allpages.py index d10e1c6..e704b48 100644 --- a/src/shard_wiki/views/allpages.py +++ b/src/shard_wiki/views/allpages.py @@ -62,8 +62,16 @@ class _UnionFind: self._parent[max(ra, rb)] = min(ra, rb) -def all_pages(union: UnionGraph) -> tuple[AllPagesEntry, ...]: - """Enumerate the union's distinct pages, collapsing chorus + equivalence-bound members.""" +def all_pages( + union: UnionGraph, + equivalence_groups: tuple[frozenset[str], ...] | None = None, +) -> tuple[AllPagesEntry, ...]: + """Enumerate the union's distinct pages, collapsing chorus + equivalence-bound members. + + ``equivalence_groups`` (string identities, decision-log form) overrides the source of + equivalence — the orchestrator passes the maintained index's groups (SHARD-WP-0011 T4); the + default falls back to the decision-log fold, so direct callers are unaffected. + """ pages: dict[str, Page] = {} by_key: dict[str, list[str]] = {} for page in union.iter_pages(): @@ -77,8 +85,9 @@ def all_pages(union: UnionGraph) -> tuple[AllPagesEntry, ...]: for idents in by_key.values(): # same key across shards → chorus for other in idents[1:]: uf.union(idents[0], other) - fold = union.log.fold(union.space) - for group in fold.equivalence_groups: # decision-log bindings + if equivalence_groups is None: + equivalence_groups = union.log.fold(union.space).equivalence_groups + for group in equivalence_groups: # curator bindings (+ maintained content edges) present = [m for m in group if m in pages] for other in present[1:]: uf.union(present[0], other) diff --git a/tests/test_git_adapter.py b/tests/test_git_adapter.py new file mode 100644 index 0000000..a8e2e96 --- /dev/null +++ b/tests/test_git_adapter.py @@ -0,0 +1,131 @@ +"""Tests for the GitShardAdapter read path + profile (SHARD-WP-0012 T1).""" + +import subprocess + +import pytest + +from shard_wiki.adapters import GitShardAdapter, run_conformance +from shard_wiki.model import ( + AttachmentMode, + History, + NotSupported, + ProfileError, + Substrate, + Verb, +) + + +def _git(repo, *args): + subprocess.run( + ["git", "-C", str(repo), *args], + check=True, + capture_output=True, + env={"GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@t", + "GIT_COMMITTER_NAME": "t", "GIT_COMMITTER_EMAIL": "t@t", + "PATH": __import__("os").environ.get("PATH", "")}, + ) + + +def _repo(tmp_path, files, name="repo"): + repo = tmp_path / name + repo.mkdir() + _git(repo, "init", "--quiet") + for rel, text in files.items(): + p = repo / rel + p.parent.mkdir(parents=True, exist_ok=True) + p.write_text(text, encoding="utf-8") + _git(repo, "add", rel) + _git(repo, "commit", "-m", "seed") + return repo + + +def test_keys_are_tracked_md_paths(tmp_path): + repo = _repo(tmp_path, {"Home.md": "h", "docs/Guide.md": "g", "ignore.txt": "x"}) + adapter = GitShardAdapter("git", repo) + assert set(adapter.keys()) == {"Home", "docs/Guide"} # only tracked *.md + + +def test_read_returns_page_with_commit_sha_rev(tmp_path): + repo = _repo(tmp_path, {"Home.md": "welcome"}) + adapter = GitShardAdapter("git", repo) + page = adapter.read("Home") + assert page.identity.shard == "git" + assert page.body == "welcome" + head = subprocess.run( + ["git", "-C", str(repo), "rev-parse", "HEAD"], capture_output=True, text=True, check=True + ).stdout.strip() + assert page.envelope.source_rev == head # source_rev is the commit sha + assert page.envelope.lineage == "git-native" + + +def test_read_missing_key_raises(tmp_path): + adapter = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"})) + with pytest.raises(KeyError): + adapter.read("Nope") + + +def test_profile_validates_implication_rules(tmp_path): + profile = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"})).profile() + assert profile.substrate is Substrate.GIT + assert profile.attachment_mode is AttachmentMode.GIT_IS_STORE + assert profile.history is History.GIT_NATIVE # git-is-store ⟹ git-native + profile.validate() # raises if the implication rule were violated + + +def test_profile_is_read_only_in_t1(tmp_path): + profile = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"})).profile() + assert profile.supports(Verb.READ) + assert not profile.supports(Verb.WRITE) + + +def test_conformance_read_path_passes(tmp_path): + adapter = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h", "Other.md": "o"})) + report = run_conformance(adapter) + assert report.ok, report.diff() + + +def test_unclaimed_write_raises_not_supported(tmp_path): + adapter = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"})) + with pytest.raises(NotSupported): + adapter.write("Home", "new") # read-only: honest absence + + +def test_empty_repo_has_no_keys(tmp_path): + repo = tmp_path / "empty" + repo.mkdir() + _git(repo, "init", "--quiet") + adapter = GitShardAdapter("git", repo) + assert list(adapter.keys()) == [] + + +def test_bad_profile_combo_is_rejected(): + # Sanity: the implication rule that backs the git profile actually bites when violated. + from shard_wiki.model import ( + AccessGrant, + Addressing, + CapabilityProfile, + ContentOpacity, + MergeModel, + NativeQuery, + OperationalEnvelope, + Translation, + WriteGranularity, + ) + from shard_wiki.provenance import Liveness + + with pytest.raises(ProfileError): + CapabilityProfile( + substrate=Substrate.FILES, # not git, but claims git-is-store + attachment_mode=AttachmentMode.GIT_IS_STORE, + write_granularity=WriteGranularity.NONE, + content_opacity=ContentOpacity.TRANSPARENT, + operational_envelope=OperationalEnvelope.LOCAL_UNBOUNDED, + access_grant=AccessGrant.OPEN, + liveness=Liveness.STATIC, + history=History.NONE, + merge_model=MergeModel.NONE, + addressing=Addressing.PATH, + native_query=NativeQuery.NONE, + translation=Translation.NATIVE, + supported_verbs=frozenset({Verb.READ}), + ).validate() diff --git a/tests/test_git_adapter_integration.py b/tests/test_git_adapter_integration.py new file mode 100644 index 0000000..9f28a2e --- /dev/null +++ b/tests/test_git_adapter_integration.py @@ -0,0 +1,116 @@ +"""GitShardAdapter history adopt + cross-substrate integration (SHARD-WP-0012 T3).""" + +import os +import subprocess + +import pytest + +from shard_wiki.adapters import FolderAdapter, GitShardAdapter +from shard_wiki.coordination import ApplyStatus +from shard_wiki.space import InformationSpace + +_ENV = { + "GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@t", + "GIT_COMMITTER_NAME": "t", "GIT_COMMITTER_EMAIL": "t@t", + "PATH": os.environ.get("PATH", ""), +} + + +def _git(repo, *args): + return subprocess.run( + ["git", "-C", str(repo), *args], check=True, capture_output=True, text=True, env=_ENV + ).stdout.strip() + + +def _git_repo(tmp_path, files, name="git"): + repo = tmp_path / name + repo.mkdir() + _git(repo, "init", "--quiet") + for rel, text in files.items(): + (repo / rel).parent.mkdir(parents=True, exist_ok=True) + (repo / rel).write_text(text, encoding="utf-8") + _git(repo, "add", rel) + _git(repo, "commit", "-m", "seed") + return repo + + +def _folder(tmp_path, name, files, writable=False): + root = tmp_path / name + for rel, text in files.items(): + p = root / rel + p.parent.mkdir(parents=True, exist_ok=True) + p.write_text(text, encoding="utf-8") + return FolderAdapter(name, root, writable=writable) + + +# -- history adopt ------------------------------------------------------------- + + +def test_history_lists_commits_newest_first(tmp_path): + repo = _git_repo(tmp_path, {"Home.md": "v1"}) + adapter = GitShardAdapter("git", repo, writable=True) + adapter.write("Home", "v2") + history = adapter.history("Home") + assert len(history) == 2 + assert history[0].message == "write Home.md" # newest first + assert history[-1].message == "seed" + assert all(rev.sha for rev in history) + + +def test_history_unknown_key_raises(tmp_path): + adapter = GitShardAdapter("git", _git_repo(tmp_path, {"Home.md": "h"})) + with pytest.raises(KeyError): + adapter.history("Nope") + + +# -- cross-substrate integration ---------------------------------------------- + + +def test_resolve_across_git_and_folder(tmp_path): + space = InformationSpace("space") + space.attach(GitShardAdapter("git", _git_repo(tmp_path, {"Home.md": "git home"}))) + space.attach(_folder(tmp_path, "notes", {"Daily.md": "folder daily"})) + assert space.read("Home").body == "git home" # resolved from the git shard + assert space.read("Daily").body == "folder daily" # resolved from the folder shard + + +def test_chorus_spans_substrates_with_divergence(tmp_path): + space = InformationSpace("space") + space.attach(GitShardAdapter("git", _git_repo(tmp_path, {"Shared.md": "from git"}))) + space.attach(_folder(tmp_path, "notes", {"Shared.md": "from folder"})) + res = space.resolve("Shared") + assert {p.body for p in res.pages} == {"from git", "from folder"} # chorus across substrates + git_page = next(p for p in res.pages if p.identity.shard == "git") + assert git_page.envelope.divergence # divergence recorded, not erased + + +def test_edit_through_git_shard_commits(tmp_path): + repo = _git_repo(tmp_path, {"Home.md": "original"}) + space = InformationSpace("space") + space.attach(GitShardAdapter("git", repo, writable=True)) + result = space.edit("Home", "edited via overlay") + assert result.status is ApplyStatus.APPLIED # write-through fast-forward on a git shard + assert space.read("Home").body == "edited via overlay" + assert int(_git(repo, "rev-list", "--count", "HEAD")) == 2 # the edit became a commit + + +def test_apply_under_drift_refuses_on_external_commit(tmp_path): + repo = _git_repo(tmp_path, {"Home.md": "original"}) + space = InformationSpace("space") + space.attach(GitShardAdapter("git", repo, writable=True)) + overlay = space.overlay("Home", "my draft") # base_rev = current git sha + # Another writer commits to the same path → the sha moves underneath the draft. + (repo / "Home.md").write_text("someone else", encoding="utf-8") + _git(repo, "add", "Home.md") + _git(repo, "commit", "-m", "external") + result = space.apply_overlay(overlay.overlay_id) + assert result.status is ApplyStatus.REFUSED_DRIFT # never clobber (sha drift detected) + # The shard itself is untouched — the external commit stands; the draft remains a draft. + assert space.union.shard("git").read("Home").body == "someone else" + + +def test_overlay_on_read_only_git_shard_kept_as_draft(tmp_path): + space = InformationSpace("space") + space.attach(GitShardAdapter("git", _git_repo(tmp_path, {"Home.md": "ro"}), writable=False)) + result = space.edit("Home", "wanted change") + assert result.status is ApplyStatus.KEPT_DRAFT # read-only target → overlay retained diff --git a/tests/test_git_adapter_write.py b/tests/test_git_adapter_write.py new file mode 100644 index 0000000..7a3a76b --- /dev/null +++ b/tests/test_git_adapter_write.py @@ -0,0 +1,89 @@ +"""Tests for GitShardAdapter write=commit + current_rev drift (SHARD-WP-0012 T2).""" + +import os +import subprocess + +from shard_wiki.adapters import GitShardAdapter, run_conformance +from shard_wiki.model import Verb + +_ENV = { + "GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@t", + "GIT_COMMITTER_NAME": "t", "GIT_COMMITTER_EMAIL": "t@t", + "PATH": os.environ.get("PATH", ""), +} + + +def _git(repo, *args, capture=False): + return subprocess.run( + ["git", "-C", str(repo), *args], check=True, capture_output=True, text=True, env=_ENV + ).stdout.strip() + + +def _repo(tmp_path, files): + repo = tmp_path / "repo" + repo.mkdir() + _git(repo, "init", "--quiet") + for rel, text in files.items(): + (repo / rel).write_text(text, encoding="utf-8") + _git(repo, "add", rel) + _git(repo, "commit", "-m", "seed") + return repo + + +def test_writable_profile_declares_write_and_version(tmp_path): + profile = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"}), writable=True).profile() + assert profile.supports(Verb.WRITE) + assert profile.supports(Verb.VERSION) + profile.validate() # PER_PAGE + WRITE is a consistent combination + + +def test_write_creates_a_commit(tmp_path): + repo = _repo(tmp_path, {"Home.md": "old"}) + adapter = GitShardAdapter("git", repo, writable=True) + before = _git(repo, "rev-list", "--count", "HEAD") + page = adapter.write("Home", "new body") + after = _git(repo, "rev-list", "--count", "HEAD") + assert int(after) == int(before) + 1 # one new commit + assert page.body == "new body" + assert page.envelope.source_rev == _git(repo, "rev-parse", "HEAD") # page is at the new sha + + +def test_write_advances_current_rev(tmp_path): + repo = _repo(tmp_path, {"Home.md": "old"}) + adapter = GitShardAdapter("git", repo, writable=True) + rev_before = adapter.current_rev("Home") + adapter.write("Home", "changed") + assert adapter.current_rev("Home") != rev_before # sha moved → drift detectable + + +def test_write_new_key_tracks_it(tmp_path): + repo = _repo(tmp_path, {"Home.md": "h"}) + adapter = GitShardAdapter("git", repo, writable=True) + adapter.write("docs/New", "fresh page") + assert "docs/New" in set(adapter.keys()) + assert adapter.read("docs/New").body == "fresh page" + + +def test_noop_write_creates_no_empty_commit(tmp_path): + repo = _repo(tmp_path, {"Home.md": "same"}) + adapter = GitShardAdapter("git", repo, writable=True) + before = _git(repo, "rev-list", "--count", "HEAD") + adapter.write("Home", "same") # identical body → nothing to commit + assert _git(repo, "rev-list", "--count", "HEAD") == before + + +def test_current_rev_reflects_external_commit(tmp_path): + repo = _repo(tmp_path, {"Home.md": "h"}) + adapter = GitShardAdapter("git", repo, writable=True) + rev = adapter.current_rev("Home") + # An out-of-band commit to the same path (another writer) moves the per-path sha. + (repo / "Home.md").write_text("externally edited", encoding="utf-8") + _git(repo, "add", "Home.md") + _git(repo, "commit", "-m", "external") + assert adapter.current_rev("Home") != rev + + +def test_conformance_positive_write_probe_passes(tmp_path): + adapter = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "body"}), writable=True) + report = run_conformance(adapter) + assert report.ok, report.diff() diff --git a/tests/test_incremental_maintenance.py b/tests/test_incremental_maintenance.py new file mode 100644 index 0000000..6a38457 --- /dev/null +++ b/tests/test_incremental_maintenance.py @@ -0,0 +1,84 @@ +"""Incremental maintenance == rebuild, with retraction + propagation (SHARD-WP-0011 T2).""" + +from shard_wiki.incremental import EquivalenceIndex +from shard_wiki.model import Identity, Page +from shard_wiki.provenance import ProvenanceEnvelope + + +def _page(shard, key, body): + return Page( + identity=Identity(shard, key), + body=body, + envelope=ProvenanceEnvelope(source_shard=shard), + ) + + +def _rebuilt(pages, curator=()): + idx = EquivalenceIndex() + idx.build(pages, curator) + return idx + + +def _equal(a, b): + return a.edges() == b.edges() and set(a.groups()) == set(b.groups()) + + +def test_add_keeps_index_equal_to_rebuild(): + pages = [_page("A", "Foo", "same content here"), _page("B", "Bar", "same content here")] + idx = EquivalenceIndex() + for p in pages: + idx.add(p) + assert _equal(idx, _rebuilt(pages)) + assert idx.groups() # the two collapse + + +def test_remove_keeps_index_equal_to_rebuild(): + pages = [ + _page("A", "Foo", "same content here"), + _page("B", "Bar", "same content here"), + _page("C", "Baz", "unrelated isolated material entirely"), + ] + idx = _rebuilt(pages) + idx.remove(Identity("B", "Bar")) + assert _equal(idx, _rebuilt([pages[0], pages[2]])) + + +def test_edit_into_new_bucket_retracts_stale_edge(): + a = _page("A", "Foo", "shared identical body text") + b = _page("B", "Bar", "shared identical body text") + idx = _rebuilt([a, b]) + assert idx.groups() # A ≡ B initially + # Edit B to something completely different: it exits A's buckets, the edge is retracted. + b2 = _page("B", "Bar", "now totally divergent unrelated prose about nothing") + idx.update(b2) + assert idx.groups() == () # stale edge gone + assert _equal(idx, _rebuilt([a, b2])) + + +def test_edit_into_equivalence_adds_edge(): + a = _page("A", "Foo", "target body to converge on later") + b = _page("B", "Bar", "initially completely separate writing here") + idx = _rebuilt([a, b]) + assert idx.groups() == () + b2 = _page("B", "Bar", "target body to converge on later") # now identical to A + idx.update(b2) + assert idx.equivalent_to(Identity("A", "Foo")) == frozenset( + {Identity("A", "Foo"), Identity("B", "Bar")} + ) + assert _equal(idx, _rebuilt([a, b2])) + + +def test_removing_connector_splits_a_chorus(): + # Curator chain A—B—C (no direct A—C): one group of three. + a, b, c = (_page("A", "X", "aaa"), _page("B", "Y", "bbb"), _page("C", "Z", "ccc")) + idx = EquivalenceIndex() + for p in (a, b, c): + idx.add(p) + idx.bind(a.identity, b.identity) + idx.bind(b.identity, c.identity) + assert idx.equivalent_to(a.identity) == {a.identity, b.identity, c.identity} + # Removing the connector B retracts/propagates: the chorus splits. + idx.remove(b.identity) + assert idx.groups() == () + chain = [(a.identity, b.identity), (b.identity, c.identity)] + assert _equal(idx, _rebuilt([a, c], curator=chain)) diff --git a/tests/test_incremental_verification.py b/tests/test_incremental_verification.py new file mode 100644 index 0000000..dfbb590 --- /dev/null +++ b/tests/test_incremental_verification.py @@ -0,0 +1,89 @@ +"""Tests for I-2 verification — digest + consistency-checker (SHARD-WP-0011 T3).""" + +from shard_wiki.incremental import ( + ConsistencyChecker, + EquivalenceIndex, + derived_digest, +) +from shard_wiki.model import Identity, Page +from shard_wiki.provenance import ProvenanceEnvelope + + +def _page(shard, key, body): + return Page( + identity=Identity(shard, key), + body=body, + envelope=ProvenanceEnvelope(source_shard=shard), + ) + + +def test_digest_is_stable_under_equivalent_event_orders(): + pages = [ + _page("A", "Foo", "shared body text here"), + _page("B", "Bar", "shared body text here"), + _page("C", "Baz", "an entirely separate unrelated document"), + ] + forward = EquivalenceIndex() + for p in pages: + forward.add(p) + reverse = EquivalenceIndex() + for p in reversed(pages): + reverse.add(p) + assert derived_digest(forward) == derived_digest(reverse) + + +def test_clean_index_reports_healthy(): + pages = [_page("A", "Foo", "same body"), _page("B", "Bar", "same body")] + idx = EquivalenceIndex() + idx.build(pages) + checker = ConsistencyChecker(idx, pages_fn := (lambda: pages)) + report = checker.check_and_repair() + assert report.drifted is False and report.healthy is True + assert pages_fn() # source unchanged + + +def test_missed_delta_drift_is_detected_and_repaired(): + a = _page("A", "Foo", "converging target body") + b = _page("B", "Bar", "initially unrelated separate text") + source = {"pages": [a, b]} + idx = EquivalenceIndex() + idx.build(source["pages"]) + assert idx.groups() == () # not equivalent yet + + # Source changes B to match A, but the index is never told (a missed delta → drift). + b2 = _page("B", "Bar", "converging target body") + source["pages"] = [a, b2] + + checker = ConsistencyChecker(idx, lambda: source["pages"]) + report = checker.check_and_repair() + assert report.drifted is True and report.repaired is True and report.healthy is True + # Self-healed: the index now reflects the equivalence. + assert idx.equivalent_to(Identity("A", "Foo")) == frozenset( + {Identity("A", "Foo"), Identity("B", "Bar")} + ) + + +def test_corrupted_internal_state_is_healed(): + a = _page("A", "Foo", "identical content") + b = _page("B", "Bar", "identical content") + idx = EquivalenceIndex() + idx.build([a, b]) + # Corrupt the derived tier directly: delete a true edge (simulated index corruption). + idx._content_edges.clear() + assert idx.groups() == () # corrupted away + + checker = ConsistencyChecker(idx, lambda: [a, b]) + report = checker.check_and_repair() + assert report.drifted is True and report.healthy is True + assert idx.groups() # edge restored by scoped recompute + + +def test_removed_source_page_is_reconciled(): + a = _page("A", "Foo", "same body") + b = _page("B", "Bar", "same body") + idx = EquivalenceIndex() + idx.build([a, b]) + checker = ConsistencyChecker(idx, lambda: [a]) # B vanished from source + report = checker.check_and_repair() + assert report.healthy is True + assert Identity("B", "Bar") not in idx.identities() diff --git a/tests/test_incremental_wiring.py b/tests/test_incremental_wiring.py new file mode 100644 index 0000000..42a531b --- /dev/null +++ b/tests/test_incremental_wiring.py @@ -0,0 +1,74 @@ +"""Wire the incremental tier behind InformationSpace views (SHARD-WP-0011 T4).""" + +from shard_wiki.adapters import FolderAdapter +from shard_wiki.coordination import EventType +from shard_wiki.model import Identity +from shard_wiki.space import InformationSpace +from shard_wiki.views import all_pages + + +def _shard(tmp_path, name, files): + root = tmp_path / name + for rel, text in files.items(): + p = root / rel + p.parent.mkdir(parents=True, exist_ok=True) + p.write_text(text, encoding="utf-8") + return FolderAdapter(name, root) + + +def test_all_pages_via_index_matches_direct_fold(tmp_path): + space = InformationSpace("space") + space.attach(_shard(tmp_path, "wiki", {"Home.md": "welcome", "Guide.md": "the guide"})) + space.attach(_shard(tmp_path, "notes", {"Daily.md": "today"})) + # Routed-through-index result equals the direct fold-based computation (behaviour unchanged). + via_index = {(e.name, e.members) for e in space.all_pages()} + direct = {(e.name, e.members) for e in all_pages(space.union)} + assert via_index == direct + + +def test_curator_binding_collapses_via_maintained_index(tmp_path): + space = InformationSpace("space") + space.attach(_shard(tmp_path, "a", {"Foo.md": "x"})) + space.attach(_shard(tmp_path, "b", {"Bar.md": "y"})) + space.log.append( + "space", EventType.BINDING_MADE, {"members": ["a:Foo", "b:Bar"]} + ) + # The maintained index re-syncs curator edges live from the log fold. + collapsed = [e for e in space.all_pages() if len(e.members) == 2] + assert len(collapsed) == 1 + assert set(collapsed[0].members) == {Identity("a", "Foo"), Identity("b", "Bar")} + + +def test_content_duplicate_collapses_via_index(tmp_path): + space = InformationSpace("space") + space.attach(_shard(tmp_path, "a", {"Foo.md": "the very same body content here"})) + space.attach(_shard(tmp_path, "b", {"Bar.md": "the very same body content here"})) + dup = [e for e in space.all_pages() if len(e.members) == 2] + assert len(dup) == 1 # content equivalence detected by the maintained index + assert set(dup[0].members) == {Identity("a", "Foo"), Identity("b", "Bar")} + + +def test_attach_invalidates_index(tmp_path): + space = InformationSpace("space") + space.attach(_shard(tmp_path, "a", {"Foo.md": "same body"})) + assert space.all_pages() # builds the index (one page, no groups) + space.attach(_shard(tmp_path, "b", {"Bar.md": "same body"})) # marks index stale + dup = [e for e in space.all_pages() if len(e.members) == 2] + assert len(dup) == 1 # rebuilt fallback picks up the new equivalent page + + +def test_verify_index_reports_healthy_when_consistent(tmp_path): + space = InformationSpace("space") + space.attach(_shard(tmp_path, "a", {"Foo.md": "same body"})) + space.attach(_shard(tmp_path, "b", {"Bar.md": "same body"})) + space.all_pages() # ensure built + report = space.verify_index() + assert report.healthy is True + + +def test_reindex_is_an_explicit_fallback(tmp_path): + space = InformationSpace("space") + space.attach(_shard(tmp_path, "a", {"Foo.md": "content"})) + before = space.index.digest() + space.reindex() + assert space.index.digest() == before # rebuild is deterministic diff --git a/workplans/SHARD-WP-0001-yawex-requirements.md b/workplans/SHARD-WP-0001-yawex-requirements.md index 83c6b77..668313f 100644 --- a/workplans/SHARD-WP-0001-yawex-requirements.md +++ b/workplans/SHARD-WP-0001-yawex-requirements.md @@ -2,7 +2,7 @@ id: SHARD-WP-0001 type: workplan title: "shard-wiki requirements from yawex prior art" -domain: whynot +domain: consumer repo: shard-wiki status: done owner: tegwick diff --git a/workplans/SHARD-WP-0002-federation-architecture.md b/workplans/SHARD-WP-0002-federation-architecture.md index 83306b3..4241e5e 100644 --- a/workplans/SHARD-WP-0002-federation-architecture.md +++ b/workplans/SHARD-WP-0002-federation-architecture.md @@ -2,7 +2,7 @@ id: SHARD-WP-0002 type: workplan title: "federation architecture design" -domain: whynot +domain: consumer repo: shard-wiki status: done owner: tegwick diff --git a/workplans/SHARD-WP-0003-engine-dives-batch.md b/workplans/SHARD-WP-0003-engine-dives-batch.md index de6498c..68769e1 100644 --- a/workplans/SHARD-WP-0003-engine-dives-batch.md +++ b/workplans/SHARD-WP-0003-engine-dives-batch.md @@ -2,7 +2,7 @@ id: SHARD-WP-0003 type: workplan title: "wiki-engine deep-dive batch (new-insight + git-forge + classic engines)" -domain: whynot +domain: consumer repo: shard-wiki status: done owner: tegwick diff --git a/workplans/SHARD-WP-0004-computational-knowledge-systems.md b/workplans/SHARD-WP-0004-computational-knowledge-systems.md index e983c1c..971aee7 100644 --- a/workplans/SHARD-WP-0004-computational-knowledge-systems.md +++ b/workplans/SHARD-WP-0004-computational-knowledge-systems.md @@ -2,7 +2,7 @@ id: SHARD-WP-0004 type: workplan title: "computational / interactive-knowledge systems research" -domain: whynot +domain: consumer repo: shard-wiki status: done owner: tegwick diff --git a/workplans/SHARD-WP-0005-architecture-hardening.md b/workplans/SHARD-WP-0005-architecture-hardening.md index 62a0a11..4521ab9 100644 --- a/workplans/SHARD-WP-0005-architecture-hardening.md +++ b/workplans/SHARD-WP-0005-architecture-hardening.md @@ -2,7 +2,7 @@ id: SHARD-WP-0005 type: workplan title: "core architecture hardening (blueprint review fixes)" -domain: whynot +domain: consumer repo: shard-wiki status: done owner: tegwick diff --git a/workplans/SHARD-WP-0006-architecture-hardening-2.md b/workplans/SHARD-WP-0006-architecture-hardening-2.md index 45cf50a..2724804 100644 --- a/workplans/SHARD-WP-0006-architecture-hardening-2.md +++ b/workplans/SHARD-WP-0006-architecture-hardening-2.md @@ -2,7 +2,7 @@ id: SHARD-WP-0006 type: workplan title: "core architecture hardening II (round-2 review fixes)" -domain: whynot +domain: consumer repo: shard-wiki status: done owner: tegwick diff --git a/workplans/SHARD-WP-0007-foundation-implementation.md b/workplans/SHARD-WP-0007-foundation-implementation.md index e8cd0fa..b9f7db4 100644 --- a/workplans/SHARD-WP-0007-foundation-implementation.md +++ b/workplans/SHARD-WP-0007-foundation-implementation.md @@ -2,7 +2,7 @@ id: SHARD-WP-0007 type: workplan title: "foundation implementation — model, contract, decision log, union read" -domain: whynot +domain: consumer repo: shard-wiki status: done owner: tegwick diff --git a/workplans/SHARD-WP-0008-write-path.md b/workplans/SHARD-WP-0008-write-path.md index 87431d5..96825f7 100644 --- a/workplans/SHARD-WP-0008-write-path.md +++ b/workplans/SHARD-WP-0008-write-path.md @@ -2,7 +2,7 @@ id: SHARD-WP-0008 type: workplan title: "write path — overlay engine, writable adapter, apply-under-drift" -domain: whynot +domain: consumer repo: shard-wiki status: done owner: tegwick diff --git a/workplans/SHARD-WP-0009-git-decision-log.md b/workplans/SHARD-WP-0009-git-decision-log.md index b1424c1..92f5bad 100644 --- a/workplans/SHARD-WP-0009-git-decision-log.md +++ b/workplans/SHARD-WP-0009-git-decision-log.md @@ -2,7 +2,7 @@ id: SHARD-WP-0009 type: workplan title: "git-backed DecisionLog + per-space append authority" -domain: whynot +domain: consumer repo: shard-wiki status: done owner: tegwick diff --git a/workplans/SHARD-WP-0010-derived-views.md b/workplans/SHARD-WP-0010-derived-views.md index d2f6f20..c768a07 100644 --- a/workplans/SHARD-WP-0010-derived-views.md +++ b/workplans/SHARD-WP-0010-derived-views.md @@ -2,7 +2,7 @@ id: SHARD-WP-0010 type: workplan title: "derived views — wikilinks, BackLinks, RecentChanges, AllPages/SiteMap" -domain: whynot +domain: consumer repo: shard-wiki status: done owner: tegwick diff --git a/workplans/SHARD-WP-0011-incremental-union.md b/workplans/SHARD-WP-0011-incremental-union.md index ba5bf45..da2e121 100644 --- a/workplans/SHARD-WP-0011-incremental-union.md +++ b/workplans/SHARD-WP-0011-incremental-union.md @@ -2,9 +2,9 @@ id: SHARD-WP-0011 type: workplan title: "incremental union maintenance + equivalence index + I-2 verification" -domain: whynot +domain: consumer repo: shard-wiki -status: active +status: done owner: tegwick topic_slug: whynot created: "2026-06-15" @@ -41,7 +41,7 @@ deployment is later. ```task id: SHARD-WP-0011-T1 -status: todo +status: done priority: high state_hub_task_id: "842f480b-7b14-47cd-818b-012dbda9c187" ``` @@ -55,7 +55,7 @@ unrelated pages don't; verified edges match a brute-force oracle on a small corp ```task id: SHARD-WP-0011-T2 -status: todo +status: done priority: high state_hub_task_id: "2da4e0b8-22cc-4ad1-a9aa-b5e991515d30" ``` @@ -70,7 +70,7 @@ stale edge. ```task id: SHARD-WP-0011-T3 -status: todo +status: done priority: high state_hub_task_id: "b602ce31-ad9a-4c7f-b596-f039722373fc" ``` @@ -85,7 +85,7 @@ equivalent event orders. ```task id: SHARD-WP-0011-T4 -status: todo +status: done priority: medium state_hub_task_id: "2f3d083c-0b2e-4b58-9e96-c0461c5eb089" ``` diff --git a/workplans/SHARD-WP-0012-git-shard-adapter.md b/workplans/SHARD-WP-0012-git-shard-adapter.md index 9049eb8..ecf6e50 100644 --- a/workplans/SHARD-WP-0012-git-shard-adapter.md +++ b/workplans/SHARD-WP-0012-git-shard-adapter.md @@ -2,9 +2,9 @@ id: SHARD-WP-0012 type: workplan title: "second adapter — git-IS-store shard (contract validation on a new substrate)" -domain: whynot +domain: consumer repo: shard-wiki -status: active +status: done owner: tegwick topic_slug: whynot created: "2026-06-15" @@ -40,7 +40,7 @@ merge beyond fast-forward (apply-under-drift refuse is enough, as in SHARD-WP-00 ```task id: SHARD-WP-0012-T1 -status: todo +status: done priority: high state_hub_task_id: "8a1c7c80-a0cc-4e02-a611-1f1fd7dec57b" ``` @@ -54,7 +54,7 @@ implication rules. Tests: read tracked files; profile validates; conformance rea ```task id: SHARD-WP-0012-T2 -status: todo +status: done priority: high state_hub_task_id: "b47dfb86-46c1-4e97-a62f-377719499ff2" ``` @@ -68,7 +68,7 @@ changes after an external commit. ```task id: SHARD-WP-0012-T3 -status: todo +status: done priority: medium state_hub_task_id: "4c895f42-671d-4948-8bdf-941fd85644bb" ``` diff --git a/workplans/SHARD-WP-0013-wiki-engine-prep.md b/workplans/SHARD-WP-0013-wiki-engine-prep.md index e63fe4b..9716b56 100644 --- a/workplans/SHARD-WP-0013-wiki-engine-prep.md +++ b/workplans/SHARD-WP-0013-wiki-engine-prep.md @@ -2,7 +2,7 @@ id: SHARD-WP-0013 type: workplan title: "wiki-engine prep — reuse-surface registration, UC-catalog systematization, WikiEngineCoreArchitecture" -domain: whynot +domain: consumer repo: shard-wiki status: done owner: tegwick diff --git a/workplans/SHARD-WP-0014-engine-implementation.md b/workplans/SHARD-WP-0014-engine-implementation.md index d354f96..19aa0c4 100644 --- a/workplans/SHARD-WP-0014-engine-implementation.md +++ b/workplans/SHARD-WP-0014-engine-implementation.md @@ -2,7 +2,7 @@ id: SHARD-WP-0014 type: workplan title: "wiki-engine implementation — kernel + typed-extension runtime + activation" -domain: whynot +domain: consumer repo: shard-wiki status: done owner: tegwick