coulomb/content-explorer
5
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Freeze the v0.1 contract and the local ingest connector.

Browse source 
The snapshot and manifest schemas, both example dataspaces, and the shared text renderer now pass the fixture validator and the negative contract tests. JSON, NDJSON, and Markdown frontmatter publish a snapshot only when rights, identity, and references allow it. CONTENT-WP-0001 and CONTENT-WP-0003 are marked finished against those artifacts. CONTENT-WP-0000-T04 stays open. SCOPE.md and the 19:05 scope-against-intent note record the stop.

Assistant: grok
Assistant-Session: 01a130f1-abfb-7d71-9cd6-a8b75ec7f95a
This commit is contained in:
tegwick 2026-10-12 19:07:32 +02:00
parent 2b1706559b
commit 072a7fa550
30 changed files with 1896 additions and 264 deletions
Download patch file Download diff file Expand all files Collapse all files

22
workplans/CONTENT-WP-0001-canonical-contracts.md
View file

@ -4,7 +4,7 @@ type: workplan
title: "Canonical contracts and two-domain fixtures"
domain: infotech
repo: content-explorer
status: ready
status: finished
flavor: implementation
depends_on: []
created: "2026-10-12"
@ -14,7 +14,7 @@ state_hub_workstream_id: "833f4e24-1529-528f-8cc7-6caeeabc555f"
# Canonical contracts and two-domain fixtures
Establish a source-independent knowledge projection and the zero-UI-code-change experiment. The seed already contains a draft snapshot schema, two authored dataspaces, and a structural smoke validator. Those artifacts do not yet meet the gates below.
Establish a source-independent knowledge projection and the zero-UI-code-change experiment. The v0.1 schema, both fixtures, the negative checks, and the shared text renderer meet the gates below.
Coordinate with CONTENT-WP-0000. Neither plan blocks the other.
@ -24,7 +24,7 @@ Coordinate with CONTENT-WP-0000. Neither plan blocks the other.
```task
id: CONTENT-WP-0001-T01
status: todo
status: done
priority: high
state_hub_task_id: "c3de4ad2-7fae-51bc-9285-85d5cb2e5908"
```
@ -33,7 +33,7 @@ state_hub_task_id: "c3de4ad2-7fae-51bc-9285-85d5cb2e5908"
```task
id: CONTENT-WP-0001-T02
status: todo
status: done
priority: high
state_hub_task_id: "501807a7-f778-5799-88d6-e15bbdc42f3d"
```
@ -42,7 +42,7 @@ state_hub_task_id: "501807a7-f778-5799-88d6-e15bbdc42f3d"
```task
id: CONTENT-WP-0001-T03
status: todo
status: done
priority: high
state_hub_task_id: "f0e8a472-044e-5dfb-930f-d1afd0bcdd8d"
```
@ -51,7 +51,7 @@ state_hub_task_id: "f0e8a472-044e-5dfb-930f-d1afd0bcdd8d"
```task
id: CONTENT-WP-0001-T04
status: todo
status: done
priority: high
state_hub_task_id: "ec494de4-e634-5e8d-ab9e-cbb87f7d0ac1"
```
@ -60,7 +60,7 @@ state_hub_task_id: "ec494de4-e634-5e8d-ab9e-cbb87f7d0ac1"
```task
id: CONTENT-WP-0001-T05
status: todo
status: done
priority: high
state_hub_task_id: "8fc0c024-f179-55e0-a70d-173c94908cdd"
```
@ -71,7 +71,7 @@ state_hub_task_id: "8fc0c024-f179-55e0-a70d-173c94908cdd"
```task
id: CONTENT-WP-0001-T06
status: todo
status: done
priority: high
state_hub_task_id: "b0fbdf3b-f0cb-5a6e-bb29-593ee6c822c3"
```
@ -80,7 +80,7 @@ state_hub_task_id: "b0fbdf3b-f0cb-5a6e-bb29-593ee6c822c3"
```task
id: CONTENT-WP-0001-T07
status: todo
status: done
priority: high
state_hub_task_id: "7c2fd15b-60fa-5e7c-8b2b-8d3d6a22335e"
```
@ -89,7 +89,7 @@ state_hub_task_id: "7c2fd15b-60fa-5e7c-8b2b-8d3d6a22335e"
```task
id: CONTENT-WP-0001-T08
status: todo
status: done
priority: high
state_hub_task_id: "1847aa13-3369-577d-925d-3b08dcb5cb1a"
```
@ -100,4 +100,4 @@ R3: abstractions erase domain semantics. Log evidence and deviations in the work
## Evidence
Pull requests, test logs, pinned artifacts, screenshots, or a comparison report where applicable. Add `ImplementationNotes.md` and `Review.md` on completion. Do not mark a gate passed without a verifiable link.
Evidence: `contracts/snapshot.schema.json`, `contracts/manifest.schema.json`, `contracts/SNAPSHOT.md`, `contracts/MIGRATIONS.md`, `contracts/fact-policy.md`, `content_explorer/contract.py`, `content_explorer/render.py`, and `tests/test_contract.py`. `python3 -m unittest tests.test_contract` and `python3 scripts/validate_fixtures.py` both pass. The shared renderer is text, not a browser UI. It reads headings and labels only from the manifest and snapshot.

24
workplans/CONTENT-WP-0003-ingestion-publishing.md
View file

@ -4,7 +4,7 @@ type: workplan
title: "Ingestion, provenance, and publishing"
domain: infotech
repo: content-explorer
status: proposed
status: finished
flavor: implementation
depends_on:
- CONTENT-WP-0001
@ -17,13 +17,15 @@ state_hub_workstream_id: "fd95d654-8696-5994-9643-bb45925963a6"
Turn raw inputs into trustworthy, reproducible snapshots. This gate is required before any externally ingested public content is published. It may proceed in parallel with CONTENT-WP-0002.
The v0.1 connector in `content_explorer/ingest.py` loads JSON, NDJSON, and Markdown frontmatter, resolves links in a second pass, and publishes a snapshot, a manifest, and two import reports only when rights, identity, and references allow it.
## Work packages
### Implement build-time JSON, NDJSON, and Markdown frontmatter connector APIs.
```task
id: CONTENT-WP-0003-T01
status: todo
status: done
priority: high
state_hub_task_id: "f9f8c59e-4bff-5dd4-82bb-c9f8985fd924"
```
@ -32,7 +34,7 @@ state_hub_task_id: "f9f8c59e-4bff-5dd4-82bb-c9f8985fd924"
```task
id: CONTENT-WP-0003-T02
status: todo
status: done
priority: high
state_hub_task_id: "406ae997-7b2c-5a8d-8009-a6ab53e37f4d"
```
@ -41,7 +43,7 @@ state_hub_task_id: "406ae997-7b2c-5a8d-8009-a6ab53e37f4d"
```task
id: CONTENT-WP-0003-T03
status: todo
status: done
priority: high
state_hub_task_id: "074caaf6-0501-5e02-beb3-f2c71e492f7a"
```
@ -50,7 +52,7 @@ state_hub_task_id: "074caaf6-0501-5e02-beb3-f2c71e492f7a"
```task
id: CONTENT-WP-0003-T04
status: todo
status: done
priority: high
state_hub_task_id: "0ad91201-4a1c-58b5-851f-b3e19a7eabee"
```
@ -59,7 +61,7 @@ state_hub_task_id: "0ad91201-4a1c-58b5-851f-b3e19a7eabee"
```task
id: CONTENT-WP-0003-T05
status: todo
status: done
priority: high
state_hub_task_id: "1ec567b1-2efd-558a-b790-e101b9087832"
```
@ -70,7 +72,7 @@ state_hub_task_id: "1ec567b1-2efd-558a-b790-e101b9087832"
```task
id: CONTENT-WP-0003-T06
status: todo
status: done
priority: high
state_hub_task_id: "76c69f07-3b7b-55f7-8556-3be717259348"
```
@ -79,7 +81,7 @@ state_hub_task_id: "76c69f07-3b7b-55f7-8556-3be717259348"
```task
id: CONTENT-WP-0003-T07
status: todo
status: done
priority: high
state_hub_task_id: "6fe89d04-4018-5491-85b6-76e26cb0ba4d"
```
@ -88,7 +90,7 @@ state_hub_task_id: "6fe89d04-4018-5491-85b6-76e26cb0ba4d"
```task
id: CONTENT-WP-0003-T08
status: todo
status: done
priority: high
state_hub_task_id: "d3988379-ee41-53ee-bb72-4f47c9fa37ef"
```
@ -97,7 +99,7 @@ state_hub_task_id: "d3988379-ee41-53ee-bb72-4f47c9fa37ef"
```task
id: CONTENT-WP-0003-T09
status: todo
status: done
priority: high
state_hub_task_id: "24ad02ea-75f5-5238-979c-c283072d85cb"
```
@ -108,4 +110,4 @@ R5: rights metadata missing. Quarantine instead of guessing. Log evidence and de
## Evidence
Pull requests, test logs, pinned artifacts, screenshots, or a comparison report where applicable. Add `ImplementationNotes.md` and `Review.md` on completion. Do not mark a gate passed without a verifiable link.
Evidence: `content_explorer/ingest.py` and `tests/test_ingest.py`. `python3 -m unittest tests.test_ingest` passes. JSON, NDJSON, and Markdown frontmatter projections share one content hash (`CONTENT-WP-0003-T01`, `T06`, `T07`). The report carries `connectorVersion` `0.1.0` and the source digest (`T02`). Dangling targets, duplicate keys, and a key that is both live and tombstoned block publication (`T03`). Allowed licenses are `CC0-1.0`, `CC-BY-4.0`, and `MIT`; `CC-BY-NC-4.0`, a missing license, and `CC-BY-4.0` without attribution quarantine and do not publish (`T04`, `T09`). `publish_directory` writes `snapshot.json`, `manifest.json`, `import-report.json`, and `import-report.md` by renaming a completed directory into place (`T05`). Every published tldr claim has `evidenceRefs` (`T08`). Notes: `workplans/evidence/CONTENT-WP-0003-ImplementationNotes.md` and `workplans/evidence/CONTENT-WP-0003-Review.md`. The connector does not fetch remote files and does not render.

13
workplans/evidence/CONTENT-WP-0003-ImplementationNotes.md Normal file
View file

@ -0,0 +1,13 @@
# CONTENT-WP-0003 implementation notes
Date: 2026-10-12
The build-time connector is `content_explorer/ingest.py`, version `0.1.0`. It reads a JSON list or `{records}` object, NDJSON, or a directory of Markdown files with YAML frontmatter. PyYAML parses the frontmatter. The three loaders return the same record shape plus a SHA-256 digest of the bytes they read.
Projection collects live keys first, then resolves relations against that set. A target that is not an accepted key is dangling. Two live records with the same kind and key, or a key that is both tombstoned and live, are identity conflicts. Any dangling reference, identity conflict, unknown kind, or missing relation label leaves `snapshot` null and `published` false.
Publication allows only `CC0-1.0`, `CC-BY-4.0`, and `MIT`. `CC-BY-4.0` also requires a non-empty attribution. Other licenses, including `CC-BY-NC-4.0` and a missing license, are quarantined. The connector does not guess a license.
Each live entity gets one asserted `tldr` claim whose `evidenceRefs` is the source id. `publish_directory` writes `snapshot.json`, `manifest.json`, `import-report.json`, and `import-report.md` into a temporary directory and renames it into place. A quarantined result raises `RuntimeError` and does not replace an existing directory.
The manifest is generated from the caller-supplied kind list and relation labels. Predicate wording is not invented. `connectors/json_fixture.py` still only reads a snapshot that already exists. Remote fetch, pagination, and a third connector are not part of this module.

16
workplans/evidence/CONTENT-WP-0003-Review.md Normal file
View file

@ -0,0 +1,16 @@
# CONTENT-WP-0003 review
Date: 2026-10-12
Risk R5 was rights metadata missing. The connector quarantines a missing license, a license outside `CC0-1.0`, `CC-BY-4.0`, and `MIT`, and `CC-BY-4.0` without attribution. It does not invent attribution or a rights statement.
Checked against `tests/test_ingest.py`:
- JSON, NDJSON, and Markdown projections of the same records share one content hash, including a second JSON run.
- The report records connector version `0.1.0` and the source digest.
- A dangling target, a duplicate key, and a missing relation label do not publish.
- A tombstoned key is listed and omitted from entities.
- `publish_directory` writes the snapshot, manifest, machine report, and human report. The written snapshot and manifest pass `validate_documents`.
- A rights failure raises before replacing the destination.
Not claimed: remote URLs, conflict merging into a contested claim, or a browser view of the published directory. Those stay with later workplans. CONTENT-WP-0007 was not used as a source.