tegwick
072a7fa550
The snapshot and manifest schemas, both example dataspaces, and the shared text renderer now pass the fixture validator and the negative contract tests. JSON, NDJSON, and Markdown frontmatter publish a snapshot only when rights, identity, and references allow it. CONTENT-WP-0001 and CONTENT-WP-0003 are marked finished against those artifacts. CONTENT-WP-0000-T04 stays open. SCOPE.md and the 19:05 scope-against-intent note record the stop. Assistant: grok Assistant-Session: 01a130f1-abfb-7d71-9cd6-a8b75ec7f95a
3.7 KiB
| id | type | title | domain | repo | status | flavor | depends_on | created | updated | state_hub_workstream_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| CONTENT-WP-0003 | workplan | Ingestion, provenance, and publishing | infotech | content-explorer | finished | implementation |
|
2026-10-12 | 2026-10-12 | fd95d654-8696-5994-9643-bb45925963a6 |
Ingestion, provenance, and publishing
Turn raw inputs into trustworthy, reproducible snapshots. This gate is required before any externally ingested public content is published. It may proceed in parallel with CONTENT-WP-0002.
The v0.1 connector in content_explorer/ingest.py loads JSON, NDJSON, and Markdown frontmatter, resolves links in a second pass, and publishes a snapshot, a manifest, and two import reports only when rights, identity, and references allow it.
Work packages
Implement build-time JSON, NDJSON, and Markdown frontmatter connector APIs.
id: CONTENT-WP-0003-T01
status: done
priority: high
state_hub_task_id: "f9f8c59e-4bff-5dd4-82bb-c9f8985fd924"
Make import idempotent with source digests and a connector version.
id: CONTENT-WP-0003-T02
status: done
priority: high
state_hub_task_id: "406ae997-7b2c-5a8d-8009-a6ab53e37f4d"
Resolve links in a second pass, report dangling references and conflicts, and distinguish tombstones.
id: CONTENT-WP-0003-T03
status: done
priority: high
state_hub_task_id: "074caaf6-0501-5e02-beb3-f2c71e492f7a"
Model source licenses, citations, and attribution, and quarantine unsupported content.
id: CONTENT-WP-0003-T04
status: done
priority: high
state_hub_task_id: "0ad91201-4a1c-58b5-851f-b3e19a7eabee"
Generate the snapshot, manifest, and machine-readable and human import reports, and publish atomically.
id: CONTENT-WP-0003-T05
status: done
priority: high
state_hub_task_id: "1ec567b1-2efd-558a-b790-e101b9087832"
Acceptance gates
Two connector types produce identical normalized snapshots for equivalent input.
id: CONTENT-WP-0003-T06
status: done
priority: high
state_hub_task_id: "76c69f07-3b7b-55f7-8556-3be717259348"
Repeated ingest yields the same content hash.
id: CONTENT-WP-0003-T07
status: done
priority: high
state_hub_task_id: "6fe89d04-4018-5491-85b6-76e26cb0ba4d"
Every published sourced claim has a retrievable evidence reference.
id: CONTENT-WP-0003-T08
status: done
priority: high
state_hub_task_id: "d3988379-ee41-53ee-bb72-4f47c9fa37ef"
Rights violations prevent publication.
id: CONTENT-WP-0003-T09
status: done
priority: high
state_hub_task_id: "24ad02ea-75f5-5238-979c-c283072d85cb"
Key risk
R5: rights metadata missing. Quarantine instead of guessing. Log evidence and deviations in the workplan review before marking complete.
Evidence
Evidence: content_explorer/ingest.py and tests/test_ingest.py. python3 -m unittest tests.test_ingest passes. JSON, NDJSON, and Markdown frontmatter projections share one content hash (CONTENT-WP-0003-T01, T06, T07). The report carries connectorVersion 0.1.0 and the source digest (T02). Dangling targets, duplicate keys, and a key that is both live and tombstoned block publication (T03). Allowed licenses are CC0-1.0, CC-BY-4.0, and MIT; CC-BY-NC-4.0, a missing license, and CC-BY-4.0 without attribution quarantine and do not publish (T04, T09). publish_directory writes snapshot.json, manifest.json, import-report.json, and import-report.md by renaming a completed directory into place (T05). Every published tldr claim has evidenceRefs (T08). Notes: workplans/evidence/CONTENT-WP-0003-ImplementationNotes.md and workplans/evidence/CONTENT-WP-0003-Review.md. The connector does not fetch remote files and does not render.