Freeze the v0.1 contract and the local ingest connector.
The snapshot and manifest schemas, both example dataspaces, and the shared text renderer now pass the fixture validator and the negative contract tests. JSON, NDJSON, and Markdown frontmatter publish a snapshot only when rights, identity, and references allow it. CONTENT-WP-0001 and CONTENT-WP-0003 are marked finished against those artifacts. CONTENT-WP-0000-T04 stays open. SCOPE.md and the 19:05 scope-against-intent note record the stop. Assistant: grok Assistant-Session: 01a130f1-abfb-7d71-9cd6-a8b75ec7f95a
This commit is contained in:
parent
2b1706559b
commit
072a7fa550
30 changed files with 1896 additions and 264 deletions
|
|
@ -12,10 +12,10 @@ Infotech already has several topics. This repo is registered by slug and classif
|
|||
|
||||
| ID | Status | File |
|
||||
|---|---|---|
|
||||
| CONTENT-WP-0000 | active | `workplans/CONTENT-WP-0000-upstream-baseline.md` |
|
||||
| CONTENT-WP-0001 | ready | `workplans/CONTENT-WP-0001-canonical-contracts.md` |
|
||||
| CONTENT-WP-0000 | active (T04 still open) | `workplans/CONTENT-WP-0000-upstream-baseline.md` |
|
||||
| CONTENT-WP-0001 | finished | `workplans/CONTENT-WP-0001-canonical-contracts.md` |
|
||||
| CONTENT-WP-0002 | proposed | `workplans/CONTENT-WP-0002-ui-extraction.md` |
|
||||
| CONTENT-WP-0003 | proposed | `workplans/CONTENT-WP-0003-ingestion-publishing.md` |
|
||||
| CONTENT-WP-0003 | finished | `workplans/CONTENT-WP-0003-ingestion-publishing.md` |
|
||||
| CONTENT-WP-0004 | proposed | `workplans/CONTENT-WP-0004-relations-journeys.md` |
|
||||
| CONTENT-WP-0005 | proposed | `workplans/CONTENT-WP-0005-quality-parity.md` |
|
||||
| CONTENT-WP-0006 | proposed | `workplans/CONTENT-WP-0006-external-pilot.md` |
|
||||
|
|
|
|||
2
.gitignore
vendored
2
.gitignore
vendored
|
|
@ -2,6 +2,8 @@ node_modules/
|
|||
.next/
|
||||
out/
|
||||
upstream-onco/
|
||||
__pycache__/
|
||||
*.pyc
|
||||
*.log
|
||||
.env*
|
||||
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ This checkout is a research seed, not a running application. There is no install
|
|||
|
||||
```sh
|
||||
python3 scripts/validate_fixtures.py
|
||||
python3 -m unittest tests.test_contract tests.test_ingest
|
||||
```
|
||||
|
||||
## State Hub
|
||||
|
|
|
|||
|
|
@ -18,6 +18,7 @@ Research grounded in publicly browsable upstream files, extraction plan, normati
|
|||
|
||||
```sh
|
||||
python3 scripts/validate_fixtures.py
|
||||
python3 -m unittest tests.test_contract tests.test_ingest
|
||||
```
|
||||
|
||||
## Implementation start
|
||||
|
|
|
|||
42
SCOPE.md
Normal file
42
SCOPE.md
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
# SCOPE — content-explorer
|
||||
|
||||
**As of:** 2026-10-12 19:05 CEST
|
||||
**Stop:** Implementation stopped at 19:05 CEST, before 24:00. No later workplan had its prerequisites met. CONTENT-WP-0007 stays blocked.
|
||||
|
||||
This file describes the tree at that stop. It is the scope a reader can compare with `INTENT.md`. The comparison itself is `history/2026-10-12-1905-scope-against-intent.md`.
|
||||
|
||||
## In scope now
|
||||
|
||||
- A research seed for a domain-independent knowledge explorer. OnCo (onco.cc) is the experience reference. MIT-licensed upstream code may be adapted from the pinned checkout. The checkout stays gitignored at `upstream-onco/`.
|
||||
- Pinned upstream baseline and reuse decision for commit `b594d7433bf4e408b44215e25e5304696d6f840e`, recorded under `research/baseline/`.
|
||||
- Frozen snapshot and manifest contracts at `schemaVersion` 0.1, fact policy, and migration note under `contracts/`.
|
||||
- Two independently authored example dataspaces, `examples/dataspaces/systems-thinking` and `examples/dataspaces/software-architecture`, each with at least three kinds and eight entities.
|
||||
- Executable checks: `python3 scripts/validate_fixtures.py`, `python3 -m unittest tests.test_contract`, and `python3 -m unittest tests.test_ingest`.
|
||||
- A shared text renderer, `content_explorer/render.py`. It takes headings and labels from the snapshot and manifest. It is not a browser UI.
|
||||
- A build-time connector, `content_explorer/ingest.py` version `0.1.0`, for local JSON, NDJSON, and Markdown frontmatter. It publishes a snapshot, a manifest, and machine-readable and human import reports only when rights, identity, and references allow it.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- The OnCo corpus, `src/data`, logos, portraits, trademarks, and CC BY-NC data. None of those are vendored in this repo.
|
||||
- A production `next build` of upstream OnCo, pixel-perfect parity, MCP, and a third connector (GitHub issues or Wikidata).
|
||||
- A browser explorer: landing, faceted index, detail page, search, graph view, and a mouse or keyboard journey. The workstation had no Chrome, Chromium, Firefox, or Playwright launcher, so those screens were not captured.
|
||||
- Remote fetch, pagination, and authenticated APIs.
|
||||
- New State Hub topics, or workplans posted by hand to the hub API. Local workplan files remain the source of truth.
|
||||
- Unblocking CONTENT-WP-0007. Its first task still waits on a human data-use authorization.
|
||||
|
||||
## Workplan status
|
||||
|
||||
| ID | Status in the workplan file | What the tree shows |
|
||||
|---|---|---|
|
||||
| CONTENT-WP-0000 | active, partial | T01–T03 and T05–T09 are done and cite the baseline report, pin, and license notes. T04 is still `todo`: no headless browser, so there is no screenshot or keyboard log. |
|
||||
| CONTENT-WP-0001 | finished | Schema, both fixtures, negative graph checks, fact policy, and the text renderer. Tests import `content_explorer/contract.py` and `content_explorer/render.py`. |
|
||||
| CONTENT-WP-0002 | proposed, not started | Depends on 0000 and 0001. 0000 is not finished while T04 is open, so this plan was not started. |
|
||||
| CONTENT-WP-0003 | finished | JSON, NDJSON, and Markdown connectors, source digest, second-pass links, tombstones, identity conflicts, rights quarantine, and atomic publish of snapshot, manifest, and both reports. Evidence is `content_explorer/ingest.py`, `tests/test_ingest.py`, and `workplans/evidence/CONTENT-WP-0003-*.md`. |
|
||||
| CONTENT-WP-0004 | proposed, not started | Depends on 0002 and 0003. 0002 has not started. |
|
||||
| CONTENT-WP-0005 | proposed, not started | Depends on 0004. |
|
||||
| CONTENT-WP-0006 | proposed, not started | Depends on 0005. A third connector was a non-goal for this evening. |
|
||||
| CONTENT-WP-0007 | blocked | Depends on 0000, 0001, 0002, 0003, and 0005. T01 remains `wait` with `needs_human: true`. No OnCo reference slice was imported. |
|
||||
|
||||
## Checks at the stop
|
||||
|
||||
`python3 -m unittest tests.test_ingest tests.test_contract` and `python3 scripts/validate_fixtures.py` exited 0 at 19:01 CEST and again after the ingest manifest and conflict work. The saved log for the closing run is outside the repo, in the session scratch file `fixture-check.txt`.
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
# Canonical data model — draft contract v0.1
|
||||
|
||||
The JSON Schema in `contracts/snapshot.schema.json` is a **first executable draft**. It intentionally models a compact display projection; Claim/Evidence and policy semantics in this document are richer than the initial schema and must be implemented in WP-03.
|
||||
The JSON Schema in `contracts/snapshot.schema.json` is frozen at `schemaVersion` 0.1, including claims. Producing those claims from an external source is still CONTENT-WP-0003. Display policy is `contracts/fact-policy.md`.
|
||||
|
||||
## Records
|
||||
|
||||
|
|
|
|||
|
|
@ -20,8 +20,8 @@ Input → `source record` → `canonical id` → `kind` → `fields/claims` →
|
|||
|
||||
## First connectors
|
||||
|
||||
1. **JSON/NDJSON**: local fixture and remote static file ingestion (local first). A simple reference mapping implementation is in `connectors/json_fixture.py`.
|
||||
2. **Markdown+frontmatter**: repo text with typed metadata and wiki-style references; parse via a robust library, not ad-hoc regex in production. Preserve headings/source locations and content license.
|
||||
1. **JSON/NDJSON**: local fixture ingestion. The build-time connector is `content_explorer/ingest.py` (`load_json_records`, `load_ndjson_records`). `connectors/json_fixture.py` only reads an already published snapshot.
|
||||
2. **Markdown+frontmatter**: repo text with typed metadata. The same module loads it with PyYAML (`load_markdown_records`). Remote static files and wiki-style references are not implemented yet.
|
||||
3. **GitHub or Wikidata** (P2): authenticated/rate-limited external API snapshot; mapping rules, pagination, ETag and license review before activation.
|
||||
|
||||
## Required diagnostics
|
||||
|
|
|
|||
3
content_explorer/__init__.py
Normal file
3
content_explorer/__init__.py
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
"""Domain-independent snapshot contracts and rendering."""
|
||||
|
||||
__version__ = "0.1.0"
|
||||
236
content_explorer/contract.py
Normal file
236
content_explorer/contract.py
Normal file
|
|
@ -0,0 +1,236 @@
|
|||
"""Validate a v0.1 snapshot against its manifest.
|
||||
|
||||
Publication checks live here. Callers pass documents they already loaded.
|
||||
`validate_dataspace` is the file entry point used by `scripts/validate_fixtures.py`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from collections import defaultdict
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import jsonschema
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
SNAPSHOT_SCHEMA_PATH = ROOT / "contracts" / "snapshot.schema.json"
|
||||
MANIFEST_SCHEMA_PATH = ROOT / "contracts" / "manifest.schema.json"
|
||||
SUPPORTED_VERSIONS = frozenset({"0.1"})
|
||||
ENTITY_ID_PREFIX = "urn:ce:"
|
||||
|
||||
|
||||
def load_json(path: Path) -> Any:
|
||||
return json.loads(path.read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
def _validator(schema_path: Path) -> jsonschema.Draft202012Validator:
|
||||
schema = load_json(schema_path)
|
||||
return jsonschema.Draft202012Validator(
|
||||
schema,
|
||||
format_checker=jsonschema.Draft202012Validator.FORMAT_CHECKER,
|
||||
)
|
||||
|
||||
|
||||
def schema_errors(document: dict, schema_path: Path) -> list[str]:
|
||||
validator = _validator(schema_path)
|
||||
found = sorted(validator.iter_errors(document), key=lambda err: list(err.path))
|
||||
messages = []
|
||||
for err in found:
|
||||
where = "/".join(str(part) for part in err.path) or "(root)"
|
||||
messages.append(f"schema {where}: {err.message}")
|
||||
return messages
|
||||
|
||||
|
||||
def version_error(document: dict, label: str) -> str | None:
|
||||
version = document.get("schemaVersion")
|
||||
if version in SUPPORTED_VERSIONS:
|
||||
return None
|
||||
return (
|
||||
f"{label} schemaVersion {version!r} is not supported; "
|
||||
f"expected one of {', '.join(sorted(SUPPORTED_VERSIONS))}"
|
||||
)
|
||||
|
||||
|
||||
def resolve_claims(claims: list[dict], key: str, policy: str) -> dict[str, Any]:
|
||||
"""Choose what a display field may show for one subject.
|
||||
|
||||
`missing` means no claim exists. `unknown` means a claim records that the
|
||||
value is not known. Differing asserted values are a contradiction. The
|
||||
manifest policy `show-all` keeps every asserted value, `withhold` keeps
|
||||
none, and `preferred` keeps the claim marked preferred or withholds when
|
||||
none is marked.
|
||||
"""
|
||||
relevant = [claim for claim in claims if claim.get("key") == key]
|
||||
if not relevant:
|
||||
return {"state": "missing", "values": []}
|
||||
asserted = [claim for claim in relevant if claim.get("status") == "asserted"]
|
||||
values = []
|
||||
for claim in asserted:
|
||||
value = claim.get("value")
|
||||
if value not in values:
|
||||
values.append(value)
|
||||
if len(values) > 1:
|
||||
if policy == "show-all":
|
||||
return {"state": "contradicted", "values": values}
|
||||
if policy == "preferred":
|
||||
chosen = [claim.get("value") for claim in asserted if claim.get("preferred") is True]
|
||||
if len(chosen) == 1:
|
||||
return {"state": "present", "values": chosen}
|
||||
return {"state": "contradicted", "values": []}
|
||||
return {"state": "contradicted", "values": []}
|
||||
if values:
|
||||
return {"state": "present", "values": values}
|
||||
if any(claim.get("status") == "unknown" for claim in relevant):
|
||||
return {"state": "unknown", "values": []}
|
||||
if any(claim.get("status") == "withheld" for claim in relevant):
|
||||
return {"state": "withheld", "values": []}
|
||||
return {"state": "missing", "values": []}
|
||||
|
||||
|
||||
def _cycles(snapshot: dict, predicates: set[str]) -> list[str]:
|
||||
if not predicates:
|
||||
return []
|
||||
adjacency: dict[str, list[tuple[str, str]]] = defaultdict(list)
|
||||
for relation in snapshot.get("relations", []):
|
||||
if relation.get("predicate") in predicates:
|
||||
adjacency[relation["from"]].append((relation["to"], relation["id"]))
|
||||
color: dict[str, int] = {}
|
||||
errors: list[str] = []
|
||||
|
||||
def walk(node: str) -> None:
|
||||
color[node] = 1
|
||||
for nxt, relation_id in adjacency.get(node, []):
|
||||
state = color.get(nxt, 0)
|
||||
if state == 1:
|
||||
errors.append(
|
||||
f"taxonomic cycle via {relation_id}: {node} -> {nxt}"
|
||||
)
|
||||
elif state == 0:
|
||||
walk(nxt)
|
||||
color[node] = 2
|
||||
|
||||
for start in list(adjacency):
|
||||
if color.get(start, 0) == 0:
|
||||
walk(start)
|
||||
return errors
|
||||
|
||||
|
||||
def graph_errors(snapshot: dict, manifest: dict) -> list[str]:
|
||||
errors: list[str] = []
|
||||
version = version_error(snapshot, "snapshot")
|
||||
if version:
|
||||
errors.append(version)
|
||||
manifest_version = version_error(manifest, "manifest")
|
||||
if manifest_version:
|
||||
errors.append(manifest_version)
|
||||
|
||||
kinds = [item.get("id") for item in snapshot.get("kinds", [])]
|
||||
if len(kinds) != len(set(kinds)):
|
||||
errors.append("duplicate kind id")
|
||||
kind_ids = set(kinds)
|
||||
entity_ids = [item.get("id") for item in snapshot.get("entities", [])]
|
||||
if len(entity_ids) != len(set(entity_ids)):
|
||||
errors.append("duplicate entity id")
|
||||
entities = {item.get("id"): item for item in snapshot.get("entities", []) if item.get("id")}
|
||||
sources = {item.get("id") for item in snapshot.get("sources", [])}
|
||||
claim_ids = [item.get("id") for item in snapshot.get("claims", [])]
|
||||
if len(claim_ids) != len(set(claim_ids)):
|
||||
errors.append("duplicate claim id")
|
||||
|
||||
dataspace_id = (snapshot.get("dataspace") or {}).get("id")
|
||||
if manifest.get("id") != dataspace_id:
|
||||
errors.append(
|
||||
f"manifest id {manifest.get('id')!r} does not match dataspace {dataspace_id!r}"
|
||||
)
|
||||
|
||||
labels = manifest.get("relationLabels") or {}
|
||||
views = manifest.get("kindViews") or {}
|
||||
for kind_id in kind_ids:
|
||||
if kind_id not in views:
|
||||
errors.append(f"kind {kind_id} has no manifest kindView")
|
||||
for featured in (manifest.get("home") or {}).get("featuredKinds", []):
|
||||
if featured not in kind_ids:
|
||||
errors.append(f"featured kind {featured} is not in the snapshot")
|
||||
|
||||
for entity in snapshot.get("entities", []):
|
||||
entity_id = entity.get("id", "")
|
||||
if not str(entity_id).startswith(ENTITY_ID_PREFIX):
|
||||
errors.append(f"entity id {entity_id} is not a urn:ce identifier")
|
||||
if entity.get("kindId") not in kind_ids:
|
||||
errors.append(f"{entity_id}: undeclared kind {entity.get('kindId')}")
|
||||
if not entity.get("sourceRefs"):
|
||||
errors.append(f"{entity_id}: missing source")
|
||||
for source_id in entity.get("sourceRefs", []):
|
||||
if source_id not in sources:
|
||||
errors.append(f"{entity_id}: missing source {source_id}")
|
||||
for claim_id in entity.get("claimRefs", []):
|
||||
if claim_id not in claim_ids:
|
||||
errors.append(f"{entity_id}: missing claim {claim_id}")
|
||||
|
||||
known_entities = set(entities)
|
||||
for relation in snapshot.get("relations", []):
|
||||
relation_id = relation.get("id")
|
||||
predicate = relation.get("predicate")
|
||||
if predicate not in labels:
|
||||
errors.append(f"relation {relation_id}: predicate {predicate} has no label")
|
||||
for field in ("from", "to"):
|
||||
target = relation.get(field)
|
||||
if target not in known_entities:
|
||||
errors.append(f"relation {relation_id}: dangling {field} {target}")
|
||||
refs = relation.get("sourceRefs") or []
|
||||
if not refs:
|
||||
errors.append(f"relation {relation_id}: missing source")
|
||||
for source_id in refs:
|
||||
if source_id not in sources:
|
||||
errors.append(f"relation {relation_id}: missing source {source_id}")
|
||||
|
||||
for claim in snapshot.get("claims", []):
|
||||
claim_id = claim.get("id")
|
||||
subject = claim.get("subjectId")
|
||||
if subject not in known_entities:
|
||||
errors.append(f"claim {claim_id}: dangling subject {subject}")
|
||||
evidence = claim.get("evidenceRefs") or []
|
||||
if not evidence:
|
||||
errors.append(f"claim {claim_id}: missing source")
|
||||
for source_id in evidence:
|
||||
if source_id not in sources:
|
||||
errors.append(f"claim {claim_id}: missing source {source_id}")
|
||||
|
||||
acyclic = set(manifest.get("acyclicPredicates") or [])
|
||||
errors.extend(_cycles(snapshot, acyclic))
|
||||
return errors
|
||||
|
||||
|
||||
def size_errors(snapshot: dict, *, min_kinds: int = 3, min_entities: int = 8) -> list[str]:
|
||||
errors = []
|
||||
kind_count = len(snapshot.get("kinds") or [])
|
||||
entity_count = len(snapshot.get("entities") or [])
|
||||
if kind_count < min_kinds:
|
||||
errors.append(f"needs at least {min_kinds} kinds, found {kind_count}")
|
||||
if entity_count < min_entities:
|
||||
errors.append(f"needs at least {min_entities} entities, found {entity_count}")
|
||||
return errors
|
||||
|
||||
|
||||
def validate_documents(
|
||||
snapshot: dict,
|
||||
manifest: dict,
|
||||
*,
|
||||
require_size: bool = False,
|
||||
) -> list[str]:
|
||||
errors = schema_errors(snapshot, SNAPSHOT_SCHEMA_PATH)
|
||||
errors.extend(schema_errors(manifest, MANIFEST_SCHEMA_PATH))
|
||||
errors.extend(graph_errors(snapshot, manifest))
|
||||
if require_size:
|
||||
errors.extend(size_errors(snapshot))
|
||||
return errors
|
||||
|
||||
|
||||
def validate_dataspace(directory: Path, *, require_size: bool = False) -> list[str]:
|
||||
snapshot_path = directory / "snapshot.json"
|
||||
manifest_path = directory / "manifest.json"
|
||||
snapshot = load_json(snapshot_path)
|
||||
manifest = load_json(manifest_path)
|
||||
errors = validate_documents(snapshot, manifest, require_size=require_size)
|
||||
return [f"{directory.name}: {message}" for message in errors]
|
||||
371
content_explorer/ingest.py
Normal file
371
content_explorer/ingest.py
Normal file
|
|
@ -0,0 +1,371 @@
|
|||
"""Build a v0.1 snapshot from JSON, NDJSON, or Markdown frontmatter.
|
||||
|
||||
Connectors stop at a snapshot and a report. They do not render.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
CONNECTOR_VERSION = "0.1.0"
|
||||
ALLOWED_LICENSES = frozenset({"CC0-1.0", "CC-BY-4.0", "MIT"})
|
||||
|
||||
|
||||
def _digest(payload: bytes) -> str:
|
||||
return hashlib.sha256(payload).hexdigest()
|
||||
|
||||
|
||||
def snapshot_hash(snapshot: dict) -> str:
|
||||
encoded = json.dumps(snapshot, sort_keys=True, separators=(",", ":")).encode("utf-8")
|
||||
return _digest(encoded)
|
||||
|
||||
|
||||
def load_json_records(path: Path) -> tuple[list[dict], str]:
|
||||
raw = path.read_bytes()
|
||||
document = json.loads(raw.decode("utf-8"))
|
||||
if isinstance(document, dict) and "records" in document:
|
||||
records = document["records"]
|
||||
elif isinstance(document, list):
|
||||
records = document
|
||||
else:
|
||||
raise ValueError(f"{path} is not a record list")
|
||||
return records, _digest(raw)
|
||||
|
||||
|
||||
def load_ndjson_records(path: Path) -> tuple[list[dict], str]:
|
||||
raw = path.read_bytes()
|
||||
records = []
|
||||
for line in raw.decode("utf-8").splitlines():
|
||||
if line.strip():
|
||||
records.append(json.loads(line))
|
||||
return records, _digest(raw)
|
||||
|
||||
|
||||
def load_markdown_records(directory: Path) -> tuple[list[dict], str]:
|
||||
parts: list[bytes] = []
|
||||
records: list[dict] = []
|
||||
for path in sorted(directory.glob("*.md")):
|
||||
raw = path.read_bytes()
|
||||
parts.append(raw)
|
||||
text = raw.decode("utf-8")
|
||||
if not text.startswith("---\n"):
|
||||
raise ValueError(f"{path.name} has no frontmatter")
|
||||
_, frontmatter, body = text.split("---", 2)
|
||||
record = yaml.safe_load(frontmatter) or {}
|
||||
if not isinstance(record, dict):
|
||||
raise ValueError(f"{path.name} frontmatter is not a mapping")
|
||||
if body.strip() and not record.get("summary"):
|
||||
record["summary"] = body.strip()
|
||||
records.append(record)
|
||||
return records, _digest(b"".join(parts))
|
||||
|
||||
|
||||
def _entity_id(dataspace: str, kind: str, key: str) -> str:
|
||||
return f"urn:ce:{dataspace}:{kind}:{key}"
|
||||
|
||||
|
||||
def _unpublished(report: dict) -> dict[str, Any]:
|
||||
return {"snapshot": None, "manifest": None, "report": report}
|
||||
|
||||
|
||||
def build_manifest(
|
||||
*,
|
||||
dataspace_id: str,
|
||||
heading: str,
|
||||
kinds: list[dict],
|
||||
relation_labels: dict[str, dict[str, str]],
|
||||
predicates: list[str],
|
||||
fact_policy: str = "show-all",
|
||||
acyclic_predicates: list[str] | None = None,
|
||||
) -> tuple[dict[str, Any] | None, list[str]]:
|
||||
"""Build a v0.1 manifest. Missing relation labels are returned and not invented."""
|
||||
missing = sorted({predicate for predicate in predicates if predicate not in relation_labels})
|
||||
if missing:
|
||||
return None, missing
|
||||
kind_ids = [kind["id"] for kind in kinds]
|
||||
labels = {
|
||||
predicate: {
|
||||
"forward": relation_labels[predicate]["forward"],
|
||||
"inverse": relation_labels[predicate]["inverse"],
|
||||
}
|
||||
for predicate in sorted(set(predicates))
|
||||
}
|
||||
manifest = {
|
||||
"schemaVersion": "0.1",
|
||||
"id": dataspace_id,
|
||||
"factPolicy": fact_policy,
|
||||
"home": {"heading": heading, "featuredKinds": kind_ids},
|
||||
"navigation": [{"label": "Browse", "kinds": kind_ids}],
|
||||
"kindViews": {
|
||||
kind["id"]: {"columns": ["title"], "facets": ["tags"]}
|
||||
for kind in kinds
|
||||
},
|
||||
"panels": ["summary", "relations", "evidence", "claims"],
|
||||
"relationLabels": labels,
|
||||
"acyclicPredicates": list(acyclic_predicates or []),
|
||||
}
|
||||
return manifest, []
|
||||
|
||||
|
||||
def human_import_report(report: dict) -> str:
|
||||
"""Provenance and rights summary a person can read next to the snapshot."""
|
||||
source = report.get("source") or {}
|
||||
lines = [
|
||||
"# Import report",
|
||||
"",
|
||||
f"Connector version: {report.get('connectorVersion', '')}",
|
||||
f"Source id: {source.get('id', '')}",
|
||||
f"Source URI: {source.get('uri', '')}",
|
||||
f"License: {source.get('license', '')}",
|
||||
f"Attribution: {source.get('attribution') or '(none)'}",
|
||||
f"Source digest: {report.get('sourceDigest') or '(none)'}",
|
||||
f"Content hash: {report.get('contentHash') or '(not published)'}",
|
||||
f"Published: {str(bool(report.get('published'))).lower()}",
|
||||
f"Candidates: {report.get('candidates', 0)}",
|
||||
f"Accepted: {report.get('accepted', 0)}",
|
||||
"",
|
||||
"## Tombstones",
|
||||
]
|
||||
tombstones = report.get("tombstoned") or []
|
||||
lines.extend(f"- {key}" for key in tombstones)
|
||||
if not tombstones:
|
||||
lines.append("- (none)")
|
||||
lines.extend(["", "## Quarantine"])
|
||||
quarantined = report.get("quarantined") or []
|
||||
if not quarantined:
|
||||
lines.append("- (none)")
|
||||
for item in quarantined:
|
||||
lines.append(f"- {json.dumps(item, sort_keys=True)}")
|
||||
lines.extend(["", "## Dangling references"])
|
||||
dangling = report.get("dangling") or []
|
||||
if not dangling:
|
||||
lines.append("- (none)")
|
||||
for item in dangling:
|
||||
lines.append(f"- {item.get('from')} -{item.get('predicate')}-> {item.get('to')}")
|
||||
lines.extend(["", "## Identity conflicts"])
|
||||
conflicts = report.get("conflicts") or []
|
||||
if not conflicts:
|
||||
lines.append("- (none)")
|
||||
for item in conflicts:
|
||||
lines.append(f"- {item.get('key')}: {item.get('reason')}")
|
||||
lines.append("")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def project_records(
|
||||
records: list[dict],
|
||||
*,
|
||||
dataspace_id: str,
|
||||
title: str,
|
||||
description: str,
|
||||
source: dict,
|
||||
kinds: list[dict],
|
||||
observed_at: str,
|
||||
source_digest: str = "",
|
||||
relation_labels: dict[str, dict[str, str]] | None = None,
|
||||
fact_policy: str = "show-all",
|
||||
acyclic_predicates: list[str] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Return a report. `snapshot` is present only when publication is allowed.
|
||||
|
||||
Live keys are collected first. Relations resolve in a second pass against
|
||||
that set. Duplicate keys and a live key that is also tombstoned are conflicts.
|
||||
"""
|
||||
license_name = str(source.get("license") or "")
|
||||
report: dict[str, Any] = {
|
||||
"connectorVersion": CONNECTOR_VERSION,
|
||||
"candidates": len(records),
|
||||
"accepted": 0,
|
||||
"tombstoned": [],
|
||||
"quarantined": [],
|
||||
"dangling": [],
|
||||
"conflicts": [],
|
||||
"published": False,
|
||||
"sourceDigest": source_digest,
|
||||
"source": {
|
||||
"id": source.get("id", ""),
|
||||
"uri": source.get("uri", ""),
|
||||
"title": source.get("title", ""),
|
||||
"license": license_name,
|
||||
"attribution": source.get("attribution") or "",
|
||||
},
|
||||
}
|
||||
if license_name not in ALLOWED_LICENSES:
|
||||
report["quarantined"].append({
|
||||
"reason": "rights",
|
||||
"license": license_name or "(missing)",
|
||||
})
|
||||
return _unpublished(report)
|
||||
if license_name == "CC-BY-4.0" and not str(source.get("attribution") or "").strip():
|
||||
report["quarantined"].append({
|
||||
"reason": "rights",
|
||||
"license": license_name,
|
||||
"detail": "missing-attribution",
|
||||
})
|
||||
return _unpublished(report)
|
||||
|
||||
kind_ids = {kind["id"] for kind in kinds}
|
||||
tombstoned_keys = set()
|
||||
for record in records:
|
||||
if record.get("deleted"):
|
||||
report["tombstoned"].append(record.get("key"))
|
||||
tombstoned_keys.add(f"{record.get('kind')}:{record.get('key')}")
|
||||
|
||||
grouped: dict[str, list[dict]] = {}
|
||||
for record in records:
|
||||
if record.get("deleted"):
|
||||
continue
|
||||
kind = record["kind"]
|
||||
key = record["key"]
|
||||
qualified = f"{kind}:{key}"
|
||||
if kind not in kind_ids:
|
||||
report["quarantined"].append({"reason": "unknown-kind", "key": key, "kind": kind})
|
||||
continue
|
||||
if qualified in tombstoned_keys:
|
||||
report["conflicts"].append({"key": qualified, "reason": "tombstone-and-live"})
|
||||
continue
|
||||
grouped.setdefault(qualified, []).append(record)
|
||||
|
||||
accepted: dict[str, dict] = {}
|
||||
for qualified, group in grouped.items():
|
||||
if len(group) > 1:
|
||||
report["conflicts"].append({
|
||||
"key": qualified,
|
||||
"reason": "duplicate-identity",
|
||||
"count": len(group),
|
||||
})
|
||||
continue
|
||||
accepted[qualified] = group[0]
|
||||
|
||||
entities = []
|
||||
relations = []
|
||||
claims = []
|
||||
predicates: list[str] = []
|
||||
# Second pass: links resolve only against keys accepted in the first pass.
|
||||
for qualified, record in accepted.items():
|
||||
kind, key = qualified.split(":", 1)
|
||||
entity_id = _entity_id(dataspace_id, kind, key)
|
||||
for link in record.get("relations") or []:
|
||||
target = link.get("to")
|
||||
predicate = link.get("predicate")
|
||||
if target not in accepted:
|
||||
report["dangling"].append({"from": qualified, "to": target, "predicate": predicate})
|
||||
claim_id = f"claim:{kind}:{key}:tldr"
|
||||
entities.append({
|
||||
"id": entity_id,
|
||||
"kindId": kind,
|
||||
"title": record["title"],
|
||||
"tldr": record["tldr"],
|
||||
"summary": record.get("summary") or record["tldr"],
|
||||
"tags": list(record.get("tags") or []),
|
||||
"sourceRefs": [source["id"]],
|
||||
"claimRefs": [claim_id],
|
||||
"asOf": observed_at,
|
||||
})
|
||||
claims.append({
|
||||
"id": claim_id,
|
||||
"subjectId": entity_id,
|
||||
"key": "tldr",
|
||||
"value": record["tldr"],
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [source["id"]],
|
||||
"observedAt": observed_at,
|
||||
})
|
||||
for index, link in enumerate(record.get("relations") or [], start=1):
|
||||
target = link["to"]
|
||||
predicate = link["predicate"]
|
||||
predicates.append(predicate)
|
||||
target_kind, target_key = target.split(":", 1)
|
||||
relations.append({
|
||||
"id": f"{kind}-{key}-{index}",
|
||||
"predicate": predicate,
|
||||
"from": entity_id,
|
||||
"to": _entity_id(dataspace_id, target_kind, target_key),
|
||||
"sourceRefs": [source["id"]],
|
||||
"asOf": observed_at,
|
||||
})
|
||||
|
||||
blocked = (
|
||||
report["dangling"]
|
||||
or report["conflicts"]
|
||||
or any(item["reason"] == "unknown-kind" for item in report["quarantined"])
|
||||
)
|
||||
if blocked:
|
||||
return _unpublished(report)
|
||||
|
||||
manifest, missing_labels = build_manifest(
|
||||
dataspace_id=dataspace_id,
|
||||
heading=title,
|
||||
kinds=kinds,
|
||||
relation_labels=relation_labels or {},
|
||||
predicates=predicates,
|
||||
fact_policy=fact_policy,
|
||||
acyclic_predicates=acyclic_predicates,
|
||||
)
|
||||
if missing_labels:
|
||||
report["quarantined"].append({
|
||||
"reason": "missing-relation-label",
|
||||
"predicates": missing_labels,
|
||||
})
|
||||
return _unpublished(report)
|
||||
|
||||
entities.sort(key=lambda item: item["id"])
|
||||
relations.sort(key=lambda item: item["id"])
|
||||
claims.sort(key=lambda item: item["id"])
|
||||
snapshot = {
|
||||
"schemaVersion": "0.1",
|
||||
"dataspace": {"id": dataspace_id, "title": title, "description": description},
|
||||
"kinds": kinds,
|
||||
"entities": entities,
|
||||
"relations": relations,
|
||||
"sources": [source],
|
||||
"claims": claims,
|
||||
}
|
||||
for claim in claims:
|
||||
if not claim["evidenceRefs"]:
|
||||
report["quarantined"].append({"reason": "missing-evidence", "claim": claim["id"]})
|
||||
return _unpublished(report)
|
||||
report["accepted"] = len(entities)
|
||||
report["published"] = True
|
||||
report["contentHash"] = snapshot_hash(snapshot)
|
||||
return {"snapshot": snapshot, "manifest": manifest, "report": report}
|
||||
|
||||
|
||||
def publish_directory(destination: Path, result: dict[str, Any]) -> None:
|
||||
"""Replace `destination` only after the snapshot, manifest, and both reports exist."""
|
||||
if not result["report"]["published"] or result["snapshot"] is None or result.get("manifest") is None:
|
||||
raise RuntimeError("refusing to publish a quarantined or dangling import")
|
||||
destination.parent.mkdir(parents=True, exist_ok=True)
|
||||
temporary = destination.parent / f".{destination.name}.incoming"
|
||||
if temporary.exists():
|
||||
shutil.rmtree(temporary)
|
||||
temporary.mkdir()
|
||||
(temporary / "snapshot.json").write_text(
|
||||
json.dumps(result["snapshot"], indent=2) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
(temporary / "manifest.json").write_text(
|
||||
json.dumps(result["manifest"], indent=2) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
(temporary / "import-report.json").write_text(
|
||||
json.dumps(result["report"], indent=2) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
(temporary / "import-report.md").write_text(
|
||||
human_import_report(result["report"]),
|
||||
encoding="utf-8",
|
||||
)
|
||||
backup = destination.parent / f".{destination.name}.backup"
|
||||
if backup.exists():
|
||||
shutil.rmtree(backup)
|
||||
if destination.exists():
|
||||
os.rename(destination, backup)
|
||||
os.rename(temporary, destination)
|
||||
60
content_explorer/render.py
Normal file
60
content_explorer/render.py
Normal file
|
|
@ -0,0 +1,60 @@
|
|||
"""Render a snapshot with whatever labels the manifest supplies.
|
||||
|
||||
The text below is the shared explorer. It must not name a dataspace, a kind,
|
||||
or a predicate. Those strings come from the snapshot and the manifest.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from content_explorer.contract import resolve_claims
|
||||
|
||||
|
||||
def render_explorer(snapshot: dict, manifest: dict) -> str:
|
||||
kinds = {kind["id"]: kind for kind in snapshot.get("kinds", [])}
|
||||
entities = {entity["id"]: entity for entity in snapshot.get("entities", [])}
|
||||
claims = snapshot.get("claims", [])
|
||||
policy = manifest.get("factPolicy", "show-all")
|
||||
lines: list[str] = [manifest["home"]["heading"], ""]
|
||||
|
||||
for section in manifest.get("navigation", []):
|
||||
names = [kinds[kind_id]["plural"] for kind_id in section["kinds"] if kind_id in kinds]
|
||||
lines.append(f"{section['label']}: {', '.join(names)}")
|
||||
lines.append("")
|
||||
|
||||
for kind_id in manifest["home"]["featuredKinds"]:
|
||||
kind = kinds.get(kind_id)
|
||||
if kind is None:
|
||||
continue
|
||||
lines.append(f"## {kind['plural']}")
|
||||
members = [entity for entity in snapshot.get("entities", []) if entity.get("kindId") == kind_id]
|
||||
if not members:
|
||||
lines.append("(none)")
|
||||
for entity in members:
|
||||
field = resolve_claims(
|
||||
[claim for claim in claims if claim.get("subjectId") == entity["id"]],
|
||||
"tldr",
|
||||
policy,
|
||||
)
|
||||
if field["state"] == "unknown":
|
||||
shown = "unknown"
|
||||
elif field["state"] == "contradicted":
|
||||
shown = "contested: " + " | ".join(str(value) for value in field["values"])
|
||||
elif field["state"] == "withheld":
|
||||
shown = "withheld"
|
||||
elif field["values"]:
|
||||
shown = str(field["values"][0])
|
||||
else:
|
||||
shown = entity.get("tldr", "")
|
||||
lines.append(f"- {entity['title']}: {shown}")
|
||||
lines.append("")
|
||||
|
||||
lines.append("## Relations")
|
||||
labels = manifest.get("relationLabels") or {}
|
||||
for relation in snapshot.get("relations", []):
|
||||
predicate = relation["predicate"]
|
||||
wording = labels.get(predicate, {}).get("forward", predicate)
|
||||
origin = entities.get(relation["from"], {}).get("title", relation["from"])
|
||||
destination = entities.get(relation["to"], {}).get("title", relation["to"])
|
||||
lines.append(f"- {origin} — {wording} → {destination}")
|
||||
lines.append("")
|
||||
return "\n".join(lines).rstrip() + "\n"
|
||||
9
contracts/MIGRATIONS.md
Normal file
9
contracts/MIGRATIONS.md
Normal file
|
|
@ -0,0 +1,9 @@
|
|||
# Snapshot and manifest migrations
|
||||
|
||||
## 0.1 — frozen 2026-10-12
|
||||
|
||||
This is the first supported `schemaVersion`. The seed draft omitted `claims`, required relation `sourceRefs`, manifest `schemaVersion`, `relationLabels`, and `factPolicy`. Those documents were edited in place. There is no 0.0 reader.
|
||||
|
||||
`content_explorer.contract.SUPPORTED_VERSIONS` accepts only `0.1`. A document with any other version fails validation. A later version must add a named migrator, for example `migrate_0_1_to_0_2`, and must not guess missing rights or claims.
|
||||
|
||||
Display text on an entity is not a second source of truth. A `tldr` claim with `status: asserted` is the statement the renderer prefers to show. `status: unknown` records that the field was considered and is not known.
|
||||
12
contracts/SNAPSHOT.md
Normal file
12
contracts/SNAPSHOT.md
Normal file
|
|
@ -0,0 +1,12 @@
|
|||
# Snapshot format 0.1
|
||||
|
||||
Normative files:
|
||||
|
||||
- `contracts/snapshot.schema.json` — entity, relation, source, and claim document.
|
||||
- `contracts/manifest.schema.json` — presentation. Kind names, facet names, and relation wording live here.
|
||||
- `contracts/MIGRATIONS.md` — version gate.
|
||||
- `contracts/fact-policy.md` — missing, unknown, and contradiction.
|
||||
|
||||
Identifiers for entities are `urn:ce:<dataspace>:<kind>:<key>`. The same key in another dataspace is a different entity. Relation ids are local to the snapshot.
|
||||
|
||||
A published example must contain at least three kinds and eight entities. `scripts/validate_fixtures.py` enforces that, plus the schema, dangling references, duplicate ids, missing sources, and cycles on `acyclicPredicates` only.
|
||||
19
contracts/fact-policy.md
Normal file
19
contracts/fact-policy.md
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
# Fact policy v0.1
|
||||
|
||||
A claim is a statement about one entity field. It is not a proof. The snapshot keeps every claim. The manifest `factPolicy` only decides what a display may show.
|
||||
|
||||
| Situation | Claim shape | `resolve_claims` state |
|
||||
|---|---|---|
|
||||
| No claim for that key | absent | `missing` |
|
||||
| The field was checked and has no value | `status: unknown` | `unknown` |
|
||||
| One asserted value | `status: asserted` | `present` |
|
||||
| The holder asked not to display it | `status: withheld` | `withheld` |
|
||||
| Two asserted values | both `status: asserted` | `contradicted` |
|
||||
|
||||
Policies when values disagree:
|
||||
|
||||
- `show-all` returns every asserted value and state `contradicted`.
|
||||
- `withhold` returns no values and state `contradicted`.
|
||||
- `preferred` returns the single claim with `preferred: true`. If none or more than one is marked, it returns no values and state `contradicted`.
|
||||
|
||||
Absence is not the same as unknown, and unknown is not the same as false. A relation with no `sourceRefs` is invalid. It is not stored as an unsourced fact. Conflicting claims stay in the snapshot either way. The policy does not delete them.
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
{
|
||||
"id": "systems-thinking",
|
||||
"home": {
|
||||
"heading": "Explore systems thinking",
|
||||
"heading": "Explore Systems Thinking",
|
||||
"featuredKinds": [
|
||||
"concept",
|
||||
"person",
|
||||
|
|
@ -11,42 +11,97 @@
|
|||
},
|
||||
"navigation": [
|
||||
{
|
||||
"label": "Explore",
|
||||
"label": "Browse",
|
||||
"kinds": [
|
||||
"concept",
|
||||
"method",
|
||||
"person",
|
||||
"publication",
|
||||
"person"
|
||||
"method"
|
||||
]
|
||||
}
|
||||
],
|
||||
"kindViews": {
|
||||
"concept": {
|
||||
"facets": [
|
||||
"discipline",
|
||||
"tradition"
|
||||
],
|
||||
"columns": [
|
||||
"name",
|
||||
"tldr",
|
||||
"tags"
|
||||
],
|
||||
"facets": [
|
||||
"tags"
|
||||
]
|
||||
}
|
||||
},
|
||||
"relationLabels": {
|
||||
"influences": {
|
||||
"forward": "Influences",
|
||||
"inverse": "Influenced by"
|
||||
},
|
||||
"discusses": {
|
||||
"forward": "Discusses",
|
||||
"inverse": "Discussed in"
|
||||
"person": {
|
||||
"columns": [
|
||||
"name",
|
||||
"tldr",
|
||||
"tags"
|
||||
],
|
||||
"facets": [
|
||||
"tags"
|
||||
]
|
||||
},
|
||||
"publication": {
|
||||
"columns": [
|
||||
"name",
|
||||
"tldr",
|
||||
"tags"
|
||||
],
|
||||
"facets": [
|
||||
"tags"
|
||||
]
|
||||
},
|
||||
"method": {
|
||||
"columns": [
|
||||
"name",
|
||||
"tldr",
|
||||
"tags"
|
||||
],
|
||||
"facets": [
|
||||
"tags"
|
||||
]
|
||||
}
|
||||
},
|
||||
"panels": [
|
||||
"summary",
|
||||
"relations",
|
||||
"evidence"
|
||||
"evidence",
|
||||
"claims"
|
||||
],
|
||||
"schemaVersion": "0.1",
|
||||
"factPolicy": "show-all",
|
||||
"relationLabels": {
|
||||
"represented-by": {
|
||||
"forward": "Represented by",
|
||||
"inverse": "Represents"
|
||||
},
|
||||
"scopes": {
|
||||
"forward": "Scopes",
|
||||
"inverse": "Scoped by"
|
||||
},
|
||||
"associated-with": {
|
||||
"forward": "Associated with",
|
||||
"inverse": "Associated with"
|
||||
},
|
||||
"authored-by": {
|
||||
"forward": "Authored by",
|
||||
"inverse": "Authored"
|
||||
},
|
||||
"related-to": {
|
||||
"forward": "Related to",
|
||||
"inverse": "Related to"
|
||||
},
|
||||
"models": {
|
||||
"forward": "Models",
|
||||
"inverse": "Modeled by"
|
||||
},
|
||||
"explores": {
|
||||
"forward": "Explores",
|
||||
"inverse": "Explored by"
|
||||
}
|
||||
},
|
||||
"acyclicPredicates": [
|
||||
"authored-by"
|
||||
],
|
||||
"theme": {
|
||||
"accent": "indigo"
|
||||
|
|
|
|||
73
contracts/manifest.schema.json
Normal file
73
contracts/manifest.schema.json
Normal file
|
|
@ -0,0 +1,73 @@
|
|||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://content-explorer.local/schemas/manifest-0.1.json",
|
||||
"title": "Content Explorer dataspace manifest v0.1",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["schemaVersion", "id", "home", "navigation", "kindViews", "panels", "relationLabels", "factPolicy"],
|
||||
"properties": {
|
||||
"schemaVersion": {"const": "0.1"},
|
||||
"id": {"type": "string", "pattern": "^[a-z0-9-]+$"},
|
||||
"home": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["heading", "featuredKinds"],
|
||||
"properties": {
|
||||
"heading": {"type": "string", "minLength": 1},
|
||||
"featuredKinds": {"type": "array", "items": {"type": "string"}, "minItems": 1}
|
||||
}
|
||||
},
|
||||
"navigation": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["label", "kinds"],
|
||||
"properties": {
|
||||
"label": {"type": "string"},
|
||||
"kinds": {"type": "array", "items": {"type": "string"}, "minItems": 1}
|
||||
}
|
||||
}
|
||||
},
|
||||
"kindViews": {
|
||||
"type": "object",
|
||||
"additionalProperties": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["columns", "facets"],
|
||||
"properties": {
|
||||
"columns": {"type": "array", "items": {"type": "string"}, "minItems": 1},
|
||||
"facets": {"type": "array", "items": {"type": "string"}}
|
||||
}
|
||||
}
|
||||
},
|
||||
"panels": {
|
||||
"type": "array",
|
||||
"items": {"enum": ["summary", "relations", "evidence", "claims"]}
|
||||
},
|
||||
"relationLabels": {
|
||||
"type": "object",
|
||||
"additionalProperties": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["forward", "inverse"],
|
||||
"properties": {
|
||||
"forward": {"type": "string"},
|
||||
"inverse": {"type": "string"}
|
||||
}
|
||||
}
|
||||
},
|
||||
"acyclicPredicates": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"}
|
||||
},
|
||||
"factPolicy": {"enum": ["show-all", "preferred", "withhold"]},
|
||||
"theme": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["accent"],
|
||||
"properties": {"accent": {"type": "string"}}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1,63 +1,33 @@
|
|||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"title": "Content Explorer compact snapshot v0.1",
|
||||
"$id": "https://content-explorer.local/schemas/snapshot-0.1.json",
|
||||
"title": "Content Explorer snapshot v0.1",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"schemaVersion",
|
||||
"dataspace",
|
||||
"kinds",
|
||||
"entities",
|
||||
"relations",
|
||||
"sources"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"required": ["schemaVersion", "dataspace", "kinds", "entities", "relations", "sources", "claims"],
|
||||
"properties": {
|
||||
"schemaVersion": {
|
||||
"const": "0.1"
|
||||
},
|
||||
"schemaVersion": {"const": "0.1"},
|
||||
"dataspace": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"title",
|
||||
"description"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"required": ["id", "title", "description"],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string"
|
||||
},
|
||||
"title": {
|
||||
"type": "string"
|
||||
},
|
||||
"description": {
|
||||
"type": "string"
|
||||
}
|
||||
"id": {"type": "string", "pattern": "^[a-z0-9-]+$"},
|
||||
"title": {"type": "string", "minLength": 1},
|
||||
"description": {"type": "string", "minLength": 1}
|
||||
}
|
||||
},
|
||||
"kinds": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"label",
|
||||
"plural"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"required": ["id", "label", "plural"],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string"
|
||||
},
|
||||
"label": {
|
||||
"type": "string"
|
||||
},
|
||||
"plural": {
|
||||
"type": "string"
|
||||
},
|
||||
"facets": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
"id": {"type": "string", "pattern": "^[a-z0-9-]+$"},
|
||||
"label": {"type": "string"},
|
||||
"plural": {"type": "string"}
|
||||
}
|
||||
}
|
||||
},
|
||||
|
|
@ -65,48 +35,22 @@
|
|||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"kindId",
|
||||
"title",
|
||||
"tldr",
|
||||
"summary",
|
||||
"sourceRefs"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"required": ["id", "kindId", "title", "tldr", "summary", "sourceRefs"],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string"
|
||||
},
|
||||
"kindId": {
|
||||
"type": "string"
|
||||
},
|
||||
"title": {
|
||||
"type": "string"
|
||||
},
|
||||
"tldr": {
|
||||
"type": "string"
|
||||
},
|
||||
"summary": {
|
||||
"type": "string"
|
||||
},
|
||||
"tags": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"properties": {
|
||||
"type": "object"
|
||||
},
|
||||
"sourceRefs": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"asOf": {
|
||||
"type": "string",
|
||||
"format": "date"
|
||||
"id": {"type": "string", "pattern": "^urn:ce:[a-z0-9-]+:[a-z0-9-]+:[a-z0-9-]+$"},
|
||||
"kindId": {"type": "string"},
|
||||
"title": {"type": "string"},
|
||||
"tldr": {"type": "string"},
|
||||
"summary": {"type": "string"},
|
||||
"aliases": {"type": "array", "items": {"type": "string"}},
|
||||
"tags": {"type": "array", "items": {"type": "string"}},
|
||||
"properties": {"type": "object"},
|
||||
"sourceRefs": {"type": "array", "items": {"type": "string"}},
|
||||
"claimRefs": {"type": "array", "items": {"type": "string"}},
|
||||
"asOf": {"type": "string", "format": "date"},
|
||||
"state": {
|
||||
"enum": ["present", "missing", "unknown", "redacted", "stale", "contradicted", "not-applicable"]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -115,63 +59,52 @@
|
|||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"predicate",
|
||||
"from",
|
||||
"to"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"required": ["id", "predicate", "from", "to", "sourceRefs"],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string"
|
||||
},
|
||||
"predicate": {
|
||||
"type": "string"
|
||||
},
|
||||
"from": {
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"type": "string"
|
||||
},
|
||||
"sourceRefs": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
"id": {"type": "string"},
|
||||
"predicate": {"type": "string", "pattern": "^[a-z0-9-]+$"},
|
||||
"from": {"type": "string"},
|
||||
"to": {"type": "string"},
|
||||
"sourceRefs": {"type": "array", "items": {"type": "string"}, "minItems": 1},
|
||||
"asOf": {"type": "string", "format": "date"}
|
||||
}
|
||||
}
|
||||
},
|
||||
"sources": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"uri",
|
||||
"title",
|
||||
"license"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"required": ["id", "uri", "title", "license"],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string"
|
||||
},
|
||||
"uri": {
|
||||
"type": "string"
|
||||
},
|
||||
"title": {
|
||||
"type": "string"
|
||||
},
|
||||
"license": {
|
||||
"type": "string"
|
||||
},
|
||||
"attribution": {
|
||||
"type": "string"
|
||||
}
|
||||
"id": {"type": "string"},
|
||||
"uri": {"type": "string"},
|
||||
"title": {"type": "string"},
|
||||
"license": {"type": "string"},
|
||||
"attribution": {"type": "string"},
|
||||
"retrievedAt": {"type": "string", "format": "date"}
|
||||
}
|
||||
}
|
||||
},
|
||||
"claims": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["id", "subjectId", "key", "status", "evidenceRefs"],
|
||||
"properties": {
|
||||
"id": {"type": "string"},
|
||||
"subjectId": {"type": "string"},
|
||||
"key": {"type": "string"},
|
||||
"value": {},
|
||||
"status": {"enum": ["asserted", "unknown", "contested", "withheld"]},
|
||||
"preferred": {"type": "boolean"},
|
||||
"evidenceRefs": {"type": "array", "items": {"type": "string"}, "minItems": 1},
|
||||
"observedAt": {"type": "string", "format": "date"}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -65,6 +65,42 @@
|
|||
"panels": [
|
||||
"summary",
|
||||
"relations",
|
||||
"evidence"
|
||||
"evidence",
|
||||
"claims"
|
||||
],
|
||||
"schemaVersion": "0.1",
|
||||
"factPolicy": "show-all",
|
||||
"relationLabels": {
|
||||
"adapts": {
|
||||
"forward": "Adapts",
|
||||
"inverse": "Adapted by"
|
||||
},
|
||||
"uses": {
|
||||
"forward": "Uses",
|
||||
"inverse": "Used by"
|
||||
},
|
||||
"addressed-by": {
|
||||
"forward": "Addressed by",
|
||||
"inverse": "Addresses"
|
||||
},
|
||||
"related-to": {
|
||||
"forward": "Related to",
|
||||
"inverse": "Related to"
|
||||
},
|
||||
"documents": {
|
||||
"forward": "Documents",
|
||||
"inverse": "Documented by"
|
||||
},
|
||||
"visualizes": {
|
||||
"forward": "Visualizes",
|
||||
"inverse": "Visualized by"
|
||||
},
|
||||
"affects": {
|
||||
"forward": "Affects",
|
||||
"inverse": "Affected by"
|
||||
}
|
||||
},
|
||||
"acyclicPredicates": [
|
||||
"documents"
|
||||
]
|
||||
}
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
"dataspace": {
|
||||
"id": "software-architecture",
|
||||
"title": "Software Architecture",
|
||||
"description": "A small illustrative collection of patterns and ideas in software design"
|
||||
"description": "A small illustrative collection of patterns and ideas in software design. The sentences were written for this demo and are not quotations from another corpus."
|
||||
},
|
||||
"kinds": [
|
||||
{
|
||||
|
|
@ -42,108 +42,153 @@
|
|||
"kindId": "concept",
|
||||
"title": "Coupling",
|
||||
"tldr": "The degree to which software elements depend on one another.",
|
||||
"summary": "The degree to which software elements depend on one another. This fixture is illustrative and does not constitute a sourced scholarly claim.",
|
||||
"tags": [],
|
||||
"summary": "Coupling is how much one software element must know about another in order to work.",
|
||||
"tags": [
|
||||
"dependency"
|
||||
],
|
||||
"sourceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"asOf": "2026-10-12"
|
||||
"asOf": "2026-10-12",
|
||||
"claimRefs": [
|
||||
"claim:coupling:tldr"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:software-architecture:concept:cohesion",
|
||||
"kindId": "concept",
|
||||
"title": "Cohesion",
|
||||
"tldr": "The degree to which a module groups related responsibilities.",
|
||||
"summary": "The degree to which a module groups related responsibilities. This fixture is illustrative and does not constitute a sourced scholarly claim.",
|
||||
"tags": [],
|
||||
"summary": "Cohesion is how closely the responsibilities inside one module belong together.",
|
||||
"tags": [
|
||||
"modularity"
|
||||
],
|
||||
"sourceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"asOf": "2026-10-12"
|
||||
"asOf": "2026-10-12",
|
||||
"claimRefs": [
|
||||
"claim:cohesion:tldr"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:software-architecture:concept:interface",
|
||||
"kindId": "concept",
|
||||
"title": "Interface contract",
|
||||
"tldr": "An agreed description of interaction between components.",
|
||||
"summary": "An agreed description of interaction between components. This fixture is illustrative and does not constitute a sourced scholarly claim.",
|
||||
"tags": [],
|
||||
"summary": "An interface contract names the interactions two parts are allowed to rely on.",
|
||||
"tags": [
|
||||
"boundary"
|
||||
],
|
||||
"sourceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"asOf": "2026-10-12"
|
||||
"asOf": "2026-10-12",
|
||||
"claimRefs": [
|
||||
"claim:interface:tldr"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:software-architecture:pattern:adapter",
|
||||
"kindId": "pattern",
|
||||
"title": "Adapter pattern",
|
||||
"tldr": "An adapter translates between interfaces.",
|
||||
"summary": "An adapter translates between interfaces. This fixture is illustrative and does not constitute a sourced scholarly claim.",
|
||||
"tags": [],
|
||||
"summary": "An adapter lets a caller use one interface while the callee offers another.",
|
||||
"tags": [
|
||||
"translation"
|
||||
],
|
||||
"sourceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"asOf": "2026-10-12"
|
||||
"asOf": "2026-10-12",
|
||||
"claimRefs": [
|
||||
"claim:adapter:tldr"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:software-architecture:pattern:ports-adapters",
|
||||
"kindId": "pattern",
|
||||
"title": "Ports and adapters",
|
||||
"tldr": "A pattern for isolating core logic from outside systems.",
|
||||
"summary": "A pattern for isolating core logic from outside systems. This fixture is illustrative and does not constitute a sourced scholarly claim.",
|
||||
"tags": [],
|
||||
"summary": "Ports and adapters keep domain logic from importing the details of a database, a UI, or a network.",
|
||||
"tags": [
|
||||
"isolation"
|
||||
],
|
||||
"sourceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"asOf": "2026-10-12"
|
||||
"asOf": "2026-10-12",
|
||||
"claimRefs": [
|
||||
"claim:ports-adapters:tldr"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:software-architecture:pattern:event-driven",
|
||||
"kindId": "pattern",
|
||||
"title": "Event-driven architecture",
|
||||
"tldr": "Software components communicate through events.",
|
||||
"summary": "Software components communicate through events. This fixture is illustrative and does not constitute a sourced scholarly claim.",
|
||||
"tags": [],
|
||||
"summary": "In an event-driven arrangement, parts announce facts and other parts react, instead of calling each other directly.",
|
||||
"tags": [
|
||||
"communication"
|
||||
],
|
||||
"sourceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"asOf": "2026-10-12"
|
||||
"asOf": "2026-10-12",
|
||||
"claimRefs": [
|
||||
"claim:event-driven:tldr"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:software-architecture:person:alexander",
|
||||
"kindId": "person",
|
||||
"title": "Christopher Alexander",
|
||||
"tldr": "Architect whose pattern-language ideas influenced software design.",
|
||||
"summary": "Architect whose pattern-language ideas influenced software design. This fixture is illustrative and does not constitute a sourced scholarly claim.",
|
||||
"tags": [],
|
||||
"summary": "Christopher Alexander described pattern languages for buildings. Software designers later borrowed that way of naming recurring forms.",
|
||||
"tags": [
|
||||
"patterns"
|
||||
],
|
||||
"sourceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"asOf": "2026-10-12"
|
||||
"asOf": "2026-10-12",
|
||||
"claimRefs": [
|
||||
"claim:alexander:tldr"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:software-architecture:artifact:adr",
|
||||
"kindId": "artifact",
|
||||
"title": "Architecture Decision Record",
|
||||
"tldr": "A lightweight record of architectural decisions and rationale.",
|
||||
"summary": "A lightweight record of architectural decisions and rationale. This fixture is illustrative and does not constitute a sourced scholarly claim.",
|
||||
"tags": [],
|
||||
"summary": "An architecture decision record states a decision, its context, and the consequences the writers accepted.",
|
||||
"tags": [
|
||||
"record"
|
||||
],
|
||||
"sourceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"asOf": "2026-10-12"
|
||||
"asOf": "2026-10-12",
|
||||
"claimRefs": [
|
||||
"claim:adr:tldr"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:software-architecture:artifact:c4",
|
||||
"kindId": "artifact",
|
||||
"title": "C4 model diagram",
|
||||
"tldr": "A set of views for communicating software architecture.",
|
||||
"summary": "A set of views for communicating software architecture. This fixture is illustrative and does not constitute a sourced scholarly claim.",
|
||||
"tags": [],
|
||||
"summary": "The C4 model offers a small set of diagrams, from system context down to code, so different readers can stop at the level they need.",
|
||||
"tags": [
|
||||
"diagram"
|
||||
],
|
||||
"sourceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"asOf": "2026-10-12"
|
||||
"asOf": "2026-10-12",
|
||||
"claimRefs": [
|
||||
"claim:c4:tldr"
|
||||
]
|
||||
}
|
||||
],
|
||||
"relations": [
|
||||
|
|
@ -210,5 +255,116 @@
|
|||
"src:software-architecture:authored-demo"
|
||||
]
|
||||
}
|
||||
],
|
||||
"claims": [
|
||||
{
|
||||
"id": "claim:coupling:tldr",
|
||||
"subjectId": "urn:ce:software-architecture:concept:coupling",
|
||||
"key": "tldr",
|
||||
"value": "The degree to which software elements depend on one another.",
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
},
|
||||
{
|
||||
"id": "claim:cohesion:tldr",
|
||||
"subjectId": "urn:ce:software-architecture:concept:cohesion",
|
||||
"key": "tldr",
|
||||
"value": "The degree to which a module groups related responsibilities.",
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
},
|
||||
{
|
||||
"id": "claim:interface:tldr",
|
||||
"subjectId": "urn:ce:software-architecture:concept:interface",
|
||||
"key": "tldr",
|
||||
"value": "An agreed description of interaction between components.",
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
},
|
||||
{
|
||||
"id": "claim:adapter:tldr",
|
||||
"subjectId": "urn:ce:software-architecture:pattern:adapter",
|
||||
"key": "tldr",
|
||||
"value": "An adapter translates between interfaces.",
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
},
|
||||
{
|
||||
"id": "claim:ports-adapters:tldr",
|
||||
"subjectId": "urn:ce:software-architecture:pattern:ports-adapters",
|
||||
"key": "tldr",
|
||||
"value": "A pattern for isolating core logic from outside systems.",
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
},
|
||||
{
|
||||
"id": "claim:event-driven:tldr",
|
||||
"subjectId": "urn:ce:software-architecture:pattern:event-driven",
|
||||
"key": "tldr",
|
||||
"value": "Software components communicate through events.",
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
},
|
||||
{
|
||||
"id": "claim:alexander:tldr",
|
||||
"subjectId": "urn:ce:software-architecture:person:alexander",
|
||||
"key": "tldr",
|
||||
"value": "Architect whose pattern-language ideas influenced software design.",
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
},
|
||||
{
|
||||
"id": "claim:adr:tldr",
|
||||
"subjectId": "urn:ce:software-architecture:artifact:adr",
|
||||
"key": "tldr",
|
||||
"value": "A lightweight record of architectural decisions and rationale.",
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
},
|
||||
{
|
||||
"id": "claim:c4:tldr",
|
||||
"subjectId": "urn:ce:software-architecture:artifact:c4",
|
||||
"key": "tldr",
|
||||
"value": "A set of views for communicating software architecture.",
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
},
|
||||
{
|
||||
"id": "claim:unknown:numeric-scale",
|
||||
"subjectId": "urn:ce:software-architecture:concept:coupling",
|
||||
"key": "numeric-scale",
|
||||
"status": "unknown",
|
||||
"evidenceRefs": [
|
||||
"src:software-architecture:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
|
|||
|
|
@ -65,6 +65,45 @@
|
|||
"panels": [
|
||||
"summary",
|
||||
"relations",
|
||||
"evidence"
|
||||
]
|
||||
"evidence",
|
||||
"claims"
|
||||
],
|
||||
"schemaVersion": "0.1",
|
||||
"factPolicy": "show-all",
|
||||
"relationLabels": {
|
||||
"represented-by": {
|
||||
"forward": "Represented by",
|
||||
"inverse": "Represents"
|
||||
},
|
||||
"scopes": {
|
||||
"forward": "Scopes",
|
||||
"inverse": "Scoped by"
|
||||
},
|
||||
"associated-with": {
|
||||
"forward": "Associated with",
|
||||
"inverse": "Associated with"
|
||||
},
|
||||
"authored-by": {
|
||||
"forward": "Authored by",
|
||||
"inverse": "Authored"
|
||||
},
|
||||
"related-to": {
|
||||
"forward": "Related to",
|
||||
"inverse": "Related to"
|
||||
},
|
||||
"models": {
|
||||
"forward": "Models",
|
||||
"inverse": "Modeled by"
|
||||
},
|
||||
"explores": {
|
||||
"forward": "Explores",
|
||||
"inverse": "Explored by"
|
||||
}
|
||||
},
|
||||
"acyclicPredicates": [
|
||||
"authored-by"
|
||||
],
|
||||
"theme": {
|
||||
"accent": "indigo"
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
"dataspace": {
|
||||
"id": "systems-thinking",
|
||||
"title": "Systems Thinking",
|
||||
"description": "A small illustrative collection of ideas in systems theory"
|
||||
"description": "A small illustrative collection of ideas in systems theory. The sentences were written for this demo and are not quotations from another corpus."
|
||||
},
|
||||
"kinds": [
|
||||
{
|
||||
|
|
@ -42,108 +42,153 @@
|
|||
"kindId": "concept",
|
||||
"title": "Feedback",
|
||||
"tldr": "Effects of actions change subsequent behavior.",
|
||||
"summary": "Effects of actions change subsequent behavior. This fixture is illustrative and does not constitute a sourced scholarly claim.",
|
||||
"tags": [],
|
||||
"summary": "A feedback loop sends the result of an action back so the next action can change.",
|
||||
"tags": [
|
||||
"regulation"
|
||||
],
|
||||
"sourceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"asOf": "2026-10-12"
|
||||
"asOf": "2026-10-12",
|
||||
"claimRefs": [
|
||||
"claim:feedback:tldr"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:systems-thinking:concept:emergence",
|
||||
"kindId": "concept",
|
||||
"title": "Emergence",
|
||||
"tldr": "Collective behavior arises through interactions.",
|
||||
"summary": "Collective behavior arises through interactions. This fixture is illustrative and does not constitute a sourced scholarly claim.",
|
||||
"tags": [],
|
||||
"summary": "Emergence is behavior of a whole that is not obvious from one part looked at alone.",
|
||||
"tags": [
|
||||
"structure"
|
||||
],
|
||||
"sourceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"asOf": "2026-10-12"
|
||||
"asOf": "2026-10-12",
|
||||
"claimRefs": [
|
||||
"claim:emergence:tldr"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:systems-thinking:concept:variety",
|
||||
"kindId": "concept",
|
||||
"title": "Requisite variety",
|
||||
"tldr": "Effective regulation requires sufficient response variety.",
|
||||
"summary": "Effective regulation requires sufficient response variety. This fixture is illustrative and does not constitute a sourced scholarly claim.",
|
||||
"tags": [],
|
||||
"summary": "Requisite variety says a regulator needs enough responses to match the situations it faces.",
|
||||
"tags": [
|
||||
"regulation"
|
||||
],
|
||||
"sourceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"asOf": "2026-10-12"
|
||||
"asOf": "2026-10-12",
|
||||
"claimRefs": [
|
||||
"claim:variety:tldr"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:systems-thinking:concept:system-boundary",
|
||||
"kindId": "concept",
|
||||
"title": "System boundary",
|
||||
"tldr": "A distinction determines what is examined as a system.",
|
||||
"summary": "A distinction determines what is examined as a system. This fixture is illustrative and does not constitute a sourced scholarly claim.",
|
||||
"tags": [],
|
||||
"summary": "A system boundary is the distinction that decides which interactions count as inside the system.",
|
||||
"tags": [
|
||||
"boundary"
|
||||
],
|
||||
"sourceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"asOf": "2026-10-12"
|
||||
"asOf": "2026-10-12",
|
||||
"claimRefs": [
|
||||
"claim:system-boundary:tldr"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:systems-thinking:person:ashby",
|
||||
"kindId": "person",
|
||||
"title": "W. Ross Ashby",
|
||||
"tldr": "Cybernetics researcher associated with requisite variety.",
|
||||
"summary": "Cybernetics researcher associated with requisite variety. This fixture is illustrative and does not constitute a sourced scholarly claim.",
|
||||
"tags": [],
|
||||
"summary": "W. Ross Ashby wrote about regulation and variety in mid-century cybernetics.",
|
||||
"tags": [
|
||||
"cybernetics"
|
||||
],
|
||||
"sourceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"asOf": "2026-10-12"
|
||||
"asOf": "2026-10-12",
|
||||
"claimRefs": [
|
||||
"claim:ashby:tldr"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:systems-thinking:person:meadows",
|
||||
"kindId": "person",
|
||||
"title": "Donella Meadows",
|
||||
"tldr": "Systems thinker known for work on leverage points.",
|
||||
"summary": "Systems thinker known for work on leverage points. This fixture is illustrative and does not constitute a sourced scholarly claim.",
|
||||
"tags": [],
|
||||
"summary": "Donella Meadows described places in a system where a small change can have a large effect.",
|
||||
"tags": [
|
||||
"leverage"
|
||||
],
|
||||
"sourceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"asOf": "2026-10-12"
|
||||
"asOf": "2026-10-12",
|
||||
"claimRefs": [
|
||||
"claim:meadows:tldr"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:systems-thinking:publication:thinking-in-systems",
|
||||
"kindId": "publication",
|
||||
"title": "Thinking in Systems",
|
||||
"tldr": "A widely read introductory systems-thinking book.",
|
||||
"summary": "A widely read introductory systems-thinking book. This fixture is illustrative and does not constitute a sourced scholarly claim.",
|
||||
"tags": [],
|
||||
"summary": "Thinking in Systems is an introductory book that explains stocks, flows, and feedback with everyday examples.",
|
||||
"tags": [
|
||||
"introduction"
|
||||
],
|
||||
"sourceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"asOf": "2026-10-12"
|
||||
"asOf": "2026-10-12",
|
||||
"claimRefs": [
|
||||
"claim:thinking-in-systems:tldr"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:systems-thinking:method:causal-loop",
|
||||
"kindId": "method",
|
||||
"title": "Causal loop diagram",
|
||||
"tldr": "A diagram representing causal influences and feedback.",
|
||||
"summary": "A diagram representing causal influences and feedback. This fixture is illustrative and does not constitute a sourced scholarly claim.",
|
||||
"tags": [],
|
||||
"summary": "A causal loop diagram draws influence arrows and marks whether each link reinforces or balances.",
|
||||
"tags": [
|
||||
"diagram"
|
||||
],
|
||||
"sourceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"asOf": "2026-10-12"
|
||||
"asOf": "2026-10-12",
|
||||
"claimRefs": [
|
||||
"claim:causal-loop:tldr"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:systems-thinking:method:stock-flow",
|
||||
"kindId": "method",
|
||||
"title": "Stock and flow model",
|
||||
"tldr": "A representation of accumulations and their rates of change.",
|
||||
"summary": "A representation of accumulations and their rates of change. This fixture is illustrative and does not constitute a sourced scholarly claim.",
|
||||
"tags": [],
|
||||
"summary": "A stock and flow model separates accumulations from the rates that fill or drain them.",
|
||||
"tags": [
|
||||
"diagram"
|
||||
],
|
||||
"sourceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"asOf": "2026-10-12"
|
||||
"asOf": "2026-10-12",
|
||||
"claimRefs": [
|
||||
"claim:stock-flow:tldr"
|
||||
]
|
||||
}
|
||||
],
|
||||
"relations": [
|
||||
|
|
@ -210,5 +255,116 @@
|
|||
"src:systems-thinking:authored-demo"
|
||||
]
|
||||
}
|
||||
],
|
||||
"claims": [
|
||||
{
|
||||
"id": "claim:feedback:tldr",
|
||||
"subjectId": "urn:ce:systems-thinking:concept:feedback",
|
||||
"key": "tldr",
|
||||
"value": "Effects of actions change subsequent behavior.",
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
},
|
||||
{
|
||||
"id": "claim:emergence:tldr",
|
||||
"subjectId": "urn:ce:systems-thinking:concept:emergence",
|
||||
"key": "tldr",
|
||||
"value": "Collective behavior arises through interactions.",
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
},
|
||||
{
|
||||
"id": "claim:variety:tldr",
|
||||
"subjectId": "urn:ce:systems-thinking:concept:variety",
|
||||
"key": "tldr",
|
||||
"value": "Effective regulation requires sufficient response variety.",
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
},
|
||||
{
|
||||
"id": "claim:system-boundary:tldr",
|
||||
"subjectId": "urn:ce:systems-thinking:concept:system-boundary",
|
||||
"key": "tldr",
|
||||
"value": "A distinction determines what is examined as a system.",
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
},
|
||||
{
|
||||
"id": "claim:ashby:tldr",
|
||||
"subjectId": "urn:ce:systems-thinking:person:ashby",
|
||||
"key": "tldr",
|
||||
"value": "Cybernetics researcher associated with requisite variety.",
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
},
|
||||
{
|
||||
"id": "claim:meadows:tldr",
|
||||
"subjectId": "urn:ce:systems-thinking:person:meadows",
|
||||
"key": "tldr",
|
||||
"value": "Systems thinker known for work on leverage points.",
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
},
|
||||
{
|
||||
"id": "claim:thinking-in-systems:tldr",
|
||||
"subjectId": "urn:ce:systems-thinking:publication:thinking-in-systems",
|
||||
"key": "tldr",
|
||||
"value": "A widely read introductory systems-thinking book.",
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
},
|
||||
{
|
||||
"id": "claim:causal-loop:tldr",
|
||||
"subjectId": "urn:ce:systems-thinking:method:causal-loop",
|
||||
"key": "tldr",
|
||||
"value": "A diagram representing causal influences and feedback.",
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
},
|
||||
{
|
||||
"id": "claim:stock-flow:tldr",
|
||||
"subjectId": "urn:ce:systems-thinking:method:stock-flow",
|
||||
"key": "tldr",
|
||||
"value": "A representation of accumulations and their rates of change.",
|
||||
"status": "asserted",
|
||||
"evidenceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
},
|
||||
{
|
||||
"id": "claim:unknown:formal-measure",
|
||||
"subjectId": "urn:ce:systems-thinking:concept:system-boundary",
|
||||
"key": "formal-measure",
|
||||
"status": "unknown",
|
||||
"evidenceRefs": [
|
||||
"src:systems-thinking:authored-demo"
|
||||
],
|
||||
"observedAt": "2026-10-12"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
|
|||
59
history/2026-10-12-1905-scope-against-intent.md
Normal file
59
history/2026-10-12-1905-scope-against-intent.md
Normal file
|
|
@ -0,0 +1,59 @@
|
|||
# SCOPE against INTENT — 2026-10-12 19:05 CEST
|
||||
|
||||
This note compares `SCOPE.md` (the tree at the stop below) with `INTENT.md` (initial intent v0.1, 2026-10-12). It can be read without the session transcript.
|
||||
|
||||
## Stop
|
||||
|
||||
Implementation stopped at **2026-10-12 19:05 CEST**, before 24:00 local. The reason is dependency order, not the clock. CONTENT-WP-0001 and CONTENT-WP-0003 are finished in the workplan files. CONTENT-WP-0002 depends on CONTENT-WP-0000 as well as 0001, and 0000 still has T04 open. CONTENT-WP-0004, 0005, and 0006 depend on plans that have not started. CONTENT-WP-0007 stays blocked on data-use authorization. No further registered plan was eligible.
|
||||
|
||||
CONTENT-WP-0000-T04 stays `todo`. At 18:52 CEST the workstation had no Chrome, Chromium, Firefox, or Playwright launcher (`command -v` found none). Live OnCo page titles were fetched over HTTP during the baseline. No screenshot and no keyboard walkthrough were produced, and T04 was not marked done. The same absence is why the v0.1 browser journey below is unmet. Chromium was not installed during this session.
|
||||
|
||||
## What already matches the mission
|
||||
|
||||
INTENT asks for a maintainable, domain-independent adaptation, with a connector projecting sources into a provenance-aware model and a manifest configuring presentation.
|
||||
|
||||
- Two unrelated authored dataspaces exist: systems thinking and software architecture. Their sentences were written for this demo. They are not an OnCo import. Both pass `python3 scripts/validate_fixtures.py`, including the size gate of at least three kinds and eight entities.
|
||||
- The canonical types Dataspace, Kind, Entity, Relation, Claim, and Source are required by `contracts/snapshot.schema.json` at `schemaVersion` 0.1. Navigation, kind views, panels, relation labels, and fact policy live in `contracts/manifest.schema.json`.
|
||||
- Domain wording is data. `content_explorer/render.py` reads the heading, kind plurals, and relation labels from the snapshot and the manifest. `tests/test_contract.py` switches the manifest while holding the systems snapshot fixed, and checks that the renderer source does not contain the fixture vocabulary.
|
||||
- Fact policy distinguishes missing, unknown, one asserted value, withheld, and two asserted values. `show-all`, `preferred`, and `withhold` are specified in `contracts/fact-policy.md` and covered by tests. Absence is not stored as unknown or false.
|
||||
- Ingestion is replaceable and does not render. `content_explorer/ingest.py` loads local JSON, NDJSON, and Markdown frontmatter (PyYAML). Equivalent records from those three inputs produce one snapshot hash. A second run of the same input repeats that hash. The report carries connector version `0.1.0` and a SHA-256 source digest.
|
||||
- Every published tldr claim has `evidenceRefs` pointing at the source id. The source carries uri, title, license, and attribution.
|
||||
- Dangling relations, duplicate identity, a key that is both live and tombstoned, an unknown kind, a missing relation label, a missing or disallowed license, and CC-BY-4.0 without attribution all refuse publication. Allowed licenses are CC0-1.0, CC-BY-4.0, and MIT. CC BY-NC is quarantined.
|
||||
- Publish writes `snapshot.json`, `manifest.json`, `import-report.json`, and a human `import-report.md` by renaming a completed directory into place.
|
||||
- Reuse limits match INTENT principle 10. The baseline record separates MIT code from CC BY-NC data, logos, and portraits. Those assets are not in this repo. CONTENT-WP-0007 was not advanced.
|
||||
|
||||
## v0.1 proof that is still unmet
|
||||
|
||||
INTENT's exit criteria are stronger than the seed that exists tonight.
|
||||
|
||||
- **No browser UI.** Switching dataspaces is proven for the text renderer, not for React components. There is still no landing page, faceted index, detail page, search, graph view, or route per entity. The browse, filter, entity, neighbor, and source journey has not been walked by mouse or keyboard.
|
||||
- **CONTENT-WP-0000 is partial.** The pin, license matrix, import trace, and reuse decision exist. Screenshots and the keyboard walkthrough do not.
|
||||
- **CONTENT-WP-0002 has not started**, so no OnCo table, facet control, or detail shell has been extracted. The earlier reuse note names ResultsTable, FacetSelect, browser sort, URL state, and a detail shell as later candidates. They were not copied in this session.
|
||||
- **Collection** is named in INTENT's common types and is not a v0.1 schema object. Navigation and view metadata are on the manifest instead.
|
||||
- **The connector is local only.** Remote static files, wiki-style references, refresh checkpoints, and the GitHub or Wikidata connector are not implemented. `connectors/json_fixture.py` only reads a snapshot that already exists.
|
||||
- **Conflicting source values are not merged into a contested claim.** Two records with the same kind and key block publication as an identity conflict. The snapshot schema can store contested claims, and the fact policy can display them, but the connector does not produce that state.
|
||||
- **No two-domain visual review** and no automated UI or end-to-end test. `research/UXParityMatrix.md` is still a research note, not a release-candidate parity result.
|
||||
- **Regression coverage is the Python contract and ingest tests**, not an upstream `next build` or the OnCo test suite.
|
||||
|
||||
## Deferrals
|
||||
|
||||
Left for a later session, in `workplans/README.md` order:
|
||||
|
||||
- Finish CONTENT-WP-0000-T04 when a headless browser is available, then start CONTENT-WP-0002.
|
||||
- CONTENT-WP-0004 (relations and journeys) after 0002. 0003, which it also needs, is finished.
|
||||
- CONTENT-WP-0005 after 0004. Quality measurement was not opened early.
|
||||
- CONTENT-WP-0006 (third connector, MCP is out of scope until then) after 0005.
|
||||
- CONTENT-WP-0007 until a person authorizes OnCo data use. Do not vendor `src/data`, `public/logos`, or `public/portraits`.
|
||||
|
||||
## Status the files claim
|
||||
|
||||
| Workplan | File status | SCOPE verdict |
|
||||
|---|---|---|
|
||||
| CONTENT-WP-0000 | active | partial |
|
||||
| CONTENT-WP-0001 | finished | finished for the contract and text renderer, not for a browser proof |
|
||||
| CONTENT-WP-0002 | proposed | not started |
|
||||
| CONTENT-WP-0003 | finished | finished for the local connector described in its evidence section |
|
||||
| CONTENT-WP-0004 | proposed | not started |
|
||||
| CONTENT-WP-0005 | proposed | not started |
|
||||
| CONTENT-WP-0006 | proposed | not started |
|
||||
| CONTENT-WP-0007 | blocked | blocked, not finished |
|
||||
|
|
@ -100,4 +100,4 @@ The detail page's first sections are Overview, State of the art, Anatomy and lym
|
|||
- R2: default-heap `tsc` ran out of memory. The same command passed with a 6 GB heap.
|
||||
- R2: production `next build` and the full test and lint runs were not executed.
|
||||
- R1: `NOTICE` is absent. Preservation of the MIT notice applies when a file is copied, which this pass does not do.
|
||||
- T04 screenshots and the keyboard walkthrough are not done.
|
||||
- T04 screenshots and the keyboard walkthrough are not done. At 18:52 CEST on 2026-10-12 the workstation had no Chrome, Chromium, Firefox, or Playwright launcher (`command -v` found none). The task stays open. Live page titles were fetched over HTTP only.
|
||||
|
|
|
|||
|
|
@ -1,27 +1,33 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Portable smoke validator: structural and cross-reference checks; not full JSON Schema compliance."""
|
||||
"""Validate every example dataspace against the v0.1 contract."""
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
import json, sys
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
errors=[]
|
||||
for path in sorted((ROOT/'examples/dataspaces').glob('*/snapshot.json')):
|
||||
d=json.loads(path.read_text())
|
||||
kinds={r['id'] for r in d['kinds']}; ids=[r['id'] for r in d['entities']]
|
||||
sources={r['id'] for r in d['sources']}
|
||||
if d['schemaVersion']!='0.1': errors.append(f'{path}: schemaVersion')
|
||||
if len(ids)!=len(set(ids)): errors.append(f'{path}: duplicate entities')
|
||||
for e in d['entities']:
|
||||
if e['kindId'] not in kinds: errors.append(f'{path}: undeclared kind {e["kindId"]}')
|
||||
for sid in e.get('sourceRefs',[]):
|
||||
if sid not in sources: errors.append(f'{path}: missing source {sid}')
|
||||
for r in d['relations']:
|
||||
for field in ('from','to'):
|
||||
if r[field] not in ids: errors.append(f'{path}: dangling {field} {r[field]}')
|
||||
for sid in r.get('sourceRefs',[]):
|
||||
if sid not in sources: errors.append(f'{path}: missing edge source {sid}')
|
||||
manifest=json.loads((path.parent/'manifest.json').read_text())
|
||||
if manifest['id']!=d['dataspace']['id']: errors.append(f'{path}: manifest ID mismatch')
|
||||
print(f'{d["dataspace"]["id"]}: {len(ids)} entities, {len(d["relations"])} relations, {len(kinds)} kinds')
|
||||
if errors:
|
||||
print('\n'.join(errors),file=sys.stderr);sys.exit(1)
|
||||
print('Smoke validation OK (full JSON Schema and claims pending).')
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from content_explorer.contract import validate_dataspace # noqa: E402
|
||||
|
||||
def main() -> int:
|
||||
errors: list[str] = []
|
||||
directories = sorted((ROOT / "examples" / "dataspaces").glob("*/snapshot.json"))
|
||||
if not directories:
|
||||
print("no dataspace fixtures found", file=sys.stderr)
|
||||
return 1
|
||||
for snapshot_path in directories:
|
||||
directory = snapshot_path.parent
|
||||
found = validate_dataspace(directory, require_size=True)
|
||||
errors.extend(found)
|
||||
if not found:
|
||||
print(f"{directory.name}: contract ok")
|
||||
if errors:
|
||||
print("\n".join(errors), file=sys.stderr)
|
||||
return 1
|
||||
print("Fixture validation OK")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
|
|
|
|||
147
tests/test_contract.py
Normal file
147
tests/test_contract.py
Normal file
|
|
@ -0,0 +1,147 @@
|
|||
"""Contract checks against the shipped validator, not a copy of it."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import json
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from content_explorer.contract import ( # noqa: E402
|
||||
resolve_claims,
|
||||
validate_dataspace,
|
||||
validate_documents,
|
||||
)
|
||||
from content_explorer.render import render_explorer # noqa: E402
|
||||
|
||||
EXAMPLES = ROOT / "examples" / "dataspaces"
|
||||
RENDERER = ROOT / "content_explorer" / "render.py"
|
||||
|
||||
|
||||
def load_pair(name: str) -> tuple[dict, dict]:
|
||||
directory = EXAMPLES / name
|
||||
snapshot = json.loads((directory / "snapshot.json").read_text(encoding="utf-8"))
|
||||
manifest = json.loads((directory / "manifest.json").read_text(encoding="utf-8"))
|
||||
return snapshot, manifest
|
||||
|
||||
|
||||
class PublishedFixtures(unittest.TestCase):
|
||||
def test_each_example_passes_the_file_entry_point(self) -> None:
|
||||
names = sorted(path.name for path in EXAMPLES.iterdir() if (path / "snapshot.json").is_file())
|
||||
self.assertGreaterEqual(len(names), 2)
|
||||
for name in names:
|
||||
errors = validate_dataspace(EXAMPLES / name, require_size=True)
|
||||
self.assertEqual(errors, [], name)
|
||||
|
||||
def test_renderer_switches_with_the_manifest(self) -> None:
|
||||
systems, systems_manifest = load_pair("systems-thinking")
|
||||
software, software_manifest = load_pair("software-architecture")
|
||||
systems_view = render_explorer(systems, systems_manifest)
|
||||
software_view = render_explorer(software, software_manifest)
|
||||
switched = render_explorer(systems, software_manifest)
|
||||
self.assertIn(systems_manifest["home"]["heading"], systems_view)
|
||||
self.assertIn(software_manifest["home"]["heading"], software_view)
|
||||
self.assertIn(software_manifest["home"]["heading"], switched)
|
||||
self.assertNotIn(systems_manifest["home"]["heading"], switched)
|
||||
self.assertNotIn(software_manifest["home"]["heading"], systems_view)
|
||||
|
||||
def test_renderer_source_has_no_domain_vocabulary(self) -> None:
|
||||
source = RENDERER.read_text(encoding="utf-8").lower()
|
||||
for word in (
|
||||
"cancer",
|
||||
"oncolog",
|
||||
"feedback",
|
||||
"coupling",
|
||||
"ashby",
|
||||
"meadows",
|
||||
"alexander",
|
||||
"systems-thinking",
|
||||
"software-architecture",
|
||||
"causal",
|
||||
"cohesion",
|
||||
):
|
||||
self.assertNotIn(word, source, word)
|
||||
|
||||
|
||||
class NegativeGraph(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.snapshot, self.manifest = load_pair("systems-thinking")
|
||||
|
||||
def test_duplicate_entity_id(self) -> None:
|
||||
snapshot = copy.deepcopy(self.snapshot)
|
||||
snapshot["entities"].append(copy.deepcopy(snapshot["entities"][0]))
|
||||
errors = validate_documents(snapshot, self.manifest)
|
||||
self.assertTrue(any("duplicate entity id" in error for error in errors))
|
||||
|
||||
def test_dangling_relation(self) -> None:
|
||||
snapshot = copy.deepcopy(self.snapshot)
|
||||
snapshot["relations"][0]["to"] = "urn:ce:systems-thinking:concept:missing"
|
||||
errors = validate_documents(snapshot, self.manifest)
|
||||
self.assertTrue(any("dangling to" in error for error in errors))
|
||||
|
||||
def test_missing_source(self) -> None:
|
||||
snapshot = copy.deepcopy(self.snapshot)
|
||||
snapshot["entities"][0]["sourceRefs"] = ["src:missing"]
|
||||
errors = validate_documents(snapshot, self.manifest)
|
||||
self.assertTrue(any("missing source src:missing" in error for error in errors))
|
||||
|
||||
def test_taxonomic_cycle_fails_and_other_cycles_do_not(self) -> None:
|
||||
snapshot = copy.deepcopy(self.snapshot)
|
||||
left = snapshot["entities"][0]["id"]
|
||||
right = snapshot["entities"][1]["id"]
|
||||
source = snapshot["sources"][0]["id"]
|
||||
snapshot["relations"].extend([
|
||||
{
|
||||
"id": "cycle-a",
|
||||
"predicate": "related-to",
|
||||
"from": left,
|
||||
"to": right,
|
||||
"sourceRefs": [source],
|
||||
},
|
||||
{
|
||||
"id": "cycle-b",
|
||||
"predicate": "related-to",
|
||||
"from": right,
|
||||
"to": left,
|
||||
"sourceRefs": [source],
|
||||
},
|
||||
])
|
||||
manifest = copy.deepcopy(self.manifest)
|
||||
manifest["relationLabels"]["related-to"] = {"forward": "Related to", "inverse": "Related to"}
|
||||
manifest["acyclicPredicates"] = []
|
||||
open_cycle = validate_documents(snapshot, manifest)
|
||||
self.assertFalse(any("taxonomic cycle" in error for error in open_cycle))
|
||||
manifest["acyclicPredicates"] = ["related-to"]
|
||||
closed = validate_documents(snapshot, manifest)
|
||||
self.assertTrue(any("taxonomic cycle" in error for error in closed))
|
||||
|
||||
|
||||
class FactPolicy(unittest.TestCase):
|
||||
def test_missing_unknown_and_conflict(self) -> None:
|
||||
claims = [
|
||||
{"key": "measure", "status": "unknown", "value": None},
|
||||
{"key": "name", "status": "asserted", "value": "one"},
|
||||
{"key": "name", "status": "asserted", "value": "two"},
|
||||
]
|
||||
self.assertEqual(resolve_claims(claims, "absent", "show-all")["state"], "missing")
|
||||
self.assertEqual(resolve_claims(claims, "measure", "show-all")["state"], "unknown")
|
||||
shown = resolve_claims(claims, "name", "show-all")
|
||||
self.assertEqual(shown["state"], "contradicted")
|
||||
self.assertEqual(shown["values"], ["one", "two"])
|
||||
withheld = resolve_claims(claims, "name", "withhold")
|
||||
self.assertEqual(withheld["values"], [])
|
||||
preferred_claims = [
|
||||
{"key": "name", "status": "asserted", "value": "one", "preferred": True},
|
||||
{"key": "name", "status": "asserted", "value": "two"},
|
||||
]
|
||||
chosen = resolve_claims(preferred_claims, "name", "preferred")
|
||||
self.assertEqual(chosen, {"state": "present", "values": ["one"]})
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
185
tests/test_ingest.py
Normal file
185
tests/test_ingest.py
Normal file
|
|
@ -0,0 +1,185 @@
|
|||
"""Ingestion gates for JSON, NDJSON, and Markdown frontmatter."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
import sys
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from content_explorer.contract import validate_documents # noqa: E402
|
||||
from content_explorer.ingest import ( # noqa: E402
|
||||
load_json_records,
|
||||
load_markdown_records,
|
||||
load_ndjson_records,
|
||||
project_records,
|
||||
publish_directory,
|
||||
)
|
||||
|
||||
LABELS = {"represented-by": {"forward": "Represented by", "inverse": "Represents"}}
|
||||
|
||||
KINDS = [
|
||||
{"id": "concept", "label": "Concept", "plural": "Concepts"},
|
||||
{"id": "method", "label": "Method", "plural": "Methods"},
|
||||
]
|
||||
SOURCE = {
|
||||
"id": "src:demo:notes",
|
||||
"uri": "https://example.invalid/content-explorer/ingest",
|
||||
"title": "Demo notes",
|
||||
"license": "CC0-1.0",
|
||||
"attribution": "Content Explorer demo",
|
||||
}
|
||||
RECORDS = [
|
||||
{
|
||||
"kind": "concept",
|
||||
"key": "feedback",
|
||||
"title": "Feedback",
|
||||
"tldr": "Results change the next action.",
|
||||
"summary": "A loop carries an outcome back to the actor.",
|
||||
"tags": ["regulation"],
|
||||
"relations": [{"predicate": "represented-by", "to": "method:loop-diagram"}],
|
||||
},
|
||||
{
|
||||
"kind": "method",
|
||||
"key": "loop-diagram",
|
||||
"title": "Loop diagram",
|
||||
"tldr": "Arrows that show influence.",
|
||||
"summary": "A drawing of influences among parts.",
|
||||
"tags": ["diagram"],
|
||||
"relations": [],
|
||||
},
|
||||
]
|
||||
def project(records, source=SOURCE, relation_labels=None):
|
||||
return project_records(
|
||||
records,
|
||||
dataspace_id="demo-notes",
|
||||
title="Demo notes",
|
||||
description="A connector test collection.",
|
||||
source=source,
|
||||
kinds=KINDS,
|
||||
observed_at="2026-10-12",
|
||||
relation_labels=LABELS if relation_labels is None else relation_labels,
|
||||
)
|
||||
|
||||
|
||||
class IngestGates(unittest.TestCase):
|
||||
def test_json_ndjson_and_markdown_match_and_repeat(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
(root / "records.json").write_text(json.dumps(RECORDS), encoding="utf-8")
|
||||
(root / "records.ndjson").write_text(
|
||||
"\n".join(json.dumps(record) for record in RECORDS) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
docs = root / "notes"
|
||||
docs.mkdir()
|
||||
for record in RECORDS:
|
||||
body = "---\n" + _yaml(record) + "---\n\n" + record["summary"] + "\n"
|
||||
(docs / f"{record['key']}.md").write_text(body, encoding="utf-8")
|
||||
json_records, json_digest = load_json_records(root / "records.json")
|
||||
ndjson_records, _ = load_ndjson_records(root / "records.ndjson")
|
||||
markdown_records, _ = load_markdown_records(docs)
|
||||
first = project_records(
|
||||
json_records,
|
||||
dataspace_id="demo-notes",
|
||||
title="Demo notes",
|
||||
description="A connector test collection.",
|
||||
source=SOURCE,
|
||||
kinds=KINDS,
|
||||
observed_at="2026-10-12",
|
||||
source_digest=json_digest,
|
||||
relation_labels=LABELS,
|
||||
)
|
||||
self.assertEqual(first["report"]["sourceDigest"], json_digest)
|
||||
second = project(ndjson_records)
|
||||
third = project(markdown_records)
|
||||
again = project(json_records)
|
||||
self.assertTrue(first["report"]["published"])
|
||||
self.assertEqual(first["report"]["contentHash"], second["report"]["contentHash"])
|
||||
self.assertEqual(first["report"]["contentHash"], third["report"]["contentHash"])
|
||||
self.assertEqual(first["report"]["contentHash"], again["report"]["contentHash"])
|
||||
self.assertEqual(first["snapshot"], third["snapshot"])
|
||||
for claim in first["snapshot"]["claims"]:
|
||||
self.assertTrue(claim["evidenceRefs"])
|
||||
errors = validate_documents(first["snapshot"], first["manifest"], require_size=False)
|
||||
self.assertEqual(errors, [])
|
||||
self.assertEqual(first["manifest"]["id"], "demo-notes")
|
||||
self.assertIn("Represented by", first["manifest"]["relationLabels"]["represented-by"]["forward"])
|
||||
|
||||
def test_rights_and_dangling_refs_do_not_publish(self) -> None:
|
||||
blocked = project(RECORDS, source={**SOURCE, "license": "CC-BY-NC-4.0"})
|
||||
self.assertFalse(blocked["report"]["published"])
|
||||
self.assertIsNone(blocked["snapshot"])
|
||||
self.assertEqual(blocked["report"]["quarantined"][0]["reason"], "rights")
|
||||
dangling = project([
|
||||
{
|
||||
**RECORDS[0],
|
||||
"relations": [{"predicate": "represented-by", "to": "method:missing"}],
|
||||
}
|
||||
])
|
||||
self.assertFalse(dangling["report"]["published"])
|
||||
self.assertTrue(dangling["report"]["dangling"])
|
||||
missing_credit = project(RECORDS, source={**SOURCE, "license": "CC-BY-4.0", "attribution": ""})
|
||||
self.assertFalse(missing_credit["report"]["published"])
|
||||
self.assertEqual(missing_credit["report"]["quarantined"][0]["detail"], "missing-attribution")
|
||||
duplicate = project(RECORDS + [{**RECORDS[0], "title": "Other feedback"}])
|
||||
self.assertFalse(duplicate["report"]["published"])
|
||||
self.assertEqual(duplicate["report"]["conflicts"][0]["reason"], "duplicate-identity")
|
||||
unlabeled = project(RECORDS, relation_labels={})
|
||||
self.assertFalse(unlabeled["report"]["published"])
|
||||
self.assertEqual(unlabeled["report"]["quarantined"][0]["reason"], "missing-relation-label")
|
||||
|
||||
def test_tombstone_is_omitted_and_publish_writes_both_files(self) -> None:
|
||||
retired = project(RECORDS + [{
|
||||
"kind": "concept",
|
||||
"key": "retired",
|
||||
"title": "Retired",
|
||||
"tldr": "Gone.",
|
||||
"deleted": True,
|
||||
}])
|
||||
self.assertEqual(retired["report"]["tombstoned"], ["retired"])
|
||||
self.assertEqual(len(retired["snapshot"]["entities"]), 2)
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
destination = Path(tmp) / "out"
|
||||
publish_directory(destination, retired)
|
||||
self.assertTrue((destination / "snapshot.json").is_file())
|
||||
self.assertTrue((destination / "manifest.json").is_file())
|
||||
self.assertTrue((destination / "import-report.json").is_file())
|
||||
human = (destination / "import-report.md").read_text(encoding="utf-8")
|
||||
self.assertIn("Connector version: 0.1.0", human)
|
||||
self.assertIn("License: CC0-1.0", human)
|
||||
self.assertIn("retired", human)
|
||||
published = json.loads((destination / "snapshot.json").read_text(encoding="utf-8"))
|
||||
manifest = json.loads((destination / "manifest.json").read_text(encoding="utf-8"))
|
||||
self.assertEqual(validate_documents(published, manifest, require_size=False), [])
|
||||
with self.assertRaises(RuntimeError):
|
||||
publish_directory(destination, project(RECORDS, source={**SOURCE, "license": ""}))
|
||||
|
||||
|
||||
def _yaml(record: dict) -> str:
|
||||
lines = [
|
||||
f"kind: {record['kind']}",
|
||||
f"key: {record['key']}",
|
||||
f"title: {record['title']}",
|
||||
f"tldr: {record['tldr']}",
|
||||
f"summary: {record['summary']}",
|
||||
"tags:",
|
||||
*[f" - {tag}" for tag in record["tags"]],
|
||||
"relations:",
|
||||
]
|
||||
if record["relations"]:
|
||||
for link in record["relations"]:
|
||||
lines.append(f" - predicate: {link['predicate']}")
|
||||
lines.append(f" to: {link['to']}")
|
||||
else:
|
||||
lines.append(" []")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -4,7 +4,7 @@ type: workplan
|
|||
title: "Canonical contracts and two-domain fixtures"
|
||||
domain: infotech
|
||||
repo: content-explorer
|
||||
status: ready
|
||||
status: finished
|
||||
flavor: implementation
|
||||
depends_on: []
|
||||
created: "2026-10-12"
|
||||
|
|
@ -14,7 +14,7 @@ state_hub_workstream_id: "833f4e24-1529-528f-8cc7-6caeeabc555f"
|
|||
|
||||
# Canonical contracts and two-domain fixtures
|
||||
|
||||
Establish a source-independent knowledge projection and the zero-UI-code-change experiment. The seed already contains a draft snapshot schema, two authored dataspaces, and a structural smoke validator. Those artifacts do not yet meet the gates below.
|
||||
Establish a source-independent knowledge projection and the zero-UI-code-change experiment. The v0.1 schema, both fixtures, the negative checks, and the shared text renderer meet the gates below.
|
||||
|
||||
Coordinate with CONTENT-WP-0000. Neither plan blocks the other.
|
||||
|
||||
|
|
@ -24,7 +24,7 @@ Coordinate with CONTENT-WP-0000. Neither plan blocks the other.
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0001-T01
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "c3de4ad2-7fae-51bc-9285-85d5cb2e5908"
|
||||
```
|
||||
|
|
@ -33,7 +33,7 @@ state_hub_task_id: "c3de4ad2-7fae-51bc-9285-85d5cb2e5908"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0001-T02
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "501807a7-f778-5799-88d6-e15bbdc42f3d"
|
||||
```
|
||||
|
|
@ -42,7 +42,7 @@ state_hub_task_id: "501807a7-f778-5799-88d6-e15bbdc42f3d"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0001-T03
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "f0e8a472-044e-5dfb-930f-d1afd0bcdd8d"
|
||||
```
|
||||
|
|
@ -51,7 +51,7 @@ state_hub_task_id: "f0e8a472-044e-5dfb-930f-d1afd0bcdd8d"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0001-T04
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "ec494de4-e634-5e8d-ab9e-cbb87f7d0ac1"
|
||||
```
|
||||
|
|
@ -60,7 +60,7 @@ state_hub_task_id: "ec494de4-e634-5e8d-ab9e-cbb87f7d0ac1"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0001-T05
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "8fc0c024-f179-55e0-a70d-173c94908cdd"
|
||||
```
|
||||
|
|
@ -71,7 +71,7 @@ state_hub_task_id: "8fc0c024-f179-55e0-a70d-173c94908cdd"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0001-T06
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "b0fbdf3b-f0cb-5a6e-bb29-593ee6c822c3"
|
||||
```
|
||||
|
|
@ -80,7 +80,7 @@ state_hub_task_id: "b0fbdf3b-f0cb-5a6e-bb29-593ee6c822c3"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0001-T07
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "7c2fd15b-60fa-5e7c-8b2b-8d3d6a22335e"
|
||||
```
|
||||
|
|
@ -89,7 +89,7 @@ state_hub_task_id: "7c2fd15b-60fa-5e7c-8b2b-8d3d6a22335e"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0001-T08
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "1847aa13-3369-577d-925d-3b08dcb5cb1a"
|
||||
```
|
||||
|
|
@ -100,4 +100,4 @@ R3: abstractions erase domain semantics. Log evidence and deviations in the work
|
|||
|
||||
## Evidence
|
||||
|
||||
Pull requests, test logs, pinned artifacts, screenshots, or a comparison report where applicable. Add `ImplementationNotes.md` and `Review.md` on completion. Do not mark a gate passed without a verifiable link.
|
||||
Evidence: `contracts/snapshot.schema.json`, `contracts/manifest.schema.json`, `contracts/SNAPSHOT.md`, `contracts/MIGRATIONS.md`, `contracts/fact-policy.md`, `content_explorer/contract.py`, `content_explorer/render.py`, and `tests/test_contract.py`. `python3 -m unittest tests.test_contract` and `python3 scripts/validate_fixtures.py` both pass. The shared renderer is text, not a browser UI. It reads headings and labels only from the manifest and snapshot.
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ type: workplan
|
|||
title: "Ingestion, provenance, and publishing"
|
||||
domain: infotech
|
||||
repo: content-explorer
|
||||
status: proposed
|
||||
status: finished
|
||||
flavor: implementation
|
||||
depends_on:
|
||||
- CONTENT-WP-0001
|
||||
|
|
@ -17,13 +17,15 @@ state_hub_workstream_id: "fd95d654-8696-5994-9643-bb45925963a6"
|
|||
|
||||
Turn raw inputs into trustworthy, reproducible snapshots. This gate is required before any externally ingested public content is published. It may proceed in parallel with CONTENT-WP-0002.
|
||||
|
||||
The v0.1 connector in `content_explorer/ingest.py` loads JSON, NDJSON, and Markdown frontmatter, resolves links in a second pass, and publishes a snapshot, a manifest, and two import reports only when rights, identity, and references allow it.
|
||||
|
||||
## Work packages
|
||||
|
||||
### Implement build-time JSON, NDJSON, and Markdown frontmatter connector APIs.
|
||||
|
||||
```task
|
||||
id: CONTENT-WP-0003-T01
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "f9f8c59e-4bff-5dd4-82bb-c9f8985fd924"
|
||||
```
|
||||
|
|
@ -32,7 +34,7 @@ state_hub_task_id: "f9f8c59e-4bff-5dd4-82bb-c9f8985fd924"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0003-T02
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "406ae997-7b2c-5a8d-8009-a6ab53e37f4d"
|
||||
```
|
||||
|
|
@ -41,7 +43,7 @@ state_hub_task_id: "406ae997-7b2c-5a8d-8009-a6ab53e37f4d"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0003-T03
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "074caaf6-0501-5e02-beb3-f2c71e492f7a"
|
||||
```
|
||||
|
|
@ -50,7 +52,7 @@ state_hub_task_id: "074caaf6-0501-5e02-beb3-f2c71e492f7a"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0003-T04
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "0ad91201-4a1c-58b5-851f-b3e19a7eabee"
|
||||
```
|
||||
|
|
@ -59,7 +61,7 @@ state_hub_task_id: "0ad91201-4a1c-58b5-851f-b3e19a7eabee"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0003-T05
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "1ec567b1-2efd-558a-b790-e101b9087832"
|
||||
```
|
||||
|
|
@ -70,7 +72,7 @@ state_hub_task_id: "1ec567b1-2efd-558a-b790-e101b9087832"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0003-T06
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "76c69f07-3b7b-55f7-8556-3be717259348"
|
||||
```
|
||||
|
|
@ -79,7 +81,7 @@ state_hub_task_id: "76c69f07-3b7b-55f7-8556-3be717259348"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0003-T07
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "6fe89d04-4018-5491-85b6-76e26cb0ba4d"
|
||||
```
|
||||
|
|
@ -88,7 +90,7 @@ state_hub_task_id: "6fe89d04-4018-5491-85b6-76e26cb0ba4d"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0003-T08
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "d3988379-ee41-53ee-bb72-4f47c9fa37ef"
|
||||
```
|
||||
|
|
@ -97,7 +99,7 @@ state_hub_task_id: "d3988379-ee41-53ee-bb72-4f47c9fa37ef"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0003-T09
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "24ad02ea-75f5-5238-979c-c283072d85cb"
|
||||
```
|
||||
|
|
@ -108,4 +110,4 @@ R5: rights metadata missing. Quarantine instead of guessing. Log evidence and de
|
|||
|
||||
## Evidence
|
||||
|
||||
Pull requests, test logs, pinned artifacts, screenshots, or a comparison report where applicable. Add `ImplementationNotes.md` and `Review.md` on completion. Do not mark a gate passed without a verifiable link.
|
||||
Evidence: `content_explorer/ingest.py` and `tests/test_ingest.py`. `python3 -m unittest tests.test_ingest` passes. JSON, NDJSON, and Markdown frontmatter projections share one content hash (`CONTENT-WP-0003-T01`, `T06`, `T07`). The report carries `connectorVersion` `0.1.0` and the source digest (`T02`). Dangling targets, duplicate keys, and a key that is both live and tombstoned block publication (`T03`). Allowed licenses are `CC0-1.0`, `CC-BY-4.0`, and `MIT`; `CC-BY-NC-4.0`, a missing license, and `CC-BY-4.0` without attribution quarantine and do not publish (`T04`, `T09`). `publish_directory` writes `snapshot.json`, `manifest.json`, `import-report.json`, and `import-report.md` by renaming a completed directory into place (`T05`). Every published tldr claim has `evidenceRefs` (`T08`). Notes: `workplans/evidence/CONTENT-WP-0003-ImplementationNotes.md` and `workplans/evidence/CONTENT-WP-0003-Review.md`. The connector does not fetch remote files and does not render.
|
||||
|
|
|
|||
13
workplans/evidence/CONTENT-WP-0003-ImplementationNotes.md
Normal file
13
workplans/evidence/CONTENT-WP-0003-ImplementationNotes.md
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
# CONTENT-WP-0003 implementation notes
|
||||
|
||||
Date: 2026-10-12
|
||||
|
||||
The build-time connector is `content_explorer/ingest.py`, version `0.1.0`. It reads a JSON list or `{records}` object, NDJSON, or a directory of Markdown files with YAML frontmatter. PyYAML parses the frontmatter. The three loaders return the same record shape plus a SHA-256 digest of the bytes they read.
|
||||
|
||||
Projection collects live keys first, then resolves relations against that set. A target that is not an accepted key is dangling. Two live records with the same kind and key, or a key that is both tombstoned and live, are identity conflicts. Any dangling reference, identity conflict, unknown kind, or missing relation label leaves `snapshot` null and `published` false.
|
||||
|
||||
Publication allows only `CC0-1.0`, `CC-BY-4.0`, and `MIT`. `CC-BY-4.0` also requires a non-empty attribution. Other licenses, including `CC-BY-NC-4.0` and a missing license, are quarantined. The connector does not guess a license.
|
||||
|
||||
Each live entity gets one asserted `tldr` claim whose `evidenceRefs` is the source id. `publish_directory` writes `snapshot.json`, `manifest.json`, `import-report.json`, and `import-report.md` into a temporary directory and renames it into place. A quarantined result raises `RuntimeError` and does not replace an existing directory.
|
||||
|
||||
The manifest is generated from the caller-supplied kind list and relation labels. Predicate wording is not invented. `connectors/json_fixture.py` still only reads a snapshot that already exists. Remote fetch, pagination, and a third connector are not part of this module.
|
||||
16
workplans/evidence/CONTENT-WP-0003-Review.md
Normal file
16
workplans/evidence/CONTENT-WP-0003-Review.md
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
# CONTENT-WP-0003 review
|
||||
|
||||
Date: 2026-10-12
|
||||
|
||||
Risk R5 was rights metadata missing. The connector quarantines a missing license, a license outside `CC0-1.0`, `CC-BY-4.0`, and `MIT`, and `CC-BY-4.0` without attribution. It does not invent attribution or a rights statement.
|
||||
|
||||
Checked against `tests/test_ingest.py`:
|
||||
|
||||
- JSON, NDJSON, and Markdown projections of the same records share one content hash, including a second JSON run.
|
||||
- The report records connector version `0.1.0` and the source digest.
|
||||
- A dangling target, a duplicate key, and a missing relation label do not publish.
|
||||
- A tombstoned key is listed and omitted from entities.
|
||||
- `publish_directory` writes the snapshot, manifest, machine report, and human report. The written snapshot and manifest pass `validate_documents`.
|
||||
- A rights failure raises before replacing the destination.
|
||||
|
||||
Not claimed: remote URLs, conflict merging into a contested claim, or a browser view of the published directory. Those stay with later workplans. CONTENT-WP-0007 was not used as a source.
|
||||
Loading…
Add table
Add a link
Reference in a new issue