Add a synthetic quality suite and upstream parity record.
Some checks failed
synthetic / suite (push) Has been cancelled
Some checks failed
synthetic / suite (push) Has been cancelled
Assistant: grok Assistant-Session: 01a1531b-6698-7732-8587-f28ea604bcdf
This commit is contained in:
parent
b5b6f6bc00
commit
f43edc1edc
14 changed files with 753 additions and 11 deletions
31
.forgejo/workflows/synthetic.yaml
Normal file
31
.forgejo/workflows/synthetic.yaml
Normal file
|
|
@ -0,0 +1,31 @@
|
|||
# Public synthetic suite. Do not fetch the OnCo corpus or its data license.
|
||||
name: synthetic
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
pull_request:
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
suite:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
- name: Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- name: Node
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "22"
|
||||
- name: Python dependencies
|
||||
run: python3 -m pip install -r requirements.txt
|
||||
- name: UI dependencies
|
||||
working-directory: ui
|
||||
run: npm ci
|
||||
- name: Schema, ingest, UI, journeys, and notices
|
||||
run: sh scripts/ci-synthetic.sh
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
# Third-party materials and licensing
|
||||
|
||||
This seed does **not** contain the OnCo corpus or OnCo oncology data. `ui/` adapts a reduced set of OnCo interface tokens and components from commit `b594d7433bf4e408b44215e25e5304696d6f840e`. Those portions stay under the MIT License. The copyright and permission notice is in `ui/NOTICE`, and each adapted file names the pin. The OnCo mark, garden art, and `src/data` were not copied. Check `LICENSE-DATA`: OnCo corpus is CC BY-NC 4.0 and is not automatically eligible for commercial product reuse. Independently clear artworks, images, logos and third-party source records. The independently authored illustrative sample descriptions are offered as CC0-1.0. The project must choose and add a definitive own-source LICENSE after review; upstream licensing does not by itself license this newly written research package.
|
||||
This seed does **not** contain the OnCo corpus or OnCo oncology data. `ui/` adapts a reduced set of OnCo interface tokens and components from commit `b594d7433bf4e408b44215e25e5304696d6f840e`. Those portions stay under the MIT License. The notice is `Copyright (c) 2026 OnCo`. The copyright and permission notice is in `ui/NOTICE`, and each adapted file names the pin. The OnCo mark, garden art, and `src/data` were not copied. Check `LICENSE-DATA`: OnCo corpus is CC BY-NC 4.0 and is not automatically eligible for commercial product reuse. Independently clear artworks, images, logos and third-party source records. The independently authored illustrative sample descriptions are offered as CC0-1.0. The project must choose and add a definitive own-source LICENSE after review; upstream licensing does not by itself license this newly written research package.
|
||||
|
||||
**Reference testing rule:** OnCo `LICENSE-DATA` explicitly treats a company internal data pipeline as commercial use requiring permission. A small test slice is not automatically exempt. See `tests/onco-reference/README.md` and keep actual records out of public CI and redistribution without appropriate rights.
|
||||
|
|
|
|||
228
content_explorer/quality.py
Normal file
228
content_explorer/quality.py
Normal file
|
|
@ -0,0 +1,228 @@
|
|||
"""Quality counts for a published snapshot and an ingest report.
|
||||
|
||||
The counts come from the documents passed in. A failure class is present even
|
||||
when its count is zero, so a missing license, a dangling reference, or a
|
||||
duplicate identity cannot disappear by omission.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from content_explorer.ingest import ALLOWED_LICENSES
|
||||
|
||||
FAILURE_CLASSES = (
|
||||
"rights",
|
||||
"missing-license",
|
||||
"dangling-reference",
|
||||
"duplicate-identity",
|
||||
"unresolved-reference",
|
||||
"missing-relation-label",
|
||||
"unknown-kind",
|
||||
"stale",
|
||||
)
|
||||
|
||||
|
||||
def _failure(kind: str, item_id: str, detail: str) -> dict[str, str]:
|
||||
return {"class": kind, "id": item_id, "detail": detail}
|
||||
|
||||
|
||||
def _days_after(earlier: str, later: str) -> int | None:
|
||||
try:
|
||||
start = date.fromisoformat(earlier)
|
||||
end = date.fromisoformat(later)
|
||||
except ValueError:
|
||||
return None
|
||||
return (end - start).days
|
||||
|
||||
|
||||
def _licensed_source(source: dict) -> bool:
|
||||
license_name = str(source.get("license") or "")
|
||||
return bool(source.get("id") and source.get("uri") and source.get("title") and license_name in ALLOWED_LICENSES)
|
||||
|
||||
|
||||
def _from_ingest(report: dict) -> list[dict[str, str]]:
|
||||
failures: list[dict[str, str]] = []
|
||||
for item in report.get("quarantined") or []:
|
||||
reason = item.get("reason")
|
||||
if reason == "rights":
|
||||
license_name = str(item.get("license") or "")
|
||||
if license_name in ("", "(missing)"):
|
||||
failures.append(_failure("missing-license", str(item.get("id") or ""), license_name or "(missing)"))
|
||||
else:
|
||||
failures.append(_failure("rights", str(item.get("id") or ""), item.get("detail") or license_name))
|
||||
elif reason == "missing-relation-label":
|
||||
failures.append(_failure("missing-relation-label", "", ",".join(item.get("predicates") or [])))
|
||||
elif reason == "unknown-kind":
|
||||
failures.append(_failure("unknown-kind", str(item.get("key") or ""), str(item.get("kind") or "")))
|
||||
elif reason:
|
||||
failures.append(_failure(str(reason), str(item.get("key") or item.get("id") or ""), str(item.get("detail") or "")))
|
||||
for item in report.get("dangling") or []:
|
||||
failures.append(_failure(
|
||||
"dangling-reference",
|
||||
str(item.get("from") or ""),
|
||||
str(item.get("to") or ""),
|
||||
))
|
||||
for item in report.get("conflicts") or []:
|
||||
reason = item.get("reason") or "conflict"
|
||||
kind = "duplicate-identity" if reason == "duplicate-identity" else str(reason)
|
||||
failures.append(_failure(kind, str(item.get("key") or ""), str(item.get("count") or reason)))
|
||||
return failures
|
||||
|
||||
|
||||
def quality_report(
|
||||
snapshot: dict | None = None,
|
||||
manifest: dict | None = None,
|
||||
ingest_report: dict | None = None,
|
||||
*,
|
||||
today: str | None = None,
|
||||
stale_days: int = 365,
|
||||
) -> dict[str, Any]:
|
||||
"""Summarise provenance, staleness, references, licenses, and failure classes."""
|
||||
failures = _from_ingest(ingest_report) if ingest_report else []
|
||||
unresolved: list[dict[str, str]] = []
|
||||
stale_ids: list[str] = []
|
||||
incomplete: list[str] = []
|
||||
missing_license: list[str] = []
|
||||
licenses: list[str] = []
|
||||
records = 0
|
||||
complete = 0
|
||||
source_count = 0
|
||||
sources_complete = 0
|
||||
dataspace_id = ""
|
||||
|
||||
if snapshot:
|
||||
dataspace_id = (snapshot.get("dataspace") or {}).get("id") or ""
|
||||
entities = list(snapshot.get("entities") or [])
|
||||
relations = list(snapshot.get("relations") or [])
|
||||
claims = list(snapshot.get("claims") or [])
|
||||
sources = list(snapshot.get("sources") or [])
|
||||
source_count = len(sources)
|
||||
by_source = {}
|
||||
for source in sources:
|
||||
source_id = str(source.get("id") or "")
|
||||
license_name = str(source.get("license") or "")
|
||||
if license_name and license_name not in licenses:
|
||||
licenses.append(license_name)
|
||||
if _licensed_source(source):
|
||||
sources_complete += 1
|
||||
by_source[source_id] = source
|
||||
else:
|
||||
missing_license.append(source_id or "(missing)")
|
||||
if not license_name:
|
||||
failures.append(_failure("missing-license", source_id, "(missing)"))
|
||||
else:
|
||||
failures.append(_failure("rights", source_id, license_name))
|
||||
|
||||
for label, items in (("entity", entities), ("claim", claims), ("relation", relations)):
|
||||
seen_here: dict[str, str] = {}
|
||||
for item in items:
|
||||
item_id = str(item.get("id") or "")
|
||||
if not item_id:
|
||||
continue
|
||||
if item_id in seen_here:
|
||||
failures.append(_failure("duplicate-identity", item_id, label))
|
||||
else:
|
||||
seen_here[item_id] = label
|
||||
|
||||
entity_ids = {str(entity.get("id")) for entity in entities if entity.get("id")}
|
||||
claim_ids = {str(claim.get("id")) for claim in claims if claim.get("id")}
|
||||
known_source_ids = {str(source.get("id")) for source in sources if source.get("id")}
|
||||
source_ids = set(by_source)
|
||||
|
||||
def provenance_ok(refs: list) -> bool:
|
||||
return any(ref in source_ids for ref in refs)
|
||||
|
||||
for entity in entities:
|
||||
entity_id = str(entity.get("id") or "")
|
||||
records += 1
|
||||
refs = list(entity.get("sourceRefs") or [])
|
||||
if provenance_ok(refs):
|
||||
complete += 1
|
||||
else:
|
||||
incomplete.append(entity_id)
|
||||
for ref in refs:
|
||||
if ref not in known_source_ids:
|
||||
unresolved.append({"id": entity_id, "field": "sourceRefs", "target": str(ref)})
|
||||
failures.append(_failure("unresolved-reference", entity_id, str(ref)))
|
||||
if entity.get("state") == "stale":
|
||||
stale_ids.append(entity_id)
|
||||
failures.append(_failure("stale", entity_id, "state"))
|
||||
elif today and entity.get("asOf"):
|
||||
age = _days_after(str(entity["asOf"]), today)
|
||||
if age is not None and age > stale_days:
|
||||
stale_ids.append(entity_id)
|
||||
failures.append(_failure("stale", entity_id, str(entity["asOf"])))
|
||||
for claim_id in entity.get("claimRefs") or []:
|
||||
if claim_id not in claim_ids:
|
||||
unresolved.append({"id": entity_id, "field": "claimRefs", "target": str(claim_id)})
|
||||
failures.append(_failure("unresolved-reference", entity_id, str(claim_id)))
|
||||
|
||||
for relation in relations:
|
||||
relation_id = str(relation.get("id") or "")
|
||||
records += 1
|
||||
refs = list(relation.get("sourceRefs") or [])
|
||||
if provenance_ok(refs):
|
||||
complete += 1
|
||||
else:
|
||||
incomplete.append(relation_id)
|
||||
for field in ("from", "to"):
|
||||
target = relation.get(field)
|
||||
if target not in entity_ids:
|
||||
unresolved.append({"id": relation_id, "field": field, "target": str(target)})
|
||||
failures.append(_failure("dangling-reference", relation_id, f"{field}:{target}"))
|
||||
for ref in refs:
|
||||
if ref not in known_source_ids:
|
||||
unresolved.append({"id": relation_id, "field": "sourceRefs", "target": str(ref)})
|
||||
failures.append(_failure("unresolved-reference", relation_id, str(ref)))
|
||||
|
||||
for claim in claims:
|
||||
claim_id = str(claim.get("id") or "")
|
||||
records += 1
|
||||
refs = list(claim.get("evidenceRefs") or [])
|
||||
if provenance_ok(refs):
|
||||
complete += 1
|
||||
else:
|
||||
incomplete.append(claim_id)
|
||||
subject = claim.get("subjectId")
|
||||
if subject not in entity_ids:
|
||||
unresolved.append({"id": claim_id, "field": "subjectId", "target": str(subject)})
|
||||
failures.append(_failure("dangling-reference", claim_id, f"subject:{subject}"))
|
||||
for ref in refs:
|
||||
if ref not in known_source_ids:
|
||||
unresolved.append({"id": claim_id, "field": "evidenceRefs", "target": str(ref)})
|
||||
failures.append(_failure("unresolved-reference", claim_id, str(ref)))
|
||||
|
||||
counts = {name: 0 for name in FAILURE_CLASSES}
|
||||
for failure in failures:
|
||||
counts[failure["class"]] = counts.get(failure["class"], 0) + 1
|
||||
|
||||
return {
|
||||
"dataspace": dataspace_id or (manifest or {}).get("id") or "",
|
||||
"provenance": {"records": records, "complete": complete, "incomplete": incomplete},
|
||||
"stale": {"count": len(stale_ids), "ids": stale_ids},
|
||||
"unresolvedReferences": unresolved,
|
||||
"failureClasses": counts,
|
||||
"failures": failures,
|
||||
"sources": {
|
||||
"count": source_count,
|
||||
"complete": sources_complete,
|
||||
"missingLicense": missing_license,
|
||||
"licenses": licenses,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def report_dataspaces(root: Path, *, today: str | None = None, stale_days: int = 365) -> list[dict[str, Any]]:
|
||||
"""One quality report for each example dataspace directory."""
|
||||
reports = []
|
||||
for snapshot_path in sorted(root.glob("*/snapshot.json")):
|
||||
directory = snapshot_path.parent
|
||||
snapshot = json.loads(snapshot_path.read_text(encoding="utf-8"))
|
||||
manifest_path = directory / "manifest.json"
|
||||
manifest = json.loads(manifest_path.read_text(encoding="utf-8")) if manifest_path.is_file() else None
|
||||
reports.append(quality_report(snapshot, manifest, today=today, stale_days=stale_days))
|
||||
return reports
|
||||
2
requirements.txt
Normal file
2
requirements.txt
Normal file
|
|
@ -0,0 +1,2 @@
|
|||
jsonschema==4.10.3
|
||||
PyYAML==6.0.1
|
||||
|
|
@ -17,6 +17,28 @@
|
|||
| Ask / MCP | upstream packages | P3 | Common read contract | Explicitly deferred |
|
||||
| Cancer anatomy/clinical actions | public site | Excluded | No generic analogue | No medical specificity leaked |
|
||||
|
||||
## Disposition after CONTENT-WP-0005
|
||||
|
||||
Each row is reused, generalized, omitted, or deferred. Pixel equality with OnCo is not a target.
|
||||
|
||||
| Pattern | Disposition | What shipped |
|
||||
|---|---|---|
|
||||
| Editorial landing and global search | generalized | Dataspace home, header, and record search. The command palette is not included. |
|
||||
| Kind directory | generalized | `/{dataspace}/{kind}/` from the snapshot registry. |
|
||||
| Faceted table | reused | `ResultsTable` and tag facets, reduced from the pinned table. |
|
||||
| Deep-linked view state | reused | `q`, repeated facet keys, and `sort` with a leading `-`. |
|
||||
| Object detail | generalized | Title, summary, sources, claims, and relation blocks. `EntityDetail.tsx` was not copied. |
|
||||
| Backlinks | generalized | Incoming relations use the manifest inverse label. |
|
||||
| Source evidence | generalized | Source title and license on the record. No separate citation browser. |
|
||||
| Glossary tooltip | generalized | Optional glossary and link preview. Absent data leaves the link in place. |
|
||||
| Graph explorer | generalized | One-hop and two-hop neighborhood, adjacency export, focus in the URL. |
|
||||
| Collections and reading paths | generalized | Declared on entity `properties`, rendered as member links and breadcrumbs. |
|
||||
| Timelines and comparisons | omitted | No timeline or comparison widget. |
|
||||
| Static API | deferred | Snapshots are files. There is no versioned HTTP snapshot endpoint. |
|
||||
| Ask / MCP | deferred | Not in this suite. |
|
||||
| Cancer anatomy and clinical actions | omitted | No medical widgets and no oncology corpus. |
|
||||
| OnCo mark, garden art, and logos | omitted | Not copied. `ui/NOTICE` keeps the MIT attribution and the pin. |
|
||||
|
||||
## Visual acceptance
|
||||
|
||||
At WP-00 capture desktop and narrow-screen reference views of homepage, directory, entity page, filtered table, related-items view. Use these screenshots as inspiration/benchmark, not redistribute without review. Compare spacing, hierarchy, readable density, color tokens, accessible contrast, empty states, keyboard paths, tooltip behavior and URL/history semantics. Record intentional deviations (branding, medical tools, sources).
|
||||
|
|
|
|||
|
|
@ -55,6 +55,18 @@ def footer_problems(text: str, pin: str) -> list[str]:
|
|||
return [f"ui/components/SiteFooter.tsx missing {', '.join(missing)}"]
|
||||
|
||||
|
||||
def third_party_problems(root: Path = ROOT, pin: str | None = None) -> list[str]:
|
||||
pin = load_pin() if pin is None else pin
|
||||
path = root / "THIRD_PARTY.md"
|
||||
if not path.is_file():
|
||||
return ["THIRD_PARTY.md is missing"]
|
||||
text = path.read_text(encoding="utf-8")
|
||||
missing = [mark for mark in ("Copyright (c) 2026 OnCo", "MIT License", pin) if mark not in text]
|
||||
if not missing:
|
||||
return []
|
||||
return [f"THIRD_PARTY.md missing {', '.join(missing)}"]
|
||||
|
||||
|
||||
def tree_problems(root: Path = UI, pin: str | None = None) -> list[str]:
|
||||
pin = load_pin() if pin is None else pin
|
||||
problems: list[str] = []
|
||||
|
|
@ -74,6 +86,7 @@ def tree_problems(root: Path = UI, pin: str | None = None) -> list[str]:
|
|||
problems.append("ui/components/SiteFooter.tsx is missing")
|
||||
else:
|
||||
problems.extend(footer_problems(footer.read_text(encoding="utf-8"), pin))
|
||||
problems.extend(third_party_problems(root.parent if root == UI else root, pin))
|
||||
return problems
|
||||
|
||||
|
||||
|
|
|
|||
12
scripts/ci-synthetic.sh
Normal file
12
scripts/ci-synthetic.sh
Normal file
|
|
@ -0,0 +1,12 @@
|
|||
#!/bin/sh
|
||||
# Synthetic suite. It does not fetch or require the OnCo corpus.
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
python3 scripts/validate_fixtures.py
|
||||
python3 -m unittest discover -s tests -p 'test_*.py'
|
||||
python3 scripts/check_ui_boundary.py
|
||||
python3 scripts/check_ui_provenance.py
|
||||
python3 scripts/quality_report.py >/dev/null
|
||||
cd ui
|
||||
npm test
|
||||
npm run typecheck
|
||||
39
scripts/quality_report.py
Normal file
39
scripts/quality_report.py
Normal file
|
|
@ -0,0 +1,39 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Print a quality report for every example dataspace."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from content_explorer.quality import report_dataspaces # noqa: E402
|
||||
|
||||
|
||||
def main() -> int:
|
||||
reports = report_dataspaces(ROOT / "examples" / "dataspaces", today=date.today().isoformat())
|
||||
if not reports:
|
||||
print("no dataspace fixtures found", file=sys.stderr)
|
||||
return 1
|
||||
problems = []
|
||||
for report in reports:
|
||||
if report["provenance"]["complete"] != report["provenance"]["records"]:
|
||||
problems.append(f"{report['dataspace']}: provenance incomplete")
|
||||
for name, count in report["failureClasses"].items():
|
||||
if count:
|
||||
problems.append(f"{report['dataspace']}: {name} {count}")
|
||||
json.dump(reports, sys.stdout, indent=2)
|
||||
sys.stdout.write("\n")
|
||||
if problems:
|
||||
print("\n".join(problems), file=sys.stderr)
|
||||
return 1
|
||||
print("Quality report OK", file=sys.stderr)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
171
tests/test_quality.py
Normal file
171
tests/test_quality.py
Normal file
|
|
@ -0,0 +1,171 @@
|
|||
"""Quality report and public snapshot rejection."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import json
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from content_explorer.ingest import project_records, publish_directory # noqa: E402
|
||||
from content_explorer.quality import FAILURE_CLASSES, quality_report, report_dataspaces # noqa: E402
|
||||
|
||||
EXAMPLES = ROOT / "examples" / "dataspaces"
|
||||
LABELS = {"represented-by": {"forward": "Represented by", "inverse": "Represents"}}
|
||||
KINDS = [
|
||||
{"id": "concept", "label": "Concept", "plural": "Concepts"},
|
||||
{"id": "method", "label": "Method", "plural": "Methods"},
|
||||
]
|
||||
SOURCE = {
|
||||
"id": "src:demo:notes",
|
||||
"uri": "https://example.invalid/content-explorer/ingest",
|
||||
"title": "Demo notes",
|
||||
"license": "CC0-1.0",
|
||||
"attribution": "Content Explorer demo",
|
||||
}
|
||||
RECORDS = [
|
||||
{
|
||||
"kind": "concept",
|
||||
"key": "feedback",
|
||||
"title": "Feedback",
|
||||
"tldr": "Results change the next action.",
|
||||
"summary": "A loop carries an outcome back to the actor.",
|
||||
"tags": ["regulation"],
|
||||
"relations": [{"predicate": "represented-by", "to": "method:loop-diagram"}],
|
||||
},
|
||||
{
|
||||
"kind": "method",
|
||||
"key": "loop-diagram",
|
||||
"title": "Loop diagram",
|
||||
"tldr": "Arrows that show influence.",
|
||||
"summary": "A drawing of influences among parts.",
|
||||
"tags": ["diagram"],
|
||||
"relations": [],
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def project(records, source=SOURCE, relation_labels=None):
|
||||
return project_records(
|
||||
records,
|
||||
dataspace_id="demo-notes",
|
||||
title="Demo notes",
|
||||
description="A connector test collection.",
|
||||
source=source,
|
||||
kinds=KINDS,
|
||||
observed_at="2026-10-12",
|
||||
relation_labels=LABELS if relation_labels is None else relation_labels,
|
||||
)
|
||||
|
||||
|
||||
def load_example(name: str) -> tuple[dict, dict]:
|
||||
directory = EXAMPLES / name
|
||||
snapshot = json.loads((directory / "snapshot.json").read_text(encoding="utf-8"))
|
||||
manifest = json.loads((directory / "manifest.json").read_text(encoding="utf-8"))
|
||||
return snapshot, manifest
|
||||
|
||||
|
||||
class ExampleReports(unittest.TestCase):
|
||||
def test_published_examples_are_complete(self) -> None:
|
||||
reports = report_dataspaces(EXAMPLES, today="2026-10-19")
|
||||
self.assertEqual(
|
||||
sorted(report["dataspace"] for report in reports),
|
||||
["software-architecture", "systems-thinking"],
|
||||
)
|
||||
for report in reports:
|
||||
self.assertGreater(report["provenance"]["records"], 0)
|
||||
self.assertEqual(report["provenance"]["complete"], report["provenance"]["records"])
|
||||
self.assertEqual(report["provenance"]["incomplete"], [])
|
||||
self.assertEqual(report["stale"]["count"], 0)
|
||||
self.assertEqual(report["unresolvedReferences"], [])
|
||||
self.assertEqual(report["sources"]["missingLicense"], [])
|
||||
self.assertEqual(report["sources"]["complete"], report["sources"]["count"])
|
||||
self.assertGreater(report["sources"]["count"], 0)
|
||||
self.assertIn("CC0-1.0", report["sources"]["licenses"])
|
||||
for name in FAILURE_CLASSES:
|
||||
self.assertEqual(report["failureClasses"][name], 0)
|
||||
|
||||
def test_missing_license_dangling_and_duplicate_are_failures(self) -> None:
|
||||
snapshot, manifest = load_example("systems-thinking")
|
||||
source_id = snapshot["sources"][0]["id"]
|
||||
missing = copy.deepcopy(snapshot)
|
||||
missing["sources"][0]["license"] = ""
|
||||
missing_report = quality_report(missing, manifest)
|
||||
self.assertIn(source_id, missing_report["sources"]["missingLicense"])
|
||||
self.assertGreater(missing_report["failureClasses"]["missing-license"], 0)
|
||||
self.assertTrue(any(
|
||||
item["class"] == "missing-license" and item["id"] == source_id
|
||||
for item in missing_report["failures"]
|
||||
))
|
||||
|
||||
dangling = copy.deepcopy(snapshot)
|
||||
relation_id = dangling["relations"][0]["id"]
|
||||
dangling["relations"][0]["to"] = "urn:ce:systems-thinking:concept:missing"
|
||||
dangling_report = quality_report(dangling, manifest)
|
||||
self.assertGreater(dangling_report["failureClasses"]["dangling-reference"], 0)
|
||||
self.assertTrue(any(
|
||||
item["id"] == relation_id and item["field"] == "to" for item in dangling_report["unresolvedReferences"]
|
||||
))
|
||||
|
||||
duplicate = copy.deepcopy(snapshot)
|
||||
entity_id = duplicate["entities"][0]["id"]
|
||||
duplicate["entities"].append(copy.deepcopy(duplicate["entities"][0]))
|
||||
duplicate_report = quality_report(duplicate, manifest)
|
||||
self.assertGreater(duplicate_report["failureClasses"]["duplicate-identity"], 0)
|
||||
self.assertTrue(any(
|
||||
item["class"] == "duplicate-identity" and item["id"] == entity_id
|
||||
for item in duplicate_report["failures"]
|
||||
))
|
||||
|
||||
allowed = copy.deepcopy(snapshot)
|
||||
allowed["sources"][0]["license"] = "MIT"
|
||||
allowed_report = quality_report(allowed, manifest)
|
||||
self.assertIn("MIT", allowed_report["sources"]["licenses"])
|
||||
self.assertEqual(allowed_report["failureClasses"]["rights"], 0)
|
||||
self.assertEqual(allowed_report["failureClasses"]["missing-license"], 0)
|
||||
|
||||
def test_stale_state_is_counted(self) -> None:
|
||||
snapshot, manifest = load_example("software-architecture")
|
||||
entity_id = snapshot["entities"][0]["id"]
|
||||
snapshot["entities"][0]["state"] = "stale"
|
||||
report = quality_report(snapshot, manifest, today="2026-10-19")
|
||||
self.assertIn(entity_id, report["stale"]["ids"])
|
||||
self.assertGreater(report["stale"]["count"], 0)
|
||||
self.assertGreater(report["failureClasses"]["stale"], 0)
|
||||
|
||||
|
||||
class IngestFailures(unittest.TestCase):
|
||||
def test_ingest_classes_and_publish_rejection(self) -> None:
|
||||
rights = project(RECORDS, source={**SOURCE, "license": "CC-BY-NC-4.0"})
|
||||
missing = project(RECORDS, source={**SOURCE, "license": ""})
|
||||
dangling = project([{
|
||||
**RECORDS[0],
|
||||
"relations": [{"predicate": "represented-by", "to": "method:missing"}],
|
||||
}])
|
||||
duplicate = project(RECORDS + [{**RECORDS[0], "title": "Other feedback"}])
|
||||
cases = {
|
||||
"rights": (rights, "rights"),
|
||||
"missing-license": (missing, "missing-license"),
|
||||
"dangling-reference": (dangling, "dangling-reference"),
|
||||
"duplicate-identity": (duplicate, "duplicate-identity"),
|
||||
}
|
||||
for result, kind in cases.values():
|
||||
self.assertFalse(result["report"]["published"])
|
||||
self.assertIsNone(result["snapshot"])
|
||||
report = quality_report(ingest_report=result["report"])
|
||||
self.assertGreater(report["failureClasses"][kind], 0)
|
||||
self.assertEqual(report["provenance"]["records"], 0)
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
destination = Path(tmp) / "out"
|
||||
with self.assertRaises(RuntimeError):
|
||||
publish_directory(destination, result)
|
||||
self.assertFalse((destination / "snapshot.json").exists())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -59,6 +59,9 @@ class UiProvenance(unittest.TestCase):
|
|||
def test_shipped_ui_keeps_headers_and_notices(self) -> None:
|
||||
self.assertEqual(provenance.tree_problems(), [])
|
||||
|
||||
def test_third_party_notice_names_the_pin(self) -> None:
|
||||
self.assertEqual(provenance.third_party_problems(ROOT, PIN), [])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
|
|||
157
ui/components/journeys.test.tsx
Normal file
157
ui/components/journeys.test.tsx
Normal file
|
|
@ -0,0 +1,157 @@
|
|||
// @vitest-environment happy-dom
|
||||
import { createRoot, type Root } from "react-dom/client";
|
||||
import { act, type ReactNode } from "react";
|
||||
import { renderToStaticMarkup } from "react-dom/server";
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import { EntityDetail } from "@/components/EntityDetail";
|
||||
import { GraphExplorer } from "@/components/GraphExplorer";
|
||||
import { RecordBrowser } from "@/components/RecordBrowser";
|
||||
import { findEntity, findGraph, findKind, type EntityView } from "@/lib/catalog";
|
||||
import type { GraphHistory } from "@/lib/graph";
|
||||
import type { ViewHistory } from "@/lib/table-view";
|
||||
|
||||
declare global {
|
||||
var IS_REACT_ACT_ENVIRONMENT: boolean;
|
||||
}
|
||||
|
||||
globalThis.IS_REACT_ACT_ENVIRONMENT = true;
|
||||
|
||||
const SPACES = ["systems-thinking", "software-architecture"] as const;
|
||||
|
||||
let root: Root | null = null;
|
||||
|
||||
afterEach(() => {
|
||||
act(() => root?.unmount());
|
||||
root = null;
|
||||
document.body.innerHTML = "";
|
||||
});
|
||||
|
||||
function memoryHistory(): ViewHistory {
|
||||
let current = "";
|
||||
return {
|
||||
read: () => (current ? `?${current}` : ""),
|
||||
push: (search) => {
|
||||
current = search;
|
||||
},
|
||||
subscribe: () => () => undefined,
|
||||
};
|
||||
}
|
||||
|
||||
function fixedHistory(search: string): GraphHistory {
|
||||
return {
|
||||
read: () => search,
|
||||
push: () => undefined,
|
||||
subscribe: () => () => undefined,
|
||||
};
|
||||
}
|
||||
|
||||
function journeyEntity(dataspaceId: string): EntityView {
|
||||
const kind = findKind(dataspaceId, "concept");
|
||||
if (!kind) throw new Error(`missing concept kind in ${dataspaceId}`);
|
||||
for (const row of kind.rows) {
|
||||
const slug = row.route.split("/").filter(Boolean).at(-1) ?? "";
|
||||
const entity = findEntity(dataspaceId, "concept", decodeURIComponent(slug));
|
||||
if (entity && (entity.outgoing.length > 0 || entity.incoming.length > 0) && entity.sources.length > 0) {
|
||||
return entity;
|
||||
}
|
||||
}
|
||||
throw new Error(`no related record in ${dataspaceId}`);
|
||||
}
|
||||
|
||||
function mount(node: ReactNode) {
|
||||
const host = document.createElement("div");
|
||||
document.body.appendChild(host);
|
||||
root = createRoot(host);
|
||||
return act(async () => {
|
||||
root?.render(node);
|
||||
});
|
||||
}
|
||||
|
||||
describe("critical journeys", () => {
|
||||
it("browses, searches, opens a record, follows a neighbour, shows a source, and opens the graph", async () => {
|
||||
for (const id of SPACES) {
|
||||
const kind = findKind(id, "concept");
|
||||
const entity = journeyEntity(id);
|
||||
expect(kind).not.toBeNull();
|
||||
document.body.innerHTML = "";
|
||||
await mount(<RecordBrowser rows={kind!.rows} columns={kind!.columns} history={memoryHistory()} />);
|
||||
const names = () => [...document.querySelectorAll("tbody tr td:first-child")].map((cell) => cell.textContent ?? "");
|
||||
expect(names().length).toBe(kind!.rows.length);
|
||||
expect(names()).toContain(entity.title);
|
||||
|
||||
const input = document.querySelector("input[aria-label='Search records']");
|
||||
if (!(input instanceof HTMLInputElement)) throw new Error("missing search");
|
||||
await act(async () => {
|
||||
const setValue = Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, "value")?.set;
|
||||
setValue?.call(input, entity.title);
|
||||
input.dispatchEvent(new Event("input", { bubbles: true }));
|
||||
input.form?.requestSubmit();
|
||||
});
|
||||
expect(names()).toEqual([entity.title]);
|
||||
|
||||
const detailHtml = renderToStaticMarkup(<EntityDetail entity={entity} />);
|
||||
const detail = new DOMParser().parseFromString(detailHtml, "text/html");
|
||||
const source = entity.sources[0];
|
||||
expect(source).toBeTruthy();
|
||||
expect(detail.body.textContent).toContain(source.title);
|
||||
expect(detail.body.textContent).toContain(source.license);
|
||||
const neighbour = entity.outgoing[0] ?? entity.incoming[0];
|
||||
expect(neighbour.other.href).toBeTruthy();
|
||||
const neighbourLink = detail.querySelector(`a[href="${neighbour.other.href}"]`);
|
||||
expect(neighbourLink?.textContent).toContain(neighbour.other.title);
|
||||
const parts = neighbour.other.href!.split("/").filter(Boolean);
|
||||
const followed = findEntity(parts[0], parts[1], decodeURIComponent(parts[2]));
|
||||
expect(followed?.title).toBe(neighbour.other.title);
|
||||
expect(renderToStaticMarkup(<EntityDetail entity={followed!} />)).toContain(followed!.title);
|
||||
expect(detailHtml).toContain(entity.graphHref);
|
||||
|
||||
act(() => root?.unmount());
|
||||
root = null;
|
||||
document.body.innerHTML = "";
|
||||
const graph = findGraph(id);
|
||||
expect(graph).not.toBeNull();
|
||||
const search = `?${new URLSearchParams({ focus: entity.id }).toString()}`;
|
||||
await mount(<GraphExplorer model={graph!} initialSearch={search} history={fixedHistory(search)} />);
|
||||
expect(document.querySelector("[data-focus]")?.getAttribute("data-focus")).toBe(entity.id);
|
||||
expect(document.body.textContent).toContain(neighbour.wording);
|
||||
expect(document.querySelector("[data-excluded-count]")).toBeTruthy();
|
||||
act(() => root?.unmount());
|
||||
root = null;
|
||||
}
|
||||
});
|
||||
|
||||
it("can focus search, a result, the record link, a relation, and the way back", async () => {
|
||||
for (const id of SPACES) {
|
||||
const kind = findKind(id, "concept");
|
||||
const entity = journeyEntity(id);
|
||||
document.body.innerHTML = "";
|
||||
await mount(<RecordBrowser rows={kind!.rows} columns={kind!.columns} history={memoryHistory()} />);
|
||||
const search = document.querySelector("input[aria-label='Search records']");
|
||||
const result = document.querySelector("tbody a");
|
||||
if (!(search instanceof HTMLElement) || !(result instanceof HTMLAnchorElement)) throw new Error("missing focus targets");
|
||||
search.focus();
|
||||
expect(document.activeElement).toBe(search);
|
||||
expect(search.tabIndex).toBeGreaterThanOrEqual(0);
|
||||
result.focus();
|
||||
expect(document.activeElement).toBe(result);
|
||||
expect(result.getAttribute("href")).toBeTruthy();
|
||||
|
||||
act(() => root?.unmount());
|
||||
root = null;
|
||||
document.body.innerHTML = "";
|
||||
const host = document.createElement("div");
|
||||
host.innerHTML = renderToStaticMarkup(<EntityDetail entity={entity} />);
|
||||
document.body.appendChild(host);
|
||||
const relation = host.querySelector("[data-relation] a");
|
||||
const back = [...host.querySelectorAll("a")].find((link) => link.textContent?.startsWith("Back to"));
|
||||
if (!(relation instanceof HTMLAnchorElement) || !(back instanceof HTMLAnchorElement)) {
|
||||
throw new Error(`missing record links in ${id}`);
|
||||
}
|
||||
relation.focus();
|
||||
expect(document.activeElement).toBe(relation);
|
||||
back.focus();
|
||||
expect(document.activeElement).toBe(back);
|
||||
expect(back.getAttribute("href")).toBe(entity.kindHref);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
|
@ -4,12 +4,12 @@ type: workplan
|
|||
title: "Quality system and upstream experience parity"
|
||||
domain: infotech
|
||||
repo: content-explorer
|
||||
status: proposed
|
||||
status: finished
|
||||
flavor: implementation
|
||||
depends_on:
|
||||
- CONTENT-WP-0004
|
||||
created: "2026-10-12"
|
||||
updated: "2026-10-12"
|
||||
updated: "2026-10-19"
|
||||
state_hub_workstream_id: "f1e0fa09-c85d-5aac-b665-4d73b0863fc2"
|
||||
---
|
||||
|
||||
|
|
@ -25,7 +25,7 @@ CONTENT-WP-0007 compares one permitted pinned OnCo slice through the original im
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0005-T01
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "9b83cd57-836e-5fac-a1cc-200481036aef"
|
||||
```
|
||||
|
|
@ -34,7 +34,7 @@ state_hub_task_id: "9b83cd57-836e-5fac-a1cc-200481036aef"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0005-T02
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "6a94c105-e66b-5aa2-a46c-a42fdd709e0f"
|
||||
```
|
||||
|
|
@ -43,7 +43,7 @@ state_hub_task_id: "6a94c105-e66b-5aa2-a46c-a42fdd709e0f"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0005-T03
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "0eb744db-302b-5cf8-82f5-d1851a009d23"
|
||||
```
|
||||
|
|
@ -52,7 +52,7 @@ state_hub_task_id: "0eb744db-302b-5cf8-82f5-d1851a009d23"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0005-T04
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "fe27e784-a99b-54e0-9634-4c62073f9835"
|
||||
```
|
||||
|
|
@ -61,7 +61,7 @@ state_hub_task_id: "fe27e784-a99b-54e0-9634-4c62073f9835"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0005-T05
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "bfbb8b00-ac88-50d7-89ee-1210d716f7df"
|
||||
```
|
||||
|
|
@ -72,7 +72,7 @@ state_hub_task_id: "bfbb8b00-ac88-50d7-89ee-1210d716f7df"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0005-T06
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "cb73eb25-85b9-5808-aa1a-b00f6dc21a9f"
|
||||
```
|
||||
|
|
@ -81,7 +81,7 @@ state_hub_task_id: "cb73eb25-85b9-5808-aa1a-b00f6dc21a9f"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0005-T07
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "92c00f5d-e3c8-55d7-92f2-45fe71eee2ab"
|
||||
```
|
||||
|
|
@ -90,7 +90,7 @@ state_hub_task_id: "92c00f5d-e3c8-55d7-92f2-45fe71eee2ab"
|
|||
|
||||
```task
|
||||
id: CONTENT-WP-0005-T08
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "f35cfc13-0f9f-5f50-871f-a9bc8d1e6148"
|
||||
```
|
||||
|
|
|
|||
51
workplans/evidence/CONTENT-WP-0005-ImplementationNotes.md
Normal file
51
workplans/evidence/CONTENT-WP-0005-ImplementationNotes.md
Normal file
|
|
@ -0,0 +1,51 @@
|
|||
# CONTENT-WP-0005 implementation notes
|
||||
|
||||
Date: 2026-10-19
|
||||
|
||||
Public checks stay on the two synthetic dataspaces, `systems-thinking` and `software-architecture`. The suite does not fetch the OnCo corpus.
|
||||
|
||||
## T01 and T07 — one suite, both journeys
|
||||
|
||||
`.forgejo/workflows/synthetic.yaml` checks out the repo, installs `requirements.txt` and `ui/` dependencies, and runs `sh scripts/ci-synthetic.sh`. That script runs fixture validation, `python3 -m unittest discover -s tests -p 'test_*.py'`, the UI boundary and provenance checks, `scripts/quality_report.py`, then `npm test` and `npm run typecheck` in `ui/`.
|
||||
|
||||
`ui/components/journeys.test.tsx` drives the shipped browser and detail markup for both dataspaces. For the first concept that has a relation and a source it searches that title, opens the record, checks the source title and license, follows the neighbour link, and opens `GraphExplorer` with that entity as focus. A second test focuses the search field, a result link, a relation link, and the way back.
|
||||
|
||||
## T02 — keyboard, viewports, screenshots, lab vitals
|
||||
|
||||
Headless Chromium on `npm run dev` at `http://127.0.0.1:3400` walked the same journeys at 1280×800 and 390×844. Focus reached search, a result, the record, a relation link, and the return link on both dataspaces. Page errors were 0. Captured under `/tmp/grok-goal-60054ed6e434/implementer/`:
|
||||
|
||||
- `a11y.txt`
|
||||
- `wide.png` (1280×971, 102155 bytes) and `narrow.png` (390×1411, 99962 bytes)
|
||||
- `screenshot-compare.txt`: the pixels differ; the wide view keeps search and the Search button on one row; the narrow view stacks search, tags, and the table. OnCo pixel equality was not measured.
|
||||
- `vitals.txt`: ttfbMs 445.1, domContentLoadedMs 664.7, loadMs 877.2, fcpMs 828, lcpMs 828. These are navigation-timing numbers from the last wide page, the software-architecture concept index.
|
||||
|
||||
## T03 and T08 — quality report
|
||||
|
||||
`content_explorer.quality.quality_report` reads a snapshot, a manifest, and an optional ingest report. It always emits every failure class, including zeros: `rights`, `missing-license`, `dangling-reference`, `duplicate-identity`, `unresolved-reference`, `missing-relation-label`, `unknown-kind`, and `stale`. It also reports provenance complete and incomplete ids, stale ids, unresolved references, and source counts with the licenses actually present.
|
||||
|
||||
`scripts/quality_report.py` prints that JSON for `examples/dataspaces` and exits non-zero when any class count is above zero or provenance is incomplete. On 2026-10-19 both examples are complete and every class count is 0. Their sources are CC0-1.0. A test that sets one source license to MIT sees `MIT` in `sources.licenses`.
|
||||
|
||||
## T04 — deviations and notices
|
||||
|
||||
`research/UXParityMatrix.md` section “Disposition after CONTENT-WP-0005” lists each upstream pattern as reused, generalized, omitted, or deferred. Faceted table and deep-linked view state are reused. Landing, kind directory, detail, backlinks, sources, glossary, graph, and collections are generalized. Timelines, comparisons, medical widgets, and the OnCo mark are omitted. A static API and Ask/MCP are deferred.
|
||||
|
||||
`scripts/check_ui_provenance.py` requires `THIRD_PARTY.md` to contain `Copyright (c) 2026 OnCo`, `MIT License`, and pin `b594d7433bf4e408b44215e25e5304696d6f840e`. `tests/test_ui_provenance.py` calls that check.
|
||||
|
||||
## T05 — publish rejection
|
||||
|
||||
`publish_directory` still refuses a quarantined or dangling import and does not write `snapshot.json`. `tests/test_quality.py` drives `project_records` and `publish_directory` on a missing license, a disallowed license, a dangling relation target, and a duplicate identity. Each case leaves `published` false, `snapshot` null, and a non-zero failure class on `quality_report`. Clearing a published source license, pointing a relation at a missing id, appending a duplicate entity, or marking an entity `stale` each raises the matching class above zero.
|
||||
|
||||
## Checks
|
||||
|
||||
Durable tests:
|
||||
|
||||
- `tests/test_quality.py`
|
||||
- `tests/test_ui_provenance.py`
|
||||
- `ui/components/journeys.test.tsx`
|
||||
- `.forgejo/workflows/synthetic.yaml` and `scripts/ci-synthetic.sh`
|
||||
|
||||
Captured on 2026-10-19 under `/tmp/grok-goal-60054ed6e434/implementer/`:
|
||||
|
||||
- `tests-1.txt` and `tests-2.txt`: `sh scripts/ci-synthetic.sh`, both exit 0. Unittest ran 26 tests. Vitest ran 12 files and 34 tests, including the two journey tests.
|
||||
- `ci-local.txt`: the workflow’s install and suite commands.
|
||||
- `fixtures.txt`: `python3 scripts/validate_fixtures.py` and `python3 scripts/check_ui_provenance.py`.
|
||||
13
workplans/evidence/CONTENT-WP-0005-Review.md
Normal file
13
workplans/evidence/CONTENT-WP-0005-Review.md
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
# CONTENT-WP-0005 review
|
||||
|
||||
Date: 2026-10-19
|
||||
|
||||
T01–T08 are done. The public suite is `.forgejo/workflows/synthetic.yaml`, which runs `sh scripts/ci-synthetic.sh` and does not fetch OnCo data.
|
||||
|
||||
Local CI result: `/tmp/grok-goal-60054ed6e434/implementer/ci-local.txt` runs the workflow commands (`pip install -r requirements.txt`, `npm ci` in `ui/`, then `sh scripts/ci-synthetic.sh`). The same suite was run twice before that and both exited 0 (`tests-1.txt`, `tests-2.txt`): 26 unittest tests, 34 Vitest tests. Fixture validation and the third-party notice check are in `fixtures.txt`.
|
||||
|
||||
Both example dataspaces pass browse, search, record open, neighbour follow, source and license, and graph focus. The keyboard pass in `a11y.txt` reached search, a result, the record, a relation link, and return at 1280×800 and 390×844, with page errors 0. Screenshots `wide.png` and `narrow.png` show the software-architecture concept index. They are not blank. Their pixels differ, and OnCo pixel equality was not measured (`screenshot-compare.txt`).
|
||||
|
||||
R7 deviation: the keyboard walk, the two screenshots, and the lab navigation timings in `vitals.txt` (ttfbMs 445.1, fcpMs 828, lcpMs 828) approximate usability. They are not field Core Web Vitals and they are not a usability proof. Screenshot comparison records the wide and narrow layout difference. It does not fail on unequal pixels.
|
||||
|
||||
The quality report over the published examples is complete, with every failure class at zero, including missing license, dangling reference, and duplicate identity. The same function reports those three as failures when the input contains them, and `publish_directory` refuses to write the snapshot. Upstream dispositions are in `research/UXParityMatrix.md`. Notices still name `Copyright (c) 2026 OnCo`, the MIT license, and pin `b594d7433bf4e408b44215e25e5304696d6f840e`.
|
||||
Loading…
Add table
Add a link
Reference in a new issue