coulomb/content-explorer
5
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Seeded repo

Browse source
This commit is contained in:
tegwick 2026-10-12 18:28:16 +02:00
parent 451fb5924c
commit 762707cefe
34 changed files with 1526 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

10
.gitignore vendored Normal file
View file

@ -0,0 +1,10 @@
node_modules/
.next/
out/
upstream-onco/
*.log
.env*
# Licensed local reference test material
tests/onco-reference/local/
tests/onco-reference/authorization.json

62
INTENT.md Normal file
View file

@ -0,0 +1,62 @@
# INTENT — content-explorer
**Project:** `content-explorer`
**Description:** A user interface to dig into a field of interconnected knowledge. Inspired by onco.cc, the very well organized information collection about everything related to cancer.
**Status:** Initial intent v0.1 — 2026-10-12
## Mission
Make any viable *infospace* navigable: a trustworthy, comprehensible, evolving map of a bounded field of knowledge, served through a consistent and delightful human interface regardless of the original sources, schemas or vocabulary.
Replicate the high-value user experience of [OnCo](https://onco.cc/) through a **maintainable, domain-independent adaptation of its MIT-licensed software**, not through indiscriminate cloning of its cancer-specific content and pages. A connector projects heterogeneous sources into a canonical, provenance-aware information model. A dataspace manifest configures the presentation without new UI code.
## Desired user experience
A visitor should be able to start with an accessible field overview; browse kinds/categories and filterable sortable lists; search across names, aliases and explanations; inspect a single knowledge object with plain-language summary first and deeper information below; explore typed incoming and outgoing relations; see evidence, freshness and uncertainty; and follow a reading path through related objects. Every aggregate is linked and every claim can identify its origin.
## Core principles
1. **Objects are addressable:** stable qualified identifiers and one canonical page per object.
2. **Connections are first-class:** directed typed relations with meaningful labels and derived reverse navigation; no forced tree model.
3. **Comprehension before expertise:** TL;DR first, domain-specific technical depth progressively disclosed.
4. **Evidence before authority:** preserve primary and secondary sources, extraction lineage, review timestamps, uncertainty and conflicting claims.
5. **Absence is data:** distinguish missing, unknown, redacted, stale, contradicted, and explicitly not-applicable; never pretend completeness.
6. **Domain vocabulary is declarative:** type names, iconography, facets, widgets, reading paths and landing navigation come from manifests.
7. **Ingestion is replaceable:** no source adapter imports UI code; source identifiers, transformations, licensing and refresh policy remain inspectable.
8. **Human and agent parity:** static/exportable graph and HTTP/JSON first; MCP and richer interfaces later.
9. **Composable and reversible:** a connector can be replaced, a source re-imported, a dataspace switched and a projection replayed without editing the UI.
10. **Reuse responsibly:** preserve notices; distinguish MIT-licensed code from separately licensed data, images and brands.
## In scope — first proof
- Two clearly unrelated, tiny authored example dataspaces (systems thinking and software architecture).
- Connectors: canonical JSON/NDJSON fixture loader and Markdown frontmatter importer; a third connector (GitHub issues or Wikidata) is a subsequent integration target.
- Common types: Dataspace, Kind, Entity, Relation, Claim, Evidence/Source, Collection, Navigation and View metadata.
- OnCo-inspired landing, faceted index, detail page, backlinks/related objects, search and basic graph view.
- Build-time static projection and validation initially; dynamic refresh later.
- Audit-grade import report with reproducible ingest inputs, warnings and provenance.
## Explicit non-goals for v0.1
- Copying the oncology corpus, OnCo logos, trademark, or bespoke medical decision-making widgets.
- A universal ontology, automatic semantic truth determination, AI-generated citations, or unrestricted content scraping.
- Distributed graph database, runtime multi-tenant writes, authentication or a production LLM answer service.
- Pixel-perfect reproduction at the cost of accessibility, performance or domain independence.
## Architectural wager
OnCo's generic browsing patterns are reusable, but its domain coupling is not fully abstract. Preserve reusable UX and presentation components where legally and technically feasible; insert a **projection boundary** between heterogeneous source records and the UI. Use **profile-driven type and view registries**, not oncology constants disguised as generic enums.
## Proof of success / exit criteria
- Switching dataspace via configuration alone shows another content field, without changes to UI component source.
- Every visible entity and relationship uses canonical IDs and resolvable routes.
- Every imported claim displays its source, observed/as-of date and rights metadata where present; unsupported assertions have explicit status.
- Dangling references, duplicate identity conflicts and forbidden rights propagation block publication or produce explicit quarantine according to policy.
- Full browse→filter→entity→neighbor→source journey works by mouse and keyboard.
- Feature parity matrix demonstrates what was reused, generalized, omitted or deferred from OnCo.
- Clean attribution and third-party notice report; regression tests and two-domain visual review.
## Long-term vision
An ecosystem of self-improving dataspaces: connectors and human editors enrich a source-aware information substrate, quality and coverage measures reveal gaps, and a portable explorer offers reading, investigation, comparison and eventually questions/answers without becoming the system of record. The experience can be embedded in larger knowledge architectures through stable JSON, links and optional agent-facing protocols.

30
README.md
View file

@ -1,3 +1,31 @@
# content-explorer
A user interface to dig into a field of interconnected knowledge. Inspired by onco.cc the very well organized information collection about everything related to cancer.
A user interface to dig into a field of interconnected knowledge. Inspired by onco.cc the very well organized information collection about everything related to cancer.# content-explorer
> A user interface to dig into a field of interconnected knowledge. Inspired by onco.cc, the very well organized information collection about everything related to cancer.
**Stage:** research / repository seed, not yet a working application. **Reference:** https://github.com/judegomila/OnCo (code MIT; OnCo corpus CC BY-NC 4.0).
Start with [INTENT.md](INTENT.md), [research/OnCoDeepDive.md](research/OnCoDeepDive.md), [architecture/ArchitectureBlueprint.md](architecture/ArchitectureBlueprint.md), and [workplans/INDEX.md](workplans/INDEX.md).
## North-star demo
A single UI build loads either `systems-thinking` or `software-architecture`, shows the same browse/search/detail/connections experience, and needs zero edits to React components for the switch. Source references and rights follow every imported assertion.
## What this archive contains
Research grounded in publicly browsable upstream files, extraction plan, normative draft contracts, two independently authored small datasets, a basic connector example, executable fixture validator, roadmap and decision records. **It does not contain or claim to be a fork of upstream source.** Fetch and pin upstream at implementation time. See `scripts/bootstrap-upstream.sh`.
## Quick seed check
```sh
python3 scripts/validate_fixtures.py
```
## Implementation start
Workplan 00 pins upstream and records code provenance; Workplan 01 establishes clean domain fixtures, Workplan 02 extracts browse/detail shell. PRs should demonstrate test-backed progress against the acceptance criteria.
## OnCo comparison track
`WP-07` specifies a small, pinned, **authorized** OnCo reference-slice test program. No actual OnCo data is included. Public fixtures remain independently authored; real-data ingestion is disabled until the relevant data rights are cleared.

5
THIRD_PARTY.md Normal file
View file

@ -0,0 +1,5 @@
# Third-party materials and licensing
This seed does **not** contain OnCo source or OnCo oncology data. It uses factual references to the upstream code and site. Prior to copying/adapting source, inspect `https://github.com/judegomila/OnCo/blob/main/LICENSE` and retain its MIT copyright and permission notice alongside relevant copied files and in shipped notices. Check `LICENSE-DATA`: OnCo corpus is CC BY-NC 4.0 and is not automatically eligible for commercial product reuse. Independently clear artworks, images, logos and third-party source records. The independently authored illustrative sample descriptions are offered as CC0-1.0. The project must choose and add a definitive own-source LICENSE after review; upstream licensing does not by itself license this newly written research package.
**Reference testing rule:** OnCo `LICENSE-DATA` explicitly treats a company internal data pipeline as commercial use requiring permission. A small test slice is not automatically exempt. See `tests/onco-reference/README.md` and keep actual records out of public CI and redistribution without appropriate rights.

62
architecture/ArchitectureBlueprint.md Normal file
View file

@ -0,0 +1,62 @@
# Architecture blueprint v0.1
## Bounded context
`content-explorer` is a **read/explore projection** over one or more source dataspaces; not an authoritative universal knowledge graph. Source adapters produce candidate facts; a normalization and governance pipeline publishes immutable queryable snapshots. The interface sees only the snapshot contract and a declarative presentation manifest.
```text
Git Markdown ─┐
JSON/NDJSON ──┼─> connector adapters ─> typed staging ─> identity/rights/evidence ─> graph validator ─> snapshot
future API ───┘ | | |
quarantine report manifest
|
+---------------- static JSON/read API ----------------+
| | |
Next.js UI search/graph views future CLI/MCP
```
## Canonical layers
1. **Source artifact**: exact immutable locator and content hash, acquired/observed timestamp, origin license, optional source-specific ID.
2. **Assertion/Claim**: typed subject–predicate–object or field value, linked evidence, status, confidence (if available), temporal `validFrom`/`validTo`, provenance and extractor version. Attribute displayed on entity may be an aggregation of claims; don't discard disagreement.
3. **Entity/Relation projection**: globally qualified stable identifiers, domain-local kind IDs, labels, TL;DR, summary, curated properties, typed relations and incoming derived backlinks. Nonsemantic `related` edges are allowed but distinguished.
4. **Presentation**: dataspace manifest defines allowed kinds, labels, facets, grouping, column specs, navigation, field layout, cards and optional widgets.
5. **Transport/export**: versioned snapshot JSON and OpenAPI later; no raw connector type reaches the UI.
## Identity
Canonical IDs: `urn:ce:<dataspace>:<kind>:<key>` using normalized escaped segments or deterministic hashes when keys cannot meet the grammar. External IDs live separately as aliases. Two identical upstream IDs in different spaces cannot collide. Same-as merging is explicit with mapping confidence and curator decision; it must not happen from name similarity alone. Preserve source native identifiers for replay.
## Relations
Each relation has its own ID, source, target, predicate (manifest-defined), labels for both directions, evidence and time. Store directed edges only once; derive reverse navigation. Allow many-to-many, multiple relation types between nodes, multi-parent hierarchies and cycles; selectively enforce acyclic property for declared taxonomic predicates only.
## Claims, truth and evidence
Claims are source statements, not verified universal facts. Each projected display field optionally refers to one or more claim IDs; policies choose `preferred`, `show-all`, `contested` or `withhold`. Evidence includes source URI, document title, retrieved/verified timestamps, optional location/quote, license and attribution. UI exposes missing versus false versus stale versus disputed, without fake confidence scores.
## Connector protocol
`discover() -> SourceDescriptor[]`; `fetch(cursor?) -> SourceBatch`; `project(batch, mapping) -> ImportCandidate[]`; `checkpoint()` for deterministic repeat. Adapter declares capabilities (full snapshot vs incremental, read-only, rights metadata, deletions), sync semantics, source authority and supported MIME/media types. Every run persists: connector version, input digest, output digest, insert/update/tombstone/quarantine counts and diagnostics. In v0.1 implement as build-time CLI jobs, no service dependency.
**Ingestion modes:** authoritative snapshot: missing source records tombstoned only within that source's ownership boundary; incremental: delete only on explicit tombstone; replay: same batch is idempotent. Namespace assertions per connector+source and preserve change history in report. Fail closed for forbidden rights and known dangling references, quarantine optional low-integrity records when policy allows.
## Configurable views
Manifest owns `home`, `kinds`, `relations`, `navigation`, `facets`, `columns`, `displayPanels`, `readingPaths`, `locale`, `theme`. UI components know generic `Entity`, `Facet`, `Claim`, `Link`, `Collection`; any widget requiring domain-specific fields is an optional registered plugin with schema validation and fallback rendering.
## Trust and deployment
Begin with static snapshots generated in CI for reproducibility, modest cost and safe public read. Source data is untrusted input: URL allowlist/scheme checks; HTML sanitization; bounded document size; no dynamic evaluation; prevent SSRF in future remote fetch adapters. No personal/sensitive data for early public demo. Record source copyright/attribution and policy before publishing. Signed/hashed snapshots and dependency scanning before production.
## Quality measures
- Model independence: 2 domain manifests, identical app build; zero code changes between.
- Graph integrity: 100% IDs resolved or quarantined with diagnostics; no silent edge drops.
- Provenance: 100% externally sourced displayed claims traceable to a source and rights record.
- Navigation: browse, filter, search, detail, backlink, source, browser back and deep links pass e2e.
- Performance budgets to baseline: static initial view and graph neighborhood size; set numeric SLOs after measuring actual upstream baseline.
## Architectural trade-offs
Start with strongly typed **core envelope** and flexible per-kind properties; resist a rigid cross-domain ontology. Treat external RDF/JSON-LD as mapping targets, not prerequisite storage. Avoid graph DB until query or volume evidence demands one. Persist raw inputs separately from normalized snapshots. Never equate source trust with correctness of individual assertions.

25
architecture/CanonicalDataModel.md Normal file
View file

@ -0,0 +1,25 @@
# Canonical data model — draft contract v0.1
The JSON Schema in `contracts/snapshot.schema.json` is a **first executable draft**. It intentionally models a compact display projection; Claim/Evidence and policy semantics in this document are richer than the initial schema and must be implemented in WP-03.
## Records
- `Dataspace`: namespace, version, title, description, schemaVersion, language, source policy, kind registry, navigation, view registry.
- `Entity`: ID, kindId, title, aliases, tldr, summary, tags, properties, effective time, sourceRefs, claimRefs, state.
- `Relation`: id, predicate, from/to, evidenceRefs, asOf, sourceRef; direction preserved.
- `Claim`: id, subject, key or predicate, value/target, assertedBy, evidenceRefs, status, valid interval, observedAt; no mandatory scalar probability.
- `Source`: id, canonical URI, title, author/publisher, retrievedAt, publishedAt, license, attribution, contentHash.
- `Collection`: ordered set of entity IDs + editorial purpose, optionally computed from a view.
- `View`: declarative facets, columns, sorts, summary cards, panels and graph traversal policy.
## Projection semantics
A `view` never changes truth; it chooses presentation. Projected `Entity.summary` is a **display decision** (may be curated), and `Claim`s retain competing assertions. Edges without provenance can be shown as `unverified` but must not be presented as sourced fact. All counts derive from the snapshot and query state.
## Mappings versus ontology
Types such as `concept`, `person`, `organization`, `document`, `pattern`, `decision`, `problem` are example `kindId`s, not privileged ontological primitives. The core representation is open-world and domain-specific names live in the manifest. A future vocabulary layer can connect manifests to SKOS/schema.org/RDF terms when useful.
## Future-proofs
Versioned migrators (`schemaVersion`), stable source-scoped IDs, lifecycle tombstones, localizations, role/perspective-aware visibility, hierarchical collection membership, temporal relations, claim-level review state, conflict handling, event stream, and computation provenance.

29
architecture/ConnectorContract.md Normal file
View file

@ -0,0 +1,29 @@
# Connector protocol and governance
**Adapter boundary:** `source format -> candidate entities, claims, relations, sources -> validated canonical snapshot`. Connectors must not render or import React components.
## Minimum connector metadata
`id`, `version`, `sourceNamespace`, `capabilities`, `licensePolicy`, `mappingVersion`, `refreshMode`, `sourcePriority`, `lastCheckpoint`.
## Functions
- `discover(config)` returns stable enumerations if supported.
- `fetch(config, checkpoint)` yields batches with immutable locators and input digests.
- `map(batch)` emits candidates and issues; preserve raw provenance.
- `validate(candidates, manifest)` checks shape, IDs, rights, provenance, references.
- `commit(validated, priorSnapshot)` atomically publishes snapshot/report or rolls back.
## Candidate mapping
Input → `source record` → `canonical id` → `kind` → `fields/claims` → `relations`. Relationships may be forward references; resolve in a second pass. No fuzzy join without explicit mapping policy. Respect deletion semantics by source ownership. Retries + replay must converge.
## First connectors
1. **JSON/NDJSON**: local fixture and remote static file ingestion (local first). A simple reference mapping implementation is in `connectors/json_fixture.py`.
2. **Markdown+frontmatter**: repo text with typed metadata and wiki-style references; parse via a robust library, not ad-hoc regex in production. Preserve headings/source locations and content license.
3. **GitHub or Wikidata** (P2): authenticated/rate-limited external API snapshot; mapping rules, pagination, ETag and license review before activation.
## Required diagnostics
`runId`, connector/version, started/ended UTC, input digests, source counts, candidates, accepted, updated, tombstoned, quarantined, warning/error codes, rights blocks, unresolved target IDs and published snapshot digest. Generate a human-readable provenance/rights report with each snapshot.

9
connectors/json_fixture.py Normal file
View file

@ -0,0 +1,9 @@
"""Illustrative JSON fixture reader. Production connector protocol is described in architecture/ConnectorContract.md."""
import json
from pathlib import Path
def load_snapshot(path: str | Path) -> dict:
return json.loads(Path(path).read_text(encoding="utf-8"))
def entities_by_kind(snapshot: dict, kind: str) -> list[dict]:
return [entity for entity in snapshot["entities"] if entity["kindId"] == kind]

54
contracts/manifest.example.json Normal file
View file

@ -0,0 +1,54 @@
{
"id": "systems-thinking",
"home": {
"heading": "Explore systems thinking",
"featuredKinds": [
"concept",
"person",
"publication",
"method"
]
},
"navigation": [
{
"label": "Explore",
"kinds": [
"concept",
"method",
"publication",
"person"
]
}
],
"kindViews": {
"concept": {
"facets": [
"discipline",
"tradition"
],
"columns": [
"name",
"tldr",
"tags"
]
}
},
"relationLabels": {
"influences": {
"forward": "Influences",
"inverse": "Influenced by"
},
"discusses": {
"forward": "Discusses",
"inverse": "Discussed in"
}
},
"panels": [
"summary",
"relations",
"evidence"
],
"theme": {
"accent": "indigo"
}
}

177
contracts/snapshot.schema.json Normal file
View file

@ -0,0 +1,177 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "Content Explorer compact snapshot v0.1",
"type": "object",
"required": [
"schemaVersion",
"dataspace",
"kinds",
"entities",
"relations",
"sources"
],
"properties": {
"schemaVersion": {
"const": "0.1"
},
"dataspace": {
"type": "object",
"required": [
"id",
"title",
"description"
],
"properties": {
"id": {
"type": "string"
},
"title": {
"type": "string"
},
"description": {
"type": "string"
}
}
},
"kinds": {
"type": "array",
"items": {
"type": "object",
"required": [
"id",
"label",
"plural"
],
"properties": {
"id": {
"type": "string"
},
"label": {
"type": "string"
},
"plural": {
"type": "string"
},
"facets": {
"type": "array",
"items": {
"type": "string"
}
}
}
}
},
"entities": {
"type": "array",
"items": {
"type": "object",
"required": [
"id",
"kindId",
"title",
"tldr",
"summary",
"sourceRefs"
],
"properties": {
"id": {
"type": "string"
},
"kindId": {
"type": "string"
},
"title": {
"type": "string"
},
"tldr": {
"type": "string"
},
"summary": {
"type": "string"
},
"tags": {
"type": "array",
"items": {
"type": "string"
}
},
"properties": {
"type": "object"
},
"sourceRefs": {
"type": "array",
"items": {
"type": "string"
}
},
"asOf": {
"type": "string",
"format": "date"
}
}
}
},
"relations": {
"type": "array",
"items": {
"type": "object",
"required": [
"id",
"predicate",
"from",
"to"
],
"properties": {
"id": {
"type": "string"
},
"predicate": {
"type": "string"
},
"from": {
"type": "string"
},
"to": {
"type": "string"
},
"sourceRefs": {
"type": "array",
"items": {
"type": "string"
}
}
}
}
},
"sources": {
"type": "array",
"items": {
"type": "object",
"required": [
"id",
"uri",
"title",
"license"
],
"properties": {
"id": {
"type": "string"
},
"uri": {
"type": "string"
},
"title": {
"type": "string"
},
"license": {
"type": "string"
},
"attribution": {
"type": "string"
}
}
}
}
},
"additionalProperties": false
}

9
decisions/ADR-0001-selective-onco-adaptation.md Normal file
View file

@ -0,0 +1,9 @@
# ADR-0001: selective reuse of OnCo UI through canonical projection
Status: Proposed (2026-10-12)
Decision: adopt Next.js/TypeScript and selectively adapt OnCo MIT-licensed UI components after pinned-source/license audit. Place normalized knowledge snapshot between ingestion and presentation; put kinds, facets, navigation and relation display in the dataspace manifest. Use a pure compatibility adapter for extracted browser components. Keep domain-specific widgets optional.
Consequences: achieves visible UX parity earlier than a complete rewrite while isolating OnCo-specific imports. Requires dependency tracing, code copyright preservation and source-maintenance strategy. Avoids copying CC BY-NC content into the seed.
Alternatives: wholesale fork (fast parity, high leakage/coupling); greenfield app (clean abstractions, high UX recreation cost). Revisit after WP-00 with concrete coupling measurements.

9
decisions/ADR-0002-static-projection-first.md Normal file
View file

@ -0,0 +1,9 @@
# ADR-0002: static build-time snapshots before live knowledge backend
Status: Proposed (2026-10-12)
Decision: begin with reproducible offline connectors and a versioned static snapshot JSON plus index; client-side/filter-in-browser at small scale. Build/validate before publish. Add remote read APIs, indexed search and graph storage only with measured limits or freshness requirements.
Rationale: onco.cc demonstrates considerable value via static-first publication, while scope here is a domain-independent exploration engine rather than real-time editing.
Trade-offs: delayed source refresh and snapshot size; address with paged static tables and incremental build if benchmarks require.

70
examples/dataspaces/software-architecture/manifest.json Normal file
View file

@ -0,0 +1,70 @@
{
"id": "software-architecture",
"home": {
"heading": "Explore Software Architecture",
"featuredKinds": [
"concept",
"pattern",
"person",
"artifact"
]
},
"navigation": [
{
"label": "Browse",
"kinds": [
"concept",
"pattern",
"person",
"artifact"
]
}
],
"kindViews": {
"concept": {
"columns": [
"name",
"tldr",
"tags"
],
"facets": [
"tags"
]
},
"pattern": {
"columns": [
"name",
"tldr",
"tags"
],
"facets": [
"tags"
]
},
"person": {
"columns": [
"name",
"tldr",
"tags"
],
"facets": [
"tags"
]
},
"artifact": {
"columns": [
"name",
"tldr",
"tags"
],
"facets": [
"tags"
]
}
},
"panels": [
"summary",
"relations",
"evidence"
]
}

214
examples/dataspaces/software-architecture/snapshot.json Normal file
View file

@ -0,0 +1,214 @@
{
"schemaVersion": "0.1",
"dataspace": {
"id": "software-architecture",
"title": "Software Architecture",
"description": "A small illustrative collection of patterns and ideas in software design"
},
"kinds": [
{
"id": "concept",
"label": "Concept",
"plural": "Concepts"
},
{
"id": "pattern",
"label": "Pattern",
"plural": "Patterns"
},
{
"id": "person",
"label": "Person",
"plural": "Persons"
},
{
"id": "artifact",
"label": "Artifact",
"plural": "Artifacts"
}
],
"sources": [
{
"id": "src:software-architecture:authored-demo",
"uri": "https://example.invalid/content-explorer/software-architecture/fixture",
"title": "Original illustrative demo fixture written for content-explorer",
"license": "CC0-1.0",
"attribution": "Content Explorer demo authors; fictional source URI is a fixture placeholder"
}
],
"entities": [
{
"id": "urn:ce:software-architecture:concept:coupling",
"kindId": "concept",
"title": "Coupling",
"tldr": "The degree to which software elements depend on one another.",
"summary": "The degree to which software elements depend on one another. This fixture is illustrative and does not constitute a sourced scholarly claim.",
"tags": [],
"sourceRefs": [
"src:software-architecture:authored-demo"
],
"asOf": "2026-10-12"
},
{
"id": "urn:ce:software-architecture:concept:cohesion",
"kindId": "concept",
"title": "Cohesion",
"tldr": "The degree to which a module groups related responsibilities.",
"summary": "The degree to which a module groups related responsibilities. This fixture is illustrative and does not constitute a sourced scholarly claim.",
"tags": [],
"sourceRefs": [
"src:software-architecture:authored-demo"
],
"asOf": "2026-10-12"
},
{
"id": "urn:ce:software-architecture:concept:interface",
"kindId": "concept",
"title": "Interface contract",
"tldr": "An agreed description of interaction between components.",
"summary": "An agreed description of interaction between components. This fixture is illustrative and does not constitute a sourced scholarly claim.",
"tags": [],
"sourceRefs": [
"src:software-architecture:authored-demo"
],
"asOf": "2026-10-12"
},
{
"id": "urn:ce:software-architecture:pattern:adapter",
"kindId": "pattern",
"title": "Adapter pattern",
"tldr": "An adapter translates between interfaces.",
"summary": "An adapter translates between interfaces. This fixture is illustrative and does not constitute a sourced scholarly claim.",
"tags": [],
"sourceRefs": [
"src:software-architecture:authored-demo"
],
"asOf": "2026-10-12"
},
{
"id": "urn:ce:software-architecture:pattern:ports-adapters",
"kindId": "pattern",
"title": "Ports and adapters",
"tldr": "A pattern for isolating core logic from outside systems.",
"summary": "A pattern for isolating core logic from outside systems. This fixture is illustrative and does not constitute a sourced scholarly claim.",
"tags": [],
"sourceRefs": [
"src:software-architecture:authored-demo"
],
"asOf": "2026-10-12"
},
{
"id": "urn:ce:software-architecture:pattern:event-driven",
"kindId": "pattern",
"title": "Event-driven architecture",
"tldr": "Software components communicate through events.",
"summary": "Software components communicate through events. This fixture is illustrative and does not constitute a sourced scholarly claim.",
"tags": [],
"sourceRefs": [
"src:software-architecture:authored-demo"
],
"asOf": "2026-10-12"
},
{
"id": "urn:ce:software-architecture:person:alexander",
"kindId": "person",
"title": "Christopher Alexander",
"tldr": "Architect whose pattern-language ideas influenced software design.",
"summary": "Architect whose pattern-language ideas influenced software design. This fixture is illustrative and does not constitute a sourced scholarly claim.",
"tags": [],
"sourceRefs": [
"src:software-architecture:authored-demo"
],
"asOf": "2026-10-12"
},
{
"id": "urn:ce:software-architecture:artifact:adr",
"kindId": "artifact",
"title": "Architecture Decision Record",
"tldr": "A lightweight record of architectural decisions and rationale.",
"summary": "A lightweight record of architectural decisions and rationale. This fixture is illustrative and does not constitute a sourced scholarly claim.",
"tags": [],
"sourceRefs": [
"src:software-architecture:authored-demo"
],
"asOf": "2026-10-12"
},
{
"id": "urn:ce:software-architecture:artifact:c4",
"kindId": "artifact",
"title": "C4 model diagram",
"tldr": "A set of views for communicating software architecture.",
"summary": "A set of views for communicating software architecture. This fixture is illustrative and does not constitute a sourced scholarly claim.",
"tags": [],
"sourceRefs": [
"src:software-architecture:authored-demo"
],
"asOf": "2026-10-12"
}
],
"relations": [
{
"id": "r1",
"predicate": "adapts",
"from": "urn:ce:software-architecture:pattern:adapter",
"to": "urn:ce:software-architecture:concept:interface",
"sourceRefs": [
"src:software-architecture:authored-demo"
]
},
{
"id": "r2",
"predicate": "uses",
"from": "urn:ce:software-architecture:pattern:ports-adapters",
"to": "urn:ce:software-architecture:concept:interface",
"sourceRefs": [
"src:software-architecture:authored-demo"
]
},
{
"id": "r3",
"predicate": "addressed-by",
"from": "urn:ce:software-architecture:concept:coupling",
"to": "urn:ce:software-architecture:pattern:ports-adapters",
"sourceRefs": [
"src:software-architecture:authored-demo"
]
},
{
"id": "r4",
"predicate": "related-to",
"from": "urn:ce:software-architecture:concept:cohesion",
"to": "urn:ce:software-architecture:concept:coupling",
"sourceRefs": [
"src:software-architecture:authored-demo"
]
},
{
"id": "r5",
"predicate": "documents",
"from": "urn:ce:software-architecture:artifact:adr",
"to": "urn:ce:software-architecture:pattern:ports-adapters",
"sourceRefs": [
"src:software-architecture:authored-demo"
]
},
{
"id": "r6",
"predicate": "visualizes",
"from": "urn:ce:software-architecture:artifact:c4",
"to": "urn:ce:software-architecture:concept:interface",
"sourceRefs": [
"src:software-architecture:authored-demo"
]
},
{
"id": "r7",
"predicate": "affects",
"from": "urn:ce:software-architecture:pattern:event-driven",
"to": "urn:ce:software-architecture:concept:coupling",
"sourceRefs": [
"src:software-architecture:authored-demo"
]
}
]
}

70
examples/dataspaces/systems-thinking/manifest.json Normal file
View file

@ -0,0 +1,70 @@
{
"id": "systems-thinking",
"home": {
"heading": "Explore Systems Thinking",
"featuredKinds": [
"concept",
"person",
"publication",
"method"
]
},
"navigation": [
{
"label": "Browse",
"kinds": [
"concept",
"person",
"publication",
"method"
]
}
],
"kindViews": {
"concept": {
"columns": [
"name",
"tldr",
"tags"
],
"facets": [
"tags"
]
},
"person": {
"columns": [
"name",
"tldr",
"tags"
],
"facets": [
"tags"
]
},
"publication": {
"columns": [
"name",
"tldr",
"tags"
],
"facets": [
"tags"
]
},
"method": {
"columns": [
"name",
"tldr",
"tags"
],
"facets": [
"tags"
]
}
},
"panels": [
"summary",
"relations",
"evidence"
]
}

214
examples/dataspaces/systems-thinking/snapshot.json Normal file
View file

@ -0,0 +1,214 @@
{
"schemaVersion": "0.1",
"dataspace": {
"id": "systems-thinking",
"title": "Systems Thinking",
"description": "A small illustrative collection of ideas in systems theory"
},
"kinds": [
{
"id": "concept",
"label": "Concept",
"plural": "Concepts"
},
{
"id": "person",
"label": "Person",
"plural": "Persons"
},
{
"id": "publication",
"label": "Publication",
"plural": "Publications"
},
{
"id": "method",
"label": "Method",
"plural": "Methods"
}
],
"sources": [
{
"id": "src:systems-thinking:authored-demo",
"uri": "https://example.invalid/content-explorer/systems-thinking/fixture",
"title": "Original illustrative demo fixture written for content-explorer",
"license": "CC0-1.0",
"attribution": "Content Explorer demo authors; fictional source URI is a fixture placeholder"
}
],
"entities": [
{
"id": "urn:ce:systems-thinking:concept:feedback",
"kindId": "concept",
"title": "Feedback",
"tldr": "Effects of actions change subsequent behavior.",
"summary": "Effects of actions change subsequent behavior. This fixture is illustrative and does not constitute a sourced scholarly claim.",
"tags": [],
"sourceRefs": [
"src:systems-thinking:authored-demo"
],
"asOf": "2026-10-12"
},
{
"id": "urn:ce:systems-thinking:concept:emergence",
"kindId": "concept",
"title": "Emergence",
"tldr": "Collective behavior arises through interactions.",
"summary": "Collective behavior arises through interactions. This fixture is illustrative and does not constitute a sourced scholarly claim.",
"tags": [],
"sourceRefs": [
"src:systems-thinking:authored-demo"
],
"asOf": "2026-10-12"
},
{
"id": "urn:ce:systems-thinking:concept:variety",
"kindId": "concept",
"title": "Requisite variety",
"tldr": "Effective regulation requires sufficient response variety.",
"summary": "Effective regulation requires sufficient response variety. This fixture is illustrative and does not constitute a sourced scholarly claim.",
"tags": [],
"sourceRefs": [
"src:systems-thinking:authored-demo"
],
"asOf": "2026-10-12"
},
{
"id": "urn:ce:systems-thinking:concept:system-boundary",
"kindId": "concept",
"title": "System boundary",
"tldr": "A distinction determines what is examined as a system.",
"summary": "A distinction determines what is examined as a system. This fixture is illustrative and does not constitute a sourced scholarly claim.",
"tags": [],
"sourceRefs": [
"src:systems-thinking:authored-demo"
],
"asOf": "2026-10-12"
},
{
"id": "urn:ce:systems-thinking:person:ashby",
"kindId": "person",
"title": "W. Ross Ashby",
"tldr": "Cybernetics researcher associated with requisite variety.",
"summary": "Cybernetics researcher associated with requisite variety. This fixture is illustrative and does not constitute a sourced scholarly claim.",
"tags": [],
"sourceRefs": [
"src:systems-thinking:authored-demo"
],
"asOf": "2026-10-12"
},
{
"id": "urn:ce:systems-thinking:person:meadows",
"kindId": "person",
"title": "Donella Meadows",
"tldr": "Systems thinker known for work on leverage points.",
"summary": "Systems thinker known for work on leverage points. This fixture is illustrative and does not constitute a sourced scholarly claim.",
"tags": [],
"sourceRefs": [
"src:systems-thinking:authored-demo"
],
"asOf": "2026-10-12"
},
{
"id": "urn:ce:systems-thinking:publication:thinking-in-systems",
"kindId": "publication",
"title": "Thinking in Systems",
"tldr": "A widely read introductory systems-thinking book.",
"summary": "A widely read introductory systems-thinking book. This fixture is illustrative and does not constitute a sourced scholarly claim.",
"tags": [],
"sourceRefs": [
"src:systems-thinking:authored-demo"
],
"asOf": "2026-10-12"
},
{
"id": "urn:ce:systems-thinking:method:causal-loop",
"kindId": "method",
"title": "Causal loop diagram",
"tldr": "A diagram representing causal influences and feedback.",
"summary": "A diagram representing causal influences and feedback. This fixture is illustrative and does not constitute a sourced scholarly claim.",
"tags": [],
"sourceRefs": [
"src:systems-thinking:authored-demo"
],
"asOf": "2026-10-12"
},
{
"id": "urn:ce:systems-thinking:method:stock-flow",
"kindId": "method",
"title": "Stock and flow model",
"tldr": "A representation of accumulations and their rates of change.",
"summary": "A representation of accumulations and their rates of change. This fixture is illustrative and does not constitute a sourced scholarly claim.",
"tags": [],
"sourceRefs": [
"src:systems-thinking:authored-demo"
],
"asOf": "2026-10-12"
}
],
"relations": [
{
"id": "r1",
"predicate": "represented-by",
"from": "urn:ce:systems-thinking:concept:feedback",
"to": "urn:ce:systems-thinking:method:causal-loop",
"sourceRefs": [
"src:systems-thinking:authored-demo"
]
},
{
"id": "r2",
"predicate": "scopes",
"from": "urn:ce:systems-thinking:concept:system-boundary",
"to": "urn:ce:systems-thinking:method:stock-flow",
"sourceRefs": [
"src:systems-thinking:authored-demo"
]
},
{
"id": "r3",
"predicate": "associated-with",
"from": "urn:ce:systems-thinking:concept:variety",
"to": "urn:ce:systems-thinking:person:ashby",
"sourceRefs": [
"src:systems-thinking:authored-demo"
]
},
{
"id": "r4",
"predicate": "authored-by",
"from": "urn:ce:systems-thinking:publication:thinking-in-systems",
"to": "urn:ce:systems-thinking:person:meadows",
"sourceRefs": [
"src:systems-thinking:authored-demo"
]
},
{
"id": "r5",
"predicate": "related-to",
"from": "urn:ce:systems-thinking:concept:emergence",
"to": "urn:ce:systems-thinking:concept:feedback",
"sourceRefs": [
"src:systems-thinking:authored-demo"
]
},
{
"id": "r6",
"predicate": "models",
"from": "urn:ce:systems-thinking:method:stock-flow",
"to": "urn:ce:systems-thinking:concept:feedback",
"sourceRefs": [
"src:systems-thinking:authored-demo"
]
},
{
"id": "r7",
"predicate": "explores",
"from": "urn:ce:systems-thinking:method:causal-loop",
"to": "urn:ce:systems-thinking:concept:emergence",
"sourceRefs": [
"src:systems-thinking:authored-demo"
]
}
]
}

17
history/2026-10-12-initial-exploration.md Normal file
View file

@ -0,0 +1,17 @@
# Research initiation — 2026-10-12
## User brief
Create `content-explorer`, a general-purpose browser for interconnected knowledge, influenced by onco.cc's excellent information presentation and MIT-licensed repository. Connect to arbitrary dataspaces by projecting varied inputs into an abstract model. Reuse and refactor OnCo's UI where possible. Deliver INTENT and research-backed implementation workplans as ZIP.
## Initial synthesis
OnCo supplies valuable generic scaffolding: graph validation/backlinks, index/detail routes, a faceted `EntityBrowser`, and static data publication. Those elements still embed cancer-specific types, icons and branching. The project should pin an upstream version, separate rights-controlled datasets from reusable code, and extract a generic UI behind a manifest and snapshot boundary.
## Selected design assumptions
Static-first MVP; Next.js/React/TypeScript continuity; canonical qualified identifiers, typed relation and evidence model; two small independent domains as a non-coupling test; initial local JSON and Markdown connectors; GitHub/Wikidata later. Claims, conflicts and license gates must not be erased by the normalized read projection.
## Known limitations
This research accessed a number of original source files and the live site but could not clone and execute the complete upstream repository. Baseline reproduction is the mandatory first workplan, not an already completed task.

78
research/OnCoDeepDive.md Normal file
View file

@ -0,0 +1,78 @@
# OnCo technical and UX deep dive
**Observed:** 2026-10-12. **Review basis:** public live site, GitHub README and selected raw source files accessible through web browsing. **Limitation:** complete Git clone/build/test was not possible in this research environment (GitHub host DNS unavailable to container). Findings about selected source files were inspected, but repository-wide coupling, tests and runtime behavior require checkout verification. Do not treat this document as a completed full-code audit.
## Primary references
- https://onco.cc/ — live information architecture and entry points.
- https://github.com/judegomila/OnCo — README, repo layout, licensing and contributor instructions.
- https://raw.githubusercontent.com/judegomila/OnCo/main/package.json — scripts and toolchain.
- https://raw.githubusercontent.com/judegomila/OnCo/main/src/lib/schema.ts — Zod entity schemas.
- https://raw.githubusercontent.com/judegomila/OnCo/main/src/lib/kinds.ts — kind registry, labels/routes.
- https://raw.githubusercontent.com/judegomila/OnCo/main/src/lib/graph.ts — validated graph and backlinks.
- https://raw.githubusercontent.com/judegomila/OnCo/main/src/components/EntityBrowser.tsx — generic sortable/filterable table with domain visuals.
- https://raw.githubusercontent.com/judegomila/OnCo/main/src/components/EntityDetail.tsx — entity details and tabs.
- https://raw.githubusercontent.com/judegomila/OnCo/main/src/app/%5Bkind%5D/page.tsx — static kind index routes.
- https://raw.githubusercontent.com/judegomila/OnCo/main/src/lib/nav.ts — navigation taxonomy.
- https://onco.cc/api/ — published machine-access contract.
## Structural anatomy (observed)
| Element | Upstream location | Reuse assessment |
|---|---|---|
| Content corpus | `src/data/`, `ALL_INPUTS` | Replace with connector-produced projection; don't bundle licensed oncology data |
| Schemas | `src/lib/schema.ts` and `kinds.ts` | Conceptual guide; reimplement as domain-neutral schemas and registries |
| Graph | `src/lib/graph.ts` | Reuse algorithmic patterns: duplicate/dangling validation, forward and reverse links; replace `forCancer` and hardcoded typed fields |
| Browser | `src/components/EntityBrowser.tsx` | High-value reuse candidate, but imports cancer icons, target schematics, approval chips, molecule views and Kind enum; split core table from renderers |
| Detail | `src/components/EntityDetail.tsx` | Extract generic title, TL;DR, evidence, relations and tab shell; create registry-driven panels |
| Index route | `src/app/[kind]/page.tsx` | Static routing and listing pattern; replace `ROUTE_TO_KIND`, `k===cancer` branches and `kindBrowser` switches |
| Nav | `src/lib/nav.ts` | Generate from manifest, not oncology-specific menu |
| Export | `scripts/build-api.ts`, API/OpenAPI scripts | Static snapshot publication is excellent v0.1 baseline; adapt canonical contract |
| CLI/MCP | `packages/`, `mcp/` | Subsequent adapters onto same read contract |
| Fetchers and validators | `scripts/` | Reuse refresh/validation workflow, not specialized cancer APIs |
## Notable implementation details
`Base` in `schema.ts` requires `id`, `kind`, `name`, `tldr`, `summary`, `asOf` and collects citation links, aliases, tags and many explicit relationship arrays such as `cancers`, `drugs`, `targets`, `trials`. Domain subtypes extend this base. In `graph.ts`, `Graph` validates via Zod, indexes by ID/kind, rejects duplicate and dangling IDs and derives backlinks; `forCancer` deliberately performs special oncology traversal. Those are clear refactoring boundaries.
`EntityBrowser.tsx` describes a shared full-width sortable/filterable table. Its `BrowserRow` already has generic `facets`, `cols`, `route`, `name`, `tldr`, `sortKeys`, but it also includes `molecule`, `cancerIcon`, `target`, `schematic`, `sectionIcon` and other domain renderers. It retains filter/search/sort state in URL parameters for shareable, static-export-compatible views. Preserve this behavior and separate domain visuals through an optional widget registry.
The Next.js `[kind]` page generates static paths from `KINDS`/`KIND_META`, which can naturally be replaced by `manifest.kinds`. Its current branches still special-case cancer, drug, target, institution and other domain pages. `package.json` defines `dev`, `build:api`, `build`, `validate`, `typecheck`, `lint`, and Vitest. Its build pipeline also emits embeddings, relatedness, triples and context snapshots: promising later extensions, not MVP dependencies.
## Experience inventory
| Experience | Prioritize | Proposed abstraction |
|---|---|---|
| Search box / command shortcut | P0 | Cross-entity full-text/alias index |
| Broad grouped navigation and spotlight landing | P0 | Dataspace home sections and featured collections |
| Faceted sortable entity browser | P0 | Kind-driven columns, facets, URL query state |
| Entity page with nontechnical TL;DR | P0 | Layout from entity+kind manifest |
| Related objects/backlinks | P0 | Typed directed edges, inverse display labels |
| Source links + freshness | P0 | Entity/claim evidence panel with visible as-of |
| Glossary/tooltips | P1 | Term kind + text reference annotations |
| Graph explorer | P1 | Bounded neighborhood or selection, graceful large-graph limits |
| Saved/bookmarkable views | P1 | URL state first; local storage later |
| Timeline/comparison/reading paths | P2 | Optional view plugins dependent on data capability |
| Ask/agent tools | P3 | Answer over cited corpus with provenance guardrails |
| Oncology-specific body map, molecule spins, clinical guides | Exclude initially | Possible domain-specific extension widgets, not core |
## Why reuse rather than replatform
Retaining Next.js, React, TypeScript, Zod, Tailwind-compatible styling and existing table/navigation patterns minimizes design drift. Introduce the normalized model **at build/projection time**, emitting a view model that existing components can consume. For the first port, a thin compatibility mapper (`CanonicalEntity` → `BrowserRow`) is safer than rewriting the entire presentation stack.
## Legal/security review
The upstream README explicitly states MIT **code** and CC BY-NC 4.0 **data**, with permission needed for commercial use of the data. Avoid shipping oncology records as tests. Original software copied/adapted from upstream must retain its MIT notice and copyright notice. Assets, illustrations, screenshots, logos and source data require independent rights review. Do not imply endorsement or use the OnCo name as the product's identity. Validate connector licenses, sanitize HTML/Markdown and outbound URL schemes, and quarantine content with ambiguous permission.
## Open verification tasks on actual clone
1. Pin SHA, enumerate package versions, licenses, source/assets boundaries and copied files.
2. Build upstream baseline, run tests, record accessibility/Lighthouse/screenshots in desktop/mobile.
3. Trace import dependency graph of `EntityBrowser`, `EntityDetail`, `graph`, `kind-browser` and `nav`.
4. Measure coupling via references to cancer-specific constants/files; identify safe incremental extraction points.
5. Run parallel before/after screenshot and interaction snapshots against representative OnCo views.
6. Check whether external package licenses and illustrations introduce obligations beyond MIT.
## Research assessment
Feasible, but **not plug-and-play**: generic graph and table abstractions coexist with extensive oncology special cases. Recommended strategy is a selective fork/extraction with repeatable upstream diff, a strict domain projection seam, and domain-independent acceptance fixtures. Avoid promising parity for the full OnCo site in the first release.

20
research/RisksAndAlternatives.md Normal file
View file

@ -0,0 +1,20 @@
# Risks, countermeasures, alternatives
| Risk | Impact | Response |
|---|---|---|
| OnCo evolves quickly | Cherry-pick conflicts | Pin SHA; maintain provenance diff and minimal extraction |
| Partial upstream audit | Surprise dependencies | WP-00 build + import graph + component classification |
| Misread MIT scope | Copyright/legal exposure | Separate code/data/assets; retain original MIT notice |
| Universal graph model too vague | Lost semantics | Domain manifest + typed predicates + preserved source claims |
| Overfitted universal model | Inflexible connector mappings | Core envelope plus extensible typed properties |
| Untraceable ingested claims | False confidence | Claim+Evidence lineage, block unsupported public claims |
| Duplicate identities | Corrupted graph | Source-qualified IDs, explicit equivalence registry |
| URL state drift | Broken shared views | Browser compatibility/e2e tests |
| Large graphs | UI unusable | Bounded neighborhood, paged indexes and lazy loading |
| Untrusted Markdown / remote content | XSS/SSRF | Safe parsers, sanitizer, URL policy, no execution |
| License contagion from imports | Publishing restrictions | Rights gate, attribution manifest, quarantine |
| Premature LLM question-answering | Hallucination | Defer until retrieval and evidence-grade evaluation |
## Alternatives
**A: Fork whole OnCo and replace content.** Fast visual parity but creates cancer-specific technical debt and major removal effort. **B: Fresh rewrite mimicking look.** Cleaner model but forfeits proven table/route work and carries UX drift risk. **C: Selective upstream extraction with canonical adapter (recommended).** Moderate integration cost and sustainable domain independence; requires careful upstream audit. Gate C against measurable coupling. If extraction crosses a predefined complexity threshold (record in WP-00), switch to B for affected components only.

22
research/UXParityMatrix.md Normal file
View file

@ -0,0 +1,22 @@
# OnCo → Content Explorer UX parity matrix
| Pattern | Reference | v0.1 target | Reuse path | Test |
|---|---|---|---|---|
| Editorial landing and global search | onco.cc home | Adapt | Header/home shell; dynamic title/category cards | Switch domain and labels update |
| Kind directory | `/[kind]/` | Adapt | Dynamic kind route from manifest | Both manifests render |
| Faceted table | `EntityBrowser.tsx` | Strong reuse | Extract `BrowserRow` / facets/columns | URL filtering, sorting |
| Deep-linked view state | `EntityBrowser.tsx` | Strong reuse | URL q/facets/sort | Reload retains state |
| Object detail | `EntityDetail.tsx` | Adapt | Generic tabs and property renderers | Same component for 2 domains |
| Backlinks | `graph.ts` | Reimplement core | Canonical typed edges and reverse index | All inbound/outbound visible |
| Source evidence | `schema.ts` / detail | Extend | Claim/source lineage display | Every cited display links to origin |
| Glossary tooltip | `Tip.tsx` | Later | Term-kind annotation | Graceful absence |
| Graph Explorer | public site | P1 | Neighbor-based generic viewer | Bounded graph, focus change |
| Collections/reading paths | public site | P2 | Manifest collection | Guided route |
| Timelines/comparisons | public site | P2 | Optional widget capabilities | Not rendered when unsupported |
| Static API | `build-api` | P1 | `snapshot.json` then versioned endpoint | Contract validation |
| Ask / MCP | upstream packages | P3 | Common read contract | Explicitly deferred |
| Cancer anatomy/clinical actions | public site | Excluded | No generic analogue | No medical specificity leaked |
## Visual acceptance
At WP-00 capture desktop and narrow-screen reference views of homepage, directory, entity page, filtered table, related-items view. Use these screenshots as inspiration/benchmark, not redistribute without review. Compare spacing, hierarchy, readable density, color tokens, accessible contrast, empty states, keyboard paths, tooltip behavior and URL/history semantics. Record intentional deviations (branding, medical tools, sources).

9
scripts/bootstrap-upstream.sh Normal file
View file

@ -0,0 +1,9 @@
#!/usr/bin/env bash
set -euo pipefail
# This is a research/bootstrap helper, not bundled upstream code.
# Pin commit after manual licensing inspection. Never merge upstream corpus into this repo by default.
: "${ONCO_COMMIT:?Set ONCO_COMMIT to a reviewed full Git SHA}"
git clone https://github.com/judegomila/OnCo.git upstream-onco
git -C upstream-onco checkout --detach "$ONCO_COMMIT"
printf 'OnCo pinned to %s\n' "$(git -C upstream-onco rev-parse HEAD)"
printf 'Review LICENSE, LICENSE-DATA, images and dependency licenses before adapting files.\n'

27
scripts/validate_fixtures.py Normal file
View file

@ -0,0 +1,27 @@
#!/usr/bin/env python3
"""Portable smoke validator: structural and cross-reference checks; not full JSON Schema compliance."""
from pathlib import Path
import json, sys
ROOT = Path(__file__).resolve().parent.parent
errors=[]
for path in sorted((ROOT/'examples/dataspaces').glob('*/snapshot.json')):
d=json.loads(path.read_text())
kinds={r['id'] for r in d['kinds']}; ids=[r['id'] for r in d['entities']]
sources={r['id'] for r in d['sources']}
if d['schemaVersion']!='0.1': errors.append(f'{path}: schemaVersion')
if len(ids)!=len(set(ids)): errors.append(f'{path}: duplicate entities')
for e in d['entities']:
if e['kindId'] not in kinds: errors.append(f'{path}: undeclared kind {e["kindId"]}')
for sid in e.get('sourceRefs',[]):
if sid not in sources: errors.append(f'{path}: missing source {sid}')
for r in d['relations']:
for field in ('from','to'):
if r[field] not in ids: errors.append(f'{path}: dangling {field} {r[field]}')
for sid in r.get('sourceRefs',[]):
if sid not in sources: errors.append(f'{path}: missing edge source {sid}')
manifest=json.loads((path.parent/'manifest.json').read_text())
if manifest['id']!=d['dataspace']['id']: errors.append(f'{path}: manifest ID mismatch')
print(f'{d["dataspace"]["id"]}: {len(ids)} entities, {len(d["relations"])} relations, {len(kinds)} kinds')
if errors:
print('\n'.join(errors),file=sys.stderr);sys.exit(1)
print('Smoke validation OK (full JSON Schema and claims pending).')

13
tests/AcceptanceScenarios.md Normal file
View file

@ -0,0 +1,13 @@
# Cross-domain acceptance scenarios (turn into Playwright suites at WP-02)
1. Launch with systems-thinking manifest. Home headline and available categories are systems-specific. Browse `concept`, filter/search `feedback`, open detail, navigate a relation, return via browser history. Show source and uncertainty context.
2. Rebuild or relaunch with software-architecture manifest **without editing UI source**. Home and kind pages change accordingly. Open `adapter`, follow `interface`, inspect backlink; `cancer` never appears in navigation or component module imports.
3. URL filter state is bookmarkable and remains after refresh and back-forward navigation.
4. Keyboard-only usage covers command search, facet control, result row and related-entity list.
5. Ingestion regression: same source batch twice yields byte-identical canonical projection (after canonical sorting); source removal behavior varies correctly by snapshot/incremental mode.
6. Validation rejects duplicate IDs, dangling relation targets, and missing rights data under strict public-publish policy.
7. Conflicting claims from two sources appear as disputed, not silently overwritten.
8. Graph handles cycles, multi-parent objects and high-degree nodes with explicit limits.
9. Every adapted upstream file includes applicable MIT notice and is traceable to a pinned commit.
10. Under the separately authorized OnCo reference-slice profile, verify forward and reverse relations, page semantics, filters, attribution, stable URLs and 3-hop journeys against the pinned upstream reference; see `tests/onco-reference/README.md`.

56
tests/onco-reference/README.md Normal file
View file

@ -0,0 +1,56 @@
# OnCo reference-slice conformance test (planned / opt-in)
Purpose: validate content-explorer against the *real* upstream UX and data topology, without treating the OnCo corpus as product data. This is an additional test track, not a replacement for CC0 generic fixture tests.
## Non-negotiable rights boundary
- OnCo **software** is MIT; OnCo **curated dataset** is CC BY-NC 4.0, with its own `LICENSE-DATA`. This repository contains **no copied OnCo dataset**.
- The upstream data terms expressly classify **a company's internal data pipeline** as commercial use requiring the rightsholder's agreement. **Small slices and private tests do not automatically create an exception**. Before importing real records in an organizational/commercial workflow, obtain written permission or a suitable license, or get independent legal approval for a defensible exception. Until then, use synthetic fixtures and references/URLs only.
- Record authorization in a *private* `authorization.json` document. Do not commit it if it contains contacts/confidential terms. Publication and CI distribution require separate permission evaluation.
- Attribution wherever permitted OnCo content is presented: `Data from OnCo (onco.cc)` linking to https://onco.cc; license link https://creativecommons.org/licenses/by-nc/4.0/; source URL, upstream SHA, accessed date, modification notice, and original upstream links. Inspect third-party assets separately; no logo/portrait copying by default.
- No patient/personal health records, no clinical conclusions, no medical-advice claims.
## Reference slice selection (target ~12–25 nodes, not a corpus fork)
Choose a connected induced/test subgraph from an immutable upstream commit. Include one of each: cancer, treatment/drug, target, technology, clinical trial, key paper, institution and glossary term; optionally a person and idea if rights permit. Choose real items **after** authorization and upstream pinning, not by hard-coded contemporary medical claims.
Coverage requirements:
1. At least two different types of edges and an inverse/backlink traversal.
2. At least one one-to-many relationship and one multi-hop path (depth >= 3).
3. At least one entity with >1 provenance link; dated TL;DR and technical summary.
4. At least one missing optional field, and at least one unresolved relation handled explicitly (not silently erased).
5. Two filter facets, full-text query and a stable URL deep link.
6. A few controlled mutations: renamed title, removed node, bad cross-reference, duplicate id, license metadata removed (all modifications clearly tagged as local tests, never represented as upstream facts).
## Proposed workflow
**Gate A — authorization:** verify rights and record intended use; CI containing the real dataset must remain disabled without sufficient grant.
**Gate B — pin & extract:** pin upstream commit (`git rev-parse HEAD`), document source paths and entity ids in `slice-selection.yaml`; extractor reads only enumerated records. Compute sha256 of originals and emitted projection. No bulk download/auto-discovery as a side effect of normal builds.
**Gate C — dual oracle:** run the pinned OnCo implementation as reference and capture results for identical user tasks, using stable screenshots and structured browser/graph assertions. Use the *same pinned records* through the connector to make comparable content states; preserve expected divergences in an explicit exception list.
**Gate D — conformance:** test source -> projection -> view model -> route using schema tests, semantic invariants, Playwright browser journeys, and visual snapshots. Record each mismatch as extraction, mapping, semantics, presentation, or upstream instability.
**Gate E — reproducibility:** publish hashes, commit SHA, normalized test descriptions and aggregate results only when permitted; distribute synthetic fixtures as the default executable public tests. Never upload the OnCo slice as a CI artifact without authorization.
## Run profiles
- `public-ci`: synthetic fixtures only; completely offline and distributable.
- `licensed-reference`: locally provisioned real slice, explicitly enabled after authorization; never default, no automatic telemetry/artifact export.
- `visual-upstream`: uses browser captures from pinned upstream locally; screenshots may have independent IP considerations; approval needed before publication.
## Acceptance gates
- [ ] Signed-off authorization and permitted scope recorded before any organization-run OnCo data extraction.
- [ ] Slice manifest enumerates pinned upstream SHA, 12–25 ids, files, types, edges, provenance and expected license/attribution.
- [ ] Extractor is allowlist-only and deterministic; no unrequested expansion to linked entities.
- [ ] Projection preserves node identity, types, relations, as-of metadata, citations, missingness and applicable rights.
- [ ] ≥95% of agreed deterministic behavioral assertions match reference, with **zero** silent link loss and **zero** missing required attribution; exact threshold can be revisited after baseline measurement.
- [ ] Visual differences categorized and reviewed; compare viewport/font/environment under pinned baseline, rather than requiring arbitrary pixel-perfect equality.
- [ ] Public CI passes with no OnCo corpus available; disabling the licensed dataset never breaks product features.
- [ ] Oncology slice and two unrelated synthetic domains exercise exactly the same UI code paths (except declarative presentation configuration).
## Evidence
`reference-manifest.json`, `baseline-report.md`, `projection-diff.json`, `journey-results.json`, `visual-review.md`, `rights-review.md` (non-confidential). Keep raw licensed data in ignored local directories.

7
tests/onco-reference/reference-manifest.example.json Normal file
View file

@ -0,0 +1,7 @@
{
"schema_version": "0.1",
"reference": {"project":"OnCo", "upstream":"https://github.com/judegomila/OnCo", "commit_sha":"REPLACE_WITH_40_CHAR_SHA", "license":"CC BY-NC 4.0", "attribution":"Data from OnCo (onco.cc)", "attribution_url":"https://onco.cc", "license_url":"https://creativecommons.org/licenses/by-nc/4.0/"},
"usage": {"mode":"LICENSED_REFERENCE_ONLY", "authorization_reference":"REQUIRED_BEFORE_REAL_DATA_IMPORT", "distribution_allowed":false, "ci_allowed":false},
"slice": {"entities":[], "source_paths":[], "expected_relations":[], "test_cases":["entity-browse","filter-search","entity-detail","forward-relation","backlink","multi-hop","provenance","stable-url"]},
"baseline": {"upstream_capture_id":null,"captured_at":null,"viewport":"1440x900","environment":null}
}

17
workplans/INDEX.md Normal file
View file

@ -0,0 +1,17 @@
# Execution plan and dependency map
| ID | Workplan | Prerequisites | Milestone |
|---|---|---|---|
| WP-00 | Upstream baseline & IP audit | none | Reproducible reference and extraction decision |
| WP-01 | Canonical contracts & fixtures | none; coordinate with WP-00 | Two independent dataspaces pass contract |
| WP-02 | UI extraction & experience skeleton | WP-00 + WP-01 | First generic explorer end-to-end |
| WP-03 | Ingestion & publishing | WP-01 (parallel WP-02) | Repeatable snapshots with evidence |
| WP-04 | Relations & journeys | WP-02 + WP-03 | Graph navigation |
| WP-05 | Quality & parity | begins WP-00, closes WP-04 | Release candidate |
| WP-06 | External pilot | WP-05 | Decision to expand |
**Minimum viable proof:** WP-00 → WP-01 → WP-02, with JSON snapshot ingestion; WP-03 provenance gate is required before any externally ingested public content is published. WP-04+ need not delay the basic demo.
Each workplan has a deliverable, tasks, verification gates and risks. No effort estimates are asserted before cloning and profiling the upstream code. Proposed priority: WP-00/01 now; WP-02 then WP-03; WP-04/05; pilot WP-06.
- **WP-07** — Opt-in, authorized OnCo reference-slice conformance and paired upstream behavioral tests (see `WP-07.md`).

28
workplans/WP-00.md Normal file
View file

@ -0,0 +1,28 @@
# WP-00 — Upstream baseline & IP audit
**Purpose:** Reproduce OnCo baseline and establish legal, behavioral and visual reference.
**Dependencies:** see `INDEX.md`. **Status:** planned. **Owner:** to assign. **Estimate:** to refine after WP-00.
## Work packages
- [ ] Pin immutable upstream git SHA and preserve original MIT license/NOTICE.
- [ ] Install exactly locked dependencies; run build/test/lint/typecheck and record results.
- [ ] Trace dependencies of browser, detail, graph, navigation and data.
- [ ] Capture responsive screenshots and keyboard interaction walkthrough.
- [ ] Inventory data, code, logos, artwork, fonts, external dependency licenses.
- [ ] Quantify oncology-specific references and decide component-by-component reuse.
## Acceptance gates
- [ ] Pinned SHA, baseline report and component inventory committed.
- [ ] License matrix separates MIT software from NC dataset/assets.
- [ ] One reference example each for browse/detail/search/relations captured.
## Key risk / mitigation focus
R1: license/source boundary; R2: unable to reproduce baseline. Log evidence and deviations in the workplan review before marking complete.
## Evidence artifacts
Pull requests, test logs, pinned artifacts, screenshots or comparison report where applicable. Add explicit `ImplementationNotes.md` and `Review.md` on completion; do not mark a gate passed without a verifiable link.

27
workplans/WP-01.md Normal file
View file

@ -0,0 +1,27 @@
# WP-01 — Canonical contracts & 2-domain fixtures
**Purpose:** Establish source-independent knowledge projection and the zero-UI-code-change experiment.
**Dependencies:** see `INDEX.md`. **Status:** planned. **Owner:** to assign. **Estimate:** to refine after WP-00.
## Work packages
- [ ] Freeze v0.1 snapshot schema and manifest schema; document migrations.
- [ ] Model entity IDs, kinds, directed relations, evidence and claims in TS/Zod.
- [ ] Normalize systems-thinking and software-architecture fixtures with independently authored prose.
- [ ] Write duplicate, dangling, cyclic non-taxonomy and missing-source tests.
- [ ] Define policy for unknown and conflicting facts.
## Acceptance gates
- [ ] Both fixtures validate and supply ≥3 kinds and ≥8 entities each.
- [ ] A shared renderer can switch manifests with no domain vocabulary in UI source.
- [ ] Snapshot format documented and versioned.
## Key risk / mitigation focus
R3: abstractions erase domain semantics. Log evidence and deviations in the workplan review before marking complete.
## Evidence artifacts
Pull requests, test logs, pinned artifacts, screenshots or comparison report where applicable. Add explicit `ImplementationNotes.md` and `Review.md` on completion; do not mark a gate passed without a verifiable link.

28
workplans/WP-02.md Normal file
View file

@ -0,0 +1,28 @@
# WP-02 — UI extraction & experience skeleton
**Purpose:** Deliver the strongest OnCo-style interaction patterns on generic data.
**Dependencies:** see `INDEX.md`. **Status:** planned. **Owner:** to assign. **Estimate:** to refine after WP-00.
## Work packages
- [ ] Extract/port UI tokens, layout, header, hero and reusable table from pinned upstream.
- [ ] Create registry-based kind index and entity detail with no oncology imports.
- [ ] Adapt BrowserRow via a pure mapper and add configurable generic cell renderers.
- [ ] Preserve shareable URL query, facets and sorting; test browser back/forward.
- [ ] Implement keyboard search, empty/loading/error states and responsive layouts.
## Acceptance gates
- [ ] Browse/search/filter/sort/detail works for two dataspaces.
- [ ] Component dependency check prohibits `/oncology|cancer|drug|target/` domain imports in generic code.
- [ ] Copying upstream components retains provenance headers/notices.
- [ ] Keyboard and small-screen smoke tests pass.
## Key risk / mitigation focus
R4: extraction too expensive; use greenfield component where warranted. Log evidence and deviations in the workplan review before marking complete.
## Evidence artifacts
Pull requests, test logs, pinned artifacts, screenshots or comparison report where applicable. Add explicit `ImplementationNotes.md` and `Review.md` on completion; do not mark a gate passed without a verifiable link.

28
workplans/WP-03.md Normal file
View file

@ -0,0 +1,28 @@
# WP-03 — Ingestion, provenance & publishing
**Purpose:** Turn raw inputs into trustworthy reproducible snapshots.
**Dependencies:** see `INDEX.md`. **Status:** planned. **Owner:** to assign. **Estimate:** to refine after WP-00.
## Work packages
- [ ] Implement build-time JSON/NDJSON and Markdown frontmatter connector APIs.
- [ ] Make import idempotent with source digests and connector version.
- [ ] Resolve links in second pass; report dangling/conflicts; distinguish tombstones.
- [ ] Model source licenses, citations and attribution; quarantine unsupported content.
- [ ] Generate snapshot, manifest and machine/human import reports; atomic publication.
## Acceptance gates
- [ ] Two connector types produce identical normalized snapshots for equivalent input.
- [ ] Repeated ingest yields same content hash.
- [ ] Every published sourced claim has a retrievable evidence reference.
- [ ] Rights violations prevent publication.
## Key risk / mitigation focus
R5: rights metadata missing; quarantine instead of guessing. Log evidence and deviations in the workplan review before marking complete.
## Evidence artifacts
Pull requests, test logs, pinned artifacts, screenshots or comparison report where applicable. Add explicit `ImplementationNotes.md` and `Review.md` on completion; do not mark a gate passed without a verifiable link.

27
workplans/WP-04.md Normal file
View file

@ -0,0 +1,27 @@
# WP-04 — Relation explorer & knowledge journeys
**Purpose:** Make interconnected knowledge truly explorable.
**Dependencies:** see `INDEX.md`. **Status:** planned. **Owner:** to assign. **Estimate:** to refine after WP-00.
## Work packages
- [ ] Implement typed outgoing/incoming relation blocks and inverse labels.
- [ ] Build bounded interactive 1-2 hop graph and adjacency export.
- [ ] Add collections, related objects, reading path breadcrumbs.
- [ ] Introduce optional glossary tooltips and link previews.
- [ ] Handle cycles, multigraph, multi-parent hierarchy and disconnected components.
## Acceptance gates
- [ ] No graph relation omitted silently.
- [ ] Graph focus and detail navigation are reversible and deep linkable.
- [ ] Large graph safeguards tested with synthetic fixture.
## Key risk / mitigation focus
R6: large graph performance. Log evidence and deviations in the workplan review before marking complete.
## Evidence artifacts
Pull requests, test logs, pinned artifacts, screenshots or comparison report where applicable. Add explicit `ImplementationNotes.md` and `Review.md` on completion; do not mark a gate passed without a verifiable link.

31
workplans/WP-05.md Normal file
View file

@ -0,0 +1,31 @@
# WP-05 — Quality system & upstream experience parity
**Purpose:** Measure architectural independence and UX faithfulness.
**Dependencies:** see `INDEX.md`. **Status:** planned. **Owner:** to assign. **Estimate:** to refine after WP-00.
## Work packages
- [ ] Automate schema/ingest/ui/e2e tests and two-domain switch tests.
- [ ] Run accessibility, screenshot comparison, core web vitals and responsive audits.
- [ ] Report provenance completeness, stale counts, unresolved refs and ingestion failure classes.
- [ ] Document deviations from upstream and verify third-party notices.
- [ ] Harden inputs and public snapshot generation.
## Acceptance gates
- [ ] Tests run in CI; no material keyboard accessibility failures.
- [ ] Two unrelated domains pass all critical journeys.
- [ ] Source and license completeness report available.
## Key risk / mitigation focus
R7: tests approximate but do not prove usability. Log evidence and deviations in the workplan review before marking complete.
## Evidence artifacts
Pull requests, test logs, pinned artifacts, screenshots or comparison report where applicable. Add explicit `ImplementationNotes.md` and `Review.md` on completion; do not mark a gate passed without a verifiable link.
## Reference-oracle testing extension
Add WP-07: compare the *same permitted pinned OnCo slice* through the original implementation and generic Content Explorer, including behavioral journeys and visual snapshots. Public CI remains synthetic-only.

27
workplans/WP-06.md Normal file
View file

@ -0,0 +1,27 @@
# WP-06 — Extensibility & first real-world pilot
**Purpose:** Attach an independently maintained knowledge source and validate user value.
**Dependencies:** see `INDEX.md`. **Status:** planned. **Owner:** to assign. **Estimate:** to refine after WP-00.
## Work packages
- [ ] Implement third connector (GitHub issues or Wikidata after rights/rate review).
- [ ] Provide connector authoring template with fixture-based certification tests.
- [ ] Pilot one non-medical viable infospace with real users.
- [ ] Prioritize API/JSON-LD/MCP/ASK enhancements from observed needs.
- [ ] Define long-term update and governance responsibilities.
## Acceptance gates
- [ ] Third connector is refreshable and auditable.
- [ ] Pilot uses no hardcoded kinds in UI.
- [ ] Document informed go/no-go for dynamic deployment and search backend.
## Key risk / mitigation focus
R8: third-party API policies and data quality. Log evidence and deviations in the workplan review before marking complete.
## Evidence artifacts
Pull requests, test logs, pinned artifacts, screenshots or comparison report where applicable. Add explicit `ImplementationNotes.md` and `Review.md` on completion; do not mark a gate passed without a verifiable link.

16
workplans/WP-07.md Normal file
View file

@ -0,0 +1,16 @@
# WP-07 — OnCo licensed reference-slice conformance
**Purpose:** validate generic Content Explorer against real OnCo topology and the OnCo UX using a small pinned, authorized cancer-data slice, without copying the broader corpus.
**Dependencies:** WP-00 license/behavior audit; WP-01 model; WP-02 UI; WP-03 connector; WP-05 test harness. **Status:** planned and BLOCKED on data-use authorization.
## Tasks
- [ ] Determine test operator/use, acquire permission/license where required by OnCo `LICENSE-DATA`; document decision and redistribution restrictions.
- [ ] Pin upstream SHA and capture behavior for ~12–25 representative entities, 8+ user journeys, and several viewport sizes.
- [ ] Build allowlist-only `onco-reference` adapter from explicitly authorized local inputs to canonical snapshot, retaining provenance, dates, rights and record links.
- [ ] Implement graph topology, inverse-links, search/facet, detail, source display, stable deep-link and negative mutation tests.
- [ ] Run paired OnCo/Content Explorer Playwright tests against pinned source baseline; report semantic and visual divergences separately.
- [ ] Run clean-room public CI with synthetic domains and no actual OnCo dataset; audit that no OnCo corpus entered distributable packages/build artifacts.
## Gate
No real data import until authorization; no missing attribution or silent relation loss; version-pinned reproducibility; comparative conformance report and documented exceptions. For full process see `tests/onco-reference/README.md`.