Seeded repo

architecture/ArchitectureBlueprint.md

@@ -0,0 +1,62 @@
+# Architecture blueprint v0.1
+
+## Bounded context
+
+`content-explorer` is a **read/explore projection** over one or more source dataspaces; not an authoritative universal knowledge graph. Source adapters produce candidate facts; a normalization and governance pipeline publishes immutable queryable snapshots. The interface sees only the snapshot contract and a declarative presentation manifest.
+
+```text
+Git Markdown ─┐
+JSON/NDJSON ──┼─> connector adapters ─> typed staging ─> identity/rights/evidence ─> graph validator ─> snapshot
+future API ───┘                           |                       |               |
+                                       quarantine              report        manifest
+                                                                               |
+                        +---------------- static JSON/read API ----------------+
+                        |                              |                       |
+                     Next.js UI                 search/graph views           future CLI/MCP
+```
+
+## Canonical layers
+
+1. **Source artifact**: exact immutable locator and content hash, acquired/observed timestamp, origin license, optional source-specific ID.
+2. **Assertion/Claim**: typed subject–predicate–object or field value, linked evidence, status, confidence (if available), temporal `validFrom`/`validTo`, provenance and extractor version. Attribute displayed on entity may be an aggregation of claims; don't discard disagreement.
+3. **Entity/Relation projection**: globally qualified stable identifiers, domain-local kind IDs, labels, TL;DR, summary, curated properties, typed relations and incoming derived backlinks. Nonsemantic `related` edges are allowed but distinguished.
+4. **Presentation**: dataspace manifest defines allowed kinds, labels, facets, grouping, column specs, navigation, field layout, cards and optional widgets.
+5. **Transport/export**: versioned snapshot JSON and OpenAPI later; no raw connector type reaches the UI.
+
+## Identity
+
+Canonical IDs: `urn:ce:<dataspace>:<kind>:<key>` using normalized escaped segments or deterministic hashes when keys cannot meet the grammar. External IDs live separately as aliases. Two identical upstream IDs in different spaces cannot collide. Same-as merging is explicit with mapping confidence and curator decision; it must not happen from name similarity alone. Preserve source native identifiers for replay.
+
+## Relations
+
+Each relation has its own ID, source, target, predicate (manifest-defined), labels for both directions, evidence and time. Store directed edges only once; derive reverse navigation. Allow many-to-many, multiple relation types between nodes, multi-parent hierarchies and cycles; selectively enforce acyclic property for declared taxonomic predicates only.
+
+## Claims, truth and evidence
+
+Claims are source statements, not verified universal facts. Each projected display field optionally refers to one or more claim IDs; policies choose `preferred`, `show-all`, `contested` or `withhold`. Evidence includes source URI, document title, retrieved/verified timestamps, optional location/quote, license and attribution. UI exposes missing versus false versus stale versus disputed, without fake confidence scores.
+
+## Connector protocol
+
+`discover() -> SourceDescriptor[]`; `fetch(cursor?) -> SourceBatch`; `project(batch, mapping) -> ImportCandidate[]`; `checkpoint()` for deterministic repeat. Adapter declares capabilities (full snapshot vs incremental, read-only, rights metadata, deletions), sync semantics, source authority and supported MIME/media types. Every run persists: connector version, input digest, output digest, insert/update/tombstone/quarantine counts and diagnostics. In v0.1 implement as build-time CLI jobs, no service dependency.
+
+**Ingestion modes:** authoritative snapshot: missing source records tombstoned only within that source's ownership boundary; incremental: delete only on explicit tombstone; replay: same batch is idempotent. Namespace assertions per connector+source and preserve change history in report. Fail closed for forbidden rights and known dangling references, quarantine optional low-integrity records when policy allows.
+
+## Configurable views
+
+Manifest owns `home`, `kinds`, `relations`, `navigation`, `facets`, `columns`, `displayPanels`, `readingPaths`, `locale`, `theme`. UI components know generic `Entity`, `Facet`, `Claim`, `Link`, `Collection`; any widget requiring domain-specific fields is an optional registered plugin with schema validation and fallback rendering.
+
+## Trust and deployment
+
+Begin with static snapshots generated in CI for reproducibility, modest cost and safe public read. Source data is untrusted input: URL allowlist/scheme checks; HTML sanitization; bounded document size; no dynamic evaluation; prevent SSRF in future remote fetch adapters. No personal/sensitive data for early public demo. Record source copyright/attribution and policy before publishing. Signed/hashed snapshots and dependency scanning before production.
+
+## Quality measures
+
+- Model independence: 2 domain manifests, identical app build; zero code changes between.
+- Graph integrity: 100% IDs resolved or quarantined with diagnostics; no silent edge drops.
+- Provenance: 100% externally sourced displayed claims traceable to a source and rights record.
+- Navigation: browse, filter, search, detail, backlink, source, browser back and deep links pass e2e.
+- Performance budgets to baseline: static initial view and graph neighborhood size; set numeric SLOs after measuring actual upstream baseline.
+
+## Architectural trade-offs
+
+Start with strongly typed **core envelope** and flexible per-kind properties; resist a rigid cross-domain ontology. Treat external RDF/JSON-LD as mapping targets, not prerequisite storage. Avoid graph DB until query or volume evidence demands one. Persist raw inputs separately from normalized snapshots. Never equate source trust with correctness of individual assertions.

architecture/CanonicalDataModel.md

@@ -0,0 +1,25 @@
+# Canonical data model — draft contract v0.1
+
+The JSON Schema in `contracts/snapshot.schema.json` is a **first executable draft**. It intentionally models a compact display projection; Claim/Evidence and policy semantics in this document are richer than the initial schema and must be implemented in WP-03.
+
+## Records
+
+- `Dataspace`: namespace, version, title, description, schemaVersion, language, source policy, kind registry, navigation, view registry.
+- `Entity`: ID, kindId, title, aliases, tldr, summary, tags, properties, effective time, sourceRefs, claimRefs, state.
+- `Relation`: id, predicate, from/to, evidenceRefs, asOf, sourceRef; direction preserved.
+- `Claim`: id, subject, key or predicate, value/target, assertedBy, evidenceRefs, status, valid interval, observedAt; no mandatory scalar probability.
+- `Source`: id, canonical URI, title, author/publisher, retrievedAt, publishedAt, license, attribution, contentHash.
+- `Collection`: ordered set of entity IDs + editorial purpose, optionally computed from a view.
+- `View`: declarative facets, columns, sorts, summary cards, panels and graph traversal policy.
+
+## Projection semantics
+
+A `view` never changes truth; it chooses presentation. Projected `Entity.summary` is a **display decision** (may be curated), and `Claim`s retain competing assertions. Edges without provenance can be shown as `unverified` but must not be presented as sourced fact. All counts derive from the snapshot and query state.
+
+## Mappings versus ontology
+
+Types such as `concept`, `person`, `organization`, `document`, `pattern`, `decision`, `problem` are example `kindId`s, not privileged ontological primitives. The core representation is open-world and domain-specific names live in the manifest. A future vocabulary layer can connect manifests to SKOS/schema.org/RDF terms when useful.
+
+## Future-proofs
+
+Versioned migrators (`schemaVersion`), stable source-scoped IDs, lifecycle tombstones, localizations, role/perspective-aware visibility, hierarchical collection membership, temporal relations, claim-level review state, conflict handling, event stream, and computation provenance.

architecture/ConnectorContract.md

@@ -0,0 +1,29 @@
+# Connector protocol and governance
+
+**Adapter boundary:** `source format -> candidate entities, claims, relations, sources -> validated canonical snapshot`. Connectors must not render or import React components.
+
+## Minimum connector metadata
+
+`id`, `version`, `sourceNamespace`, `capabilities`, `licensePolicy`, `mappingVersion`, `refreshMode`, `sourcePriority`, `lastCheckpoint`.
+
+## Functions
+
+- `discover(config)` returns stable enumerations if supported.
+- `fetch(config, checkpoint)` yields batches with immutable locators and input digests.
+- `map(batch)` emits candidates and issues; preserve raw provenance.
+- `validate(candidates, manifest)` checks shape, IDs, rights, provenance, references.
+- `commit(validated, priorSnapshot)` atomically publishes snapshot/report or rolls back.
+
+## Candidate mapping
+
+Input → `source record` → `canonical id` → `kind` → `fields/claims` → `relations`. Relationships may be forward references; resolve in a second pass. No fuzzy join without explicit mapping policy. Respect deletion semantics by source ownership. Retries + replay must converge.
+
+## First connectors
+
+1. **JSON/NDJSON**: local fixture and remote static file ingestion (local first). A simple reference mapping implementation is in `connectors/json_fixture.py`.
+2. **Markdown+frontmatter**: repo text with typed metadata and wiki-style references; parse via a robust library, not ad-hoc regex in production. Preserve headings/source locations and content license.
+3. **GitHub or Wikidata** (P2): authenticated/rate-limited external API snapshot; mapping rules, pagination, ETag and license review before activation.
+
+## Required diagnostics
+
+`runId`, connector/version, started/ended UTC, input digests, source counts, candidates, accepted, updated, tombstoned, quarantined, warning/error codes, rights blocks, unresolved target IDs and published snapshot digest. Generate a human-readable provenance/rights report with each snapshot.