content-explorer/architecture/ConnectorContract.md

# Connector protocol and governance

**Adapter boundary:** `source format -> candidate entities, claims, relations, sources -> validated canonical snapshot`. Connectors must not render or import React components.

## Minimum connector metadata

`id`, `version`, `sourceNamespace`, `capabilities`, `licensePolicy`, `mappingVersion`, `refreshMode`, `sourcePriority`, `lastCheckpoint`.

## Functions

- `discover(config)` returns stable enumerations if supported.
- `fetch(config, checkpoint)` yields batches with immutable locators and input digests.
- `map(batch)` emits candidates and issues; preserve raw provenance.
- `validate(candidates, manifest)` checks shape, IDs, rights, provenance, references.
- `commit(validated, priorSnapshot)` atomically publishes snapshot/report or rolls back.

## Candidate mapping

Input → `source record` → `canonical id` → `kind` → `fields/claims` → `relations`. Relationships may be forward references; resolve in a second pass. No fuzzy join without explicit mapping policy. Respect deletion semantics by source ownership. Retries + replay must converge.

## First connectors

1. **JSON/NDJSON**: local fixture ingestion. The build-time connector is `content_explorer/ingest.py` (`load_json_records`, `load_ndjson_records`). `connectors/json_fixture.py` only reads an already published snapshot.
2. **Markdown+frontmatter**: repo text with typed metadata. The same module loads it with PyYAML (`load_markdown_records`). Remote static files and wiki-style references are not implemented yet.
3. **GitHub or Wikidata** (P2): authenticated/rate-limited external API snapshot; mapping rules, pagination, ETag and license review before activation.

## Required diagnostics

`runId`, connector/version, started/ended UTC, input digests, source counts, candidates, accepted, updated, tombstoned, quarantined, warning/error codes, rights blocks, unresolved target IDs and published snapshot digest. Generate a human-readable provenance/rights report with each snapshot.
Seeded repo 2026-10-12 18:28:16 +02:00			`# Connector protocol and governance`

			Adapter boundary: `source format -> candidate entities, claims, relations, sources -> validated canonical snapshot`. Connectors must not render or import React components.

			`## Minimum connector metadata`

			`id`, `version`, `sourceNamespace`, `capabilities`, `licensePolicy`, `mappingVersion`, `refreshMode`, `sourcePriority`, `lastCheckpoint`.

			`## Functions`

			- `discover(config)` returns stable enumerations if supported.
			- `fetch(config, checkpoint)` yields batches with immutable locators and input digests.
			- `map(batch)` emits candidates and issues; preserve raw provenance.
			- `validate(candidates, manifest)` checks shape, IDs, rights, provenance, references.
			- `commit(validated, priorSnapshot)` atomically publishes snapshot/report or rolls back.

			`## Candidate mapping`

			Input → `source record` → `canonical id` → `kind` → `fields/claims` → `relations`. Relationships may be forward references; resolve in a second pass. No fuzzy join without explicit mapping policy. Respect deletion semantics by source ownership. Retries + replay must converge.

			`## First connectors`

Freeze the v0.1 contract and the local ingest connector. The snapshot and manifest schemas, both example dataspaces, and the shared text renderer now pass the fixture validator and the negative contract tests. JSON, NDJSON, and Markdown frontmatter publish a snapshot only when rights, identity, and references allow it. CONTENT-WP-0001 and CONTENT-WP-0003 are marked finished against those artifacts. CONTENT-WP-0000-T04 stays open. SCOPE.md and the 19:05 scope-against-intent note record the stop. Assistant: grok Assistant-Session: 01a130f1-abfb-7d71-9cd6-a8b75ec7f95a 2026-10-12 19:07:32 +02:00			1. JSON/NDJSON: local fixture ingestion. The build-time connector is `content_explorer/ingest.py` (`load_json_records`, `load_ndjson_records`). `connectors/json_fixture.py` only reads an already published snapshot.
			2. Markdown+frontmatter: repo text with typed metadata. The same module loads it with PyYAML (`load_markdown_records`). Remote static files and wiki-style references are not implemented yet.
Seeded repo 2026-10-12 18:28:16 +02:00			`3. GitHub or Wikidata (P2): authenticated/rate-limited external API snapshot; mapping rules, pagination, ETag and license review before activation.`

			`## Required diagnostics`

			`runId`, connector/version, started/ended UTC, input digests, source counts, candidates, accepted, updated, tombstoned, quarantined, warning/error codes, rights blocks, unresolved target IDs and published snapshot digest. Generate a human-readable provenance/rights report with each snapshot.