29 lines
1.9 KiB
Markdown
29 lines
1.9 KiB
Markdown
# Connector protocol and governance
|
|
|
|
**Adapter boundary:** `source format -> candidate entities, claims, relations, sources -> validated canonical snapshot`. Connectors must not render or import React components.
|
|
|
|
## Minimum connector metadata
|
|
|
|
`id`, `version`, `sourceNamespace`, `capabilities`, `licensePolicy`, `mappingVersion`, `refreshMode`, `sourcePriority`, `lastCheckpoint`.
|
|
|
|
## Functions
|
|
|
|
- `discover(config)` returns stable enumerations if supported.
|
|
- `fetch(config, checkpoint)` yields batches with immutable locators and input digests.
|
|
- `map(batch)` emits candidates and issues; preserve raw provenance.
|
|
- `validate(candidates, manifest)` checks shape, IDs, rights, provenance, references.
|
|
- `commit(validated, priorSnapshot)` atomically publishes snapshot/report or rolls back.
|
|
|
|
## Candidate mapping
|
|
|
|
Input → `source record` → `canonical id` → `kind` → `fields/claims` → `relations`. Relationships may be forward references; resolve in a second pass. No fuzzy join without explicit mapping policy. Respect deletion semantics by source ownership. Retries + replay must converge.
|
|
|
|
## First connectors
|
|
|
|
1. **JSON/NDJSON**: local fixture and remote static file ingestion (local first). A simple reference mapping implementation is in `connectors/json_fixture.py`.
|
|
2. **Markdown+frontmatter**: repo text with typed metadata and wiki-style references; parse via a robust library, not ad-hoc regex in production. Preserve headings/source locations and content license.
|
|
3. **GitHub or Wikidata** (P2): authenticated/rate-limited external API snapshot; mapping rules, pagination, ETag and license review before activation.
|
|
|
|
## Required diagnostics
|
|
|
|
`runId`, connector/version, started/ended UTC, input digests, source counts, candidates, accepted, updated, tombstoned, quarantined, warning/error codes, rights blocks, unresolved target IDs and published snapshot digest. Generate a human-readable provenance/rights report with each snapshot.
|