evidence-source/SCOPE.md

# SCOPE

> This file helps you quickly understand what this repository is about,
> when it is relevant, and when it is not.
> It is intentionally lightweight and may be incomplete.

---

## One-liner

Headless document ingest that turns PDF, HTML, and Markdown bytes into
engine-owned evidence contracts (fingerprint, canonical text, page/offset maps).

---

## Core Idea

The rest of the citation-evidence ecosystem (anchoring, evidence linking,
binders) needs a stable, runtime-agnostic way to go from *raw document bytes*
to a `Document` + `DocumentRepresentation`. This repo owns that transformation
for PDF, HTML, and Markdown — pure over bytes, no persistence, no viewer,
no UI framework.

---

## In Scope

- PDF byte ingest → `{ document, representation }` (`ingestPdf`)
- HTML byte ingest → `{ document, representation }` (`ingestHtml`)
- Markdown byte ingest → `{ document, representation }` (`ingestMarkdown`)
- PDF text extraction → canonical text + page map + gap-free offset map (`extractPdf`)
- PDF intrinsic metadata extraction with caller-wins merge (`extractPdfMetadata`)
- SHA-256 byte fingerprinting (`fingerprintBytes`)
- Citation recovery primitives: re-ingest/reconcile, local quote search, discovery hooks
- Browser upload helpers behind a separate entry point (`createPdfByteStore`, `ingestPdfFromFile`)

---

## Out of Scope

- Persisting documents or representations (caller's job)
- Viewer URL resolution and blob-vs-fixture policy (stays in `citation-evidence`)
- Stale selector detection and anchor resolution (stays in `evidence-anchor`)
- Network-backed external source discovery implementations (host registers hooks)
- Defining domain contracts — those are owned by `citation-engine`

---

## Relevant When

- You have PDF bytes and need engine-shaped evidence contracts
- You need a deterministic content fingerprint of document bytes
- You are wiring a new consumer to ingest without pulling in a viewer

---

## Not Relevant When

- You need formats beyond PDF/HTML/Markdown (not yet implemented)
- You need viewer/session/UI behavior (see `citation-evidence`)
- You are changing the `Document`/`DocumentRepresentation` contract (see `citation-engine`)

---

## Current State

- Status: active
- Implementation: partial (PDF/HTML/MD ingest, metadata enrichment, recovery primitives)
- Stability: evolving
- Usage: internal (consumed by `citation-evidence` across the repo boundary)

---

## How It Fits

- Upstream dependencies: `citation-engine` (`@citation-evidence/engine/shared`)
- Downstream consumers: `citation-evidence` (umbrella app)
- Often used with: `citation-engine`, `citation-evidence`

---

## Terminology

- Preferred terms: ingest, canonical text, fingerprint, representation
- Also known as: "source" (its former location was `citation-evidence/src/source/`)
- Potentially confusing terms: "representation" (a derived view of a document, not the document itself)

---

## Related / Overlapping Repositories

- `citation-engine` — owns the shared `Document`/`DocumentRepresentation` contracts this repo produces
- `citation-evidence` — umbrella app that consumes this ingest core; still owns viewer-url resolution

---

## Getting Oriented

- Start with: `README.md`, then `docs/ADR-0001-extraction-boundary.md`
- Key files / directories: `src/pdf/`, `src/html/`, `src/markdown/`, `src/recovery/`, `src/browser/`
- Entry points: `src/index.ts` (headless), `src/browser/index.ts` (browser)

---

## Provided Capabilities

- `capability.infotech.pdf-evidence-ingest` — headless PDF → evidence contracts
  (see `registry/capabilities/`)

---

## Notes

Fixture corpus is owned upstream by `citation-evidence` and resolved from a
sibling checkout (ADR-0002); it is not duplicated here.