evidence-source/README.md

# evidence-source

Headless document ingest for the citation-evidence ecosystem. Turns raw PDF,
HTML, and Markdown bytes into engine-owned evidence contracts — a `Document`
(media type, SHA-256 fingerprint, optional title/uri/metadata) and a
`DocumentRepresentation` (canonical text, page/offset maps where applicable).
Ingest is pure over bytes: no persistence, no viewer state, no React.

## Status

**Implemented:** PDF ingest (ESRC-WP-0001), HTML/Markdown ingest
(ESRC-WP-0002), PDF metadata enrichment (ESRC-WP-0003), and citation recovery
primitives (ESRC-WP-0004). See `workplans/` for history.

## Install

Requires a sibling checkout of `citation-engine` (domain contracts) and, for
tests, `citation-evidence` (fixture corpus — see `docs/ADR-0002-fixture-ownership.md`).

```bash
pnpm install   # resolves @citation-evidence/engine via link:../citation-engine
```

## Usage

```ts
import {
  ingestPdf,
  ingestHtml,
  ingestMarkdown,
} from "@citation-evidence/evidence-source";

const pdf = await ingestPdf(pdfBytes, { filename: "contract.pdf" });
const html = await ingestHtml(htmlBytes, { filename: "brief.html" });
const md = await ingestMarkdown("# Title\n\nBody text.", { filename: "notes.md" });
// document.fingerprint -> SHA-256 hex
// representation.canonicalText / pageMap / offsetMap (format-dependent)
```

Browser upload helpers (in-memory `blob:` byte store + `ingestPdfFromFile`)
live behind a separate, clearly-isolated entry point so headless consumers do
not pull in browser machinery:

```ts
import { createPdfByteStore, ingestPdfFromFile }
  from "@citation-evidence/evidence-source/browser";
```

Extraction requires the host to configure the PDF.js worker
(`GlobalWorkerOptions.workerSrc`) before calling `extractPdf`/`ingestPdf`; the
module itself does no worker setup so it loads cleanly in Node and browsers.

## Dev commands

```bash
pnpm test        # vitest (fixture suites skip if the corpus is absent)
pnpm typecheck   # tsc --noEmit
pnpm lint        # eslint
```

Point `EVIDENCE_SOURCE_FIXTURE_DIR` at the PDF corpus when the sibling
`citation-evidence` checkout is not at `../citation-evidence`.

## Architecture

- `src/pdf/` — PDF ingest, extraction, fingerprinting, metadata enrichment.
- `src/html/`, `src/markdown/` — reflowable-format ingest (ADR-0003).
- `src/recovery/` — re-ingest/reconcile, local quote search, discovery hooks (ADR-0004).
- `src/browser/` — browser upload surface: `byte-store`, `upload`. Separated by
  an ESLint boundary so the core never imports it.
- Domain contracts come from `@citation-evidence/engine/shared`; none are
  copied locally.
- Boundary and fixture decisions: `docs/ADR-0001` through `docs/ADR-0004`.

`viewer-url` resolution stays in the consuming app (`citation-evidence`), which
imports this package's ingest core through a thin façade.