feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001)
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 2s

Bootstrap evidence-source from the citation-evidence src/source PDF slice.

- Headless core (src/pdf): ingest/extract/fingerprint, importing domain
  contracts from @citation-evidence/engine/shared (no local copies)
- Browser upload helpers isolated under src/browser behind a ./browser
  entry point, with an eslint boundary keeping the core browser-free
- pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism)
- Fixtures resolved from the sibling citation-evidence checkout, not
  duplicated (real PII) — see docs/ADR-0002; suites skip when absent
- Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002
- README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest
  registered, NO_CAPABILITIES removed
- Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-07-08 20:55:28 +02:00
parent 2fd715ba45
commit cb93c322c0
31 changed files with 5066 additions and 105 deletions

View file

@ -0,0 +1,78 @@
# ADR-0001 — PDF extraction boundary for `evidence-source`
- **Status:** accepted
- **Date:** 2026-07-08
- **Workplan:** ESRC-WP-0001 (T01)
## Context
`evidence-source` is being bootstrapped from the PDF slice that currently
lives in `../citation-evidence/src/source/`. That slice already implements a
working, tested headless ingest pipeline plus a set of browser/session helpers.
This ADR locks *which* files cross the boundary, *where* they land, and *which
contracts* they may depend on, so the extraction is mechanical rather than a
series of ad-hoc judgement calls.
The shared domain contracts (`Document`, `DocumentRepresentation`, `PageMap`,
`OffsetMap`, `newId`, `normalize`, …) are owned by `citation-engine` and are
published from its package as `@citation-evidence/engine/shared`. Nothing in
this extraction copies those types locally.
## Decision
### In-scope — headless ingest core (`src/pdf/`)
Moved into this repo immediately; these are pure, runtime-agnostic, and the
reason this repo exists:
| Upstream file | New home | Notes |
| --- | --- | --- |
| `src/source/pdf/ingest.ts` | `src/pdf/ingest.ts` | `ingestPdf` |
| `src/source/pdf/extract.ts` | `src/pdf/extract.ts` | `extractPdf` |
| `src/source/pdf/fingerprint.ts` | `src/pdf/fingerprint.ts` | `fingerprintBytes` |
The only change to these files is the import specifier: `@shared/*` becomes the
published package subpath `@citation-evidence/engine/shared`.
### In-scope but explicitly isolated — browser helpers (`src/browser/`)
Useful for browser upload flows, but *not* headless ingest. They are kept in
this repo behind a separate `./browser` entry point and documented as
browser-facing so they never blur into the headless core:
| Upstream file | New home | Notes |
| --- | --- | --- |
| `src/source/pdf/byte-store.ts` | `src/browser/byte-store.ts` | in-memory `blob:` byte store |
| `src/source/pdf/upload.ts` | `src/browser/upload.ts` | `ingestPdfFromFile` — thin wrapper over `ingestPdf` |
### Out of scope — stays in `citation-evidence`
| Upstream file | Reason |
| --- | --- |
| `src/source/pdf/viewer-url.ts` | Pure viewer/app concern: it encodes the umbrella's `/fixtures/pdfs/…` URL convention and the app's blob-vs-fixture fallback policy. It consumes `PdfByteStore` (a type this repo now owns) but the resolution *policy* belongs to the viewer. It stays upstream and imports the `PdfByteStore` type from this package. |
| `tests/integration/anchor-source-roundtrip.test.ts` | Cross-subsystem (source ↔ anchor) contract. Remains an umbrella integration test; it is not a private test of this repo. |
### Contracts
All domain contracts come from `@citation-evidence/engine/shared`. This repo
declares `@citation-evidence/engine` as a `link:../citation-engine` dependency
and never re-declares `Document`/`DocumentRepresentation`/etc. locally.
### Fixture corpus
The PDF fixture corpus (`../citation-evidence/fixtures/pdfs/`) is **not** copied
into this repo. The files are real personal/legal documents (utility statements,
court letters, admission forms) containing PII; duplicating them into a package
that advertises reusable ingest capability is undesirable. Instead this repo's
tests resolve the corpus from the sibling checkout (default
`../citation-evidence/fixtures/pdfs`, override via
`EVIDENCE_SOURCE_FIXTURE_DIR`). This mirrors the sibling-checkout `link:` model
already used for the engine dependency and keeps the corpus single-owned
upstream. See ADR-0002.
## Consequences
- The extraction is a 5-file move (3 core + 2 browser) plus import rewrites.
- `citation-evidence` keeps `viewer-url.ts` and the round-trip integration test.
- Tests here require a sibling `citation-evidence` checkout for fixtures — an
accepted coupling, identical in spirit to the engine `link:` dependency.

View file

@ -0,0 +1,43 @@
# ADR-0002 — Fixture corpus ownership
- **Status:** accepted
- **Date:** 2026-07-08
- **Workplan:** ESRC-WP-0001 (T05)
## Context
The PDF ingest contract tests are driven by a fixture corpus
(`fixtures/pdfs/` + `manifest.json`) currently owned by `citation-evidence`.
The corpus is made of **real** personal and legal documents — utility-cost
statements, a settlement letter, court correspondence, cemetery admission
forms — containing names, addresses, and case details.
The workplan (T05) asks us to either copy the corpus into this repo or define a
stable shared-fixture path.
## Decision
Do **not** copy the corpus into `evidence-source`. Resolve it from the sibling
`citation-evidence` checkout instead:
- default: `<repo>/../citation-evidence/fixtures/pdfs`
- override: `EVIDENCE_SOURCE_FIXTURE_DIR` environment variable
## Rationale
- **Privacy.** `evidence-source` advertises reusable ingest capability and is a
candidate for wider distribution. Duplicating real PII into it multiplies the
places that data lives and can be leaked.
- **Single ownership.** The corpus and its `manifest.json` (page counts,
known-good quotes) were curated and re-verified upstream. One owner avoids
drift between two copies.
- **Consistent coupling.** This repo already assumes a sibling checkout for its
`@citation-evidence/engine` (`link:../citation-engine`) dependency. Reading
fixtures from `../citation-evidence` is the same coupling model, not a new one.
## Consequences
- Tests require a sibling `citation-evidence` checkout. In CI or a standalone
clone, point `EVIDENCE_SOURCE_FIXTURE_DIR` at wherever the corpus is mounted.
- When the corpus is absent, the fixture-driven suites skip with a clear message
rather than failing spuriously, so unit tests that need no corpus still run.