feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001)
Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
2fd715ba45
commit
cb93c322c0
31 changed files with 5066 additions and 105 deletions
78
docs/ADR-0001-extraction-boundary.md
Normal file
78
docs/ADR-0001-extraction-boundary.md
Normal file
|
|
@ -0,0 +1,78 @@
|
|||
# ADR-0001 — PDF extraction boundary for `evidence-source`
|
||||
|
||||
- **Status:** accepted
|
||||
- **Date:** 2026-07-08
|
||||
- **Workplan:** ESRC-WP-0001 (T01)
|
||||
|
||||
## Context
|
||||
|
||||
`evidence-source` is being bootstrapped from the PDF slice that currently
|
||||
lives in `../citation-evidence/src/source/`. That slice already implements a
|
||||
working, tested headless ingest pipeline plus a set of browser/session helpers.
|
||||
This ADR locks *which* files cross the boundary, *where* they land, and *which
|
||||
contracts* they may depend on, so the extraction is mechanical rather than a
|
||||
series of ad-hoc judgement calls.
|
||||
|
||||
The shared domain contracts (`Document`, `DocumentRepresentation`, `PageMap`,
|
||||
`OffsetMap`, `newId`, `normalize`, …) are owned by `citation-engine` and are
|
||||
published from its package as `@citation-evidence/engine/shared`. Nothing in
|
||||
this extraction copies those types locally.
|
||||
|
||||
## Decision
|
||||
|
||||
### In-scope — headless ingest core (`src/pdf/`)
|
||||
|
||||
Moved into this repo immediately; these are pure, runtime-agnostic, and the
|
||||
reason this repo exists:
|
||||
|
||||
| Upstream file | New home | Notes |
|
||||
| --- | --- | --- |
|
||||
| `src/source/pdf/ingest.ts` | `src/pdf/ingest.ts` | `ingestPdf` |
|
||||
| `src/source/pdf/extract.ts` | `src/pdf/extract.ts` | `extractPdf` |
|
||||
| `src/source/pdf/fingerprint.ts` | `src/pdf/fingerprint.ts` | `fingerprintBytes` |
|
||||
|
||||
The only change to these files is the import specifier: `@shared/*` becomes the
|
||||
published package subpath `@citation-evidence/engine/shared`.
|
||||
|
||||
### In-scope but explicitly isolated — browser helpers (`src/browser/`)
|
||||
|
||||
Useful for browser upload flows, but *not* headless ingest. They are kept in
|
||||
this repo behind a separate `./browser` entry point and documented as
|
||||
browser-facing so they never blur into the headless core:
|
||||
|
||||
| Upstream file | New home | Notes |
|
||||
| --- | --- | --- |
|
||||
| `src/source/pdf/byte-store.ts` | `src/browser/byte-store.ts` | in-memory `blob:` byte store |
|
||||
| `src/source/pdf/upload.ts` | `src/browser/upload.ts` | `ingestPdfFromFile` — thin wrapper over `ingestPdf` |
|
||||
|
||||
### Out of scope — stays in `citation-evidence`
|
||||
|
||||
| Upstream file | Reason |
|
||||
| --- | --- |
|
||||
| `src/source/pdf/viewer-url.ts` | Pure viewer/app concern: it encodes the umbrella's `/fixtures/pdfs/…` URL convention and the app's blob-vs-fixture fallback policy. It consumes `PdfByteStore` (a type this repo now owns) but the resolution *policy* belongs to the viewer. It stays upstream and imports the `PdfByteStore` type from this package. |
|
||||
| `tests/integration/anchor-source-roundtrip.test.ts` | Cross-subsystem (source ↔ anchor) contract. Remains an umbrella integration test; it is not a private test of this repo. |
|
||||
|
||||
### Contracts
|
||||
|
||||
All domain contracts come from `@citation-evidence/engine/shared`. This repo
|
||||
declares `@citation-evidence/engine` as a `link:../citation-engine` dependency
|
||||
and never re-declares `Document`/`DocumentRepresentation`/etc. locally.
|
||||
|
||||
### Fixture corpus
|
||||
|
||||
The PDF fixture corpus (`../citation-evidence/fixtures/pdfs/`) is **not** copied
|
||||
into this repo. The files are real personal/legal documents (utility statements,
|
||||
court letters, admission forms) containing PII; duplicating them into a package
|
||||
that advertises reusable ingest capability is undesirable. Instead this repo's
|
||||
tests resolve the corpus from the sibling checkout (default
|
||||
`../citation-evidence/fixtures/pdfs`, override via
|
||||
`EVIDENCE_SOURCE_FIXTURE_DIR`). This mirrors the sibling-checkout `link:` model
|
||||
already used for the engine dependency and keeps the corpus single-owned
|
||||
upstream. See ADR-0002.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The extraction is a 5-file move (3 core + 2 browser) plus import rewrites.
|
||||
- `citation-evidence` keeps `viewer-url.ts` and the round-trip integration test.
|
||||
- Tests here require a sibling `citation-evidence` checkout for fixtures — an
|
||||
accepted coupling, identical in spirit to the engine `link:` dependency.
|
||||
43
docs/ADR-0002-fixture-ownership.md
Normal file
43
docs/ADR-0002-fixture-ownership.md
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
# ADR-0002 — Fixture corpus ownership
|
||||
|
||||
- **Status:** accepted
|
||||
- **Date:** 2026-07-08
|
||||
- **Workplan:** ESRC-WP-0001 (T05)
|
||||
|
||||
## Context
|
||||
|
||||
The PDF ingest contract tests are driven by a fixture corpus
|
||||
(`fixtures/pdfs/` + `manifest.json`) currently owned by `citation-evidence`.
|
||||
The corpus is made of **real** personal and legal documents — utility-cost
|
||||
statements, a settlement letter, court correspondence, cemetery admission
|
||||
forms — containing names, addresses, and case details.
|
||||
|
||||
The workplan (T05) asks us to either copy the corpus into this repo or define a
|
||||
stable shared-fixture path.
|
||||
|
||||
## Decision
|
||||
|
||||
Do **not** copy the corpus into `evidence-source`. Resolve it from the sibling
|
||||
`citation-evidence` checkout instead:
|
||||
|
||||
- default: `<repo>/../citation-evidence/fixtures/pdfs`
|
||||
- override: `EVIDENCE_SOURCE_FIXTURE_DIR` environment variable
|
||||
|
||||
## Rationale
|
||||
|
||||
- **Privacy.** `evidence-source` advertises reusable ingest capability and is a
|
||||
candidate for wider distribution. Duplicating real PII into it multiplies the
|
||||
places that data lives and can be leaked.
|
||||
- **Single ownership.** The corpus and its `manifest.json` (page counts,
|
||||
known-good quotes) were curated and re-verified upstream. One owner avoids
|
||||
drift between two copies.
|
||||
- **Consistent coupling.** This repo already assumes a sibling checkout for its
|
||||
`@citation-evidence/engine` (`link:../citation-engine`) dependency. Reading
|
||||
fixtures from `../citation-evidence` is the same coupling model, not a new one.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Tests require a sibling `citation-evidence` checkout. In CI or a standalone
|
||||
clone, point `EVIDENCE_SOURCE_FIXTURE_DIR` at wherever the corpus is mounted.
|
||||
- When the corpus is absent, the fixture-driven suites skip with a clear message
|
||||
rather than failing spuriously, so unit tests that need no corpus still run.
|
||||
Loading…
Add table
Add a link
Reference in a new issue