feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001)
Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
2fd715ba45
commit
cb93c322c0
31 changed files with 5066 additions and 105 deletions
107
SCOPE.md
107
SCOPE.md
|
|
@ -8,130 +8,103 @@
|
|||
|
||||
## One-liner
|
||||
|
||||
<!-- Describe the purpose of this repository in one precise sentence. -->
|
||||
<!-- Example: "Provides a lightweight event router for Kubernetes-native systems." -->
|
||||
Headless PDF ingest that turns document bytes into engine-owned evidence
|
||||
contracts (fingerprint, canonical text, page/offset maps).
|
||||
|
||||
---
|
||||
|
||||
## Core Idea
|
||||
|
||||
<!-- What is the main capability or idea behind this repository? -->
|
||||
<!-- What problem does it try to solve? -->
|
||||
The rest of the citation-evidence ecosystem (anchoring, evidence linking,
|
||||
binders) needs a stable, runtime-agnostic way to go from *raw document bytes*
|
||||
to a `Document` + `DocumentRepresentation`. This repo owns that transformation
|
||||
for PDFs — pure over bytes, no persistence, no viewer, no UI framework.
|
||||
|
||||
---
|
||||
|
||||
## In Scope
|
||||
|
||||
<!-- What this repository is responsible for. -->
|
||||
<!-- Be explicit and concrete. -->
|
||||
|
||||
-
|
||||
-
|
||||
-
|
||||
- PDF byte ingest → `{ document, representation }` (`ingestPdf`)
|
||||
- PDF text extraction → canonical text + page map + gap-free offset map (`extractPdf`)
|
||||
- SHA-256 byte fingerprinting (`fingerprintBytes`)
|
||||
- Browser upload helpers behind a separate entry point (`createPdfByteStore`, `ingestPdfFromFile`)
|
||||
|
||||
---
|
||||
|
||||
## Out of Scope
|
||||
|
||||
<!-- What this repository deliberately does NOT do. -->
|
||||
<!-- This is often more important than "In Scope". -->
|
||||
|
||||
-
|
||||
-
|
||||
-
|
||||
- HTML / Markdown representations (deferred — see `workplans/`)
|
||||
- Persisting documents or representations (caller's job)
|
||||
- Viewer URL resolution and blob-vs-fixture policy (stays in `citation-evidence`)
|
||||
- Citation recovery / external source discovery (deferred)
|
||||
- Defining domain contracts — those are owned by `citation-engine`
|
||||
|
||||
---
|
||||
|
||||
## Relevant When
|
||||
|
||||
<!-- When should someone consider using or exploring this repository? -->
|
||||
|
||||
-
|
||||
-
|
||||
-
|
||||
- You have PDF bytes and need engine-shaped evidence contracts
|
||||
- You need a deterministic content fingerprint of document bytes
|
||||
- You are wiring a new consumer to ingest without pulling in a viewer
|
||||
|
||||
---
|
||||
|
||||
## Not Relevant When
|
||||
|
||||
<!-- When should someone ignore this repository? -->
|
||||
|
||||
-
|
||||
-
|
||||
-
|
||||
- You need HTML/Markdown ingest (not yet implemented)
|
||||
- You need viewer/session/UI behavior (see `citation-evidence`)
|
||||
- You are changing the `Document`/`DocumentRepresentation` contract (see `citation-engine`)
|
||||
|
||||
---
|
||||
|
||||
## Current State
|
||||
|
||||
<!-- Rough indication of maturity. No strict format required. -->
|
||||
|
||||
- Status: <!-- e.g. concept / experimental / active / stable / deprecated -->
|
||||
- Implementation: <!-- e.g. idea / partial / substantial / complete -->
|
||||
- Stability: <!-- e.g. unstable / evolving / stable -->
|
||||
- Usage: <!-- e.g. none / personal / internal / production -->
|
||||
|
||||
<!-- Add any notes that help set expectations. -->
|
||||
- Status: active
|
||||
- Implementation: partial (PDF slice implemented; HTML/MD and recovery deferred)
|
||||
- Stability: evolving
|
||||
- Usage: internal (consumed by `citation-evidence` across the repo boundary)
|
||||
|
||||
---
|
||||
|
||||
## How It Fits
|
||||
|
||||
<!-- Where does this repository sit in the bigger picture? -->
|
||||
|
||||
- Upstream dependencies:
|
||||
- Downstream consumers:
|
||||
- Often used with:
|
||||
- Upstream dependencies: `citation-engine` (`@citation-evidence/engine/shared`)
|
||||
- Downstream consumers: `citation-evidence` (umbrella app)
|
||||
- Often used with: `citation-engine`, `citation-evidence`
|
||||
|
||||
---
|
||||
|
||||
## Terminology
|
||||
|
||||
<!-- Terms that are important to understand this repo. -->
|
||||
<!-- Especially useful if naming differs from other repos. -->
|
||||
|
||||
- Preferred terms:
|
||||
- Also known as:
|
||||
- Potentially confusing terms:
|
||||
- Preferred terms: ingest, canonical text, fingerprint, representation
|
||||
- Also known as: "source" (its former location was `citation-evidence/src/source/`)
|
||||
- Potentially confusing terms: "representation" (a derived view of a document, not the document itself)
|
||||
|
||||
---
|
||||
|
||||
## Related / Overlapping Repositories
|
||||
|
||||
<!-- List repositories that have similar or adjacent responsibilities. -->
|
||||
<!-- Helps detect duplication and navigate the ecosystem. -->
|
||||
|
||||
- <repo-name> — <!-- how it relates -->
|
||||
- `citation-engine` — owns the shared `Document`/`DocumentRepresentation` contracts this repo produces
|
||||
- `citation-evidence` — umbrella app that consumes this ingest core; still owns viewer-url resolution
|
||||
|
||||
---
|
||||
|
||||
## Getting Oriented
|
||||
|
||||
<!-- If someone decides to look deeper, where should they start? -->
|
||||
|
||||
- Start with:
|
||||
- Key files / directories:
|
||||
- Entry points:
|
||||
- Start with: `README.md`, then `docs/ADR-0001-extraction-boundary.md`
|
||||
- Key files / directories: `src/pdf/` (headless core), `src/browser/` (upload helpers)
|
||||
- Entry points: `src/index.ts` (headless), `src/browser/index.ts` (browser)
|
||||
|
||||
---
|
||||
|
||||
## Provided Capabilities
|
||||
|
||||
<!-- What can this repo's domain provide to other domains on request? -->
|
||||
<!-- Each capability block is parsed by the state-hub capability catalog ingest. -->
|
||||
<!-- Remove the examples and add your own, or leave empty if none. -->
|
||||
|
||||
<!--
|
||||
```capability
|
||||
type: infrastructure
|
||||
title: Example capability title
|
||||
description: What this capability provides, in one or two sentences.
|
||||
keywords: [keyword1, keyword2, keyword3]
|
||||
```
|
||||
-->
|
||||
- `capability.infotech.pdf-evidence-ingest` — headless PDF → evidence contracts
|
||||
(see `registry/capabilities/`)
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
<!-- Anything else worth knowing. Keep it short. -->
|
||||
Fixture corpus is owned upstream by `citation-evidence` and resolved from a
|
||||
sibling checkout (ADR-0002); it is not duplicated here.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue