feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001)
Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
2fd715ba45
commit
cb93c322c0
31 changed files with 5066 additions and 105 deletions
|
|
@ -4,7 +4,7 @@ type: workplan
|
|||
title: "Bootstrap standalone source repo from citation-evidence PDF slice"
|
||||
domain: infotech
|
||||
repo: evidence-source
|
||||
status: ready
|
||||
status: finished
|
||||
owner: codex
|
||||
topic_slug: citation_evidence_mvp
|
||||
created: "2026-06-21"
|
||||
|
|
@ -64,7 +64,7 @@ T01 extraction boundary and import plan
|
|||
|
||||
```task
|
||||
id: ESRC-WP-0001-T01
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
|
|
@ -93,7 +93,7 @@ the extraction relies on copying `citation-engine` shared types locally.
|
|||
|
||||
```task
|
||||
id: ESRC-WP-0001-T02
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
|
|
@ -119,7 +119,7 @@ runtime host.
|
|||
|
||||
```task
|
||||
id: ESRC-WP-0001-T03
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
|
|
@ -147,7 +147,7 @@ Acceptance: a consumer can pass PDF bytes to this repo and receive a valid
|
|||
|
||||
```task
|
||||
id: ESRC-WP-0001-T04
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
|
|
@ -176,7 +176,7 @@ browser/session-only behavior after extraction.
|
|||
|
||||
```task
|
||||
id: ESRC-WP-0001-T05
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
|
|
@ -203,7 +203,7 @@ source-to-anchor round-trip remains an umbrella integration test.
|
|||
|
||||
```task
|
||||
id: ESRC-WP-0001-T06
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
|
|
@ -227,7 +227,7 @@ package rather than the local `src/source/` copy for the PDF slice.
|
|||
|
||||
```task
|
||||
id: ESRC-WP-0001-T07
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
```
|
||||
|
||||
|
|
|
|||
55
workplans/ESRC-WP-0002-html-markdown-representations.md
Normal file
55
workplans/ESRC-WP-0002-html-markdown-representations.md
Normal file
|
|
@ -0,0 +1,55 @@
|
|||
---
|
||||
id: ESRC-WP-0002
|
||||
type: workplan
|
||||
title: "HTML and Markdown source representations"
|
||||
domain: infotech
|
||||
repo: evidence-source
|
||||
status: proposed
|
||||
owner: codex
|
||||
topic_slug: citation_evidence_mvp
|
||||
created: "2026-07-08"
|
||||
updated: "2026-07-08"
|
||||
spec_refs:
|
||||
- README.md
|
||||
- docs/ADR-0001-extraction-boundary.md
|
||||
- ../citation-engine/src/shared/document.ts
|
||||
---
|
||||
|
||||
# ESRC-WP-0002 — HTML and Markdown source representations
|
||||
|
||||
The PDF slice (ESRC-WP-0001) established the ingest boundary and contract shape.
|
||||
This workplan extends ingest to HTML and Markdown sources, producing the same
|
||||
`{ document, representation }` pair over engine-owned contracts.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Which `representationType` values do HTML/MD map to, and do they need new
|
||||
`DocumentRepresentation` fields in `citation-engine`?
|
||||
- How is canonical text + offset map derived for reflowable formats that have
|
||||
no fixed pages? (Page map may be empty or synthetic.)
|
||||
|
||||
## Sketch
|
||||
|
||||
```task
|
||||
id: ESRC-WP-0002-T01
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
Define the HTML/MD representation contract with `citation-engine`; agree
|
||||
`representationType` and any offset/page semantics for pageless formats.
|
||||
|
||||
```task
|
||||
id: ESRC-WP-0002-T02
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
Implement `ingestHtml` / `ingestMarkdown` in `src/html/` and `src/markdown/`,
|
||||
reusing `fingerprintBytes` and the canonical-text normalizer.
|
||||
|
||||
```task
|
||||
id: ESRC-WP-0002-T03
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
Add fixture-driven contract tests mirroring the PDF suite; decide fixture
|
||||
ownership per ADR-0002.
|
||||
48
workplans/ESRC-WP-0003-metadata-enrichment.md
Normal file
48
workplans/ESRC-WP-0003-metadata-enrichment.md
Normal file
|
|
@ -0,0 +1,48 @@
|
|||
---
|
||||
id: ESRC-WP-0003
|
||||
type: workplan
|
||||
title: "Document metadata enrichment beyond pass-through options"
|
||||
domain: infotech
|
||||
repo: evidence-source
|
||||
status: proposed
|
||||
owner: codex
|
||||
topic_slug: citation_evidence_mvp
|
||||
created: "2026-07-08"
|
||||
updated: "2026-07-08"
|
||||
spec_refs:
|
||||
- README.md
|
||||
- src/pdf/ingest.ts
|
||||
---
|
||||
|
||||
# ESRC-WP-0003 — Metadata enrichment
|
||||
|
||||
Today `ingestPdf` only passes through caller-supplied `title`/`uri`/`metadata`.
|
||||
This workplan extracts intrinsic document metadata (PDF info dictionary,
|
||||
embedded XMP, page-derived signals like author/creation date/title) and folds
|
||||
it into the `Document` record without breaking the pure-over-bytes contract.
|
||||
|
||||
## Sketch
|
||||
|
||||
```task
|
||||
id: ESRC-WP-0003-T01
|
||||
status: todo
|
||||
priority: low
|
||||
```
|
||||
Enumerate available PDF metadata sources (info dict, XMP) via PDF.js and decide
|
||||
precedence vs. caller-supplied options (caller wins).
|
||||
|
||||
```task
|
||||
id: ESRC-WP-0003-T02
|
||||
status: todo
|
||||
priority: low
|
||||
```
|
||||
Implement extraction into a normalized metadata shape; keep it optional so
|
||||
minimal-metadata PDFs still ingest cleanly.
|
||||
|
||||
```task
|
||||
id: ESRC-WP-0003-T03
|
||||
status: todo
|
||||
priority: low
|
||||
```
|
||||
Contract tests over the fixture corpus asserting extracted vs. overridden
|
||||
metadata precedence.
|
||||
47
workplans/ESRC-WP-0004-citation-recovery.md
Normal file
47
workplans/ESRC-WP-0004-citation-recovery.md
Normal file
|
|
@ -0,0 +1,47 @@
|
|||
---
|
||||
id: ESRC-WP-0004
|
||||
type: workplan
|
||||
title: "Local citation recovery and external source discovery hooks"
|
||||
domain: infotech
|
||||
repo: evidence-source
|
||||
status: proposed
|
||||
owner: codex
|
||||
topic_slug: citation_evidence_mvp
|
||||
created: "2026-07-08"
|
||||
updated: "2026-07-08"
|
||||
spec_refs:
|
||||
- INTENT.md
|
||||
- README.md
|
||||
---
|
||||
|
||||
# ESRC-WP-0004 — Citation recovery and source discovery hooks
|
||||
|
||||
`evidence-source`'s original intent includes recovering citations when a stored
|
||||
selector goes stale and discovering the external source a document was drawn
|
||||
from. This is the largest deferred strand and depends on the anchoring layer's
|
||||
selector model.
|
||||
|
||||
## Open questions
|
||||
|
||||
- What is the recovery contract — does this repo *detect* stale selectors, or
|
||||
only provide the re-ingest/re-fingerprint primitives the anchor layer calls?
|
||||
- Where does external source discovery (URL/DOI resolution) belong relative to
|
||||
`citation-engine` and the umbrella?
|
||||
|
||||
## Sketch
|
||||
|
||||
```task
|
||||
id: ESRC-WP-0004-T01
|
||||
status: todo
|
||||
priority: low
|
||||
```
|
||||
Design the recovery boundary with the anchor layer: define what
|
||||
`evidence-source` owns vs. what stays in anchoring.
|
||||
|
||||
```task
|
||||
id: ESRC-WP-0004-T02
|
||||
status: todo
|
||||
priority: low
|
||||
```
|
||||
Prototype re-ingest/re-fingerprint recovery primitives and a discovery hook
|
||||
interface (no network coupling baked into the core).
|
||||
Loading…
Add table
Add a link
Reference in a new issue