Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery
Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
This commit is contained in:
parent
df6acad828
commit
97437dfa18
28 changed files with 1295 additions and 69 deletions
|
|
@ -4,14 +4,15 @@ type: workplan
|
|||
title: "HTML and Markdown source representations"
|
||||
domain: infotech
|
||||
repo: evidence-source
|
||||
status: proposed
|
||||
status: finished
|
||||
owner: codex
|
||||
topic_slug: citation_evidence_mvp
|
||||
created: "2026-07-08"
|
||||
updated: "2026-07-08"
|
||||
updated: "2026-07-09"
|
||||
spec_refs:
|
||||
- README.md
|
||||
- docs/ADR-0001-extraction-boundary.md
|
||||
- docs/ADR-0003-html-markdown-representation.md
|
||||
- ../citation-engine/src/shared/document.ts
|
||||
state_hub_workstream_id: "c0d28886-e02f-4df2-abb6-ebe179171c94"
|
||||
---
|
||||
|
|
@ -22,18 +23,19 @@ The PDF slice (ESRC-WP-0001) established the ingest boundary and contract shape.
|
|||
This workplan extends ingest to HTML and Markdown sources, producing the same
|
||||
`{ document, representation }` pair over engine-owned contracts.
|
||||
|
||||
## Open questions
|
||||
## Decisions (ADR-0003)
|
||||
|
||||
- Which `representationType` values do HTML/MD map to, and do they need new
|
||||
`DocumentRepresentation` fields in `citation-engine`?
|
||||
- How is canonical text + offset map derived for reflowable formats that have
|
||||
no fixed pages? (Page map may be empty or synthetic.)
|
||||
- HTML maps to `representationType: "html-dom"`; Markdown to
|
||||
`"markdown-rendered"`.
|
||||
- Pageless formats omit `pageMap`; `offsetMap` is a single synthetic page-1
|
||||
range covering `[0, canonicalText.length)`.
|
||||
- Inline fixtures owned by this repo (no PII); PDF corpus rules unchanged.
|
||||
|
||||
## Sketch
|
||||
|
||||
```task
|
||||
id: ESRC-WP-0002-T01
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "bf1d2a2f-41b4-44b5-b1d1-4f0eec5fbf59"
|
||||
```
|
||||
|
|
@ -42,7 +44,7 @@ Define the HTML/MD representation contract with `citation-engine`; agree
|
|||
|
||||
```task
|
||||
id: ESRC-WP-0002-T02
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "66320911-a4b2-404f-b025-8b23dc0c9680"
|
||||
```
|
||||
|
|
@ -51,9 +53,9 @@ reusing `fingerprintBytes` and the canonical-text normalizer.
|
|||
|
||||
```task
|
||||
id: ESRC-WP-0002-T03
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "266d168c-d93e-4387-9527-70aa0d5b2a6b"
|
||||
```
|
||||
Add fixture-driven contract tests mirroring the PDF suite; decide fixture
|
||||
ownership per ADR-0002.
|
||||
ownership per ADR-0002.
|
||||
Loading…
Add table
Add a link
Reference in a new issue