coulomb/evidence-source
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
evidence-source/workplans/ESRC-WP-0002-html-markdown-representations.md

61 lines
1.8 KiB
Markdown
Raw Normal View History

feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
---
id: ESRC-WP-0002
type: workplan
title: "HTML and Markdown source representations"
domain: infotech
repo: evidence-source
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00
status: finished
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
owner: codex
topic_slug: citation_evidence_mvp
created: "2026-07-08"
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00
updated: "2026-07-09"
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
spec_refs:
- README.md
- docs/ADR-0001-extraction-boundary.md
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00
- docs/ADR-0003-html-markdown-representation.md
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
- ../citation-engine/src/shared/document.ts
chore(consistency): write back state-hub workplan/task IDs [auto] fix-consistency C-06/C-11 registered ESRC-WP-0002..0004 and the ESRC-WP-0001 tasks; IDs written back into the workplan files. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:56:39 +02:00
state_hub_workstream_id: "c0d28886-e02f-4df2-abb6-ebe179171c94"
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
---
# ESRC-WP-0002 — HTML and Markdown source representations
The PDF slice (ESRC-WP-0001) established the ingest boundary and contract shape.
This workplan extends ingest to HTML and Markdown sources, producing the same
`{ document, representation }` pair over engine-owned contracts.
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00
## Decisions (ADR-0003)
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00
- HTML maps to `representationType: "html-dom"`; Markdown to
`"markdown-rendered"`.
- Pageless formats omit `pageMap`; `offsetMap` is a single synthetic page-1
range covering `[0, canonicalText.length)`.
- Inline fixtures owned by this repo (no PII); PDF corpus rules unchanged.
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
## Sketch
```task
id: ESRC-WP-0002-T01
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00
status: done
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
priority: medium
chore(consistency): write back state-hub workplan/task IDs [auto] fix-consistency C-06/C-11 registered ESRC-WP-0002..0004 and the ESRC-WP-0001 tasks; IDs written back into the workplan files. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:56:39 +02:00
state_hub_task_id: "bf1d2a2f-41b4-44b5-b1d1-4f0eec5fbf59"
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
```
Define the HTML/MD representation contract with `citation-engine`; agree
`representationType` and any offset/page semantics for pageless formats.
```task
id: ESRC-WP-0002-T02
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00
status: done
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
priority: medium
chore(consistency): write back state-hub workplan/task IDs [auto] fix-consistency C-06/C-11 registered ESRC-WP-0002..0004 and the ESRC-WP-0001 tasks; IDs written back into the workplan files. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:56:39 +02:00
state_hub_task_id: "66320911-a4b2-404f-b025-8b23dc0c9680"
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
```
Implement `ingestHtml` / `ingestMarkdown` in `src/html/` and `src/markdown/`,
reusing `fingerprintBytes` and the canonical-text normalizer.
```task
id: ESRC-WP-0002-T03
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00
status: done
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
priority: medium
chore(consistency): write back state-hub workplan/task IDs [auto] fix-consistency C-06/C-11 registered ESRC-WP-0002..0004 and the ESRC-WP-0001 tasks; IDs written back into the workplan files. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:56:39 +02:00
state_hub_task_id: "266d168c-d93e-4387-9527-70aa0d5b2a6b"
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
```
Add fixture-driven contract tests mirroring the PDF suite; decide fixture
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00
ownership per ADR-0002.
Reference in a new issue Copy permalink