coulomb/evidence-source
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
evidence-source/workplans/ESRC-WP-0002-html-markdown-representations.md

61 lines
1.8 KiB
Markdown
Raw Normal View History

feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
---
id: ESRC-WP-0002
type: workplan
title: "HTML and Markdown source representations"
domain: infotech
repo: evidence-source
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00
status: finished
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
owner: codex
topic_slug: citation_evidence_mvp
created: "2026-07-08"
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00
updated: "2026-07-09"
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
spec_refs:
- README.md
- docs/ADR-0001-extraction-boundary.md
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00
- docs/ADR-0003-html-markdown-representation.md
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
- ../citation-engine/src/shared/document.ts
fix(workplans): adopt ADR-007 derived identifiers for unregistered records These workplans exist only in the retired local hub. Their random pre-ADR-007 identifiers are refused by C-06 as stale references, so they cannot be registered. Deriving from the canonical record id takes no identity from anything: central does not hold them and the old ids die with the cache. Records central already holds were deliberately left untouched. Refs CUST-WP-0068-T06 Assistant: claude-code Assistant-Model: opus Assistant-Process: 2583210@bnt-lap001 Assistant-Session: f2bff2d5-e9b2-4338-92ca-10282a927006
2026-08-25 20:10:10 +02:00
state_hub_workstream_id: "12f0b386-0df1-50d9-8cf7-4829d5257382"
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
---
# ESRC-WP-0002 — HTML and Markdown source representations
The PDF slice (ESRC-WP-0001) established the ingest boundary and contract shape.
This workplan extends ingest to HTML and Markdown sources, producing the same
`{ document, representation }` pair over engine-owned contracts.
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00
## Decisions (ADR-0003)
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00
- HTML maps to `representationType: "html-dom"`; Markdown to
`"markdown-rendered"`.
- Pageless formats omit `pageMap`; `offsetMap` is a single synthetic page-1
range covering `[0, canonicalText.length)`.
- Inline fixtures owned by this repo (no PII); PDF corpus rules unchanged.
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
## Sketch
```task
id: ESRC-WP-0002-T01
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00
status: done
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
priority: medium
fix(workplans): adopt ADR-007 derived identifiers for unregistered records These workplans exist only in the retired local hub. Their random pre-ADR-007 identifiers are refused by C-06 as stale references, so they cannot be registered. Deriving from the canonical record id takes no identity from anything: central does not hold them and the old ids die with the cache. Records central already holds were deliberately left untouched. Refs CUST-WP-0068-T06 Assistant: claude-code Assistant-Model: opus Assistant-Process: 2583210@bnt-lap001 Assistant-Session: f2bff2d5-e9b2-4338-92ca-10282a927006
2026-08-25 20:10:10 +02:00
state_hub_task_id: "f875c025-2c08-56d7-bbde-60e0643a7b9c"
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
```
Define the HTML/MD representation contract with `citation-engine`; agree
`representationType` and any offset/page semantics for pageless formats.
```task
id: ESRC-WP-0002-T02
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00
status: done
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
priority: medium
fix(workplans): adopt ADR-007 derived identifiers for unregistered records These workplans exist only in the retired local hub. Their random pre-ADR-007 identifiers are refused by C-06 as stale references, so they cannot be registered. Deriving from the canonical record id takes no identity from anything: central does not hold them and the old ids die with the cache. Records central already holds were deliberately left untouched. Refs CUST-WP-0068-T06 Assistant: claude-code Assistant-Model: opus Assistant-Process: 2583210@bnt-lap001 Assistant-Session: f2bff2d5-e9b2-4338-92ca-10282a927006
2026-08-25 20:10:10 +02:00
state_hub_task_id: "e7936d67-188d-50f3-8dc8-684c9f03fb4a"
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
```
Implement `ingestHtml` / `ingestMarkdown` in `src/html/` and `src/markdown/`,
reusing `fingerprintBytes` and the canonical-text normalizer.
```task
id: ESRC-WP-0002-T03
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00
status: done
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
priority: medium
fix(workplans): adopt ADR-007 derived identifiers for unregistered records These workplans exist only in the retired local hub. Their random pre-ADR-007 identifiers are refused by C-06 as stale references, so they cannot be registered. Deriving from the canonical record id takes no identity from anything: central does not hold them and the old ids die with the cache. Records central already holds were deliberately left untouched. Refs CUST-WP-0068-T06 Assistant: claude-code Assistant-Model: opus Assistant-Process: 2583210@bnt-lap001 Assistant-Session: f2bff2d5-e9b2-4338-92ca-10282a927006
2026-08-25 20:10:10 +02:00
state_hub_task_id: "18fcce8a-92f6-5e52-b613-ffa52f18bc71"
feat(pdf): extract standalone PDF ingest package (ESRC-WP-0001) Bootstrap evidence-source from the citation-evidence src/source PDF slice. - Headless core (src/pdf): ingest/extract/fingerprint, importing domain contracts from @citation-evidence/engine/shared (no local copies) - Browser upload helpers isolated under src/browser behind a ./browser entry point, with an eslint boundary keeping the core browser-free - pnpm/TS/vitest/eslint scaffold; 52 tests (contract + determinism) - Fixtures resolved from the sibling citation-evidence checkout, not duplicated (real PII) — see docs/ADR-0002; suites skip when absent - Boundary + fixture decisions recorded as docs/ADR-0001 / ADR-0002 - README/SCOPE rewritten; capability.infotech.pdf-evidence-ingest registered, NO_CAPABILITIES removed - Follow-on workplans ESRC-WP-0002..0004 queued; ESRC-WP-0001 finished Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:55:28 +02:00
```
Add fixture-driven contract tests mirroring the PDF suite; decide fixture
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00
ownership per ADR-0002.
Reference in a new issue Copy permalink