coulomb/evidence-source
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
evidence-source/docs/ADR-0003-html-markdown-representation.md
tegwick 97437dfa18
Some checks failed
CI Smoke / host-smoke (push) Successful in 0s
Details
CI Smoke / container-smoke (push) Failing after 15m18s
Details
Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery 
Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics,
PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and
citation recovery primitives including re-ingest reconcile, local quote
search, and pluggable discovery hooks (ADR-0004). Mark all three workplans
finished with contract tests (80 passing).
2026-07-09 01:48:28 +02:00

2 KiB
Raw Blame History

ADR-0003 — HTML and Markdown representation contract

  • Status: accepted
  • Date: 2026-07-09
  • Workplan: ESRC-WP-0002 (T01)

Context

ESRC-WP-0001 established the PDF ingest boundary. ESRC-WP-0002 extends ingest to HTML and Markdown while reusing engine-owned contracts from @citation-evidence/engine/shared.

RepresentationType already reserves html-dom and markdown-rendered in citation-engine; no engine schema change is required for this slice.

Decision

Representation types

Source format mediaType representationType
HTML text/html html-dom
Markdown text/markdown markdown-rendered

Pageless offset semantics

Reflowable formats have no fixed pages:

  • pageMap is omitted (undefined).
  • offsetMap is a single synthetic range on page 1 covering [0, canonicalText.length) with no gaps. This keeps TextPositionSelector usable without inventing physical page geometry.
  • structureMap remains never in the engine contract and is not populated.

Canonical text

Both pipelines decode source bytes as UTF-8, derive plain text from the format, then apply normalize() from @citation-evidence/engine/shared. HTML strips active content (script, style, iframe, object, embed, noscript) before text extraction.

Content hash

contentHash is the SHA-256 fingerprint of the source bytes (same rule as PDF ingest), not a hash of canonical text.

Fixtures

HTML/Markdown contract tests use small inline fixtures owned by this repo. They contain no PII and do not duplicate the upstream PDF corpus (ADR-0002).

Consequences

  • ingestHtml and ingestMarkdown mirror the PDF { document, representation } shape.
  • Anchoring layers may add DOM/structural selectors later; this slice delivers canonical text and position offsets only.
  • Sanitization is extraction-time stripping, not a sandboxed renderer.