Implement ESRC-WP-0002/0003/0004: HTML/MD ingest, metadata, recovery
Add ingestHtml and ingestMarkdown with ADR-0003 pageless offset semantics, PDF intrinsic metadata extraction with caller-wins merge (WP-0003), and citation recovery primitives including re-ingest reconcile, local quote search, and pluggable discovery hooks (ADR-0004). Mark all three workplans finished with contract tests (80 passing).
This commit is contained in:
parent
df6acad828
commit
97437dfa18
28 changed files with 1295 additions and 69 deletions
40
README.md
40
README.md
|
|
@ -1,18 +1,16 @@
|
|||
# evidence-source
|
||||
|
||||
Headless document ingest for the citation-evidence ecosystem. Turns raw PDF
|
||||
bytes into engine-owned evidence contracts — a `Document` (media type,
|
||||
SHA-256 fingerprint, optional title/uri/metadata) and a
|
||||
`DocumentRepresentation` (`pdf-text`: canonical text, page map, gap-free
|
||||
offset map). Ingest is pure over bytes: no persistence, no viewer state, no
|
||||
React.
|
||||
Headless document ingest for the citation-evidence ecosystem. Turns raw PDF,
|
||||
HTML, and Markdown bytes into engine-owned evidence contracts — a `Document`
|
||||
(media type, SHA-256 fingerprint, optional title/uri/metadata) and a
|
||||
`DocumentRepresentation` (canonical text, page/offset maps where applicable).
|
||||
Ingest is pure over bytes: no persistence, no viewer state, no React.
|
||||
|
||||
## Status
|
||||
|
||||
**Implemented: the PDF slice.** As of ESRC-WP-0001 this repo hosts the
|
||||
extracted PDF ingest core that previously lived in `citation-evidence/src/source/`.
|
||||
HTML/Markdown representations, richer metadata enrichment, and citation
|
||||
recovery are deferred to follow-on workplans (see `workplans/`).
|
||||
**Implemented:** PDF ingest (ESRC-WP-0001), HTML/Markdown ingest
|
||||
(ESRC-WP-0002), PDF metadata enrichment (ESRC-WP-0003), and citation recovery
|
||||
primitives (ESRC-WP-0004). See `workplans/` for history.
|
||||
|
||||
## Install
|
||||
|
||||
|
|
@ -26,13 +24,17 @@ pnpm install # resolves @citation-evidence/engine via link:../citation-engine
|
|||
## Usage
|
||||
|
||||
```ts
|
||||
import { ingestPdf } from "@citation-evidence/evidence-source";
|
||||
import {
|
||||
ingestPdf,
|
||||
ingestHtml,
|
||||
ingestMarkdown,
|
||||
} from "@citation-evidence/evidence-source";
|
||||
|
||||
const { document, representation } = await ingestPdf(pdfBytes, {
|
||||
filename: "contract.pdf",
|
||||
});
|
||||
// document.fingerprint -> SHA-256 hex
|
||||
// representation.canonicalText / pageMap / offsetMap
|
||||
const pdf = await ingestPdf(pdfBytes, { filename: "contract.pdf" });
|
||||
const html = await ingestHtml(htmlBytes, { filename: "brief.html" });
|
||||
const md = await ingestMarkdown("# Title\n\nBody text.", { filename: "notes.md" });
|
||||
// document.fingerprint -> SHA-256 hex
|
||||
// representation.canonicalText / pageMap / offsetMap (format-dependent)
|
||||
```
|
||||
|
||||
Browser upload helpers (in-memory `blob:` byte store + `ingestPdfFromFile`)
|
||||
|
|
@ -61,12 +63,14 @@ Point `EVIDENCE_SOURCE_FIXTURE_DIR` at the PDF corpus when the sibling
|
|||
|
||||
## Architecture
|
||||
|
||||
- `src/pdf/` — headless core: `ingest`, `extract`, `fingerprint`. Runtime-agnostic.
|
||||
- `src/pdf/` — PDF ingest, extraction, fingerprinting, metadata enrichment.
|
||||
- `src/html/`, `src/markdown/` — reflowable-format ingest (ADR-0003).
|
||||
- `src/recovery/` — re-ingest/reconcile, local quote search, discovery hooks (ADR-0004).
|
||||
- `src/browser/` — browser upload surface: `byte-store`, `upload`. Separated by
|
||||
an ESLint boundary so the core never imports it.
|
||||
- Domain contracts come from `@citation-evidence/engine/shared`; none are
|
||||
copied locally.
|
||||
- Boundary and fixture decisions: `docs/ADR-0001`, `docs/ADR-0002`.
|
||||
- Boundary and fixture decisions: `docs/ADR-0001` through `docs/ADR-0004`.
|
||||
|
||||
`viewer-url` resolution stays in the consuming app (`citation-evidence`), which
|
||||
imports this package's ingest core through a thin façade.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue