artifact-store/src/artifactstore/dataplane/spi.py

105 lines
2.9 KiB
Python
Raw Normal View History

WP-0001-T012: data plane SPI and in-process implementation src/artifactstore/dataplane/: - spi.py: DataPlane Protocol with the five operations ingest_stream, serve_object, verify_object, delete_object, backend_health (ADR-0004). Dataclasses: IngestHints (size_hint, primary_algorithm, backend_id overrides), IngestResult (primary_digest + sha256_digest + size_bytes + StorageReceipt), VerifyResult (verified bool, mismatch reason, actual digests + size). - inproc.py: InProcessDataPlane wraps one StorageBackend. ingest_stream is two-pass against a tempfile (drain stream while dual-hashing into BLAKE3+SHA-256, then forward the tempfile to backend.put under the primary content address); fsync+cleanup on exception. serve_object passes byte ranges through; verify_object re-reads bytes via backend.get, re-digests with the stored algorithm, and reports mismatches. delete and health are thin pass-throughs. tests/unit/test_dataplane_inproc.py (11 cases): - ingest_stream computes correct dual digests, returns receipt, stores bytes at the content-addressed path. - empty-input ingest returns the BLAKE3/SHA-256 of empty. - serve_object round-trips ingested bytes; supports byte_range. - verify_object verifies intact bytes; detects on-disk corruption. - delete_object passes through (True then False). - backend_health passes through. - IngestHints override of primary_algorithm (sha256-as-primary path). - Missing-object serve raises ObjectNotFoundError. - Architectural test (ADR-0004 invariant): no control-plane module (api / registry / retention / audit) imports artifactstore.storage.backends.* or artifactstore.dataplane.inproc directly. Enforced via AST scan of every .py file in those packages. Gates: ruff clean, mypy --strict clean on 44 files, 70 tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 02:07:05 +02:00
"""Data plane SPI (ADR-0004).
The control plane (``registry`` / ``api`` / ``retention`` / ``audit``)
interacts with bytes exclusively through this surface. v1 ships an
in-process implementation backed by a :class:`StorageBackend`; a future
Rust daemon will satisfy the same protocol over a Unix socket.
The SPI uses CBOR-serialisable input and output types so the in-process
implementation and a future RPC implementation share representations.
"""
from __future__ import annotations
from collections.abc import AsyncIterator
from dataclasses import dataclass
from typing import Protocol, runtime_checkable
from artifactstore.identity import ContentAddress, Digest
from artifactstore.storage.spi import BackendStatus, DeletionResult, StorageReceipt
__all__ = [
"DataPlane",
"IngestHints",
"IngestResult",
"VerifyResult",
]
@dataclass(frozen=True, slots=True)
class IngestHints:
"""Optional hints attached to an ingest call.
``size_hint`` lets backends pre-allocate or pick an upload strategy.
``primary_algorithm`` overrides the default primary hash (BLAKE3).
``backend_id`` lets the data plane select among configured backends
(ignored when only one backend is configured).
"""
size_hint: int | None = None
primary_algorithm: str | None = None
backend_id: str | None = None
@dataclass(frozen=True, slots=True)
class IngestResult:
"""Return value of :meth:`DataPlane.ingest_stream`."""
primary_digest: Digest
sha256_digest: Digest
size_bytes: int
receipt: StorageReceipt
@dataclass(frozen=True, slots=True)
class VerifyResult:
"""Return value of :meth:`DataPlane.verify_object`.
``verified`` is ``True`` when the bytes currently held at
``content_address`` re-digest to the expected primary digest. ``mismatch``
is populated with a short explanation when verification fails.
"""
content_address: ContentAddress
verified: bool
actual_primary_digest: Digest
actual_sha256_digest: Digest
actual_size_bytes: int
mismatch: str | None = None
@runtime_checkable
class DataPlane(Protocol):
"""Byte-handling surface used by the control plane."""
async def ingest_stream(
self,
stream: AsyncIterator[bytes],
*,
hints: IngestHints | None = None,
) -> IngestResult: ...
async def serve_object(
self,
content_address: ContentAddress,
*,
byte_range: tuple[int, int] | None = None,
backend_id: str | None = None,
WP-0001-T012: data plane SPI and in-process implementation src/artifactstore/dataplane/: - spi.py: DataPlane Protocol with the five operations ingest_stream, serve_object, verify_object, delete_object, backend_health (ADR-0004). Dataclasses: IngestHints (size_hint, primary_algorithm, backend_id overrides), IngestResult (primary_digest + sha256_digest + size_bytes + StorageReceipt), VerifyResult (verified bool, mismatch reason, actual digests + size). - inproc.py: InProcessDataPlane wraps one StorageBackend. ingest_stream is two-pass against a tempfile (drain stream while dual-hashing into BLAKE3+SHA-256, then forward the tempfile to backend.put under the primary content address); fsync+cleanup on exception. serve_object passes byte ranges through; verify_object re-reads bytes via backend.get, re-digests with the stored algorithm, and reports mismatches. delete and health are thin pass-throughs. tests/unit/test_dataplane_inproc.py (11 cases): - ingest_stream computes correct dual digests, returns receipt, stores bytes at the content-addressed path. - empty-input ingest returns the BLAKE3/SHA-256 of empty. - serve_object round-trips ingested bytes; supports byte_range. - verify_object verifies intact bytes; detects on-disk corruption. - delete_object passes through (True then False). - backend_health passes through. - IngestHints override of primary_algorithm (sha256-as-primary path). - Missing-object serve raises ObjectNotFoundError. - Architectural test (ADR-0004 invariant): no control-plane module (api / registry / retention / audit) imports artifactstore.storage.backends.* or artifactstore.dataplane.inproc directly. Enforced via AST scan of every .py file in those packages. Gates: ruff clean, mypy --strict clean on 44 files, 70 tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 02:07:05 +02:00
) -> AsyncIterator[bytes]: ...
async def verify_object(
self,
content_address: ContentAddress,
*,
backend_id: str | None = None,
) -> VerifyResult: ...
WP-0001-T012: data plane SPI and in-process implementation src/artifactstore/dataplane/: - spi.py: DataPlane Protocol with the five operations ingest_stream, serve_object, verify_object, delete_object, backend_health (ADR-0004). Dataclasses: IngestHints (size_hint, primary_algorithm, backend_id overrides), IngestResult (primary_digest + sha256_digest + size_bytes + StorageReceipt), VerifyResult (verified bool, mismatch reason, actual digests + size). - inproc.py: InProcessDataPlane wraps one StorageBackend. ingest_stream is two-pass against a tempfile (drain stream while dual-hashing into BLAKE3+SHA-256, then forward the tempfile to backend.put under the primary content address); fsync+cleanup on exception. serve_object passes byte ranges through; verify_object re-reads bytes via backend.get, re-digests with the stored algorithm, and reports mismatches. delete and health are thin pass-throughs. tests/unit/test_dataplane_inproc.py (11 cases): - ingest_stream computes correct dual digests, returns receipt, stores bytes at the content-addressed path. - empty-input ingest returns the BLAKE3/SHA-256 of empty. - serve_object round-trips ingested bytes; supports byte_range. - verify_object verifies intact bytes; detects on-disk corruption. - delete_object passes through (True then False). - backend_health passes through. - IngestHints override of primary_algorithm (sha256-as-primary path). - Missing-object serve raises ObjectNotFoundError. - Architectural test (ADR-0004 invariant): no control-plane module (api / registry / retention / audit) imports artifactstore.storage.backends.* or artifactstore.dataplane.inproc directly. Enforced via AST scan of every .py file in those packages. Gates: ruff clean, mypy --strict clean on 44 files, 70 tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 02:07:05 +02:00
async def delete_object(
self,
content_address: ContentAddress,
*,
backend_id: str | None = None,
) -> DeletionResult: ...
WP-0001-T012: data plane SPI and in-process implementation src/artifactstore/dataplane/: - spi.py: DataPlane Protocol with the five operations ingest_stream, serve_object, verify_object, delete_object, backend_health (ADR-0004). Dataclasses: IngestHints (size_hint, primary_algorithm, backend_id overrides), IngestResult (primary_digest + sha256_digest + size_bytes + StorageReceipt), VerifyResult (verified bool, mismatch reason, actual digests + size). - inproc.py: InProcessDataPlane wraps one StorageBackend. ingest_stream is two-pass against a tempfile (drain stream while dual-hashing into BLAKE3+SHA-256, then forward the tempfile to backend.put under the primary content address); fsync+cleanup on exception. serve_object passes byte ranges through; verify_object re-reads bytes via backend.get, re-digests with the stored algorithm, and reports mismatches. delete and health are thin pass-throughs. tests/unit/test_dataplane_inproc.py (11 cases): - ingest_stream computes correct dual digests, returns receipt, stores bytes at the content-addressed path. - empty-input ingest returns the BLAKE3/SHA-256 of empty. - serve_object round-trips ingested bytes; supports byte_range. - verify_object verifies intact bytes; detects on-disk corruption. - delete_object passes through (True then False). - backend_health passes through. - IngestHints override of primary_algorithm (sha256-as-primary path). - Missing-object serve raises ObjectNotFoundError. - Architectural test (ADR-0004 invariant): no control-plane module (api / registry / retention / audit) imports artifactstore.storage.backends.* or artifactstore.dataplane.inproc directly. Enforced via AST scan of every .py file in those packages. Gates: ruff clean, mypy --strict clean on 44 files, 70 tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 02:07:05 +02:00
async def backend_health(self) -> BackendStatus: ...