WP-0001-T012: data plane SPI and in-process implementation
src/artifactstore/dataplane/:
- spi.py: DataPlane Protocol with the five operations ingest_stream,
serve_object, verify_object, delete_object, backend_health
(ADR-0004). Dataclasses: IngestHints (size_hint, primary_algorithm,
backend_id overrides), IngestResult (primary_digest + sha256_digest +
size_bytes + StorageReceipt), VerifyResult (verified bool, mismatch
reason, actual digests + size).
- inproc.py: InProcessDataPlane wraps one StorageBackend. ingest_stream
is two-pass against a tempfile (drain stream while dual-hashing into
BLAKE3+SHA-256, then forward the tempfile to backend.put under the
primary content address); fsync+cleanup on exception. serve_object
passes byte ranges through; verify_object re-reads bytes via backend.get,
re-digests with the stored algorithm, and reports mismatches. delete
and health are thin pass-throughs.
tests/unit/test_dataplane_inproc.py (11 cases):
- ingest_stream computes correct dual digests, returns receipt, stores
bytes at the content-addressed path.
- empty-input ingest returns the BLAKE3/SHA-256 of empty.
- serve_object round-trips ingested bytes; supports byte_range.
- verify_object verifies intact bytes; detects on-disk corruption.
- delete_object passes through (True then False).
- backend_health passes through.
- IngestHints override of primary_algorithm (sha256-as-primary path).
- Missing-object serve raises ObjectNotFoundError.
- Architectural test (ADR-0004 invariant): no control-plane module
(api / registry / retention / audit) imports
artifactstore.storage.backends.* or artifactstore.dataplane.inproc
directly. Enforced via AST scan of every .py file in those packages.
Gates: ruff clean, mypy --strict clean on 44 files, 70 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 02:07:05 +02:00
|
|
|
"""Data plane SPI (ADR-0004).
|
|
|
|
|
|
|
|
|
|
The control plane (``registry`` / ``api`` / ``retention`` / ``audit``)
|
|
|
|
|
interacts with bytes exclusively through this surface. v1 ships an
|
|
|
|
|
in-process implementation backed by a :class:`StorageBackend`; a future
|
|
|
|
|
Rust daemon will satisfy the same protocol over a Unix socket.
|
|
|
|
|
|
|
|
|
|
The SPI uses CBOR-serialisable input and output types so the in-process
|
|
|
|
|
implementation and a future RPC implementation share representations.
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
from __future__ import annotations
|
|
|
|
|
|
|
|
|
|
from collections.abc import AsyncIterator
|
|
|
|
|
from dataclasses import dataclass
|
|
|
|
|
from typing import Protocol, runtime_checkable
|
|
|
|
|
|
|
|
|
|
from artifactstore.identity import ContentAddress, Digest
|
|
|
|
|
from artifactstore.storage.spi import BackendStatus, DeletionResult, StorageReceipt
|
|
|
|
|
|
|
|
|
|
__all__ = [
|
|
|
|
|
"DataPlane",
|
|
|
|
|
"IngestHints",
|
|
|
|
|
"IngestResult",
|
|
|
|
|
"VerifyResult",
|
|
|
|
|
]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@dataclass(frozen=True, slots=True)
|
|
|
|
|
class IngestHints:
|
|
|
|
|
"""Optional hints attached to an ingest call.
|
|
|
|
|
|
|
|
|
|
``size_hint`` lets backends pre-allocate or pick an upload strategy.
|
|
|
|
|
``primary_algorithm`` overrides the default primary hash (BLAKE3).
|
|
|
|
|
``backend_id`` lets the data plane select among configured backends
|
|
|
|
|
(ignored when only one backend is configured).
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
size_hint: int | None = None
|
|
|
|
|
primary_algorithm: str | None = None
|
|
|
|
|
backend_id: str | None = None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@dataclass(frozen=True, slots=True)
|
|
|
|
|
class IngestResult:
|
|
|
|
|
"""Return value of :meth:`DataPlane.ingest_stream`."""
|
|
|
|
|
|
|
|
|
|
primary_digest: Digest
|
|
|
|
|
sha256_digest: Digest
|
|
|
|
|
size_bytes: int
|
|
|
|
|
receipt: StorageReceipt
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@dataclass(frozen=True, slots=True)
|
|
|
|
|
class VerifyResult:
|
|
|
|
|
"""Return value of :meth:`DataPlane.verify_object`.
|
|
|
|
|
|
|
|
|
|
``verified`` is ``True`` when the bytes currently held at
|
|
|
|
|
``content_address`` re-digest to the expected primary digest. ``mismatch``
|
|
|
|
|
is populated with a short explanation when verification fails.
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
content_address: ContentAddress
|
|
|
|
|
verified: bool
|
|
|
|
|
actual_primary_digest: Digest
|
|
|
|
|
actual_sha256_digest: Digest
|
|
|
|
|
actual_size_bytes: int
|
|
|
|
|
mismatch: str | None = None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@runtime_checkable
|
|
|
|
|
class DataPlane(Protocol):
|
|
|
|
|
"""Byte-handling surface used by the control plane."""
|
|
|
|
|
|
|
|
|
|
async def ingest_stream(
|
|
|
|
|
self,
|
|
|
|
|
stream: AsyncIterator[bytes],
|
|
|
|
|
*,
|
|
|
|
|
hints: IngestHints | None = None,
|
|
|
|
|
) -> IngestResult: ...
|
|
|
|
|
|
|
|
|
|
async def serve_object(
|
|
|
|
|
self,
|
|
|
|
|
content_address: ContentAddress,
|
|
|
|
|
*,
|
|
|
|
|
byte_range: tuple[int, int] | None = None,
|
2026-05-16 23:26:03 +02:00
|
|
|
backend_id: str | None = None,
|
WP-0001-T012: data plane SPI and in-process implementation
src/artifactstore/dataplane/:
- spi.py: DataPlane Protocol with the five operations ingest_stream,
serve_object, verify_object, delete_object, backend_health
(ADR-0004). Dataclasses: IngestHints (size_hint, primary_algorithm,
backend_id overrides), IngestResult (primary_digest + sha256_digest +
size_bytes + StorageReceipt), VerifyResult (verified bool, mismatch
reason, actual digests + size).
- inproc.py: InProcessDataPlane wraps one StorageBackend. ingest_stream
is two-pass against a tempfile (drain stream while dual-hashing into
BLAKE3+SHA-256, then forward the tempfile to backend.put under the
primary content address); fsync+cleanup on exception. serve_object
passes byte ranges through; verify_object re-reads bytes via backend.get,
re-digests with the stored algorithm, and reports mismatches. delete
and health are thin pass-throughs.
tests/unit/test_dataplane_inproc.py (11 cases):
- ingest_stream computes correct dual digests, returns receipt, stores
bytes at the content-addressed path.
- empty-input ingest returns the BLAKE3/SHA-256 of empty.
- serve_object round-trips ingested bytes; supports byte_range.
- verify_object verifies intact bytes; detects on-disk corruption.
- delete_object passes through (True then False).
- backend_health passes through.
- IngestHints override of primary_algorithm (sha256-as-primary path).
- Missing-object serve raises ObjectNotFoundError.
- Architectural test (ADR-0004 invariant): no control-plane module
(api / registry / retention / audit) imports
artifactstore.storage.backends.* or artifactstore.dataplane.inproc
directly. Enforced via AST scan of every .py file in those packages.
Gates: ruff clean, mypy --strict clean on 44 files, 70 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 02:07:05 +02:00
|
|
|
) -> AsyncIterator[bytes]: ...
|
|
|
|
|
|
2026-05-16 23:26:03 +02:00
|
|
|
async def verify_object(
|
|
|
|
|
self,
|
|
|
|
|
content_address: ContentAddress,
|
|
|
|
|
*,
|
|
|
|
|
backend_id: str | None = None,
|
|
|
|
|
) -> VerifyResult: ...
|
WP-0001-T012: data plane SPI and in-process implementation
src/artifactstore/dataplane/:
- spi.py: DataPlane Protocol with the five operations ingest_stream,
serve_object, verify_object, delete_object, backend_health
(ADR-0004). Dataclasses: IngestHints (size_hint, primary_algorithm,
backend_id overrides), IngestResult (primary_digest + sha256_digest +
size_bytes + StorageReceipt), VerifyResult (verified bool, mismatch
reason, actual digests + size).
- inproc.py: InProcessDataPlane wraps one StorageBackend. ingest_stream
is two-pass against a tempfile (drain stream while dual-hashing into
BLAKE3+SHA-256, then forward the tempfile to backend.put under the
primary content address); fsync+cleanup on exception. serve_object
passes byte ranges through; verify_object re-reads bytes via backend.get,
re-digests with the stored algorithm, and reports mismatches. delete
and health are thin pass-throughs.
tests/unit/test_dataplane_inproc.py (11 cases):
- ingest_stream computes correct dual digests, returns receipt, stores
bytes at the content-addressed path.
- empty-input ingest returns the BLAKE3/SHA-256 of empty.
- serve_object round-trips ingested bytes; supports byte_range.
- verify_object verifies intact bytes; detects on-disk corruption.
- delete_object passes through (True then False).
- backend_health passes through.
- IngestHints override of primary_algorithm (sha256-as-primary path).
- Missing-object serve raises ObjectNotFoundError.
- Architectural test (ADR-0004 invariant): no control-plane module
(api / registry / retention / audit) imports
artifactstore.storage.backends.* or artifactstore.dataplane.inproc
directly. Enforced via AST scan of every .py file in those packages.
Gates: ruff clean, mypy --strict clean on 44 files, 70 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 02:07:05 +02:00
|
|
|
|
2026-05-16 23:26:03 +02:00
|
|
|
async def delete_object(
|
|
|
|
|
self,
|
|
|
|
|
content_address: ContentAddress,
|
|
|
|
|
*,
|
|
|
|
|
backend_id: str | None = None,
|
|
|
|
|
) -> DeletionResult: ...
|
WP-0001-T012: data plane SPI and in-process implementation
src/artifactstore/dataplane/:
- spi.py: DataPlane Protocol with the five operations ingest_stream,
serve_object, verify_object, delete_object, backend_health
(ADR-0004). Dataclasses: IngestHints (size_hint, primary_algorithm,
backend_id overrides), IngestResult (primary_digest + sha256_digest +
size_bytes + StorageReceipt), VerifyResult (verified bool, mismatch
reason, actual digests + size).
- inproc.py: InProcessDataPlane wraps one StorageBackend. ingest_stream
is two-pass against a tempfile (drain stream while dual-hashing into
BLAKE3+SHA-256, then forward the tempfile to backend.put under the
primary content address); fsync+cleanup on exception. serve_object
passes byte ranges through; verify_object re-reads bytes via backend.get,
re-digests with the stored algorithm, and reports mismatches. delete
and health are thin pass-throughs.
tests/unit/test_dataplane_inproc.py (11 cases):
- ingest_stream computes correct dual digests, returns receipt, stores
bytes at the content-addressed path.
- empty-input ingest returns the BLAKE3/SHA-256 of empty.
- serve_object round-trips ingested bytes; supports byte_range.
- verify_object verifies intact bytes; detects on-disk corruption.
- delete_object passes through (True then False).
- backend_health passes through.
- IngestHints override of primary_algorithm (sha256-as-primary path).
- Missing-object serve raises ObjectNotFoundError.
- Architectural test (ADR-0004 invariant): no control-plane module
(api / registry / retention / audit) imports
artifactstore.storage.backends.* or artifactstore.dataplane.inproc
directly. Enforced via AST scan of every .py file in those packages.
Gates: ruff clean, mypy --strict clean on 44 files, 70 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 02:07:05 +02:00
|
|
|
|
|
|
|
|
async def backend_health(self) -> BackendStatus: ...
|