build: package CLI with pinned producer verifier

Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a140c4-354a-7823-8f0d-8f717397f786

README.md

@@ -86,3 +86,5 @@ speaks in lanes and guarded plans.
   not write.
 - Temporary bootstrap OpenBao credentials must live outside repos, use mode
   0600, be revocable, and be removed after narrower auth is working.
+
+Packaged CLI deployment uses the reviewed [wheel + verifier bundle](docs/cli-release-bundle.md). A Python wheel alone omits the mandatory signature helper.

docs/cli-release-bundle.md

@@ -0,0 +1,82 @@
+# CLI release bundle with authenticated decision verifier
+
+The supported packaged CLI path is an explicit wheel-plus-verifier bundle. The
+existing editable `uv` workflow remains for development. Installing the Python
+wheel alone omits the Go helper and is incomplete for production authorization;
+it continues to fail closed. This is a CLI package, not a service or the retired
+`rapp-secrets-engine` deployment.
+
+The bundle contains the Secrets wheel, the existing PyYAML runtime wheel pinned
+by `uv.lock`, the verifier executable, producer license attribution, a narrow CLI
+entry point and an offline installer. It never contains the test signer, signing
+seed, caller credential, runtime trust/key config or backend entitlement. Optional
+`clock` extras are outside this base bundle; configuring clock trust without the
+separately reviewed extra continues to refuse, rather than fall back.
+
+## Build
+
+Use Python >=3.11, uv and the Go toolchain required by `verifier/go.mod` on the
+intended Linux architecture. Supply a local compatible PyYAML wheel whose exact
+filename and SHA-256 appear in `uv.lock`. The builder never discovers/downloads a
+runtime dependency. The existing Hatchling build backend is now version-pinned.
+
+```bash
+python3 scripts/build_release_bundle.py \
+  --output /tmp/secrets-cli-reviewed-build \
+  --dependency-wheel /path/to/locked-compatible-pyyaml.whl
+```
+
+Builds are offline by default. In a reviewed build environment, the explicit
+`--allow-build-downloads` flag permits uv to fetch the existing pinned build
+backend and its build dependencies. It does not change the runtime dependency
+contract. Set writable uv/Go caches as required by that build environment.
+
+The builder first verifies the exact producer API/signature source and native
+fixture hashes, builds only the verifier root package with CGO disabled, explicit
+host architecture, trimmed paths, disabled VCS stamping and an empty build ID,
+then builds the wheel with commit-derived `SOURCE_DATE_EPOCH`. Two builds using
+the same source snapshot/toolchain/dependency wheel produce matching artifacts
+and manifest. The manifest records exact source-file and artifact SHA-256s, base
+commit, producer snapshot, platform and tool versions. It contains no observed
+build timestamp; retain observed execution time in a separate receipt.
+
+## Review and offline install
+
+**Checksums prove artifact integrity, not independent package admission.** The
+owner must review/admit the source, toolchain, complete bundle and manifest hash,
+and control the bundle/install prefix. A manifest obtained from an untrusted
+bundle does not establish that admission. Key/trust admission remains a distinct
+WP14 gate. None of these steps authorizes protected OpenBao work.
+
+Execute the reviewed installer with the independently retained manifest digest:
+
+```bash
+python3 /path/to/reviewed-bundle/install.py \
+  --prefix /path/to/new-owner-controlled-prefix \
+  --manifest-sha256 REVIEWED_MANIFEST_SHA256
+/path/to/new-owner-controlled-prefix/bin/secrets-engine --help
+```
+
+The parent directory must already be owner controlled. The installer validates
+platform, manifest and the entire artifact set before creating a private new
+prefix, creates its venv through the existing uv convention, and installs entirely
+offline from bundled wheels. It never replaces an existing prefix or installs
+into standard user/system tools. A failed partial prefix is retained and marked;
+retry with a new prefix after diagnosis. Removing a failed/disposable prefix is
+an explicit owner operation, not an automatic cleanup of an existing path.
+
+Use `PREFIX/bin/secrets-engine` as the admitted CLI entry point. It checks the
+bundled helper's SHA-256 before CLI import or credential access and fixes
+`SECRETS_ENGINE_PDP_VERIFIER` to that helper; ambient executable overrides do not
+select another verifier. Missing or changed helpers refuse. There is no runtime
+compilation or unsigned fallback. The internal venv scripts/programmatic API are
+not a second admitted entry point and do not create package/trust admission.
+Catalog/policy and independently reviewed responder trust configuration still
+belong to the deploying owner. Neither is silently copied from a developer repo.
+
+For source verification, supply `SECRETS_ENGINE_TEST_DEPENDENCY_WHEEL` and run
+`pytest tests/test_release_bundle.py`. The disposable proof compares two builds,
+installs offline, invokes the installed wheel's signed HTTP boundary with public
+synthetic fixtures, rejects unsigned decisions, checks helper/bundle tampering,
+source drift and existing-prefix refusal. No native credentials or protected
+backend are involved.

pyproject.toml

@@ -20,7 +20,7 @@ dev = [
 secrets-engine = "secrets_engine.cli:main"
 
 [build-system]
-requires = ["hatchling"]
+requires = ["hatchling==1.27.0"]
 build-backend = "hatchling.build"
 
 [tool.hatch.build.targets.wheel]

scripts/build_release_bundle.py

@@ -0,0 +1,86 @@
+#!/usr/bin/env python3
+"""Build wheel + pinned producer verifier; install is a separate reviewed step."""
+import argparse
+import hashlib
+import json
+import os
+from pathlib import Path
+import platform
+import shutil
+import subprocess
+import sys
+import tomllib
+
+
+def digest(path):
+    return hashlib.sha256(path.read_bytes()).hexdigest()
+
+
+def run(command, **kwargs):
+    return subprocess.check_output(command,text=True,**kwargs).strip()
+
+
+def build(root, output, dependency_wheel, uv, allow_build_downloads=False):
+    if platform.system() != 'Linux':
+        raise ValueError('this bundle supports Linux CLI only')
+    # No discovered keys, source rewriting, or runtime compilation.
+    subprocess.run([sys.executable,str(root/'scripts/check_pdp_producer_source.py')],check=True,stdout=sys.stderr)
+    locked=next(p for p in tomllib.loads((root/'uv.lock').read_text())['package'] if p['name']=='pyyaml')
+    admitted={Path(item['url']).name:item['hash'].removeprefix('sha256:') for item in locked['wheels']}
+    if dependency_wheel.name not in admitted or digest(dependency_wheel)!=admitted[dependency_wheel.name]:
+        raise ValueError('supplied PyYAML wheel does not match uv.lock')
+    output.mkdir(mode=0o700,parents=False,exist_ok=False)
+    epoch=run(['git','log','-1','--format=%ct'],cwd=root)
+    commit=run(['git','rev-parse','HEAD'],cwd=root)
+    architecture={'x86_64':'amd64','aarch64':'arm64'}.get(platform.machine())
+    if architecture is None:
+        raise ValueError('unsupported Linux architecture')
+    env={**os.environ,'CGO_ENABLED':'0','GOOS':'linux','GOARCH':architecture,
+         'GOFLAGS':'','GOWORK':'off','SOURCE_DATE_EPOCH':epoch}
+    (output/'bin').mkdir();(output/'wheels').mkdir()
+    subprocess.run(['go','build','-trimpath','-buildvcs=false','-ldflags=-buildid=',
+                    '-o',str(output/'bin/secrets-engine-pdp-verify'),'.'],cwd=root/'verifier',env=env,check=True)
+    command=[uv,'--no-config','build','--wheel','--out-dir',str(output/'wheels'),str(root)]
+    if not allow_build_downloads:command.append('--offline')
+    subprocess.run(command,env=env,check=True)
+    wheel,=list((output/'wheels').glob('secrets_engine-*.whl'))
+    shutil.copyfile(dependency_wheel,output/'wheels'/dependency_wheel.name)
+    helper_hash=digest(output/'bin/secrets-engine-pdp-verify')
+    launcher=(root/'scripts/release_cli.py').read_text().replace('@HELPER_SHA256@',helper_hash)
+    (output/'release_cli.py').write_text(launcher)
+    shutil.copyfile(root/'scripts/install_release_bundle.py',output/'install.py')
+    shutil.copyfile(root/'verifier/LICENSE.access-engine',output/'LICENSE.access-engine')
+    sources=[root/'pyproject.toml',root/'uv.lock',root/'verifier/producer-source.json']
+    sources+=list((root/'src/secrets_engine').glob('*.py'))
+    sources+=list((root/'verifier/pkg/api').glob('*.go'))+list((root/'verifier/internal/sign').glob('*.go'))
+    sources+=[root/'verifier/main.go',root/'scripts/check_pdp_producer_source.py',
+              root/'scripts/build_release_bundle.py',root/'scripts/install_release_bundle.py',root/'scripts/release_cli.py']
+    source_hashes={str(path.relative_to(root)):digest(path) for path in sorted(sources)}
+    manifest={"schema":"secrets-engine.cli-bundle.v1","source_commit":commit,"source_files":source_hashes,
+              "source_date_epoch":int(epoch),"system":platform.system(),"machine":platform.machine(),
+              "python":platform.python_version(),"go":run(['go','version']),"uv":run([uv,'--version']),
+              "build_backend":"hatchling==1.27.0","wheel":str(wheel.relative_to(output)),
+              "producer":json.loads((root/'verifier/producer-source.json').read_text()),
+              "artifacts":{str(path.relative_to(output)):digest(path) for path in sorted(output.rglob('*')) if path.is_file()},
+              "admission":"checksums are integrity evidence; owner must admit the bundle/prefix independently"}
+    (output/'manifest.json').write_text(json.dumps(manifest,sort_keys=True,indent=2)+'\n')
+    return {'bundle':str(output),'manifest_sha256':digest(output/'manifest.json'),'helper_sha256':helper_hash}
+
+
+def main():
+    parser=argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('--output',type=Path,required=True)
+    parser.add_argument('--dependency-wheel',type=Path,required=True)
+    parser.add_argument('--uv',default='uv')
+    parser.add_argument('--allow-build-downloads',action='store_true',help='permit uv to fetch existing build-system dependencies')
+    args=parser.parse_args()
+    try:
+        print(json.dumps(build(Path(__file__).resolve().parents[1],args.output.resolve(),args.dependency_wheel.resolve(),
+                               args.uv,args.allow_build_downloads),sort_keys=True))
+    except (OSError,ValueError,KeyError,subprocess.SubprocessError) as exc:
+        print('bundle build refused: '+str(exc),file=sys.stderr);return 1
+    return 0
+
+
+if __name__=='__main__':
+    raise SystemExit(main())

scripts/install_release_bundle.py

@@ -0,0 +1,82 @@
+#!/usr/bin/env python3
+"""Install one reviewed CLI bundle offline into a new owner-controlled prefix."""
+import argparse
+import hashlib
+import json
+from pathlib import Path
+import platform
+import shutil
+import subprocess
+import sys
+
+
+def digest(path):
+    return hashlib.sha256(path.read_bytes()).hexdigest()
+
+
+def install(bundle, prefix, expected_manifest, python, uv):
+    manifest_path = bundle / "manifest.json"
+    if digest(manifest_path) != expected_manifest:
+        raise ValueError("bundle manifest differs from reviewed digest")
+    manifest = json.loads(manifest_path.read_text())
+    if manifest["schema"] != "secrets-engine.cli-bundle.v1":
+        raise ValueError("unsupported bundle")
+    if manifest['system'] != platform.system() or manifest['machine'] != platform.machine():
+        raise ValueError("bundle platform differs from installer host")
+    expected_files = set(manifest['artifacts']) | {'manifest.json'}
+    actual_files = {str(p.relative_to(bundle)) for p in bundle.rglob('*') if p.is_file()}
+    if actual_files != expected_files:
+        raise ValueError("bundle file set differs")
+    for name, expected in manifest['artifacts'].items():
+        relative = Path(name)
+        if relative.is_absolute() or '..' in relative.parts:
+            raise ValueError("invalid artifact path")
+        artifact = bundle / relative
+        if any(parent.is_symlink() for parent in (artifact,*artifact.parents)) or digest(artifact) != expected:
+            raise ValueError("bundle artifact missing or changed")
+    # Reserve once; never replace an installed prefix or follow an existing link.
+    prefix.mkdir(mode=0o700, parents=False, exist_ok=False)
+    try:
+        subprocess.run([uv,'--no-config','venv','--python',python,'--no-python-downloads',str(prefix/'venv')],check=True)
+        subprocess.run([uv,'--no-config','pip','install','--python',str(prefix/'venv/bin/python'),
+                        '--no-index','--find-links',str(bundle/'wheels'),str(bundle/manifest['wheel'])],check=True)
+        (prefix/'libexec').mkdir(mode=0o700)
+        helper=prefix/'libexec/secrets-engine-pdp-verify'
+        shutil.copyfile(bundle/'bin/secrets-engine-pdp-verify',helper);helper.chmod(0o700)
+        (prefix/'bin').mkdir(mode=0o700)
+        launcher=prefix/'bin/secrets-engine'
+        # An absolute venv interpreter fixes the installed dependency environment.
+        launcher.write_text('#!'+str(prefix/'venv/bin/python')+'\n'+(bundle/'release_cli.py').read_text())
+        launcher.chmod(0o700)
+        receipt={"schema":"secrets-engine.cli-install.v1","manifest_sha256":expected_manifest,
+                 "helper_sha256":digest(helper),"launcher_sha256":digest(launcher),"prefix":str(prefix),
+                 "admission":"artifact integrity only; trust and keys not admitted"}
+        (prefix/'install-receipt.json').write_text(json.dumps(receipt,sort_keys=True,indent=2)+'\n')
+        return receipt
+    except BaseException:
+        # Preserve a failed prefix as evidence; never retry into a partial install.
+        (prefix/'INSTALL-FAILED').write_text('Incomplete installation; use a new prefix.\n')
+        raise
+
+
+def main():
+    parser=argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('--bundle',type=Path,default=Path(__file__).resolve().parent)
+    parser.add_argument('--prefix',type=Path,required=True)
+    parser.add_argument('--manifest-sha256',required=True)
+    parser.add_argument('--python',default=sys.executable)
+    parser.add_argument('--uv',default='uv')
+    args=parser.parse_args()
+    try:
+        if not args.prefix.is_absolute() or args.prefix.parent.is_symlink():
+            raise ValueError('prefix must be absolute with an admitted parent')
+        receipt=install(args.bundle.resolve(),args.prefix,args.manifest_sha256,args.python,args.uv)
+        print(json.dumps(receipt,sort_keys=True))
+    except (OSError,ValueError,KeyError,subprocess.SubprocessError) as exc:
+        print('bundle install refused: '+str(exc),file=sys.stderr)
+        return 1
+    return 0
+
+
+if __name__=='__main__':
+    raise SystemExit(main())

scripts/release_cli.py

@@ -0,0 +1,29 @@
+"""Narrow installed CLI entry point; rendered by build_release_bundle.py."""
+import hashlib
+import os
+from pathlib import Path
+import sys
+
+EXPECTED_HELPER_SHA256 = "@HELPER_SHA256@"
+
+
+def main():
+    prefix = Path(__file__).resolve().parent.parent
+    helper = prefix / "libexec" / "secrets-engine-pdp-verify"
+    try:
+        if helper.is_symlink() or not helper.is_file():
+            raise ValueError("missing helper")
+        if hashlib.sha256(helper.read_bytes()).hexdigest() != EXPECTED_HELPER_SHA256:
+            raise ValueError("changed helper")
+    except (OSError, ValueError):
+        print("secrets-engine: installed verifier missing or changed", file=sys.stderr)
+        return 1
+    # The bundle always uses its reviewed helper; ambient overrides do not select
+    # another executable. The owner admits runtime trust separately.
+    os.environ["SECRETS_ENGINE_PDP_VERIFIER"] = str(helper)
+    from secrets_engine.cli import main as cli_main
+    return cli_main()
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

tests/test_release_bundle.py

@@ -0,0 +1,137 @@
+"""Actual offline installation proof; only supplied locked public dependency wheel.
+
+Set SECRETS_ENGINE_TEST_DEPENDENCY_WHEEL to a local PyYAML wheel from uv.lock.
+No production install, trust admission, backend or credential is used.
+"""
+import hashlib
+import json
+import os
+from pathlib import Path
+import shutil
+import subprocess
+import sys
+
+import pytest
+
+from tests.test_decision_signature import FIXTURE, PUBLIC
+from tests.test_decision_transport import server
+
+ROOT = Path(__file__).resolve().parents[1]
+
+
+def run(command, **kwargs):
+    return subprocess.run(command, text=True, capture_output=True, timeout=120, **kwargs)
+
+
+@pytest.fixture(scope='module')
+def bundles(tmp_path_factory):
+    dependency=os.environ.get('SECRETS_ENGINE_TEST_DEPENDENCY_WHEEL')
+    if not dependency:
+        pytest.skip('provide the locked public PyYAML wheel for actual bundle build/install tests')
+    directory=tmp_path_factory.mktemp('cli-bundles')
+    result=[]
+    for name in ['one','two']:
+        bundle=directory/name
+        built=run([sys.executable,str(ROOT/'scripts/build_release_bundle.py'),'--output',str(bundle),
+                   '--dependency-wheel',dependency])
+        assert built.returncode==0,built.stderr
+        result.append(bundle)
+    return result
+
+
+def manifest_hash(bundle):
+    return hashlib.sha256((bundle/'manifest.json').read_bytes()).hexdigest()
+
+
+def install(bundle,prefix):
+    return run([sys.executable,str(bundle/'install.py'),'--bundle',str(bundle),'--prefix',str(prefix),
+                '--manifest-sha256',manifest_hash(bundle)])
+
+
+@pytest.fixture
+def installed(bundles,tmp_path):
+    prefix=tmp_path/'installed'
+    result=install(bundles[0],prefix)
+    assert result.returncode==0,result.stderr
+    return prefix
+
+
+def test_two_builds_match_and_contain_no_signer(bundles):
+    one,two=bundles
+    names={str(path.relative_to(one)) for path in one.rglob('*') if path.is_file()}
+    assert names=={str(path.relative_to(two)) for path in two.rglob('*') if path.is_file()}
+    for name in names:
+        assert (one/name).read_bytes()==(two/name).read_bytes(),name
+    assert not any('signer' in name or 'testsupport' in name for name in names)
+    assert (one/'bin/secrets-engine-pdp-verify').is_file()
+    assert len(list((one/'wheels').glob('secrets_engine-*.whl')))==1
+
+
+def test_clean_install_uses_installed_wheel_and_signed_boundary(installed,tmp_path):
+    env={**os.environ,'SECRETS_ENGINE_PDP_VERIFIER':'/must-not-use-ambient-override'}
+    help_result=run([str(installed/'bin/secrets-engine'),'--help'],env=env,cwd='/tmp')
+    assert help_result.returncode==0,help_result.stderr
+    envelope=json.loads(FIXTURE.read_bytes())
+    trust={name:envelope['provenance'][name] for name in ('evaluator','mode','policy_package','policy_version','policy_package_digest')}
+    trust.update(schema='secrets-engine.pdp-trust.v1',keys={'testdata-ed25519':PUBLIC})
+    trust_file=tmp_path/'synthetic-trust.json';trust_file.write_text(json.dumps(trust))
+    token=tmp_path/'synthetic.token';token.write_text('synthetic-pdp-credential');token.chmod(0o600)
+    script='''import pathlib,sys
+import secrets_engine.decision_check as installed
+from secrets_engine.errors import DecisionError
+assert str(pathlib.Path(installed.__file__).resolve()).startswith(sys.argv[1]+'/venv/')
+try:
+ result=installed.check_decision(base_url=sys.argv[2],token_file=pathlib.Path(sys.argv[3]),request={},trust_file=pathlib.Path(sys.argv[4]),verifier=pathlib.Path(sys.argv[1]+'/libexec/secrets-engine-pdp-verify'))
+except DecisionError:
+ if sys.argv[5]=='refuse':sys.exit(0)
+ raise
+assert sys.argv[5]=='accept' and result['effect']=='allow'
+'''
+    clean_env={key:value for key,value in os.environ.items() if key not in ('PYTHONPATH','PYTHONHOME')}
+    for body,expect in [(FIXTURE.read_bytes(),'accept'),(b'{"effect":"allow"}','refuse')]:
+        with server(body=body) as (origin,seen):
+            result=run([str(installed/'venv/bin/python'),'-c',script,str(installed),origin,str(token),str(trust_file),expect],env=clean_env,cwd='/tmp')
+            assert result.returncode==0,result.stderr
+            assert seen==[True]
+
+
+@pytest.mark.parametrize('change',['missing','tampered'])
+def test_installed_cli_refuses_missing_or_tampered_helper(installed,change):
+    helper=installed/'libexec/secrets-engine-pdp-verify'
+    if change=='missing':helper.rename(helper.with_suffix('.held'))
+    else:helper.write_bytes(helper.read_bytes()+b'tampered')
+    result=run([str(installed/'bin/secrets-engine'),'--help'],cwd='/tmp')
+    assert result.returncode==1
+    assert 'verifier missing or changed' in result.stderr
+    assert not result.stdout
+
+
+def test_installer_refuses_changed_artifact_and_existing_prefix(bundles,tmp_path):
+    altered=tmp_path/'altered';shutil.copytree(bundles[0],altered)
+    helper=altered/'bin/secrets-engine-pdp-verify';helper.write_bytes(helper.read_bytes()+b'tampered')
+    prefix=tmp_path/'must-not-create'
+    result=install(altered,prefix)
+    assert result.returncode==1 and not prefix.exists()
+    existing=tmp_path/'existing';existing.mkdir();marker=existing/'keep';marker.write_text('unchanged')
+    result=install(bundles[0],existing)
+    assert result.returncode==1 and marker.read_text()=='unchanged'
+    assert sorted(path.name for path in existing.iterdir())==['keep']
+
+
+def test_installer_requires_exact_reviewed_manifest(bundles,tmp_path):
+    prefix=tmp_path/'must-not-create'
+    result=run([sys.executable,str(bundles[0]/'install.py'),'--prefix',str(prefix),'--manifest-sha256','0'*64])
+    assert result.returncode==1 and not prefix.exists()
+
+
+def test_changed_producer_source_stops_before_build(tmp_path):
+    root=tmp_path/'source';(root/'scripts').mkdir(parents=True)
+    for name in ['build_release_bundle.py','check_pdp_producer_source.py']:
+        shutil.copyfile(ROOT/'scripts'/name,root/'scripts'/name)
+    shutil.copytree(ROOT/'verifier',root/'verifier')
+    shutil.copytree(ROOT/'tests/fixtures/pdp-signature',root/'tests/fixtures/pdp-signature')
+    source=root/'verifier/internal/sign/sign.go';source.write_bytes(source.read_bytes()+b'// drift\n')
+    output=tmp_path/'must-not-build'
+    result=run([sys.executable,str(root/'scripts/build_release_bundle.py'),'--output',str(output),
+                '--dependency-wheel',str(tmp_path/'not-reached.whl')])
+    assert result.returncode==1 and not output.exists()

verifier/README.md

@@ -28,3 +28,8 @@ The Check API defaults to mandatory authentication. Its deliberately named
 `allow_unsigned_test_fixture` argument is used only by synthetic transport tests;
 production `authorize_action` never passes it. The helper runs with an empty
 environment, so ambient backend and identity credentials cannot reach it.
+
+For the complete reproducible CLI build and offline fresh-prefix installation,
+use [the release bundle contract](../docs/cli-release-bundle.md). This replaces
+manual helper copying for packaged deployments; package and key admission remain
+separate owner decisions.

workplans/SECRETS-WP-0014-pdp-transport-and-responder-authentication.md

@@ -77,6 +77,15 @@ key admission remains part of its scope; T03 stays wait. WP13 execution and
 identity-only CCR38 are unchanged. Custodian source/verification handoff:
 `docs/changes/netkingdom-target-2026-10-16/decision-auth/` in the Custodian repo.
 
+2026-10-16 packaging preparation: `scripts/build_release_bundle.py` and its
+separate offline installer deliver the existing wheel plus the reviewed producer
+verifier together into a new owner-controlled prefix. Source/hash drift checks,
+artifact/toolchain receipts and a narrow fixed-helper launcher remove the silent
+wheel-only deployment gap. Disposable build/install tests prove deterministic
+artifacts, signed HTTP consumption and missing/tampered artifact refusal; they do
+not admit packages, keys or a production prefix. T02 remains progress and T03
+wait; no native installation, trust change or protected entitlement is claimed.
+
 ## Verify the admitted native consumer and rotation path
 
 ```task