railiance-infra/workplans/RAIL-HO-WP-0011-reproducible-s1-declaration-and-handoff.md
repo-manager b3884a8c2d
All checks were successful
CI Smoke / source-contract (push) Successful in 11s
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s
chore(registrar): assign State Hub identifiers
Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 2583210@bnt-lap001
Assistant-Session: f2bff2d5-e9b2-4338-92ca-10282a927006
2026-08-25 17:46:22 +02:00

314 lines
14 KiB
Markdown

---
id: RAIL-HO-WP-0011
type: workplan
title: "Make the S1 declaration reproducible and the handoff verifiably green"
domain: financials
repo: railiance-infra
status: active
owner: codex
topic_slug: railiance
created: "2026-08-23"
updated: "2026-08-23"
related:
- RAIL-HO-WP-0002
- RAIL-HO-WP-0009
state_hub_workstream_id: "5738f113-1c4e-5d27-95b2-b6655e0b7279"
---
# RAIL-HO-WP-0011 — reproducible S1 declaration and handoff
## Goal
Close the highest-leverage gaps identified in
`history/2026-08-23-scope-against-intent.md`: distinguish adopted resources
from provider-managed resources, make the host baseline contract executable,
produce a green per-host handoff gate, and make every declared secret input
fail closed. Then add the evidence and rotation mechanics needed for an
auditable provisioning path.
This workplan improves the source and validation path. It does **not** authorize
a Terraform apply or destroy, a live firewall change, a credential rotation,
or any other live-host mutation. Those actions require their normal reviewed
plan and operator approval.
## Delivery Order
The critical path is T01 → T02 and T03 → T04 → T05. T06 can proceed
independently. T07 depends on T02 and T05. T08 depends on T06.
## T01 — Define and validate adopted-versus-managed inventory
```task
id: RAIL-HO-WP-0011-T01
status: done
priority: high
state_hub_task_id: "5f89e545-ff95-5216-8369-da10e5eec727"
```
Define one explicit inventory contract for common host identity and
provider-specific lifecycle fields. At minimum it must distinguish:
- provider (`hosteurope`, `hetzner`, or a documented extension value)
- lifecycle mode (`adopted` versus provider-managed)
- connection identity used by Ansible
- provider-specific provisioning fields used by Terraform
Add a side-effect-free validator and fixtures for the two current adopted Host
Europe hosts, a valid provisionable Hetzner host, and invalid mixed records.
Preserve stable host names and connection behavior.
**Done when:** the current inventory validates as adopted Host Europe
resources, a complete Hetzner fixture validates, incomplete or contradictory
records fail with actionable errors, and no provider command is needed to run
the validation.
**Done 2026-08-23.** `scripts/inventory_contract.py` validates schema version,
provider, lifecycle mode, connection identity, baseline profile, and nested
Hetzner provisioning fields. Both current hosts are explicit adopted Host
Europe records; valid Hetzner and invalid mixed fixtures have negative/positive
unit coverage.
## T02 — Make Hetzner planning select only managed Hetzner resources
```task
id: RAIL-HO-WP-0011-T02
status: done
priority: high
state_hub_task_id: "66024380-3afb-50a7-8908-02eadd96925a"
```
Update the Terraform and helper path to consume the T01 contract and exclude
adopted or non-Hetzner records. Audit any existing Terraform state before
changing resource addresses or selection logic. Correct provisioning docs and
Make targets so they name the provider-token source actually used.
Add an isolated plan test or equivalent deterministic validation that proves
the current Host Europe records cannot become Hetzner creates or destroys.
**Done when:** current adopted records pass through the inventory-to-Terraform
boundary without missing-field errors or managed resources, a provisionable
Hetzner fixture produces the expected resource shape, and a reviewed plan shows
no unintended create, replace, or destroy. Do not apply the plan in this task.
**Done 2026-08-23.** Terraform filters on `provider: hetzner` plus
`lifecycle_mode: provider-managed`; the shared SSH-key resource is also absent
when selection is empty. No local Terraform state exists in the module. In an
isolated Terraform 1.9.8 container with mocked providers, the adopted-only plan
selected zero resources and the managed fixture selected exactly its named
host (2/2 native Terraform tests passed). Apply/destroy Make targets now refuse
before init without exact approval variables; no provider mutation occurred.
## T03 — Model host-specific baseline profiles and reconcile declared state
```task
id: RAIL-HO-WP-0011-T03
status: done
priority: high
state_hub_task_id: "fdfc7974-c491-555d-8fbd-6b2b42dd7ebb"
```
Replace the permanent `CoulombCore` expected failure with an explicit baseline
profile or declared exception. Reconcile `spec/server-baseline.yaml`, Ansible,
and the intended live posture for packages, users/sudo, SSH, firewall,
fail2ban, SOPS tooling, swap, and resource limits.
Exceptions must say which control replaces the default, why it is accepted,
and who owns removing it. An unmanaged control must not silently count as
verified.
**Done when:** both current hosts resolve to complete declared profiles, the
UFW-managed and externally-filtered postures have explicit assertions, and the
spec no longer claims properties that convergence neither establishes nor
deliberately delegates.
**Done 2026-08-23.** `spec/server-baseline.yaml` v2 defines shared defaults and
the `ufw-managed` / `external-firewall` profiles. Dynamic inventory resolves
the model into both consumers. The external profile carries its replacement
control, owner, removal condition, and an iptables INPUT default-drop check.
Both rendered Goss files parse as YAML and contain their distinct firewall
assertions. No live firewall change occurred.
## T04 — Add automated baseline contract-parity tests
```task
id: RAIL-HO-WP-0011-T04
status: done
priority: high
state_hub_task_id: "188321a8-d39e-5e88-9acd-986765699dbd"
```
Add repository tests that fail when the human baseline, convergence roles, and
Goss assertions diverge on governed properties. Prefer a single
machine-readable model or generated artifacts where that reduces duplicated
declarations; otherwise implement explicit parity checks with clear failure
messages.
Cover the profiles introduced by T03 and run the tests in Forgejo CI without
requiring host access or secrets.
**Done when:** a deliberate mismatch in a governed package, SSH setting,
firewall rule, service, or user property fails locally and in CI, while all
declared profiles pass from a clean checkout.
**Done 2026-08-23.** The baseline validator enforces governed
minimum packages, services, SSH directives, user/sudo posture, both required
profiles, and consumer markers. Unit tests prove deliberate `htop` removal and
SSH weakening fail. Forgejo Actions run 79 passed the source-contract,
host-smoke, and container-smoke jobs for revision `4f2312a`; the source job
runs in the repository's disposable `ubuntu-latest` lane and needs no host
access or secret.
## T05 — Provide a fresh green S1 handoff gate
```task
id: RAIL-HO-WP-0011-T05
status: wait
priority: high
state_hub_task_id: "ba526585-6141-5df1-9094-a1322394d8b5"
```
Provide one operator-facing, non-ambiguous command that checks inventory and
contract validity, runs the applicable profile for every selected host,
collects evidence, and exits non-zero if any required control is failed,
unknown, stale, or skipped without an accepted declaration.
The receipt must identify the source revision, inventory digest, host/profile,
check time, and result without including credentials or sensitive host output.
Define freshness and the exact interface S2 can consume.
**Done when:** both current hosts can return green against their declared
profiles in an attended verification run, a failing or stale host makes the
aggregate gate fail, and the evidence is sufficient for an S2 handoff without
interpreting an expected-red exception.
**Implemented; live gate pending 2026-08-23.** `scripts/s1_handoff.py` and
`make s1-handoff` validate source contracts, require a clean revision, run each
host separately, fail the aggregate on any non-zero result, require TAP hashes
for a pass, and record a freshness boundary. Dry-run output is forcibly
`not-run`. The live playbook is now remotely read-only: a static contract
rejects mutating modules and arbitrary commands, it requires the installed
Goss surface to match the locally rendered profile digest, and it writes only
controller-side evidence. Refresh is a separate exact-approval interface. The
attended all-host run now waits only for Ansible plus short-lived SSH access;
no host was contacted.
## T06 — Repair and enforce the secret-source contract
```task
id: RAIL-HO-WP-0011-T06
status: done
priority: high
state_hub_task_id: "040b5cdb-6e48-588d-b472-f734b99ad4b2"
```
Remove, relocate, or SOPS-encrypt the plaintext
`inventory/group_vars/secrets.sops.yaml` placeholder. Inventory every path that
documentation or automation describes as secret-bearing, extend the
pre-commit/CI check to those paths, and make playbooks fail clearly when an
expected encrypted input is absent or malformed.
Align README, provisioning, convergence, Make targets, and playbooks on the
actual Hetzner token source and on the fact that the host role installs tools
but does not distribute an age private key.
**Done when:** plaintext fixtures at any declared secret path are rejected,
the committed tree contains no file falsely presented as encrypted input, and
the documented controller and host secret flows match the executable paths.
**Done 2026-08-23.** The plaintext placeholder was removed and bootstrap no
longer loads it. `scripts/check_secret_paths.py` protects both `secrets/` and
inventory `secrets*` paths in the pre-commit hook, Make target, and CI. Docs,
helpers, and Make consistently use `secrets/hetzner-token.yaml` field
`hetzner.token`; tests reject plaintext and empty encrypted-file fixtures.
## T07 — Emit non-secret provisioning and handoff receipts
```task
id: RAIL-HO-WP-0011-T07
status: done
priority: medium
state_hub_task_id: "d296b442-83df-5bb8-abed-bbb848477a26"
```
Define a versioned receipt linking inventory and source revisions, provider
plan/apply identity, provider resource ID, cloud-init completion, Ansible
convergence, and the T05 handoff result. Separate plan, apply, and verification
events so a plan receipt cannot be mistaken for proof of a live change.
Exercise the format with a dry-run or synthetic fixture first. A live provider
apply remains separately approved and is not required merely to prove schema
and redaction behavior.
**Done when:** the receipt schema validates, sensitive fields and provider
tokens cannot be serialized, incomplete phases fail closed, and a synthetic
end-to-end example is consumable without prose interpretation.
**Done 2026-08-23.** `schemas/s1-receipt.schema.json` documents the v1 shape
and `scripts/s1_receipt.py` enforces phase-specific completeness plus recursive
secret-shaped key/value rejection. The committed synthetic chain separates all
five phases and validates; tests reject an incomplete passing chain, a passing
verification without host evidence, and a token-shaped field.
## T08 — Automate bounded SOPS recipient rotation
```task
id: RAIL-HO-WP-0011-T08
status: wait
priority: medium
state_hub_task_id: "920f869a-2554-58a9-b0da-8a0bbd7c5ef1"
```
Build a dry-run-first rotation workflow for the repository's SOPS files. It
must inventory affected files and recipients, detect recipient drift, require a
reviewed change set, verify decryption through an approved non-printing check,
and emit a metadata-only receipt. Never print a decrypted value or private age
key.
Live recipient removal or credential replacement requires explicit operator
approval after the dry run. Scheduling may be proposed only after rollback and
recovery-key custody are documented.
**Done when:** CI can detect recipient drift without secrets, an attended dry
run identifies the exact files and before/after recipient set, rollback is
documented, and a sample receipt proves verification without exposing values.
**Implemented; attended verification pending 2026-08-23.**
`scripts/sops_rotation.py` reports zero metadata drift for the current file,
supports null-output decryption verification, exact approval-file binding, and
suppressed-output `sops updatekeys`; CI runs metadata-only `--check`. The local
workstation has no `sops` executable or approved age-key session, so no passing
decryption receipt or recipient change was attempted.
## Acceptance
- [x] Adopted Host Europe records and provider-managed Hetzner records cannot
be confused by validation or Terraform selection.
- [x] The declared baseline, Ansible convergence, and Goss verification have an
automated parity contract.
- [ ] Every current host can produce a fresh green result against an explicit
profile, and the aggregate handoff fails closed.
- [x] Every documented secret-bearing path is encrypted or deliberately absent
and protected by local and CI checks.
- [ ] Provisioning/handoff and rotation paths emit metadata-only receipts with
reviewed redaction behavior.
- [x] No live provider, firewall, credential, or host mutation occurs without a
separately reviewed plan and the required operator approval.
## Completion Evidence
Current evidence (2026-08-23):
- Python unit suite: 30 tests pass, including rejection of a remote template,
arbitrary remote command, non-delegated controller write, and unreviewed
Goss command surface in the handoff path.
- Terraform 1.9.8 mock-provider tests: 2 pass.
- Ansible 2.17.13 syntax checks: bootstrap, read-only verify, explicit
verify-refresh, and firewall pass in a disposable environment. A disposable
render comparison also proves the template lookup digest matches the bytes
produced by Ansible's deployment template action.
- Both profile-specific Goss templates render and parse as YAML.
- Inventory, baseline, secret metadata, receipt, shell syntax, Python compile,
and whitespace checks pass.
- Forgejo Actions run 79 is green for revision `4f2312a` across all three jobs.
- Pending before finish: an attended fresh all-host handoff receipt and an
attended non-printing SOPS decryption receipt.