2026-07-07 20:04:54 +02:00
|
|
|
|
# Resilience Policy
|
|
|
|
|
|
|
|
|
|
|
|
**Status:** adopted (initial)
|
|
|
|
|
|
**Updated:** 2026-07-07
|
|
|
|
|
|
**Owner:** Bernd
|
|
|
|
|
|
**Scope:** Coulomb Social / Helix Forge
|
|
|
|
|
|
|
|
|
|
|
|
## Operating principle
|
|
|
|
|
|
|
|
|
|
|
|
> Every capability that becomes valuable must declare its source of truth,
|
|
|
|
|
|
> recovery method, backup schedule, RTO, RPO, owner, and restore test evidence.
|
|
|
|
|
|
|
|
|
|
|
|
Servers are disposable. Data, identity, secrets, evidence, and payment records
|
|
|
|
|
|
are not. Recovery means **rebuild** (Ansible/IaC + data restore + secret
|
|
|
|
|
|
recovery + DNS cutover), not server resurrection.
|
|
|
|
|
|
|
|
|
|
|
|
After a security breach: **never restore into the compromised machine.** Build
|
|
|
|
|
|
clean, restore known-good data, rotate credentials, preserve evidence.
|
|
|
|
|
|
|
|
|
|
|
|
## Asset tiers
|
|
|
|
|
|
|
|
|
|
|
|
| Tier | Definition | Requirement |
|
|
|
|
|
|
| --- | --- | --- |
|
|
|
|
|
|
| **T0** | Identity / trust / money | Recoverable even after breach; recovery path documented |
|
|
|
|
|
|
| **T1** | Irreplaceable source of truth | Strong backups, tested restore, offsite copy where feasible |
|
|
|
|
|
|
| **T2** | Runtime platforms | Rebuild from IaC + restore data |
|
|
|
|
|
|
| **T3** | Derived / rebuildable | Recompute where possible |
|
|
|
|
|
|
| **T4** | External dependencies | Export, monitor, fallback, document limits |
|
|
|
|
|
|
|
|
|
|
|
|
Canonical inventory: `AssetRegister.yaml`, `DataRegister.yaml`.
|
|
|
|
|
|
|
|
|
|
|
|
## Recovery objectives
|
|
|
|
|
|
|
|
|
|
|
|
Global baseline:
|
|
|
|
|
|
|
|
|
|
|
|
- **RPO ≤ 24h** everywhere (unless vendor-limited)
|
|
|
|
|
|
- **RTO ≤ 1 day** for full organizational recovery
|
|
|
|
|
|
- **RTO ≤ 4–8h** for code, content, and control-plane recovery
|
|
|
|
|
|
|
|
|
|
|
|
Per-system targets: `RTO-RPO-Matrix.md`.
|
|
|
|
|
|
Backup mechanisms: `BackupPolicy.md`.
|
|
|
|
|
|
|
|
|
|
|
|
## Drill and evidence cadence
|
|
|
|
|
|
|
|
|
|
|
|
| Activity | Frequency | Output |
|
|
|
|
|
|
| --- | --- | --- |
|
|
|
|
|
|
| Backup job success check (M-1) | Daily | Operator review / log check |
|
|
|
|
|
|
| Backup integrity check (M-3) | Weekly | age-decrypt sample per lane |
|
|
|
|
|
|
| T1 restore drill | Quarterly | `TestEvidence/restore-drill-YYYY-MM-DD-<system>.md` |
|
|
|
|
|
|
| T0 cold decrypt drill | Quarterly | Evidence file + register update |
|
|
|
|
|
|
| Full server rebuild drill | Quarterly | `RestoreRunbooks/01-full-server-loss.md` + evidence |
|
|
|
|
|
|
| Asset register review | Monthly | Update `AssetRegister.yaml` statuses |
|
|
|
|
|
|
| Critical secret rotation | Quarterly or after incident | Evidence note in State Hub |
|
|
|
|
|
|
| Cross-repo pickup queue review | Monthly | Update `BackupPickupQueue.md` |
|
|
|
|
|
|
|
|
|
|
|
|
A workplan is not **finished** until at least one restore drill evidence file
|
|
|
|
|
|
exists in `TestEvidence/` for an untested T1 system.
|
|
|
|
|
|
|
|
|
|
|
|
## Roles
|
|
|
|
|
|
|
|
|
|
|
|
| Role | Holder | Responsibilities |
|
|
|
|
|
|
| --- | --- | --- |
|
|
|
|
|
|
| Policy owner | Bernd | Approve targets, run drills, update registers |
|
|
|
|
|
|
| Alerting owner | Bernd | Respond to backup failures (M-1–M-3) until on-call defined |
|
|
|
|
|
|
| Implementation | Owner repos | Backup automation, CNPG, cron, platform tools |
|
|
|
|
|
|
|
|
|
|
|
|
## Control plane independence (R10)
|
|
|
|
|
|
|
|
|
|
|
|
`disaster-control` must survive the disasters it describes:
|
|
|
|
|
|
|
2026-07-07 20:28:58 +02:00
|
|
|
|
1. At least one git mirror off primary Gitea — `forgejo.coulomb.social/coulomb/disaster-control` (see `docs/mirror-sync.md`).
|
2026-07-07 20:04:54 +02:00
|
|
|
|
2. Registers, policies, and evidence readable without production infrastructure.
|
|
|
|
|
|
3. Panic sheet (`PanicSheet.md`) executable by a competent stranger.
|
|
|
|
|
|
|
|
|
|
|
|
## Incident response
|
|
|
|
|
|
|
|
|
|
|
|
Five playbooks under `IncidentPlaybooks/`:
|
|
|
|
|
|
|
|
|
|
|
|
1. Ransomware / hostile access
|
|
|
|
|
|
2. Credential leak
|
|
|
|
|
|
3. Accidental delete / data corruption
|
|
|
|
|
|
4. Provider outage
|
|
|
|
|
|
5. Loss of admin identity
|
|
|
|
|
|
|
|
|
|
|
|
Each playbook: numbered steps, one page, links to runbooks and registers.
|
|
|
|
|
|
|
|
|
|
|
|
## Related artifacts
|
|
|
|
|
|
|
|
|
|
|
|
| Artifact | Purpose |
|
|
|
|
|
|
| --- | --- |
|
|
|
|
|
|
| `BackupPolicy.md` | Backup architecture, encryption, monitoring |
|
|
|
|
|
|
| `RTO-RPO-Matrix.md` | Per-system RTO/RPO targets and measured drills |
|
|
|
|
|
|
| `BackupPickupQueue.md` | Cross-repo implementation tracker |
|
|
|
|
|
|
| `PanicSheet.md` | One-page emergency reference |
|
|
|
|
|
|
| `RestoreRunbooks/` | Step-by-step recovery procedures |
|
|
|
|
|
|
| `TestEvidence/` | Dated drill reports |
|
