Start DISCTL-WP-0002: policies, registers, runbooks, playbooks
Expand AssetRegister and DataRegister with T0/T4 stack assets. Add ResiliencePolicy, PanicSheet, full-server-loss runbook, and five incident playbooks. Update README with mirror status. Mark T02-T05, T07-T08 done; T01 and T06 in progress.
This commit is contained in:
parent
4288ad21a9
commit
d011073e16
13 changed files with 814 additions and 29 deletions
23
IncidentPlaybooks/accidental-delete.md
Normal file
23
IncidentPlaybooks/accidental-delete.md
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
# Playbook: Accidental Delete / Data Corruption
|
||||
|
||||
**Triggers:** deleted repo, dropped table, bad migration, destructive script, bad Ansible run.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Stop writes** — scale deployment to zero, enable maintenance mode, or revoke DB write access.
|
||||
2. **Assess scope** — what was deleted/corrupted, when, last known good state.
|
||||
3. **Clone backup to staging** — never restore directly over production first.
|
||||
4. **Identify last good point** — backup timestamp, git ref, DB snapshot.
|
||||
5. **Restore selectively** — one repo, one table, or one PVC — not full blast restore unless required.
|
||||
6. **Diff** — compare staging restore against current production; list deltas.
|
||||
7. **Apply to production** — merge recovered data; re-run migrations if needed.
|
||||
8. **Resume writes** — remove maintenance mode; monitor.
|
||||
9. **Document** — `TestEvidence/` note with RPO achieved; update registers if backup gap contributed.
|
||||
|
||||
## Fast rollback
|
||||
|
||||
If local snapshot or recent git push exists, prefer that over full restore.
|
||||
|
||||
## References
|
||||
|
||||
`RTO-RPO-Matrix.md` · `BackupPolicy.md` · `AssetRegister.yaml`
|
||||
18
IncidentPlaybooks/credential-leak.md
Normal file
18
IncidentPlaybooks/credential-leak.md
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
# Playbook: Credential Leak
|
||||
|
||||
**Triggers:** token in Git/logs/chat, stolen laptop, exposed `.env`, compromised CI secret.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Identify scope** — which credential, which systems, exposure window.
|
||||
2. **Revoke immediately** — invalidate leaked token/password before cleanup discussion.
|
||||
3. **Rotate downstream** — every system that trusted the credential (OpenBao, Stripe, Gitea/Forgejo, backup drop, OpenRouter, webhooks).
|
||||
4. **Scan for misuse** — auth logs, Stripe dashboard, API usage anomalies, unexpected git pushes.
|
||||
5. **Remove secret from exposure** — git history scrub if committed; rotate not revert.
|
||||
6. **Verify separation** — confirm backup credentials still isolated (`BackupPolicy.md` CS-1).
|
||||
7. **Re-deploy** — refresh Kubernetes secrets via External Secrets / OpenBao.
|
||||
8. **Document** — incident note; no secret values in State Hub or evidence files.
|
||||
|
||||
## References
|
||||
|
||||
`AssetRegister.yaml` T0 entries · `PanicSheet.md` §4 · ops-warden credential routing
|
||||
25
IncidentPlaybooks/loss-of-admin-identity.md
Normal file
25
IncidentPlaybooks/loss-of-admin-identity.md
Normal file
|
|
@ -0,0 +1,25 @@
|
|||
# Playbook: Loss of Admin Identity
|
||||
|
||||
**Triggers:** locked out of registrar, Gitea, Stripe, Bubble, email, cloud; lost MFA device.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Identify which identity** — registrar, payment, forge, email, secrets vault.
|
||||
2. **Break-glass account** — use documented secondary admin if available (see `AssetRegister.yaml` recovery_path).
|
||||
3. **Offline recovery codes** — MFA reset via vendor recovery flow; requires pre-stored codes.
|
||||
4. **Vendor support** — IONOS/Stripe identity verification; allow 24–48h; have domain ownership proof ready.
|
||||
5. **Verify ownership** — confirm domain/registrar records unchanged; check for hostile transfer.
|
||||
6. **Rotate compromised identity** — new password, new MFA, revoke active sessions.
|
||||
7. **Restore dependent access** — Gitea/Forgejo admin, DNS, backup admin, OpenBao.
|
||||
8. **Update registers** — record recovery paths discovered; close gaps in `AssetRegister.yaml` T0 section.
|
||||
9. **Document** — evidence log; **no recovery codes in Git**.
|
||||
|
||||
## Prevention (ongoing)
|
||||
|
||||
- Record backup admin contacts per T0 asset.
|
||||
- Store MFA recovery codes in password manager.
|
||||
- Registrar lock enabled.
|
||||
|
||||
## References
|
||||
|
||||
`AssetRegister.yaml` T0 · `PanicSheet.md` §2 · `ResiliencePolicy.md`
|
||||
19
IncidentPlaybooks/provider-outage.md
Normal file
19
IncidentPlaybooks/provider-outage.md
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
# Playbook: Provider Outage
|
||||
|
||||
**Triggers:** Bubble, Stripe, OpenRouter, IONOS DNS/email, or other vendor unavailable.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Confirm outage** — vendor status page, `dig`, `curl`, third-party monitor; rule out local DNS/SSH issue.
|
||||
2. **Communicate** — internal status note; disable affected features in UI if possible.
|
||||
3. **Graceful degradation** — queue writes; serve static fallback; read-only mode.
|
||||
4. **Bubble.io** — document member-facing impact; no local DB to restore; wait for vendor.
|
||||
5. **Stripe** — pause checkout if webhooks failing; queue events; reconcile on return.
|
||||
6. **OpenRouter** — switch to alternate LLM provider if configured; reduce non-critical inference.
|
||||
7. **DNS/email** — if IONOS down, existing TTLs may mask briefly; avoid panic changes until vendor confirms.
|
||||
8. **Reconcile** — after recovery, replay webhooks, verify Stripe/Bubble state, run smoke tests.
|
||||
9. **Document** — outage window, impact, follow-up (export cadence, alternate provider).
|
||||
|
||||
## References
|
||||
|
||||
`AssetRegister.yaml` T4 entries · `ResiliencePolicy.md` · Bubble/Stripe export gaps in `DataRegister.yaml`
|
||||
27
IncidentPlaybooks/ransomware.md
Normal file
27
IncidentPlaybooks/ransomware.md
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
# Playbook: Ransomware / Hostile Access
|
||||
|
||||
**Triggers:** suspicious encryption, unknown admin sessions, altered repos, exfiltration indicators.
|
||||
**Tier:** L2 / I5 — treat as breach until proven otherwise.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Isolate** — disconnect affected hosts from network; disable compromised accounts; preserve VM/disk snapshots for evidence.
|
||||
2. **Preserve evidence** — do not wipe logs; capture auth logs, k8s audit, Gitea/Forgejo access logs, firewall logs.
|
||||
3. **Revoke credentials** — rotate all tokens/passwords that may be exposed: OpenBao, Stripe, API keys, backup tokens, Gitea/Forgejo admin, SSH certs.
|
||||
4. **Assess backup integrity** — verify offsite backups (Nextcloud) were not deleted; check `.last-backup` stamps; assume on-node backups on compromised host are untrusted.
|
||||
5. **Build clean environment** — new machines; fresh OS; Ansible from known-good Git refs. **Never restore into compromised host.**
|
||||
6. **Restore known-good data** — from backups timestamped **before** compromise; age-decrypt from offsite only.
|
||||
7. **Rotate all secrets** — full rotation, not selective; update OpenBao paths; re-issue `warden` certs.
|
||||
8. **Force resets** — user sessions, MFA re-enrollment where applicable.
|
||||
9. **Validate** — smoke tests; compare restored repos against known-good SHAs.
|
||||
10. **Document** — State Hub progress event; `TestEvidence/` incident note; update `AssetRegister.yaml` if exposure changed posture.
|
||||
|
||||
## Do not
|
||||
|
||||
- Pay ransom without executive decision recorded.
|
||||
- Restore production DB over live data without identifying last good point.
|
||||
- Paste secrets into incident notes.
|
||||
|
||||
## References
|
||||
|
||||
`BackupPolicy.md` §8 · `RestoreRunbooks/01-full-server-loss.md` · `PanicSheet.md` §4
|
||||
Loading…
Add table
Add a link
Reference in a new issue