Finish RAILIANCE-WP-0016 apps-pg resource evidence
Publish capacity, recovery, labor, and allocation-driver evidence for resource:railiance:apps-pg so resource-control can forecast and allocate without reading application data or inventing booked cost.
This commit is contained in:
codex
parent
b5fb44b53e
commit
6033ae1aef
6 changed files with 566 additions and 6 deletions
5
Makefile
5
Makefile
|
|
@ -386,6 +386,9 @@ forgejo-package-prune: ## Prune Forgejo packages — keep newest 3 versions per
|
||||||
reef-deployables: ## Capture live reef-railiance deployables for the family validator
|
reef-deployables: ## Capture live reef-railiance deployables for the family validator
|
||||||
python3 scripts/capture-reef-deployables.py -o docs/evidence/reef-railiance-deployables.json
|
python3 scripts/capture-reef-deployables.py -o docs/evidence/reef-railiance-deployables.json
|
||||||
|
|
||||||
|
apps-pg-evidence: ## Capture non-secret apps-pg capacity evidence for resource-control
|
||||||
|
python3 scripts/capture-apps-pg-evidence.py -o docs/evidence/RAILIANCE-WP-0016-apps-pg-observation.json
|
||||||
|
|
||||||
##@ Help
|
##@ Help
|
||||||
|
|
||||||
help: ## Show this help
|
help: ## Show this help
|
||||||
|
|
@ -393,4 +396,4 @@ help: ## Show this help
|
||||||
/^[a-zA-Z_-]+:.*?##/ { printf " \033[36m%-22s\033[0m %s\n", $$1, $$2 } \
|
/^[a-zA-Z_-]+:.*?##/ { printf " \033[36m%-22s\033[0m %s\n", $$1, $$2 } \
|
||||||
/^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) }' $(MAKEFILE_LIST)
|
/^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) }' $(MAKEFILE_LIST)
|
||||||
|
|
||||||
.PHONY: db-deploy db-status db-shell db-logs forgejo-db-deploy forgejo-db-status forgejo-db-shell apps-pg-deploy apps-pg-status apps-pg-shell apps-pg-logs net-kingdom-pg-inter-hub-networkpolicy-deploy pg-deploy pg-status pg-pgpool-check valkey-deploy valkey-status openbao-repo openbao-dry-run openbao-overlay-apply openbao-verify-login-overlay openbao-deploy openbao-status openbao-verify openbao-verify-post-unseal openbao-configure-initial openbao-configure-ssh openbao-verify-ssh openbao-verify-authenticated openbao-configure-external-secrets-issue-core openbao-configure-external-secrets-activity-core openbao-configure-external-secrets-forgejo openbao-validate-restore-evidence openbao-validate-emergency-evidence credential-grants-validate credential-change-applier-dry-run credential-change-applier-apply-plan credential-change-applier-apply credential-change-runbook credential-change-record-evidence credential-change-lifecycle-plan credential-change-lifecycle-event credential-change-import-inventory openbao-credential-change-appliers-dry-run openbao-configure-credential-change-appliers openbao-token-grants-dry-run openbao-configure-token-grants openbao-verify-token-grants-dry-run openbao-verify-token-grants openbao-verify-token-grants-smoke credential-helper-dry-run credential-tests credential-exec-ops-warden-smoke argocd-bootstrap-dry-run argocd-bootstrap-deploy argocd-repo-apply argocd-status backup forgejo-backup forgejo-backup-dry-run forgejo-backup-status forgejo-package-prune forgejo-package-prune-dry-run reef-deployables help
|
.PHONY: db-deploy db-status db-shell db-logs forgejo-db-deploy forgejo-db-status forgejo-db-shell apps-pg-deploy apps-pg-status apps-pg-shell apps-pg-logs net-kingdom-pg-inter-hub-networkpolicy-deploy pg-deploy pg-status pg-pgpool-check valkey-deploy valkey-status openbao-repo openbao-dry-run openbao-overlay-apply openbao-verify-login-overlay openbao-deploy openbao-status openbao-verify openbao-verify-post-unseal openbao-configure-initial openbao-configure-ssh openbao-verify-ssh openbao-verify-authenticated openbao-configure-external-secrets-issue-core openbao-configure-external-secrets-activity-core openbao-configure-external-secrets-forgejo openbao-validate-restore-evidence openbao-validate-emergency-evidence credential-grants-validate credential-change-applier-dry-run credential-change-applier-apply-plan credential-change-applier-apply credential-change-runbook credential-change-record-evidence credential-change-lifecycle-plan credential-change-lifecycle-event credential-change-import-inventory openbao-credential-change-appliers-dry-run openbao-configure-credential-change-appliers openbao-token-grants-dry-run openbao-configure-token-grants openbao-verify-token-grants-dry-run openbao-verify-token-grants openbao-verify-token-grants-smoke credential-helper-dry-run credential-tests credential-exec-ops-warden-smoke argocd-bootstrap-dry-run argocd-bootstrap-deploy argocd-repo-apply argocd-status backup forgejo-backup forgejo-backup-dry-run forgejo-backup-status forgejo-package-prune forgejo-package-prune-dry-run reef-deployables apps-pg-evidence help
|
||||||
|
|
|
||||||
|
|
@ -105,3 +105,10 @@ are follow-up platform work items.
|
||||||
Until backup coverage is explicitly added, consumer onboarding should
|
Until backup coverage is explicitly added, consumer onboarding should
|
||||||
record whether app data is disposable, externally reproducible, or
|
record whether app data is disposable, externally reproducible, or
|
||||||
requires an immediate backup follow-up before production use.
|
requires an immediate backup follow-up before production use.
|
||||||
|
|
||||||
|
Resource evidence for `resource:railiance:apps-pg` (capacity, recovery,
|
||||||
|
labor, allocation drivers) is published under
|
||||||
|
`docs/evidence/RAILIANCE-WP-0016-apps-pg-resource-evidence.md`. The
|
||||||
|
2026-08-14 observation: no ScheduledBackup, railiance01 `apps-pg` is
|
||||||
|
not an Option A target, RPO is unbounded, RTO is undefined. That is
|
||||||
|
unresolved local-storage risk, not verified recovery.
|
||||||
|
|
|
||||||
139
docs/evidence/RAILIANCE-WP-0016-apps-pg-observation.json
Normal file
139
docs/evidence/RAILIANCE-WP-0016-apps-pg-observation.json
Normal file
|
|
@ -0,0 +1,139 @@
|
||||||
|
{
|
||||||
|
"schema_version": "0.1",
|
||||||
|
"record_type": "usage_observation",
|
||||||
|
"resource_id": "resource:railiance:apps-pg",
|
||||||
|
"source": "railiance-platform",
|
||||||
|
"workplan_id": "RAILIANCE-WP-0016",
|
||||||
|
"reef": "reef-railiance",
|
||||||
|
"captured_at": "2026-08-14T00:04:19Z",
|
||||||
|
"identity": {
|
||||||
|
"cluster": "apps-pg",
|
||||||
|
"namespace": "databases",
|
||||||
|
"pod": "apps-pg-1",
|
||||||
|
"image": "ghcr.io/cloudnative-pg/postgresql:16",
|
||||||
|
"postgres_version": "PostgreSQL 16.13 (Debian 16.13-1.pgdg11+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit",
|
||||||
|
"created_at": "2026-08-09T00:15:53Z",
|
||||||
|
"phase": "Cluster in healthy state"
|
||||||
|
},
|
||||||
|
"capacity": {
|
||||||
|
"instances_provisioned": 1,
|
||||||
|
"instances_ready": 1,
|
||||||
|
"cpu_request": null,
|
||||||
|
"memory_request": null,
|
||||||
|
"cpu_limit": null,
|
||||||
|
"memory_limit": null,
|
||||||
|
"storage_provisioned": "10Gi",
|
||||||
|
"storage_class": "local-path",
|
||||||
|
"max_connections": 100
|
||||||
|
},
|
||||||
|
"utilization": {
|
||||||
|
"cpu_observed": "9m",
|
||||||
|
"memory_observed": "102Mi",
|
||||||
|
"pgdata_bytes": 619083120,
|
||||||
|
"wal_bytes_since_init": 587202560,
|
||||||
|
"sessions": 7,
|
||||||
|
"databases": [
|
||||||
|
{
|
||||||
|
"name": "apps_meta",
|
||||||
|
"bytes": 7830551,
|
||||||
|
"backends": 0,
|
||||||
|
"xact_commit": 14374,
|
||||||
|
"xact_rollback": 0,
|
||||||
|
"blks_read": 126,
|
||||||
|
"blks_hit": 525989,
|
||||||
|
"tup_inserted": 0,
|
||||||
|
"tup_updated": 0,
|
||||||
|
"tup_deleted": 0
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "coulomb_social_db",
|
||||||
|
"bytes": 8952855,
|
||||||
|
"backends": 0,
|
||||||
|
"xact_commit": 14754,
|
||||||
|
"xact_rollback": 0,
|
||||||
|
"blks_read": 41,
|
||||||
|
"blks_hit": 574766,
|
||||||
|
"tup_inserted": 982,
|
||||||
|
"tup_updated": 77,
|
||||||
|
"tup_deleted": 11
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "postgres",
|
||||||
|
"bytes": 7830551,
|
||||||
|
"backends": 1,
|
||||||
|
"xact_commit": 146383,
|
||||||
|
"xact_rollback": 0,
|
||||||
|
"blks_read": 356,
|
||||||
|
"blks_hit": 3445068,
|
||||||
|
"tup_inserted": 0,
|
||||||
|
"tup_updated": 4,
|
||||||
|
"tup_deleted": 0
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "vergabe_db",
|
||||||
|
"bytes": 7668239,
|
||||||
|
"backends": 0,
|
||||||
|
"xact_commit": 0,
|
||||||
|
"xact_rollback": 0,
|
||||||
|
"blks_read": 0,
|
||||||
|
"blks_hit": 0,
|
||||||
|
"tup_inserted": 0,
|
||||||
|
"tup_updated": 0,
|
||||||
|
"tup_deleted": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"consumers": [
|
||||||
|
{
|
||||||
|
"database": "coulomb_social_db",
|
||||||
|
"workload": "coulomb-social",
|
||||||
|
"role": "coulomb_social",
|
||||||
|
"cost_attribution_key": "platform:coulomb-social",
|
||||||
|
"bytes": 8952855,
|
||||||
|
"xact_commit": 14754,
|
||||||
|
"backends": 0,
|
||||||
|
"share_of_consumer_bytes": 0.5386
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"database": "vergabe_db",
|
||||||
|
"workload": "vergabe-teilnahme",
|
||||||
|
"role": "vergabe",
|
||||||
|
"cost_attribution_key": "platform:vergabe-teilnahme",
|
||||||
|
"bytes": 7668239,
|
||||||
|
"xact_commit": 0,
|
||||||
|
"backends": 0,
|
||||||
|
"share_of_consumer_bytes": 0.4614
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"recovery": {
|
||||||
|
"cluster_ready": true,
|
||||||
|
"continuous_archiving": true,
|
||||||
|
"scheduled_backup_present": false,
|
||||||
|
"option_a_target": false,
|
||||||
|
"restore_drill_recorded": false,
|
||||||
|
"rpo": "unbounded",
|
||||||
|
"rto": "undefined",
|
||||||
|
"failure_domain": "single-host local-path on railiance01"
|
||||||
|
},
|
||||||
|
"allocation": {
|
||||||
|
"method": "proportional",
|
||||||
|
"driver": "database_gb",
|
||||||
|
"method_version": "apps-pg-dbbytes-v1",
|
||||||
|
"consumer_bytes": 16621094,
|
||||||
|
"residual_bytes": 586800924,
|
||||||
|
"residual_key": "platform:apps-pg-overhead",
|
||||||
|
"notes": [
|
||||||
|
"Consumer share is pg_database_size of declared consumer databases only.",
|
||||||
|
"Residual is PGDATA minus all non-template databases (WAL and catalogs)."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"provenance": {
|
||||||
|
"commands": [
|
||||||
|
"kubectl get cluster apps-pg -n databases -o json",
|
||||||
|
"kubectl top pod -n databases -l cnpg.io/cluster=apps-pg",
|
||||||
|
"kubectl exec -n databases apps-pg-1 -c postgres -- du -sb /var/lib/postgresql/data/pgdata",
|
||||||
|
"psql non-secret catalog and pg_stat_database queries"
|
||||||
|
],
|
||||||
|
"authority": "reef-railiance Kubernetes API + local postgres catalog"
|
||||||
|
}
|
||||||
|
}
|
||||||
118
docs/evidence/RAILIANCE-WP-0016-apps-pg-resource-evidence.md
Normal file
118
docs/evidence/RAILIANCE-WP-0016-apps-pg-resource-evidence.md
Normal file
|
|
@ -0,0 +1,118 @@
|
||||||
|
# apps-pg resource evidence
|
||||||
|
|
||||||
|
Work record: `RAILIANCE-WP-0016`
|
||||||
|
Resource: `resource:railiance:apps-pg`
|
||||||
|
Captured: `2026-08-14T00:04:19Z` (refresh:
|
||||||
|
`scripts/capture-apps-pg-evidence.py -o docs/evidence/RAILIANCE-WP-0016-apps-pg-observation.json`)
|
||||||
|
Machine-readable companion: `docs/evidence/RAILIANCE-WP-0016-apps-pg-observation.json`
|
||||||
|
|
||||||
|
This is the platform-owned evidence `RESOURCE-WP-0003-T04` asked for.
|
||||||
|
It does not contain credentials, DSNs, or application rows. It is not a
|
||||||
|
fin-hub ledger: no EUR amounts are invented here. resource-control can
|
||||||
|
forecast and allocate from these technical facts; booked spend stays in
|
||||||
|
fin-hub.
|
||||||
|
|
||||||
|
## T01 — Capacity and utilization
|
||||||
|
|
||||||
|
| Fact | Value | Kind | Source |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| Instances | 1 provisioned, 1 ready | declared + observed | Cluster `apps-pg` |
|
||||||
|
| Image | `ghcr.io/cloudnative-pg/postgresql:16` (live 16.13) | declared | Cluster spec / `version()` |
|
||||||
|
| Created | 2026-08-09T00:15:53Z | observed | Cluster metadata |
|
||||||
|
| Phase | Cluster in healthy state | observed | Cluster status |
|
||||||
|
| CPU / memory requests | unset (BestEffort) | declared | empty `spec.resources` |
|
||||||
|
| CPU / memory observed | 9m / 102Mi | observed | `kubectl top` |
|
||||||
|
| Storage provisioned | 10Gi, `local-path`, RWO | declared | PVC `apps-pg-1` |
|
||||||
|
| PGDATA used | 619 083 120 B (~590.4 MiB) | observed | `du -sb …/pgdata` |
|
||||||
|
| WAL since init | 587 202 560 B (~560 MiB) | observed | `pg_wal_lsn_diff` |
|
||||||
|
| max_connections | 100 | declared | `SHOW max_connections` |
|
||||||
|
| Sessions at capture | 7 (1 `postgres` + 6 background) | observed | `pg_stat_activity` |
|
||||||
|
|
||||||
|
Consumer catalogs, five days after commission:
|
||||||
|
|
||||||
|
| Database | Workload | Bytes | xact_commit | backends |
|
||||||
|
| --- | --- | --- | --- | --- |
|
||||||
|
| `coulomb_social_db` | coulomb-social | 8 952 855 | 14 754 | 0 |
|
||||||
|
| `vergabe_db` | vergabe-teilnahme | 7 668 239 | 0 | 0 |
|
||||||
|
| `apps_meta` | platform bootstrap | 7 830 551 | 14 374 | 0 |
|
||||||
|
| `postgres` | engine catalog | 7 830 551 | 146 383 | 1 |
|
||||||
|
|
||||||
|
`vergabe_db` has never been used. `coulomb_social_db` is the only consumer
|
||||||
|
with writes (982 inserts). Cache hit on that database is >99.9%. There is
|
||||||
|
no I/O history older than cluster creation and no metrics retention;
|
||||||
|
saturation proxies are the snapshot above plus WAL growth.
|
||||||
|
|
||||||
|
Headroom: ~9.4 GiB of the 10Gi PVC is free. The instance has no CPU or
|
||||||
|
memory request, so it does not reserve cluster capacity. Observed use is
|
||||||
|
idle-class.
|
||||||
|
|
||||||
|
## T02 — Recovery and operations
|
||||||
|
|
||||||
|
| Check | Result |
|
||||||
|
| --- | --- |
|
||||||
|
| Cluster Ready | true since 2026-08-09T00:16:16Z |
|
||||||
|
| ContinuousArchiving | true — CNPG `wal-archive` to the instance manager, **on-node** |
|
||||||
|
| CNPG `Backup` / `ScheduledBackup` | none |
|
||||||
|
| Option A logical dump | **not a target**. Default list covers CoulombCore `apps-pg` and railiance01 `forgejo-db` / `net-kingdom-pg` / `state-hub-db` / `user-engine-pg` only |
|
||||||
|
| Restore drill for this cluster | none recorded |
|
||||||
|
| Replica / pooler | not configured (single instance) |
|
||||||
|
| Failure domain | `railiance01` + `local-path`. Host loss is data loss |
|
||||||
|
|
||||||
|
Recovery objectives: **RPO unbounded, RTO undefined**. This is unresolved
|
||||||
|
local-storage risk, not verified recovery. `docs/apps-pg.md` already said
|
||||||
|
backup was follow-up work; this observation confirms it has not landed for
|
||||||
|
the railiance01 cluster.
|
||||||
|
|
||||||
|
Maintenance: image tag `postgresql:16` is a major pin, not a patch pin.
|
||||||
|
The live minor is 16.13. No upgrade or incident record exists for this
|
||||||
|
cluster. Cadence is unscheduled.
|
||||||
|
|
||||||
|
Labor (high uncertainty, not a timesheet):
|
||||||
|
|
||||||
|
| Class | Estimate | Basis |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| Setup | 6 operator-hours | RAILIANCE-WP-0003 cluster + two consumer onboardings + docs, 2026-08-09 |
|
||||||
|
| Recurring | 0.5 operator-hours / month | ad-hoc `make apps-pg-status`; no backup, restore, or upgrade job |
|
||||||
|
| Incidents | 0 recorded | none in workplans or Cluster conditions |
|
||||||
|
|
||||||
|
resource-control may convert hours at its own labor rate. Do not treat
|
||||||
|
these hours as booked cost.
|
||||||
|
|
||||||
|
## T03 — Allocation drivers
|
||||||
|
|
||||||
|
`apps-pg` is a shared platform service. The recommended first method is
|
||||||
|
the one resource-control already sketched, with the residual made
|
||||||
|
explicit.
|
||||||
|
|
||||||
|
**Method `apps-pg-dbbytes-v1`**
|
||||||
|
|
||||||
|
- Driver: `database_gb` = `pg_database_size` of each **declared consumer**
|
||||||
|
database.
|
||||||
|
- Denominator: sum of those consumer sizes (16 621 094 B at capture).
|
||||||
|
- Shares at capture: coulomb-social **0.5386**, vergabe-teilnahme **0.4614**.
|
||||||
|
- Residual: PGDATA minus all non-template databases = 586 800 924 B
|
||||||
|
(**94.8%** of used volume). Attribute to
|
||||||
|
`platform:apps-pg-overhead`. This is WAL plus engine catalogs, not
|
||||||
|
missing consumer data.
|
||||||
|
- Uncertainty: **high**. Five days of life, empty `vergabe_db`, no history.
|
||||||
|
|
||||||
|
Candidates considered and not selected as v1:
|
||||||
|
|
||||||
|
| Driver | Why not first |
|
||||||
|
| --- | --- |
|
||||||
|
| Connections | All consumers idle at capture; peak is our own `psql` |
|
||||||
|
| `xact_commit` / tuples | Would assign ~100% to coulomb-social and 0% to a reserved empty database |
|
||||||
|
| Equal instance share | Fair while data is tiny, but it hides growth; keep as fallback `apps-pg-instance-v1` if residual stays above 50% after the first month |
|
||||||
|
|
||||||
|
Shared overhead stays unattributed to a consumer on purpose. Converting
|
||||||
|
the residual into booked EUR is fin-hub / resource-control work. Adding a
|
||||||
|
third consumer is a new row in `consumers` plus a Database CR; the method
|
||||||
|
does not change.
|
||||||
|
|
||||||
|
## What this is not
|
||||||
|
|
||||||
|
- Not a cost figure. The host bill belongs to `resource:hosteurope:railiance01`.
|
||||||
|
- Not a backup program. Covering railiance01 `apps-pg` is a later platform
|
||||||
|
change, gated on the same object-store decision as `platform-pg`.
|
||||||
|
- Not application telemetry. No table counts, no tenant identifiers, no
|
||||||
|
Secret values.
|
||||||
276
scripts/capture-apps-pg-evidence.py
Executable file
276
scripts/capture-apps-pg-evidence.py
Executable file
|
|
@ -0,0 +1,276 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Capture non-secret apps-pg capacity evidence for resource-control.
|
||||||
|
|
||||||
|
Used by RAILIANCE-WP-0016. Never prints credentials or role passwords.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import shlex
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
DEFAULT_REMOTE = "railiance01"
|
||||||
|
NAMESPACE = "databases"
|
||||||
|
CLUSTER = "apps-pg"
|
||||||
|
POD = "apps-pg-1"
|
||||||
|
RESOURCE_ID = "resource:railiance:apps-pg"
|
||||||
|
|
||||||
|
CONSUMERS = {
|
||||||
|
"vergabe_db": {
|
||||||
|
"name": "vergabe-teilnahme",
|
||||||
|
"role": "vergabe",
|
||||||
|
"cost_attribution_key": "platform:vergabe-teilnahme",
|
||||||
|
},
|
||||||
|
"coulomb_social_db": {
|
||||||
|
"name": "coulomb-social",
|
||||||
|
"role": "coulomb_social",
|
||||||
|
"cost_attribution_key": "platform:coulomb-social",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def parse_args() -> argparse.Namespace:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__.split("\n\n", 1)[0])
|
||||||
|
parser.add_argument("--remote", default=DEFAULT_REMOTE)
|
||||||
|
parser.add_argument("-o", "--output")
|
||||||
|
return parser.parse_args()
|
||||||
|
|
||||||
|
|
||||||
|
def ssh_run(remote: str, remote_cmd: str) -> str:
|
||||||
|
cmd = ["ssh", "-o", "BatchMode=yes", remote, remote_cmd]
|
||||||
|
result = subprocess.run(cmd, check=False, capture_output=True, text=True)
|
||||||
|
if result.returncode != 0:
|
||||||
|
raise RuntimeError(f"{remote_cmd} failed: {(result.stderr or '').strip()}")
|
||||||
|
return result.stdout
|
||||||
|
|
||||||
|
|
||||||
|
def ssh_kubectl(remote: str, *args: str) -> str:
|
||||||
|
return ssh_run(remote, "kubectl " + " ".join(shlex.quote(arg) for arg in args))
|
||||||
|
|
||||||
|
|
||||||
|
def ssh_psql(remote: str, sql: str) -> str:
|
||||||
|
remote_cmd = (
|
||||||
|
"kubectl exec -n {ns} {pod} -c postgres -- "
|
||||||
|
"psql -U postgres -d postgres -At -F , -c {sql}"
|
||||||
|
).format(ns=shlex.quote(NAMESPACE), pod=shlex.quote(POD), sql=shlex.quote(sql))
|
||||||
|
return ssh_run(remote, remote_cmd)
|
||||||
|
|
||||||
|
|
||||||
|
def as_int(value: str) -> int:
|
||||||
|
return int(value.strip())
|
||||||
|
|
||||||
|
|
||||||
|
def collect(remote: str) -> dict[str, Any]:
|
||||||
|
cluster = json.loads(
|
||||||
|
ssh_kubectl(remote, "get", "cluster", CLUSTER, "-n", NAMESPACE, "-o", "json")
|
||||||
|
)
|
||||||
|
pvc = json.loads(
|
||||||
|
ssh_kubectl(
|
||||||
|
remote,
|
||||||
|
"get",
|
||||||
|
"pvc",
|
||||||
|
"-n",
|
||||||
|
NAMESPACE,
|
||||||
|
"-l",
|
||||||
|
f"cnpg.io/cluster={CLUSTER}",
|
||||||
|
"-o",
|
||||||
|
"json",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
top = ssh_kubectl(
|
||||||
|
remote, "top", "pod", "-n", NAMESPACE, "-l", f"cnpg.io/cluster={CLUSTER}", "--no-headers"
|
||||||
|
).strip()
|
||||||
|
cpu_obs, mem_obs = None, None
|
||||||
|
if top:
|
||||||
|
parts = top.split()
|
||||||
|
if len(parts) >= 3:
|
||||||
|
cpu_obs, mem_obs = parts[1], parts[2]
|
||||||
|
|
||||||
|
pgdata = ssh_kubectl(
|
||||||
|
remote,
|
||||||
|
"exec",
|
||||||
|
"-n",
|
||||||
|
NAMESPACE,
|
||||||
|
POD,
|
||||||
|
"-c",
|
||||||
|
"postgres",
|
||||||
|
"--",
|
||||||
|
"du",
|
||||||
|
"-sb",
|
||||||
|
"/var/lib/postgresql/data/pgdata",
|
||||||
|
).split()[0]
|
||||||
|
|
||||||
|
db_rows = []
|
||||||
|
for line in ssh_psql(
|
||||||
|
remote,
|
||||||
|
"SELECT datname, pg_database_size(datname) FROM pg_database "
|
||||||
|
"WHERE datistemplate = false ORDER BY 1",
|
||||||
|
).splitlines():
|
||||||
|
if not line.strip():
|
||||||
|
continue
|
||||||
|
name, size = line.split(",", 1)
|
||||||
|
db_rows.append({"name": name, "bytes": as_int(size)})
|
||||||
|
|
||||||
|
stats = {}
|
||||||
|
for line in ssh_psql(
|
||||||
|
remote,
|
||||||
|
"SELECT datname, numbackends, xact_commit, xact_rollback, blks_read, blks_hit, "
|
||||||
|
"tup_inserted, tup_updated, tup_deleted FROM pg_stat_database "
|
||||||
|
"WHERE datname IS NOT NULL AND datname NOT LIKE 'template%' ORDER BY 1",
|
||||||
|
).splitlines():
|
||||||
|
if not line.strip():
|
||||||
|
continue
|
||||||
|
cols = line.split(",")
|
||||||
|
stats[cols[0]] = {
|
||||||
|
"backends": as_int(cols[1]),
|
||||||
|
"xact_commit": as_int(cols[2]),
|
||||||
|
"xact_rollback": as_int(cols[3]),
|
||||||
|
"blks_read": as_int(cols[4]),
|
||||||
|
"blks_hit": as_int(cols[5]),
|
||||||
|
"tup_inserted": as_int(cols[6]),
|
||||||
|
"tup_updated": as_int(cols[7]),
|
||||||
|
"tup_deleted": as_int(cols[8]),
|
||||||
|
}
|
||||||
|
|
||||||
|
sessions = as_int(ssh_psql(remote, "SELECT count(*) FROM pg_stat_activity"))
|
||||||
|
max_conn = as_int(ssh_psql(remote, "SELECT current_setting('max_connections')"))
|
||||||
|
wal = ssh_psql(
|
||||||
|
remote, "SELECT pg_wal_lsn_diff(pg_current_wal_lsn(), '0/0')"
|
||||||
|
).strip()
|
||||||
|
version = ssh_psql(remote, "SELECT version()").strip()
|
||||||
|
|
||||||
|
backups = ssh_kubectl(
|
||||||
|
remote, "get", "backup,scheduledbackup", "-n", NAMESPACE, "--ignore-not-found"
|
||||||
|
).strip()
|
||||||
|
|
||||||
|
spec = cluster.get("spec") or {}
|
||||||
|
status = cluster.get("status") or {}
|
||||||
|
resources = spec.get("resources") or {}
|
||||||
|
pvc_item = (pvc.get("items") or [{}])[0]
|
||||||
|
provisioned = ((pvc_item.get("status") or {}).get("capacity") or {}).get("storage")
|
||||||
|
storage_class = (pvc_item.get("spec") or {}).get("storageClassName")
|
||||||
|
|
||||||
|
conditions = {c.get("type"): c for c in status.get("conditions") or []}
|
||||||
|
consumers = []
|
||||||
|
consumer_bytes = 0
|
||||||
|
for db in db_rows:
|
||||||
|
meta = CONSUMERS.get(db["name"])
|
||||||
|
if not meta:
|
||||||
|
continue
|
||||||
|
stat = stats.get(db["name"], {})
|
||||||
|
consumer_bytes += db["bytes"]
|
||||||
|
consumers.append(
|
||||||
|
{
|
||||||
|
"database": db["name"],
|
||||||
|
"workload": meta["name"],
|
||||||
|
"role": meta["role"],
|
||||||
|
"cost_attribution_key": meta["cost_attribution_key"],
|
||||||
|
"bytes": db["bytes"],
|
||||||
|
"xact_commit": stat.get("xact_commit", 0),
|
||||||
|
"backends": stat.get("backends", 0),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
pgdata_bytes = as_int(pgdata)
|
||||||
|
residual_bytes = max(pgdata_bytes - sum(db["bytes"] for db in db_rows), 0)
|
||||||
|
for row in consumers:
|
||||||
|
row["share_of_consumer_bytes"] = (
|
||||||
|
round(row["bytes"] / consumer_bytes, 4) if consumer_bytes else None
|
||||||
|
)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"schema_version": "0.1",
|
||||||
|
"record_type": "usage_observation",
|
||||||
|
"resource_id": RESOURCE_ID,
|
||||||
|
"source": "railiance-platform",
|
||||||
|
"workplan_id": "RAILIANCE-WP-0016",
|
||||||
|
"reef": "reef-railiance",
|
||||||
|
"captured_at": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
|
||||||
|
"identity": {
|
||||||
|
"cluster": CLUSTER,
|
||||||
|
"namespace": NAMESPACE,
|
||||||
|
"pod": status.get("currentPrimary") or POD,
|
||||||
|
"image": status.get("image") or spec.get("imageName"),
|
||||||
|
"postgres_version": version,
|
||||||
|
"created_at": cluster.get("metadata", {}).get("creationTimestamp"),
|
||||||
|
"phase": status.get("phase"),
|
||||||
|
},
|
||||||
|
"capacity": {
|
||||||
|
"instances_provisioned": spec.get("instances"),
|
||||||
|
"instances_ready": status.get("readyInstances"),
|
||||||
|
"cpu_request": (resources.get("requests") or {}).get("cpu"),
|
||||||
|
"memory_request": (resources.get("requests") or {}).get("memory"),
|
||||||
|
"cpu_limit": (resources.get("limits") or {}).get("cpu"),
|
||||||
|
"memory_limit": (resources.get("limits") or {}).get("memory"),
|
||||||
|
"storage_provisioned": provisioned or spec.get("storage", {}).get("size"),
|
||||||
|
"storage_class": storage_class,
|
||||||
|
"max_connections": max_conn,
|
||||||
|
},
|
||||||
|
"utilization": {
|
||||||
|
"cpu_observed": cpu_obs,
|
||||||
|
"memory_observed": mem_obs,
|
||||||
|
"pgdata_bytes": pgdata_bytes,
|
||||||
|
"wal_bytes_since_init": as_int(wal) if wal else None,
|
||||||
|
"sessions": sessions,
|
||||||
|
"databases": [{**db, **stats.get(db["name"], {})} for db in db_rows],
|
||||||
|
},
|
||||||
|
"consumers": consumers,
|
||||||
|
"recovery": {
|
||||||
|
"cluster_ready": (conditions.get("Ready") or {}).get("status") == "True",
|
||||||
|
"continuous_archiving": (conditions.get("ContinuousArchiving") or {}).get(
|
||||||
|
"status"
|
||||||
|
)
|
||||||
|
== "True",
|
||||||
|
"scheduled_backup_present": bool(backups),
|
||||||
|
"option_a_target": False,
|
||||||
|
"restore_drill_recorded": False,
|
||||||
|
"rpo": "unbounded",
|
||||||
|
"rto": "undefined",
|
||||||
|
"failure_domain": "single-host local-path on railiance01",
|
||||||
|
},
|
||||||
|
"allocation": {
|
||||||
|
"method": "proportional",
|
||||||
|
"driver": "database_gb",
|
||||||
|
"method_version": "apps-pg-dbbytes-v1",
|
||||||
|
"consumer_bytes": consumer_bytes,
|
||||||
|
"residual_bytes": residual_bytes,
|
||||||
|
"residual_key": "platform:apps-pg-overhead",
|
||||||
|
"notes": [
|
||||||
|
"Consumer share is pg_database_size of declared consumer databases only.",
|
||||||
|
"Residual is PGDATA minus all non-template databases (WAL and catalogs).",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
"provenance": {
|
||||||
|
"commands": [
|
||||||
|
f"kubectl get cluster {CLUSTER} -n {NAMESPACE} -o json",
|
||||||
|
f"kubectl top pod -n {NAMESPACE} -l cnpg.io/cluster={CLUSTER}",
|
||||||
|
f"kubectl exec -n {NAMESPACE} {POD} -c postgres -- du -sb /var/lib/postgresql/data/pgdata",
|
||||||
|
"psql non-secret catalog and pg_stat_database queries",
|
||||||
|
],
|
||||||
|
"authority": "reef-railiance Kubernetes API + local postgres catalog",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
args = parse_args()
|
||||||
|
try:
|
||||||
|
payload = collect(args.remote)
|
||||||
|
except (RuntimeError, json.JSONDecodeError, ValueError) as exc:
|
||||||
|
print(f"capture failed: {exc}", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
text = json.dumps(payload, indent=2) + "\n"
|
||||||
|
if args.output:
|
||||||
|
with open(args.output, "w", encoding="utf-8") as fh:
|
||||||
|
fh.write(text)
|
||||||
|
else:
|
||||||
|
sys.stdout.write(text)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
|
|
@ -4,11 +4,11 @@ type: workplan
|
||||||
title: "Publish apps-pg resource and recovery evidence"
|
title: "Publish apps-pg resource and recovery evidence"
|
||||||
domain: financials
|
domain: financials
|
||||||
repo: railiance-platform
|
repo: railiance-platform
|
||||||
status: ready
|
status: finished
|
||||||
owner: codex
|
owner: codex
|
||||||
topic_slug: railiance
|
topic_slug: railiance
|
||||||
created: "2026-08-11"
|
created: "2026-08-11"
|
||||||
updated: "2026-08-11"
|
updated: "2026-08-14"
|
||||||
state_hub_workstream_id: "49084fb8-de63-4f32-a4a9-3a42d4e708ac"
|
state_hub_workstream_id: "49084fb8-de63-4f32-a4a9-3a42d4e708ac"
|
||||||
related:
|
related:
|
||||||
- RAILIANCE-WP-0003
|
- RAILIANCE-WP-0003
|
||||||
|
|
@ -27,7 +27,7 @@ utilization, recovery, operations labor, and consumer allocation. Origin:
|
||||||
|
|
||||||
```task
|
```task
|
||||||
id: RAILIANCE-WP-0016-T01
|
id: RAILIANCE-WP-0016-T01
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "c772bcea-7ae0-4532-b434-31f75c905431"
|
state_hub_task_id: "c772bcea-7ae0-4532-b434-31f75c905431"
|
||||||
```
|
```
|
||||||
|
|
@ -43,7 +43,7 @@ application data or credentials.
|
||||||
|
|
||||||
```task
|
```task
|
||||||
id: RAILIANCE-WP-0016-T02
|
id: RAILIANCE-WP-0016-T02
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "4eafebd5-0434-42a2-a126-c292c29c1f21"
|
state_hub_task_id: "4eafebd5-0434-42a2-a126-c292c29c1f21"
|
||||||
```
|
```
|
||||||
|
|
@ -59,7 +59,7 @@ storage risk, and platform labor cost.
|
||||||
|
|
||||||
```task
|
```task
|
||||||
id: RAILIANCE-WP-0016-T03
|
id: RAILIANCE-WP-0016-T03
|
||||||
status: todo
|
status: done
|
||||||
priority: medium
|
priority: medium
|
||||||
state_hub_task_id: "fc329443-a2fc-4409-b670-dfb103df2794"
|
state_hub_task_id: "fc329443-a2fc-4409-b670-dfb103df2794"
|
||||||
```
|
```
|
||||||
|
|
@ -70,3 +70,20 @@ unattributed residual, method version, and uncertainty.
|
||||||
|
|
||||||
Done when resource-control can allocate a referenced apps-pg cost without
|
Done when resource-control can allocate a referenced apps-pg cost without
|
||||||
making railiance-platform a financial ledger.
|
making railiance-platform a financial ledger.
|
||||||
|
|
||||||
|
## Outcome
|
||||||
|
|
||||||
|
Published 2026-08-14. Observation
|
||||||
|
`docs/evidence/RAILIANCE-WP-0016-apps-pg-observation.json` and narrative
|
||||||
|
`docs/evidence/RAILIANCE-WP-0016-apps-pg-resource-evidence.md`.
|
||||||
|
|
||||||
|
- T01: 1/1 instance, 10Gi local-path, 590 MiB PGDATA, no CPU/memory
|
||||||
|
requests, two consumer catalogs (~16.6 MiB), `vergabe_db` unused.
|
||||||
|
- T02: no ScheduledBackup, not an Option A target, RPO unbounded, RTO
|
||||||
|
undefined — unresolved local-storage risk. Labor: ~6h setup, ~0.5h/month
|
||||||
|
recurring, high uncertainty, no EUR.
|
||||||
|
- T03: method `apps-pg-dbbytes-v1`, driver `database_gb`, residual
|
||||||
|
`platform:apps-pg-overhead` (94.8% of used volume is WAL/catalog).
|
||||||
|
|
||||||
|
Refresh: `make apps-pg-evidence`. resource-control can now replace the
|
||||||
|
illustrative `examples/control-cycle/apps-pg-*.json` fixtures.
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue