From 35dbe4dd42e1f468630d6f217707f2e192d645c8 Mon Sep 17 00:00:00 2001 From: tegwick Date: Sat, 22 Aug 2026 12:34:25 +0200 Subject: [PATCH] Publish Forge resource economics evidence Assistant: codex Assistant-Model: gpt-5.6-sol Assistant-Session: 01a02669-87ee-7a31-b111-edc95a16e0fa --- INTENT.md | 17 +- Makefile | 20 +- SCOPE.md | 13 +- WORK-RECORDS.md | 8 +- .../allocation-drivers.yaml | 20 ++ .../observations/20260822T102256Z.json | 115 +++++++++ .../observations/latest.json | 115 +++++++++ .../service-objectives-and-labor.yaml | 31 +++ docs/forge-resource-evidence.md | 18 ++ tests/test_forge_resources.py | 33 +++ tools/observe_forge_resources.py | 232 ++++++++++++++++++ ...-0002-forge-resource-economics-evidence.md | 40 ++- 12 files changed, 632 insertions(+), 30 deletions(-) create mode 100644 docs/evidence/resource-railiance-forgejo/allocation-drivers.yaml create mode 100644 docs/evidence/resource-railiance-forgejo/observations/20260822T102256Z.json create mode 100644 docs/evidence/resource-railiance-forgejo/observations/latest.json create mode 100644 docs/evidence/resource-railiance-forgejo/service-objectives-and-labor.yaml create mode 100644 docs/forge-resource-evidence.md create mode 100644 tests/test_forge_resources.py create mode 100755 tools/observe_forge_resources.py diff --git a/INTENT.md b/INTENT.md index 245c2dc..d8f251b 100644 --- a/INTENT.md +++ b/INTENT.md @@ -122,10 +122,12 @@ description left it unplaced - which is how layer boundaries erode. Placement was decided 2026-08-11 (hub decision `d151d817`): * the **Forgejo workload** becomes `rapp-forgejo`, a managed workload package -* the **layer responsibility** - runner placement and labels, registry - retention, artifact lifecycle, package credentials - folds into **S4 - `railiance-enablement`**, which already declares a handoff contract for - exactly these concerns +* the **consumer-facing contract** - reusable workflow expectations for runner + labels, registry interfaces, package credentials, and artifact evidence - + belongs to **S4 `railiance-enablement`** +* this repo remains the **operational provider** for the live forge, registry, + runner substrate, retention, custody references, and operating evidence + until a separately approved migration changes that boundary A retirement hypothesis was tested and rejected: this repo and `helix-forge` are different lineages, not one maturing into the other. `helix-forge` is @@ -133,8 +135,11 @@ methodological (capability-first ecosystem, OAS dimensions, VSM vocabulary); this repo is operational. Retiring it would orphan responsibilities nothing else owns. -Until the migration completes, this repo remains the operating home for forge -concerns. See `railiance-platform/ArchitectureBlueprint.md` section 5.2. +This split does not require a physical merge: enablement owns the paved +consumer path, while this repo owns the capability that path consumes. The +Forgejo workload is expected to become `rapp-forgejo` without moving its +operator contract into enablement. See `railiance-platform/ArchitectureBlueprint.md` +section 5.2. --- diff --git a/Makefile b/Makefile index a3e0f9d..004b566 100644 --- a/Makefile +++ b/Makefile @@ -1,15 +1,15 @@ SHELL := /usr/bin/env bash .DEFAULT_GOAL := help -GITEA_RELEASE ?= gitea -GITEA_NAMESPACE ?= default +GITEA_RELEASE ?= forgejo +GITEA_NAMESPACE ?= forgejo GITEA_CHART ?= gitea-charts/gitea GITEA_VALUES ?= helm/gitea-values.sops.yaml GITEA_REGISTRY_VALUES ?= helm/gitea-registry-values.yaml GITEA_INGRESS ?= manifests/gitea-ingress.yaml -GITEA_HTTP_SERVICE ?= gitea-http -GITEA_SSH_SERVICE ?= gitea-ssh-nodeport -GITEA_DB_CLUSTER ?= gitea-db +GITEA_HTTP_SERVICE ?= forgejo-gitea-http +GITEA_SSH_SERVICE ?= forgejo-ssh-nodeport +GITEA_DB_CLUSTER ?= forgejo-db GITEA_DB_NAMESPACE ?= databases REGISTRY_DOCS ?= docs/gitea-container-registry.md docs/gitea-package-registry.md EVIDENCE_DOCS ?= docs/observability-operating-evidence.md docs/ci-runner-actions-gitops-ownership.md docs/backup-restore-secret-handoff.md @@ -62,6 +62,14 @@ runner-status: ## Read-only Actions runner, host, and inter-hub registry probes forgejo-runner-status: ## Read-only Forgejo Actions runner and endpoint probes bash tools/forgejo-runner-status.sh +forge-observe: ## Capture non-secret forge capacity, recovery, and allocation evidence + python3 tools/observe_forge_resources.py \ + $(if $(REMOTE),--remote $(REMOTE),) \ + --output-dir docs/evidence/resource-railiance-forgejo/observations + +test-unit: ## Run non-live forge evidence tests + python3 -m unittest discover -s tests -p 'test_*.py' -v + check-runner-tools: ## Check local tools used by runner inspection targets @missing=0; \ for tool in curl ssh docker; do \ @@ -112,4 +120,4 @@ help: ## Show this help /^[a-zA-Z0-9_-]+:.*?##/ { printf " \033[36m%-20s\033[0m %s\n", $$1, $$2 } \ /^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) }' $(MAKEFILE_LIST) -.PHONY: check-tools check-sops registry-docs evidence-docs runner-docs runner-status forgejo-runner-status check-runner-tools gitea-deploy gitea-ingress-deploy gitea-status help +.PHONY: check-tools check-sops registry-docs evidence-docs runner-docs runner-status forgejo-runner-status forge-observe test-unit check-runner-tools gitea-deploy gitea-ingress-deploy gitea-status help diff --git a/SCOPE.md b/SCOPE.md index dd2e827..f83551f 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -3,7 +3,7 @@ This file defines what `railiance-forge` owns, when to use it, and where its boundaries stop. -Last reviewed: 2026-06-13 +Last reviewed: 2026-08-22 --- @@ -113,14 +113,15 @@ The Fabric graph declarations for forge capabilities and edges live in ## Current State -- Status: active forge extraction. +- Status: active forge operation with explicit S4 consumer-contract handoff. - Implementation: repository contract, registry docs, initial operating contracts, deploy-capable Gitea files, and operator targets are present. - Stability: emerging but now live-facing; forge owns the reviewed public Gitea HTTPS ingress for the web UI, package registry, and OCI registry. Raw node IP HTTP access is not part of the supported forge surface. -- Usage: canonical reference point for forge and registry responsibilities - currently transitioning out of `railiance-apps`. +- Usage: canonical operational reference for forge, registry, runner, retention, + recovery, and resource evidence. S4 owns reusable consumer templates; this + repo remains their provider and does not duplicate those templates. Known starting point: @@ -128,8 +129,8 @@ Known starting point: ingress, operating contracts, and deploy/status entry points. - `railiance-apps` keeps app release ownership and transitional compatibility wrappers for old Gitea commands. -- `railiance-enablement` owns the intent for delivery templates and developer - paved paths, but not forge runtime operation. +- `railiance-enablement` owns delivery templates and developer paved paths, + but not forge runtime, registry, runner, retention, or recovery operation. - `railiance-forge` should absorb forge runtime and artifact infrastructure without moving S5 app release ownership. diff --git a/WORK-RECORDS.md b/WORK-RECORDS.md index 68b1fd7..81c5952 100644 --- a/WORK-RECORDS.md +++ b/WORK-RECORDS.md @@ -8,7 +8,7 @@ | Kind | ID | Status | Lane | Source | | --- | --- | --- | --- | --- | -| workplan | RFORGE-WP-0002 | ready | — | workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md | -| task | RFORGE-WP-0002-T01 | todo | — | workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md | -| task | RFORGE-WP-0002-T02 | todo | — | workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md | -| task | RFORGE-WP-0002-T03 | todo | — | workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md | +| workplan | RFORGE-WP-0002 | finished | — | workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md | +| task | RFORGE-WP-0002-T01 | done | — | workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md | +| task | RFORGE-WP-0002-T02 | done | — | workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md | +| task | RFORGE-WP-0002-T03 | done | — | workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md | diff --git a/docs/evidence/resource-railiance-forgejo/allocation-drivers.yaml b/docs/evidence/resource-railiance-forgejo/allocation-drivers.yaml new file mode 100644 index 0000000..13bafc0 --- /dev/null +++ b/docs/evidence/resource-railiance-forgejo/allocation-drivers.yaml @@ -0,0 +1,20 @@ +schema_version: railiance.forge-allocation-drivers.v1 +resource_id: resource:railiance:forgejo +method_version: forge-raw-drivers-v1 +candidate_drivers: + - repository_count + - package_bytes + - runner_job_minutes +consumer_groups: + - helix-forge + - coulomb-social + - railiance +selected_driver: null +attributed: [] +residual: + share: 1.0 + reason: current read-only surfaces do not bind repository, package, and runner usage to those consumer groups +selection_owner: resource-control +provenance: + - Kubernetes deployment, PVC, CNPG, runner, and storage observations + - Forgejo aggregate API evidence when an approved read-only route becomes available diff --git a/docs/evidence/resource-railiance-forgejo/observations/20260822T102256Z.json b/docs/evidence/resource-railiance-forgejo/observations/20260822T102256Z.json new file mode 100644 index 0000000..afdbbc9 --- /dev/null +++ b/docs/evidence/resource-railiance-forgejo/observations/20260822T102256Z.json @@ -0,0 +1,115 @@ +{ + "schema_version": "railiance.forge-resource-observation.v1", + "record_type": "usage_observation", + "resource_id": "resource:railiance:forgejo", + "source": "railiance-forge", + "workplan_id": "RFORGE-WP-0002", + "reef": "reef-railiance", + "captured_at": "2026-08-22T10:22:56Z", + "capacity": { + "deployments": [ + { + "name": "forgejo-gitea", + "desired_replicas": 1, + "available_replicas": 1, + "requests": { + "cpu_millicores": 0, + "memory_bytes": 0 + }, + "limits": { + "cpu_millicores": 0, + "memory_bytes": 0 + }, + "observed": { + "cpu_millicores": 115, + "memory_bytes": 629145600 + } + }, + { + "name": "forgejo-runner", + "desired_replicas": 1, + "available_replicas": 1, + "requests": { + "cpu_millicores": 0, + "memory_bytes": 0 + }, + "limits": { + "cpu_millicores": 0, + "memory_bytes": 0 + }, + "observed": { + "cpu_millicores": 3, + "memory_bytes": 563085312 + } + } + ], + "persistent_volumes": [ + { + "name": "forgejo-runner-data", + "storage_class": "local-path", + "requested_bytes": 5368709120, + "phase": "Bound" + }, + { + "name": "gitea-shared-storage", + "storage_class": "local-path", + "requested_bytes": 10737418240, + "phase": "Bound" + } + ], + "pvc_requested_bytes": 16106127360, + "forge_data_used_bytes": 2377375744, + "runner_data_used_bytes": 247693312, + "database": { + "instances": 1, + "ready_instances": 1, + "storage_requested_bytes": 10737418240, + "phase": "Cluster in healthy state" + } + }, + "recovery": { + "database_continuous_archiving": true, + "database_backups_seen": 0, + "database_backups_completed": 0, + "shared_storage_restore_drill": false, + "runner_restore_drill": false + }, + "allocation": { + "method_version": "forge-raw-drivers-v1", + "candidate_drivers": [ + "repository_count", + "package_bytes", + "runner_job_minutes" + ], + "consumer_groups": [ + "helix-forge", + "coulomb-social", + "railiance" + ], + "attributed": [], + "residual": { + "forge_data_used_bytes": 2377375744, + "reason": "consumer attribution source is not yet available" + }, + "selection_owner": "resource-control" + }, + "measurement_gaps": [ + "forgejo-db has no Backup or ScheduledBackup evidence", + "material network traffic is unavailable", + "repository and package usage are not yet attributable to named consumers", + "runner job counts and durations are unavailable from the current read-only surface" + ], + "provenance": { + "commands": [ + "kubectl get deploy -n forgejo -o json", + "kubectl get pvc -n forgejo -o json", + "kubectl top pods -n forgejo --no-headers", + "kubectl get cluster forgejo-db -n databases -o json", + "kubectl get backup,scheduledbackup -n databases -o json", + "kubectl exec deploy/forgejo-gitea -- du -sk /data", + "kubectl exec deploy/forgejo-runner -- du -sk /data" + ], + "repository_contents_read": false, + "secret_surfaces_read": false + } +} diff --git a/docs/evidence/resource-railiance-forgejo/observations/latest.json b/docs/evidence/resource-railiance-forgejo/observations/latest.json new file mode 100644 index 0000000..afdbbc9 --- /dev/null +++ b/docs/evidence/resource-railiance-forgejo/observations/latest.json @@ -0,0 +1,115 @@ +{ + "schema_version": "railiance.forge-resource-observation.v1", + "record_type": "usage_observation", + "resource_id": "resource:railiance:forgejo", + "source": "railiance-forge", + "workplan_id": "RFORGE-WP-0002", + "reef": "reef-railiance", + "captured_at": "2026-08-22T10:22:56Z", + "capacity": { + "deployments": [ + { + "name": "forgejo-gitea", + "desired_replicas": 1, + "available_replicas": 1, + "requests": { + "cpu_millicores": 0, + "memory_bytes": 0 + }, + "limits": { + "cpu_millicores": 0, + "memory_bytes": 0 + }, + "observed": { + "cpu_millicores": 115, + "memory_bytes": 629145600 + } + }, + { + "name": "forgejo-runner", + "desired_replicas": 1, + "available_replicas": 1, + "requests": { + "cpu_millicores": 0, + "memory_bytes": 0 + }, + "limits": { + "cpu_millicores": 0, + "memory_bytes": 0 + }, + "observed": { + "cpu_millicores": 3, + "memory_bytes": 563085312 + } + } + ], + "persistent_volumes": [ + { + "name": "forgejo-runner-data", + "storage_class": "local-path", + "requested_bytes": 5368709120, + "phase": "Bound" + }, + { + "name": "gitea-shared-storage", + "storage_class": "local-path", + "requested_bytes": 10737418240, + "phase": "Bound" + } + ], + "pvc_requested_bytes": 16106127360, + "forge_data_used_bytes": 2377375744, + "runner_data_used_bytes": 247693312, + "database": { + "instances": 1, + "ready_instances": 1, + "storage_requested_bytes": 10737418240, + "phase": "Cluster in healthy state" + } + }, + "recovery": { + "database_continuous_archiving": true, + "database_backups_seen": 0, + "database_backups_completed": 0, + "shared_storage_restore_drill": false, + "runner_restore_drill": false + }, + "allocation": { + "method_version": "forge-raw-drivers-v1", + "candidate_drivers": [ + "repository_count", + "package_bytes", + "runner_job_minutes" + ], + "consumer_groups": [ + "helix-forge", + "coulomb-social", + "railiance" + ], + "attributed": [], + "residual": { + "forge_data_used_bytes": 2377375744, + "reason": "consumer attribution source is not yet available" + }, + "selection_owner": "resource-control" + }, + "measurement_gaps": [ + "forgejo-db has no Backup or ScheduledBackup evidence", + "material network traffic is unavailable", + "repository and package usage are not yet attributable to named consumers", + "runner job counts and durations are unavailable from the current read-only surface" + ], + "provenance": { + "commands": [ + "kubectl get deploy -n forgejo -o json", + "kubectl get pvc -n forgejo -o json", + "kubectl top pods -n forgejo --no-headers", + "kubectl get cluster forgejo-db -n databases -o json", + "kubectl get backup,scheduledbackup -n databases -o json", + "kubectl exec deploy/forgejo-gitea -- du -sk /data", + "kubectl exec deploy/forgejo-runner -- du -sk /data" + ], + "repository_contents_read": false, + "secret_surfaces_read": false + } +} diff --git a/docs/evidence/resource-railiance-forgejo/service-objectives-and-labor.yaml b/docs/evidence/resource-railiance-forgejo/service-objectives-and-labor.yaml new file mode 100644 index 0000000..7e01eed --- /dev/null +++ b/docs/evidence/resource-railiance-forgejo/service-objectives-and-labor.yaml @@ -0,0 +1,31 @@ +schema_version: railiance.forge-service-objectives.v1 +resource_id: resource:railiance:forgejo +workplan_id: RFORGE-WP-0002 +service_objectives: + availability: + objective: web, Git SSH, OCI, package, and runner readiness checks pass at each observation + percentage_target: null + measurement_gap: no approved monthly availability SLO or continuous evidence plane + recovery: + rpo: null + rto: null + requirement: isolated restore must prove clone, login, package pull, and runner re-registration before production trust + retention: + policy: null + requirement: cleanup must preserve referenced release artifacts and evidence + performance: + objective: null + measurement_gap: no approved latency or throughput target +operations_labor: + basis: activities declared; hours remain unknown until time records exist + activities: + - forge and registry health review + - runner health and label review + - storage growth and retention review + - database and shared-storage backup review + - upgrade and migration rehearsal + - incident response and restore drill + setup_hours: null + recurring_hours_per_month: null + measurement_gap: no attributable time records +cost_owner: resource-control/fin-hub diff --git a/docs/forge-resource-evidence.md b/docs/forge-resource-evidence.md new file mode 100644 index 0000000..89c5d3b --- /dev/null +++ b/docs/forge-resource-evidence.md @@ -0,0 +1,18 @@ +# Forge resource evidence + +Run `make forge-observe REMOTE=railiance01` to capture a timestamped, +non-secret observation for `resource:railiance:forgejo`. The collector reads +Kubernetes deployment, metrics, PVC, CNPG, backup-resource, and aggregate +`du` surfaces. It does not read repositories, packages, Secrets, or pod +environments. + +The observation separates requested/limited compute from observed usage, +records storage and recovery posture, and preserves unavailable dimensions as +measurement gaps. The companion YAML files define service/labor assumptions +and candidate allocation drivers. Unknown targets, labor, consumer +attribution, and booked cost stay unknown; `resource-control` owns allocation +method selection and `fin-hub` owns financial reconciliation. + +The current capture also makes the recovery gap explicit: CNPG reports +continuous archiving, but no `Backup` or `ScheduledBackup` resource exists for +`forgejo-db`, and shared-storage/runner restore drills have not been evidenced. diff --git a/tests/test_forge_resources.py b/tests/test_forge_resources.py new file mode 100644 index 0000000..ee600a9 --- /dev/null +++ b/tests/test_forge_resources.py @@ -0,0 +1,33 @@ +from __future__ import annotations + +import importlib.util +import unittest +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +SPEC = importlib.util.spec_from_file_location("forge_resources", ROOT / "tools" / "observe_forge_resources.py") +assert SPEC and SPEC.loader +module = importlib.util.module_from_spec(SPEC) +SPEC.loader.exec_module(module) + + +class ForgeResourceTests(unittest.TestCase): + def test_resource_aggregation_and_residual(self) -> None: + deployment = {"metadata": {"name": "forgejo-gitea"}, "spec": {"replicas": 1, "template": {"spec": {"containers": [{"resources": {"requests": {"cpu": "100m", "memory": "128Mi"}, "limits": {"cpu": "1", "memory": "1Gi"}}}]}}}, "status": {"availableReplicas": 1}} + snapshot = { + "deployments": {"items": [deployment]}, + "pvcs": {"items": [{"metadata": {"name": "data"}, "spec": {"storageClassName": "local-path", "resources": {"requests": {"storage": "10Gi"}}}, "status": {"phase": "Bound"}}]}, + "database": {"spec": {"instances": 1, "storage": {"size": "10Gi"}}, "status": {"readyInstances": 1, "phase": "ready", "conditions": []}}, + "backups": {"items": []}, + "top": None, + "forge_data_kib": 1024, + "runner_data_kib": None, + } + result = module.build_observation(snapshot, "2026-08-22T00:00:00Z") + self.assertEqual(100, result["capacity"]["deployments"][0]["requests"]["cpu_millicores"]) + self.assertEqual(1024**2, result["allocation"]["residual"]["forge_data_used_bytes"]) + self.assertIn("forgejo-db has no Backup or ScheduledBackup evidence", result["measurement_gaps"]) + + +if __name__ == "__main__": + unittest.main() diff --git a/tools/observe_forge_resources.py b/tools/observe_forge_resources.py new file mode 100755 index 0000000..76ff36e --- /dev/null +++ b/tools/observe_forge_resources.py @@ -0,0 +1,232 @@ +#!/usr/bin/env python3 +"""Capture non-secret Forgejo resource and recovery evidence.""" + +from __future__ import annotations + +import argparse +import json +import re +import subprocess +import sys +from datetime import UTC, datetime +from pathlib import Path +from typing import Any + + +CPU_FACTORS = {"n": 0.000001, "u": 0.001, "m": 1.0, "": 1000.0} +BYTE_FACTORS = {"": 1, "Ki": 1024, "Mi": 1024**2, "Gi": 1024**3, "Ti": 1024**4, "K": 1000, "M": 1000**2, "G": 1000**3} + + +def cpu_millicores(value: str | None) -> int: + match = re.fullmatch(r"([0-9]+(?:\.[0-9]+)?)(n|u|m)?", str(value or "0")) + if not match: + raise ValueError(f"unsupported CPU quantity: {value}") + return round(float(match.group(1)) * CPU_FACTORS[match.group(2) or ""]) + + +def bytes_value(value: str | None) -> int: + match = re.fullmatch(r"([0-9]+(?:\.[0-9]+)?)(Ki|Mi|Gi|Ti|K|M|G)?", str(value or "0")) + if not match: + raise ValueError(f"unsupported byte quantity: {value}") + return round(float(match.group(1)) * BYTE_FACTORS[match.group(2) or ""]) + + +def resource_totals(deployment: dict[str, Any]) -> dict[str, Any]: + spec = (((deployment.get("spec") or {}).get("template") or {}).get("spec") or {}) + result = { + "requests": {"cpu_millicores": 0, "memory_bytes": 0}, + "limits": {"cpu_millicores": 0, "memory_bytes": 0}, + } + replicas = int((deployment.get("spec") or {}).get("replicas") or 0) + for container in spec.get("containers") or []: + resources = container.get("resources") or {} + for kind in ("requests", "limits"): + values = resources.get(kind) or {} + result[kind]["cpu_millicores"] += cpu_millicores(values.get("cpu")) * replicas + result[kind]["memory_bytes"] += bytes_value(values.get("memory")) * replicas + return result + + +def parse_top(text: str | None) -> dict[str, dict[str, int]]: + result: dict[str, dict[str, int]] = {} + for line in (text or "").splitlines(): + fields = line.split() + if len(fields) >= 3: + result[fields[0]] = {"cpu_millicores": cpu_millicores(fields[1]), "memory_bytes": bytes_value(fields[2])} + return result + + +def build_observation(snapshot: dict[str, Any], captured_at: str) -> dict[str, Any]: + gaps: list[str] = [] + top = parse_top(snapshot.get("top")) + if snapshot.get("top") is None: + gaps.append("forge pod metrics unavailable") + deployments = snapshot["deployments"].get("items") or [] + deployment_rows = [] + for deployment in deployments: + meta = deployment.get("metadata") or {} + status = deployment.get("status") or {} + name = str(meta.get("name")) + matching = [metric for pod, metric in top.items() if pod.startswith(f"{name}-")] + observed = None + if snapshot.get("top") is not None: + observed = { + "cpu_millicores": sum(item["cpu_millicores"] for item in matching), + "memory_bytes": sum(item["memory_bytes"] for item in matching), + } + deployment_rows.append({ + "name": name, + "desired_replicas": int((deployment.get("spec") or {}).get("replicas") or 0), + "available_replicas": int(status.get("availableReplicas") or 0), + **resource_totals(deployment), + "observed": observed, + }) + + pvcs = [] + total_pvc = 0 + for item in snapshot["pvcs"].get("items") or []: + meta = item.get("metadata") or {} + spec = item.get("spec") or {} + requested = bytes_value(((spec.get("resources") or {}).get("requests") or {}).get("storage")) + total_pvc += requested + pvcs.append({ + "name": meta.get("name"), + "storage_class": spec.get("storageClassName"), + "requested_bytes": requested, + "phase": (item.get("status") or {}).get("phase"), + }) + + database = snapshot["database"] + db_spec = database.get("spec") or {} + db_status = database.get("status") or {} + backups = [ + item for item in snapshot["backups"].get("items") or [] + if ((item.get("spec") or {}).get("cluster") or {}).get("name") == "forgejo-db" + ] + completed = [item for item in backups if (item.get("status") or {}).get("phase") == "completed"] + if not backups: + gaps.append("forgejo-db has no Backup or ScheduledBackup evidence") + if snapshot.get("forge_data_kib") is None: + gaps.append("forge shared-storage used bytes unavailable") + if snapshot.get("runner_data_kib") is None: + gaps.append("runner storage used bytes unavailable") + gaps.extend([ + "repository and package usage are not yet attributable to named consumers", + "runner job counts and durations are unavailable from the current read-only surface", + "material network traffic is unavailable", + ]) + + return { + "schema_version": "railiance.forge-resource-observation.v1", + "record_type": "usage_observation", + "resource_id": "resource:railiance:forgejo", + "source": "railiance-forge", + "workplan_id": "RFORGE-WP-0002", + "reef": "reef-railiance", + "captured_at": captured_at, + "capacity": { + "deployments": deployment_rows, + "persistent_volumes": pvcs, + "pvc_requested_bytes": total_pvc, + "forge_data_used_bytes": int(snapshot["forge_data_kib"]) * 1024 if snapshot.get("forge_data_kib") is not None else None, + "runner_data_used_bytes": int(snapshot["runner_data_kib"]) * 1024 if snapshot.get("runner_data_kib") is not None else None, + "database": { + "instances": db_spec.get("instances"), + "ready_instances": db_status.get("readyInstances"), + "storage_requested_bytes": bytes_value((db_spec.get("storage") or {}).get("size")), + "phase": db_status.get("phase"), + }, + }, + "recovery": { + "database_continuous_archiving": any(c.get("type") == "ContinuousArchiving" and c.get("status") == "True" for c in db_status.get("conditions") or []), + "database_backups_seen": len(backups), + "database_backups_completed": len(completed), + "shared_storage_restore_drill": False, + "runner_restore_drill": False, + }, + "allocation": { + "method_version": "forge-raw-drivers-v1", + "candidate_drivers": ["repository_count", "package_bytes", "runner_job_minutes"], + "consumer_groups": ["helix-forge", "coulomb-social", "railiance"], + "attributed": [], + "residual": {"forge_data_used_bytes": int(snapshot["forge_data_kib"]) * 1024 if snapshot.get("forge_data_kib") is not None else None, "reason": "consumer attribution source is not yet available"}, + "selection_owner": "resource-control", + }, + "measurement_gaps": sorted(set(gaps)), + "provenance": { + "commands": [ + "kubectl get deploy -n forgejo -o json", + "kubectl get pvc -n forgejo -o json", + "kubectl top pods -n forgejo --no-headers", + "kubectl get cluster forgejo-db -n databases -o json", + "kubectl get backup,scheduledbackup -n databases -o json", + "kubectl exec deploy/forgejo-gitea -- du -sk /data", + "kubectl exec deploy/forgejo-runner -- du -sk /data", + ], + "repository_contents_read": False, + "secret_surfaces_read": False, + }, + } + + +def run(prefix: list[str], args: list[str], *, optional: bool = False) -> str | None: + completed = subprocess.run(prefix + args, text=True, capture_output=True, check=False) + if completed.returncode != 0: + if optional: + return None + raise subprocess.CalledProcessError(completed.returncode, prefix + args, completed.stdout, completed.stderr) + return completed.stdout + + +def collect(remote: str | None, kubectl: str) -> dict[str, Any]: + prefix = ["ssh", "-o", "BatchMode=yes", remote, kubectl] if remote else [kubectl] + load = lambda args: json.loads(run(prefix, args) or "{}") + data = { + "deployments": load(["get", "deploy", "-n", "forgejo", "-o", "json"]), + "pvcs": load(["get", "pvc", "-n", "forgejo", "-o", "json"]), + "database": load(["get", "cluster", "forgejo-db", "-n", "databases", "-o", "json"]), + "backups": load(["get", "backup,scheduledbackup", "-n", "databases", "-o", "json"]), + "top": run(prefix, ["top", "pods", "-n", "forgejo", "--no-headers"], optional=True), + } + # Avoid an inner shell here: when kubectl is executed through SSH, shell + # quoting can otherwise turn ``sh -c 'du ...'`` into ``sh -c du`` and + # silently measure the container working directory instead of /data. + forge_du = run(prefix, ["exec", "-n", "forgejo", "deploy/forgejo-gitea", "-c", "gitea", "--", "du", "-sk", "/data"], optional=True) + runner_du = run(prefix, ["exec", "-n", "forgejo", "deploy/forgejo-runner", "-c", "runner", "--", "du", "-sk", "/data"], optional=True) + + def du_kib(output: str | None) -> int | None: + fields = (output or "").split() + return int(fields[0]) if fields and fields[0].isdigit() else None + + data["forge_data_kib"] = du_kib(forge_du) + data["runner_data_kib"] = du_kib(runner_du) + captured_at = datetime.now(UTC).replace(microsecond=0).isoformat().replace("+00:00", "Z") + return build_observation(data, captured_at) + + +def main() -> int: + parser = argparse.ArgumentParser() + parser.add_argument("--remote") + parser.add_argument("--kubectl", default="kubectl") + parser.add_argument("--output-dir", type=Path) + args = parser.parse_args() + try: + observation = collect(args.remote, args.kubectl) + except (OSError, subprocess.CalledProcessError, json.JSONDecodeError, ValueError) as exc: + print(f"forge observation failed: {exc}", file=sys.stderr) + return 1 + rendered = json.dumps(observation, indent=2) + "\n" + if args.output_dir: + args.output_dir.mkdir(parents=True, exist_ok=True) + stamp = observation["captured_at"].replace(":", "").replace("-", "") + destination = args.output_dir / f"{stamp}.json" + destination.write_text(rendered, encoding="utf-8") + (args.output_dir / "latest.json").write_text(rendered, encoding="utf-8") + print(destination) + else: + sys.stdout.write(rendered) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md b/workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md index 42e5a9f..2c8df3c 100644 --- a/workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md +++ b/workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md @@ -4,11 +4,11 @@ type: workplan title: "Publish Forgejo resource economics evidence" domain: financials repo: railiance-forge -status: ready +status: finished owner: codex topic_slug: railiance created: "2026-08-11" -updated: "2026-08-15" +updated: "2026-08-22" related: - RESOURCE-WP-0003 - ADR-0008 @@ -43,7 +43,7 @@ a separate granted surface, not this plan. Do not treat ```task id: RFORGE-WP-0002-T01 -status: todo +status: done priority: high state_hub_task_id: "7048e354-0844-4f6c-aeaa-acf188b04ce2" ``` @@ -55,11 +55,17 @@ and material network traffic using stable units and timestamps. Done when resource-control can forecast capacity and identify saturation or idle capacity without accessing repository content or credentials. +**Outcome (2026-08-22):** `make forge-observe REMOTE=railiance01` publishes a +timestamped, versioned observation for deployment requests/limits and observed +CPU/memory, PVC and database capacity, shared/runner storage usage, recovery +posture, and unavailable axes. It reads neither repository contents nor Secret +surfaces. + ## T02 — Declare service objectives and operations labor ```task id: RFORGE-WP-0002-T02 -status: todo +status: done priority: high state_hub_task_id: "643c2393-2ee5-4264-972c-04c3b358dc86" ``` @@ -71,11 +77,16 @@ database, backup, upgrade, and incident labor where evidence permits. Done when total-cost and optimization cases can preserve the forge's service constraints and labor assumptions. +**Outcome (2026-08-22):** the service-objective and labor evidence file names +the availability, restore, retention, performance, operational, upgrade, and +incident dimensions. Unknown targets and hours remain explicit nulls rather +than invented estimates. + ## T03 — Publish shared-consumer allocation evidence ```task id: RFORGE-WP-0002-T03 -status: todo +status: done priority: medium state_hub_task_id: "498cc0c7-ba07-4e7f-a0ef-2e6b34bef04c" ``` @@ -87,8 +98,21 @@ and unattributed residuals; do not implement customer billing. Done when a versioned driver report can feed resource-control allocation without making the forge repository authoritative for booked cost. +**Outcome (2026-08-22):** `forge-raw-drivers-v1` proposes repository count, +package bytes, and runner job minutes, preserves all current use as an +unattributed residual, and assigns method selection to `resource-control`. + ## Acceptance -- [ ] Demand and utilization include storage, compute, runner, and backup axes. -- [ ] Service objectives and labor assumptions are explicit. -- [ ] Shared allocation evidence exposes residuals and provenance. +- [x] Demand and utilization include storage, compute, runner, and backup axes. +- [x] Service objectives and labor assumptions are explicit. +- [x] Shared allocation evidence exposes residuals and provenance. + +## Completion evidence + +- `python3 -m unittest discover -s tests -p 'test_*.py' -v`: 1 test passed. +- Live read-only capture: + `docs/evidence/resource-railiance-forgejo/observations/20260822T102256Z.json`. +- Current observation records 2,377,375,744 forge data bytes, 247,693,312 + runner data bytes, healthy single-instance CNPG state, continuous archiving, + and no Forge database Backup/ScheduledBackup or restore-drill evidence.