Publish Forge resource economics evidence
Assistant: codex Assistant-Model: gpt-5.6-sol Assistant-Session: 01a02669-87ee-7a31-b111-edc95a16e0fa
This commit is contained in:
parent
a0f4c493a3
commit
35dbe4dd42
12 changed files with 632 additions and 30 deletions
17
INTENT.md
17
INTENT.md
|
|
@ -122,10 +122,12 @@ description left it unplaced - which is how layer boundaries erode.
|
|||
Placement was decided 2026-08-11 (hub decision `d151d817`):
|
||||
|
||||
* the **Forgejo workload** becomes `rapp-forgejo`, a managed workload package
|
||||
* the **layer responsibility** - runner placement and labels, registry
|
||||
retention, artifact lifecycle, package credentials - folds into **S4
|
||||
`railiance-enablement`**, which already declares a handoff contract for
|
||||
exactly these concerns
|
||||
* the **consumer-facing contract** - reusable workflow expectations for runner
|
||||
labels, registry interfaces, package credentials, and artifact evidence -
|
||||
belongs to **S4 `railiance-enablement`**
|
||||
* this repo remains the **operational provider** for the live forge, registry,
|
||||
runner substrate, retention, custody references, and operating evidence
|
||||
until a separately approved migration changes that boundary
|
||||
|
||||
A retirement hypothesis was tested and rejected: this repo and `helix-forge` are
|
||||
different lineages, not one maturing into the other. `helix-forge` is
|
||||
|
|
@ -133,8 +135,11 @@ methodological (capability-first ecosystem, OAS dimensions, VSM vocabulary);
|
|||
this repo is operational. Retiring it would orphan responsibilities nothing
|
||||
else owns.
|
||||
|
||||
Until the migration completes, this repo remains the operating home for forge
|
||||
concerns. See `railiance-platform/ArchitectureBlueprint.md` section 5.2.
|
||||
This split does not require a physical merge: enablement owns the paved
|
||||
consumer path, while this repo owns the capability that path consumes. The
|
||||
Forgejo workload is expected to become `rapp-forgejo` without moving its
|
||||
operator contract into enablement. See `railiance-platform/ArchitectureBlueprint.md`
|
||||
section 5.2.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
20
Makefile
20
Makefile
|
|
@ -1,15 +1,15 @@
|
|||
SHELL := /usr/bin/env bash
|
||||
.DEFAULT_GOAL := help
|
||||
|
||||
GITEA_RELEASE ?= gitea
|
||||
GITEA_NAMESPACE ?= default
|
||||
GITEA_RELEASE ?= forgejo
|
||||
GITEA_NAMESPACE ?= forgejo
|
||||
GITEA_CHART ?= gitea-charts/gitea
|
||||
GITEA_VALUES ?= helm/gitea-values.sops.yaml
|
||||
GITEA_REGISTRY_VALUES ?= helm/gitea-registry-values.yaml
|
||||
GITEA_INGRESS ?= manifests/gitea-ingress.yaml
|
||||
GITEA_HTTP_SERVICE ?= gitea-http
|
||||
GITEA_SSH_SERVICE ?= gitea-ssh-nodeport
|
||||
GITEA_DB_CLUSTER ?= gitea-db
|
||||
GITEA_HTTP_SERVICE ?= forgejo-gitea-http
|
||||
GITEA_SSH_SERVICE ?= forgejo-ssh-nodeport
|
||||
GITEA_DB_CLUSTER ?= forgejo-db
|
||||
GITEA_DB_NAMESPACE ?= databases
|
||||
REGISTRY_DOCS ?= docs/gitea-container-registry.md docs/gitea-package-registry.md
|
||||
EVIDENCE_DOCS ?= docs/observability-operating-evidence.md docs/ci-runner-actions-gitops-ownership.md docs/backup-restore-secret-handoff.md
|
||||
|
|
@ -62,6 +62,14 @@ runner-status: ## Read-only Actions runner, host, and inter-hub registry probes
|
|||
forgejo-runner-status: ## Read-only Forgejo Actions runner and endpoint probes
|
||||
bash tools/forgejo-runner-status.sh
|
||||
|
||||
forge-observe: ## Capture non-secret forge capacity, recovery, and allocation evidence
|
||||
python3 tools/observe_forge_resources.py \
|
||||
$(if $(REMOTE),--remote $(REMOTE),) \
|
||||
--output-dir docs/evidence/resource-railiance-forgejo/observations
|
||||
|
||||
test-unit: ## Run non-live forge evidence tests
|
||||
python3 -m unittest discover -s tests -p 'test_*.py' -v
|
||||
|
||||
check-runner-tools: ## Check local tools used by runner inspection targets
|
||||
@missing=0; \
|
||||
for tool in curl ssh docker; do \
|
||||
|
|
@ -112,4 +120,4 @@ help: ## Show this help
|
|||
/^[a-zA-Z0-9_-]+:.*?##/ { printf " \033[36m%-20s\033[0m %s\n", $$1, $$2 } \
|
||||
/^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) }' $(MAKEFILE_LIST)
|
||||
|
||||
.PHONY: check-tools check-sops registry-docs evidence-docs runner-docs runner-status forgejo-runner-status check-runner-tools gitea-deploy gitea-ingress-deploy gitea-status help
|
||||
.PHONY: check-tools check-sops registry-docs evidence-docs runner-docs runner-status forgejo-runner-status forge-observe test-unit check-runner-tools gitea-deploy gitea-ingress-deploy gitea-status help
|
||||
|
|
|
|||
13
SCOPE.md
13
SCOPE.md
|
|
@ -3,7 +3,7 @@
|
|||
This file defines what `railiance-forge` owns, when to use it, and where its
|
||||
boundaries stop.
|
||||
|
||||
Last reviewed: 2026-06-13
|
||||
Last reviewed: 2026-08-22
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -113,14 +113,15 @@ The Fabric graph declarations for forge capabilities and edges live in
|
|||
|
||||
## Current State
|
||||
|
||||
- Status: active forge extraction.
|
||||
- Status: active forge operation with explicit S4 consumer-contract handoff.
|
||||
- Implementation: repository contract, registry docs, initial operating
|
||||
contracts, deploy-capable Gitea files, and operator targets are present.
|
||||
- Stability: emerging but now live-facing; forge owns the reviewed public
|
||||
Gitea HTTPS ingress for the web UI, package registry, and OCI registry. Raw
|
||||
node IP HTTP access is not part of the supported forge surface.
|
||||
- Usage: canonical reference point for forge and registry responsibilities
|
||||
currently transitioning out of `railiance-apps`.
|
||||
- Usage: canonical operational reference for forge, registry, runner, retention,
|
||||
recovery, and resource evidence. S4 owns reusable consumer templates; this
|
||||
repo remains their provider and does not duplicate those templates.
|
||||
|
||||
Known starting point:
|
||||
|
||||
|
|
@ -128,8 +129,8 @@ Known starting point:
|
|||
ingress, operating contracts, and deploy/status entry points.
|
||||
- `railiance-apps` keeps app release ownership and transitional compatibility
|
||||
wrappers for old Gitea commands.
|
||||
- `railiance-enablement` owns the intent for delivery templates and developer
|
||||
paved paths, but not forge runtime operation.
|
||||
- `railiance-enablement` owns delivery templates and developer paved paths,
|
||||
but not forge runtime, registry, runner, retention, or recovery operation.
|
||||
- `railiance-forge` should absorb forge runtime and artifact infrastructure
|
||||
without moving S5 app release ownership.
|
||||
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@
|
|||
|
||||
| Kind | ID | Status | Lane | Source |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| workplan | RFORGE-WP-0002 | ready | — | workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md |
|
||||
| task | RFORGE-WP-0002-T01 | todo | — | workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md |
|
||||
| task | RFORGE-WP-0002-T02 | todo | — | workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md |
|
||||
| task | RFORGE-WP-0002-T03 | todo | — | workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md |
|
||||
| workplan | RFORGE-WP-0002 | finished | — | workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md |
|
||||
| task | RFORGE-WP-0002-T01 | done | — | workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md |
|
||||
| task | RFORGE-WP-0002-T02 | done | — | workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md |
|
||||
| task | RFORGE-WP-0002-T03 | done | — | workplans/RFORGE-WP-0002-forge-resource-economics-evidence.md |
|
||||
|
|
|
|||
|
|
@ -0,0 +1,20 @@
|
|||
schema_version: railiance.forge-allocation-drivers.v1
|
||||
resource_id: resource:railiance:forgejo
|
||||
method_version: forge-raw-drivers-v1
|
||||
candidate_drivers:
|
||||
- repository_count
|
||||
- package_bytes
|
||||
- runner_job_minutes
|
||||
consumer_groups:
|
||||
- helix-forge
|
||||
- coulomb-social
|
||||
- railiance
|
||||
selected_driver: null
|
||||
attributed: []
|
||||
residual:
|
||||
share: 1.0
|
||||
reason: current read-only surfaces do not bind repository, package, and runner usage to those consumer groups
|
||||
selection_owner: resource-control
|
||||
provenance:
|
||||
- Kubernetes deployment, PVC, CNPG, runner, and storage observations
|
||||
- Forgejo aggregate API evidence when an approved read-only route becomes available
|
||||
|
|
@ -0,0 +1,115 @@
|
|||
{
|
||||
"schema_version": "railiance.forge-resource-observation.v1",
|
||||
"record_type": "usage_observation",
|
||||
"resource_id": "resource:railiance:forgejo",
|
||||
"source": "railiance-forge",
|
||||
"workplan_id": "RFORGE-WP-0002",
|
||||
"reef": "reef-railiance",
|
||||
"captured_at": "2026-08-22T10:22:56Z",
|
||||
"capacity": {
|
||||
"deployments": [
|
||||
{
|
||||
"name": "forgejo-gitea",
|
||||
"desired_replicas": 1,
|
||||
"available_replicas": 1,
|
||||
"requests": {
|
||||
"cpu_millicores": 0,
|
||||
"memory_bytes": 0
|
||||
},
|
||||
"limits": {
|
||||
"cpu_millicores": 0,
|
||||
"memory_bytes": 0
|
||||
},
|
||||
"observed": {
|
||||
"cpu_millicores": 115,
|
||||
"memory_bytes": 629145600
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "forgejo-runner",
|
||||
"desired_replicas": 1,
|
||||
"available_replicas": 1,
|
||||
"requests": {
|
||||
"cpu_millicores": 0,
|
||||
"memory_bytes": 0
|
||||
},
|
||||
"limits": {
|
||||
"cpu_millicores": 0,
|
||||
"memory_bytes": 0
|
||||
},
|
||||
"observed": {
|
||||
"cpu_millicores": 3,
|
||||
"memory_bytes": 563085312
|
||||
}
|
||||
}
|
||||
],
|
||||
"persistent_volumes": [
|
||||
{
|
||||
"name": "forgejo-runner-data",
|
||||
"storage_class": "local-path",
|
||||
"requested_bytes": 5368709120,
|
||||
"phase": "Bound"
|
||||
},
|
||||
{
|
||||
"name": "gitea-shared-storage",
|
||||
"storage_class": "local-path",
|
||||
"requested_bytes": 10737418240,
|
||||
"phase": "Bound"
|
||||
}
|
||||
],
|
||||
"pvc_requested_bytes": 16106127360,
|
||||
"forge_data_used_bytes": 2377375744,
|
||||
"runner_data_used_bytes": 247693312,
|
||||
"database": {
|
||||
"instances": 1,
|
||||
"ready_instances": 1,
|
||||
"storage_requested_bytes": 10737418240,
|
||||
"phase": "Cluster in healthy state"
|
||||
}
|
||||
},
|
||||
"recovery": {
|
||||
"database_continuous_archiving": true,
|
||||
"database_backups_seen": 0,
|
||||
"database_backups_completed": 0,
|
||||
"shared_storage_restore_drill": false,
|
||||
"runner_restore_drill": false
|
||||
},
|
||||
"allocation": {
|
||||
"method_version": "forge-raw-drivers-v1",
|
||||
"candidate_drivers": [
|
||||
"repository_count",
|
||||
"package_bytes",
|
||||
"runner_job_minutes"
|
||||
],
|
||||
"consumer_groups": [
|
||||
"helix-forge",
|
||||
"coulomb-social",
|
||||
"railiance"
|
||||
],
|
||||
"attributed": [],
|
||||
"residual": {
|
||||
"forge_data_used_bytes": 2377375744,
|
||||
"reason": "consumer attribution source is not yet available"
|
||||
},
|
||||
"selection_owner": "resource-control"
|
||||
},
|
||||
"measurement_gaps": [
|
||||
"forgejo-db has no Backup or ScheduledBackup evidence",
|
||||
"material network traffic is unavailable",
|
||||
"repository and package usage are not yet attributable to named consumers",
|
||||
"runner job counts and durations are unavailable from the current read-only surface"
|
||||
],
|
||||
"provenance": {
|
||||
"commands": [
|
||||
"kubectl get deploy -n forgejo -o json",
|
||||
"kubectl get pvc -n forgejo -o json",
|
||||
"kubectl top pods -n forgejo --no-headers",
|
||||
"kubectl get cluster forgejo-db -n databases -o json",
|
||||
"kubectl get backup,scheduledbackup -n databases -o json",
|
||||
"kubectl exec deploy/forgejo-gitea -- du -sk /data",
|
||||
"kubectl exec deploy/forgejo-runner -- du -sk /data"
|
||||
],
|
||||
"repository_contents_read": false,
|
||||
"secret_surfaces_read": false
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,115 @@
|
|||
{
|
||||
"schema_version": "railiance.forge-resource-observation.v1",
|
||||
"record_type": "usage_observation",
|
||||
"resource_id": "resource:railiance:forgejo",
|
||||
"source": "railiance-forge",
|
||||
"workplan_id": "RFORGE-WP-0002",
|
||||
"reef": "reef-railiance",
|
||||
"captured_at": "2026-08-22T10:22:56Z",
|
||||
"capacity": {
|
||||
"deployments": [
|
||||
{
|
||||
"name": "forgejo-gitea",
|
||||
"desired_replicas": 1,
|
||||
"available_replicas": 1,
|
||||
"requests": {
|
||||
"cpu_millicores": 0,
|
||||
"memory_bytes": 0
|
||||
},
|
||||
"limits": {
|
||||
"cpu_millicores": 0,
|
||||
"memory_bytes": 0
|
||||
},
|
||||
"observed": {
|
||||
"cpu_millicores": 115,
|
||||
"memory_bytes": 629145600
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "forgejo-runner",
|
||||
"desired_replicas": 1,
|
||||
"available_replicas": 1,
|
||||
"requests": {
|
||||
"cpu_millicores": 0,
|
||||
"memory_bytes": 0
|
||||
},
|
||||
"limits": {
|
||||
"cpu_millicores": 0,
|
||||
"memory_bytes": 0
|
||||
},
|
||||
"observed": {
|
||||
"cpu_millicores": 3,
|
||||
"memory_bytes": 563085312
|
||||
}
|
||||
}
|
||||
],
|
||||
"persistent_volumes": [
|
||||
{
|
||||
"name": "forgejo-runner-data",
|
||||
"storage_class": "local-path",
|
||||
"requested_bytes": 5368709120,
|
||||
"phase": "Bound"
|
||||
},
|
||||
{
|
||||
"name": "gitea-shared-storage",
|
||||
"storage_class": "local-path",
|
||||
"requested_bytes": 10737418240,
|
||||
"phase": "Bound"
|
||||
}
|
||||
],
|
||||
"pvc_requested_bytes": 16106127360,
|
||||
"forge_data_used_bytes": 2377375744,
|
||||
"runner_data_used_bytes": 247693312,
|
||||
"database": {
|
||||
"instances": 1,
|
||||
"ready_instances": 1,
|
||||
"storage_requested_bytes": 10737418240,
|
||||
"phase": "Cluster in healthy state"
|
||||
}
|
||||
},
|
||||
"recovery": {
|
||||
"database_continuous_archiving": true,
|
||||
"database_backups_seen": 0,
|
||||
"database_backups_completed": 0,
|
||||
"shared_storage_restore_drill": false,
|
||||
"runner_restore_drill": false
|
||||
},
|
||||
"allocation": {
|
||||
"method_version": "forge-raw-drivers-v1",
|
||||
"candidate_drivers": [
|
||||
"repository_count",
|
||||
"package_bytes",
|
||||
"runner_job_minutes"
|
||||
],
|
||||
"consumer_groups": [
|
||||
"helix-forge",
|
||||
"coulomb-social",
|
||||
"railiance"
|
||||
],
|
||||
"attributed": [],
|
||||
"residual": {
|
||||
"forge_data_used_bytes": 2377375744,
|
||||
"reason": "consumer attribution source is not yet available"
|
||||
},
|
||||
"selection_owner": "resource-control"
|
||||
},
|
||||
"measurement_gaps": [
|
||||
"forgejo-db has no Backup or ScheduledBackup evidence",
|
||||
"material network traffic is unavailable",
|
||||
"repository and package usage are not yet attributable to named consumers",
|
||||
"runner job counts and durations are unavailable from the current read-only surface"
|
||||
],
|
||||
"provenance": {
|
||||
"commands": [
|
||||
"kubectl get deploy -n forgejo -o json",
|
||||
"kubectl get pvc -n forgejo -o json",
|
||||
"kubectl top pods -n forgejo --no-headers",
|
||||
"kubectl get cluster forgejo-db -n databases -o json",
|
||||
"kubectl get backup,scheduledbackup -n databases -o json",
|
||||
"kubectl exec deploy/forgejo-gitea -- du -sk /data",
|
||||
"kubectl exec deploy/forgejo-runner -- du -sk /data"
|
||||
],
|
||||
"repository_contents_read": false,
|
||||
"secret_surfaces_read": false
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,31 @@
|
|||
schema_version: railiance.forge-service-objectives.v1
|
||||
resource_id: resource:railiance:forgejo
|
||||
workplan_id: RFORGE-WP-0002
|
||||
service_objectives:
|
||||
availability:
|
||||
objective: web, Git SSH, OCI, package, and runner readiness checks pass at each observation
|
||||
percentage_target: null
|
||||
measurement_gap: no approved monthly availability SLO or continuous evidence plane
|
||||
recovery:
|
||||
rpo: null
|
||||
rto: null
|
||||
requirement: isolated restore must prove clone, login, package pull, and runner re-registration before production trust
|
||||
retention:
|
||||
policy: null
|
||||
requirement: cleanup must preserve referenced release artifacts and evidence
|
||||
performance:
|
||||
objective: null
|
||||
measurement_gap: no approved latency or throughput target
|
||||
operations_labor:
|
||||
basis: activities declared; hours remain unknown until time records exist
|
||||
activities:
|
||||
- forge and registry health review
|
||||
- runner health and label review
|
||||
- storage growth and retention review
|
||||
- database and shared-storage backup review
|
||||
- upgrade and migration rehearsal
|
||||
- incident response and restore drill
|
||||
setup_hours: null
|
||||
recurring_hours_per_month: null
|
||||
measurement_gap: no attributable time records
|
||||
cost_owner: resource-control/fin-hub
|
||||
18
docs/forge-resource-evidence.md
Normal file
18
docs/forge-resource-evidence.md
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
# Forge resource evidence
|
||||
|
||||
Run `make forge-observe REMOTE=railiance01` to capture a timestamped,
|
||||
non-secret observation for `resource:railiance:forgejo`. The collector reads
|
||||
Kubernetes deployment, metrics, PVC, CNPG, backup-resource, and aggregate
|
||||
`du` surfaces. It does not read repositories, packages, Secrets, or pod
|
||||
environments.
|
||||
|
||||
The observation separates requested/limited compute from observed usage,
|
||||
records storage and recovery posture, and preserves unavailable dimensions as
|
||||
measurement gaps. The companion YAML files define service/labor assumptions
|
||||
and candidate allocation drivers. Unknown targets, labor, consumer
|
||||
attribution, and booked cost stay unknown; `resource-control` owns allocation
|
||||
method selection and `fin-hub` owns financial reconciliation.
|
||||
|
||||
The current capture also makes the recovery gap explicit: CNPG reports
|
||||
continuous archiving, but no `Backup` or `ScheduledBackup` resource exists for
|
||||
`forgejo-db`, and shared-storage/runner restore drills have not been evidenced.
|
||||
33
tests/test_forge_resources.py
Normal file
33
tests/test_forge_resources.py
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
SPEC = importlib.util.spec_from_file_location("forge_resources", ROOT / "tools" / "observe_forge_resources.py")
|
||||
assert SPEC and SPEC.loader
|
||||
module = importlib.util.module_from_spec(SPEC)
|
||||
SPEC.loader.exec_module(module)
|
||||
|
||||
|
||||
class ForgeResourceTests(unittest.TestCase):
|
||||
def test_resource_aggregation_and_residual(self) -> None:
|
||||
deployment = {"metadata": {"name": "forgejo-gitea"}, "spec": {"replicas": 1, "template": {"spec": {"containers": [{"resources": {"requests": {"cpu": "100m", "memory": "128Mi"}, "limits": {"cpu": "1", "memory": "1Gi"}}}]}}}, "status": {"availableReplicas": 1}}
|
||||
snapshot = {
|
||||
"deployments": {"items": [deployment]},
|
||||
"pvcs": {"items": [{"metadata": {"name": "data"}, "spec": {"storageClassName": "local-path", "resources": {"requests": {"storage": "10Gi"}}}, "status": {"phase": "Bound"}}]},
|
||||
"database": {"spec": {"instances": 1, "storage": {"size": "10Gi"}}, "status": {"readyInstances": 1, "phase": "ready", "conditions": []}},
|
||||
"backups": {"items": []},
|
||||
"top": None,
|
||||
"forge_data_kib": 1024,
|
||||
"runner_data_kib": None,
|
||||
}
|
||||
result = module.build_observation(snapshot, "2026-08-22T00:00:00Z")
|
||||
self.assertEqual(100, result["capacity"]["deployments"][0]["requests"]["cpu_millicores"])
|
||||
self.assertEqual(1024**2, result["allocation"]["residual"]["forge_data_used_bytes"])
|
||||
self.assertIn("forgejo-db has no Backup or ScheduledBackup evidence", result["measurement_gaps"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
232
tools/observe_forge_resources.py
Executable file
232
tools/observe_forge_resources.py
Executable file
|
|
@ -0,0 +1,232 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Capture non-secret Forgejo resource and recovery evidence."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
CPU_FACTORS = {"n": 0.000001, "u": 0.001, "m": 1.0, "": 1000.0}
|
||||
BYTE_FACTORS = {"": 1, "Ki": 1024, "Mi": 1024**2, "Gi": 1024**3, "Ti": 1024**4, "K": 1000, "M": 1000**2, "G": 1000**3}
|
||||
|
||||
|
||||
def cpu_millicores(value: str | None) -> int:
|
||||
match = re.fullmatch(r"([0-9]+(?:\.[0-9]+)?)(n|u|m)?", str(value or "0"))
|
||||
if not match:
|
||||
raise ValueError(f"unsupported CPU quantity: {value}")
|
||||
return round(float(match.group(1)) * CPU_FACTORS[match.group(2) or ""])
|
||||
|
||||
|
||||
def bytes_value(value: str | None) -> int:
|
||||
match = re.fullmatch(r"([0-9]+(?:\.[0-9]+)?)(Ki|Mi|Gi|Ti|K|M|G)?", str(value or "0"))
|
||||
if not match:
|
||||
raise ValueError(f"unsupported byte quantity: {value}")
|
||||
return round(float(match.group(1)) * BYTE_FACTORS[match.group(2) or ""])
|
||||
|
||||
|
||||
def resource_totals(deployment: dict[str, Any]) -> dict[str, Any]:
|
||||
spec = (((deployment.get("spec") or {}).get("template") or {}).get("spec") or {})
|
||||
result = {
|
||||
"requests": {"cpu_millicores": 0, "memory_bytes": 0},
|
||||
"limits": {"cpu_millicores": 0, "memory_bytes": 0},
|
||||
}
|
||||
replicas = int((deployment.get("spec") or {}).get("replicas") or 0)
|
||||
for container in spec.get("containers") or []:
|
||||
resources = container.get("resources") or {}
|
||||
for kind in ("requests", "limits"):
|
||||
values = resources.get(kind) or {}
|
||||
result[kind]["cpu_millicores"] += cpu_millicores(values.get("cpu")) * replicas
|
||||
result[kind]["memory_bytes"] += bytes_value(values.get("memory")) * replicas
|
||||
return result
|
||||
|
||||
|
||||
def parse_top(text: str | None) -> dict[str, dict[str, int]]:
|
||||
result: dict[str, dict[str, int]] = {}
|
||||
for line in (text or "").splitlines():
|
||||
fields = line.split()
|
||||
if len(fields) >= 3:
|
||||
result[fields[0]] = {"cpu_millicores": cpu_millicores(fields[1]), "memory_bytes": bytes_value(fields[2])}
|
||||
return result
|
||||
|
||||
|
||||
def build_observation(snapshot: dict[str, Any], captured_at: str) -> dict[str, Any]:
|
||||
gaps: list[str] = []
|
||||
top = parse_top(snapshot.get("top"))
|
||||
if snapshot.get("top") is None:
|
||||
gaps.append("forge pod metrics unavailable")
|
||||
deployments = snapshot["deployments"].get("items") or []
|
||||
deployment_rows = []
|
||||
for deployment in deployments:
|
||||
meta = deployment.get("metadata") or {}
|
||||
status = deployment.get("status") or {}
|
||||
name = str(meta.get("name"))
|
||||
matching = [metric for pod, metric in top.items() if pod.startswith(f"{name}-")]
|
||||
observed = None
|
||||
if snapshot.get("top") is not None:
|
||||
observed = {
|
||||
"cpu_millicores": sum(item["cpu_millicores"] for item in matching),
|
||||
"memory_bytes": sum(item["memory_bytes"] for item in matching),
|
||||
}
|
||||
deployment_rows.append({
|
||||
"name": name,
|
||||
"desired_replicas": int((deployment.get("spec") or {}).get("replicas") or 0),
|
||||
"available_replicas": int(status.get("availableReplicas") or 0),
|
||||
**resource_totals(deployment),
|
||||
"observed": observed,
|
||||
})
|
||||
|
||||
pvcs = []
|
||||
total_pvc = 0
|
||||
for item in snapshot["pvcs"].get("items") or []:
|
||||
meta = item.get("metadata") or {}
|
||||
spec = item.get("spec") or {}
|
||||
requested = bytes_value(((spec.get("resources") or {}).get("requests") or {}).get("storage"))
|
||||
total_pvc += requested
|
||||
pvcs.append({
|
||||
"name": meta.get("name"),
|
||||
"storage_class": spec.get("storageClassName"),
|
||||
"requested_bytes": requested,
|
||||
"phase": (item.get("status") or {}).get("phase"),
|
||||
})
|
||||
|
||||
database = snapshot["database"]
|
||||
db_spec = database.get("spec") or {}
|
||||
db_status = database.get("status") or {}
|
||||
backups = [
|
||||
item for item in snapshot["backups"].get("items") or []
|
||||
if ((item.get("spec") or {}).get("cluster") or {}).get("name") == "forgejo-db"
|
||||
]
|
||||
completed = [item for item in backups if (item.get("status") or {}).get("phase") == "completed"]
|
||||
if not backups:
|
||||
gaps.append("forgejo-db has no Backup or ScheduledBackup evidence")
|
||||
if snapshot.get("forge_data_kib") is None:
|
||||
gaps.append("forge shared-storage used bytes unavailable")
|
||||
if snapshot.get("runner_data_kib") is None:
|
||||
gaps.append("runner storage used bytes unavailable")
|
||||
gaps.extend([
|
||||
"repository and package usage are not yet attributable to named consumers",
|
||||
"runner job counts and durations are unavailable from the current read-only surface",
|
||||
"material network traffic is unavailable",
|
||||
])
|
||||
|
||||
return {
|
||||
"schema_version": "railiance.forge-resource-observation.v1",
|
||||
"record_type": "usage_observation",
|
||||
"resource_id": "resource:railiance:forgejo",
|
||||
"source": "railiance-forge",
|
||||
"workplan_id": "RFORGE-WP-0002",
|
||||
"reef": "reef-railiance",
|
||||
"captured_at": captured_at,
|
||||
"capacity": {
|
||||
"deployments": deployment_rows,
|
||||
"persistent_volumes": pvcs,
|
||||
"pvc_requested_bytes": total_pvc,
|
||||
"forge_data_used_bytes": int(snapshot["forge_data_kib"]) * 1024 if snapshot.get("forge_data_kib") is not None else None,
|
||||
"runner_data_used_bytes": int(snapshot["runner_data_kib"]) * 1024 if snapshot.get("runner_data_kib") is not None else None,
|
||||
"database": {
|
||||
"instances": db_spec.get("instances"),
|
||||
"ready_instances": db_status.get("readyInstances"),
|
||||
"storage_requested_bytes": bytes_value((db_spec.get("storage") or {}).get("size")),
|
||||
"phase": db_status.get("phase"),
|
||||
},
|
||||
},
|
||||
"recovery": {
|
||||
"database_continuous_archiving": any(c.get("type") == "ContinuousArchiving" and c.get("status") == "True" for c in db_status.get("conditions") or []),
|
||||
"database_backups_seen": len(backups),
|
||||
"database_backups_completed": len(completed),
|
||||
"shared_storage_restore_drill": False,
|
||||
"runner_restore_drill": False,
|
||||
},
|
||||
"allocation": {
|
||||
"method_version": "forge-raw-drivers-v1",
|
||||
"candidate_drivers": ["repository_count", "package_bytes", "runner_job_minutes"],
|
||||
"consumer_groups": ["helix-forge", "coulomb-social", "railiance"],
|
||||
"attributed": [],
|
||||
"residual": {"forge_data_used_bytes": int(snapshot["forge_data_kib"]) * 1024 if snapshot.get("forge_data_kib") is not None else None, "reason": "consumer attribution source is not yet available"},
|
||||
"selection_owner": "resource-control",
|
||||
},
|
||||
"measurement_gaps": sorted(set(gaps)),
|
||||
"provenance": {
|
||||
"commands": [
|
||||
"kubectl get deploy -n forgejo -o json",
|
||||
"kubectl get pvc -n forgejo -o json",
|
||||
"kubectl top pods -n forgejo --no-headers",
|
||||
"kubectl get cluster forgejo-db -n databases -o json",
|
||||
"kubectl get backup,scheduledbackup -n databases -o json",
|
||||
"kubectl exec deploy/forgejo-gitea -- du -sk /data",
|
||||
"kubectl exec deploy/forgejo-runner -- du -sk /data",
|
||||
],
|
||||
"repository_contents_read": False,
|
||||
"secret_surfaces_read": False,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def run(prefix: list[str], args: list[str], *, optional: bool = False) -> str | None:
|
||||
completed = subprocess.run(prefix + args, text=True, capture_output=True, check=False)
|
||||
if completed.returncode != 0:
|
||||
if optional:
|
||||
return None
|
||||
raise subprocess.CalledProcessError(completed.returncode, prefix + args, completed.stdout, completed.stderr)
|
||||
return completed.stdout
|
||||
|
||||
|
||||
def collect(remote: str | None, kubectl: str) -> dict[str, Any]:
|
||||
prefix = ["ssh", "-o", "BatchMode=yes", remote, kubectl] if remote else [kubectl]
|
||||
load = lambda args: json.loads(run(prefix, args) or "{}")
|
||||
data = {
|
||||
"deployments": load(["get", "deploy", "-n", "forgejo", "-o", "json"]),
|
||||
"pvcs": load(["get", "pvc", "-n", "forgejo", "-o", "json"]),
|
||||
"database": load(["get", "cluster", "forgejo-db", "-n", "databases", "-o", "json"]),
|
||||
"backups": load(["get", "backup,scheduledbackup", "-n", "databases", "-o", "json"]),
|
||||
"top": run(prefix, ["top", "pods", "-n", "forgejo", "--no-headers"], optional=True),
|
||||
}
|
||||
# Avoid an inner shell here: when kubectl is executed through SSH, shell
|
||||
# quoting can otherwise turn ``sh -c 'du ...'`` into ``sh -c du`` and
|
||||
# silently measure the container working directory instead of /data.
|
||||
forge_du = run(prefix, ["exec", "-n", "forgejo", "deploy/forgejo-gitea", "-c", "gitea", "--", "du", "-sk", "/data"], optional=True)
|
||||
runner_du = run(prefix, ["exec", "-n", "forgejo", "deploy/forgejo-runner", "-c", "runner", "--", "du", "-sk", "/data"], optional=True)
|
||||
|
||||
def du_kib(output: str | None) -> int | None:
|
||||
fields = (output or "").split()
|
||||
return int(fields[0]) if fields and fields[0].isdigit() else None
|
||||
|
||||
data["forge_data_kib"] = du_kib(forge_du)
|
||||
data["runner_data_kib"] = du_kib(runner_du)
|
||||
captured_at = datetime.now(UTC).replace(microsecond=0).isoformat().replace("+00:00", "Z")
|
||||
return build_observation(data, captured_at)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--remote")
|
||||
parser.add_argument("--kubectl", default="kubectl")
|
||||
parser.add_argument("--output-dir", type=Path)
|
||||
args = parser.parse_args()
|
||||
try:
|
||||
observation = collect(args.remote, args.kubectl)
|
||||
except (OSError, subprocess.CalledProcessError, json.JSONDecodeError, ValueError) as exc:
|
||||
print(f"forge observation failed: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
rendered = json.dumps(observation, indent=2) + "\n"
|
||||
if args.output_dir:
|
||||
args.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
stamp = observation["captured_at"].replace(":", "").replace("-", "")
|
||||
destination = args.output_dir / f"{stamp}.json"
|
||||
destination.write_text(rendered, encoding="utf-8")
|
||||
(args.output_dir / "latest.json").write_text(rendered, encoding="utf-8")
|
||||
print(destination)
|
||||
else:
|
||||
sys.stdout.write(rendered)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
|
|
@ -4,11 +4,11 @@ type: workplan
|
|||
title: "Publish Forgejo resource economics evidence"
|
||||
domain: financials
|
||||
repo: railiance-forge
|
||||
status: ready
|
||||
status: finished
|
||||
owner: codex
|
||||
topic_slug: railiance
|
||||
created: "2026-08-11"
|
||||
updated: "2026-08-15"
|
||||
updated: "2026-08-22"
|
||||
related:
|
||||
- RESOURCE-WP-0003
|
||||
- ADR-0008
|
||||
|
|
@ -43,7 +43,7 @@ a separate granted surface, not this plan. Do not treat
|
|||
|
||||
```task
|
||||
id: RFORGE-WP-0002-T01
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "7048e354-0844-4f6c-aeaa-acf188b04ce2"
|
||||
```
|
||||
|
|
@ -55,11 +55,17 @@ and material network traffic using stable units and timestamps.
|
|||
Done when resource-control can forecast capacity and identify saturation or
|
||||
idle capacity without accessing repository content or credentials.
|
||||
|
||||
**Outcome (2026-08-22):** `make forge-observe REMOTE=railiance01` publishes a
|
||||
timestamped, versioned observation for deployment requests/limits and observed
|
||||
CPU/memory, PVC and database capacity, shared/runner storage usage, recovery
|
||||
posture, and unavailable axes. It reads neither repository contents nor Secret
|
||||
surfaces.
|
||||
|
||||
## T02 — Declare service objectives and operations labor
|
||||
|
||||
```task
|
||||
id: RFORGE-WP-0002-T02
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "643c2393-2ee5-4264-972c-04c3b358dc86"
|
||||
```
|
||||
|
|
@ -71,11 +77,16 @@ database, backup, upgrade, and incident labor where evidence permits.
|
|||
Done when total-cost and optimization cases can preserve the forge's service
|
||||
constraints and labor assumptions.
|
||||
|
||||
**Outcome (2026-08-22):** the service-objective and labor evidence file names
|
||||
the availability, restore, retention, performance, operational, upgrade, and
|
||||
incident dimensions. Unknown targets and hours remain explicit nulls rather
|
||||
than invented estimates.
|
||||
|
||||
## T03 — Publish shared-consumer allocation evidence
|
||||
|
||||
```task
|
||||
id: RFORGE-WP-0002-T03
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "498cc0c7-ba07-4e7f-a0ef-2e6b34bef04c"
|
||||
```
|
||||
|
|
@ -87,8 +98,21 @@ and unattributed residuals; do not implement customer billing.
|
|||
Done when a versioned driver report can feed resource-control allocation
|
||||
without making the forge repository authoritative for booked cost.
|
||||
|
||||
**Outcome (2026-08-22):** `forge-raw-drivers-v1` proposes repository count,
|
||||
package bytes, and runner job minutes, preserves all current use as an
|
||||
unattributed residual, and assigns method selection to `resource-control`.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- [ ] Demand and utilization include storage, compute, runner, and backup axes.
|
||||
- [ ] Service objectives and labor assumptions are explicit.
|
||||
- [ ] Shared allocation evidence exposes residuals and provenance.
|
||||
- [x] Demand and utilization include storage, compute, runner, and backup axes.
|
||||
- [x] Service objectives and labor assumptions are explicit.
|
||||
- [x] Shared allocation evidence exposes residuals and provenance.
|
||||
|
||||
## Completion evidence
|
||||
|
||||
- `python3 -m unittest discover -s tests -p 'test_*.py' -v`: 1 test passed.
|
||||
- Live read-only capture:
|
||||
`docs/evidence/resource-railiance-forgejo/observations/20260822T102256Z.json`.
|
||||
- Current observation records 2,377,375,744 forge data bytes, 247,693,312
|
||||
runner data bytes, healthy single-instance CNPG state, continuous archiving,
|
||||
and no Forge database Backup/ScheduledBackup or restore-drill evidence.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue